IBM Power Systems AIX/Linux
Microcode Update for PM985

800/1600/3200 GB
PM985 U.2 NVMe SSD
Microcode Level MN3A

DATE: May 10, 2021

Revised: May 19, 2021

Table of Contents

1. General Information

1.1     Disk Drives Affected

1.2 Microcode Change History

2.  AIX and VIOS(PowerVM Virtual I/O Server) Operating System

2.1 Installation Time

2.2 Determine Drive Model, FRU&PN and Current Microcode Level

2.3.1 Internet AIX self extracting file (.bin)

2.3.2 Internet RPM package

2.4 Microcode Download Procedure

3.   Linux Operating System

3.1 Determine the Current Microcode Level for Linux:

3.2 Downloading and Unpacking the Firmware Update Package

3.3 Verifying microcode before download and notes

3.4 Microcode Download Procedure-NVMe Adapters for Linux

4.   Revisions

===============================================================

1. General Information

The Microcode files provided in this package are for SAS disk drives used in IBM system P Servers

with AIX or Linux operating systems

1.1    Disk Drives Affected 

Below is the list of microcode files in this package:

NOTE: To insure the integrity of microcode files, please make sure the checksum value is correct after extraction

 

 

Size

Disk Drive Model

FRU#

P/N (VPD)

Microcode File Name

Check Sum

800 GB

MZQLB800HAJR

01LU763

01LU760

A1800101.4d4e3341

03386

1600 GB

MZQLB1T6HALS

01LU764

01LU761

3200 GB

MZQLB3T2HALS

01LU765

01LU762

1.2 Microcode Change History

Release date

Code level

Severity/

Impact

Change Description

May 2021

MN3A

Function/

ATT

-GPIO Detection: Change the detection method from edge detection to level detection

- NAND Recovery: Optimize the setting to applicate NAND recovery code on Die1 as well

-PLP recovery: Write request for PLP dump is allocated into SRAM so dump speed is improved

- Optimize operation ratio between GC and host write in GC level “Low” → Decrease GC operation ratio in GC level “Low”

July 2020

MN39

Function/

ATT

Fixed the following:
Gen3-Gen1 speed down transition
Performance Enhancement
Erase count correction
TCG Transaction Exception Handling corner case fix
FW Activation Policy Change on FW image Crash
ARM exception by TRIM during TCG operation
Device did not appear as NVMe/PCIe device after hot-insertion (Hot-plug)
Other fixes and enhancements (Network Roll)

Feb 2019

MN38

Function/

ATT

Command timeout (8sec) caused by a racing condition between write and read when specific bank receives intensive read commands

 

October 2018

MN37

Function/

ATT

- VPD Change to show correct FW Level (in lscfg) after code

   update
- Fixed Namespace Creation error

August 2018

MN36

New/

New

Initial Release

 

 

===============================================================

2.  AIX and VIOS(PowerVM Virtual I/O Server) Operating System

Note: For VIOS systems, microcode of shared physical I/O resources can only be updated from VIOS partition.

To update disk drive microcode, from VIOS partition, enter the following command:     

oem_setup_env   

Then follow the instruction in section 2.3 to transfer/extract the microcode file, and section 2.4.1 to update the disk drive code.

2.1 Installation Time

Approximately 20 minutes if using Diagnostics CD; Approximately 5 minutes/drive using AIX concurrent diagnostics.
 

2.2 Determine Drive Model, FRU&PN and Current Microcode Level
To see the drive type and current microcode level, Execute the following command:
lscfgvl nvme*

 

 Vital Product Data (VPD)

# lscfg -vl nvme0

nvme0            U78D4.ND1.CSS10FA-P2-C1  PCIe3 x4 NVMe Flash Adapter

 

      800GB NVMe Gen3 U.2 Slim SSD    :

        Part Number.................01CM501

        EC Level....................P02092

        FRU Number..................01LU763

        Product Specific.(AN).......01LU760

        Feature Code/Marketing ID...EC5J

        Customer Card ID Number.....59B4

        Serial Number...............84YAB0AF

        Product Specific.(Z0).......10140637

        Product Specific.(Z1).......2.4

        Product Specific.(Z2).......800

        Product Specific.(Z3).......3.50

        Product Specific.(Z4).......0

        Product Specific.(Z5).......0

        Product Specific.(Z6).......A1800101

        Product Specific.(Z7).......0YA

        Product Specific.(Z8).......2637

        Product Specific.(Z9).......08

        Product Specific.(ZA).......5A

        Product Specific.(ZB).......08

        Manufacture ID..............34NY0K100406

        Product Specific.(RV)........

        ROM Level.(alterable).......REV.MN3A               ß-- Code Level MN3A

        Hardware Location Code......U78D4.ND1.CSS10FA-P2-C1       

 

You can also use lsmcode to find the microcode level:

lsmcoderd nvme0

A1800101.4d4e3341

Note that lsmcode output FW level is displayed as ASCII/Hex

4d = M

4e = N

33 = 3

41 = A

===============================================================

2.3 Transferring and Unpacking the Microcode Update Package

2.3.1 Internet AIX self extracting file (.bin)

a) Make a directory on an AIX system to receive the AIX format file.
    Enter:      mkdir /tmp/fwupdate
b) Transfer the AIX format file to /tmp/fwupdate directory (using "Save as ...").
    You'll see the filename is  PM985.bin
c) Unpack the file by executing the instructions below.
  Enter the commands:
  cd /tmp/fwupdate
  chmod +x PM985.bin
  ./PM985.bin
[Don't overlook the periods (.) in the above command.]

The directory pm985 with microcode files listed in section 1.1 will be added to /tmp/fwupdate.
d-Enter the following command to copy the microcode files to " /etc/microcode " directory :
   cp /tmp/fwupdate/ PM985/*   /etc/microcode

2.3.2 Internet RPM package

Note: The instructions that follow are specific AIX commands. AIX commands are CASE (lower and upper) SENSITIVE,

and must be entered exactly as shown, including filenames.

a) Make a directory on an AIX system to receive the RPM format file.
    Enter:      mkdir /tmp/microcode

b) Transfer the RPM format file to the /tmp/microcode directory.
    You'll see the filename is PM985-MN3A-AIX.rpm

c) Unpack the file by executing the instructions below:
    Enter the commands:
        rpm -ihv --ignoreos  --force PM985-MN3A-AIX.rpm   ;(please note, there are two dashes)

The file size and checksum will be verified.

The microcode files will be added to /etc/microcode/
 d) If you are using stand alone AIX diag (bootable USB flash or CD/DVD) to update drive’s microcode, transfer microcode files to a USB drive and place it in “/etc/microcode” directory.
    Please refer to this page for USB flash drive usage  
   
https://www.ibm.com/support/pages/using-and-taking-advantage-usb-devices-and-aix

 

========================================================

2.4 Microcode Download Procedure

Warning: Do not power off the system or the drive during microcode download as this may permanently damage the disk drive

NOTE: Please save current and older versions of the microcode update files in case you need to restore your system.

Caution: The microcode download process is nondestructive to the hard disks; however, as a precaution, ensure that a backup operation has been performed on the affected systems PRIOR to the microcode installation.

Can microcode be installed Concurrently?  YES
Is system reboot needed to activate the microcode: Yes if stand alone diagnostics CD is used, NO if concurrent diagnostics is used.

Installation Time: Approximately 20 minutes/drive if stand alone diagnostics is used(using Diagnostics CD); Approximately 5 minutes/drive if concurrent diagnostics is used.

2.4.1 Concurrent Download

1. From AIX prompt type diag, then select the "Task Selection" from diagnostics menu.
2. Select "Microcode Download" from "Task Selection" menu.
3. Select resource that microcode will be applied to (nvmek?) and press ENTER, then press "F7" to commit..
4- A small screen displays the message " *** NOTICE***  The microcode installation occurs while the attached drives are available for use..... "
    Press enter to continue
5.  Select the source for microcode, use arrow key to go up/down, Enter to continue.
                      /etc/microcode
                      usb mass storage

    - Use /etc/microcode if microcode file is in the system
    - Use “usb mass storage” if using usb flash drive

6- Display shows a list of microcodes available for the selected nvme(s), highlight option "M  4d4e3341", the latest microcode for the drive and press ENTER.
   A message may warn you that current level of microcode is not available on the media, press ENTER to continue.

7. You will receive a msg. "Current Microcode is...." and "Download has completed successfully"
       (this will take a several minutes)
8. Return to the Tasks Selection menu and repeat this procedure for each nvme (in the same partition) that requires this microcode.
9. Exit diagnostic

2.4.2 Standalone (Service Mode) Download

This procedure is used for all nvme’s  including rootvg or paging space drives
1. Properly shutdown the system.
2. Boot up the system in standalone diagnostics mode with the AIX diag CD/DVD or USB Thumb drive
3. Select the "Task Selection" from diagnostics menu.
4. Select "Microcode Download" from "Task Selection" menu.
5. Select resource that microcode will be applied to and PRESS ENTER.
6. Press "F7" to commit.
      (The current microcode level of the resource you selected earlier will be displayed at the top of the screen)
7. Make the following selections on the next screen:
    file system
      /etc/microcode
      usb mass storage    (ISO 9660 file system format)

8. You will receive a msg. "Current Microcode is...." and "Download has completed successfully"
       (this may take a few minutes)
9. Return to the Tasks Selection menu and repeat this procedure for each hard disk that requires this microcode.
10. Exit diagnostic and reboot system in normal mode

                             

======================================================

3.   Linux Operating System

 

Note: The following procedure is for Red Hat Linux, please use microcode file package PM985-MN3A-LINUX.tgz for Ubuntu

tar -C-xzvf PM985-MN3A-LINUX.tgz


Red Hat Enterprise Linux for IBM Power LE

 

3.1 Determine the Current Microcode Level for Linux:

These steps assume you are logged in with root permissions.

 

Prerequisites for Red Hat Linux:

Use the ibm-power-repo to configure the repositories and then use yum to download the nvme-cli package.

http://www14.software.ibm.com/support/customercare/sas/f/lopdiags/home.html

 

1. Download the IBM Linux on Power Tools Repository configuration RPM file available at:

http://www14.software.ibm.com/webapp/set2/sas/f/lopdiags/yum.html

 

2. Copy the file to the target system and install the RPM

# rpm -ivh ibm-power-repo-latest.noarch.rpm

 

3. To add the Tools Repository to the package management configuration, run the following command as root user (or with root privilege) and accept the Licensing agreement.

# /opt/ibm/lop/configure

 

Instructions for installing nvme-cli tool using Red Hat Enterprise Linux

 

To list the currently enabled repositories, enter:

# yum repolist enabled

 

Install the nvme-cli package, enter:

# yum install nvme-cli

 

After nvme-cli is installed on the system, to check the current microcode level for the adapter enter the following command:

nvme list

 

Note: "nvme list" command uses libudev-devel, if it doesn't work, may need to install “libudev-devel"

 

1) Type "nvme list"
2) The column “FW Rev” will list the firmware version.  For example:

 
     Node                            SN                                  Model                            Namespace                 Usage                        Format              FW Rev 

----------------     -----------------------   -----------------------------------------     ------------     -----------------------------      ----------------     ------ -------

/dev/nvme1n1    S434NY0K400206  800GB NVMe Gen3 U.2 Slim SSD           1           800.17  GB / 800.17  GB      4 KiB +  0 B     REV.MN3A

 

If the Firmware Version is below than MN3A9 then you should update the microcode.

3.2 Downloading and Unpacking the Firmware Update Package

1. Download feature code EC5J/EC5K/EC5L package from FixCentral, and transfer the file, to a temporary directory on the target server such as /tmp.

2. Unpack the package

     rpm -ivh --ignoreos PM985-MN3A-LINUX.rpm

3. The microcode package extracts the firmware file to the /lib/firmware/ directory

 

3.3 Verifying microcode before download and notes

NOTE: Microcode download must be performed separately for each nvme under distinct Logical Partitions (LPAR's).
              Please save current and older versions of the microcode update files in case you need to restore your system.

Please verify the file size and checksum of the raw microcode files matches the below.
Filesize:  1376256
Checksum:  03386

For Linux:  "ls -l /lib/firmware/ A1800101.4d4e3341" to verify Filesize.
                    "sum /lib/firmware/A1800101.4d4e3341" to verify Checksum.

For systems that have LPAR installed, the microcode file must be placed on all partitions that have(owns) an NVMe device in order to update the code. 
Follow this procedure for each partition:
   

1.       Log into the target partition.

2.       Type: "cd /lib/firmware"

3.       SFTP to the partition that has the media bay assigned. (ie.  "sftp partition1.ibm.com")

4.       Type: "cd /lib/firmware"

5.       Type: "get A1800101.4d4e3341"

6.       Type: "quit"

3.4 Microcode Download Procedure-NVMe Adapters for Linux

1) Type "nvme list" and press "Enter".  This will list all of the NVMe adapters in your partition.  In the Node column, the NVMe adapters will be listed as /dev/nvme#n1 where # is the number of the device.
2) For each NVMe adapter to be update, issue the following commands:

a)     Type “nvme fw-log /dev/nvme<#>” and press “Enter”, where <#> is the number of the device. 

                For example, for device /dev/nvme0n1, type

                nvme fw-log /dev/nvme0” and press “Enter”.  This will list the firmware level currently loaded on the adapter as well as the number of slots in the adapter.  
               These 3 slots are listed as frs1, frs3.

       b)     Download and activate firmware to the adapter by typing:

               Note that there are 4 FW “slots” in this nvme, each slot must be updated AND activated, example below is for nvme0

 

nvme fw-download /dev/nvme0 --fw=/lib/firmware/ A1800101.4d4e3341

nvme fw-activate /dev/nvme0 -a 0 –s 1

                               

Repeat for all slots (-s 2; -s 3)

 

 

nvme fw-activate /dev/nvme0 -a 3 –s 3   ß this step may be omitted if the nvme device does not support the function

nvme reset /dev/nvme0

 

4.   Revisions

May 19, 2021: Changed section 2.4.1 to indicate no power cycle or reboot is required