How to update the InfiniBand HCA firmware on Exalytics X2-4 and X3-4 systems

Asset ID:	1-71-1967637.1
Update Date:	2016-08-29
Keywords:

Solution Type Technical Instruction Sure

Solution 1967637.1 : How to update the InfiniBand HCA firmware on Exalytics X2-4 and X3-4 systems

Applies to:

Exalytics In-Memory Machine X3-4 - Version All Versions to All Versions [Release All Releases]
Exalytics In-Memory Machine X2-4 - Version All Versions to All Versions [Release All Releases]
x86 64 bit

Goal

How to update the InfiniBand HCA firmware on Exalytics X2-4 and X3-4 systems

Solution

To update the firmware on an InfiniBand HCA used in an Exalytics X2-4 or X3-4 server the following steps should be followed.

First find the pci address of the HCA using the lspci command

[root@exalytics0 ~]# lspci | grep InfiniBand
b1:00.0 InfiniBand: Mellanox Technologies MT26428 [ConnectX VPI PCIe 2.0 5GT/s - IB QDR / 10GigE] (rev b0)

Exalytics X2-4 and X3-4 systems should have the InfiniBand HCA card located in pci slot 6 so the lspci command should show the address in use as "b1:00.0"

Use this address in the mstflint query command to check the current firmware level of the card

[root@exalytics0 ~]# mstflint -d b1:00.0 q
Image type:      ConnectX
FW Version:      2.7.8130
Device ID:       26428
Description:     Node             Port1            Port2            Sys image
GUIDs:           0021280001cf6112 0021280001cf6113 0021280001cf6114 0021280001cf6115
MACs:                                 002128cf6112     002128cf6113
Board ID:         (SUN0170000009)
VSD:
PSID:            SUN0170000009

The latest recommended firmware revision for use in the Exalytics X2-4 or X3-4 servers is 2.11.2010. If the previous command shows this revision reported on the "FW Version" line then there is no need for update. If the output shows an older revision (as in our example 2.7.8130) then proceed with the update steps.

Download the firmware update from MOS patch 16340059 and unzip the file. The firmware image file fw-ConnectX2-rel-2_11_2010-be-375-3696-01.bin will be in the download, transfer it to the Exalytics server. In the following example the file is located in the /tmp directory, if your file is in a different location on the server change the command as appropriate.

The InfiniBand interfaces should be brought down before performing the update. This will interupt any connection to the InfiniBand fabric and should be done when there is no active use of the interface required. Plan for downtime of this connection before proceeding.

If unsure of which interfaces the InfiniBand HCA are then use the following ifconfig command to discover them.

[root@exalytics0 ~]# ifconfig -a | grep InfiniBand
bond0     Link encap:InfiniBand HWaddr 80:00:00:4A:FE:80:00:00:00:00:00:00:00:00:00:00:00:00:00:00
ib0       Link encap:InfiniBand HWaddr 80:00:00:4A:FE:80:00:00:00:00:00:00:00:00:00:00:00:00:00:00
ib1       Link encap:InfiniBand HWaddr 80:00:00:4B:FE:80:00:00:00:00:00:00:00:00:00:00:00:00:00:00

then bring down each of the interfaces

[root@exalytics0 ~]# ifconfig bond0 down
[root@exalytics0 ~]# ifconfig ib0 down
[root@exalytics0 ~]# ifconfig ib1 down

After the interfaces are brought down perform the firmware update using the following mstflint command

[root@exalytics0 ~]# /usr/bin/mstflint -d b1:00.0 -i /tmp/fw-ConnectX2-rel-2_11_2010-be-375-3696-01.bin burn

    Current FW version on flash: 2.7.8130
    New FW version:               2.11.2010

Burning FW image without signatures - OK
Restoring signature                  - OK

when the firmware update is complete bring the interfaces back up

[root@exalytics0 ~]# ifconfig ib0 up
[root@exalytics0 ~]# ifconfig ib1 up
[root@exalytics0 ~]# ifconfig bond0 up

To make sure the firmware update was successful the HCA can be queried once again (note the following example now shows the FW Version as 2.11.2010 confirming that the firmware update succeeded)

[root@exalytics0 ~]# mstflint -d b1:00.0 q
Image type:      ConnectX
FW Version:      2.11.2010
Device ID:       26428
Description:     Node             Port1            Port2            Sys image
GUIDs:           0021280001cf6112 0021280001cf6113 0021280001cf6114 0021280001cf6115
MACs:                                 002128cf6112     002128cf6113
Board ID:         (SUN0170000009)
VSD:
PSID:            SUN0170000009

Attachments

This solution has no attachment