How to Perform IB Diagnostics on Oracle Fabric Interconnect (Formerly Xsigo)

Asset ID:	1-71-1522847.1
Update Date:	2017-08-01
Keywords:

Solution Type Technical Instruction Sure

Solution 1522847.1 : How to Perform IB Diagnostics on Oracle Fabric Interconnect (Formerly Xsigo)

Applies to:

Oracle Fabric Interconnect F1-4 - Version All Versions to All Versions [Release All Releases]
Oracle Fabric Interconnect F1-15 - Version All Versions to All Versions [Release All Releases]
Information in this document applies to any platform.

Goal

Learn how to perform IB diagnostic steps on Oracle Fabric Interconnect during Fabric setup.

Solution

Here is a rough guideline of how to bring-up a fabric.

1. Cable the IB fabric.
2. Power on a "manageable section" of the fabric.
3. Clear counters when logged into Fabric Director as user 'admin':

#set diagnostics ib-clear-counters

4. Watch the SM (Subnet Manager) sweep and look for errors. Run the command 'showlog opensm.log' to watch SM sweep. Additionally run this command:

#show diagnostics ofed

Look for section titled 'ibnetdiscover', in the output of above command and note any symbol or other errors as well as any 'linkwidth' issues. Any linkwidth negotiating at 1x instead of 4x means a possible bad IB cable, bad IB port, bad HCA, or poorly seated IB cable or HCA.

5. If the SM sees errors on an idle fabric, fix errors, go back to step "3".
6. Clear counters.

#set diagnostics ib-clear-counters

7. Run traffic on all nodes powered on (ideally to all other nodes).
8. Watch the SM log and sweep and look for errors - (showlog opensm.log)
9. If errors, fix the errors, and go back to step "5".

10. If the fabric is not all powered on, power on an additional "manageable section" of the fabric, go to step 3.
11. Done!

Attachments

This solution has no attachment