Sun Datacenter InfiniBand Switch 36
 

PLA-Support>Sun Systems>SAND>Network>SN-SND: Sun Network Infiniband
 

A number of Sun Datacenter Infiniband Switch 36 units were shipped with incorrect settings in the /conf/ifcfg-ib0 file. This allows for the customer IP to be applied during the EIS Checklist activities but because the DEVICE entry in the affected file has eth0 instead of ib0 there is a conflict of settings. The affected serial number range is AK00380358-AK00383371 and is documented in Bug 25022730.

We understand that about 50 switches were shipped in this state, manufactured in the two weeks up to 1st November 2016.

The issue enables the customer IP address to be assigned but a number of network-related issues have been seen including the following:

In this Document

Symptoms

Cause

Solution

Applies to:

Symptoms

The issue enables the customer IP address to be assigned but a number of network-related issues have been seen including the following:

1. ibswitches output & browser show incorrect IP Address
2. The command smpartition commit fails
3. The affected switch can login to itself with 2 x IPs - ilom /SP/network address & IP address assigned in /conf/ifcfg-ib0
4. Merging of switches with a mismatch of these files causes huge latency over the IP stack within InfiniBand fabric - (ping tests between nodes) - and will cause an outage.

Cause

A number of Sun Datacenter Infiniband Switch 36 units were shipped with incorrect settings in the /conf/ifcfg-ib0 file. This file allows the customer IP to be applied during configuration but because the DEVICE entry in the affected file has eth0 instead of ib0 there is a conflict of settings. The affected serial number range is AK00380358-AK00383371 and is documented in Bug 25022730.

The signature of a such a switch is that the /conf/ifcfg-ib0 contains:
DEVICE=eth0

Below is an example output of an affected switch (IP address & netmask will vary):
DEVICE=eth0
BOOTPROTO=static
IPADDR=10.196.67.32
NETMASK=255.255.224.0
ONBOOT=yes

Solution

To restore the switch back to the factory default, execute the following commands:

1. Log into the switch as root
2. Overwrite the content of the /conf/ifcfg-ib0 file
   1. cat /dev/null > /conf/ifcfg-ib0
3. From the root CLI execute:
   1. delete_ipoib 0x7fff
4. Verify the logical link:
   1. ls -l /etc/sysconfig/network-scripts/ifcfg-ib0
      1. lrwxrwxrwx 1 root root 15 2016-09-28 12:15 /etc/sysconfig/network-scripts/ifcfg-ib0 -> /conf/ifcfg-ib0
5. Reboot each switch after making all changes.

If IPoIB needed, follow the product documentation to config the interface.

This solution has no attachment