Sun Storage 7000 Unified Storage System: 7x20 Datalink Device Errors caused by path_to

Asset ID:	1-72-1472000.1
Update Date:	2017-01-17
Keywords:

Solution Type Problem Resolution Sure

Solution 1472000.1 : Sun Storage 7000 Unified Storage System: 7x20 Datalink Device Errors caused by path_to_inst not configured correctly

Applies to:

Sun ZFS Storage 7120 - Version All Versions and later
Sun ZFS Storage 7320 - Version All Versions and later
Sun ZFS Storage 7420 - Version All Versions and later
7000 Appliance OS (Fishworks)

Symptoms

Example of Datalink Devices output.

BUI shows devices e1000g0 thru e1000g3 on one node and the other node reports e1000g1 thru e1000g4
BUI shows devices igb0 thru igb3 on one node and the other node reports igb1 thru igb8
BUI shows devices ixgbe0 thru ixgbe3 on one node and the other node reports ixgbe4 thru ixgbe7

1. Check net directory and all information related to the devices
2. Check Appliance Kit version, BIOS and SP
3. Check known bugs or issues with this discrepancy
4. Escalate to backline engineer to review supportbundle and work in shared shell/webex
5. Document information found related to BUI network interfaces and CLI network interfaces
6. Open a bug if the code is current, problem is reproducible and collect data for development

Changes

Engage Oracle L2 NAS Support with a collaboration SR to resolve this problem

Cause

path_to_inst system configuration file was not configured with the correct paths, or was "fixed" incorrectly during a previous issue.

Nothing has changed on the network infrastructure side of things. All cables are plugged in and working as confirmed by the interfaces being UP.

The issue is that somehow the device names changed and becomes the problem. Issues are seen with Exadata configuration, upgrades, and editing of the path_to_inst file on the effected node.

The device names in the BUI used to be igb1 thru igb4. Now the device names are igb0 thru igb3. Because of the name changes in the BUI, the interfaces are having anomalies.

NOTE: on all of the 7x20 series arrays, the device are igb1 thru igb4 in both the BUI and the command line. Something happened to cause the devices to change.

(In some cases, both heads are showing the same discrepancy between the command line and the BUI. The BUI is igb0 thru igb3 and the command line is igb1 thru igbg4).

Comment : The only change that occurred recently was that an additional 8 disk shelves which just got cabled-up. This required installing an additional SAS hba in each head.

An upgrade of the Appliance Kit code or the path_to_inst edits were not done correctly (Doc ID 1560329.1) and allowed a new instance that does not match any physical instance.

CLI:>configuration net devices show

shows different instances for network instances below :

nas01-a:> configuration net devices show
---------------------
ibp0        true   32000 Mbit/s not available
ibp1        true   32000 Mbit/s not available
ibp2        true   32000 Mbit/s not available
ibp3        true   32000 Mbit/s not available
igb0        true   1000 Mbit/s   0:21:28:ff:c2:52
igb1        false 0 Mbit/s      0:21:28:ff:c2:53
igb2        false 0 Mbit/s      0:21:28:ff:c2:54
igb3        false 0 Mbit/s      0:21:28:ff:c2:55
ixgbe0      true   10000 Mbit/s 90:e2:ba:c:2d:b4
ixgbe1      true   10000 Mbit/s 90:e2:ba:c:2d:b5
ixgbe2      true   10000 Mbit/s 90:e2:ba:c:2c:c8
ixgbe3      true   10000 Mbit/s 90:e2:ba:c:2c:c9

nas01-b:> configuration net devices show
---------------------
DEVICE      UP     SPEED         MAC

e1000g1     true   1000 Mbit/s   0:1b:21:cb:8:45
e1000g2     true   1000 Mbit/s   0:1b:21:cb:8:44
e1000g3     false 0 Mbit/s      0:1b:21:cb:8:47
e1000g4     false 0 Mbit/s      0:1b:21:cb:8:46
e1000g5     true   1000 Mbit/s   0:1b:21:c5:89:a1
e1000g6     true   1000 Mbit/s   0:1b:21:c5:89:a0
e1000g7     false 0 Mbit/s      0:1b:21:c5:89:a3
e1000g8     false 0 Mbit/s      0:1b:21:c5:89:a2
ibp0        false 0 Mbit/s      not available
ibp1        false 0 Mbit/s      not available
ibp4        false 0 Mbit/s      not available   >>>>>>>
ibp5        false 0 Mbit/s      not available         >>>>>>>>
igb0        false 0 Mbit/s      0:21:28:ff:5f:40
igb1        true   1000 Mbit/s   0:21:28:ff:5f:41
igb4        false 0 Mbit/s      0:21:28:ff:5f:42        >>>>>>>
igb5        false 0 Mbit/s      0:21:28:ff:5f:43        >>>>>>>>>
ixgbe4      true   10000 Mbit/s 90:e2:ba:c:2e:fc   >>>>>>
ixgbe5      true   10000 Mbit/s 90:e2:ba:c:2e:fd >>>>>>>
ixgbe6      true   10000 Mbit/s 90:e2:ba:c:2f:0   >>>>>>>
ixgbe7      true   10000 Mbit/s 90:e2:ba:c:2f:1   >>>>>>>>

7420/7720 check http://docs.oracle.com/cd/E22471_01/html/821-1792/maintenance__hardware__overview__7420_7720.html#maintenance__hardware__overview__7420_7720__pcie_option

PCIe Options for DP Optical 10GE NIC is slot 2, slot 4. slot 5, slot 6 and slot 7 --- DP SAS-2 HBA is slot 1, slot 2, slot 7 and slot 8 --- 8Gb DP FC HBA is slot 0, slot 3, slot 6 and 9 are supported.

If you show 10G is in slot 0, slot 9..... please contact Oracle Support for help in correcting the configuration.

Solution

Recommended action for the customer:

You will need to engage Oracle Support, by opening a Service Request, so that Oracle Support Services can provide confirmation of this issue and then carry out the appropriate activities to resolve the issue.

Recommended actions for the Oracle Support engineer:

Please use the resolution procedure given in the following document:

1560329.1 - Sun Storage 7000 Unified Storage System: Device Enumeration Mismatch when inserting Additional Network Cards in a Cluster

If you cannot access this document engage NAS Storage-TSC for assistance

References

<NOTE:1396100.1> - Sun Storage 7000 Unified Storage System: Causes and Solutions for Well Known General Networking Problems
<NOTE:1560329.1> - Sun Storage 7000 Unified Storage System: Device Enumeration Mismatch when inserting Additional Network Cards in a Cluster
<BUG:17194712> - NEED RCA AND FIX/WORKAROUND TOCORRECT PATH_TO_INST ISSUE

Attachments

This solution has no attachment