Sun Fire[TM] 12K/15K/E20K/E25K: SMS 1.6 + Solaris[TM] 10 deployed CP1500 based SC's C1 network interfaces reported with non-unique MAC address

Asset ID:	1-71-1008055.1
Update Date:	2016-10-04
Keywords:

Solution Type Technical Instruction Sure

Solution 1008055.1 : Sun Fire[TM] 12K/15K/E20K/E25K: SMS 1.6 + Solaris[TM] 10 deployed CP1500 based SC's C1 network interfaces reported with non-unique MAC address

Applies to:

Sun Fire 12K Server - Version All Versions and later
Sun Fire 15K Server - Version All Versions and later
Sun Fire E20K Server - Version All Versions and later
Sun Fire E25K Server - Version All Versions and later
All Platforms

Goal

On the CP1500 based System Controller (SC), the default network interfaces for the user community network C1 are hme0 and eri1. These external network interfaces are grouped using IP Multi-Pathing (IPMP) to provide reliability.

This document discusses an anomaly whereby the external interfaces that makes up the user community network C1 are shown to share identical MAC addresses.

Solution

Due to this anomaly, the external interfaces of the C1 network share identical MAC addresses: this in turns causes IPMP, managed by the in.mpathd daemon, to fail either a specific network interface in the C1 network community or all network interfaces in C1 network.
This anomaly is only manifested in the following System Controller (SC) environment:

# /opt/SUNWSMS/bin/smsversion -t
1.6 <-- SMS 1.6
# uname -r
5.10 <-- Solaris 10
# uname -i
SUNW,UltraSPARC-IIi-cEngine <-- CP1500 based System Controller

The error condition with regards to the network interfaces in the C1 network community is manifested as follows:

# ifconfig -a
eri1: flags=19040843 mtu 1500 index 2
inet 129.158.142.240 netmask ffffff00 broadcast 129.158.142.255
groupname C1
ether 0:3:ba:12:45:63

hme0: flags=9040843 mtu 1500 index 3
inet 129.158.142.238 netmask ffffff00 broadcast 129.158.142.255
groupname C1
ether 0:3:ba:12:45:63

hme0:1: flags=1000843 mtu 1500 index 3
inet 129.158.142.236 netmask ffffff00 broadcast 129.158.142.255

scman0: flags=1008843 mtu 1500 index 4
inet 192.168.102.1 netmask ffffffe0 broadcast 192.168.102.31
ether 0:3:ba:12:45:63

scman1: flags=1008843 mtu 1500 index 5
inet 192.168.102.42 netmask fffffffc broadcast 192.168.102.43
ether 0:3:ba:12:45:63

As observed from the above output generated from the "ifconfig -a" command, the network interfaces in the C1 network community (eri1 and hme0) share the same ethernet (MAC) address 0:3:ba:12:45:63. This state of configuration happens in spite of the fact that the OBP parameter "local-mac-address?" is assigned to the expected value (local-mac-address?=true).

The above identical ethernet (MAC) address assignment to both network interfaces in the C1 network community can potentially lead to in.mpathd failing a specific network interface within the C1 community:

Sep 15 11:09:37 v4u-15kb-sc1 in.mpathd[139]: [ID 594170 daemon.error] NIC failure detected on eri1 of group C1
Sep 15 11:09:37 v4u-15kb-sc1 in.mpathd[139]: [ID 832587 daemon.error] Successfully failed over from NIC eri1 to NIC hme0

The root-cause behind the above error condition lies with the incorrect application of the Solaris[TM] 10 Layered Driver Interface (LDI) calls within the Sun Fire[TM] 15000 management network device driver, scman. The incorrect application of LDI codes within v1.47 scman is primarily attributable to the identical MAC address being shared between the external network interfaces in the C1 network community observations on the CP1500 SC boards loaded with Solaris 10 and SMS 1.6.

The issue described above is addressed in v1.48 of the scman driver:

# modinfo | grep scman
120 7bb54000 11cd0 299 1 scman (MAN MetaDriver v1.48)

which is officially available via the scman driver patch 124243-01.

Internal Section

See Bug ID 6448961 for more info.

Keywords: starcat, ldi_open_by_name, ldi_open_by_dev, 6448961, cp1500, scman, 1.47, 1.48

Previously Published As 87105

Attachments

This solution has no attachment