Sun Microsystems, Inc.  Oracle System Handbook - ISO 7.0 May 2018 Internal/Partner Edition
   Home | Current Systems | Former STK Products | EOL Systems | Components | General Info | Search | Feedback

Asset ID: 1-72-2089322.1
Update Date:2017-05-04
Keywords:

Solution Type  Problem Resolution Sure

Solution  2089322.1 :   Fabric Interconnect (Xsigo) After XgOS Install or Upgrade 'ethmgmt' Network Interface Will Not Come up, Cannot Configure Networking  


Related Items
  • Oracle Fabric Interconnect F1-15
  •  
  • Oracle Fabric Interconnect F1-4
  •  
  • Oracle Virtual Compute Appliance X4-2 Hardware
  •  
  • Private Cloud Appliance X5-2 Hardware
  •  
Related Categories
  • PLA-Support>Sun Systems>SAND>Network>SN-SND: Oracle Virtual Networking
  •  




In this Document
Symptoms
Changes
Cause
Solution


Created from <SR 3-11513443891>

Applies to:

Oracle Fabric Interconnect F1-15 - Version All Versions to All Versions [Release All Releases]
Oracle Fabric Interconnect F1-4 - Version All Versions to All Versions [Release All Releases]
Private Cloud Appliance X5-2 Hardware - Version All Versions to All Versions [Release All Releases]
Oracle Virtual Compute Appliance X4-2 Hardware - Version All Versions to All Versions [Release All Releases]
Information in this document applies to any platform.

Symptoms

After XgOS install or upgrade or installing a new Gen2 Front Panel, the 'ethmgmt' network interface is not found.  Without the 'ethmgmt' network interface present, networking cannot be configured to connect the Fabric Interconnect to a network.  When looking at the Fabric Interconnect over serial console the errors show that eth1 is being renamed to eth4.  The errors are also found in /var/log/kern.log.  Here is the errors where the network devices are being renamed:

(snip below is also found also in /var/log/kern.log)

xvio01:~# grep eth /var/log/dmesg
[   25.558411] Driver 'sd' needs updating - please use bus_type methods
[   25.767591] e1000e 0000:00:19.0: eth0: (PCI Express:2.5GB/s:Width x1) ec:9e:cd:13:42:58
[   25.775713] e1000e 0000:00:19.0: eth0: Intel(R) PRO/1000 Network Connection
[   25.782808] e1000e 0000:00:19.0: eth0: MAC: 10, PHY: 10, PBA No: ffffff-0ff
[   25.850887] e1000e 0000:01:00.0: eth1: (PCI Express:2.5GB/s:Width x1) 00:13:97:00:a5:9a
[   25.859007] e1000e 0000:01:00.0: eth1: Intel(R) PRO/1000 Network Connection
[   25.866069] e1000e 0000:01:00.0: eth1: MAC: 4, PHY: 8, PBA No: ffffff-0ff
[   25.930569] e1000e 0000:02:00.0: eth2: (PCI Express:2.5GB/s:Width x1) 00:13:97:00:a5:9b
[   25.938687] e1000e 0000:02:00.0: eth2: Intel(R) PRO/1000 Network Connection
[   25.945751] e1000e 0000:02:00.0: eth2: MAC: 4, PHY: 8, PBA No: ffffff-0ff
[   38.703819] udev: renamed network interface eth1 to eth4  <<-- !!!
[   38.705424] udev: renamed network interface eth2 to eth3  <<-- !!!
[   44.441422] ADDRCONF(NETDEV_UP): ethiop: link is not ready
[   44.482647] ADDRCONF(NETDEV_UP): ethfabric: link is not ready

The issue here is eth1 gets renamed to eth4.  When XgOS expects to rename eth1 to ethmgmt, this is naturally going to fail.

In /etc/udev/rules.d/70-persistent-net.rules, can see the result of "udev: renamed network interface eth1 to eth4" where 00:13:97:00:a5:9a is associated to eth4 instead of eth1.

In /etc/udev/rules.d/70-persistent-net.rules, normally would expect to see only 3 lines for eth0 ~ eth2.  However, for some reason, there are 2 ADDITIONAL interfaces listed in /etc/udev/rules.d/70-persistent-net.rules, taking up eth1 and eth2, so this pushed the management interface to eth4.

The Gen2 FP comes with the COMX motherboard.

Based on the logic in /etc/init.d/xgifrename.sh, would expect the renaming to be:

eth0 -> ethiop
eth1 -> ethmgmt
eth2 -> ethha
eth3 -> ethfabric

XgOS is expecting the management interface to be eth1 before renaming it to ethmgmt, but, In this case, the management interface is eth4, so eth4 is left as it is.

eth0 -> ethiop           Link encap:Ethernet  HWaddr ec:9e:cd:13:42:58
eth1 -> eth4             Link encap:Ethernet  HWaddr 00:13:97:00:a5:9a
eth2 -> eth3 ->         ethfabric Link encap:Ethernet  HWaddr 00:13:97:00:a5:9b

Manually editing /etc/init.d/xgifrename.sh and changing all instances of eth4 back to eth1 works until the Fabric Interconnect is rebooted.

Also the manual changes made using 'ifconfig' to force to use eth4 revert to original whenever the Fabric Interconnect is rebooted.

For example to force ethmgmt to use eth4 and configure IP reverts when the Fabric Interconnect is rebooted - using ifconfig to manually plumb an IP address to eth4 is a workaround, but will be lost every time the Fabric Interconnect is rebooted.

xvio01:~# ifconfig eth4 <ip address>/24 up
xvio01:~# route -v add default gw <ip address> eth4
xvio01:~# netstat -nr

 Reinstalling the XgOS from scratch also doesn't resolve the problem.

Changes

 XgOS install or upgrade/downgrade or brand new Gen2 Front Panel installation.

Cause

The 'forcebaseos' option forces a new fresh install of the Linux kernel in addition to the XgOS downgrade/upgrade.   It is possible if this option is needed, that there was a bad XgOS install at the factory, and the 'forcebaseos' option forces a new install of the Linux kernel.

Solution

Reinstall the XgOS using 'forcebaseos' option.  After the XgOS .xpf file is placed on the Fabric Interconnect as user root under /var/fsroot/admin, as admin user run this command:

system upgrade <version.xpf> forcebaseos

 


Attachments
This solution has no attachment
  Copyright © 2018 Oracle, Inc.  All rights reserved.
 Feedback