Oracle Database Appliance X7-2 HA Hardware
 

PLA-Support>Sun Systems>x86>Engineered Systems HW>SN-x64: ORA-DATA-APP
 

In this Document

Symptoms

Changes

Cause

Solution

References

Created from <SR 3-16787419701>

Applies to:

Symptoms

After reimaging with the OVM iso, ifconfig does not show the device names for the onboard Broadcom ports, specifically eth2/eth3. Therefore, after executing “oakcli configure firstnet”, the Broadcom ports still do not have link lights and the system cannot be accessed remotely.

Changes

 Customer has reimaged the ODA X7-2 nodes with the 12.2.1.2.0 Virtualized iso image

Cause

 It appears that when the fiber cables are plugged in, this somehow messes up the OVM device udev creations, and the appropriate device names and mac associations are not created. When you unplug the fiber cables completely (including transceivers), reimage with Virtualized iso, it seems to create all the devices.

Solution

1. Unplug all network cables, including transceivers

2. Reimage with VM iso

3. Plug in transceivers/cables

4. Reboot both nodes

5. Check the firmware of the Broadcom ports

Now when you do an ifconfig, you will see eth0-eth4, etc...

eth2 and eth3 are the Broadcom ports
Check both nodes to make sure the broadcom ports are at 20.06.04.07

If the firmware is not at the above level, update the firmware. Otherwise, skip to next step, step 6.
To update the firmware on the Broadcom ports, download the attached ODA_SFP28_FW.tar file to each node, unpack it in the /tmp directory, and execute the updates.  If you need a network port to transfer the ODA_SFP28_FW.tar file to the nodes, you can plumb up the copper 10G port (eth4) temporarily, and transfer the files over (ifconfig eth4 <ip addr> netmask <netmask addr>), for example:

Once the file has been transferred over and unpacked, execute the following commands to update the Broadcom firmware.

After the system has rebooted, execute the following ethtool commands to manually force the speed to 10000 and turn off autoneg.

You will only have to execute the above two commands if your switch speed is set to 10Gb. The ports come from the factory set to 25Gb and the optical transceivers do not auto-negotiate back to 10Gb when a 10Gb optical transceiver is used.

To make the network settings persistent across reboots, you can hard code this in the system bios.  In order to enter system bios, reset the system, and press F2 during startup. 

Select Advanced -> Oracle Dual Port 10Gbase-T Ethernet Controller -> Device Configuration Menu -> Operational Link Speed -> 10Gb (from AutoNeg). This change needs to be made for each of the two ports.

Alternatively, you can add the following options to each interface /etc/sysconfig/network-scripts/ifcfg-eth2 & ifcfg-eth3 files:

ETHTOOL_OPTS="speed 10000 duplex full autoneg off"

6. run “oakcli configure firstnet”
Now, you should be able to access the nodes remotely.

References

This solution has no attachment