How To Change Existing VLAN IDs in Physical Exalogic Environments

Asset ID:	1-79-2120490.1
Update Date:	2016-04-25
Keywords:

Solution Type Predictive Self-Healing Sure

Solution 2120490.1 : How To Change Existing VLAN IDs in Physical Exalogic Environments

Applies to:

Oracle Exalogic Elastic Cloud Software
Oracle Exalogic Elastic Cloud X2-2 Hardware - Version X2 to X5 [Release X2 to X5]
Linux x86-64

Purpose

Network reconfiguration in customer’s data center may result in changes to VLAN IDs for networks provisioned on Exalogic. Such changes require reconfiguration on the InfiniBand NM2 Gateway switches on Exalogic. The VLANs provisioned on the switches must match those for the modified VLAN IDs and vNICs must be plumbed to the right VLANs.

This document describes a procedure to reconfigure NM2 Gateway Switches on Exalogic to accommodate the changed VLAN IDs.

NOTE: This procedure is applicable to Exalogic racks running Linux on bare metal (physical configuration) only, and does not apply to Virtual or Solaris environments.

The following shows the network traffic flow from the Exalogic rack to the customer’s client network:

Compute Nodes -> EOIB Network Interface -> NM2-GW -> 10Gb-Uplink(VLAN) -> 3 layer Switch(VLAN) -> Customer’s Client Network -> Other Hosts

This document provides a step-by-step procedure to reconfigure Exalogic to match the new VLAN IDs.

Details

Perform the following procedure to reconfigure Exalogic:

Backup the Network Configurations of InfiniBand Switches and Compute Nodes

Using ExaBR, backup the configurations of all the NM2 Gateway and 36p Switches on Exalogic.

For details on how to use ExaBR, refer to the following MOS note:

<Note 1912063.1>: Exalogic Lifecycle Toolkit Release 14.2

Modify vNICs and vLANs on NM2 Gateway Switches

Perform the following steps to modify the vLANs and vNICs configured on the InfiniBand Gateway switches:

1. Login to the one of the NM2 Gateway Switches as root

2. Identify all the VNICs configured on the original (old) VLAN ID (300 in this example)

e.g.

[root@el01gw01-adm ~]# showvnics | grep “ 300 “
0 UP N D4C186E7068BEB63 el01cn01 EL-C 192.168.10.101 0000 00:21:F6:55:E4:54 300 0xffff 0A-ETH-1
1 UP N B1E79BB655B07CBA el01cn04 EL-C 192.168.10.104 0000 00:21:F6:01:69:5D 300 0xffff 0A-ETH-1
[root@el01gw01-adm ~]#

3. Manually delete all VNICs configured on the VLAN ID (the vNICs displayed in the example for VLAN ID 300)

e.g.

[root@el01gw01-adm ~]# deletevnic 0A-ETH-1 0
vNIC ID 0 deleted
IO Adapter for vNIC deleted

[root@el01gw01-adm ~]# deletevnic 0A-ETH-1 1
vNIC ID 1 deleted
IO Adapter for vNIC deleted

[root@el01gw01-adm ~]# showvnics | grep “ 300 “
[root@el01gw01-adm ~]

NOTE: Delete all vNICS that are configured on the old VLAN ID, including those in WAIT-IOA or any other states.

4. Identify the Partition KEY and Connector for the old VLAN ID (e.g. 300)

e.g.

[root@el01gw01-adm ~]# showvlan | grep “ 300 “

0A-ETH-1 300 0xffff

5. Manually delete the old VLAN

e.g.

[root@el01gw01-adm ~]# deletevlan 0A-ETH-1 -vlan 300

6. Create a new VLAN with the new ID (e.g. 305), using the same connector and the same partition KEY.

e.g.

[root@el01gw01-adm ~]# createvlan 0A-ETH-1 -VLAN 305 -PKEY default

7. Recreate the deleted VNICs using the new VLAN ID, but with the original values of the connector, GUID, MAC address, and PKEY

e.g.

[root@el01gw01-adm ~]# createvnic 0A-ETH-1 -GUID B1:E7:9B:B6:55:B0:7C:BA -MAC 00:21:F6:01:69:5D -PKEY default -VLAN 305
vNIC created

[root@el01gw01-adm ~]# createvnic 0A-ETH-1 -GUID D4:C1:86:E7:06:8B:EB:63 -MAC 00:21:F6:55:E4:54 -PKEY default -VLAN 305
vNIC created

[root@el01gw01-adm ~]#

8. Repeat steps 1 to 7 on all remaining NM2 Gateway Switches on Exalogic.

Restart the Network on Compute Nodes

Restart the network service on all compute nodes:

[root@el01cn01 ~]# service network restart

Verify the Changes to VLAN IDs

Ping the compute node over the EoIB address from outside the rack.

Connect to the compute node over ssh.

Confirm that ALL compute nodes are accessible over their EoIB addresses from outside the rack, and both ping and ssh work.

References

<NOTE:1912063.1> - Exalogic Lifecycle Toolkit Release 14.2
<NOTE:1551388.1> - How To Change Predefined VLAN ID for the EoIB-external-mgmt Network Configured in Oracle Enterprise Manager Ops Center (EMOC)
https://docs.oracle.com/cd/E18476_01/doc.220/e18478/commproc.htm#ELMOG155
<NOTE:1374890.1> - Exalogic FAQs

Attachments

This solution has no attachment