ODA X6-2S/X6-2M/X6-2L: Deploy using VLAN

Asset ID:	1-71-2216567.1
Update Date:	2018-01-16
Keywords:

Solution Type Technical Instruction Sure

Solution 2216567.1 : ODA X6-2S/X6-2M/X6-2L: Deploy using VLAN

Applies to:

Oracle Database Appliance X6-2 Hardware - Version All Versions and later
Oracle Database Appliance Software - Version 12.1.2.8 to 12.1.2.9 [Release 12.1]
Information in this document applies to any platform.

Goal

The goal of this bulletin is to show how you could deploy an ODA X6-2S/X6-2M/X6-2L with an 8021.Q VLAN Tagged Interfaces for external client access networks.

This note assumes :
ODA X6-2S/X6-2M/X6-2L
ODA software version 12.1.2.8.1 or 12.1.2.9.0
The deploy will be executed by command-line using a JSON configuration file

Note: the deploy done through the graphical web interface does not support VLAN interface

Solution

VLAN Tagged Interfaces - Background

Virtual Local Area Networks (VLANs) are used to separate Broadcast Domains in the Network. To share the separation information on which packet belongs to which VLAN between switches and end systems, Ethernet frames gets the VLAN identification number inserted into the header. This number is called the VLAN tag.
If you need additional networks for external access and prefer to enable isolation among them, 8021.Q based VLAN tagging is a solution. This document provides detailed instructions on how to configure and use such additional VLAN tagged client access networks.

1. Setting Up VLAN Tagging at OS level - (Using ODA_vlan_config utility)

To setup VLAN the "ODA_vlan_config.sh" utility can be used:

# ./ODA_vlan_config.sh del -v 620 -i sfpbond1 -h

Oracle Database Appliance BareMetal VLAN Configurator
Copyright Oracle, Inc. 2015
version 1.03 Dated 21-Dec-2016

Usage: ODA_vlan_config.sh [ help | man | -h ]
       ODA_vlan_config.sh add
                              -v <VLAN ID>
                              -i <Interface>
                              -ip <IPv4 address>
                              -n <Netmask>
                              -g <Gateway>
                              -c
       ODA_vlan_config.sh del
                              -v <VLAN ID>
                              -i <Interface>
                             [-f] #only if the VLAN is not in use by the clusterware
ODA_vlan_config.sh show

Example:

# ./ODA_vlan_config.sh add -v 620 -i sfpbond1 -ip 10.24.128.243 -n 255.255.255.0 -g 10.24.128.1

Oracle Database Appliance BareMetal VLAN Configurator
Copyright Oracle, Inc. 2015
version 1.03 Dated 21-Dec-2016

Using iproute2 table ID: 221
Created config files in directory /etc/sysconfig/network-scripts
==> ifcfg-sfpbond1.620
==> rule-sfpbond1.620
==> route-sfpbond1.620

Device "sfpbond1.620" does not exist.
Determining if ip address 10.24.128.243 is already in use for device sfpbond1.620...

Following VLAN(s) are now configured:
sfpbond1.620 | 620 | sfpbond1 | 10.24.128.243 | 255.255.255.0 | up

If you need to delete a VLAN, before run "ODA_vlan_config.sh del -v <VLAN ID> -i <Interface>" you must setup the SRVCTL environment variable, pointing to Grid Infrastructure srvctl command, example:

# export SRVCTL=/u01/app/12.1.0.2/grid/bin/srvctl
# ./ODA_vlan_config.sh del -v 620 -i sfpbond1

If you did not run the deploy yet, srvctl is not installed, then you should use the "-f" option

# ODA_vlan_config.sh del -v <VLAN ID> -i <Interface> -f
example:
# ./ODA_vlan_config.sh del -v 620 -i sfpbond1 -f

2. DCS VLAN Setup

Once the VLAN is up&running and working properly you need execute the following steps:

Check the DCS-Agent Version

You could check the DCS-agent version doing:

rpm -qa|grep dcs-agent

example:

# rpm -qa|grep dcs-agent
dcs-agent-12.1.2.8.1_LINUX.X64_160925-1.x86_64

If on 12.1.2.9.0

1) initctl stop initdcsagent
2) initctl start initdcsagent
3) odacli list-networkinterfaces

example:

# ./ODA_vlan_config.sh add -v 620 -i sfpbond1 -ip 10.24.128.243 -n 255.255.255.0 -g 10.24.128.1

Oracle Database Appliance BareMetal VLAN Configurator
Copyright Oracle, Inc. 2015
version 1.02 Dated 16-Dec-2016

Using iproute2 table ID: 221
Created config files in directory /etc/sysconfig/network-scripts
==> ifcfg-sfpbond1.620
==> rule-sfpbond1.620
==> route-sfpbond1.620

Device "sfpbond1.620" does not exist.
Determining if ip address 10.24.128.243 is already in use for device sfpbond1.620...

Following VLAN(s) are now configured:
sfpbond1.620 | 620 | sfpbond1 | 10.24.128.243 | 255.255.255.0 | up

# ifconfig sfpbond1.620
sfpbond1.620 Link encap:Ethernet HWaddr 90:E2:BA:C6:D5:D8
inet addr:10.24.128.243 Bcast:10.24.128.255 Mask:255.255.255.0
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)

# initctl stop initdcsagent
initdcsagent stop/waiting

# initctl start initdcsagent
initdcsagent start/running, process 31470

# /opt/oracle/dcs/bin/odacli list-networkinterfaces

ID Name NIC Type
---------------------------------------- -------------------- ------------- ----------
b324f474-627e-4b15-af83-2ed21eda845a btbond1 btbond1 Bond
595dfda9-67a6-4fc9-96af-afe966c287a7 em1 em1 Physical
2033de33-99fb-4d53-8235-6ef088db54fe em2 em2 Physical
62a09165-d9a5-4b8d-8eaa-45022988215b em3 em3 Physical
40070762-6e87-42b4-8eb2-b8f9aa642849 em4 em4 Physical
d70bbb83-e7de-462a-8892-42bdb687d6dc sfpbond1 sfpbond1 Bond
26a0c940-5705-460a-b51d-ab8723d06f35 priv0 priv0 Dummy
49cf2709-3c75-4c2c-bf53-718757385335 sfpbond1.620 sfpbond1.620 Bond

Note the command "odacli list-networkinterfaces" must show your VLAN

example:

# /opt/oracle/dcs/bin/odacli list-networkinterfaces

If on 12.1.2.8.1

1) initctl stop initdcsagent
2) cp -r /opt/oracle/dcs/repo/node_0 /tmp/DerbyDB_backup
3) rm -rf /opt/oracle/dcs/repo/node_0
4) initctl start initdcsagent
5) odacli list-networkinterfaces

Note:due to step #3, we are cleaning up the metadata database, so adding VLAN's should be the first step before adding any more resources to the system, you should never execute such step post-deploy

Note the command "odacli list-networkinterfaces" must show your VLAN

example:

# /opt/oracle/dcs/bin/odacli list-networkinterfaces

3. Deploy, create-appliance

For an example about JSON file to create-appliance you may check on Database Appliance X6-2S and X6-2M Deployment and User’s Guide

$ cat create-appliance.json
{
"instance" : {
"name" : "odas100",
"instanceBaseName" : "odas100",
"dbEdition" : "EE",
"timeZone" : "UTC",
"ntpServers" : ["10.0.3.14"],
"dnsServers" : ["10.0.4.10","10.0.4.11","10.0.4.12"],
"domainName" : "example.com",
"isRoleSeparated" : true,
"osUserGroup" : {
"groups" : [ {
"groupId" : 1001,
"groupName" : "oinstall",
"groupRole" : "oinstall"
}, {
"groupId" : 1002,
"groupName" : "dbaoper",
"groupRole" : "dbaoper"
}, {
"groupId" : 1003,
"groupName" : "dba",
"groupRole" : "dba"
}, {
"groupId" : 1004,
"groupName" : "asmadmin",
"groupRole" : "asmadmin"
}, {
"groupId" : 1005,
"groupName" : "asmoper",
"groupRole" : "asmoper"
}, {
"groupId" : 1006,
"groupName" : "asmdba",
"groupRole" : "asmdba"
} ],
"users" : [ {
"userId" : 1000,
"userName" : "oracle",
"userRole" : "oracleUser"
}, {
"userId" : 1001,
"userName" : "grid",
"userRole" : "gridUser"
} ]
}
},
"nodes" : [ {
"nodeNumber" : "0",
"nodeName" : "odas100",
"network" : [ {
"nicName" : "sfpbond1.620",
"ipAddress" : "10.24.128.243",
"subNetMask" : "255.255.255.0",
"gateway" : "10.24.128.1",
"networkType" : [ "Public" ],
"isDefaultNetwork" : true
} ],
"ilom" : {
"ilomName":"odambox-c",
"ipAddress":"10.0.2.10",
"subNetMask":"255.255.255.0",
"gateway":"10.0.2.1"
}
} ],
"grid" : {
"diskGroup" : [ {
"diskGroupName" : "DATA",
"redundancy" : "NORMAL",
"diskPercentage" :70
}, {
"diskGroupName" : "RECO",
"redundancy" : "NORMAL",
"diskPercentage" :30
} ],
"language" : "en"
},
"database" : {
"dbName" : "db1",
"dbVersion" : "12.1.0.2",
"instanceOnly" : false,
"isCdb" : true,
"pdBName" : "pdb1",
"pdbAdminuserName" : "pdbuser",
"adminPassword" : "welcome1",
"dbType" : "SI",
"dbTargetNodeNumber" : "0",
"dbClass" : "OLTP",
"dbShape" : "odb1",
"dbStorage" : "ACFS",
"dbCharacterSet" : {
"characterSet" : "AL32UTF8",
"nlsCharacterset" : "AL16UTF16",
"dbTerritory" : "AMERICA",
"dbLanguage" : "AMERICAN"
},
"dbConsoleEnable" : false
},
"asr" :{
"userName":"john.smith@example.com",
"password":"xxxxx",
"proxyServerName":"www-proxy.example.com",
"proxyPort":"80",
"proxyUserName":"",
"proxyPassword":"",
"snmpVersion":"v3"
}
}

Note as on the JSON file you must use as "nicName" your VLAN device, in this example: "sfpbond1.620"

Then you can now execute:

/opt/oracle/dcs/bin/odacli create-appliance -r /tmp/odas100.json

References

<NOTE:1500709.1> - ODA (Oracle Database Appliance): Enable 8021.Q VLAN Tagged Interfaces for external client access networks
<BUG:25297818> - ODALITE: CMDLINE DEPLOY IS FAILING USING VLAN
<BUG:25291350> - ODALITE: PLEASE PROVIDE VLAN SUPPORT

Attachments

This solution has no attachment