Enabling 8021.Q VLAN Tagging in Zero Data Loss Recovery Appliance Over Ingest Networks

Asset ID:	1-71-2047411.1
Update Date:	2017-10-30
Keywords:

Solution Type Technical Instruction Sure

Solution 2047411.1 : Enabling 8021.Q VLAN Tagging in Zero Data Loss Recovery Appliance Over Ingest Networks

Applies to:

Zero Data Loss Recovery Appliance X5 Hardware - Version All Versions and later
Zero Data Loss Recovery Appliance Software - Version 12.1.1.1.1 and later
Linux x86-64

Goal

Enable 8021.Q VLAN Tagged Interfaces in Zero Data Loss Recovery Appliance for Ethernet ingest networks.

Solution

IMPORTANT!!!

Please follow ALL naming conventions as suggested by ORACLE.

Listeners => LIST_<VLAN_ID>

Sample VLAN IP Information

Verify the needed ip addresses: (use nslookup)

Checking Host: zdlra01ingest01-v3101.us.oracle.com (10.128.181.111) ..
Checking Host: zdlra01ingest01-vip-v3101.us.oracle.com (10.128.181.112) ..
Checking Host: zdlra01ingest02-v3101.us.oracle.com (10.128.181.113) ..
Checking Host: zdlra01ingest02-vip-v3101.us.oracle.com (10.128.181.114) ..
Checking Host: zdlra01ingest-scan1-v3101.us.oracle.com
(10.128.181.116, 10.128.181.115, 10.128.181.117) ..

There are currently two scenarios for VLAN configuration on the ZDLRA.

Pre-install VLAN Configuration
Post-install VLAN Configuration

Pre-install VLAN Configuration: Converting an Untagged Ingest Network to Tagged Before Install

Prerequisites

This process assumes:
- The system has the base Exadata image on it.
- The ra_preinstall.pl step has been run
- OEDA install.sh has NOT been run
- The ip addresses and network information for bondeth0 that were entered into OEDA are correct.

Ensure that the ingest IP address has been configured with the VLAN IP interface.

Note:
For this example;
• VLAN_ID = 3101

1. Create the tagged interface names for the ingest network

[root@zdlra01adm01 ~]# cd /etc/sysconfig/network-scripts/

2. Copy the untagged interface files to a tagged interface file

root@zdlra01adm01 network-scripts]# cp ifcfg-bondeth0 ifcfg-bondeth0.3101

3. Edit ingest network ifcfg-bondeth0

Remove the lines: USERCTL, IPADDR, NETMASK, GATEWAY, NETWORK, BROADCAST from the ifcfg-bondeth0 interface.

Before Edits:

[root@zdlra01adm01 network-scripts]# cat ifcfg-bondeth0
#### DO NOT REMOVE THESE LINES ####
#### %GENERATED BY CELL% ####
DEVICE=bondeth0
USERCTL=no
BOOTPROTO=none
ONBOOT=yes
BONDING_OPTS="mode=802.3ad miimon=100 downdelay=200 updelay=200 lacp_rate=1 xmit_hash_policy=layer3+4"
IPV6INIT=no
IPADDR=10.128.181.111
NETMASK=255.255.248.0
GATEWAY=10.128.176.1
NETWORK=10.128.176.0
BROADCAST=10.128.181.255
[root@zdlra01adm01 network-scripts]#

After Edits:

[root@zdlra01adm01 network-scripts]# cat ifcfg-bondeth0
#### DO NOT REMOVE THESE LINES ####
#### %GENERATED BY CELL% ####
DEVICE=bondeth0
BOOTPROTO=none
ONBOOT=yes
BONDING_OPTS="mode=802.3ad miimon=100 downdelay=200 updelay=200 lacp_rate=1 xmit_hash_policy=layer3+4"
IPV6INIT=no
[root@zdlra01adm01 network-scripts]#

4. Edit the tagged interface ifcfg-bondeth0.VLAN_ID file to reflect the interface name

In this example, the VLAN_ID is 3101 so the file is ifcfg-bondeth0.3101
Remove lines: BONDING_OPTS
Change Lines: DEVICE
Add Lines: VLAN=yes

Before Edits:

[root@ zdlra01adm01 network-scripts]# cat ifcfg-bondeth0.3101
#### DO NOT REMOVE THESE LINES ####
#### %GENERATED BY CELL% ####
DEVICE=bondeth0
USERCTL=no
BOOTPROTO=none
ONBOOT=yes
BONDING_OPTS="mode=802.3ad miimon=100 downdelay=200 updelay=200 lacp_rate=1 xmit_hash_policy=layer3+4"
IPV6INIT=no
IPADDR=10.128.181.111
NETMASK=255.255.248.0
GATEWAY=10.128.176.1

After Edits:

[root@ zdlra01adm01 network-scripts]# cat ifcfg-bondeth0.3101
#### DO NOT REMOVE THESE LINES ####
#### %GENERATED BY CELL% ####
DEVICE=bondeth0.3101
BOOTPROTO=none
ONBOOT=yes
IPADDR=10.128.181.111
NETMASK=255.255.248.0
GATEWAY=10.128.176.1
VLAN=yes
[root@ zdlra01adm01 network-scripts]#

5. Rename the untagged rule file to be tagged

[root@zdlra01adm01 network-scripts]# mv rule-bondeth0 rule-bondeth0.3101

6. Rename the untagged route file to be tagged

[root@zdlra01adm01 network-scripts]# mv route-bondeth0 route-bondeth0.3101

7. Edit the tagged route file to reflect the tagged interface

- sed –i.orig –e ‘s/bondeth0/bondeth0.<VLAN_ID>/g' /etc/sysconfig/network-scripts/route-bondeth0.<VLAN_ID>

In our example VLAN_ID is 3101.

Example:

[root@zdlra01adm01 network-scripts]# sed –i.orig –e ‘s/bondeth0/bondeth0.3101/g' /etc/sysconfig/network-scripts/route-bondeth0.3101

Before Edits:

[root@zdlra01adm01 network-scripts]# cat route-bondeth0.3101
10.128.176.0/21 dev bondeth0 table 210
default via 10.128.176.1 dev bondeth0 table 210
[root@zdlra01adm01 network-scripts]#

After Edits:

[root@zdlra01adm01 network-scripts]# cat route-bondeth0.3101
10.128.176.0/21 dev bondeth0.3101 table 210
default via 10.128.176.1 dev bondeth0.3101 table 210
[root@zdlra01adm01 network-scripts]#

8. Edit the /etc/sysconfig/network file to reflect the tagged interface name

- sed –i.orig –e ‘s/ GATEWAYDEV=bondeth0/GATEWAYDEV=bondeth0.<VLAN_ID>/' /etc/sysconfig/network

In our example VLAN_ID is 3101.

Example:

sed -i.orig -e 's/GATEWAYDEV=bondeth0/GATEWAYDEV=bondeth0.3101/' /etc/sysconfig/network

9. Restart the network by rebooting the host

[root@zdlra01adm01 ~]# reboot

[root@zdlra01adm01 ~]# netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
0.0.0.0 10.128.176.1 0.0.0.0 UG 0 0 0 bondeth0.3101
10.128.176.0 0.0.0.0 255.255.248.0 U 0 0 0 bondeth0.3101
10.133.40.0 0.0.0.0 255.255.248.0 U 0 0 0 eth0
10.133.241.192 0.0.0.0 255.255.255.192 U 0 0 0 bondeth1
192.168.40.0 0.0.0.0 255.255.248.0 U 0 0 0 ib0
192.168.40.0 0.0.0.0 255.255.248.0 U 0 0 0 ib1
[root@zdlra01adm01 ~]#

*** Repeat the above steps 1-9 on each remaining ZDLRA compute node ***

10. Edit the XML file to reflect the new tagged interface

Edit the configuration xml change bondeth0 to bondeth0.VLAN_ID (In our case 3101)

[root@zdlraadm01]# sed –i.orig -e ’s/bondeth0/bondeth0.3101/g’ OEDA_FILE;

11. Continue with the install starting with OEDA install.sh step 1

Follow the manuals to continue the install as you would any other ZDLRA installation from install.sh step 1 forward.

Post-Install VLAN Configuration: Adding an Additional Tagged Ingest Network After Install

VLAN Information

Download the script from Oracle Support Document Enabling 8021.Q VLAN Tagging in Exadata Database Machine over client networks (Doc ID 1423676.1)

IMPORTANT!!!

Please follow ALL naming conventions as suggested by ORACLE.

Listeners => LIST_<VLAN_ID>

1. Stage the VLAN script in /home/oracle (ALL Compute nodes)

[root@zdlra01adm01 oracle]# ls dbm_vlan_config.sh
dbm_vlan_config.sh
[root@zdlra01adm01 oracle]#

[root@zdlra01adm02 oracle]# ls dbm_vlan_config.sh
dbm_vlan_config.sh
[root@zdlra01adm02 oracle]#

Verify the needed ip addresses: (use nslookup)

Run the dbm_vlan_config.sh script (ALL Compute Nodes): (root user)

On Node1:

[root@zdlra01adm01 oracle]# /home/oracle/dbm_vlan_config.sh add -v 3101 -i bondeth0 -ip 10.128.181.111 -n 255.255.248.0 -g 10.128.176.1

Exadata Database Machine VLAN Configurator
Copyright Oracle, Inc 2012
version 1.01 Dated 16-Apr-2015

Using iproute2 table ID: 221
Created config files in directory /etc/sysconfig/network-scripts
==> ifcfg-bondeth0.3101
==> rule-bondeth0.3101
==> route-bondeth0.3101

Determining if ip address 10.128.181.111 is already in use for device bondeth0.3101...
[root@zdlra01adm01 oracle]#

On Node2:

[root@zdlra01adm02 oracle]# /home/oracle/dbm_vlan_config.sh add -v 3101 -i bondeth0 -ip 10.128.181.113 -n 255.255.248.0 -g 10.128.176.1

Exadata Database Machine VLAN Configurator
Copyright Oracle, Inc 2012
version 1.01 Dated 16-Apr-2015

Using iproute2 table ID: 221
Created config files in directory /etc/sysconfig/network-scripts
==> ifcfg-bondeth0.3101
==> rule-bondeth0.3101
==> route-bondeth0.3101

Determining if ip address 10.128.181.113 is already in use for device bondeth0.3101...
[root@zdlra01adm02 oracle]#

Note:
All following commands are run on the First Compute Node(zdlra01adm01) as the identified user by the prompt unless specified.

2. Add the interface to Oracle Cluster Registry

[root@zdlra01adm01 oracle]# su - oracle
[oracle@zdlra01adm01 ~]$ . oraenv
ORACLE_SID = [zdlra31] ? +ASM1
The Oracle base has been set to /u01/app/oracle
[oracle@zdlra01adm01 ~]$ oifcfg setif -global bondeth0.3101/10.128.176.0:public

[oracle@zdlra01adm01 ~]$ oifcfg getif
bondeth0 10.10.62.0 global public
ib0 192.168.40.0 global cluster_interconnect
ib1 192.168.40.0 global cluster_interconnect
bondeth0.3101 10.128.176.0 global public
[oracle@zdlra01adm01 ~]$

Add the Network, VIPs and SCAN Addresses to OCR
As the root user add and then query the new network and VLAN to the OCR

Add:

[root@zdlra01adm01 oracle]# . oraenv
ORACLE_SID = [root] ? +ASM1
The Oracle base has been set to /u01/app/oracle
[root@zdlra01adm01 oracle]# srvctl add network -k 3101 -S 10.128.176.0/255.255.248.0/bondeth0.3101 -w static

Query:

[root@zdlra01adm01 oracle]# srvctl config network -k 3101
Network 3,101 exists
Subnet IPv4: 10.128.176.0/255.255.248.0/bondeth0.3101, static
Subnet IPv6:
Ping Targets:
Network is enabled
Network is individually enabled on nodes:
Network is individually disabled on nodes:
[root@zdlra01adm01 oracle]#

With the network defined, the VIP's and SCAN Addresses can be added to the OCR

[root@zdlra01adm01 oracle]# srvctl add vip -n zdlra01adm01 -k 3101 -A zdlra01ingest01-vip-v3101.us.oracle.com/255.255.248.0/bondeth0.3101
[root@zdlra01adm01 oracle]# srvctl add vip -n zdlra01adm02 -k 3101 -A zdlra01ingest02-vip-v3101.us.oracle.com/255.255.248.0/bondeth0.3101
[root@zdlra01adm01 oracle]# srvctl add scan -netnum 3101 -scanname zdlra01ingest-scan1-v3101.us.oracle.com
[root@zdlra01adm01 oracle]#

[root@zdlra01adm01 oracle]# su - oracle
[oracle@zdlra01adm01 ~]$ srvctl config vip -vip zdlra01ingest01-vip-v3101.us.oracle.com
VIP exists: network number 3,101, hosting node zdlra01adm01
VIP Name: zdlra01ingest01-vip-v3101.us.oracle.com
VIP IPv4 Address: 10.128.181.112
VIP IPv6 Address:
VIP is enabled.
VIP is individually enabled on nodes:
VIP is individually disabled on nodes:
[oracle@zdlra01adm01 ~]$

[oracle@zdlra01adm01 ~]$ srvctl config vip -vip zdlra01ingest02-vip-v3101.us.oracle.com
VIP exists: network number 3,101, hosting node zdlra01adm02
VIP Name: zdlra01ingest02-vip-v3101.us.oracle.com
VIP IPv4 Address: 10.128.181.114
VIP IPv6 Address:
VIP is enabled.
VIP is individually enabled on nodes:
VIP is individually disabled on nodes:
[oracle@zdlra01adm01 ~]$

[oracle@zdlra01adm01 ~]$ srvctl config scan -netnum 3101
SCAN name: zdlra01ingest-scan1-v3101.us.oracle.com, Network: 3,101
Subnet IPv4: 10.128.176.0/255.255.248.0/bondeth0.3101, static
Subnet IPv6:
SCAN 0 IPv4 VIP: 10.128.181.116
SCAN VIP is enabled.
SCAN VIP is individually enabled on nodes:
SCAN VIP is individually disabled on nodes:
SCAN 1 IPv4 VIP: 10.128.181.115
SCAN VIP is enabled.
SCAN VIP is individually enabled on nodes:
SCAN VIP is individually disabled on nodes:
SCAN 2 IPv4 VIP: 10.128.181.117
SCAN VIP is enabled.
SCAN VIP is individually enabled on nodes:
SCAN VIP is individually disabled on nodes:
[oracle@zdlra01adm01 ~]$

3. Start the VIPs (oracle user)

[oracle@zdlra01adm01 ~]$ srvctl start vip -vip zdlra01ingest01-vip-v3101.us.oracle.com
[oracle@zdlra01adm01 ~]$

[oracle@zdlra01adm02 ~]$ srvctl start vip -vip zdlra01ingest02-vip-v3101.us.oracle.com
[oracle@zdlra01adm02 ~]$

4. Check the status of the VIPs (oracle user)

[oracle@zdlra01adm01 ~]$ srvctl status vip -vip zdlra01ingest01-vip-v3101.us.oracle.com
VIP zdlra01ingest01-vip-v3101.us.oracle.com is enabled
VIP zdlra01ingest01-vip-v3101.us.oracle.com is running on node: zdlra01adm01
[oracle@zdlra01adm01 ~]$

[oracle@zdlra01adm01 ~]$ srvctl status vip -vip zdlra01ingest02-vip-v3101.us.oracle.com
VIP zdlra01ingest02-vip-v3101.us.oracle.com is enabled
VIP zdlra01ingest02-vip-v3101.us.oracle.com is running on node: zdlra01adm02
[oracle@zdlra01adm01 ~]$

5. Add and Start the DB Listener and SCAN Listener (oracle user)

Please follow ALL naming conventions as suggested by ORACLE.

Listeners => LIST_<VLAN_ID>

[oracle@zdlra01adm01 ~]$ srvctl add listener -l LIST_3101 -p 1521 -k 3101 -s
[oracle@zdlra01adm01 ~]$

[oracle@zdlra01adm01 ~]$ srvctl start listener -l LIST_3101
[oracle@zdlra01adm01 ~]$

[oracle@zdlra01adm01 ~]$ srvctl status listener -l LIST_3101
Listener LIST_3101 is enabled
Listener LIST_3101 is running on node(s): zdlra01adm01,zdlra01adm02
[oracle@zdlra01adm01 ~]$

[oracle@zdlra01adm01 ~]$ srvctl add scan_listener -netnum 3101 -listener LIST_SCAN_3101 -skip
[oracle@zdlra01adm01 ~]$

[oracle@zdlra01adm01 ~]$ srvctl start scan_listener -netnum 3101
[oracle@zdlra01adm01 ~]$

[oracle@zdlra01adm01 ~]$ srvctl status scan_listener -netnum 3101
SCAN Listener LIST_SCAN_3101_SCAN1_NET3101 is enabled
SCAN listener LIST_SCAN_3101_SCAN1_NET3101 is running on node zdlra01adm02
SCAN Listener LIST_SCAN_3101_SCAN2_NET3101 is enabled
SCAN listener LIST_SCAN_3101_SCAN2_NET3101 is running on node zdlra01adm02
SCAN Listener LIST_SCAN_3101_SCAN3_NET3101 is enabled
SCAN listener LIST_SCAN_3101_SCAN3_NET3101 is running on node zdlra01adm02
You have new mail in /var/spool/mail/oracle
[oracle@zdlra01adm01 ~]$

6. Add the database service that is associated with the network

[oracle@zdlra01adm01 ~]$ srvctl add service -d zdlra3 -s zdlra3_3101 -netnum 3101 -preferred zdlra31,zdlra32
[oracle@zdlra01adm01 ~]$

[oracle@zdlra01adm01 ~]$ srvctl start service -d zdlra3 -s zdlra3_3101
[oracle@zdlra01adm01 ~]$

[oracle@zdlra01adm01 ~]$ srvctl status service -d zdlra3 -s zdlra3_3101
Service zdlra3_3101 is running on instance(s) zdlra31,zdlra32
[oracle@zdlra01adm01 ~]$

Due to the init.ora parameter length limit of 255 characters, a TNS Alias must be created for subsequent use with the dispatcher registration step below. Ensure that the fully qualified domain name are used.

On Node1:
Add the following entry to:
- /u01/app/oracle/product/12.1.0.2/dbhome_1/network/admin/tnsnames.ora

ZDLRA31_3101 =

(DESCRIPTION =

(ADDRESS_LIST =

(ADDRESS = (PROTOCOL = TCP)(HOST = zdlra01ingest01-vip-v3101.us.oracle.com)(PORT = 1521))

(ADDRESS = (PROTOCOL = TCP)(HOST = zdlra01ingest-scan1-v3101.us.oracle.com)(PORT = 1521))

)

Now verify that the tnsnames.ora file is configured correctly using tnsping

[oracle@zdlra01adm01 ~]$ tnsping zdlra31_3101

TNS Ping Utility for Linux: Version 12.1.0.2.0 - Production on 01-APR-2016 15:46:59

Copyright (c) 1997, 2014, Oracle. All rights reserved.
Used parameter files:
/u01/app/oracle/product/12.1.0.2/dbhome_1/network/admin/sqlnet.ora
Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS_LIST = (ADDRESS = (PROTOCOL = TCP)(HOST = zdlra01ingest01-vip-v3101.us.oracle.com)(PORT = 1521)) (ADDRESS = (PROTOCOL = TCP)(HOST = zdlra01ingest-scan1-v3101.us.oracle.com)(PORT = 1521))))
OK (0 msec)
You have new mail in /var/spool/mail/oracle
[oracle@zdlra01adm01 ~]$

On Node2:
Add the following entry to:
- /u01/app/oracle/product/12.1.0.2/dbhome_1/network/admin/tnsnames.ora

ZDLRA32_3101 =

(DESCRIPTION =

(ADDRESS_LIST =

(ADDRESS = (PROTOCOL = TCP)(HOST = zdlra01ingest02-vip-v3101.us.oracle.com)(PORT = 1521))

(ADDRESS = (PROTOCOL = TCP)(HOST = zdlra01ingest-scan1-v3101.us.oracle.com)(PORT = 1521))

)

[oracle@zdlra01adm02 ~]$ tnsping zdlra32_3101

TNS Ping Utility for Linux: Version 12.1.0.2.0 - Production on 01-APR-2016 15:54:47
Copyright (c) 1997, 2014, Oracle. All rights reserved.
Used parameter files:
/u01/app/oracle/product/12.1.0.2/dbhome_1/network/admin/sqlnet.ora
Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS_LIST = (ADDRESS = (PROTOCOL = TCP)(HOST = zdlra01ingest02-vip-v3101.us.oracle.com)(PORT = 1521)) (ADDRESS = (PROTOCOL = TCP)(HOST = zdlra01ingest-scan1-v3101.us.oracle.com)(PORT = 1521))))
OK (10 msec)
[oracle@zdlra01adm01 ~]$

Cluster Configuration Complete!!!

7. Add the Dispatcher Entry

IMPORTANT!!!

Each Nodes dispatchers command MUST be run on the actual node.

On Node1:

[oracle@zdlra01adm01 ~]$ sqlplus / as sysdba

SQL*Plus: Release 12.1.0.2.0 Production on Mon Apr 4 09:52:54 2016

Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters and Automatic Storage Management options

SQL> alter system set dispatchers='(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=zdlra01ingest01-vip-v3101.us.oracle.com))(SDU=65536))(SERVICE=zdlra3XDB3101)(DISPATCHERS=4)(LISTENER=zdlra31_3101)' scope=both sid='zdlra31';

System altered.

SQL>

STOP!!!

Please login to the second/other compute node before executing the node's Dispatchers.

On Node2:

[root@zdlra01adm02 ~]# su - oracle

[oracle@zdlra01adm02 ~]$ sqlplus / as sysdba

SQL*Plus: Release 12.1.0.2.0 Production on Mon Apr 4 10:23:07 2016

Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters and Automatic Storage Management options

SQL>

SQL> alter system set dispatchers='(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=zdlra01ingest02-vip-v3101.us.oracle.com))(SDU=65536))(SERVICE=zdlra3XDB3101)(DISPATCHERS=4)(LISTENER=zdlra32_3101)' scope=both sid='zdlra32';

System altered.

SQL>

8. Run the following sql commands to address a known bug for listener networks:

Bug 22264951 - CRSD_ORAAGENT_ORACLE FAILS TO SET LISTENER_NETWORKS AND REMOTE_LISTENER

Introduced in Ver: 12.1.0.2.13

On Node1:

SQL> ALTER SYSTEM SET
listener_networks='((NAME=net3101)(LOCAL_LISTENER=(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=10.128.181.112)(PORT=1521)))))','((NAME=net3101)(REMOTE_LISTENER=zdlra01ingest-scan1-v3101.us.oracle.com:1521))'
SCOPE=BOTH SID='zdlra31';System altered.

On Node2:

SQL> ALTER SYSTEM SET
listener_networks='((NAME=net3101)(LOCAL_LISTENER=(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=10.128.181.114)(PORT=1521)))))','((NAME=net3101)(REMOTE_LISTENER=zdlra01ingest-scan1-v3101.us.oracle.com:1521))'
SCOPE=BOTH SID='zdlra32';

System altered.

9. Restart ZDLRA Database:

After modifying the database,it is essential to restart the database to ensure the configuration comes up cleanly with the newly defined VLAN Listener information.

Refer to the Recovery Appliance Owners Guide, Chapter 12 for procedure to Shutting Down Recovery Appliance and Starting Up Recovery Appliance.

10. Reboot the All the Compute Nodes

On Node1:

# shutdown -r -y now

On Node2:

# shutdown -r -y now

11. Add the interface to host table ( as rasys/ra )

SQL> update host set BACKUP_IP_ADDRESS='10.128.181.112,’ || BACKUP_IP_ADDRESS where NODE_NAME='zdlra01adm01.us.oracle.com';

1 row updated.

SQL> update host set BACKUP_IP_ADDRESS='10.128.181.114,’ || BACKUP_IP_ADDRESS where NODE_NAME='zdlra01adm02.us.oracle.com';

1 row updated.

SQL>

SQL> commit;
Commit complete.
SQL>

12. Configure Protected Databases (Performed on the Client/Protected DB System!)

This needs to be done on CLI (Manually).
Not yet Supported in Enterprise Manager

Create the sqlnet.ora file on all compute nodes in the cluster if not already defined.
If the wallet definition already exists, do NOT change it.
- Include WALLET_LOCATION
- Include SQLNET.EXPIRE_TIME

[oracle@client175 ~]$ vi /u01/app/oracle/product/12.1.0.1.0/dbhome_1/network/admin/sqlnet.ora

SQLNET.WALLET_OVERRIDE = true

NAMES.DIRECTORY_PATH= (TNSNAMES, EZCONNECT)

WALLET_LOCATION =

(SOURCE =

(METHOD = FILE)

(METHOD_DATA =

(DIRECTORY = /u01/app/oracle/product/12.1.0.2/dbhome_1/dbs/zdlra)

)

SQLNET.EXPIRE_TIME = 10

13. Create or update the Wallet on all nodes in the cluster

Verify the existence of the Auto Login Wallet which will hold the VPC Credentials on all nodes in the cluster

[oracle@client175 dbs]$ ls -al /u01/app/oracle/product/12.1.0.1.0/dbhome_1/dbs/zdlra

total 16
drwxr-x--- 2 oracle oinstall 4096 Apr 23 2015 .
drwxr-xr-x 7 oracle oinstall 4096 Apr 5 07:28 ..
-rwxr-x--- 1 oracle oinstall 4925 Mar 31 15:52 cwallet.sso
-rw------- 1 oracle oinstall 0 Apr 23 2015 cwallet.sso.lck
[oracle@client175 dbs]$

If the wallet exists, then query the current credentials - the key thing is you should not be asked for a password

$ mkstore -wrl file:/u01/app/oracle/product/12.1.0.2/dbhome_1/dbs/zdlra -listCredential
Oracle Secret Store Tool : Version 12.1.0.2
Copyright (c) 2004, 2014, Oracle and/or its affiliates. All rights reserved.

List credential (index: connect_string username)

3: zdlra07ingest-scan7:1521/zdlra7:dedicated vpcuser

2: zdlra03ingest-scan3:1521/zdlra1b:dedicated vpcuser

1: zdlra05ingest-scan5:1521/ZDLRA5:dedicated vpcuser

[oracle@client175 dbs]$

14. Now add the vpcuser wallet credential

[oracle@client175 dbs]$ mkstore -wrl file:/u01/app/oracle/product/12.1.0.1.0/dbhome_1/dbs/zdlra -createCredential zdlra01ingest-scan1-v3101:1521/zdlra3_3101:dedicated vpcuser welcome123
Oracle Secret Store Tool : Version 12.1.0.1
Copyright (c) 2004, 2012, Oracle and/or its affiliates. All rights reserved.
Create credential oracle.security.client.connect_string6
[oracle@client175 dbs]$ mkstore -wrl file:/u01/app/oracle/product/12.1.0.1.0/dbhome_1/dbs/zdlra -listCredential
Oracle Secret Store Tool : Version 12.1.0.1
Copyright (c) 2004, 2012, Oracle and/or its affiliates. All rights reserved.

List credential (index: connect_string username)

4: zdlra01ingest-scan1-v3101:1521/zdlra3_3101:dedicated vpcuser

3: zdlra07ingest-scan7:1521/zdlra7:dedicated vpcuser

2: zdlra03ingest-scan3:1521/zdlra1b:dedicated vpcuser

1: zdlra05ingest-scan5:1521/ZDLRA5:dedicated vpcuser

oracle@client175 dbs]$

If the wallet does NOT exist, then create the auto login wallet.

$ mkstore -wrl file:/u01/app/oracle/product/12.1.0.2/dbhome_1/dbs/zdlra -createALO -createCredential zdlra01ingest-scan1-v3101:1521/zdlra3_3101:dedicated vpcuser
Oracle Secret Store Tool : Version 12.1.0.2
Copyright (c) 2004, 2014, Oracle and/or its affiliates. All rights reserved.

Your secret/Password is missing in the command line
Enter your secret/Password:
Re-enter your secret/Password:
Create credential oracle.security.client.connect_string1
You have new mail in /var/spool/mail/oracle

$ mkstore -wrl file:/u01/app/oracle/product/12.1.0.2/dbhome_1/dbs/zdlra -listCredential

List credential (index: connect_string username)

4: zdlra01ingest-scan1-v3101:1521/zdlra3_3101:dedicated vpcuser

3: zdlra07ingest-scan7:1521/zdlra7:dedicated vpcuser

2: zdlra03ingest-scan3:1521/zdlra1b:dedicated vpcuser

1: zdlra05ingest-scan5:1521/ZDLRA5:dedicated vpcuser

[oracle@client175 dbs]$

15. Register the Protected Database in the ZDLRA

[oracle@client175 dbs]$ rman target / catalog /@zdlra01ingest-scan1-v3101:1521/zdlra3_3101:dedicated

Recovery Manager: Release 12.1.0.1.0 - Production on Wed Apr 6 16:14:09 2016

connected to target database: DB1210SC (DBID=973184904)
connected to recovery catalog database
recovery catalog schema release 12.01.00.02. is newer than RMAN release

RMAN> register database;

database registered in recovery catalog
starting full resync of recovery catalog
full resync complete

RMAN>

16. Configure the Backup Devices

RMAN> CONFIGURE CHANNEL DEVICE TYPE 'SBT_TAPE' FORMAT '%d_%U' PARMS "SBT_LIBRARY=/u01/app/oracle/product/12.1.0.1.0/dbhome_1/lib/libra.so, ENV=(RA_WALLET='location=file:/u01/app/oracle/product/12.1.0.1.0/dbhome_1/dbs/zdlra credential_alias=zdlra01ingest-scan1-v3101:1521/zdlra3_3101:dedicated')";

old RMAN configuration parameters:
CONFIGURE CHANNEL DEVICE TYPE 'SBT_TAPE' FORMAT '%d_%U' PARMS "SBT_LIBRARY=/u01/app/oracle/product/12.1.0.1.0/dbhome_1/lib/libra.so, ENV=(RA_WALLET='location=file:/u01/app/oracle/product/12.1.0.1.0/dbhome_1/dbs/zdlra credential_alias=zdlra01ingest-scan1:1521/zdlra3:dedicated')";
new RMAN configuration parameters:
CONFIGURE CHANNEL DEVICE TYPE 'SBT_TAPE' FORMAT '%d_%U' PARMS "SBT_LIBRARY=/u01/app/oracle/product/12.1.0.1.0/dbhome_1/lib/libra.so, ENV=(RA_WALLET='location=file:/u01/app/oracle/product/12.1.0.1.0/dbhome_1/dbs/zdlra credential_alias=zdlra01ingest-scan1-v3101:1521/zdlra3_3101:dedicated')";
new RMAN configuration parameters are successfully stored
starting full resync of recovery catalog
full resync complete

RMAN>

17. Run a Test Backup

RMAN> backup device type sbt current controlfile;

Starting backup at 06-APR-16
using channel ORA_SBT_TAPE_1
channel ORA_SBT_TAPE_1: starting full datafile backup set
channel ORA_SBT_TAPE_1: specifying datafile(s) in backup set
including current control file in backup set
channel ORA_SBT_TAPE_1: starting piece 1 at 06-APR-16
channel ORA_SBT_TAPE_1: finished piece 1 at 06-APR-16
piece handle=DB1210SC_q0r2cku0_1_1 tag=TAG20160406T195751 comment=API Version 2.0,MMS Version 3.15.1.22
channel ORA_SBT_TAPE_1: backup set complete, elapsed time: 00:00:05
Finished backup at 06-APR-16

RMAN>

18. Configuring Real-Time Redo Transport

SQL> alter system set log_archive_config='dg_config=(zdlra3,vlan3101)';
System altered.
SQL>

SQL> alter system set redo_transport_user=vpcuser;
System altered.
SQL>

SQL> alter system set log_archive_dest_2='SERVICE="zdlra01ingest-scan1-v3101:1521/zdlra3_3101:dedicated", VALID_FOR=(ALL_LOGFILES, ALL_ROLES) ASYNC DB_UNIQUE_NAME=zdlra3';
System altered.
SQL>

19. Bounce the Protected Database

The database must next be restarted to pickup the new credentials for Real-Time Redo Transport.

This can be performed in a rolling fashion if needed, but for ease of the procedure we shutdown and restart the database.

[oracle@client175 ~]$ srvctl stop database -db db1210sc
[oracle@client175 ~]$ srvctl status database -db db1210sc
Instance db1210sc is not running on node client175
[oracle@client175 ~]$ srvctl start database -db db1210sc
[oracle@client175 ~]$ srvctl status database -db db1210sc
Instance db1210sc is running on node client175
[oracle@client175 ~]$

Attachments

This solution has no attachment