SuperCluster Grid Infrastructure Application VIP interface alias is created on the wrong netmask

Asset ID:	1-72-1993385.1
Update Date:	2015-04-25
Keywords:

Solution Type Problem Resolution Sure

Solution 1993385.1 : SuperCluster Grid Infrastructure Application VIP interface alias is created on the wrong netmask

Applies to:

SPARC SuperCluster T4-4 - Version All Versions to All Versions [Release All Releases]
Oracle Database - Enterprise Edition - Version 11.2.0.1 to 12.1.0.1 [Release 11.2 to 12.1]
Oracle SuperCluster T5-8 Hardware - Version All Versions and later
Oracle SuperCluster M6-32 Hardware - Version All Versions and later
Oracle Solaris on SPARC (64-bit)

Symptoms

Grid Infrastructure Application VIP alias created on 22bit network however ifconfig/dladm is reporting this interface as 24bit address incorrectly.
Underlying native interface is using the correct 255.255.252.0 / 22bit Netmask however creating VIP alias defaults to 255.255.255.0 / 24bit netmask value

E.g.
grid@vempdb01:~$ srvctl config network -k 2
Network exists: 2/10.53.16.0/255.255.252.0/bondeth0_v350, type static

grid@vempdb01:~$ srvctl config vip -n vempdb01
VIP exists: /10.53.16.132/10.53.16.132/10.53.16.0/255.255.252.0/bondeth0_v350, hosting node vempdb01

grid@vempdb01:~$ srvctl config vip -n vempdb02
VIP exists: /10.53.16.133/10.53.16.133/10.53.16.0/255.255.252.0/bondeth0_v350, hosting node vempdb02

rid@vempdb01:~$ oifcfg iflist -p -n
bondmgt0 10.51.206.0 PRIVATE 255.255.255.0
bondib0 172.20.4.0 PRIVATE 255.255.252.0
bondib1 172.20.4.0 PRIVATE 255.255.252.0
bondeth0_v2 10.53.4.0 PRIVATE 255.255.252.0
bondeth0_v321 10.53.116.0 PUBLIC 255.255.255.0
bondeth0_v350 10.53.16.0 PUBLIC 255.255.255.0               ---------> Wrong !
bondeth0_v626 172.20.0.0 PRIVATE 255.255.255.0

grid@vempdb01:~$ ifconfig bondeth0_v350
bondeth0_v350: flags=100201000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS,PHYSRUNNING> mtu 1500 index 16
       inet 10.53.16.44 netmask fffffc00 broadcast 10.53.19.255

grid@vempdb01:~$ ifconfig bondeth0_v350:1
bondeth0_v350:1: flags=100201040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,CoS,PHYSRUNNING> mtu 1500 index 16
       inet 10.53.16.132 netmask ffffff00 broadcast 10.53.16.255
grid@vempdb01:~$

# ipadm show-addr
bondeth0_v350/v4 static   ok           10.53.16.44/22
bondeth0_v350/v4a static   ok           10.53.16.132/24

Underlying native interface is using the correct 255.255.252.0 / 22bit Netmask however creating VIP alias defaults to 255.255.255.0 / 24bit netmask value

Cause

Based on the IP address value and the lack of entry in /etc/netmasks file its defaulting to 255.255.255.0 netmask range based on the IP Class.

Solution

To avoid the class mapping based on the IP# value, define the subnet in /etc/netmasks file.

E.g.
grid@vempdb01:~$ cat /etc/netmasks
172.20.4.0      255.255.252.0 # Generated by zonetools
10.53.116.0     255.255.255.0 # Generated by zonetools
172.20.4.0      255.255.252.0 # Generated by zonetools
10.53.16.0      255.255.252.0 # Added manually                ---------> Added 22bit address
10.51.206.0     255.255.255.0 # Generated by zonetools

Ensure the network is defined correctly in OCR, an ocrdump can be used to verify the network entry.

E.g.
% $GRID_HOME/bin/ocrdump /tmp/ocr.out

Then look for the network #2 entry..

Here is our key -->

[SYSTEM.CRSD.RESOURCES.ora!net2!network.CONFIG]
ORATEXT : ACL=owner:root:rwx,pgrp:root:r-x,other::r--,group:oinstall:r-x,user:grid:r-x~ACTION_FAILURE_TEMPLATE=~ACTION_SCRIPT=~AGENT_FILENAME=%CRS_HOME%/bin/orarootagent%CRS_EXE_SUFFIX%~ALIAS_NAME=~AUTO_START=restore~BASE_TYPE=ora.local_resource.type~CHECK_INTERVAL=1~DEFAULT_TEMPLATE=~DEGREE=1~DESCRIPTION=Oracle Network resource~ENABLED=1~LOAD=1~LOGGING_LEVEL=1~NAME=ora.net2.network~NLS_LANG=~NOT_RESTARTING_TEMPLATE=~OFFLINE_CHECK_INTERVAL=60~PROFILE_CHANGE_TEMPLATE=~RESTART_ATTEMPTS=5~SCRIPT_TIMEOUT=60~START_DEPENDENCIES=~START_TIMEOUT=0~STATE_CHANGE_TEMPLATE=~STOP_DEPENDENCIES=~STOP_TIMEOUT=0~TYPE=ora.network.type~TYPE_ACL=owner:root:rwx,pgrp:root:r-x,other::r--~TYPE_NAME=ora.network.type~TYPE_VERSION=2.2~UPTIME_THRESHOLD=1d~USR_ORA_AUTO=static~USR_ORA_ENV=~USR_ORA_IF=bondeth0_v350~USR_ORA_NETMASK=255.255.252.0~USR_ORA_SUBNET=10.53.16.0~VERSION=11.2.0.4.0~
SECURITY : {USER_PERMISSION : PROCR_ALL_ACCESS, GROUP_PERMISSION : PROCR_NONE, OTHER_PERMISSION : PROCR_NONE, USER_NAME : root, GROUP_NAME : root}

Above is used solely to confirm correct netmask is stored.

Restart Grid Infrastructure on all nodes.
This step is required since the value of the older incorrect netmask may still be held in CRSD master cache.

Post the restart dladm reports correct netmask value.

# ipadm show-addr
bondeth0_v350/v4 static ok 10.53.16.44/22
bondeth0_v350/v4a static ok 10.53.16.132/22

Attachments

This solution has no attachment