IXP Apply Changes is Failing Due to VIP Reachability Issue During or After Upgrade To PIC 9

Asset ID:	1-72-1929462.1
Update Date:	2016-12-27
Keywords:

Solution Type Problem Resolution Sure

Solution 1929462.1 : IXP Apply Changes is Failing Due to VIP Reachability Issue During or After Upgrade To PIC 9

Applies to:

Oracle Communications Performance Intelligence Center (PIC) Software - Version 9.0.3 to 9.0.4 [Release 9.0]
Information in this document applies to any platform.

Symptoms

When doing the "Apply Changes" in NSP CCM after configurations changes, there is error message and changes are not applied:

"Failed to apply changes because of the following errors : Unable to connect host - <IP>. Please contact system administrator

Changes

Cause

The VIP is not started after upgrade or datawarehouse are taking VIP owneship.

Solution

Be careful this action could impact severely the good behaviour of the system, so if in doubt please open a SR to check if this is the right command to use.

Issue during upgrade

VIP must be restarted:

This must be done simultaneously to the "Apply Changes" action on CCM. It means the menu on GUI must be ready.
On an IXP, type the command:
sudo setVIP <IP> start
On CCM from NSP GUI, launch the "Apply Changes"
Next time "Apply Changes" will be launched, there should be no issue.

Issue after upgrade

Check

During PIC 9 upgrade the Xdr storage are changed to DWS, in some occasion the VIP can remain attached to the server even if it no longer appears in the comcol/IDB config.

Check to recognize this solution will help:

ip addr show

Expected output is the following:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth01: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 qlen 1000
link/ether 78:e7:d1:59:ac:78 brd ff:ff:ff:ff:ff:ff
3: eth02: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 qlen 1000
link/ether 78:e7:d1:59:ac:78 brd ff:ff:ff:ff:ff:ff
4: sit0: <NOARP> mtu 1480 qdisc noop
link/sit 0.0.0.0 brd 0.0.0.0
5: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue
link/ether 78:e7:d1:59:ac:78 brd ff:ff:ff:ff:ff:ff
inet 169.254.100.13/24 brd 169.254.100.255 scope global bond0
inet6 fe80::7ae7:d1ff:fe59:ac78/64 scope link
valid_lft forever preferred_lft forever
6: bond1: <BROADCAST,MULTICAST,MASTER> mtu 1500 qdisc noop
link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
7: bond2: <BROADCAST,MULTICAST,MASTER> mtu 1500 qdisc noop
link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
8: bond3: <BROADCAST,MULTICAST,MASTER> mtu 1500 qdisc noop
link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
9: bond0.3@bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue
link/ether 78:e7:d1:59:ac:78 brd ff:ff:ff:ff:ff:ff
inet 10.129.125.75/26 brd 10.129.125.127 scope global bond0.3
inet 10.129.125.84/26 brd 10.129.125.127 scope global secondary bond0.3
inet6 fe80::7ae7:d1ff:fe59:ac78/64 scope link
valid_lft forever preferred_lft forever

This example is for blade where we have a bond0.3 interface. the IP of the server is 10.129.125.75 and the VIP is 10.129.125.84 which is tagged secondary address.

Solution

On a DWS this secondary address must not exist as the DWS is no longer part of a subsystem.

Ensure daqConfig and daqManager process are off:

pm.set off daqConfig daqManager
+ iset -fcntl=off PmControl where "procTag like 'daqConfig|daqManager'"
: procTag cntl command
NEW: daqConfig off daqConfig
NEW: daqManager off daqManager
=== changed 2 records ===
+ pm.wakeup

To remove the secondary IP use the following command as root:

ip addr del 10.129.125.84/26 dev bond0.3

Of course this is an example and the IP/subnet as well as interface name must be customized depending on the customer config/hardware.

Attachments

This solution has no attachment