ODA Significant Difference In Latency Between Same Computer and Each ODA Node

Asset ID:	1-72-1940062.1
Update Date:	2014-11-04
Keywords:

Solution Type Problem Resolution Sure

Solution 1940062.1 : ODA Significant Difference In Latency Between Same Computer and Each ODA Node

Applies to:

Oracle Database Appliance - Version All Versions to All Versions [Release All Releases]
Information in this document applies to any platform.

Symptoms

Latency and/or packet loss is noticed for connections to/from one or both ODA nodes. I/O to mounted NFS shares may also seem slow.
A trace route also highlights differences:

Second [problem] Node

Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.

C:\Documents and Settings\abc>tracert oda2-2

Tracing route to oda2-2 [10.11.12.13]
over a maximum of 30 hops:

1 2 42 ms 99 ms 99 ms 10.2.64.59 <
Trace complete.

First [good] Node

C:\Documents and Settings\abc>tracert oda2-1

Tracing route to oda2-1 [10.11.12.12]
over a maximum of 30 hops:

1 6 ms 2
Trace complete.

C:\>tracert oda2-1

Tracing route to oda2-1
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms 10.3.4.5
2 <1 ms <1 ms <1 ms 10.2.3.4

Trace complete.

C:\>tracert oda2-2

Tracing route to oda2-2
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms 170.111.66.11
2 9 ms 99 ms 99 ms 10.2.64.59

Trace complete.

I have also included the ifconfig:

login as: root
root@oda2-2's password:
Last login: Tue Sep 30 16:28:49 2014 from 170.111.66.11
[root@fjd-oda2-2 ~]# clear
[root@fjd-oda2-2 ~]# ifconfig -a
bond0 Link encap:Ethernet HWaddr 00:21:28:FF:7F:50
inet addr:10.2.64.59 Bcast:10.2.67.255 Mask:255.255.252.0
UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
RX packets:3204391 errors:0 dropped:859430 overruns:0 frame:0
TX packets:542478 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:334413088 (318.9 MiB) TX bytes:82210490 (78.4 MiB)

Cause

On node 2, the /etc/sysconfig/network-scripts/ifcfg-bond0 file is missing two parameters:

ifcfg-bond0 on first node:

#Comet Auto generated config file. Bond interface bond0
DEVICE=bond0
IPADDR=10.2.58.58
NETMASK=255.255.252.0
GATEWAY=10.5.3.1
ONBOOT=yes
BOOTPROTO=none
USERCTL=no
BONDING_OPTS="mode=active-backup miimon=100 primary=eth2"
TYPE=BOND

ifcfg-bond0 on second node:

#Comet Auto generated config file. Bond interface bond0
DEVICE=bond0
IPADDR=10.2.58.59
NETMASK=255.255.252.0
GATEWAY=10.5.3.1
ONBOOT=yes
BOOTPROTO=none
USERCTL=no

Discussion

Second node is missing BONDING_OPTS and TYPE.

This leads to the bond0 created with balance-rr (round robin) as bonding mode, instead of the active-backup mode.
In balance-rr mode, packets are transmitted in sequential order, using both slave interfaces.
This mode requires specific configuration on the switches, where an "etherchannel" or "trunk group" (depending on vendor) has to be configured across the two ports.

In contrast, the active-backup mode requires no special configuration for the switch ports.

The below two parameters ensure that the bond is created with active-backup as bonding mode.

BONDING_OPTS="mode=active-backup miimon=100 primary=eth2"
TYPE=BOND

Solution

1) Add the following lines to /etc/sysconfig/network-scripts/ifcfg-bond0:

BONDING_OPTS="mode=active-backup miimon=100 primary=eth2"
TYPE=BOND

2) Bring the bond down and back up by running as root on a single line:

ifdown bond0; ifup bond0

- True for BM and ODAVP ? - not sure about ODAVP

Attachments

This solution has no attachment