Sun Virtual Library Extension (VLE)
 

PLA-Support>Sun Systems>TAPE>Virtual Tape>SN-TP: VLE
 

In this Document

Symptoms

Cause

Solution

Applies to:

Symptoms

Problem Description:

This potential problem applies only to VLEs running VLE 1.4.1 code with 3, 6, 9 or 12 remote copy links.

The problem was introduced in VLE1.4.1 and affects remote copy link load balancing.  If the VLE has exactly 3 remote copy links defined, the data transfer load will drift to using only a single link for data transfer, while using the other 2 links for metadata transfers only.  Because metadata transfers are small, from the GUI 'Port Status' tab it appears as if only a single link is being used.  This same load balance anomaly also appears if exactly 6 copy links are defined; the data transfer load eventually shifts to using only 3 of the 6 links, transferring only metadata on the remaining 3 links.  Similar behavior can be seen if 9 or 12 remote copy links are defined.

If you have 2,4,5,7, 8, 10 or 11 links defined the load balancing works okay.

An improperly configured link can also cause this error condition. For example if you have two valid links defined and an invalid link (inoperable copy link), the total links defined to the node will be three, and the faulty logic will be invoked.  Please ensure that all copy links defined are operable using the 'Test Remote VLE' button in the Port Card Configuration tab of the VLE GUI.

Cause

Problem Identification:

There are at least 3 different methods for determining how many remote copy links are configured.

If you need to have the number of remote copy links on your VLE confirmed please open a service request to have this work done by an Oracle field person.

Method One:

The number of copy links defined to a VLE node can be obtained by looking at the GUI Port Card Configuration screen.

Method Two:

The same information can also be obtained by using the Coherence cache viewer via a putty session:

view_cache

Starting SimpleCacheExplorer with grid address: 172.17.1.1

Command: cache

VmvcMetadata (100 objects)
DeviceLocation (0 objects)
HardwareNodeInfo (1 objects)
VleProperty (1 objects)
DeviceInfo (0 objects)
VleRequest (0 objects)
VmvcLocation (100 objects)
coherence.common.sequencegenerators (2 objects)
UiLastUpdate (2 objects)
VleResponse (0 objects)
VtvDataServiceProxyLocation (1 objects)
ServiceInfo (7 objects)
VtvDataServiceLocation (1 objects)
RtdLocation (0 objects)
SoftwareNodeInfo (2 objects)
PortInfo (25 objects)

Command: cache VtvDataServiceProxyLocation

Command: keys

>> VLE4

Command: get VLE4

>> Value is VtvDataServiceProxyLocation: INDEX = 0 LAST = -1 PROXIES = { 172.17.1.1:62001//192.168.10.40 (true), 172.17.1.1:62002//192.168.11.41 (true), 172.17.1.1:62003//192.168.12.42 (true) }

Command: quit

NOTE: In the example above the number of links can be determined by counting the number of IP address pairs inside the curly brackets of the last display (in this case there are three).

Method Three:

The number of remote copy links can be determined by examining the output of the getVleInfo script as shown below:

vleadmin@vle4ga:~$ sudo getVleInfo
VLE Name: VLE4
vmvcLimit: 220
familyCode: 590
fruSerialNumber: 1042EN0002
gridAddress: 172.17.1.1
iffSecPortStart: 51000
iffSecPortEnd: 55000
listenOnNetName: 192.168.4.4 192.168.6.4
uuiListenOnNetName: 192.168.2.44
vtvDataServiceListenOnNetName: 192.168.8.45 192.168.9.46
isEncryptionEnabled: false


Remote VLEs configured on this node:
    id: 69, remoteVleName: VLE5, hardwareNodeName: vle4ga, remoteAddress: 192.168.9.56
    id: 70, remoteVleName: VLE3, hardwareNodeName: vle4ga, remoteAddress: 192.168.8.55


Nodes currently included in this VLE:
    name: vle4ga, gridAddress: 172.17.1.1, serialNumber: 1042EN0002

The number of remote copy links can be determined by examining the number of IP addresses provided on the “vtvDataServiceListenOnNetName:” line.  In this example there are two.

Solution

Workaround:

1. Do not configure the VLE with 3, 6, 9, or 12 copy links while running VLE 1.4.1 code.  Either remove a link or add another operable or dummy (false) link to avoid the problem configurations.
   
   
2. If you currently have a VLE with 3, 6, 9, or 12 copy links defined either change the configuration, or add a "dummy" copy link to the node changing the number of entries in the Coherence VtvDataServiceProxyLocation cache.  This will cause an ERROR in the vlelog when this dummy link is selected for the first time for a copy operation.  The link selection will time out after five minutes and the state of the link in the cache will be set to 'false'.  The copy operation will select a different link and continue. Once the link has been set to 'false' it will not be selected again.

If option 2 is selected the following messages will be found in the VLE log file:

When the dummy link is selected for the first time:
2014-04-11 18:16:05,881 [VtvDataServiceProxyReqHndlr-VLE4@192.168.22.52-pool-19-thread-17] ERROR messaging.interconnect.IcMessageFactory - Timeout (420000) while reading the socket

and

When the dummy remote link is polled every 60 seconds:
2014-04-11 18:28:15,116 [VtvDataServiceProxyMonitor-VLE4-@-192.168.22.52] WARN  vtvdataservice.proxy.VtvDataServiceProxyMonitor - VdsException while attempting to connect to remote VLE VLE4 at remote address 192.168.22.52 ( DataGridAddress: ADDRESS: 172.17.1.1 PORT: 62006 ): Failed to connect to local VDS proxy server: Connection timed out
2014-04-11 18:28:15,122 [VtvDataServiceProxyMonitor-VLE4-@-192.168.22.52] INFO  vtvdataservice.proxy.VtvDataServiceProxyMonitor - Still not-operational proxy for VLE VLE4 at remote address 192.168.22.52 ( DataGridAddress: ADDRESS: 172.17.1.1 PORT: 62006 ) will be tested again in 60 seconds

Option 1 is recommended to avoid additional error messages being recorded in the vle logs.  However, if adding or removing a copy link is not possible, no harm will result from selecting option 2, just a few ERROR messages and many more WARN messages will be posted in the log file.

Fix:

This problem is addressed in VLE maintenance release 1.4.2 which we anticipate releasing in May, 2014.

This solution has no attachment