Oracle Key Manager (OKM) - SOAP Fault String = End of file or no input: Operation interrupted or timed out

Asset ID:	1-72-1612377.1
Update Date:	2016-02-24
Keywords:

Solution Type Problem Resolution Sure

Solution 1612377.1 : Oracle Key Manager (OKM) - SOAP Fault String = End of file or no input: Operation interrupted or timed out

Applies to:

Oracle Key Manager - Version 2.3 to 2.5.2 [Release 2.0]
Information in this document applies to any platform.

Symptoms

Multiple drives lose connection to KMA03. The other KMAs are not reporting the problem.

KMA03 is responding on both service and management networks but it is showing SOAP errors like this in the KMS audit log:
---

9CDE28809E5CE7AD0000000000011859 9CDE28809E5CE7AD KMA03 Communication and Authentication Medium Term Accept Agent Service Connection SOAP Error Warning 000079000025 2013-10-21 10:23:47 1861 xxx.xx.xx.xx SOAP Fault Code = SOAP-ENV:Client, SOAP Fault String = End of file or no input: Operation interrupted or timed out (60 s receive delay) (60 s send delay), SOAP Fault Detail = Unknown, Error Code = -1 Check the network connection on the KMA reporting the issue. If this condition persists, then the KMA might need to be rebooted.

---
SOAP Fault String = End of file or no input: Operation interrupted or timed out
---

Drives go through multiple network switches to the KMA

One KMA in the cluster is out of service. It is not responding on both management and service networks.
The KMA is not locked and not enrolled.

Changes

Took a KMA out of service for maintenance

Cause

Any of the following could cause this problem:

1. If there is a KMA in the cluster that is down but is still being contacted by the drive agents for keys, the drive agents could get timeout errors
while trying to retrieve keys from this KMA

2. The drives go through non-managed 3com switches that may be causing latency issues due to multiple hops needed before reaching the KMAs.

3. The SOAP errors can also be caused by a bad switch, cable or connector.

Solution

1 If there is a KMA that is not responding on both management and service networks, make sure the KMA is in lock mode so that drive agents do not request keys from this KMA.
If drive agents access this KMA to get keys, the drive agents could get SOAP timeout errors.

2. Request for a network diagram and look for possible routing issue with drives going through multiple switches to get to the KMA
Identify which KMA is losing connection to the drives and the switch the KMA is connected to. Try isolating the connection problem to the switch, the cable, link speed, etc.
Enable the KMA Support account, login as the "support" and check for interface errors, network statistics, etc.
Enable Wireshark and capture packet traces while the problem is occurring

3. Identify which drives are reporting the SOAP errors and if the drives have common identifiable properties such as code level, subnets, connected to same switch, etc.

Open a service request in MyOracle Support if you need assistance in troubleshooting the SOAP connection errors.

References

<NOTE:1159885.1> - How To Configure And Use WireShark?

Attachments

This solution has no attachment