Pillar Axiom: How to Resolve a Heartbeat Link or Connection Dead Issue

Asset ID:	1-71-1578182.1
Update Date:	2017-09-08
Keywords:

Solution Type Technical Instruction Sure

Solution 1578182.1 : Pillar Axiom: How to Resolve a Heartbeat Link or Connection Dead Issue

Applies to:

Pillar Axiom 600 Storage System - Version Not Applicable to Not Applicable [Release N/A]
Pillar Axiom 300 Storage System - Version Not Applicable to Not Applicable [Release N/A]
Pillar Axiom 500 Storage System - Version Not Applicable to Not Applicable [Release N/A]
Information in this document applies to any platform.

Goal

How to recover from a HeartbeatNetworkConnectionDead, HeartbeatSerialLinkDead, PCP_EVT_PILOT_NETWORK_CONNECTION_DEAD or PCP_EVT_PILOT_SERIAL_LINK_DEAD event.

Solution

Purpose

There are many reasons for these error messages to be generated. Some are benign and require no further action. This document explains these reasons so as to assist the reader in deciding if further action is needed or not.

If you still have questions after reading this article, go to the My Oracle Support Community - Pillar Axiom Storage System

Scope

In an Axiom system, there are two redundant Pilots that act as a cluster. The events listed above can occur whenever there is a communication issue between them. This will not affect data access as the Pilots only perform configuration and monitor functions and have no impact on the data paths.

Details

These events are typically seen if one of the Pilots reboots. In turn this creates a communication problem between it and the surviving Pilot. If you receive the events above, please check the following steps.

Note: Before proceeding with the following maintenance steps, if you have ASR (Auto Service Request) enabled please refer to Document 1535352.1 Pillar Axiom: How to Disable Call Home to Prevent Automatic Service Request ASR Generation in order to disable call-home and prevent the generation of spurious Service Requests. Be sure to re-enable call-home after any maintenance activities are completed.

.

From the Pilot:

Was any maintenance activity being done at the time the event was generated?
Check power to both Pilots. Power cycle the Pilot with the issue.
Check Pilot status from within the GUI.
Check the serial and ethernet cable connections between the two Pilots and make sure both are secured correctly. Disconnect and reconnect them to ensure a firm connection to the serial and network ports.
On some Pilot models there is a power button behind the front bezel. Remove the bezel and press the power button on the front of the Pilot.
Connect a USB keyboard and VGA monitor to the Pilot in question, and check the condition of the console (you may have to hit the return key to get a display).
If the problem cannot be resolved by the steps above it might mean that the Pilot needs to be replaced. Support will check the event logs and take appropriate action to resolve the issue.

Additional Information:

While working on the Axiom Pilot any disconnections of the serial link or Ethernet cable, this will generate the events above.
Disable the Call-Home setting whenever any maintenance activity is attempted so it will not generate call-home events.
Pilot related events are occurring often in connection with power outlet issues and cabling issue.

Note: The Call-Homes are generated only when a fault occurs, not when the fault is resolved.

Attachments

This solution has no attachment