Pillar Axiom: Pilot failure due to pilot internal drive fault

Asset ID:	1-72-1902520.1
Update Date:	2018-01-08
Keywords:

Solution Type Problem Resolution Sure

Solution 1902520.1 : Pillar Axiom: Pilot failure due to pilot internal drive fault

Applies to:

Pillar Axiom 600 Storage System - Version All Versions and later
Pillar Axiom 500 Storage System - Version All Versions and later
Information in this document applies to any platform.

Symptoms

(Firmware release 5.x) Pilot error ASR event containing

event_code: PCP_EVT_PILOT_SERIAL_LINK_DEAD

AND IMPORTANTLY, in the event logset named 'PCP_EVT_PILOT_SERIAL_LINK_DEAD' , the log/pilot*/messages* files may contain errors similar to :-

Jun 29 03:36:41.392 pilot1 kernel: ata2.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen
Jun 29 03:36:41.392 pilot1 kernel: ata2.00: failed command: WRITE DMA EXT
Jun 29 03:36:41.392 pilot1 kernel: ata2.00: cmd 35/00:08:42:9c:5f/00:00:35:00:00/e0 tag 0 dma 4096 out
Jun 29 03:36:41.392 pilot1 kernel: res 40/00:01:00:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
Jun 29 03:36:41.392 pilot1 kernel: ata2.00: status: { DRDY }
Jun 29 03:36:51.435 pilot1 kernel: ata2.00: SRST failed (errno=-16)
Jun 29 03:37:01.478 pilot1 kernel: ata2.00: SRST failed (errno=-16)
Jun 29 03:37:36.511 pilot1 kernel: ata2.00: SRST failed (errno=-16)
Jun 29 03:37:41.556 pilot1 kernel: ata2.00: SRST failed (errno=-16)
Jun 29 03:37:41.556 pilot1 kernel: ata2.00: reset failed, giving up
Jun 29 03:37:41.556 pilot1 kernel: end_request: I/O error, dev sda, sector 895458370
Jun 29 03:37:41.556 pilot1 kernel: Buffer I/O error on device sda10, logical block 105006273
Jun 29 03:37:41.556 pilot1 kernel: end_request: I/O error, dev sda, sector 895579706
Jun 29 03:37:41.556 pilot1 kernel: Buffer I/O error on device sda10, logical block 105021440
Jun 29 03:37:41.556 pilot1 kernel: end_request: I/O error, dev sda, sector 895607962

Jun 22 04:17:48.569 pilot1 kernel: end_request: I/O error, dev sda, sector 47258796
Jun 22 04:17:48.569 pilot1 kernel: EXT3-fs error (device sda10) in ext3_orphan_add: Readonly filesystem
Jun 22 04:17:48.569 pilot1 kernel: Aborting journal on device sda9.
Jun 22 04:17:48.569 pilot1 kernel: EXT3-fs error (device sda9) in ext3_reserve_inode_write: Journal has aborted
Jun 22 04:17:48.569 pilot1 kernel: end_request: I/O error, dev sda, sector 47221124

-----

In less frequent observations, the following line might also be found in the messages* files.

smartd[2751]: Device: /dev/sda, FAILED SMART self-check. BACK UP DATA NOW!

Cause

pilot internal hard disk errors

Solution

replace affected pilot

References

<NOTE:1539019.1> - Pillar Axiom: How to replace a Pilot Control Unit:ATR:1539019.1:2

Attachments

This solution has no attachment