Sun Storage 7000 Unified Storage System: If cluster is unconfigured - and is going to remain that way

Asset ID:	1-72-1577381.1
Update Date:	2018-03-20
Keywords:

Solution Type Problem Resolution Sure

Solution 1577381.1 : Sun Storage 7000 Unified Storage System: If cluster is unconfigured - and is going to remain that way - clustron cards should be removed

Applies to:

Sun ZFS Storage 7420 - Version All Versions and later
Sun ZFS Storage 7320 - Version All Versions and later
Sun Storage 7410 Unified Storage System - Version All Versions and later
Sun Storage 7310 Unified Storage System - Version All Versions and later
Oracle ZFS Storage ZS3-2 - Version All Versions and later
7000 Appliance OS (Fishworks)

Symptoms

The AKD (Appliance Kit Management Daemon) process dumps a corefile causing a short outage.

Cause

When looking at the corefile of AKD, we see that the clustron card drivers are causing this issue - even though cluster is unconfigured.

debugging core file of akd (32-bit) from opsdbnas01
initial argv: /usr/lib/ak/akd
threading model: native threads
status: process terminated by SIGABRT (Abort), pid=17440 uid=0 code=-1
panic message: ABORT: cluster I/O high priority thread timeslice expired <---- Cluster message

fa5a7448 libc_hwcap2.so.1`_lwp_kill+0x15(28, 6, fa5a7468, fee7a5ee)
fa5a7468 libc_hwcap2.so.1`raise+0x22(6, 0, fa5a74b8, fee51b5a)
fa5a74b8 libc_hwcap2.so.1`abort+0xf2(0, 0, 0, f01c810c, 524f4241, 63203a54)
fa5a78e8 libak.so.1`ak_vpanic+0x27c(0, f0171930, fa5a7914, f0118b2f)
fa5a7908 libak.so.1`ak_panic+0x27(f0171930, fcabb980, 0, f0075f83)
fa5a7928 libak.so.1`ak_cio_deadman+0x25(29, 0, fa5a7a00, 0, feea7d64, fb237340)
fa5a794c libc_hwcap2.so.1`__sighndlr+0x15(29, 0, fa5a7a00, f0075f74)
fa5a79bc libc_hwcap2.so.1`call_user_handler+0x2a2(29)
fa5a79ec libc_hwcap2.so.1`sigacthandler+0xf2(29, 0, fa5a7a00)
fa5a7c08 libc_hwcap2.so.1`set_parking_flag+0x14(fb237340, 0)
fa5a7c48 libc_hwcap2.so.1`cond_sleep_queue+0x11d(8ccbac4, 8ccbad4)
fa5a7c78 libc_hwcap2.so.1`cond_wait_queue+0x60(8ccbac4, 8ccbad4, fa5a7ce8, feec7866)
fa5a7cc8 libc_hwcap2.so.1`cond_wait_common+0x1eb(8ccbac4, 8ccbad4, fa5a7ce8, feec7dfc)
fa5a7cf8 libc_hwcap2.so.1`__cond_reltimedwait+0x58(8ccbac4, 8ccbad4, fa5a7da8, feec7e6f)
fa5a7d18 libc_hwcap2.so.1`cond_reltimedwait+0x27(8ccbac4, 8ccbad4, fa5a7da8, feec7eb4)
fa5a7d38 libc_hwcap2.so.1`pthread_cond_reltimedwait_np+0x24(8ccbac4, 8ccbad4, fa5a7da8, f01184d8)
fa5a7d78 libak.so.1`ak_cond_wait+0x8e(8120c08, 8ccbac4, 8ccbad4, fa5a7da8)
fa5a7dc8 libak.so.1`ak_mq_wait+0x7f(8ccbaa8, fa5a7df8, fa5a7df8, ffffffff)
fa5a7f98 libak.so.1`ak_cio_link_tx+0x328(92bf6c8, 0, 0, f0135e21) <---- Cluster driver message
fa5a7fc8 libak.so.1`ak_thread_start+0x7b(94a3248, fef61000, fa5a7fe8, feecdcee)
fa5a7fe8 libc_hwcap2.so.1`_thrp_setup+0x9b(fb237340)
fa5a7ff8 libc_hwcap2.so.1`_lwp_start(fb237340, 0, 0, 0, 0, 0)

Solution

If the two nodes in a cluster are to be operating as single machines, the clustron cards should be removed physically from both nodes.

References

<NOTE:1019887.1> - Sun Storage 7000 Unified Storage System: How to Collect a Support Bundle using the BUI or CLI
<BUG:16852889> - AKD CORES DUE TO 'CLUSTER I/O HIGH PRIORITY THREAD TIMESLICE EXPIRED '
<NOTE:1392492.1> - Oracle ZFS Storage Appliance: Performance Issue when Pool is almost Full

Attachments

This solution has no attachment