Sun Storage 7000 Unified Storage System: Aksh Script Caused Appliance Kit Daemon to Restart

Asset ID:	1-72-1564183.1
Update Date:	2018-01-05
Keywords:

Solution Type Problem Resolution Sure

Solution 1564183.1 : Sun Storage 7000 Unified Storage System: Aksh Script Caused Appliance Kit Daemon to Restart

Applies to:

Sun ZFS Storage 7420 - Version All Versions and later
Sun ZFS Storage 7320 - Version All Versions and later
Sun ZFS Storage 7120 - Version All Versions and later
Sun Storage 7410 Unified Storage System - Version All Versions and later
Sun Storage 7110 Unified Storage System - Version All Versions and later
7000 Appliance OS (Fishworks)

Symptoms

To discuss this information further with Oracle experts and industry peers, we encourage you to review, join or start a discussion in the My Oracle Support Community - Disk Storage ZFS Storage Appliance

Aksh script caused appliance kit daemon (akd) to restart.

Cause

Appliance Kit Daemon (akd) dumped core (and restarted automatically) with segmentation fault when executing an aksh script.

Error Received:

aksh: uncaught internal exception: { akStack: { aks_content: [{ args:
[], caller: <function> }, { args: [], caller: <function> }] },
akWrapped: { stack: 'akshCall("utask.listTasks",[object
Array])@:0\n()@akService.js:50\n()@akInterpreter.js:1879\n@/usr/lib/ak/js/shell/akLocore.js:629\n',
lineNumber: 50, fileName: 'akService.js', message: 'couldn\'t make door
call: Bad file number' }, message: 'XML-RPC failed' }

Issue occured during running this block of code:
(the code had already performed "shares select default", "select dataset snapshots")

try {
run ("snapshot " + clone + "_clone");
run ("select " + clone + "_clone");
run ("clone " + proj + " " + dataset + "_clone_" + clone);
run ("set mountpoint=/export/" + dataset + "_clone_" + clone);
run ("commit");
printf ("INFO: Clone: " + clone + " created as dataset " + dataset + "_clone_" + clone);
} catch (err) {
printf ("ERROR: Problem creating clone");
}

Supportbundle showed akd dumped core and restarted sucessfully.

svcs/log/appliance-kit-akd:default.log
[ Mar 22 19:50:11 Stopping because process dumped core. ]
[ Mar 22 19:50:11 Executing stop method (:kill). ]
[ Mar 22 19:50:12 Executing start method ("exec /usr/lib/ak/akd"). ]
[ Mar 22 19:50:33 Method "start" exited with status 0. ]
[ Mar 22 19:50:45 Rereading configuration. ]
[ Mar 22 19:50:45 No 'refresh' method defined. Treating as :true. ]
[ Mar 22 19:50:45 Rereading configuration. ]

Peer akd reported "remote akd went away" and came back up.

Fri Mar 22 19:50:11 2013: CIO: remote akd went away
Fri Mar 22 19:50:11 2013: CIO: channel 0 reset by cio disable
Fri Mar 22 19:50:11 2013: CIO: channel 1 reset by cio disable
Fri Mar 22 19:50:11 2013: CIO: channel 0 closed (failed to send to 0): CIO connection reset by peer
Fri Mar 22 19:50:11 2013: CIO disabled: EAK_CIO_NOLISTENER
Fri Mar 22 19:50:11 2013: ALUA proxy communication with cluster peer interrupted by error: detected dropped or reset peer
Fri Mar 22 19:50:22 2013: CIO: remote akd now up

Akd core showed stack trace as per Bug Id 15777558

> ::status
debugging core file of akd (32-bit) from eis-zfs-w2
initial argv: /usr/lib/ak/akd
threading model: native threads
status: process terminated by SIGSEGV (Segmentation Fault), addr=703a6b61

> ::stack
libnvpair.so.1`nvlist_next_nvpair+0x4d(27f61988, 0, e841c304, 23cefb30)
nas.so`nas_list_common+0x69e(e841e930, 238b8058, 0, f0156ef4, fccd8630, e841e4a4)
nas.so`nas_list+0x19(e841e930, 238b8058, 656d2700, f0157011)
libak.so.1`akx_invoke+0x9a(e841e930, fccf1170, fccf1470, 238b8058, 0, 0)
libak.so.1`ak_singlecall+0xb4f(e841e930, cef0d98, 2, 0, 0, e841e6f0)
libak.so.1`ak_singlecall+0x792(e841e930, 22d70950, 1, 0, 0, e841e810)
libak.so.1`ak_singlecall+0x792(e841e930, 1d576378, 0, 0, 0, 0)
libak.so.1`ak_multicall+0xe6(e841e930, 17d51568, e841e8b8, f0157011)
libak.so.1`akx_invoke+0x9a(e841e930, f01c2310, f01c26b0, 17d51568, 0, 2)
libak.so.1`akx_call+0x7a(e841e930, e841f5d4, 82a, f015a89d)
libak.so.1`akx_rpc_svc+0xb2(8120c08, aeb0cac, e841f5d4, 82a, 25da4608, 0)
libak.so.1`ak_rpc_svc_door_process+0x33(8120c08, aeb0cac, 0, e841f5d4, 82a, 25da4608)
libak.so.1`ak_rpc_svc+0xea(8120c08, aeb0cac, e841f45c, 25da4608, 0, 0)
libak.so.1`ak_sidedoor+0x8e(aeb0c88, e841f45c, 9a2, 0, 0)
libak.so.1`ak_door_serve+0x77(aeb0c88, e841f45c, 9a2, 0, 0, f00a4c20)
libc_hwcap2.so.1`__door_return+0x4c()

Solution

This is a known issue - which has now been resolved.

Bug 15807681 SUNBT7151925-AK-2011.04.2 nas_list_common() stumbles on a stale nas cache entry

The fix is available in 2011.1.5 (and above) Appliance Firmware Release - please upgrade to the latest release.

References

<BUG:15807681> - SUNBT7151925-AK-2011.04.24 NAS_LIST_COMMON() STUMBLES ON A STALE NAS CACHE ENTRY
<BUG:15777558> - SUNBT7151925-AK-8 NAS_LIST_COMMON STUMBLES ON A STALE NAS CACHE ENTRY - AKD SE
<NOTE:1019887.1> - Sun Storage 7000 Unified Storage System: How to Collect a Support Bundle using the BUI or CLI

Attachments

This solution has no attachment