SAM-QFS - T10000C 3.62-108 Firmware Using Emulex HBAs Breaks SAM-FS With I/O Errors

Asset ID:	1-72-1954293.1
Update Date:	2017-02-28
Keywords:

Solution Type Problem Resolution Sure

Solution 1954293.1 : SAM-QFS - T10000C 3.62-108 Firmware Using Emulex HBAs Breaks SAM-FS With I/O Errors - Bad Media

Applies to:

StorageTek T10000C Tape Drive - Version All Versions to All Versions [Release All Releases]
Sun QFS and Storage Archive Manager (SAM) - Version 5.0 to 5.4 [Release 5.0]
Information in this document applies to any platform.

Symptoms

The sam-log file reporting the following type of errors:

Dec 10 23:13:57 qbi-mdc genu-4000[2247]: [ID 702911 local7.warning] Device 4002, VSN QBI275: Bad media. Check device log.
Dec 10 23:13:57 qbi-mdc sam-arcopy[2255]: [ID 702911 local7.error] err Fatal OS call error: ioctl:F_UNLOAD(eq: 4002 ti.QBI275) called from: rmarchive.c:1043: I/O error

Devlog messages, found in /var/opt/SUNWsamfs/devlog, show "driver command timeout" error while writing file mark. Tape is flagged as bad.

2014/12/10 22:01:53 3247 [2352:16986] tape2.c:2360 detail Space: Drive-reported space(5386268160K)
2014/12/10 22:01:53 1059 [2352:16986] dir_io.c:320 detail Ready for data transfer
2014/12/10 22:02:01 1066 [2352:16986] dir_io.c:477 time Wrote 1266679808 bytes, time 8 seconds
2014/12/10 22:02:01 3205 [2352:16986] tape.c:149 resource ptr: 0xfe11b5d8, process_eox: 0, process_wtm: 0, wrote_tm: 0
2014/12/10 22:02:01 3220 [2352:16986] tape.c:158 detail Creating EOF/EOV label
2014/12/10 22:02:01 3001 [2352:16986] tape.c:220 Error: mt_erreg = 0x11 [Driver Command Timeout]
2014/12/10 22:02:01 3237 [2352:16986] tape.c:315 detail Write TM at 0x260
2014/12/10 22:03:10 3229 [2352:16986] tape.c:1675 detail Check EOT [key 0 eom 0]
2014/12/10 22:03:10 3202 [2352:16986] tape.c:325 Error: Write of file mark fails - error 5, cleaning 0
2014/12/10 22:03:10 0000 [2352:16986] dev_log.c:331 all sense - 00 00 00 00 00 00 00 00 00 00
2014/12/10 22:03:10 0000 [2352:16986] dev_log.c:333 all 00 00 00 00 00 00 00 00 00 00
2014/12/10 22:03:10 3193 [2352:16986] tape2.c:406 Error: Unable to determine position
2014/12/10 22:03:10 0000 [2352:16986] dev_log.c:304 all cdb - 34 00 00 00 00 00 00 00
2014/12/10 22:03:10 0000 [2352:16986] dev_log.c:306 all 00 00 00 00 00 00 00 00
2014/12/10 22:03:10 0000 [2352:16986] dev_log.c:331 all sense - 00 00 00 00 00 00 00 00 00 00
2014/12/10 22:03:10 0000 [2352:16986] dev_log.c:333 all 00 00 00 00 00 00 00 00 00 00
2014/12/10 22:03:10 3109 [2352:16986] tape2.c:2323 Error: Unable to determine capacity
2014/12/10 22:03:10 3193 [2352:16986] tape2.c:406 Error: Unable to determine position
2014/12/10 22:03:10 3109 [2352:16986] tape2.c:2323 Error: Unable to determine capacity
2014/12/10 22:03:10 3193 [2352:16986] tape2.c:406 Error: Unable to determine position
2014/12/10 22:03:10 0000 [2352:16986] dev_log.c:304 all cdb - 34 00 00 00 00 00 00 00
2014/12/10 22:03:10 0000 [2352:16986] dev_log.c:306 all 00 00 00 00 00 00 00 00
2014/12/10 22:03:10 0000 [2352:16986] dev_log.c:331 all sense - 00 00 00 00 00 00 00 00 00 00
2014/12/10 22:03:10 0000 [2352:16986] dev_log.c:333 all 00 00 00 00 00 00 00 00 00 00
2014/12/10 22:03:10 3203 [2352:16986] tape.c:351 detail Check for End of Tape : eot 0x1 read_pos err 0xffffffff
2014/12/10 22:03:10 3230 [2352:16986] tape.c:361 detail Copy in for EOV processing [eot = 1]
2014/12/10 22:03:10 3049 [2352:16986] tape.c:555 Error: Unable to create EOF at 0
2014/12/10 22:03:10 3044 [2352:16986] misc.c:721 Error: Bad medium
2014/12/10 22:03:10 1067 [2352:16986] dir_io.c:501 Io_count: diff -4, count 604
2014/12/10 22:03:10 1068 [2352:16986] dir_io.c:504 Error: Io_count from 0x4 to 0 final 0x25c
2014/12/10 22:03:10 10046 [2352:16986] dir_io.c:529 all Error 5 on write to media, catalog not updated.

During read of the tape at the time of loading or re-labeling, positioning errors like these could be observed:

2014/12/10 21:51:24 1047 [2352:16974] mount.c:114 detail Load VSN QBI227
2014/12/10 21:51:24 1050 [2352:16974] mount.c:147 detail Open /dev/rmt/15cbn as 16, open count = 1
2014/12/10 21:51:24 0000 [2352:16974] mount.c:160 all resource handles DIV I/O
2014/12/10 21:51:24 3233 [2352:16974] tape2.c:131 detail Position timeout set to 1000 secs for 532
2014/12/10 21:54:28 3066 [2352:16974] tape2.c:151 time Positioning time: 184 seconds
2014/12/10 21:54:28 3067 [2352:16974] tape2.c:158 Error: Positioning error to 0x33bbc8
2014/12/10 21:54:28 0000 [2352:16974] dev_log.c:304 all cdb - 2b 00 00 00 33 bb c8 00
2014/12/10 21:54:28 0000 [2352:16974] dev_log.c:306 all 00 00 00 00 00 00 00 00
2014/12/10 21:54:28 0000 [2352:16974] dev_log.c:331 all sense - 00 00 00 00 00 00 00 00 00 00
2014/12/10 21:54:28 0000 [2352:16974] dev_log.c:333 all 00 00 00 00 00 00 00 00 00 00
2014/12/10 21:54:28 3063 [2352:16974] tape.c:1002 detail append_tape: Unable to position to 0x33bbc8
2014/12/10 21:54:28 3070 [2352:16974] tape.c:1011 detail Find EOD - skip to EOD
2014/12/10 21:55:07 3233 [2352:16974] tape2.c:131 detail Position timeout set to 1000 secs for 532
2014/12/10 21:57:24 3066 [2352:16974] tape2.c:151 time Positioning time: 137 seconds
2014/12/10 21:57:24 3067 [2352:16974] tape2.c:158 Error: Positioning error to 0x79e674fe
2014/12/10 21:57:24 0000 [2352:16974] dev_log.c:304 all cdb - 2b 00 00 79 e6 74 fe 00
2014/12/10 21:57:24 0000 [2352:16974] dev_log.c:306 all 00 00 00 00 00 00 00 00
2014/12/10 21:57:24 0000 [2352:16974] dev_log.c:331 all sense - 00 00 00 00 00 00 00 00 00 00
2014/12/10 21:57:24 0000 [2352:16974] dev_log.c:333 all 00 00 00 00 00 00 00 00 00 00
2014/12/10 21:57:24 3053 [2352:16974] tape.c:1780 Error: Unable to position to EOD
2014/12/10 21:57:24 3071 [2352:16974] tape.c:1017 time Skip to EOD time: 176 seconds
2014/12/10 21:57:28 3056 [2352:16974] tape.c:1074 detail EOD at position 0x8
2014/12/10 21:57:50 3223 [2352:16974] tape.c:1375 detail Back Space 1 records : erreg 8 resid 8
2014/12/10 21:57:50 3160 [2352:16974] tape.c:1083 Error: Unable to find EOF at position 0x8
2014/12/10 21:57:50 3044 [2352:16974] misc.c:721 Error: Bad medium
2014/12/10 21:57:50 1058 [2352:16974] dir_io.c:271 System error: Load error: No space left on device

Tape drive is T10000C ( T10KC ). HBA is Emulex.

Drive dump shows 2B5C FSC.

Changes

Upgraded T10000C firmware from version 1.61.301 to 3.62-108.

Cause

T10KC code 3.62-108 will not work with Emulex HBAs.
Problem has been reproduced in the lab.

Solution

This problem is fixed in the T10KC 3.62.109 code. This is available in MOS Patches & Updates as Patch ID 20524555 .

Recycle all the tape media that have been flagged as bad.

This was the old solution prior to the release of patch 20524555
-------------------------------------------------------------------------

As a workaround, you could back down the T10000C code to previous code version that was running ( like for example, 1.61.301 )

To acquire new firmware that addresses the problem, open a Service Request and ask for special build firmware RB362108, that can be provided by Oracle Tape Engineering.

Recycle all the tape media that have been flagged as bad.

Future released T10kC microcode will include a fix, made against bug 20200257.

References

<BUG:20200257> - UNIV OF QUEENSLAND - 2B5C SNO ERRORS
<NOTE:1954193.1> - T10000C - 2B5C SNO Running On 3.62.108 Code Under Emulex HBA's

Attachments

This solution has no attachment