![]() | Oracle System Handbook - ISO 7.0 May 2018 Internal/Partner Edition | ||
|
|
![]() |
||||||||||||||||||||||||||||||||||||||||||||
Solution Type Technical Instruction Sure Solution 1333828.1 : How to identify SCSI JBOD arrays commonly used as boot device for disk-less Starcat (12K/15K/E20K/E25K) and Serengeti (3800/48x0/6800/4900/6900) platforms
In this Document
Created from <SR 3-3925000581> Applies to:Sun Fire 3800 Server - Version Not Applicable and laterSun Fire E20K Server - Version Not Applicable and later Sun Fire 6800 Server - Version Not Applicable and later Sun Fire E6900 Server - Version Not Applicable and later Sun Fire 4810 Server - Version Not Applicable and later Information in this document applies to any platform. GoalThe following High-End platforms do not have internal disks that can be used as boot device, so usually this function is accomplished by an attached SCSI JBOD (Just a Bunch Of Disks) array:
SolutionTo discuss this information further with Oracle experts and industry peers, we encourage you to review, join or start a discussion in an appropriate My Oracle Support Community - Oracle Sun Technologies Community - SPARC Legacy Servers.
Most of the simple JBOD arrays commonly used as SCSI boot device are equipped with a SAF-TE chip (SCSI Accessed Fault-tolerant Enclosure protocol chip) used to monitor some chassis environmental parameters; Solaris will manage SAF-TE devices with ses driver (SCSI Enclosure Services Driver). SAF-TE is accessed via the SCSI data bus and inquired by Solaris with luxadm command, that will tell also the specific array type. Also the "cfgadm -alv" command ( <explorer>/sysconfig/cfgadm-alv.out ) will show the enclosure type.
Below are some examples of the SAF-TE equipped devices. Sun StorEdge D1000 (also applicable to Sun StorEdge D2 array)D1000 has a maximum of twelve disks on two different SCSI buses that can be accesses separately (split bus configuration) or simultaneously from the same scsi initiator (full bus configuration: this requires External SCSI Jumper Cable); each bus has its SAF-TE chip. 5. c8t1d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107>
/sbus@2,0/QLGC,isp@2,10000/sd@1,0 6. c8t2d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107> /sbus@2,0/QLGC,isp@2,10000/sd@2,0 7. c8t3d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107> /sbus@2,0/QLGC,isp@2,10000/sd@3,0 8. c8t4d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107> /sbus@2,0/QLGC,isp@2,10000/sd@4,0 9. c8t5d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107> /sbus@2,0/QLGC,isp@2,10000/sd@5,0 10. c8t8d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107> /sbus@2,0/QLGC,isp@2,10000/sd@8,0 11. c8t9d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107> /sbus@2,0/QLGC,isp@2,10000/sd@9,0 12. c8t10d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107> /sbus@2,0/QLGC,isp@2,10000/sd@a,0 13. c8t11d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107> /sbus@2,0/QLGC,isp@2,10000/sd@b,0 14. c8t12d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107> /sbus@2,0/QLGC,isp@2,10000/sd@c,0 15. c8t13d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107> /sbus@2,0/QLGC,isp@2,10000/sd@d,0 As you can see from the above output, all disks are accessed by the same SCSI path (full bus), were both SAF-TE devices will be shown by luxadm command: # luxadm inquiry /dev/es/ses0
INQUIRY: Physical Path: /devices/sbus@2,0/QLGC,isp@2,10000/ses@e,0:0 Vendor: SYMBIOS Product: D1000 <--- array model Revision: 2 Serial Number Unsupported Device type: 0x3 (Processor device) Removable media: no ISO version: 0 ECMA version: 0 ANSI version: 2 (Device complies to ANSI X3.131-1994 (SCSI-2)) Response data format: 2 Additional length: 0x31 VENDOR-SPECIFIC PARAMETERS Byte# Hex Value ASCII 36 4e 29 87 20 20 20 20 20 53 41 46 2d 54 45 31 2e N). SAF-TE1. 30 30 00 # luxadm inquiry /dev/es/ses1 INQUIRY: Physical Path: /devices/sbus@2,0/QLGC,isp@2,10000/ses@f,0:0 Vendor: SYMBIOS Product: D1000 Revision: 2 Serial Number Unsupported Device type: 0x3 (Processor device) Removable media: no ISO version: 0 ECMA version: 0 ANSI version: 2 (Device complies to ANSI X3.131-1994 (SCSI-2)) Response data format: 2 Additional length: 0x31 VENDOR-SPECIFIC PARAMETERS Byte# Hex Value ASCII 36 4e 29 87 20 20 20 20 20 53 41 46 2d 54 45 31 2e N). SAF-TE1. 30 30 00 References: D1000 Service and Administration Guide: http://docs.oracle.com/cd/E19696-01/805-2624-12/805-2624-12.pdf
Sun StorEdge 3120Sun StorEdge 3120 has a maximum of four disks on two different SCSI buses that can be accesses separately (split bus configuration) or simultaneously from the same scsi initiator (single or full bus configuration: this requires Single Bus SCSI Jumper cable); there is only one SAF-TE chip however. AVAILABLE DISK SELECTIONS:
0. c0t12d0 <drive type unknown> <--- failed disk drive /pci@1c,700000/pci@1/pci@1/scsi@2/sd@c,0 1. c0t13d0 <SUN146G cyl 14087 alt 2 hd 24 sec 848> /pci@1c,700000/pci@1/pci@1/scsi@2/sd@d,0 2. c0t14d0 <SUN146G cyl 14087 alt 2 hd 24 sec 848> /pci@1c,700000/pci@1/pci@1/scsi@2/sd@e,0 3. c0t15d0 <SUN146G cyl 14087 alt 2 hd 24 sec 848> /pci@1c,700000/pci@1/pci@1/scsi@2/sd@f,0 4. c1t8d0 <SUN146G cyl 14087 alt 2 hd 24 sec 848> /pci@3c,700000/pci@1/pci@1/scsi@2/sd@8,0 5. c1t9d0 <SUN146G cyl 14087 alt 2 hd 24 sec 848> /pci@3c,700000/pci@1/pci@1/scsi@2/sd@9,0 6. c1t10d0 <SUN146G cyl 14087 alt 2 hd 24 sec 848> /pci@3c,700000/pci@1/pci@1/scsi@2/sd@a,0 7. c1t11d0 <SUN146G cyl 14087 alt 2 hd 24 sec 848> /pci@3c,700000/pci@1/pci@1/scsi@2/sd@b,0 As you can see from the above output, two 3120 arrays are connected to different SCSI paths and also have different SCSI IDs assigned to disks (this is not strictly required in such a scenario as there would be no conflict even with same disks IDs, as array are accessed by different SCSI buses). # luxadm inquiry /dev/es/ses0
INQUIRY: Physical Path: /devices/pci@1c,700000/pci@1/pci@1/scsi@2/ses@6,0:0 Vendor: SUN Product: StorEdge 3120 D <--- array model Revision: 1180 Serial Number 0A83BB Device type: 0x3 (Processor device) Removable media: no ISO version: 0 ECMA version: 0 ANSI version: 2 (Device complies to ANSI X3.131-1994 (SCSI-2)) Response data format: 2 Additional length: 0x5f VENDOR-SPECIFIC PARAMETERS Byte# Hex Value ASCII 52 30 30 20 46 00 F 96 06 00 00 00 .... # luxadm inquiry /dev/es/ses1 INQUIRY: Physical Path: /devices/pci@3c,700000/pci@1/pci@1/scsi@2/ses@5,0:0 Vendor: SUN Product: StorEdge 3120 D Revision: 1180 Serial Number 0A8163 Device type: 0x3 (Processor device) Removable media: no ISO version: 0 ECMA version: 0 ANSI version: 2 (Device complies to ANSI X3.131-1994 (SCSI-2)) Response data format: 2 Additional length: 0x5f VENDOR-SPECIFIC PARAMETERS Byte# Hex Value ASCII 52 30 30 20 46 00 F 96 05 00 00 00 .... References:
Sun StorEdge D240 Media TraySun StorEdge D240 Media Tray has two different SCSI buses that can be accesses separately (split bus configuration) or simultaneously from the same SCSI initiator (single or full bus configuration: this requires a Short SCSI I/O cable to be connected to one "in" and one "out" port on opposite sides on the rear of the chassis, as junction for the two SCSI buses). Each SCSI bus has its own SAF-TE chip. Each SCSI bus can host up to two devices, one located in a central bay of the D240 the chassis ("upper drive" or "lower drive"), one located into a side bay ("left drive" or "right drive"). When in split bus configuration, upper central drive shares the same bus with the right device (looking at the front of the array). By means of a configuration switch on the back of the chassis, SCSI target IDs for the installed devices may be assigned as follows:
While central bays may host hard disk drives only, D240 side bays may host Tape drives or CD-ROM drives also; this sometimes makes the array tricky to be detected by looking at Solaris format output only; in such a scenario, cfgadm command will give better view of the devices installed in the D240 chassis; please check examples below for details.
In the first example below, two different D240 arrays are connected to the Solaris host; both arrays have only hard disk drives installed and are configured in full SCSI bus: 0. c0t0d0 <SUN72G cyl 14087 alt 2 hd 24 sec 424>
/ssm@0,0/pci@18,700000/pci@1/scsi@2/sd@0,0 1. c0t1d0 <SUN72G cyl 14087 alt 2 hd 24 sec 424> /ssm@0,0/pci@18,700000/pci@1/scsi@2/sd@1,0 2. c0t4d0 <SUN72G cyl 14087 alt 2 hd 24 sec 424> /ssm@0,0/pci@18,700000/pci@1/scsi@2/sd@4,0 3. c0t6d0 <SUN72G cyl 14087 alt 2 hd 24 sec 424> /ssm@0,0/pci@18,700000/pci@1/scsi@2/sd@6,0 4. c1t0d0 <SUN72G cyl 14087 alt 2 hd 24 sec 424> /ssm@0,0/pci@1c,700000/pci@1/scsi@2/sd@0,0 5. c1t1d0 <SUN72G cyl 14087 alt 2 hd 24 sec 424> /ssm@0,0/pci@1c,700000/pci@1/scsi@2/sd@1,0 6. c1t4d0 <SUN72G cyl 14087 alt 2 hd 24 sec 424> /ssm@0,0/pci@1c,700000/pci@1/scsi@2/sd@4,0 7. c1t6d0 <SUN72G cyl 14087 alt 2 hd 24 sec 424> /ssm@0,0/pci@1c,700000/pci@1/scsi@2/sd@6,0 SAF-TE monitoring chips can be inquired by luxadm command, and this will help in revealing the array type and bus configuration: # luxadm inquiry /dev/es/ses0
INQUIRY: Physical Path: /devices/ssm@0,0/pci@18,700000/pci@1/scsi@2/ses@2,0:0 Vendor: Sun Product: D240 <--- array model Revision: 1.09 Serial Number Unsupported Device type: 0x3 (Processor device) Removable media: no ISO version: 0 ECMA version: 0 ANSI version: 2 (Device complies to ANSI X3.131-1994 (SCSI-2)) Response data format: 2 Additional length: 0x31 VENDOR-SPECIFIC PARAMETERS Byte# Hex Value ASCII 36 30 5a 30 30 55 6e 73 75 70 70 6f 72 74 65 64 00 0Z00Unsupported. 30 30 00 # luxadm inquiry /dev/es/ses1 INQUIRY: Physical Path: /devices/ssm@0,0/pci@18,700000/pci@1/scsi@2/ses@3,0:0 <--- as both SAF-TE are accessed on the same path, this is a full bus configuration Vendor: Sun Product: D240 Revision: 1.09 Serial Number Unsupported Device type: 0x3 (Processor device) Removable media: no ISO version: 0 ECMA version: 0 ANSI version: 2 (Device complies to ANSI X3.131-1994 (SCSI-2)) Response data format: 2 Additional length: 0x31 VENDOR-SPECIFIC PARAMETERS Byte# Hex Value ASCII 36 30 5a 30 30 55 6e 73 75 70 70 6f 72 74 65 64 00 0Z00Unsupported. 30 30 00 # luxadm inquiry /dev/es/ses2 INQUIRY: Physical Path: /devices/ssm@0,0/pci@1c,700000/pci@1/scsi@2/ses@2,0:0 Vendor: Sun Product: D240 Revision: 1.09 Serial Number Unsupported Device type: 0x3 (Processor device) Removable media: no ISO version: 0 ECMA version: 0 ANSI version: 2 (Device complies to ANSI X3.131-1994 (SCSI-2)) Response data format: 2 Additional length: 0x31 VENDOR-SPECIFIC PARAMETERS Byte# Hex Value ASCII 36 30 5a 30 30 55 6e 73 75 70 70 6f 72 74 65 64 00 0Z00Unsupported. 30 30 00 # luxadm inquiry /dev/es/ses3 INQUIRY: Physical Path: /devices/ssm@0,0/pci@1c,700000/pci@1/scsi@2/ses@3,0:0 Vendor: Sun Product: D240 Revision: 1.09 Serial Number Unsupported Device type: 0x3 (Processor device) Removable media: no ISO version: 0 ECMA version: 0 ANSI version: 2 (Device complies to ANSI X3.131-1994 (SCSI-2)) Response data format: 2 Additional length: 0x31 VENDOR-SPECIFIC PARAMETERS Byte# Hex Value ASCII 36 30 5a 30 30 55 6e 73 75 70 70 6f 72 74 65 64 00 0Z00Unsupported. 30 30 00
In the second example below, a single D240 array is connected to the Solaris host; array is configured in split SCSI bus and has hard disk drives installed into central chassis bays only: 0. c0t0d0 <SUN72G cyl 14087 alt 2 hd 24 sec 424> bootdisk
/pci@17d,700000/pci@1/scsi@2/sd@0,0 1. c3t0d0 <SUN72G cyl 14087 alt 2 hd 24 sec 424> /pci@19d,700000/pci@1/scsi@2/sd@0,0 Here, both disk drives has SCSI target IDs assigned to 0 because of the split bus config; a single SAF-TE per SCSI path will confirm this: # luxadm inquiry /dev/es/ses0
INQUIRY: Physical Path: /devices/pci@17d,700000/pci@1/scsi@2/ses@2,0:0 Vendor: Sun Product: D240 Revision: 1.09 Serial Number Unsupported Device type: 0x3 (Processor device) Removable media: no ISO version: 0 ECMA version: 0 ANSI version: 2 (Device complies to ANSI X3.131-1994 (SCSI-2)) Response data format: 2 Additional length: 0x31 VENDOR-SPECIFIC PARAMETERS Byte# Hex Value ASCII 36 30 30 32 36 32 38 39 00 53 41 46 2d 54 45 31 2e 0026289.SAF-TE1. 30 30 00 # luxadm inquiry /dev/es/ses1 INQUIRY: Physical Path: /devices/pci@19d,700000/pci@1/scsi@2/ses@3,0:0 Vendor: Sun Product: D240 Revision: 1.09 Serial Number Unsupported Device type: 0x3 (Processor device) Removable media: no ISO version: 0 ECMA version: 0 ANSI version: 2 (Device complies to ANSI X3.131-1994 (SCSI-2)) Response data format: 2 Additional length: 0x31 VENDOR-SPECIFIC PARAMETERS Byte# Hex Value ASCII 36 30 30 32 36 32 37 33 01 53 41 46 2d 54 45 31 2e 0026273.SAF-TE1. 30 30 00
In the third example below, cfgadm command output shows all devices installed into a D240 array configured in full bus (here, disk drives are installed in central bays only): hostname:$ grep "^c1" cfgadm-alv.out
cfgadm-alv.out:c1 connected configured unknown cfgadm-alv.out:c1::dsk/c1t0d0 connected configured unknown FUJITSU MAP3367N SUN36G cfgadm-alv.out:c1::dsk/c1t1d0 connected configured unknown SEAGATE ST336605LSUN36G cfgadm-alv.out:c1::dsk/c1t6d0 connected configured unknown TOSHIBA DVD-ROM SD-M1401 cfgadm-alv.out:c1::es/ses7 connected configured unknown Sun D240 cfgadm-alv.out:c1::es/ses8 connected configured unknown Sun D240 cfgadm-alv.out:c1::rmt/0 connected configured unknown HP C5683A References:
Below are some examples of JBOD devices that are instead not equipped with the SAF-TE chip. Sun StorEdge S1Sun StorEdge S1 JBOD array is equipped with a single SCSI bus that host up to three disk drives, and has no SAF-TE device available, so luxadm command does not report any output as ses driver is not used. S1 may be recognized as disk drives SCSI target IDs are usually 0, 1 and 2. 0. c0t0d0 <SUN72G cyl 14087 alt 2 hd 24 sec 424>
/pci@11c,600000/pci@1/scsi@2/sd@0,0 1. c0t1d0 <SUN72G cyl 14087 alt 2 hd 24 sec 424> /pci@11c,600000/pci@1/scsi@2/sd@1,0 2. c0t2d0 <SUN72G cyl 14087 alt 2 hd 24 sec 424> /pci@11c,600000/pci@1/scsi@2/sd@2,0 3. c1t0d0 <SUN72G cyl 14087 alt 2 hd 24 sec 424> /pci@11d,600000/pci@1/scsi@2/sd@0,0 4. c1t1d0 <SUN72G cyl 14087 alt 2 hd 24 sec 424> /pci@11d,600000/pci@1/scsi@2/sd@1,0 5. c1t2d0 <SUN72G cyl 14087 alt 2 hd 24 sec 424> /pci@11d,600000/pci@1/scsi@2/sd@2,0 References:
Internal Section References:
JBOD disk troubleshooting:
References<NOTE:1449322.2> - Information Center: Sun Storage S1 Array<NOTE:1012070.1> - Sun StorEdge[TM] D1000:InfoDoc:How to tell if you have a D1000 attached <NOTE:1449240.2> - Information Center: Sun Storage D1000 Array <NOTE:1018089.1> - Sun StorEdge[TM] D1000 array cabling and address <NOTE:1449353.2> - Information Center: Sun Storage 3120 SCSI Array <NOTE:1006857.1> - Verifying and Troubleshooting Sun Storage 3120, 3310 and 3320 Cabling <NOTE:1531979.1> - How To Remove and Replace a Sun Storage 3120 SCSI Array HDD <NOTE:1449265.2> - Information Center: Sun Storage D240 (StorEdge) Media Tray <NOTE:1499481.1> - How To Replace A Failed Disk in a D240 Media Tray:ATR:1499481.1:3 <NOTE:1002790.1> - Sun Storage 3000 Arrays: Troubleshooting JBOD Failures <NOTE:1010946.1> - General Guidance for Diagnosis (Disk Failures/Errors) and Replacing Internal Server Disks and JBOD Disks within Solaris Attachments This solution has no attachment |
||||||||||||||||||||||||||||||||||||||||||||
|