Fujitsu M10 / M12 Introduction to HW Raid

Asset ID:	1-71-1623067.1
Update Date:	2017-04-13
Keywords:

Solution Type Technical Instruction Sure

Solution 1623067.1 : Fujitsu M10 / M12 Introduction to HW Raid

Applies to:

Fujitsu M10-1 - Version All Versions to All Versions [Release All Releases]
Fujitsu M10-4S - Version All Versions to All Versions [Release All Releases]
Fujitsu M10-4 - Version All Versions to All Versions [Release All Releases]
Fujitsu SPARC M12-2 - Version All Versions to All Versions [Release All Releases]
Fujitsu SPARC M12-2S - Version All Versions to All Versions [Release All Releases]
Information in this document applies to any platform.

Goal

The goal of this document is to show how HW Raid is used on the M10 / M12 servers

Solution

Introduction to HW RAID

HW Raid is covered in section 12.2 in the Systems System Operation and Administration Guide

OBP command set

show-children Lists all the connected logical drives and logical volumes.
show-volumes Lists the details of all the connected logical volumes.
create-raid0-volume Creates a RAID0 volume (Stripe no protection if a disk fails). Specify at least 2 targets.
create-raid1-volume Creates a RAID1 volume (Mirror). Specify 2 targets.
create-raid1e-volume Creates a RAID1E volume. Specify at least 3 targets.
delete-volume Deletes a RAID volume.
activate-volume Enables a RAID volume again after disk replacement

Introduction to the LSI SAS2IRCU utility

The same information and configuration can be done at the OS level if the RAID management utility, sas2ircu is installed. It is highly recommended that the utility be installed as it makes the management of HW Raid volumes much easier. The SAS2IRCU utility and detailed documentation about sas2ircu can be found on the LSI site:

LSI SAS2IRCU utility

Doc ID 1513610.1 for the T5 also goes over the OS commands.

explorer will collect the HW Raid information in the disk directory only if the sas2ircu utility is installed.

format will show a HW Raid volume as follows:

AVAILABLE DISK SELECTIONS:
0. c3t39B1A2D9A7F0E0F3d0 <LSI-Logical Volume-3000 cyl 9998 alt 2 hd 16 sec 128>
/pci@8000/pci@4/pci@0/pci@0/scsi@0/iport@v0/disk@w39b1a2d9a7f0e0f3,0

Note the LSI-Logical Volume

sas2ircu utility command set:

create creates an integrated RAID volume. The first disk identified becomes the primary.
delete deletes all RAID volumes and hot spare drives from a controller.
deletevolume deletes a specific RAID volume and associated hot spares on a controller
display displays information such as configuration, controller type, FW version, BIOS, etc...
hotspare adds or deletes a hot spare drive to the spare pool
list displays a list of all controllers and corresponding controller index in the system
status displays the current status of any existing RAID volumes or operations in progress
constchk requests FW to start a consistency check on the specified volume
activate activates an inactive RAID volume
locate turns on and flashes drive LED. (Does not work for drives directly attached to enclosure) The yellow locate light will be directly below the green light
logdir uploads or clears log information
bootir selects an existing RAID volume as the primary boot device
bootencl selects a specific enclosure/slot as the primary boot device
help displays usage for the specified command

The syntax for a SAS2IRCU command is as follows:

# sas2ircu <controller_#> <command> <parameters>

Creating a mirror from OBP

First run either probe-scsi-all or show-devs to obtain the path to the internal disk controller(s)

{0} ok probe-scsi-all
/pci@8000/pci@4/pci@0/pci@0/scsi@0

FCode Version 1.00.56, MPT Version 2.00, Firmware Version 13.00.66.00

Target a
Unit 0 Disk TOSHIBA MBF2600RC 3706 1172123568 Blocks, 600 GB
SASDeviceName 50000394f808cd88 SASAddress 50000394f808cd8a PhyNum 0
Target b
Unit 0 Disk TOSHIBA MBF2600RC 3706 1172123568 Blocks, 600 GB
SASDeviceName 50000394f808bb28 SASAddress 50000394f808bb2a PhyNum 1
Target c
Unit 0 Disk TOSHIBA MBF2600RC 3706 1172123568 Blocks, 600 GB
SASDeviceName 500003942823fb38 SASAddress 500003942823fb3a PhyNum 2
Target d
Unit 0 Disk TOSHIBA MBF2600RC 3706 1172123568 Blocks, 600 GB
SASDeviceName 500003940832182c SASAddress 500003940832182e PhyNum 3
Target e
Unit 0 Encl Serv device FUJITSU NBBEXP 0d32
SASAddress 500000e0e04919bd PhyNum 14

then select the controller:

{0} ok select /pci@8000/pci@4/pci@0/pci@0/scsi@0

{0} ok show-children

FCode Version 1.00.56, MPT Version 2.00, Firmware Version 13.00.66.00

Target is used to create the volume and PhyNum is the physical slot the disk is in

{0} ok show-volumes
No volumes to show

{0} ok c d create-raid1-volume
Target c size is 1169920000 Blocks, 598 GB
Target d size is 1169920000 Blocks, 598 GB
The volume can be any size from 1 MB to 571250 MB
What size do you want? [571250] 10000
Volume size will be 20480000 Blocks, 10 GB
Enter a volume name: [0 to 15 characters] c-d-volume
Volume has been created

Only 1 volume can be created on a disk pair so accept the maximum size for the volume

{0} ok show-children

FCode Version 1.00.56, MPT Version 2.00, Firmware Version 13.00.66.00

Target a
Unit 0 Disk TOSHIBA MBF2600RC 3706 1172123568 Blocks, 600 GB
SASDeviceName 50000394f808cd88 SASAddress 50000394f808cd8a PhyNum 0
Target b
Unit 0 Disk TOSHIBA MBF2600RC 3706 1172123568 Blocks, 600 GB
SASDeviceName 50000394f808bb28 SASAddress 50000394f808bb2a PhyNum 1
Target e
Unit 0 Encl Serv device FUJITSU NBBEXP 0d32
SASAddress 500000e0e04919bd PhyNum 14
Target 11e Volume 0
Unit 0 Disk LSI Logical Volume 3000 20480000 Blocks, 10 GB
VolumeDeviceName 3e48d01ec24d306e VolumeWWID 0e48d01ec24d306e

Once the volume is created the Volume will be shown, but not the physical disks that make it up

{0} ok show-volumes
Volume 0 Target 11e Type RAID1 (Mirroring)
Name c-d-volume WWID 0e48d01ec24d306e
Optimal Enabled Background Init In Progress Data Scrub In Progress
2 Members 20480000 Blocks, 10 GB
Disk 0
Primary Optimal
Target c TOSHIBA MBF2600RC 3706 PhyNum 2
Disk 1
Secondary Optimal
Target d TOSHIBA MBF2600RC 3706 PhyNum 3

The show-volume shows the PhyNum (slot numbers)

If a disk fails

{0} ok show-volumes
Volume 0 Target 11e Type RAID1 (Mirroring)
Name c-d-volume WWID 0e48d01ec24d306e
Degraded Enabled Data Scrub In Progress
2 Members 20480000 Blocks, 10 GB
Disk 0
Primary Optimal
Target c TOSHIBA MBF2600RC 3706 PhyNum 2
Disk 1
Secondary Offline Out Of Sync
Target d TOSHIBA MBF2600RC 3706

will show that a disk, Disk 1 in this case, is Offline or Failed, but the PhyNum is cleared. So determining the slot of the failed disk is hard. The disk Part # MBF2600RC is shown.

{0} ok show-children

FCode Version 1.00.56, MPT Version 2.00, Firmware Version 13.00.66.00

Target a
Unit 0 Disk TOSHIBA MBF2600RC 3706 1172123568 Blocks, 600 GB
SASDeviceName 50000394f808cd88 SASAddress 50000394f808cd8a PhyNum 0
Target b
Unit 0 Disk TOSHIBA MBF2600RC 3706 1172123568 Blocks, 600 GB
SASDeviceName 50000394f808bb28 SASAddress 50000394f808bb2a PhyNum 1
Target e
Unit 0 Encl Serv device FUJITSU NBBEXP 0d32
SASAddress 500000e0e04919bd PhyNum 14
Target 11e Volume 0
Unit 0 Disk LSI Logical Volume 3000 20480000 Blocks, 10 GB
VolumeDeviceName 3e48d01ec24d306e VolumeWWID 0e48d01ec24d306e

To determine the failed disks physical location, ask the customer what slots have disks installed in them. For this example the customer will indicate that there are disks in slots 0, 1, 2 and 3. The show-volumes and show-children show 3 disks, slot 0, 1 and 2 and a failed disk in the volume. By comparing the customers information to the show-volumes and show-children information, the failed disk is in slot 3. Once the slot is determined, simply remove the disk and insert the replacement disk. For a mirror the volume will be sync ed up. For a stripe, delete the volume, recreate the volume and restore the data.

Bug 18247156 has been created for this issue

Creating a mirror using sas2ircu

The list command will show the controller numbers which are based on the BB#s

# sas2ircu LIST
LSI Corporation SAS2 IR Configuration Utility.
Version 18.00.00.00 (2013.11.18)
Copyright (c) 2009-2013 LSI Corporation. All rights reserved.

Adapter Vendor Device SubSys SubSys
Index Type ID ID Pci Address Ven ID Dev ID
----- ------------ ------ ------ ----------------- ------ ------
0 SAS2308_2 1000h 87h 00h:03h:00h:00h 10cfh 177fh

Adapter Vendor Device SubSys SubSys
Index Type ID ID Pci Address Ven ID Dev ID
----- ------------ ------ ------ ----------------- ------ ------
1 SAS2308_2 1000h 87h 00h:03h:00h:00h 10cfh 177fh

the above output shows two BBs / controllers

the display command will show the disk and enclosure information

# sas2ircu 0 display
LSI Corporation SAS2 IR Configuration Utility.
Version 18.00.00.00 (2013.11.18)
Copyright (c) 2009-2013 LSI Corporation. All rights reserved.

Read configuration has been initiated for controller 0
------------------------------------------------------------------------
Controller information
------------------------------------------------------------------------
Controller type                         : SAS2308_2
BIOS version                            : 0.00.00.00
Firmware version                        : 13.00.66.00
Channel description                     : 1 Serial Attached SCSI
Initiator ID                            : 0
Maximum physical devices                : 255
Concurrent commands supported           : 3072
Slot                                    : Unknown
Segment                                 : 0
Bus                                     : 3
Device                                  : 0
Function                                : 0
RAID Support                            : Yes
------------------------------------------------------------------------
IR Volume information
------------------------------------------------------------------------
------------------------------------------------------------------------
Physical device information
------------------------------------------------------------------------
Initiator at ID #0

Device is a Hard disk
Enclosure #                             : 2
Slot #                                  : 0
SAS Address                             : 5000039-4-2828-b15a
State                                   : Ready (RDY)
Size (in MB)/(in sectors)               : 572325/1172123567
Manufacturer                            : TOSHIBA
Model Number                            : MBF2600RC
Firmware Revision                       : 3706
Serial No                               : EA25PC70082K
GUID                                    : 500003942828b158
Protocol                                : SAS
Drive Type                              : SAS_HDD

Device is a Enclosure services device
Enclosure #                             : 2
Slot #                                  : 0
SAS Address                             : 500000e-0-e06d-247d
State                                   : Standby (SBY)
Manufacturer                            : FUJITSU
Model Number                            : BBEXP
Firmware Revision                       : 0d32
Serial No                               : x3625413500
GUID                                    : N/A
Protocol                                : SAS
Device Type                             : Enclosure services device

Device is a Hard disk
Enclosure #                             : 2
Slot #                                  : 1
SAS Address                             : 5000039-4-0832-1952
State                                   : Ready (RDY)
Size (in MB)/(in sectors)               : 572325/1172123567
Manufacturer                            : TOSHIBA
Model Number                            : MBF2600RC
Firmware Revision                       : 3706
Serial No                               : EA25PC5007B1
GUID                                    : 5000039408321950
Protocol                                : SAS
Drive Type                              : SAS_HDD

Device is a Hard disk
Enclosure #                             : 2
Slot #                                  : 2
SAS Address                             : 5000039-4-2823-fb3a
State                                   : Ready (RDY)
Size (in MB)/(in sectors)               : 572325/1172123567
Manufacturer                            : TOSHIBA
Model Number                            : MBF2600RC
Firmware Revision                       : 3706
Serial No                               : EA25PC70080F
GUID                                    : 500003942823fb38
Protocol                                : SAS
Drive Type                              : SAS_HDD

Device is a Hard disk
Enclosure #                             : 2
Slot #                                  : 3
SAS Address                             : 5000039-4-0832-182e
State                                   : Ready (RDY)
Size (in MB)/(in sectors)               : 572325/1172123567
Manufacturer                            : TOSHIBA
Model Number                            : MBF2600RC
Firmware Revision                       : 3706
Serial No                               : EA25PC5007AK
GUID                                    : 500003940832182c
Protocol                                : SAS
Drive Type                              : SAS_HDD
------------------------------------------------------------------------
Enclosure information
------------------------------------------------------------------------
Enclosure#                              : 1
Logical ID                              : 500000e0:e0450000
Numslots                                : 8
StartSlot                               : 0
Enclosure#                              : 2
Logical ID                              : 500000e0:e06d247f
Numslots                                : 9
StartSlot                               : 0
------------------------------------------------------------------------

The above display shows the Enclosure, 2, and there are 4 physical disks in slots 0, 1, 2 and 3.

To create a 10GB mirror on controller 0 using physical disks 2 and 3:

You are about to create an IR volume.

WARNING: Proceeding with this operation may cause data loss or data
corruption. Are you sure you want to proceed (YES/NO)? yes

WARNING: This is your last chance to abort this operation. Do you wish
to abort (YES/NO)? no
WARNING: You are creating a volume with size other than 'MAX'.
Do you want to continue with volume creation (YES/NO)? yes

Note if MAX is used instead of 10000 (this is in MB, so this mirror is 10GB) the largest possible volume will be created.

Since only 1 volume can be created on a disk pair, use MAX

Background command progress status for controller 0...
IR Volume 1
Volume ID : 286
Current operation : None
Volume status : Enabled
Volume state : Optimal
Volume wwid : 0e98755112ed9dfb
Physical disk I/Os : Not quiesced
SAS2IRCU: Command STATUS Completed Successfully.
SAS2IRCU: Utility Completed Successfully.

showing all internal disk information and volumes on controller 0

Read configuration has been initiated for controller 0
------------------------------------------------------------------------
Controller information
------------------------------------------------------------------------
Controller type : SAS2308_2
BIOS version : 0.00.00.00
Firmware version : 13.00.66.00
Channel description : 1 Serial Attached SCSI
Initiator ID : 0
Maximum physical devices : 255
Concurrent commands supported : 3072
Slot : Unknown
Segment : 0
Bus : 3
Device : 0
Function : 0
RAID Support : Yes
------------------------------------------------------------------------
IR Volume information
------------------------------------------------------------------------
IR volume 1
Volume ID : 286
Volume Name : c-d-mirror
Status of volume : Okay (OKY)
Volume wwid : 0e98755112ed9dfb
RAID level : RAID1
Size (in MB) : 10000
Physical hard disks :
PHY[0] Enclosure#/Slot# : 2:2
PHY[1] Enclosure#/Slot# : 2:3
------------------------------------------------------------------------
Physical device information
------------------------------------------------------------------------
Initiator at ID #0

Device is a Hard disk
Enclosure # : 2
Slot # : 0
SAS Address : 5000039-4-2828-b15a
State : Ready (RDY)
Size (in MB)/(in sectors) : 572325/1172123567
Manufacturer : TOSHIBA
Model Number : MBF2600RC
Firmware Revision : 3706
Serial No : EA25PC70082K
GUID : 500003942828b158
Protocol : SAS
Drive Type : SAS_HDD

Device is a Enclosure services device
Enclosure # : 2
Slot # : 0
SAS Address : 500000e-0-e06d-247d
State : Standby (SBY)
Manufacturer : FUJITSU
Model Number : BBEXP
Firmware Revision : 0d32
Serial No : x3625413500
GUID : N/A
Protocol : SAS
Device Type : Enclosure services device

Device is a Hard disk
Enclosure # : 2
Slot # : 1
SAS Address : 5000039-4-0832-1952
State : Ready (RDY)
Size (in MB)/(in sectors) : 572325/1172123567
Manufacturer : TOSHIBA
Model Number : MBF2600RC
Firmware Revision : 3706
Serial No : EA25PC5007B1
GUID : 5000039408321950
Protocol : SAS
Drive Type : SAS_HDD

Device is a Hard disk
Enclosure # : 2
Slot # : 2
SAS Address : 5000039-4-2823-fb3a
State : Optimal (OPT)
Size (in MB)/(in sectors) : 572325/1172123567
Manufacturer : TOSHIBA
Model Number : MBF2600RC
Firmware Revision : 3706
Serial No : EA25PC70080F
GUID : 500003942823fb38
Protocol : SAS
Drive Type : SAS_HDD

Device is a Hard disk
Enclosure # : 2
Slot # : 3
SAS Address : 5000039-4-0832-182e
State : Optimal (OPT)
Size (in MB)/(in sectors) : 572325/1172123567
Manufacturer : TOSHIBA
Model Number : MBF2600RC
Firmware Revision : 3706
Serial No : EA25PC5007AK
GUID : 500003940832182c
Protocol : SAS
Drive Type : SAS_HDD
------------------------------------------------------------------------
Enclosure information
------------------------------------------------------------------------
Enclosure# : 1
Logical ID : 500000e0:e0450000
Numslots : 8
StartSlot : 0
Enclosure# : 2
Logical ID : 500000e0:e06d247f
Numslots : 9
StartSlot : 0
Primary Boot Slot : 3
------------------------------------------------------------------------

note the c-d-mirror volume information shows disks 2:2 and 2:3. These disks are physically in slots 2 and 3.

When a disk fails the volume information will look like this:

Read configuration has been initiated for controller 0
------------------------------------------------------------------------
Controller information
------------------------------------------------------------------------
Controller type : SAS2308_2
BIOS version : 0.00.00.00
Firmware version : 13.00.66.00
Channel description : 1 Serial Attached SCSI
Initiator ID : 0
Maximum physical devices : 255
Concurrent commands supported : 3072
Slot : Unknown
Segment : 0
Bus : 3
Device : 0
Function : 0
RAID Support : Yes
------------------------------------------------------------------------
IR Volume information
------------------------------------------------------------------------
IR volume 1
Volume ID : 286
Volume Name : c-d-mirror
Status of volume : Degraded (DGD)
Volume wwid : 0e98755112ed9dfb
RAID level : RAID1
Size (in MB) : 10000
Physical hard disks :
PHY[0] Enclosure#/Slot# : 2:2
PHY[1] Enclosure#/Slot# : 0:0
------------------------------------------------------------------------
Physical device information
------------------------------------------------------------------------
Initiator at ID #0

Device is a Hard disk
Enclosure # : 0
Slot # : 0
SAS Address : 0000000-0-0000-0000
State : Missing (MIS)
Manufacturer : TOSHIBA
Model Number : MBF2600RC
Firmware Revision : 3706
Serial No : EA25PC5007AK
GUID : N/A
Drive Type : Undetermined

For the Failed disk the enclosure:slot information has been set to 0:0. To determine the failed disks physical location, ask the customer what slots have disks installed in them. For this example the customer will indicate that there are disks in slots 0, 1, 2 and 3. The show-volumes and show-children show 3 disks, slot 0, 1 and 2 and a failed disk in the volume. By comparing the customers information to the show-volumes and show-children information, the failed disk is in slot 3. Once the slot is determined, simply remove the disk and insert the replacement disk. For a mirror the volume will be sync ed up. For a stripe, delete the volume, recreate the volume and restore the data.

Bug 18247156 has been created for this issue

Attachments

This solution has no attachment