Sun Microsystems, Inc.  Oracle System Handbook - ISO 7.0 May 2018 Internal/Partner Edition
   Home | Current Systems | Former STK Products | EOL Systems | Components | General Info | Search | Feedback

Asset ID: 1-79-1416397.1
Update Date:2018-04-09
Keywords:

Solution Type  Predictive Self-Healing Sure

Solution  1416397.1 :   How to Identify Which Flash card is Installed in an Exadata Storage Cell and Order the Correct FRU  


Related Items
  • Exadata X3-2 Hardware
  •  
  • Exadata Database Machine X2-8
  •  
  • Exadata X3-8 Hardware
  •  
  • Exadata Database Machine X2-2 Hardware
  •  
  • Oracle Exadata Hardware
  •  
  • Exadata Database Machine V2
  •  
Related Categories
  • PLA-Support>Sun Systems>x86>Engineered Systems HW>SN-x64: EXADATA
  •  


This is a How-to guide detailing the Flash components used in Exadata systems.

Oracle Confidential PARTNER - Available to partners (SUN).
Reason: Intended for resolving SR's, and provide proactive FRU replacement guidelines requiring an Oracle engineer to review

Applies to:

Oracle Exadata Hardware - Version 11.2.0.1 and later
Exadata Database Machine X2-2 Hardware - Version All Versions and later
Exadata X3-8 Hardware - Version All Versions and later
Exadata Database Machine V2 - Version All Versions and later
Exadata X3-2 Hardware - Version All Versions and later
Information in this document applies to any platform.
Exadata, Flash, FMOD, FDOM, Aura, V2, X2-2, X2-8, X3-2, X3-8


Purpose

How to identify which Flash card is present in Exadata Storage cell to order the correct FRU


Scope

For internal and HW support partner use only.  This document contains engineering level information, not suitable for customer distribution. Do not distribute to customers.

Exadata support engineers in hardware TSC and software EEST groups, and hardware Field Support
working Flash related Service Requests.

Exadata customer account managers fielding Flash related Service Requests and customer questions.

TSC and field support hardware engineers servicing systems containing the Flash cards referenced.

Details

The Exadata Storage cells contain Solid State Flash memory. A Flash 1.0 and 1.1 assembly consists of:

  • PCI Express Flash Board 
  • 24GB Solid State Flash Memory Module
  • Energy Storage Module (ESM)

A Flash Card assembly is also known as an Aura 1.0 or 1.1 card. Aura was the engineering code name.

A Flash 2.0 assembly is a self-contained FRU ,the PCI Flash Board and 400GB SSD Module are a complete assembly. The engineering code name is Aura 2.



 FRU part number  Description Where Used Firmware Version and Image
      511-1275  PCI Express Flash Board - F20 1.0 SAS HBA Sun Fire X4275 in Exadata V2 LSI 1.27.03 Image 11.2.1.2.0
LSI 1.27.90 Image 11.2.1.3.1
LSI 1.27.91 Image 11.2.2.2.0
LSI 1.27.92 Image 11.2.2.3.2 or later
 
     511-1275  PCI Express Flash Board - F20 1.0 SAS HBA Sun Fire X4270-M2 in Exadata X2-2 and X2-8 LSI 1.27.90 Image 11.2.2.1.0
LSI 1.27.91 Image 11.2.2.2.0
LSI 1.27.92 Image 11.2.2.3.2 or later
     511-1500  PCI Express Flash Board - F20 1.0 SAS HBA Sun Fire X4275 in Exadata V2 LSI 1.27.03 Image 11.2.1.2.0
LSI 1.27.90 Image 11.2.1.3.1
LSI 1.27.91 Image 11.2.2.2.0
LSI 1.27.92 Image 11.2.2.3.2 or later
     511-1500  PCI Express Flash Board - F20 1.0 SAS HBA Sun Fire X4270-M2 in Exadata X2-2 and X2-8 LSI 1.27.90 Image 11.2.2.1.0
LSI 1.27.91 Image 11.2.2.2.0
LSI 1.27.92 Image 11.2.2.3.2 or later
     541-4416 PCI Express Flash Board - F20 M2 1.1 SAS HBA Sun Fire X4275 in Exadata V2 LSI 1.27.92 Image 11.2.2.3.2 or later
     541-4416 PCI Express Flash Board - F20 M2 1.1 SAS HBA Sun Fire X4270-M2 in Exadata X2-2 and X2-8 LSI 1.27.92 Image 11.2.2.3.2 or later
      7026993 PCI Express Flash Board and SSD - F40 2.0 SAS HBA Exadata X3-2 and X3-8 LSI 06.05.10.00 Image 11.2.3.2.0 or later
      7070787 PCI Express Flash Board and SSD - F40 2.0 SAS HBA Exadata X3-2 and X3-8 LSI 06.05.10.00 Image 11.2.3.2.0 or later

 

Note:
1. The minimum OS image required to support 541-4416 F20 M2 is Exadata image patch update 11.2.2.3.2 or later
2. If for some reason, a version 1.0 flash board 511-1500 or 511-1275 is being replaced with a version 1.1 flash board 541-4416, it is also necessary to replace the ESM with part 371-4953 to avoid the ESM cable coming into contact with other PCI boards on the same riser, and to ensure the FMOD's have D20Y firmware or above
3.The minimum OS image required to support 7026993 or  7070787 is Exadata image 11.2.3.2.0 or later
4.7026993 and 7070787 are fully compatible and can be used as alternatives for either part





 FRU part number  Description  Where Used Firmware Version and Image
      371-4415 24GB Solid State Flash Memory Module Sun Fire X4275 in Exadata V2

 D20R Image 11.2.1.2.0
 D20Y Image 11.2.2.1.1 or later

 D21Y firmware available with patch 14793859

      371-4415 24GB Solid State Flash Memory Module Sun Fire X4270-M2 in Exadata X2-2 and X2-8

 D20Y Image 11.2.2.1.1 or later

 D21Y firmware available with patch 14793859

      371-5014 24GB Solid State Flash Memory Module Sun Fire X4275 in Exadata V2 on F20 M2 boards

 D20Y Image 11.2.2.1.1 or later

 D21Y firmware available with patch 14793859

      371-5014 24GB Solid State Flash Memory Module Sun Fire X4270-M2 in Exadata X2-2 and X2-8 on F20 M2 boards

 D20Y Image 11.2.2.1.1 or later

 D21Y firmware available with patch 14793859

     7061269  24GB Solid State Flash Memory Module

Sun Fire X4275 in Exadata V2

Sun Fire X4270-M2 in Exadata X2-2 and X2-8

 Firmware D21Y

 

Notes:
1. The minimum image version required to support 371-5014 is Exadata image 11.2.2.1.1

2. 371-4415 and 371-5014 are different only in firmware, however they are not substitutable per logistics rules even though the Exadata will update the firmware according to the image
3. The FMOD output model string such as 'SD88SA02' in 'lsscsi' and firmware version is NOT an indicator of the FRU part number. Only the F20 card type should be used to determine which FMOD part number to use .
4. The 371-5014 was reworked to 7061269 in December 2012.
5. The 371-4415 was reworked to 7061269 in December 2012.
 
The SSD contained on 7026993 and 7070787 is not removeable,it is not a separate FRU item





 FRU part number  Description  Where Used
      371-4650 Energy Storage Module (ESM) Must only be fitted to 511-1275 or 511-1500
      371-4953 Energy Storage Module (ESM) Must only be fitted to 541-4416

 

Note: Ensure ESM is fitted to the correct Flash board
The 7026993 and 7070787 do not contain an ESM




How to identify the model of Flash Board

Use the command lspci to identify the model of flash board.This command will show the hardware version of the card. The output to lspci can be found in a "sundiag" collection.

For 1.0 and 1.1 F20 cards:

( rev 08) = 511-1275 or 511-1500
( rev 10) = 541-4416

 

For 2.0 F40 card

( rev 03) = 7026993 or 7070787

 


This example shows 511-1275 or 511-1500:

# lspci | grep MPT 1b:00.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1068E PCI-Express Fusion-MPT SAS (rev 08)
21:00.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1068E PCI-Express Fusion-MPT SAS (rev 08)
29:00.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1068E PCI-Express Fusion-MPT SAS (rev 08)
2f:00.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1068E PCI-Express Fusion-MPT SAS (rev 08) 


This example shows 541-4416:

# lspci | grep MPT
1b:00.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1068E PCI-Express Fusion-MPT SAS (rev 10)
21:00.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1068E PCI-Express Fusion-MPT SAS (rev 10)
29:00.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1068E PCI-Express Fusion-MPT SAS (rev 10)
2f:00.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1068E PCI-Express Fusion-MPT SAS (rev 10)

 

This example shows 7026993 or 7070787: 

# lspci | grep SSD
20:00.0 Serial Attached SCSI controller: LSI Logic / Symbios Logic SSS6200 PCI-Express Flash SSD (rev 03) 
30:00.0 Serial Attached SCSI controller: LSI Logic / Symbios Logic SSS6200 PCI-Express Flash SSD (rev 03) 
90:00.0 Serial Attached SCSI controller: LSI Logic / Symbios Logic SSS6200 PCI-Express Flash SSD (rev 03) 
b0:00.0 Serial Attached SCSI controller: LSI Logic / Symbios Logic SSS6200 PCI-Express Flash SSD (rev 03) 


An alternative command to use for 1.0 and 1.1 F20 cards ONLY,this command is not applicable to 2.0 F40 cards.

/usr/bin/flash_dom -l

command as follows.The output to flash_dom -l can be found in "sundiag" collection. The "Rev" of the card can be B3 or C0

 

B3 = 511-1275 or 511-1500

C0 = 541-4416

This example shows a truncated output from the command. Each flash board will be displayed with its corresponding flash modules. Note also the FMOD Firmware. The example only shows the flash card /proc/mpt/ioc0.

This example is of a 511-1275 or 511-1500:

# /usr/bin/flash_dom -l ...
 HBA# Port Name         Chip Vendor/Type/Rev    MPT Rev  Firmware Rev  IOC

 1.  /proc/mpt/ioc0    LSI Logic SAS1068E B3     105      011b0300     0
        Current active firmware version is 011b0300 (1.27.03)
        Firmware image's version is MPTFW-01.27.03.00-IT
        x86 BIOS image's version is MPTBIOS-6.26.00.00 (2008.10.14)
        FCode image's version is MPT SAS FCode Version 1.00.49 (2007.09.21)

          D#  B___T  Type       Vendor   Product          Rev
          1.  0   0  Disk       ATA      MARVELL SD88SA02 D20R
          2.  0   1  Disk       ATA      MARVELL SD88SA02 D20R
          3.  0   2  Disk       ATA      MARVELL SD88SA02 D20R
          4.  0   3  Disk       ATA      MARVELL SD88SA02 D20R

 

This example is of a 541-4416 card:

# /usr/bin/flash_dom -l
...
HBA# Port Name Chip Vendor/Type/Rev MPT Rev Firmware Rev IOC WWID Serial Number

1. /proc/mpt/ioc0 LSI Logic SAS1068E C0 105 011b5c00 0 5080020000c11bc0 465769T+1111A400JY

Current active firmware version is 011b5c00 (1.27.92)
Firmware image's version is MPTFW-01.27.92.00-IT
x86 BIOS image's version is MPTBIOS-6.26.00.00 (2008.10.14)
FCode image's version is MPT SAS FCode Version 1.00.49 (2007.09.21)


D# B___T Type Vendor Product Rev Operating System Device Name
1. 0 0 Disk ATA MARVELL SD88SA02 D20Y /dev/sdn [8:0:0:0]
2. 0 1 Disk ATA MARVELL SD88SA02 D20Y /dev/sdo [8:0:1:0]
3. 0 2 Disk ATA MARVELL SD88SA02 D20Y /dev/sdp [8:0:2:0]
4. 0 3 Disk ATA MARVELL SD88SA02 D20Y /dev/sdq [8:0:3:0]


 

It is also possible to identify the type of version flash board from the ILOM. 

Use the "show" command from within the ILOM to read the FRUID information using the PCIE riser board targets:

For Exadata V2 , X2-2 and X2-8


/SYS/MB/RISER1/PCIE1/F20CARD
/SYS/MB/RISER1/PCIE4/F20CARD
/SYS/MB/RISER2/PCIE2/F20CARD
/SYS/MB/RISER2/PCIE5/F20CARD

 

For Exadata X3-2 and X3-8

/System/PCI_Devices/Add-on/Device_1

/System/PCI_Devices/Add-on/Device_2

/System/PCI_Devices/Add-on/Device_4

/System/PCI_Devices/Add-on/Device_5



This example shows a 511-1275 or 511-1500 located in PCIE slot 1 within V2 ,X2-2 and X2-8 :


-> show /SYS/MB/RISER1/PCIE1/F20CARD

/SYS/MB/RISER1/PCIE1/F20CARD
Targets:
PRSNT
UPTIME

Properties:
type = F20 Card
ipmi_name = PCIE1/F20CARD
fru_name = BD,ASY,PCIE1_FLASH,AURA_P1.0
fru_part_number = 511-1275-04
fru_serial_number = 0111APO-0941AU0022
fru_extra_1 = 50 Aura
fault_state = OK
clear_fault_action = (none)


 

 This example shows a 541-4416 located in PCIE slot 1 within V2 , X2-2 and X2-8:

-> show /SYS/MB/RISER1/PCIE1/F20CARD

Targets:
PRSNT
UPTIME

Properties:
type = F20 Card
ipmi_name = PCIE1/F20CARD
fru_name = ASY,BD,PCIE_FLASH,F20 M2
fru_part_number = 511-1646-01
fru_serial_number = 465769T+1111A400JV
fru_extra_1 = 04 F20 M2
fault_state = OK
clear_fault_action = (none)

 

 

Note:For the Exadata V2 image version 11.2.2.3.2 or later is required to see the Flash board in slot 4.Before this version the slot could not be displayed,the other slots are displayed in full


This example shows 7103066 or 7070787 within Exadata X3-2 or X3-8 in PCIE slot 4:

-> show /System/PCI_Devices/Add-on/Device_4

 /System/PCI_Devices/Add-on/Device_4
    Targets:

    Properties:
        part_number = 7103066
        description = Sun Flash Accelerator F40 400GB eMLC PCI-E 2.0 Low
                      Profile Adapter
        location = PCIE4 (PCIe Slot 4)
        pci_vendor_id = 0x1000
        pci_device_id = 0x007e
        pci_subvendor_id = 0x108e
        pci_subdevice_id = 0x0581


 

References

<NOTE:1501410.1> - INTERNAL Exadata Database Machine Hardware Firmware and Patch History (X3-2, X4-2, X3-8, X4-8 w/X4-2L)
<NOTE:1358327.1> - INTERNAL Exadata Database Machine Hardware Firmware and Patch History (V2, X2-2, X2-8)
<NOTE:888828.1> - Exadata Database Machine and Exadata Storage Server Supported Versions

Attachments
This solution has no attachment
  Copyright © 2018 Oracle, Inc.  All rights reserved.
 Feedback