Sun Fire[TM] 12K/15K/E20K/E25K: Cheat Sheet: hsPCI, hsPCI+ and hsPCI-X Cartridge Locations

Asset ID:	1-72-1017493.1
Update Date:	2017-02-28
Keywords:

Solution Type Problem Resolution Sure

Solution 1017493.1 : Sun Fire[TM] 12K/15K/E20K/E25K: Cheat Sheet: hsPCI, hsPCI+ and hsPCI-X Cartridge Locations

Applies to:

Sun Fire 12K Server - Version All Versions and later
Sun Fire 15K Server - Version All Versions and later
Sun Fire E20K Server - Version All Versions and later
Sun Fire E25K Server - Version All Versions and later
All Platforms

Symptoms

When upgrading a hsPCI slot1 board to hsPCI+ or hsPCI-X on a Sun Fire[TM] 12K/15K/E20K/E25K, with the domain's boot storage attached to a HBA in slot1 (C5V0, upper right), your domain will fail to boot. See notes.

Changes

Cause

Various naming conventions exist for the hsPCI, hsPCI+ and hsPCI-X cartridges in Sun Fire[TM] 12K/15K/E20K/E25K I/O boards.
A translation between them is often needed in diagnosis.

Solution

Use the chart below to translate the various cartridge designations for an hsPCI I/O board.

+-------------------------------------------------------+
|                       IO BOARD                        |
|  +-----------------------+ +-----------------------+  |
|  |     Silk Screen       | |     Silk Screen       |  |
|  |         C5V1          | |         C5V0          |  |
|  |                       | |                       |  |
|  |    Voltage/Speed      | |    Voltage/Speed      |  |
|  |     5.0V, 33Mhz       | |     5.0V, 33Mhz       |  |
|  |                       | |                       |  |
|  |     Designations      | |     Designations      |  |
|  |       Slot 1.1        | |       Slot 0.1        |  |
|  |   PCI IOC 1 Leaf B    | |   PCI IOC 0 Leaf B    |  |
|  |        Slot 3         | |        Slot 1         |  |
|  |                       | |                       |  |
|  |      Device Path      | |      Device Path      |  |
|  |    /pci@+1,7000000    | |    /pci@,7000000      |  |
|  |                       | |                       |  |
|  +-----------------------+ +-----------------------+  |
|  +-----------------------+ +-----------------------+  |
|  |     Silk Screen       | |     Silk Screen       |  |
|  |         C3V1          | |         C3V0          |  |
|  |                       | |                       |  |
|  |    Voltage/Speed      | |    Voltage/Speed      |  |
|  |    3.3V, 33/66Mhz     | |    3.3V, 33/66Mhz     |  |
|  |                       | |                       |  |
|  |     Designations      | |     Designations      |  |
|  |       Slot 1.0        | |       Slot 0.0        |  |
|  |   PCI IOC 1 Leaf A    | |   PCI IOC 0 Leaf A    |  |
|  |        Slot 2         | |        Slot 0         |  |
|  |                       | |                       |  |
|  |      Device Path      | |      Device Path      |  |
|  |    /pci@+1,6000000    | |     /pci@,6000000     |  |
|  |                       | |                       |  |
|  +-----------------------+ +-----------------------+  |
|                                                       |
+-------------------------------------------------------+

Notes:

Slots 0 and 1 cartridges are driven by PCI IOC 0.
Slots 2 and 3 cartridges are driven by PCI IOC 1.
Slot 1 (C5V0) shares its bus with the onboard RIO and SBBC. This slot should be avoided for performance-sensitive interfaces.
Slot x.x designations are with respect to the I/O board.
<a-id> refers to the Safari Agent ID of the IOC, ex. 3c.
<a-id>+1 refers to the next higher Safari Agent ID of the IOC, ex 3d.
See Document 1001879.1 to decode the Safari Agent ID

Use the chart below to translate the various cartridge designations for a
hsPCI+ I/O board ( pn 501-6082 ) or a hsPCI++ board ( pn 501-7274 ).

+-------------------------------------------------------+
|                       IO BOARD                        |
|  +-----------------------+ +-----------------------+  |
|  |     Silk Screen       | |     Silk Screen       |  |
|  |         C3V2          | |         C5V0          |  |
|  |                       | |                       |  |
|  |    Voltage/Speed      | |    Voltage/Speed      |  |
|  |    3.3V, 33/66Mhz     | |     5.0V, 33Mhz       |  |
|  |                       | |                       |  |
|  |     Designations      | |     Designations      |  |
|  |       Slot 1.1        | |       Slot 0.1        |  |
|  |   PCI IOC 1 Leaf B    | |   PCI IOC 0 Leaf B    |  |
|  |        Slot 3         | |        Slot 1         |  |
|  |                       | |                       |  |
|  |      Device Path      | |      Device Path      |  |
|  |    /pci@+1,7000000    | |     /pci@,7000000     |  |
|  |                       | |                       |  |
|  +-----------------------+ +-----------------------+  |
|  +-----------------------+ +-----------------------+  |
|  |     Silk Screen       | |     Silk Screen       |  |
|  |         C3V1          | |         C3V0          |  |
|  |                       | |                       |  |
|  |    Voltage/Speed      | |    Voltage/Speed      |  |
|  |    3.3V, 33/66Mhz     | |    3.3V, 33/66Mhz     |  |
|  |                       | |                       |  |
|  |     Designations      | |     Designations      |  |
|  |       Slot 1.0        | |       Slot 0.0        |  |
|  |   PCI IOC 1 Leaf A    | |   PCI IOC 0 Leaf A    |  |
|  |        Slot 2         | |        Slot 0         |  |
|  |                       | |                       |  |
|  |      Device Path      | |      Device Path      |  |
|  |   /pci@+1,6000000     | |     /pci@,6000000     |  |
|  |                       | |                       |  |
|  +-----------------------+ +-----------------------+  |
|                                                       |
+-------------------------------------------------------+

Notes:

Slots 0 and 1 cartridges are driven by PCI IOC0 (XMITS0).
Slots 2 and 3 cartridges are driven by PCI IOC1 (XMITS1).
Slots 0, 2, and 3 cartridges are on separate PCI buses.
Slot 1 cartridges share the same PCI bus with the onboard RIO and SBBC. There is a PCI-PCI bridge between XMITS0 and the 5v slot, slot 0.1, to provide the slot, the SBBC and RIO a 5v interface.
Slot x.x designations are with respect to the I/O board.
<a-id> refers to the Safari Agent ID of the IOC, ex. 3c
<a-id>+1 refers to the next higher Safari Agent ID of the IOC, ex 3d
see Document 1006575.1 to decode a-id
the hardware path to slot1 (C5V0, upper right) has an extra PCI bridge in it, as a result of which the device path to everything attached to that slot will change, see Document 1006575.1

Use the chart below to translate the various cartridge designations for an
hsPCI-X I/O board (pn 501-7357 ).

+-------------------------------------------------------+
|                       IO BOARD                        |
|  +-----------------------+ +-----------------------+  |
|  |     Silk Screen       | |     Silk Screen       |  |
|  |         C3V2          | |         C5V0          |  |
|  |                       | |                       |  |
|  |    Voltage/Speed      | |    Voltage/Speed      |  |
|  |    3.3V, 90Mhz        | |     5.0V, 33Mhz       |  |
|  |                       | |                       |  |
|  |     Designations      | |     Designations      |  |
|  |       Slot 1.1        | |       Slot 0.1        |  |
|  |   PCI IOC 1 Leaf B    | |   PCI IOC 0 Leaf B    |  |
|  |        Slot 3         | |        Slot 1         |  |
|  |                       | |                       |  |
|  |      Device Path      | |      Device Path      |  |
|  |   /pci@+1,7000000     | |     /pci@,7000000     |  |
|  |                       | |                       |  |
|  +-----------------------+ +-----------------------+  |
|  +-----------------------+ +-----------------------+  |
|  |     Silk Screen       | |     Silk Screen       |  |
|  |         C3V1          | |         C3V0          |  |
|  |                       | |                       |  |
|  |    Voltage/Speed      | |    Voltage/Speed      |  |
|  |    3.3V, 90Mhz        | |    3.3V, 90Mhz        |  |
|  |                       | |                       |  |
|  |     Designations      | |     Designations      |  |
|  |       Slot 1.0        | |       Slot 0.0        |  |
|  |   PCI IOC 1 Leaf A    | |   PCI IOC 0 Leaf A    |  |
|  |        Slot 2         | |        Slot 0         |  |
|  |                       | |                       |  |
|  |      Device Path      | |      Device Path      |  |
|  |   /pci@+1,6000000     | |     /pci@,6000000     |  |
|  |                       | |                       |  |
|  +-----------------------+ +-----------------------+  |
|                                                       |
+-------------------------------------------------------+

Notes:

Slots 0 and 1 cartridges are driven by PCI IOC0 (XMITS0).
Slots 2 and 3 cartridges are driven by PCI IOC1 (XMITS1).
Slots 0, 2, and 3 cartridges are on separate PCI buses.
Slot 1 cartridges share the same PCI bus with the onboard RIO and SBBC. There is a PCI-PCI bridge between XMITS0 and the 5v slot, slot 0.1, to provide the slot, the SBBC and RIO a 5v interface.
Slot x.x designations are with respect to the I/O board.
<a-id> refers to the Safari Agent ID of the IOC, ex. 3c
<a-id>+1 refers to the next higher Safari Agent ID of the IOC, ex 3d
see Document 1006575.1 to decode a-id
the hardware path to slot1 (C5V0, upper right) has an extra PCI bridge in it, as a result of which the device path to everything attached to that slot will change, see Document 1006575.1

Additional Information
The following is an example in order to help in matching what described above with the output of an hpost:

I/O_Brds:         IOC  P1/Bus/Adapt   IOC  P0/Bus/Adapt
Slot  Gen  Type   P1   B1/10 B0/10    P0   B1/eb10 B0/10  (e=ENet, b=BBC)
IO00:  P   hsPCI   P    p _e  p _p     P    p PP_p  p _m

  IOC/P1/B1 = slot 1.1 - C5V1 (hsPCI)   slot 3
- C3V2 (hsPCI+)  slot 3
- C3V2 (hsPCI-X) slot 3
IOC/P1/B0 = slot 1.0 - C3V1           slot 2
IOC/P0/B1 = slot 0.1 - C5V0           slot 1
IOC/P0/B0 = slot 0.0 - C3V0           slot 0

IOC/P1 = IO Controller 1 (schizo on hsPCI, xmits on hsPCI+)
IOC/P0 = IO Controller 0 (schizo on hsPCI, xmits on hsPCI+)

In this example:

the slot C5V1 (hsPCI) contains an empty cassette
the slot C3V0 does not contain any cassette

based on the values from the following key (which is printed in the hpost output):

Key to resource status value codes:
?=Unknown       p=Present       c=Crunched      _=Undefined     m=Missing
i=Misconfig     o=FailedOBP     f=Failed        b=Blacklisted   r=Redlisted
x=NotInDomain   u=G,unconfig    P=Passed        ==G,lockstep    l=NoLicense
e=EmptyCasstt

-----------------------------------------------------------------------------

There are a couple different ways to determine which type of I/O Board is
installed in your system, without the part number. In the examples below,
IO9 is the hsPCI I/O board, while IO10 is the hsPCI+ I/O Board.

From the domain:

# cfgadm -l
Ap_Id                          Type         Receptacle   Occupant     Condition

IO9                            HPCI         connected    configured   ok
IO10                           HPCI+        connected    configured   ok

From the MAIN System Controller:

sms-svc% showboards

Location    Pwr    Type of Board   Board Status  Test Status   Domain
--------    ---    -------------   ------------  -----------   ------

IO9         On     HPCI            Active        Passed        Dom A
IO10        On     HPCI+          Available     Unknown       Isolated

-----------------------------------------------------------------------------

References:

Document 1001879.1 - Sun Fire[TM] 12K/15K/E20K/E25K: Determining slot numbers from the I/O device paths
Document 1004773.1 - Sun Fire[TM] 12K/15K/E20K/E25K: Solaris[TM] Operating System IO device probing and configuration
Document 1006575.1 - After upgrading hsPCI to hsPCI+ domain fails to boot

Product
Sun Fire 15K Server
Sun Fire 12K Server
Sun Fire E25K Server
Sun Fire E20K Server

Internal Comments

To differentiate between hsPCI+ and hsPCI-X, you can run the redx command "seepr slot <expander> 1".

Slot 6 contains a hsPCI+ board ("redx -cx {seepr slot 6 1 }" ):
Slot IO6 (6.1) Seeprom manufacturing data:
Part Number 5016082-11 Rev 02
Serial Number Y03MUM Manufacturer's Code 0x301
Programmed Thu Jan 8 09:11:12 2004
FRUID,PRGM,SCAT,HPCI+,HPCI+-11
Manuf At: CHARLOTTE NC USA

Slot 7 contains a hsPCI-X board ("redx -cx {seepr slot 7 1 }" ):
Slot IO7 (7.1) Seeprom manufacturing data:
Part Number 5017357-01 Rev 50
Serial Number Y0ACYW Manufacturer's Code 0x301
Programmed Wed Jun 14 09:41:47 2006
ROHS ASSY,MECH,HSPCIX,E20K/E25K
Manuf At: CHARLOTTE NC USA

Slot 10 contains a hsPCI board ("redx -cx {seepr slot 10 1 }" ):
Slot IO10 (10.1) Seeprom manufacturing data:
Part Number 5015397-11 Rev 54
Serial Number Y03KL6 Manufacturer's Code 0x301
Programmed Wed May 28 22:51:57 2003
ASSY,ECB,HOT-SWAP PCI,STARCAT
Manuf At: CHARLOTTE NC USA

Keywords: hsPCI, hsPCI+, hsPCI-X, cartridge, 15K, 12K, SF15K, SF12K, starcat, E20K, E25K, upgrade, boot

Previously Published As 48199

Attachments

This solution has no attachment