Sun Fire[TM] 12K/15K/E20K/E25K, Sun Fire V1280/4800/6800/E2900/E4900/E6900, Netra 1280/1290 servers: UltraSPARC IV+ DR restrictions on Solaris10 HW1

Asset ID:	1-71-1009465.1
Update Date:	2017-02-02
Keywords:

Solution Type Technical Instruction Sure

Solution 1009465.1 : Sun Fire[TM] 12K/15K/E20K/E25K, Sun Fire V1280/4800/6800/E2900/E4900/E6900, Netra 1280/1290 servers: UltraSPARC IV+ DR restrictions on Solaris10 HW1

Applies to:

Sun Netra 1290 Server - Version Not Applicable and later
Sun Fire V1280 Server - Version Not Applicable and later
Sun Fire 12K Server - Version Not Applicable and later
Sun Fire 4800 Server - Version Not Applicable and later
Sun Fire 6800 Server - Version Not Applicable and later
All Platforms

Goal

With the introduction of UltraSPARC(R) IV+ 1.5/1.8/1.95Ghz processor based system boards:

Sun Fire[TM] 12K/15K/E20K/E25K platforms will require SMS 1.6 with patches. For details please see .

Midrange Sun Fire V1280/4800/6800/E2900/E4900/E6900, Netra 1280/1290 platforms will require the ScApp firmware version 5.19 to facilitate support for the 1.5Ghz board, 5.20.1 for the 1.8Ghz and 5.20.4 for the 1.95GHz version. Please see for details.

All domain configurations resident on the above platforms employing UltraSPARC(R) IV+ processor based system boards will require Solaris[TM] 10 HW1 or Solaris[TM] 9 Update 8 operating system as their minimum Solaris releases.

For Solaris 9 please see "Sun Fire[TM] 12K/15K/E20K/E25K, Sun Fire V1280/4800/6800/E2900/E4900/E6900, Netra 1280/1290 servers: UltraSPARC IV+ DR restrictions on Solaris9 Update8"

Fix

This technical instruction discusses a restriction introduced to Dynamic Reconfiguration ( DR ) for USIV+ based system boards on Sun Fire[TM] 12K/15K/E20K/E25K , Sun Fire V1280/4800/6800/E2900/E4900/E6900, or Netra 1280/1290 platforms running Solaris10 HW1.

The data memory management unit (MMU) in the USIV+ has three Translation Lookaside Buffers (TLBs) -- a 16-entry fully-associative TLB and two 512-entry 2-way set-associative TLBs. The second of the two 512-entry TLBs in USIV+ is designed to support page sizes of 8 KB, 64 KB, 32 MB and 256 MB. The 512-entry TLBs are programmable independently and only support one page size (per context) at any given time. Hence, for applications that need large page sizes to map extremely large data sets, USIV+ based processors will facilitate support for 32MB or 256MB page sizes.

With the introduction of Solaris10 HW1 OS environment, the OS will integrate support for these new larger page sizes when booted in a homogeneous USIV+ based processors environments.

For example, a Solaris10 HW1 environment booted in a homogeneous USIV+ processors, the "prtdiag -v" output would display the following output : i.e.,

            CPU      Run    E$   CPU      CPU
FRU Name    ID       MHz    MB   Impl.    Mask
---------- ------- ---- ---- ------- ----
/N0/SB4/P0   16,528 1500 32.0 US-IV+   2.1
/N0/SB4/P1   17,529 1500 32.0 US-IV+   2.1
/N0/SB4/P2   18,530 1500 32.0 US-IV+   2.1
/N0/SB4/P3   19,531 1500 32.0 US-IV+   2.1

# pagesize -a
8192
65536
524288
4194304
33554432
268435456
#

Conversely, the same Solaris10 HW1 environment booted into a non-all USIV+
setup will report the following observations :

            CPU      Run    E$   CPU      CPU
FRU Name    ID       MHz    MB   Impl.    Mask
---------- ------- ---- ---- ------- ----
/N0/SB0/P0    0      1050   8.0 US-III+ 11.0
/N0/SB0/P1    1      1050   8.0 US-III+ 11.0
/N0/SB0/P2    2      1050   8.0 US-III+ 11.0
/N0/SB0/P3    3      1050   8.0 US-III+ 11.0
/N0/SB4/P0   16,528 1500 32.0 US-IV+   2.1
/N0/SB4/P1   17,529 1500 32.0 US-IV+   2.1
/N0/SB4/P2   18,530 1500 32.0 US-IV+   2.1
/N0/SB4/P3   19,531 1500 32.0 US-IV+   2.1

# pagesize -a
8192
65536
524288
4194304
#

As observed from the above two sample "pagesize -a" command output, the Solaris10 HW1 will only support 32MB & 256MB as supported hardware address translation sizes where the OS is booted into an all-USIV+ processor configuration.

As a direct consequence of the fact that Solaris10 HW1 environment integrates support for page sizes that are only available on USIV+ based processors (and not USIII , USIII+ and USIV based processors), it would mean that any attempts to employ Dynamic Reconfiguration (DR) operation(s) to connect or configure any USIII+ or USIV based system boards into an S10 HW1 OS environment, that is booted in an all-USIV+ configuration, will fail.

An example of this DR 'failure' is as follows :

            CPU      Run    E$   CPU      CPU
FRU Name    ID       MHz    MB   Impl.    Mask
---------- ------- ---- ---- ------- ----
/N0/SB4/P0   16,528 1500 32.0 US-IV+   2.1
/N0/SB4/P1   17,529 1500 32.0 US-IV+   2.1
/N0/SB4/P2   18,530 1500 32.0 US-IV+   2.1
/N0/SB4/P3   19,531 1500 32.0 US-IV+   2.1

# pagesize -a
8192
65536
524288
4194304
33554432
268435456
#

# cfgadm
Ap_Id                          Type         Receptacle   Occupant     Condition
N0.IB6                         PCI_I/O_Boa connected    configured   ok
N0.IB8                         PCI_I/O_Boa connected    configured   ok
N0.SB0                         CPU_V2       disconnected unconfigured unknown
N0.SB4                         CPU_V3       connected    configured   ok
c0                             scsi-bus     connected    configured   unknown
c1                             scsi-bus     connected    unconfigured unknown
#

# cfgadm -v -c configure N0.SB0
assign N0.SB0
assign N0.SB0 done
poweron N0.SB0
poweron N0.SB0 done
test N0.SB0

     :
test N0.SB0 done
connect N0.SB0
Aug 10 01:51:17 v4u-4800b-doma sbdp: WARNING: Domain shutdown is required to add a non-UltraSPARC-IV+ board into an all UltraSPARC-IV+ domain
cfgadm: Hardware specific failure: connect N0.SB0: Operation not supported

The above DR restriction applies regardless of whether the new 32MB or 256MB page sizes are employed in the OS (or any resident applications). In addition, it would also imply that the above DR restriction will not apply to the following configuration types :

i. non-all USIV+ based configurations ;

&

ii. Solaris10 HW1 environments booted in a non-all USIV+ based setup but was modified into an all USIV+ based setup using DR techniques.

Product
Sun Fire 12K Server
Sun Fire 15K Server
Sun Fire E20K Server
Sun Fire E25K Server
Sun Fire E6900 Server
Sun Fire 6800 Server
Sun Fire 4800 Server
Sun Fire E4900 Server
Sun Fire V1280 Server
Sun Fire E2900 Server
Netra 1280 Server
Sun Netra 1290 Server

To discuss this information further with Oracle experts and industry peers, we encourage you to review, join or start a discussion in an appropriate My Oracle Support Community, Oracle Sun Technologies Community.

Please see Bug ID 6291264 or Bug ID 6292514 for background information on the implementation.
Please be aware of Sun Fire[TM] 12k/15k/E20/E25k: Using DR during upgrade to USIV+ (Panther) Uniboards could panic domain (Doc ID 1011109.1)

usiv+, dr, cfgadm, s10, pagesize, 32mb, 256mb, dmmu, tlb

References

Attachments

This solution has no attachment