![]() | Oracle System Handbook - ISO 7.0 May 2018 Internal/Partner Edition | ||
|
|
![]() |
||||||||||||
Solution Type Technical Instruction Sure Solution 1017499.1 : Sun Fire[TM] 3800/48x0/6800/E4900/E6900/E2900/V1280 and Netra[TM] 1280/1290 server: Understanding Memory Interleaving
PreviouslyPublishedAs 228626 Applies to:Sun Fire 3800 Server - Version All Versions and laterSun Fire 4800 Server - Version All Versions and later Sun Fire 4810 Server - Version All Versions and later Sun Fire 6800 Server - Version All Versions and later All Platforms Goal*Goal Explains the different memory interleaving options and shows lot of examples.
SolutionEach physical bank (4 DIMMS) of memory has 2 logical banks.
The 1st logical bank is the front of all 4 DIMMS The 2nd logical bank is the back of all 4 DIMMS Memory interleaving, interleaves logical banks together. --------------------------------------------------------------------- Pictorial representation of physical bank 0's 4 DIMMs --------------------------------------------------------------------- Logical bank L0 Logical bank L2 Front of DIMM Back of DIMM --------------------------- --------------------------- | O O O O O | | O O O O O | | O O O O | | O O O O | | 9 16bit drams | | 9 16bit drams | ------------U-------------- ------------U-------------- Front of DIMM Back of DIMM --------------------------- --------------------------- | O O O O O | | O O O O O | | O O O O | | O O O O | | 9 16bit drams | | 9 16bit drams | ------------U-------------- ------------U-------------- Front of DIMM Back of DIMM --------------------------- --------------------------- | O O O O O | | O O O O O | | O O O O | | O O O O | | 9 16bit drams | | 9 16bit drams | ------------U-------------- ------------U-------------- Front of DIMM Back of DIMM --------------------------- --------------------------- | O O O O O | | O O O O O | | O O O O | | O O O O | | 9 16bit drams | | 9 16bit drams | ------------U-------------- ------------U-------------- --------------------------------------------------------------------- Below is a top view of 4 DIMMS (1 physical bank or 2 logical banks). To get the logical bank size, simply add up all the front or backs: 128mb+128mb+128mb+128mb = 512mb logical bank size. --------------------------------------------------------------------- Top view of 4 DIMMs; Each DIMM is 256mb size (128mb, front/back). --------------------------------------------------------------------- 128MB Front ==============256MB DIMM============ DIMM 0 128MB Back 128MB Front ==============256MB DIMM============ DIMM 1 128MB Back 128MB Front ==============256MB DIMM============ DIMM 2 128MB Back 128MB Front ==============256MB DIMM============ DIMM 3 128MB Back --------------------------------------------------------------------- Some technical details:
...and more:
...and still more:
There are 2 memory interleave settings: interleave-scope = within-cpu within-board across-boards interleave-mode = off fixed (only interleave DIMMS of the same size) optimal (interleave DIMMS of different sizes) Example 1 interleave-scope = within-cpu interleave-mode = optimal Looking at interleave segment 0 from a prtdiag example: Logical Logical Logical Port Bank Bank Bank DIMM Interleave Interleave FRU Name ID Num Size Status Size Factor Segment ------------- ---- ---- ------ ------- ------ ---------- ---------- /N0/SB0/P0/B0 0 0 512MB pass 256MB 4-way 0 /N0/SB0/P0/B1 0 1 512MB pass 256MB 4-way 0 /N0/SB0/P0/B0 0 2 512MB pass 256MB 4-way 0 /N0/SB0/P0/B1 0 3 512MB pass 256MB 4-way 0 We have 2 physical banks, so 4 logical banks. Logical bank 0 is the front of DIMMS in physical bank 0 shown below: /N0/SB0/P0/B0/D0 J13300 /N0/SB0/P0/B0/D1 J13400 /N0/SB0/P0/B0/D2 J13500 /N0/SB0/P0/B0/D3 J13600 The size of logical bank 0 = 512MB = 4 DIMMS X 128MB (front of 256MB DIMM); 4 logical banks are interleaved together (4-way) to form Interleave Segment 0.
Example 2 interleave-scope = within-board interleave-mode = optimal Looking at SB4 we see one 4-way segment and two 2-way segments.
Logical Logical Logical Port Bank Bank Bank DIMM Interleave Interleave FRU Name ID Num Size Status Size Factor Segment ------------- ---- ---- ------ ------- ------ ---------- ---------- /N0/SB4/P0/B0 16 0 1024MB pass 512MB 4-way 2 /N0/SB4/P0/B0 16 2 1024MB pass 512MB 4-way 2 /N0/SB4/P1/B0 17 0 1024MB pass 512MB 4-way 2 /N0/SB4/P1/B0 17 2 1024MB pass 512MB 2-way 3 /N0/SB4/P2/B0 18 0 512MB pass 256MB 2-way 4 /N0/SB4/P2/B0 18 2 512MB pass 256MB 2-way 4 /N0/SB4/P3/B0 19 0 1024MB pass 512MB 4-way 2 /N0/SB4/P3/B0 19 2 1024MB pass 512MB 2-way 3 Example 3 interleave-scope = within-cpu interleave-mode = optimal Lets look at SB0/P0.
========================= Memory Configuration =============================== Logical Logical Logical Port Bank Bank Bank DIMM Interleave Interleave FRU Name ID Num Size Status Size Factor Segment ------------- ---- ---- ------ ------- ------ ---------- ---------- /N0/SB0/P0/B1 0 1 512MB pass 256MB 2-way 0 /N0/SB0/P0/B0 0 2 512MB pass 256MB 2-way 0 /N0/SB0/P0/B1 0 3 512MB pass 256MB 1-way 1 6800c-sc:A> showcomp sb0 Component Status Pending POST Description --------- ------ ------- ---- ----------- /N0/SB0/P0 enabled - pass UltraSPARC-III, 750MHz, 8M ECache /N0/SB0/P1 enabled - pass UltraSPARC-III, 750MHz, 8M ECache /N0/SB0/P2 enabled - pass UltraSPARC-III, 750MHz, 8M ECache /N0/SB0/P3 enabled - pass UltraSPARC-III, 750MHz, 8M ECache /N0/SB0/P0/B0/L0 disabled - untest 512M DRAM /N0/SB0/P0/B0/L2 enabled - pass 512M DRAM /N0/SB0/P0/B1/L1 enabled - pass 512M DRAM /N0/SB0/P0/B1/L3 enabled - pass 512M DRAM Example 4 interleave-scope = across-boards interleave-mode = off Below is an example with no interleaving . It's all 1-way. ========================= Memory Configuration =============================== Logical Logical Logical Port Bank Bank Bank DIMM Interleave Interleave FRU Name ID Num Size Status Size Factor Segment ------------- ---- ---- ------ ------- ------ ---------- ---------- /N0/SB0/P0/B1 0 1 512MB pass 256MB 1-way 0 /N0/SB0/P0/B0 0 2 512MB pass 256MB 1-way 1 /N0/SB0/P0/B1 0 3 512MB pass 256MB 1-way 2 /N0/SB0/P1/B0 1 0 512MB pass 256MB 1-way 3 /N0/SB0/P1/B1 1 1 512MB pass 256MB 1-way 4 /N0/SB0/P1/B0 1 2 512MB pass 256MB 1-way 5 /N0/SB0/P1/B1 1 3 512MB pass 256MB 1-way 6 /N0/SB0/P2/B0 2 0 512MB pass 256MB 1-way 7 /N0/SB0/P2/B1 2 1 512MB pass 256MB 1-way 8 /N0/SB0/P2/B0 2 2 512MB pass 256MB 1-way 9 /N0/SB0/P2/B1 2 3 512MB pass 256MB 1-way 10 /N0/SB0/P3/B0 3 0 512MB pass 256MB 1-way 11 /N0/SB0/P3/B1 3 1 512MB pass 256MB 1-way 12 /N0/SB0/P3/B0 3 2 512MB pass 256MB 1-way 13 /N0/SB0/P3/B1 3 3 512MB pass 256MB 1-way 14 /N0/SB3/P0/B0 12 0 512MB pass 256MB 1-way 15 /N0/SB3/P0/B1 12 1 512MB pass 256MB 1-way 16 /N0/SB3/P0/B0 12 2 512MB pass 256MB 1-way 17 /N0/SB3/P0/B1 12 3 512MB pass 256MB 1-way 18 /N0/SB3/P1/B0 13 0 512MB pass 256MB 1-way 19 /N0/SB3/P1/B1 13 1 512MB pass 256MB 1-way 20 /N0/SB3/P1/B0 13 2 512MB pass 256MB 1-way 21 /N0/SB3/P1/B1 13 3 512MB pass 256MB 1-way 22 /N0/SB3/P2/B0 14 0 512MB pass 256MB 1-way 23 /N0/SB3/P2/B1 14 1 512MB pass 256MB 1-way 24 /N0/SB3/P2/B0 14 2 512MB pass 256MB 1-way 25 /N0/SB3/P2/B1 14 3 512MB pass 256MB 1-way 26 /N0/SB3/P3/B0 15 0 512MB pass 256MB 1-way 27 /N0/SB3/P3/B1 15 1 512MB pass 256MB 1-way 28 /N0/SB3/P3/B0 15 2 512MB pass 256MB 1-way 29 /N0/SB3/P3/B1 15 3 512MB pass 256MB 1-way 30 /N0/SB4/P0/B0 16 0 1024MB pass 512MB 1-way 31 /N0/SB4/P0/B0 16 2 1024MB pass 512MB 1-way 32 /N0/SB4/P1/B0 17 0 1024MB pass 512MB 1-way 33 /N0/SB4/P1/B0 17 2 1024MB pass 512MB 1-way 34 /N0/SB4/P2/B0 18 0 512MB pass 256MB 1-way 35 /N0/SB4/P2/B0 18 2 512MB pass 256MB 1-way 36 /N0/SB4/P3/B0 19 0 1024MB pass 512MB 1-way 37 /N0/SB4/P3/B0 19 2 1024MB pass 512MB 1-way 38 Example 5 interleave-scope = within-board interleave-mode = optimal
========================= Memory Configuration =============================== Logical Logical Logical Port Bank Bank Bank DIMM Interleave Interleave FRU Name ID Num Size Status Size Factor Segment ------------- ---- ---- ------ ------- ------ ---------- ---------- /N0/SB0/P0/B0 0 0 512MB pass 256MB 16-way 0 /N0/SB0/P0/B1 0 1 512MB pass 256MB 16-way 0 /N0/SB0/P0/B0 0 2 512MB pass 256MB 16-way 0 /N0/SB0/P0/B1 0 3 512MB pass 256MB 16-way 0 /N0/SB0/P1/B0 1 0 512MB pass 256MB 16-way 0 /N0/SB0/P1/B1 1 1 512MB pass 256MB 16-way 0 /N0/SB0/P1/B0 1 2 512MB pass 256MB 16-way 0 /N0/SB0/P1/B1 1 3 512MB pass 256MB 16-way 0 /N0/SB0/P2/B0 2 0 512MB pass 256MB 16-way 0 /N0/SB0/P2/B1 2 1 512MB pass 256MB 16-way 0 /N0/SB0/P2/B0 2 2 512MB pass 256MB 16-way 0 /N0/SB0/P2/B1 2 3 512MB pass 256MB 16-way 0 /N0/SB0/P3/B0 3 0 512MB pass 256MB 16-way 0 /N0/SB0/P3/B1 3 1 512MB pass 256MB 16-way 0 /N0/SB0/P3/B0 3 2 512MB pass 256MB 16-way 0 /N0/SB0/P3/B1 3 3 512MB pass 256MB 16-way 0 /N0/SB3/P0/B0 12 0 512MB pass 256MB 16-way 1 /N0/SB3/P0/B1 12 1 512MB pass 256MB 16-way 1 /N0/SB3/P0/B0 12 2 512MB pass 256MB 16-way 1 /N0/SB3/P0/B1 12 3 512MB pass 256MB 16-way 1 /N0/SB3/P1/B0 13 0 512MB pass 256MB 16-way 1 /N0/SB3/P1/B1 13 1 512MB pass 256MB 16-way 1 /N0/SB3/P1/B0 13 2 512MB pass 256MB 16-way 1 /N0/SB3/P1/B1 13 3 512MB pass 256MB 16-way 1 /N0/SB3/P2/B0 14 0 512MB pass 256MB 16-way 1 /N0/SB3/P2/B1 14 1 512MB pass 256MB 16-way 1 /N0/SB3/P2/B0 14 2 512MB pass 256MB 16-way 1 /N0/SB3/P2/B1 14 3 512MB pass 256MB 16-way 1 /N0/SB3/P3/B0 15 0 512MB pass 256MB 16-way 1 /N0/SB3/P3/B1 15 1 512MB pass 256MB 16-way 1 /N0/SB3/P3/B0 15 2 512MB pass 256MB 16-way 1 /N0/SB3/P3/B1 15 3 512MB pass 256MB 16-way 1 /N0/SB4/P2/B0 18 0 512MB pass 256MB 1-way 2 /N0/SB4/P3/B0 19 0 1024MB pass 512MB 1-way 3 6800c-sc:A> showcomp sb4 Component Status Pending POST Description --------- ------ ------- ---- ----------- /N0/SB4/P0 enabled - pass UltraSPARC-III, 750MHz, 8M ECache /N0/SB4/P1 enabled - pass UltraSPARC-III, 750MHz, 8M ECache /N0/SB4/P2 enabled - pass UltraSPARC-III, 750MHz, 8M ECache /N0/SB4/P3 enabled - pass UltraSPARC-III, 750MHz, 8M ECache /N0/SB4/P0/B0/L0 disabled - untest 1024M DRAM /N0/SB4/P0/B0/L2 disabled - untest 1024M DRAM /N0/SB4/P0/B1/L1 enabled - untest empty /N0/SB4/P0/B1/L3 enabled - untest empty /N0/SB4/P1/B0/L0 disabled - untest 1024M DRAM /N0/SB4/P1/B0/L2 disabled - untest 1024M DRAM /N0/SB4/P1/B1/L1 enabled - untest empty /N0/SB4/P1/B1/L3 enabled - untest empty /N0/SB4/P2/B0/L0 enabled - pass 512M DRAM /N0/SB4/P2/B0/L2 disabled - untest 512M DRAM /N0/SB4/P2/B1/L1 enabled - untest empty /N0/SB4/P2/B1/L3 enabled - untest empty /N0/SB4/P3/B0/L0 enabled - pass 1024M DRAM /N0/SB4/P3/B0/L2 disabled - untest 1024M DRAM /N0/SB4/P3/B1/L1 enabled - untest empty /N0/SB4/P3/B1/L3 enabled - untest empty Example 6 interleave-scope = within-board interleave-mode = fixed For SB4 we have two 4-way segments and one 2-way segment.
========================= Memory Configuration =============================== Logical Logical Logical Port Bank Bank Bank DIMM Interleave Interleave FRU Name ID Num Size Status Size Factor Segment ------------- ---- ---- ------ ------- ------ ---------- ---------- /N0/SB4/P0/B0 16 0 1024MB pass 512MB 4-way 0 /N0/SB4/P0/B0 16 2 1024MB pass 512MB 4-way 0 /N0/SB4/P1/B0 17 0 1024MB pass 512MB 4-way 0 /N0/SB4/P1/B0 17 2 1024MB pass 512MB 2-way 1 /N0/SB4/P2/B0 18 0 512MB pass 256MB 4-way 2 /N0/SB4/P2/B1 18 1 512MB pass 256MB 4-way 2 /N0/SB4/P2/B0 18 2 512MB pass 256MB 4-way 2 /N0/SB4/P2/B1 18 3 512MB pass 256MB 4-way 2 /N0/SB4/P3/B0 19 0 1024MB pass 512MB 4-way 0 /N0/SB4/P3/B0 19 2 1024MB pass 512MB 2-way 1 Example 7 interleave-scope = within-board interleave-mode = optimal This is a complicated interleave example, because "optimal" setting is mixing different DIMM sizes.
========================= Memory Configuration =============================== Logical Logical Logical Port Bank Bank Bank DIMM Interleave Interleave FRU Name ID Num Size Status Size Factor Segment ------------- ---- ---- ------ ------- ------ ---------- ---------- /N0/SB4/P0/B0 16 0 1024MB pass 512MB 8-way 0 /N0/SB4/P0/B0 16 2 1024MB pass 512MB 8-way 0 /N0/SB4/P1/B0 17 0 1024MB pass 512MB 8-way 0 /N0/SB4/P1/B0 17 2 1024MB pass 512MB 8-way 0 /N0/SB4/P2/B0 18 0 512MB pass 256MB 16-way 0 /N0/SB4/P2/B1 18 1 512MB pass 256MB 16-way 0 /N0/SB4/P2/B0 18 2 512MB pass 256MB 16-way 0 /N0/SB4/P2/B1 18 3 512MB pass 256MB 16-way 0 /N0/SB4/P3/B0 19 0 1024MB pass 512MB 8-way 0 /N0/SB4/P3/B0 19 2 1024MB pass 512MB 8-way 0 Component J-No. Size Reason --------- ----- ---- ------ SB0 - - No board power SB1 - - No board power SB2 - - No board power SB3 - - No board power /N0/SB4/P0/B0/D0 J13300 512 MB /N0/SB4/P0/B0/D1 J13400 512 MB /N0/SB4/P0/B0/D2 J13500 512 MB /N0/SB4/P0/B0/D3 J13600 512 MB /N0/SB4/P0/B1 - - DRAM DIMM Group 1 Empty /N0/SB4/P1/B0/D0 J14300 512 MB /N0/SB4/P1/B0/D1 J14400 512 MB /N0/SB4/P1/B0/D2 J14500 512 MB /N0/SB4/P1/B0/D3 J14600 512 MB /N0/SB4/P1/B1 - - DRAM DIMM Group 1 Empty /N0/SB4/P2/B0/D0 J15300 256 MB /N0/SB4/P2/B0/D1 J15400 256 MB /N0/SB4/P2/B0/D2 J15500 256 MB /N0/SB4/P2/B0/D3 J15600 256 MB /N0/SB4/P2/B1/D0 J15301 256 MB /N0/SB4/P2/B1/D1 J15401 256 MB /N0/SB4/P2/B1/D2 J15501 256 MB /N0/SB4/P2/B1/D3 J15601 256 MB /N0/SB4/P3/B0/D0 J16300 512 MB /N0/SB4/P3/B0/D1 J16400 512 MB /N0/SB4/P3/B0/D2 J16500 512 MB /N0/SB4/P3/B0/D3 J16600 512 MB /N0/SB4/P3/B1 - - DRAM DIMM Group 1 Empty 6800c-sc:A> showcomp sb4 Component Status Pending POST Description --------- ------ ------- ---- ----------- /N0/SB4/P0 enabled - pass UltraSPARC-III, 750MHz, 8M ECache /N0/SB4/P1 enabled - pass UltraSPARC-III, 750MHz, 8M ECache /N0/SB4/P2 enabled - pass UltraSPARC-III, 750MHz, 8M ECache /N0/SB4/P3 enabled - pass UltraSPARC-III, 750MHz, 8M ECache /N0/SB4/P0/B0/L0 enabled - pass 1024M DRAM /N0/SB4/P0/B0/L2 enabled - pass 1024M DRAM /N0/SB4/P0/B1/L1 enabled - untest empty /N0/SB4/P0/B1/L3 enabled - untest empty /N0/SB4/P1/B0/L0 enabled - pass 1024M DRAM /N0/SB4/P1/B0/L2 enabled - pass 1024M DRAM /N0/SB4/P1/B1/L1 enabled - untest empty /N0/SB4/P1/B1/L3 enabled - untest empty /N0/SB4/P2/B0/L0 enabled - pass 512M DRAM /N0/SB4/P2/B0/L2 enabled - pass 512M DRAM /N0/SB4/P2/B1/L1 enabled - pass 512M DRAM /N0/SB4/P2/B1/L3 enabled - pass 512M DRAM /N0/SB4/P3/B0/L0 enabled - pass 1024M DRAM /N0/SB4/P3/B0/L2 enabled - pass 1024M DRAM /N0/SB4/P3/B1/L1 enabled - untest empty /N0/SB4/P3/B1/L3 enabled - untest empty
The following doc can be used to understand DIMM sizes: DIMM Size Reference Sheet for Sun Fire Midrange and Highend Servers (Doc ID 1325039.1)
To discuss this information further with Oracle experts and industry peers, we encourage you to review, join or start a discussion in an appropriate
My Oracle Support Community - Oracle Sun Technologies Community. Attachments This solution has no attachment |
||||||||||||
|