Sun Microsystems, Inc.  Oracle System Handbook - ISO 7.0 May 2018 Internal/Partner Edition
   Home | Current Systems | Former STK Products | EOL Systems | Components | General Info | Search | Feedback

Asset ID: 1-79-2160395.1
Update Date:2017-10-11
Keywords:
Solution Type  Predictive Self-Healing Sure

Solution  2160395.1 :   SPARC M7 Series Servers : Mixing 32GB RDIMM and 32GB LRDIMM  

Related Items 
    

  • Oracle SuperCluster M7 Hardware
    •  
  • SPARC M7-16
    •  
  • SPARC M7-8
    •  
Related Categories 
    

  • PLA-Support>Sun Systems>SPARC>Enterprise>SN-SPARC: M7
    •  






In this Document


Purpose


Details


 DIMM configuration


 DIMM identification


 One DIMM replacement


 Two DIMMs replacement


References


Applies to:  

SPARC M7-16 - Version All Versions and later
SPARC M7-8 - Version All Versions and later
Oracle SuperCluster M7 Hardware - Version All Versions and later
Information in this document applies to any platform.



Purpose


 The SPARC M7 Servers support different types and sizes of DIMMs :


    

  • 16GB Registered DDR4-2133 DIMM
    • 

  • 32GB Load Reduced DDR4-2133 DIMM
    • 

  • 32GB Registered DDR4-2400 DIMM
    • 

  • 64GB Load Reduced DDR4-2400 DIMM
    • 



Because the SPARC M7 Servers support two types of 32GB DIMMs, this document provides information about mixing 32GB RDIMM and 32GB LRDIMM.


 


Details


DIMM configuration


For SPARC M7 Servers, the DIMM configuration guidelines are :


    

  • Use either all 16- or all 32-Gbyte DDR4 DIMMs,
    • 

  • Ensure that all DIMMs in a CMIOU are the same size and type,
    • 

  • If you are using DIMMs of different capacities in the same host, ensure that all DIMMs in a CMIOU have the same capacity,
    • 

  • Do not mix 32GB LRDIMMs (4R x 4Gb) and 32 GB RDIMMs (2R x 8Gb) in the same CMIOU.
    • 



This implies that the proper 32GB DIMM type must be used when:


    

  • upgrading the memory configuration from half to full population,
    • 

  • replacing a 32GB DIMM in an existing configuration.
    • 



When replacing LRDIMMs with RDIMMs or vice versa, replace all of the DIMMs in the CMIOU.


 


DIMM identification


For DIMM identification, refer to SPARC M7 Series Servers Service Manual - Servicing DIMMs.


See also, SPARC M7 Series Servers: How to identify DIMM type and size from ILOM (Doc ID 2077118.1).


 


One DIMM replacement


When replacing one 32GB DIMM with the incorrect DIMM type, no FMA fault will be reported and there will be no specific event logs entry.


This will not prevent the host to start but the following message will be reported during the host startup sequence :


    

  • Full population of DIMM - DIMM Sparing enabled
    • 



2016-07-13 14:20:47 0:00:0> NOTICE: Configuring Memory
2016-07-13 14:20:48 0:00:0> ERROR: /SYS/CMIOU0/CM/CMP/BOB30/CH0/DIMM: DIMM population chip symmetry rule violation. Not configured
...
WARNING: One or more resources have been disabled, please run 'show disabled' on the SP.


    

  • DIMM Sparing disabled (Full or Half population of DIMM)
    • 



2016-07-13 15:02:46 0:00:0> NOTICE: Configuring Memory
2016-07-13 15:02:47 0:00:0> ERROR: /SYS/CMIOU0/CM/CMP/BOB30/CH0/DIMM: DIMM population chip symmetry rule violation. Not configured
2016-07-13 15:02:47 0:00:0> WARNING: Running with a nonstandard DIMM configuration. Refer to service document for details.
...
WARNING: One or more resources have been disabled, please run 'show disabled' on the SP.


As a result, the DIMM is removed from the configuration.




-> start -script /SP/faultmgmt/shell/


faultmgmtsp> fmdump -eV


...


2016-07-13/15:45:51 ereport.hc.component_disabled@/SYS/CMIOU0/CM/CMP/BOB30/CH0/DIMM
reason = DIMM population chip symmetry rule violation
system_component_firmware_manufacturer = Oracle Corporation
system_component_firmware_versions = (ILOM)3.2.6.2.b,(POST)5.5.1,(OBP)4.40.1,(HV)1.17.1
system_component_firmware_releases = (ILOM)2016.05.20,(POST)2016.04.25,(OBP)2016.04.25,(HV)2016.04.25


 


-> show faulty
Target | Property | Value
----------------------------------------+-----------------------------------------------+--------------------------------------------------------------------


->


-> show disabled
Target | Property | Value
----------------------------------------+-----------------------------------------------+--------------------------------------------------------------------
/SYS/CMIOU0/CM/CMP/BOB30/CH0/DIMM | disable_reason | Configuration Rules


 


-> show -d properties /SYS/CMIOU0/CM/CMP/BOB30/CH0/DIMM
/SYS/CMIOU0/CM/CMP/BOB30/CH0/DIMM
Properties:
type = DIMM
requested_config_state = Enabled
current_config_state = Disabled
disable_reason = Configuration Rules
fru_name = 32768MB DDR4 SDRAM DIMM
fru_manufacturer = Samsung
fru_part_number = 07075400,M393A4K40BB1-CRC
fru_rev_level = 01
fru_serial_number = 00CE02161932591867
fault_state = OK
clear_fault_action = (none)


 


-> show -t /System/Memory/DIMMs/DIMM_12
Target | Property | Value
----------------------------------------+-----------------------------------------------+--------------------------------------------------------------------
/System/Memory/DIMMs/DIMM_12 | health | Disabled
/System/Memory/DIMMs/DIMM_12 | health_details | Disabled due to configuration rules
/System/Memory/DIMMs/DIMM_12 | requested_state | Enabled
/System/Memory/DIMMs/DIMM_12 | part_number | 07075400,M393A4K40BB1-CRC
/System/Memory/DIMMs/DIMM_12 | serial_number | 00CE02161932591867
/System/Memory/DIMMs/DIMM_12 | location | CMIOU0/CM/CMP/BOB30/CH0/DIMM (CPU Memory IO Unit 0 Memory Branch
| | 30 Memory Channel 0)
/System/Memory/DIMMs/DIMM_12 | manufacturer | Samsung
/System/Memory/DIMMs/DIMM_12 | memory_size | 32 GB




 


[(flash)root@m7-8-sin06-a-sp0:~]# capiasrtest ddb
/SYS/CMIOU0/CM/CMP/BOB30/CH0/DIMM State:0x02 STATE_DISABLED
/SYS/CMIOU0/CM/CMP/BOB30/CH0 State:0x08 CHILD_AFFECTED
/SYS/CMIOU0/CM/CMP/BOB30 State:0x08 CHILD_AFFECTED
/SYS/CMIOU0/CM/CMP State:0x08 CHILD_AFFECTED
/SYS/CMIOU0/CM State:0x08 CHILD_AFFECTED
/SYS/CMIOU0 State:0x08 CHILD_AFFECTED




The impact on the host depends on the DIMM Sparing status :


    

  • If DIMM Sparing is enabled, 

      

    • there will be no impact on the amount of memory available to the host,  
      • 

    • the system will be using 100% of the other DIMMs in the CMIOU.
      • 

    

    • 

  • If DIMM Sparing is disabled (only 8 DIMMs installed),

      

    • the 32GB will be missing for the host.
      • 

    

    • 



The impact on the host may also depends on the location of the DIMM being replaced. As a result of the incorrect DIMM being deconfigured, some other DIMMs may need to be deconfigured as well for the system to operate properly.


 


Two DIMMs replacement


Upon DIMM Sparing events in the same CMIOU, two DIMMs are called out for replacement by the Fault Manager (FMA). 


If two 32GB DIMMs are replaced with the incorrect type,  the impact on the deconfiguration will depend on the locations of the 2 DIMMs being replaced. 


    

  • In the following example,  DIMMs /SYS/CMIOU0/CM/CMP/BOB00/CH0/DIMM and /SYS/CMIOU0/CM/CMP/BOB30/CH0/DIMM are 32GB RDIMM while the other 14 DIMMs are 32GB LRDIMM. As a result, the type of the first DIMM is considered as the reference; the other DIMMs are deconfigured.
    • 



2016-07-14 16:01:52 0:00:0> ERROR: /SYS/CMIOU0/CM/CMP/BOB00/CH1/DIMM: DIMM population chip symmetry rule violation. Not configured
2016-07-14 16:01:52 0:00:0> ERROR: /SYS/CMIOU0/CM/CMP/BOB01/CH0/DIMM: DIMM population chip symmetry rule violation. Not configured
2016-07-14 16:01:52 0:00:0> ERROR: /SYS/CMIOU0/CM/CMP/BOB01/CH1/DIMM: DIMM population chip symmetry rule violation. Not configured
2016-07-14 16:01:52 0:00:0> ERROR: /SYS/CMIOU0/CM/CMP/BOB10/CH0/DIMM: DIMM population chip symmetry rule violation. Not configured
2016-07-14 16:01:52 0:00:0> ERROR: /SYS/CMIOU0/CM/CMP/BOB10/CH1/DIMM: DIMM population chip symmetry rule violation. Not configured
2016-07-14 16:01:53 0:00:0> ERROR: /SYS/CMIOU0/CM/CMP/BOB11/CH0/DIMM: DIMM population chip symmetry rule violation. Not configured 
2016-07-14 16:01:53 0:00:0> ERROR: /SYS/CMIOU0/CM/CMP/BOB11/CH1/DIMM: DIMM population chip symmetry rule violation. Not configured
2016-07-14 16:01:53 0:00:0> ERROR: /SYS/CMIOU0/CM/CMP/BOB20/CH0/DIMM: DIMM population chip symmetry rule violation. Not configured
2016-07-14 16:01:53 0:00:0> ERROR: /SYS/CMIOU0/CM/CMP/BOB20/CH1/DIMM: DIMM population chip symmetry rule violation. Not configured
2016-07-14 16:01:53 0:00:0> ERROR: /SYS/CMIOU0/CM/CMP/BOB21/CH0/DIMM: DIMM population chip symmetry rule violation. Not configured
2016-07-14 16:01:53 0:00:0> ERROR: /SYS/CMIOU0/CM/CMP/BOB21/CH1/DIMM: DIMM population chip symmetry rule violation. Not configured
2016-07-14 16:01:54 0:00:0> ERROR: /SYS/CMIOU0/CM/CMP/BOB30/CH1/DIMM: DIMM population chip symmetry rule violation. Not configured
2016-07-14 16:01:54 0:00:0> ERROR: /SYS/CMIOU0/CM/CMP/BOB31/CH0/DIMM: DIMM population chip symmetry rule violation. Not configured
2016-07-14 16:01:54 0:00:0> ERROR: /SYS/CMIOU0/CM/CMP/BOB31/CH1/DIMM: DIMM population chip symmetry rule violation. Not configured
2016-07-14 16:01:54 0:00:0> WARNING: Running with a nonstandard DIMM configuration. Refer to service document for details.
...
WARNING: One or more resources have been disabled, please run 'show disabled' on the SP.


-> show -t location part_number health
Target | Property | Value
---------------------------------------------+-----------------------------------------------------+-----------------------------------------------------------------------------
/System/Memory/DIMMs/DIMM_0 | health | OK
/System/Memory/DIMMs/DIMM_0 | part_number | 07075400,M393A4K40BB1-CRC
/System/Memory/DIMMs/DIMM_0 | location | CMIOU0/CM/CMP/BOB00/CH0/DIMM (CPU Memory IO Unit 0 Memory Branch 00 Memory
| | Channel 0)
/System/Memory/DIMMs/DIMM_1 | health | Disabled
/System/Memory/DIMMs/DIMM_1 | part_number | 07075400,M386A4G40DM0-CPB
/System/Memory/DIMMs/DIMM_1 | location | CMIOU0/CM/CMP/BOB00/CH1/DIMM (CPU Memory IO Unit 0 Memory Branch 00 Memory
| | Channel 1)
...
/System/Memory/DIMMs/DIMM_11 | health | Disabled
/System/Memory/DIMMs/DIMM_11 | part_number | 07075400,M386A4G40DM0-CPB
/System/Memory/DIMMs/DIMM_11 | location | CMIOU0/CM/CMP/BOB21/CH1/DIMM (CPU Memory IO Unit 0 Memory Branch 21 Memory
| | Channel 1)
/System/Memory/DIMMs/DIMM_12 | health | OK
/System/Memory/DIMMs/DIMM_12 | part_number | 07075400,M393A4K40BB1-CRC
/System/Memory/DIMMs/DIMM_12 | location | CMIOU0/CM/CMP/BOB30/CH0/DIMM (CPU Memory IO Unit 0 Memory Branch 30 Memory
| | Channel 0)
/System/Memory/DIMMs/DIMM_13 | health | Disabled
/System/Memory/DIMMs/DIMM_13 | part_number | 07075400,M386A4G40DM0-CPB
/System/Memory/DIMMs/DIMM_13 | location | CMIOU0/CM/CMP/BOB30/CH1/DIMM (CPU Memory IO Unit 0 Memory Branch 30 Memory
| | Channel 1)..


 


    

  • In the following example, the DIMMs /SYS/CMIOU0/CM/CMP/BOB30/CH0/DIMM and /SYS/CMIOU0/CM/CMP/BOB31/CH0/DIMM are 32GB RDIMM while the other 14 DIMMs are 32GB LRDIMM. As a result, the pair of DIMMs on the BOB are deconfigured.
    • 



 




2016-07-20 14:23:19 0:00:0> NOTICE: Configuring Memory
2016-07-20 14:23:20 0:00:0> ERROR: /SYS/CMIOU0/CM/CMP/BOB30/CH0/DIMM: DIMM population chip symmetry rule violation. Not configured
2016-07-20 14:23:20 0:00:0> ERROR: /SYS/CMIOU0/CM/CMP/BOB30/CH1/DIMM: DIMM population chip symmetry rule violation. Not configured
2016-07-20 14:23:21 0:00:0> ERROR: /SYS/CMIOU0/CM/CMP/BOB31/CH0/DIMM: DIMM population chip symmetry rule violation. Not configured
2016-07-20 14:23:21 0:00:0> ERROR: /SYS/CMIOU0/CM/CMP/BOB31/CH1/DIMM: DIMM population chip symmetry rule violation. Not configured
2016-07-20 14:23:21 0:00:0> WARNING: Running with a nonstandard DIMM configuration. Refer to service document for details. 


...


WARNING: One or more resources have been disabled, please run 'show disabled' on the SP.


-> show -t location part_number health
Target | Property | Value
---------------------------------------------+-----------------------------------------------------+-----------------------------------------------------------------------------


...
/System/Memory/DIMMs/DIMM_11 | health | OK
/System/Memory/DIMMs/DIMM_11 | part_number | 07075400,M386A4G40DM0-CPB
/System/Memory/DIMMs/DIMM_11 | location | CMIOU0/CM/CMP/BOB21/CH1/DIMM (CPU Memory IO Unit 0 Memory Branch 21 Memory
| | Channel 1)
/System/Memory/DIMMs/DIMM_12 | health | Disabled
/System/Memory/DIMMs/DIMM_12 | part_number | 07075400,M393A4K40BB1-CRC
/System/Memory/DIMMs/DIMM_12 | location | CMIOU0/CM/CMP/BOB30/CH0/DIMM (CPU Memory IO Unit 0 Memory Branch 30 Memory
| | Channel 0)
/System/Memory/DIMMs/DIMM_13 | health | Disabled
/System/Memory/DIMMs/DIMM_13 | part_number | 07075400,M386A4G40DM0-CPB
/System/Memory/DIMMs/DIMM_13 | location | CMIOU0/CM/CMP/BOB30/CH1/DIMM (CPU Memory IO Unit 0 Memory Branch 30 Memory
| | Channel 1)
/System/Memory/DIMMs/DIMM_14 | health | Disabled
/System/Memory/DIMMs/DIMM_14 | part_number | 07075400,M393A4K40BB1-CRC
/System/Memory/DIMMs/DIMM_14 | location | CMIOU0/CM/CMP/BOB31/CH0/DIMM (CPU Memory IO Unit 0 Memory Branch 31 Memory
| | Channel 0)
/System/Memory/DIMMs/DIMM_15 | health | Disabled
/System/Memory/DIMMs/DIMM_15 | part_number | 07075400,M386A4G40DM0-CPB
/System/Memory/DIMMs/DIMM_15 | location | CMIOU0/CM/CMP/BOB31/CH1/DIMM (CPU Memory IO Unit 0 Memory Branch 31 Memory
| | Channel 1)
/System/Memory/DIMMs/DIMM_16 | health | OK




-> show disabled
Target | Property | Value
---------------------------------------------+-----------------------------------------------------+-----------------------------------------------------------------------------
/SYS/CMIOU0/CM/CMP/BOB30/CH0/DIMM | disable_reason | Configuration Rules
/SYS/CMIOU0/CM/CMP/BOB30/CH1/DIMM | disable_reason | Configuration Rules
/SYS/CMIOU0/CM/CMP/BOB31/CH0/DIMM | disable_reason | Configuration Rules
/SYS/CMIOU0/CM/CMP/BOB31/CH1/DIMM | disable_reason | Configuration Rules


 






 




As a result of 23314637, only ereport.hc.component_disabled are reported. 


Starting from SysFW 9.7.3, with the fix for 23314637, ireport and alert (ireport.hc.dev_mismatch / alert.ilom.chassis.config.component.misconfigured) should be reported in the FMA Shell to report the DIMM configuration mismatch.




 


References :



 


References
<NOTE:1967858.1> - SPARC M7-16 Server : Product Information Page
<NOTE:2037793.1> - SPARC T7 / M7 Servers : DIMM sparing FAQ
<NOTE:1967511.1> - SPARC M7-8 Server : Product Information Page
<NOTE:2016870.1> - SPARC T7 / M7 Servers : DIMM sparing
<NOTE:2077118.1> - SPARC M7 Series Servers: How to identify DIMM type and size from ILOM



Attachments

This solution has no attachment


















       

Copyright © 2018 Oracle, Inc.  All rights reserved.

 Feedback