Oracle Exadata Hardware
 
Exadata Database Machine X2-8
 

PLA-Support>Eng Systems>Exadata/ODA/SSC>Oracle Exadata>DB: Exadata_EST
 

Applies to:

Purpose

Document written to identify the system parameters set in the /etc/sysctl.conf of the Exadata DB Compute node which are candidates for adjustment when physical memory (RAM) is either added or removed.

This note is not relevant for Storage Cells

1. Please refer to the Exadata Best Practice recommendations listed in the following note: Oracle Exadata Best Practices (Doc ID 757552.1)
2. To ensure best practices from are being referenced, users should always run the most current version of Exachk, and compare results with values in current /etc/sysctl.conf.  
   1. Any values below Exachk identified minimums are candidates for increases.
   2. The current version of Exachk can be downloaded from <Note:1070954.1>  Oracle Exadata Database Machine exachk or HealthCheck
3. If there are Duplicate values, the last value is the one used.  Make sure that the un-needed duplicates are commented out (do not remove)
   
   1. Comment lines begin with a hash -- #
4. DO NOT use inline comments in the /etc/sysctl.conf as they cause misreads in the file.
   1. Example:  kernel.shmmax = 4294967295                     # Shared Memory Maximum          <<< This will cause read errors.
5. The current version of Exachk looks for this and validates /etc/sysctl.conf parameters based on  <Note: 1274318.1> - Oracle Sun Database Machine Setup/Configuration Best Practice

Details

 Kernel / VM Parameters that may require re-adjustment if RAM is added or removed, use of hugepages, or changes made in the SGA Size.

• kernel.shmmax                                                                        Modify to no more than 85% of new physical RAM and not less than largest SGA
• kernel.shmmal                                                                         Modify to RAM / Pagesize (4K or if hugepage 2M)
• vm.nr_hugepages                                                                     Modify based on calculated value see Document 401749.1 to calculate the recommended value for the vm.nr_hugepages kernel parameter

Commonly Tuned Parameters For Exadata.

Shared Memory Segments

                                  Oracle Default                          Exadata Default

kernel.shmmax         4GB                                         85% of RAM (Exadata Recommended)

• The maximum size of a shared memory segment and is limited by the size of the available user address space. On 64-bit systems, this is a theoretical 2^64bytes.  The "theoretical limit" for SHMMAX is the amount of physical RAM that you have.  A more realistic "physical limit" for SHMMAX would probably be "physical RAM - 2Gb".
• Setting the SHMMAX to the "physical limit" leaves inadequate system memory for the operating system and other necessary functions. Normally, Oracle Support officially recommends a "maximum" for SHMMAX of "1/2 of physical RAM" or "maximum" for SHMMAX of just less than 4Gb, or 4294967295.
• Exadata systems are configured to use a value of SHMMAX much higher than the standard support recommended value.
  • On Engineered systems, the recommendation is up to 85% of physical RAM.
• On systems with a large amount of memory, Oracle customers may chose a higher fraction, at their discretion.
• SHMMAX is the Maximum size of a memory segment.  If the actual call for a memory segment is less than the set maximum, only what is called for will be allocated.
• Ideally, the SGA of a database instance will fit in one single shared memory segment at startup by having SGA less than the SHMMAX.
• Multiple database instances can not share a single shared memory segment.
• Sometimes DBA's are confused in thinking that that setting the SHMMAX limits the total SGA size, this is false. 
  
  • Setting the SHMMAX lower than the requested SGA will just cause additional memory segments to be allocated to a database instance to achieve the SGA requested.
• See Note 567506.1 for more information.

kernel.shmall           2097152                                 2097152  (RAM / Pagesize (4096))

• Value is based on default level of system RAM
• Can be recalculated if system RAM is changed.
• The total amount of shared memory (in pages) that can be used at one time on the system.
• The use of hugepages does not effect this value.
• See Note 301830.1 for more information.

Virtual Memory

vm.nr_hugepages                                                     Calculated based on RAM and Combined size of all SGA’s

• In an Exadata environment it is always recommended to use hugepages.
• If you are using hugepages and change the size of any SGA’s, you need to re-calculate this value. 
• If you add or remove RAM and use hugepages, after re-adjusting the SGA’s of the instances, you will need to re-calculate this value.
• For information see Document 401749.1 to calculate the recommended value for the vm.nr_hugepages kernel parameter

*** Exachk already includes this check. See section in our BP Note: Validate key sysctl.conf parameters on database servers on    ->  Oracle Sun Database Machine Setup/Configuration Best Practices (Doc ID 1274318.1)

References

This solution has no attachment