How to Replace an Oracle Exalogic X2-2, X3-2, X4-2, X5-2 or X6-2 Motherboard in a Compute Node

Asset ID:	1-71-1496559.1
Update Date:	2018-04-16
Keywords:

Solution Type Technical Instruction Sure

Solution 1496559.1 : How to Replace an Oracle Exalogic X2-2, X3-2, X4-2, X5-2 or X6-2 Motherboard in a Compute Node

Applies to:

Oracle Exalogic Elastic Cloud X2-2 Half Rack - Version X2 and later
Exalogic Elastic Cloud X4-2 Hardware - Version X4 and later
Exalogic Elastic Cloud X3-2 Full Rack - Version X3 and later
Exalogic Elastic Cloud X4-2 Eighth Rack - Version X4 and later
Exalogic Elastic Cloud X4-2 Full Rack - Version X4 and later
Information in this document applies to any platform.
It is extremely important for the Field Engineer to verify which Operating System is installed on the Exalogic Machine prior to replacing the motherboard.
The Three possibilities are: Solaris Physical, Oracle Linux Physical or Oracle Virtual Server.
Special instructions for Physical and Virtual installations are in the attached Motherboard Replacement Procedure.

Goal

Provide specific additional steps required to successfully replace a Motherboard on a compute node server that is part of an Exalogic Elastic Cloud.

Purpose

This note describes what steps to take when an Exalogic Elastic Cloud compute node motherboard replacement is done.

Scope

The note applies to Compute Nodes on the Exalogic Elastic Cloud X2-2 (Sun Fire X4170M2), X3-2 (Sun Server X3-2), X4-2 (Sun Server x4-2) and X5-2 (Oracle Server X5-2)

If you are looking for the NON-Engineered Systems motherboard replacement procedures, see the following documents:

How to replace a Sun Server X4170M2 Motherboard assembly (Doc ID 1347513.1)

How to Replace a Sun Server X3-2 (X4170M3) Motherboard assembly (Doc ID 1495251.1)

How to replace a Sun Server X4-2 Motherboard assembly (Doc ID 1592250.1)

How to Replace an Oracle Server X5-2 Motherboard Assembly [VCAP] (Doc ID 1992420.1)

How to Replace an Oracle Exalogic X2-2, X3-2, X4-2, X5-2 or X6-2 Motherboard in a Compute Node(Doc ID 1496559.1)

Solution

DISPATCH INSTRUCTIONS
- WHAT SKILLS DOES THE ENGINEER NEED: The Field Engineer must be Exalogic Elastic Cloud trained.
- TIME ESTIMATE: 120 Minutes
- TASK COMPLEXITY: 4

FIELD ENGINEER INSTRUCTIONS:
- PROBLEM OVERVIEW

This details the steps required to successfully replace a Motherboard in a compute node server that is part of an Exalogic Elastic Cloud x2-2, X3-2 or X4-2.

- WHAT STATE SHOULD THE SYSTEM BE IN TO BE READY TO PERFORM THE RESOLUTION ACTIVITY?

The compute node needs to be off-lined by the customer's system administrator if it is still operational.

If the system is not down already due to whatever problem is causing the motherboard to be replaced, the system administrator should prepare the system for service by performing any application related functions required to shutdown the compute node.

This might include but is not limited to performing a system backup, fail-over, of applications or services, and finally a system shutdown.

If this is an OVS Virtual installation please confirm the system administrator has put the compute node in maintenance mode from OVMM/EMOC.

If assistance is needed with this procedure have the system administrator contact the EEST engineer that owns the parent service request for details.

- WHAT ACTION DOES THE ENGINEER NEED TO TAKE:

The general steps are:
1. Complete the pre-install steps to backup ILOM and gather serial numbers.

2. Physically replace the motherboard.

3. Complete the post-install steps to reconfigure the new motherboard to be part of the Exalogic Elastic Cloud

The following steps are described in the Motherboard Replacement Procedure.

Backup the current SP configuration
Update ILOM firmware
Update Serial Numbers
Update BIOS settings for OVS Virtual Installs
Network settings and timezone validation
Verify Network Interface has correct MAC address
1. Known Issue: Network Interfaces Not Operational After Motherboard Replacement on Exalogic Nodes : (Doc ID 1569243.1)

Details

The details have been attached to this document due to the lengthy procedure and special instructions per Operating System and Hardware platform.

The Field Engineer must download the attached Motherboard Replacement Procedure document in order to replace the motherboard properly.

Please read through this document carefully and follow the steps in each section relevant to the Compute node being serviced.

Special Instructions for Exalogic OVS:

If the customer has Oracle Virtual Server installed, there are documented BIOS changes required when replacing the motherboard.

The following Table lists the BIOS changes that must be made per platform type.

The following table lists the BIOS changes that must be made on the compute nodes after motherboard replacement.

BIOS Change	X2-2	X3-2	X4-2
Disable C-State	Required	Required	Not required
Enable SR-IOV	Required	Required	Not required
Set the maximum PCI payload to 256 bytes	Required	Not required	Not required
Disable PCI 64-bit resource allocation	Not required	Not required	Required

These are also referenced in the attached Motherboard Replacement Procedure document.

Also document: Updating the BIOS Settings for X2-2, X3-2, X4-2, and X5-2 Compute nodes before installing EECS on Exalogic (Doc ID 1608959.1)

<span style="font-size: medium;" size="3" data-mce-style="font-size: medium;"><span class="awizStartDelimiter">Example of BIOS changes for Exalogic X2-2 OVS:
</span></span>

<strong>Login to the compute node ILOM</strong>

```
set /HOST boot_device=bios
```
```
start /SYS
```
```
start /SP/console
```
```
Wait for menu
```
```
Advanced tab
```

<strong>Make sure that the PCI Payload is set to 256.&nbsp; Follow the steps below:</strong>

```
PCI Express Configuration
```
```
Change Maximum Payload Size to '256'
```

<strong>Make sure that the CPU C-State is DISABLED. Follow the steps below:</strong>

```
CPU Configuration
```
```
scroll to bottom
```
```
confirm DISABLED
```

<strong>Enable SRIOV:&nbsp; To enable SRIOV do the following:</strong>

```
Select I/O Virtualization and enable it
```
```
Save Changes and Exit
```

References: Refer to these guides for BIOS menu information :

x2-2: http://docs.oracle.com/cd/E19762-01/E22369-02/preparing.html#50581549_pgfId-1288223
x3-2: http://docs.oracle.com/cd/E22368_01/html/E27242/z400012a1446073.html
x4-2 : http://docs.oracle.com/cd/E36975_01/html/E38045/z400012a165586.html#scrolltoc
Ilom Network Sideband configuration reference: Ilom Sideband Management configuration
Network Interfaces Not Operational After Motherboard Replacement on Exalogic Nodes : (Doc ID 1569243.1)
Cannot Create vserver in a Compute Node after Motherboard Replacement in that Compute Node (Doc ID 2168395.1)

Other helpful reference links:

OBTAIN CUSTOMER ACCEPTANCE
- WHAT ACTION DOES THE CUSTOMER NEED TO TAKE TO RETURN THE SYSTEM TO AN OPERATIONAL STATE:

1. Verify the system can boot up into the OS.
2. Ensure the customer's system administrator has successfully been able to access the compute node. The system administrator should check all services are up and also if this was an OVS Virtual install, they will need to verify the Virtual Machines are able to come up properly. If the system administrator requires assistance beyond this, then you should direct them to callback the parent SR owner in EEST for assistance.

References

<NOTE:1608959.1> - Updating the BIOS Settings for X2-2, X3-2, X4-2, X5-2 and X6-2 Compute nodes before installing EECS on Exalogic
<NOTE:1569243.1> - Network Interfaces Not Operational After Motherboard Replacement on Exalogic Nodes
<NOTE:2168395.1> - Exalogic Virtual: Cannot Create vServer in a Compute Node after Motherboard Replacement in that Compute Node

Attachments

This solution has no attachment