Oracle SuperCluster M7 Hardware
 
Solaris Operating System
 
Oracle SuperCluster T5-8 Half Rack
 
Oracle SuperCluster T5-8 Full Rack
 

PLA-Support>Eng Systems>Exadata/ODA/SSC>SPARC SuperCluster>DB: SuperCluster_EST
 

In this Document

Purpose

Scope

Details

Overview

Possible recovery scenarios:

Requirements

Known Issues and Workarounds

Recover Domains Using ZFS streams

References

Applies to:

Purpose

This article describes how to recover SuperCluster components that were backed up with the osc-config-backup tool. It also highlights critical issues and workarounds which SuperCluster administrators should be aware of before attempting a recovery scenario.

Scope

The recovery process is only currently supported on T5-8 SuperCluster and M7 SuperCluster. Detailed instructions and examples that you can use to recover SuperCluster components previously backed up by osc-config-backup can be found in the Oracle SuperCluster Configuration Backup Utility - Recovery Guide.

Please note that data storage on the Exadata Storage Servers is not covered by osc-config-backup, and this article does not cover storage server recovery. Use the Engineered Systems Backup Utility (ESBU) to back up and recover this data. Refer to the article titled RMAN Backup From SPARC SuperCluster to Sun ZFS Backup Appliance (Doc ID 1517107.1)

For further details and best practices, refer to the whitepaper titled Oracle Optimized Solution for Backup and Recovery of Oracle SuperCluster.

Details

Overview

When osc-config-backup is used to backup SuperCluster components prior to a failure, you can use the osc-config-backup data to recover SuperCluster components should the need arise. During the backup process, osc-config-backup copies critical component data to a central location on the internal ZFS Storage Appliance which in the event of an incident can be used to recover from a number of components.

Components that can be recovered using osc-config-backup data:

• Domains – Configuration, ZFS pools (rpool, bpool if present, and u01-pool for Database Domains)
• Solaris zones – Configuration, ZFS pools (rpool and u01-pool for Database Zones)
• IB switches – Configuration
• Ethernet management switch – Configuration
• ZFS storage appliance – Configuration
• SuperCluster – Configuration information (ssc-cu.zip file)
• Explorer – Data from each Database and Application Domain

Possible recovery scenarios:

These recovery scenarios cover some situations with which customers could potentially find themselves having to recover from. In most situations where a single component failure has occured in the rack, no loss of service should occur assuming the configuration has been implemended with maximum availability in mind. In some situations the loss of an entire SuperCluster platform might not have too much impact on a business as it's becoming increasing common for customer's to implement a secondary DR SuperCluster platform.

• Restoring an Oracle Solaris Zone
• Rolling Back an I/O Domain
• Restore a Domain Configuration
• Restore an IB Switch
• Restore the Ethernet Management Switch
• Restore an Oracle ILOM Configuration

For complex situations or if customers are unsure of the recovery situation then please raise a support ticket for further assistance.

Requirements

To perform the recovery procedures in this article, ensure that these requirements are met:

SuperCluster components must have been backed up with the osc-config-backup tool prior to the disaster. Refer to the article titled How To Back Up the Oracle SuperCluster Configuration with osc-config-backup (Doc ID 1934129.1).

• You must have access to all of the data that was collected by osc-config-backup. This includes Explorer data, OS images for domains and zones, IB switch and Ethernet management switch configuration data.
• The ZFS storage appliance contains the osc-config-backup data and must be functional, or you must have access to your ZFS storage appliance backups.
• All hardware components of the SuperCluster that you plan to recover must be functional with no faults.
• You must have access to SuperCluster's Explorer data that was collected prior to the disaster. You might need to access the Explorer data from your ZFS storage appliance backup.
• You must have a proxy system, such as a laptop, with network access to the SuperCluster ZFS storage appliance and to the node you are restoring. The proxy system must be able to ssh to SuperCluster.

Known Issues and Workarounds

If you have opted to not backup the SuperCluster platform using iscsi luns then the only available recovery process is to use the zfs stream dumps collected within the /osc-config-backup mountpoint. A domain can be fully recovered by both methods but the zfs stream recovery process is slightly more involved that following the documented iscsi recovery methodology.

Recover Domains Using ZFS streams

This additional step only applies to T5-8 SuperCluster platforms and you only need to perform this task if you were not able to restore the OS via an iSCSI backup or you opted not to take iSCSI backups in the first place. This proceedure assumes the following conditions:

• The ZFS root pool cannot be recovered.
• The ZFS root pool snapshots are stored on the SuperCluster internal ZFS storage appliance and have been shared via NFS over the management network. Refer to Where to download Oracle Solaris ISO images and Update Releases (Doc ID 1277964.1) for additional details.
• Primary domains don't have access to a local CDROM drive so the Oracle Solaris image downloaded earlier can be made available to the ilom. Refer to How to use NFS or SAMBA server as attached host storage device for ILOM remote KVMs (Doc ID 2084013.1) for further details on how to implement this.
• Guest domains can boot from a virtual disk service provided from the Primary domain. Refer to How to Install the Oracle Solaris OS on a Guest Domain From an Oracle Solaris ISO File  for further details.

1/ Boot the domain using a CDROM image into single user mode

NOTE: Username=root password=solaris

2/ Based on Explorer data, create the pools using the first two internal drives (HDD0 labeled as solaris, and HDD1). Run the format utility to identify the drive's cntndnsn numbers (such as c0t5000CCA02523F5F4d0), then quit the utility.

NOTE: if the physical drives have been replaced then please ensure the partition table matches the original drive. Also be aware that I/O domains on T5-8 SuperCluster only have a single disk supplied by the primary control domain from the ZFS-SA.

Run this command for Oracle Solaris 11 domains:

Run this command for Oracle Solaris 10 domains.

Verify the zpools:

Create any additional pools you have on this domain.

3/ Mount the ZFS backup directory on the guest domain.

Assign a temporary network address on the same management subnet of the ZFS storage appliance on the guest domain.

For example, if the ZFS storage appliance management IP address is 10.163.238.110, you can assign an IP address of 10.163.238.10 to the guest domain.

4/ Restore the rpool from the ZFS snapshots.

5/ Activate the BE and boot from the rpool disk.

Oracle Solaris 11 example:

The guest domain is up and running.

6/ Create swap and dump file systems.

Example:

The guest domain is now restored and operational.

References

This solution has no attachment