Sun Fire T1000 Server
 
Sun Netra T5440 Server
 
SPARC S7-2
 
Netra SPARC T5-1B Server Module
 
SPARC T4-2
 
SPARC T7-1
 
Sun SPARC Enterprise T5440 Server
 
Sun Blade T6300 Server Module
 
Sun Fire T2000 Server
 
SPARC T3-2
 
SPARC T3-4
 
Sun Netra T2000 Server
 
Sun SPARC Enterprise T5120 Server
 
SPARC T5-4
 
Solaris Operating System
 
SPARC T5-2
 
Sun SPARC Enterprise T5220 Server
 
SPARC T4-1
 
SPARC T7-2
 
SPARC T7-4
 
Sun SPARC Enterprise T1000 Server
 
Sun SPARC Enterprise T5240 Server
 
SPARC T3-1
 
SPARC T4-4
 
SPARC T4-1B
 
Sun Blade T6340 Server Module
 
SPARC T5-8
 
Sun Blade T6320 Server Module
 
Sun Netra T5220 Server
 
Sun SPARC Enterprise T5140 Server
 
SPARC T5-1B
 
Sun SPARC Enterprise T2000 Server
 
SPARC T3-1B
 

PLA-Support>Sun Systems>SAND>Operating System>SN-SND: Sun OS Virtualization LDOM
 
_Old GCS Categories>Sun Microsystems>Servers>Blade Servers
 
_Old GCS Categories>Sun Microsystems>Servers>CMT Servers
 
_Old GCS Categories>Sun Microsystems>Operating Systems>Solaris Operating System
 
_Old GCS Categories>Sun Microsystems>Servers>NEBS-Certified Servers
 

Applies to:

Goal

This document details how to backup and recover the Oracle VM for SPARC (aka LDom) environment following the replacement of the service processor or motherboard in a CMT capable system.

Depending on platform type the LDom configuration is stored within the Service Processor or Motherboard/Main Module - when these components are replaced the LDom configuration will be lost;

[T1000, T2000, Netra T2000]

Stored within the /persist filesystem residing on the Service Processor

[T5x20, T5x40, Sun Blade T63x0, Netra T5xx0]

Stored within Host Data Flash embedded on the Service Processor

[T3-1, T3-2, T3-1B, Netra T3-1, T4-1, T4-2, T4-1B, Netra T4-x, T5-2, T5-1B]

Stored within Host data Flash embedded on the System Board

[T3-4, T4-4, T5-4, T5-8 S7-2, S7-2L, T7-1, T7-2, T7-4]

Stored within Host Data Flash embedded on the Main Module (Motherboard).

This procedure explains the steps needed to recover the Logical Domains in a basic LDom configuration. It does not include procedures for recovery of Solaris ZFS or Veritas Storage management.

Sometimes it is not possible to save the LDom configuration prior to the System Board, Main Module or SP replacement. Refer to doc 1464421.1 for the steps on How-To restore LDom config after loss of (or not saved) configuration.

Please note:

Solution

The method used to backup and recover your Oracle VM for SPARC (aka LDom) configuration depends on which version of Oracle VM for SPARC you are running:

Oracle VM for SPARC 2.0 and later only:

The backup and recovery process for Oracle VM for SPARC 2.0 and later is documented in the Oracle VM for SPARC administration guide:

Oracle VM for SPARC 2.0 Administration Guide > Saving Domain Configurations for Future Rebuilding

Oracle VM for SPARC 2.1 Administration Guide > Saving Domain Configurations for Future Rebuilding

Oracle VM for SPARC 2.2 Administration Guide > Saving Domain Configurations for Future Rebuilding

Oracle VM for SPARC 3.0 Administration Guide > Saving Domain Configurations for Future Rebuilding

Oracle VM for SPARC 3.1 Administration Guide > Saving Domain Configurations

The procedure is the same for Oracle VM for SPARC 2.0, 2.1, 2.2, 3.0, 3.1, 3.2 and 3.3.  The links provided above are just for convenience.

Oracle VM for SPARC 1.3 and earlier only:

This section provides a basic example of recovering your Oracle VM for SPARC (aka LDom's) 1.3 and lower configuration as per the documentation at Logical Domains 1.3 Administration Guide > Saving Logical Domain Configurations for Future Rebuilding

1. Backup of the LDom environment before hardware replacement.

2. Identify the LDom names using the ldm list-domain command:

   Example:

   # ldm list-domain
   NAME         STATE    FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
   primary      active   -n-cv   SP      8     4G       0.1%  1d 4h 52m
   guest1       active   -----           8     4G            

3. Capture the LDom environment

4. Log in as an authorized user or superuser

5. Use each of the following methods and save the output to the root-disk of the primary domain to ensure the required information is captured

6. Run the command:# ldm ls -e -p > ldm_ls-e-p.out

7. Run the command:# ldm list-constraints -x <ldom> > <ldom>.xml for each of the ldom names identified earlier

8. Create an explorer output of the primary domain (use explorer rev 6.7 or higher) with the command:# /opt/SUNWexplo/bin/explorer -w default,ldom

9. If this is not available then the system configuration can be obtained from the file /var/opt/SUNWldm/ldom-db.xml

   *See the Appendix to this document for examples of the outputs from these commands. These examples are used in this procedure.

10. Shutdown all the domains and power off the system

11. Replace the motherboard or SC following the procedures shown in the appropriate service guide. See http://www.oracle.com/technetwork/documentation/sparc-tseries-servers-252697.html and http://docs.oracle.com/cd/E19076-01/index.html

12. Boot the operating system.

13. The system will display an errors similar to the following:

    Oct 14 12:15:18 svc.startd[7]: svc:/network/physical:default: Method "/lib/svc/method/net-physical" failed with exit status 96.
    Oct 14 12:15:18 svc.startd[7]: network/physical:default misconfigured: transitioned to maintenance (see 'svcs -xv' for details)
    Oct 14 12:15:34 svc.startd[7]: svc:/ldoms/vntsd:default: Method "/lib/svc/method/svc-vntsd" failed with exit status 1.
    Oct 14 12:15:34 svc.startd[7]: svc:/ldoms/vntsd:default: Method "/lib/svc/method/svc-vntsd" failed with exit status 1.
    Oct 14 12:15:35 svc.startd[7]: svc:/ldoms/vntsd:default: Method "/lib/svc/method/svc-vntsd" failed with exit status 1.
    Oct 14 12:15:35 svc.startd[7]: ldoms/vntsd:default failed: transitioned to maintenance (see 'svcs -xv' for details)

    It will still be possible to login to the system, however there will be no network access.

14. Login to the system as root

15. Move the virtual network interface from /etc/hostname.vsw0 to the physical network interface. In this example the physical interface is e1000g0

    # mv /etc/hostname.vsw0 /etc/hostname.e1000g0

16. Configure the physical network interface

    # ifconfig e1000g0 plumb
    # ifconfig e1000g0 ip_of_e1000g0 netmask netmask_of_e1000g0 broadcast + up

17. Reboot the system and then login again.

18. Verify that the firmware on the system is on the same level or higher. Please check this URL to get information about system firmware: http://www.oracle.com/technetwork/systems/patches/firmware/release-history-jsp-138416.html

    In some instances the previous svc-vntsd failure messages may still be displayed. However, it will still be possible to login.

19. Manually re-create the Primary Domain.

    The following examples of the commands use the parameters from the example files in the Appendix. Substitute the true values from the saved files.

    1. Recreate the Virtual Disk Server (vds)

       <ldom>
       <ldom_name>primary</ldom_name>
       -
       -
       -
       <vds>
       <vds_instance>
       <service_name>primary-vds0</service_name>
       <vds_volumes>
       <vds_volume>
       
       # ldm add-vds primary-vds0 primary

    2. Re-create the Virtual Console Concentrator service (vcc)

       <vcc>
       <vcc_instance>
       <service_name>primary-vcc0</service_name>
       <min_port>5000</min_port>
       <max_port>5100</max_port>
       </vcc_instance>
       </vcc>
       
       # ldm add-vcc port-range=5000-5100 primary-vcc0 primary

    3. Re-create the cryptographic resource

       <mau>
       <number>1</number>
       </mau>
       
       # ldm set-mau 1 primary

    4. Re-create the Virtual CPUs

       <cpu>
       <number>8</number>
       </cpu>
       
       # ldm set-vcpu 8 primary

    5. Re-create the memory

       <memory>
       <size>4294967296</size>
       </memory>
       
       # ldm set-memory 4G primary

    6. Save the configuration to the service controller

       # ldm add-config initial

    7. List the saved configs

       # ldm list-config
       
       factory-default [current]
       initial [next]

    8. Perform a re-configuration boot of the system

       # touch /reconfigure
       # shutdown -y -g0 -i6

20. Power off and then power on the platform and allow the system to boot

21. Restore the virtual network

    # ifconfig -a
    # ifconfig vsw0 plumb
    # ifconfig e1000g0 down unplumb
    # ifconfig vsw0 <ip_of_e1000g0> netmask <netamsk_of_e1000g0> broadcast + up
    # mv /etc/hostname.e1000g0 /etc/hostname.vsw0

22. Check the virtual network terminal server daemon is enabled.

    # svcs -l vntsd
    
    fmri svc:/ldoms/vntsd:default
    name virtual network terminal server
    enabled true
    state online
    next_state none
    state_time Wed Oct 15 16:03:54 2008
    logfile /var/svc/log/ldoms-vntsd:default.log
    restarter svc:/system/svc/restarter:default
    contract_id 57
    dependency optional_all/error svc:/milestone/network (online)
    dependency optional_all/none svc:/system/system-log (online)

    If this is not enabled use the following command

    # svcadm enable vntsd

23. Restore the virtual disk server to the guest domains

    <vds>
    <vds_instance>
    <service_name>primary-vds0</service_name>
    <vds_volumes>
    <vds_volume>
    <block_dev>/dev/dsk/c1t1d0s2</block_dev>
    <vol_name>vol1</vol_name>
    <vol_opts/>
    </vds_volume>
    </vds_volumes>
    </vds_instance>
    </vds>
    
    # ldm add-vdsdev /dev/dsk/c1t1d0s2 vol1@primary-vds0

24. Run the following command for each guest domain’s XML file you created in Step 7.

    # ldm add-domain -i ldom.xml

25. Finally Bind and Start the remaining domains.

    # ldm bind-domain guest1
    # ldm start-domain guest1

Appendix

Output from command:ldm ls -e -p

Output from ldm list-constraints command for control/primary domain

Contents of the file /var/opt/SUNWldm/ldom-db.xml

Internal Comments
comments

This document contains normalized content and is managed by the the Domain Lead(s) of the respective domains. To notify content owners of a knowledge gap contained in this document, and/or prior to updating this document, please contact the domain engineers that are managing this document via the "Document Feedback" alias(es) listed below:

Domain Lead: dencho.kojucharov@sun.com

LDom, ldom, CMT, recovery, "logical domain"

References

This solution has no attachment