Sun SPARC Enterprise M8000 Server
 
Sun SPARC Enterprise M4000 Server
 
Sun SPARC Enterprise M3000 Server
 
Sun SPARC Enterprise M9000-32 Server
 
Sun SPARC Enterprise M9000-64 Server
 
Sun SPARC Enterprise M5000 Server
 

PLA-Support>Sun Systems>SPARC>Enterprise>SN-SPARC: Mx000
 
_Old GCS Categories>Sun Microsystems>Servers>OPL Servers
 

Applies to:

Goal

This document describes XSCF snapshot collection by running a domain Explorer. The Explorer module xscfextended is completely re-written, starting with Services Tool Bundle (STB) 7.3. The various ways to gather diagnostic data from either the XSCF or the domain and XSCF combined manually, or in an automated fashion using STB 7.2 or older, are described in "Gathering diagnostic data for SPARC Enterprise M3000/M4000/M5000/M8000/M9000 (OPL) Servers (Doc ID 1008229.1)". How to upload the gathered data to Oracle, is described in How to Upload Files to Oracle Support (Doc ID 1547088.2) .

This document applies to all Mx000 servers, and describes the usage of xscfextended module of Explorer 7.3 (as part of STB7.3) or newer, on both Solaris 10 and Solaris 11. Where needed, some Explorer basic facts are listed, to get a complete picture on how to use the reworked module. For reference purposes, we will list the available manual options for collecting the XSCF snapshot, too.

Solution

Manually running snapshot on SPARC Enterprise Mx000 (OPL) eXtended System Control Facility ( XSCF ).

Snapshot stores the collected data on a remote network host or on an external media device, based upon the use of the -T,  -t or -d option. When storing data on a remote network host, snapshot opens a network connection using SSH to act as a data pipe to the remote file.

Example of collecting a snapshot using the value for user@host:directory previously set using the setarchiving command:

Example of collecting a snapshot to host m9000-x-a.uk, as user joe, writing the snapshot file to directory /tmp/joe:

Example of collecting a snapshot to an external USB stick:

Bear in mind, that the external media device connected to the XSCF's USB port is expected to have a partition 1, formatted with the FAT32 filesystem. The external USB device can have multiple partitions, as long as partition 1 is FAT32. That partition will then be used by the snapshot command. For more details on the snapshot command please visit the manual page.

Running Explorer on  SPARC Enterprise Mx000 (OPL) Servers

Like on any other Oracle SPARC Server, you can run Explorer on the Solaris instance(s) on that server. When you run Explorer with the option -w xscfextended, it is capable of collecting a Snapshot on the specified XSCF(s). The Snapshot data will be in the xscf sub-directory of the host's Explorer. It does make sense when running an Explorer on a Mx000  Server domain, to run it with -w xscfextended, that way a single Explorer data file contains information about the domain, its main XSCF and its spare XSCF where applicable. The xscfextended module can be run manually, and in an automated version. This document will describe both options in detail. Be aware that to collect the XSCF snapshot via Explorer, the Explorer does not need to run on the platform containing that XSCF. As long as Explorer can reach the XSCF over the network, it is capable or collecting the snapshot data via the -w xscfextended option. For simplicity, this document will describe to run Explorer on an Mx000 server domain.

This document applies to Explorer version 7.3 or above for Mx000 machines. We strongly recommend to run the latest version of Explorer, which is currently 7.3.  Explorer may be downloaded from "Oracle Services Tools Bundle (STB) - RDA/Explorer, SNEEP, ACT (Doc ID 1153444.1)".

NOTE: By choosing the Mx000 Solaris domain as the host to run the Explorer command on,  it is assumed it is healthy enough to stay up long enough for the customer to run the Explorer and then get the Explorer output and send to Oracle for analysis. If the domain will not boot or stay up, consider using another machine as the Explorer host or simply gather just the XSCF snapshot manually as described in the first section of this document.

The new Explorer 7.3 xscfextended module explained

This paragraph will describe the new features that come with the Explorer 7.3 xscfextended module and how to properly set them up. You can run the xscfextended module in interactive mode and in non-interactive mode. Furthermore, there are now two methods of collecting one or more XSCF snapshots:

• LOG: this method relies on running the snapshot command on the XSCF using the -T option
• SNP: this method relies on running the snapshot command on the XSCF using the -t option

The differences between the two methods, and how to properly set them up, will be described further in How to setup the archival host.

After installing STB7.3,  you first need to run "explorer -g" to generated a defaults file that is usable by Explorer 7.3. Starting with Explorer 7.0 the package  is distributed both as a SVR4 package and as an IPS package. The standard location for the output and configuration files is as follows:

• Solaris10:
  • the default output files location is /opt/SUNWexplo/output
  • the default configuration files location is /etc/opt/SUNWexplo
• Solaris11:
  • the default output files location is /var/explorer/output
  • the default configuration files location is /etc/explorer
• throughout this document  ${XPL_ETC} will be used to indicate the location of configuration files
• throughout this document  ${XPL_OUT} will be used to indicate the location of output files

For more details on Explorer, please visit the explorer manual pages.

As always, on the Solaris instance where you want to collect an Explorer, you run the command explorer as user root. If you want the Explorer program to collect one of more XSCF snapshots, you specify the xscfextended module to be run. The host on which you run the Explorer program is called the Explorer host, in this document we will refer to it as the EXPLO_HOST. The xscfextended module does need to be told for which XSCF units to collect the snapshot. This can be done by creating the file xscfinput.txt in ${XPL_ETC), which only will be used in non-interactive mode. If running in interactive mode, the user will be prompted to enter the appropriate information. The xscfinput.txt layout and file attributes will be described in How to setup the Explorer host.

For each XSCF snapshot we tell the xscfextended module to collect, it will contact what we call the archival host; in this document we will refer to it as ARCH_HOST. To contact ARCH_HOST, we will be using the special user called archival user, to which we will refer in this document as ARCH_USER. In the home directory of the ARCH_USER, there must be a file called .explorer, which will contain information describing how to contact the XSCF, and what method to use, LOG or SNP. The content of the file .explorer will be described in How to setup the archival host. With the information found in the .explorer file, the Explorer program on EXPLO_HOST will connect to the XSCF, and start the snapshot command. The snapshot will be stored on ARCH_HOST, and after the complete snapshot file has arrived, the Explorer program will pull over the zip file to EXPLO_HOST.

This is in a nutshell how it works. Run explorer -w default,xscfextended on EXPLO_HOST. The Explorer program will login to ARCH_HOST as user ARCH_USER, for each specified XSCF. It will find all necessary details for each XSCF in the .explorer file, and start a snapshot collection accordingly. When the snapshot zip file has arrived on ARCH_HOST, EXPLO_HOST will pull that file over and put it in the ./xscf subdirectory of the Explorer output directory.

Please read on for a detailed description of the various bits and pieces that make this work. Although the procedure turns to be not too complex, at first the various variables, files, names and locations can easily make you lose track of how your setup looks. Please use the attached little variables used  table that is attached to this document, and fill in the values you are going to use, while going through the procedure.

How to setup the archival host

The archival host must be a host that can be reached over the network by both the EXPLO_HOST and the XSCF. On the ARCH_HOST, you must have user ARCH_USER with a home directory to hold one of more snapshots. The xscfextended module connects to ARCH_HOST through su or ssh, depending on the content of the xscfinput.txt file. The Explorer program uses user ARCH_USER, and will go and find information on how to contact the XSCF in the .explorer file. The ARCH_USER can only be configured with a standard shell , setting up the user with a restricted shell will cause the xscfextended module to fail. To allow for non-interactive mode to run without prompting for any passwords, the password of user ARCH_USER should be recorded in the xscfinput.txt file. Also, the password of user ARCH_USER is needed by the snapshot command, to be able to write back the snapshot file to ARCH_HOST.

The file xscfinput.txt will be further explained in How to setup the Explorer host .

User ARCH_USER on host ARCH_HOST should have a file ${HOME}/.explorer, and its permissions should be 0400 or 0600. The .explorer file will have one or more sections, one section for each XSCF we want to be able to collect a snapshot on. A typical section looks like this:

This is what the usage of each line is:

• [XSCF:m3000-x-m-xscf]:
  • this is the section header, it marks the start of a new section
  • the section header format is <LABEL>:<NAME>, the Explorer program will search for <NAME>, in this example m3000-x-m-xscf
  • the section ends at EOF or at the next section header
  • the section name must be a word which is composed of letters, digits, periods, minus-sign/dash/hyphen, and underscores
  • to make the section header self-explaning, it is best named [XSCF:<XSCF-HOSTNAME>] or [XSCF:<XSCF-IP_ADDRESS>]
• HST="m3000-x-m-xscf":
  • specifies the XSCF hostname or IP address
  • when <hostname>[.<domain>] is used, the XSCF must be able to resolve it
• USR="explo_dom0":
  • specifies the XSCF user, used by the explorer command on EXPLO_HOST, to login to the XSCF;
  • an XSCF user has limited space to store public SSH keys; to be able to setup each domain to automatically collect a snapshot of its XSCF, a per domain dedicated XSCF user is needed
  • to keep track which domain is collecting snapshots, it is recommended to create a dedicated XSCF user with a self-explaining name
  • the dedicated XSCF user must have fieldeng privileges to be able to run the snapshot command
• TYP="SNP"
  • "LOG":
    • when type is LOG, the XSCF SSH connection details, as setup with setarchiving, will be used to collect a snapshot
    • on the XSCF,  snapshot -T, will be run; details on using this method are described in detail in Doc ID 1008229.1
    • the LOG type is provided for backward compatability, Oracle recommends to use type SNP, this document will describe the use of the SNP method
  • "SNP":
    • when type is SNP, on the XSCF  snapshot -t will be run
    • the explorer program on EXPLO_HOST will provide all necessary parameters to write the snapshot to ARCH_HOST
    • Oracle recommends using type SNP moving forward.
• DIR="snapshots"
  • indicates the snapshot directory location
  • if the specified directory is not a full path name, it will be relative to user ARCH_USER's home directory;
  • its full path cannot contain single and double quote characters;
  • the specified  directory should be owned by user ARCH_USER and its permissions should be 0700;
• CLN="NONE"
  • specifies the cleanup strategy to apply to the destination directory specified by DIR;
  • "ALL": removes all files from destination directory after snapshot has been transferred to EXPLO_HOST;
  • "FILE": removes the snapshot file from destination directory after snapshot has been transferred to EXPLO_HOST
  • "NONE": disables any cleanup.
  • "PAT": removes all files from the destination directory, with a name matching the indicated pattern ( for example, CLN="PAT:*.zip" )
• KEY="m3000-x"
  • specifies an optional identification key, which can be used to indicate that multiple entries are using the same password
  • such a password would be stored in a specific type of record (type T) in the xscfinput.txt file

Finally, it must be understood the xscfextended module always works with an archival host. This section has described how to setup such an environment. The archival host environment is used regardless if the xscfextended module is run in interactive or non-interactive mode. You can only run explorer -w default,xscfextended in an automated fashion, if  the setup is done in a way that Explorer does not prompt for passwords.  Just wanted to make that very clear. Now on to running in interactive mode.

To check if you have properly setup the archival host, you should be able to:

• either su ${ARCH_USER} if you are already on ${ARCH_HOST}, or run ssh -l ${ARCH_USER} ${ARCH_HOST}, to get to the home directory of ${ARCH_USER} on ${ARCH_HOST}
• find user's ${ARCH_USER}  home directory to contain the .explorer file, with the permissions 0400 or 0600
• on XSCF ${HST}, logged in as user ${USR}, run snapshot -L F -t ${ARCH_USER}@${ARCH_HOST}:${DIR} to successfully collect the snapshot ( the password used in the process should go into the xscfinput.txt file )

How to run the Explorer 7.3 xscfextended module in interactive mode

To run the xscfextended module in interactive mode, first of all you need to run explorer -w default,xscfextended,interactive.Then, one or more of the following criteria must be met:

• ${XPL_ETC}/xscfinput.txt does not exist
• ${XPL_ETC}/xscfinput.txt does not have any active A lines (see How to setup the Explorer host)
• ${XPL_ETC}/xscfinput.txt does have file permissions other then 0400 and 0600

If you only are going to run in interactive mode, there is no need to setup the Explorer host. An interactive session of the module looks like this (non-relevant output omitted):

When running the xscfextended module in interactive mode, you are presented with a menu to enter one of more XSCFs from which you want to have a snapshot collected. To add an XSCF, type D and hit return:

• you need to fill in the section name, which will be looked for in the .explorer file (m3000-x-m)
• you need to fill in the ARCH_USER, which will be used for connecting to ARCH_HOST (rfxscf)
• you need to fill in the hostname or IP-address of ARCH_HOST (- for localhost)
• when you are done adding XSCF information, you choose R and hit return to start the snapshot collection

Please be aware of the effect of the value of ARCH_HOST. If you enter  "-",  the Explorer program will use the su command to become user ARCH_USER on your EXPLO_HOST. All other input, including "localhost" , will be considered an IP address or IP hostname to which the Explorer program will connect using ssh .

In the process, you will be asked to provide the ARCH_USER password for accessing ARCH_HOST, both for use by explorer on EXPLO_HOST and snapshot on the XSCF. In the above example, we collected a single snapshot for the XSCF as specified in the m3000-x-m-xscf section (see How to setup the archival host ).To automate all of the above, and to run in non-interactive mode, you first need to setup the Explorer host. Please read on.

How to setup the Explorer host

To run the xscfextended module in non-interactive mode, you need to setup the Explorer host. You setup an Explorer host by creating the input file xscfinput.txt in the ${XPL_ETC} directory, the file should be owned by user root and its permissions should be 0400 or 0600. A line in the xscfinput.txt file can be a type T, type X or type A:

• T: T <KEY> <XSCF password>, used by all sections in .explorer with same KEY, if XSCF password is needed, not in scope for this document
• X: X <HST> <USR> <XSCF password>, used by section in .explorer, if XSCF password is needed, not in scope for this document
• A: A <ARCH_HOST> <ARCH_USER> <section_header> <ARCH_USER password>, used to search section in .explorer, then connect to XSCF

A type A line describes how to connect to ARCH_HOST, and what section to use:

• in the example used in this section, ARCH_HOST is "-", ARCH_USER is rfxscf and section header is m3000-x-m-xscf
• the ARCH_USER password is used for both the connection from the EXPLO_HOST to ARCH_HOST and the connection between XSCF and ARCH_HOST
• if you specify "-" for ARCH_HOST,  the Explorer program will use the su command to become user ARCH_USER on your EXPLO_HOST, all other input, including "localhost" , will be considered an IP address or IP hostname to which the Explorer program will connect using ssh .
• the A line used in the examples in this doc contains "A - rfxscf m3000-x-m-xscf Gehe1mpje"

If you now start explorer -w default,xscfextended, the module will process the A lines in xscfinput.txt one by one:

• user root on EXPLO_HOST becomes user ARCH_USER through su
• it will look in .explorer for section <section_header>
• with the information found, user root on EXPLO_HOST will ssh into the XSCF, and start a snapshot collection to the specified location on ARCH_HOST
• depending on the value of  TYP in the used section, the snapshot is run with -t (SNP) or -T (LOG)
• when the snapshot collection has completed, it will be placed in a subdirectory of the Explorer output

We have not covered one thing. For using the non-interactive xscfextended module in an automated fashion, user root will need to be able to login to the XSCF without being prompted for a password. How to make that work, is described in How to run the Explorer 7.3 xscfextended module in non-interactive, automated mode.

To check if you have properly setup the explorer host, you should be able to:

• find directory ${XPL_ETC} to contain the file xscfinput.txt , owned by user root and having permissions 0400 or 0600
• either su ${ARCH_USER} if you are already on ${ARCH_HOST}, or run ssh -l ${ARCH_USER} ${ARCH_HOST}, to get to the home directory of ${ARCH_USER} on ${ARCH_HOST}, using the values from the A line in the xscfinput.txt file
• this home directory should contain the .explorer file, with the permissions 0400 or 0600

How to run the Explorer 7.3 xscfextended module in non-interactive, automated mode

In the previous section we learned how to setup EXPLO_HOST, and what happens if you run it in non-interactive mode. The missing step is to eliminate the XSCF asking for the password of  the user defined in the XSCF section (lines HST and USR, see How to setup archival host). To allow for non-interactive mode to run without prompting for the XSCF password, we need to ensure that user root on EXPLO_HOST is able to run ssh -l USR HST without being prompted for a password:

• on XSCF HST, add root's public SSH key from EXPLO_HOST to user USR's public ssh keys, by login on to the XSCF HST as user USR, and running the command setssh -c addpubkey
• these steps are described in detail in steps 3-6 of "Gathering diagnostic data for SPARC Enterprise M3000/M4000/M5000/M8000/M9000 (OPL) Servers (Doc ID 1008229.1)"

At this point, you will be able to run Explorer on M-series server domain, and in an automated fashion collect a snapshot of the XSCF(s) as part of the Explorer data, without the Explorer program prompting you for any information. You are now ready to run the Explorer program in an automated fashion, for example through cron .

To check if you have properly eliminated the XSCF asking for a password, you should be able to:

• as user root on EXPLO_HOST, run ssh -l ${USR}  ${HST}, without being prompted for a password; ${USR} and ${HST} are values from the .explorer file in the home directory of user ${ARCH_USER} on host ${ARCH_HOST}

Final observations

If you don't get this to work, make the Explorer module more verbose, by running explorer -verbose -w default,xscfextended. If that does not clear up any confusion, you can turn on "tracing" in two ways:

• in the file ${XPL_ETC}/rdainput.txt you can add the line RDA_XPLR_TRACE="1"
• before running the explorer command, run EXP_TRACE_RDA=1 ; export EXP_TRACE_RDA

This will cause a very,very verbose run, with a ton of output, but it will tell you exactly what happens. After activating tracing, you can run /opt/SUNWexplo/bin/explorer -verbose -w default,xscfextended 2>&1 | tee -a /tmp/log.txt  . All screen output will be in the specified output file, there might be a clue in the trace output just before any issue you experience.

What can go wrong ? Some example output of common mistakes (I'm sure there are more) that can happen, using the input files as listed at the end of this document ...

If user root on EXPLO_HOST needs a password to login to the XSCF as user USR, it will result in an explorer without a xscf directory:

If ARCH_USER is configured with a restricted shell, the XSCF snapshot cannot be received, but an explorer with xscf directory is created:

If ARCH_USER is configured with a restricted shell, section information cannot be found, and the explorer does not have the xscf directory:

This is how the output of a successful run looks:

What files are used in the examples in this document

On the EXPLO_HOST (m3000-x-m), we used the following xscfinput.txt file

On the ARCH_HOST (m3000-x-m), ARCH_USER (rfxscf) had the following .explorer file

explorer, xscfextended, OPL

This solution has no attachment