Pillar Axiom 600 Storage System
 
Pillar Axiom 500 Storage System
 

PLA-Support>Sun Systems>DISK>Axiom>SN-DK: Ax600
 

In this Document

Goal

Solution

References

Applies to:

Goal

The intent of this document is to provide the instructions to run the internal Private Interconnect Fabric diagnostic utility (PITMan) on a field system as part of:

• An installation,
• Significant hardware reconfiguration,
• Or health check.

This diagnostic is superior to PI Stats because traffic is sent on all paths simultaneously at levels higher than can typically be achieved by the host based generated traffic used with PI Stats. The Axiom can be online and does not interfere with data traffic during the run of this diagnostic utility. The diagnostic does not use user file systems or LUNs to function, so it has no effect on user data.

The commands can be run in either:

• Via the GUI.
• Via the CLI.
• Via the getPITMAN utility.

If you want to understand how to interpret the data collection, please refer to <Document 1906880.1> Pillar Axiom: How to Interpret a Pitman Output on AxiomOne R5.X Systems.

Solution

Summary

Overview

This procedure invokes commands that are part of the Axiom software to generate traffic on the Private Interconnect Fibre channel fabric connections. This includes the connections on the slammer control unit PIM to the buddy control unit PIM and all the bricks that are attached. Pitman diagnostics are to be executed as part of a post installation validation, as part of a post maintenance validation when significant changes are made to the PI network, or as part of a system health check.

• • The initial run of the commands should run for 5 minutes.  This will identify any immediate PI issues.  If the test is free of errors, then proceed with running a 15 minute test.
  • If the initial 5 minute run produces errors, reseat the PI cables identified in the path producing errors and run another 5 minute test. If that test runs clean, a full 30 minute test needs to run with no errors reported.
  • If the second 5 minute test fails after reseating the cables in the chain, contact the Support Center for assistance in identifying hardware replacement. These will usually be Cables, RAID Controllers, or Private Interconnect Modules. You may be asked to disconnect a given subset of the hardware to isolate difficult problems.
  • After replacing any hardware, run another 5 minute test. If that test fails, contact the Support Center for assistance.  If the 5 minute test succeeds, run a 30 minute test which must complete without errors.

Pitman Diagnostic Options

These may be used in any combination. Some combinations are highly disruptive to host traffic. As of Patch 05.04.05, the diagstate option is inhibited on Axioms with more than two Slammers.

• • maxtraffic=on    Disruptive                Minimum 5.2.4, 5.3.6 or higher recommended.  Increases traffic to levels which will disrupt any host traffic.
  • diagstate=on     Disruptive                 Minimum 5.3.10.  Places the PIM FC switches in hub mode.  Also known as PortBlaster Mode.  All hosts should be turned off before using this mode.
  • pattern=on        Non-disruptive          Minimum 5.4.6.  Transmits rotating worst case patterns to isolate pattern sensitive components.
    
    

Special Handling Instructions for Disruptive Pitman, using the maxtraffic and diagstate parameters

The maxtraffic mode requires Axiom Release 05.02.04 and higher. The maxtraffic mode sends traffic to a single destination port at a time, but changes that port every two seconds.
The diagstate mode is available in Axiom Release 05.03.10 and higher. The diagstate mode puts the PIM FC switches in hub mode to produce more internal traffic.

PITMan Diagnostics Testing Runs in the GUI

1. Open the Axiom Storage Service Manager to view the GUI of the Axiom. The hardware should be green and normal before proceeding.
2. Select the Support Tab.
3. Select System Trouble in the list.
4. Right mouse click in the right hand window and choose Run Pitman Diagnostics. Additionally this can be invoked from the Action menu.
5. In the command parameter window type SetAutoModeOff and click Execute. This sets the operational state to manual mode.
6. Next type:
   
   
   • For non-disruptive:
     
     TrafficGenOn mode=auto peer=all pattern=on and click Execute.
     
     This will start the traffic generator in auto mode. This mode automatically selects the initiator and target ports to equally distribute the generated traffic through a particular FC network.
     The 5.4.x option pattern=on sends worst case rotating patterns, but is non-disruptive to user data. The Axiom must be in 5.4.6 or higher to use this option
     
     
   • For disruptive:
     
     TrafficGenOn mode=auto peer=all maxtraffic=on diagstate=on pattern=on and click Execute.
     
     maxtraffic mode requires 05.03.06, diagstate mode requires 05.04.06 or higher, and pattern mode requires 5.4.0 or higher.
     This will start the traffic generator in auto mode. This mode automatically selects the initiator and target ports to equally distribute the generated traffic through a particular FC network.
     The maxtraffic mode removes the limits on data traffic.
     The diagstate mode puts the FCIM switches in hub mode. Either maxtraffic or diagstate are disruptive to host traffic.
     Pattern mode sends rotating bit patterns to isolate pattern-sensitive components.
     
     
7. Start the statistics monitor by typing StartRecordingStats interval=x duration=x.  The interval and duration can be changed for each test being run.  The time is listed in seconds.
   1. For the 5, 15, and 30 minute test the commands would look like the following. The interval timing represents how often the system records the system stats for that time frame.
      
      
      StartRecordingStats interval=60 duration=300
      StartRecordingStats interval=180 duration=900
      StartRecordingStats interval=360 duration=1800
      
      
      
8. Once the time has expired on each test, run StopRecordingStats.
9. Then it's time to collect the data. Type GetMarksDb RecordCount=all and execute.  It will work for a few moments and put the output in the window. Then select all the text and copy/paste to a text editor where it can be saved, zipped, and emailed to Oracle Support for evaluation.
10. Once the file is safely captured run TrafficGenOff to stop the extra traffic on the PI network.
11. Repeat the steps 7-9 if additional tests are needed.
12. Please refer to the <Document 1547088.2> How to Upload Files to Oracle Support to transfer the results to the Oracle support for an existing SR.

PITMan Diagnostics Testing Runs in the CLI

Download the CLI from the Axiom GUI in the Support tab Utilities or from the http page of the Axiom IP address. Once installed here are the commands to run to obtain the same information as above.

1. Set the operation state to manual.
   
   
   pcli.exe submit -H <AxiomIP> PerformPitmanCommand CommandParameter="setAutoModeOff"
   
   
   
2. Start the traffic generator in auto mode.
   
   
   • For non-disruptive:
     
     
     pcli.exe submit -H <AxiomIP> PerformPitmanCommand CommandParameter="TrafficGenOn mode=auto peer=all pattern=on"
     
     
     
   • For disruptive:
     
     
     pcli.exe submit -H <AxiomIP> PerformPitmanCommand CommandParameter="TrafficGenOn mode=auto peer=all pattern=on maxtraffic=on diagstate=on"
     
     
     maxtraffic mode requires 05.03.06, diagstate mode requires 05.04.06 or higher, and pattern mode requires 5.4.0 or higher.
     This mode automatically selects the initiator and target ports to equally distribute the generated traffic through a particular FC network.
     The maxtraffic mode removes the limits on data traffic.
     The diagstate mode puts the FCIM switches in hub mode. Either maxtraffic or diagstate are disruptive to host traffic.
     Pattern mode sends rotating bit patterns to isolate pattern-sensitive components.
     
     
3. Start the statistics monitor. The interval and duration can be changed for each test being run. The time is listed in seconds.
   
   
   pcli.exe submit -H <AxiomIP> PerformPitmanCommand CommandParameter="StartRecordingStats interval=x duration=x"
   
   
   
   1. For the 5,15,and 30 minute test the commands would look like the following. The interval timing represents how often the system records the system stats for that time frame.
      
      
      StartRecordingStats interval=60 duration=300
      StartRecordingStats interval=180 duration=900
      StartRecordingStats interval=360 duration=1800
      
      
      
4. Once the time has expired on each test, you can stop the recording.
   
   
   pcli.exe submit -H <AxiomIP> PerformPitmanCommand CommandParameter="StopRecordingStats"
   
   
   
5. To collect the statistics file for review. Run the follwing command and then redirect the output to a text file.
   
   
   pcli.exe submit -H <AxiomIP> -u <username> -p <password> PerformPitmanCommand CommandParameter="GetMarksDb RecordCount=all" > <output filename>
    
   
   
6. Once the text file has been generated the traffic can then be turned off.
   
   
   pcli.exe submit -H <AxiomIP> PerformPitmanCommand CommandParameter="TrafficGenOff"
   
   
   
7. Please refer to the <Document 1547088.2> How to Upload Files to Oracle Support to transfer the results to the Oracle support for an existing SR.
   
   

Using getPITMAN

Oracle Support has made available a tool called getPITMAN for Windows/Solaris/Linux. The tool automates all the above commands and rules depending on the code version. The tool will also warn you if you choose to use the PortBlaster option.

A compressed archive is generated at the end and the console will advise if the Pitman was successful or if it needs further analysis by Support.

Syntax usage:

Example:

The getPITMAN file is attached to this Knowledge article (getPITMAN.zip for Windows and getPITMAN.sh for Linux/Solaris).

Please refer to the <Document 1547088.2> How to Upload Files to Oracle Support to transfer the results to the Oracle support for an existing SR.

References

This solution has no attachment