Sun Microsystems, Inc.  Oracle System Handbook - ISO 7.0 May 2018 Internal/Partner Edition
   Home | Current Systems | Former STK Products | EOL Systems | Components | General Info | Search | Feedback

Asset ID: 1-75-1933232.1
Update Date:2018-05-17
Keywords:

Solution Type  Troubleshooting Sure

Solution  1933232.1 :   Sparc Enterprise M3000/M4000/M5000/M8000/M9000 Servers XSCFU NETWORK: packet routing basic troubleshooting steps  


Related Items
  • Sun SPARC Enterprise M4000 Server
  •  
  • Sun SPARC Enterprise M9000-32 Server
  •  
  • Sun SPARC Enterprise M9000-64 Server
  •  
  • Sun SPARC Enterprise M5000 Server
  •  
  • Sun SPARC Enterprise M8000 Server
  •  
  • Sun SPARC Enterprise M3000 Server
  •  
Related Categories
  • PLA-Support>Sun Systems>SPARC>Enterprise>SN-SPARC: Mx000
  •  




In this Document
Purpose
Troubleshooting Steps
References


Applies to:

Sun SPARC Enterprise M3000 Server - Version All Versions to All Versions [Release All Releases]
Sun SPARC Enterprise M9000-64 Server - Version All Versions to All Versions [Release All Releases]
Sun SPARC Enterprise M9000-32 Server - Version All Versions to All Versions [Release All Releases]
Sun SPARC Enterprise M4000 Server - Version All Versions to All Versions [Release All Releases]
Sun SPARC Enterprise M8000 Server - Version All Versions to All Versions [Release All Releases]
Information in this document applies to any platform.

Purpose

The purpose of this document is to illustrate some common mistakes in packets routing at the XSCFU level, and some typical XSCF network misconfigurations.
Troubleshooting document for anyone dealing with Mx000/XSCF network issues.

Troubleshooting Steps

Data collection and information to be provided to Oracle Support

The XSCFU network settings can be gathered by running the shownetwork -a command.

Let's analyze the following output performed on a M9000 platform:

xscf#0-lan#0 ---------------------------------------------------------------> xscf#0-lan#0 port
Link encap:Ethernet HWaddr 00:14:4F:7F:32:94
inet addr:10.23.35.7 Bcast:10.23.35.255 Mask:255.255.254.0 -----------------> xscf#0-lan#0 - IP/Broadcat/Netmask addresses
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:116186 errors:0 dropped:0 overruns:0 frame:0
TX packets:56524 errors:155 dropped:0 overruns:0 carrier:155
collisions:0 txqueuelen:1000
RX bytes:10125870 (9.6 MiB) TX bytes:13372712 (12.7 MiB)
Base address:0xe000


lan#0 Link encap:Ethernet HWaddr 00:14:4F:7F:32:94
inet addr:10.23.35.6 Bcast:10.23.35.255 Mask:255.255.254.0 -----------------> lan#0 failover IP/Bradcast/Netmask addresses
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Base address:0xe000


xscf#0-lan#1 ---------------------------------------------------------------> xscf#0-lan#1 port
Link encap:Ethernet HWaddr 00:14:4F:7F:32:95
inet addr:192.168.11.11 Bcast:192.168.11.255 Mask:255.255.255.0 ------------> xscf#0-lan#1 - IP/Broadcat/Netmask addresses
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
Base address:0xc000


lan#1 Link encap:Ethernet HWaddr 00:14:4F:7F:32:95
inet addr:192.168.11.10 Bcast:192.168.11.255 Mask:255.255.255.0 ------------> lan#1 failover IP/Bradcast/Netmask addresses
UP BROADCAST MULTICAST MTU:1500 Metric:1
Base address:0xc000


xscf#0-if Link encap:Ethernet HWaddr 00:14:4F:7F:32:96
inet addr:192.168.1.1 Bcast:192.168.1.255 Mask:255.255.255.0----------------> Inter SCF Network IP address. The Inter-SCF Network (ISN) provides an internal communication link 
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1                              between the two Service Processors in a high-end server. This redundancy of two Service Processors
RX packets:1525894 errors:0 dropped:0 overruns:0 frame:0                      allows them to exchange system management information and, in case of failover, to change roles.
TX packets:3869942 errors:0 dropped:0 overruns:0 carrier:0                    The ISN network is pointless on the M3000/M4000/M5000 servers. The ISN addresses are set to:
collisions:0 txqueuelen:1000 192.168.1.1 for XSCF#0 and 192.168.1.2           for XSCF#1 by default: these values can be modified.
RX bytes:113353633 (108.1 MiB) TX bytes:2619650606 (2.4 GiB)
Base address:0x8400


xscf#1-lan#0 ---------------------------------------------------------------> Standby XSCFU xscf#1-lan#0 configuration is listed as well.
HWaddr 00:14:4F:7F:32:97
inet addr:10.23.35.8 Mask:255.255.254.0


xscf#1-lan#1 ---------------------------------------------------------------> Standby XSCFU xscf#1-lan#1 configuration is listed as well.
HWaddr 00:14:4F:7F:32:98
inet addr:192.168.11.12 Mask:255.255.255.0


xscf#1-if ------------------------------------------------------------------> Standby XSCFU SCF configuration is listed as well.

The Active XSCF in the above example is XSCFU#0. The redundant networking is set (lan#0 and lan#1 connected to two different subnets).

The routing table can be gathered via showroute -a(n) command. When the showroute –a is run without the -n option, and if the customer has DNS (Domain Name System) enabled, the IP addresses will be resolved into hostnames. The default router is represented by 0.0.0 in showroute -an, and resolved in "default" (*) in showroute -a.

xscf> showroute -a

Destination    Gateway       Netmask         Flags    Interface
172.17.8.0           *       255.255.252.0   U        xscf#0-lan#0
default        somedns       0.0.0.0         UG       xscf#0-lan#0

Destination    Gateway       Netmask         Interface
172.17.8.0           *       255.255.252.0   xscf#1-lan#0
default        somedns       0.0.0.0         xscf#1-lan#0

xscf> showroute -a -n

Destination    Gateway       Netmask         Flags    Interface
172.17.8.0     0.0.0.0       255.255.252.0   U        xscf#0-lan#0
0.0.0.0        172.17.9.175  0.0.0.0         UG       xscf#0-lan#0

Destination    Gateway       Netmask         Interface
172.17.8.0     0.0.0.0       255.255.252.0   xscf#1-lan#0
0.0.0.0        172.17.9.175  0.0.0.0         xscf#1-lan#0

 

Explanation:

1) 172.17.8.0 ---------------------> Network Address

2) 172.17.9.175 ----------------->  Default gateway address (resolved in somedns when a Domain Name System configuration is set)

3) 255.255.252.0 ----------------> Xscf#1-lan#0 Netmask (subnet address)

4) Gateway 0.0.0.0 --------------> Represents the default router (resolved in default or * when a Domain Name System configuration is set)

5) Netmask 0.0.0.0 --------------> Represents the default subnet mask address.

Running the showroute command on the XSCF will display the routing table. The fields in the output include the Destination IP address, Gateway, and Netmask. In the Flags column, a U indicates the route is up and a G indicates to use this gateway. The last field is the Network interface. The first line in the example output with the U flag is the netmask setting for the XSCF LAN port. In the first example on the slide, XSCF#1-LAN#0 has a netmask setting of 255.255.252.0

Note: both shotnetwork -a and showroute -a (n) are collected by the XSCF snapshot

 Snapshot: How to collect snapshot from the XSCFU: Gathering diagnostic data for SPARC Enterprise M3000/M4000/M5000/M8000/M9000 (OPL) Servers (Doc ID 1008229.1)

 

HOW TO CORRECTLY SET ROUTES WHEN YOU'RE TRYING TO REACH THE XSCFU NETWORK (LAN#0 and LAN#1) FROM A DIFFERENT SUBNET.

The XSCFU network configuration in a single subnet, has the same difficulties than any other "computer". It is an administrative task, in which the Oracle support has very little to add. The difficulty arises almost always in complex environments, where there are two or more subnets between the client machine and the XSCFU. The TSC role is to find and avoid some of the common routing mistakes, through the analysis of shownetwork-a  and showroute-a (n) commands. The customer's network infrastructure is not supposed to be checked (customer's duty).


1) Always check that the XSCFU default gateway address, matches the subnet router IP address:

routersettings

 

HOW TO FIX:
 

I) xscf> showroute -a -n

Destination      Gateway       Netmask        Flags Interface
192.168.1.0      0.0.0.0       255.255.252.0  U     xscf#0-lan#0
0.0.0.0          192.168.1.1   0.0.0.0        UG    xscf#0-lan#0

II) Add the correct default route first (the XSCFU firmware does not allow to delete a route if at least another one is set)

xscf> setroute –c add –n 0.0.0.0 –m 0.0.0.0 –g 172.17.9.175 xscf#1-lan#0
xscf> applynetwork –y
xscf> rebootxscf -y

III) Once XSCF has rebooted, delete the bad route

xscf> setroute –c del –n 0.0.0.0 –m 0.0.0.0 –g 192.168.1.1 xscf#1-lan#0
xscf> applynetwork –y
xscf> rebootxscf -y

IV) xscf> showroute -a -n

Destination      Gateway       Netmask         Flags Interface
172.17.9.0       0.0.0.0       255.255.252.0   U     xscf#0-lan#0
0.0.0.0          172.17.9.175  0.0.0.0         UG    xscf#0-lan#0

 

 

2) Always check that there is only one default router set

xscf> showroute -a -n

Destination     Gateway         Netmask         Flags Interface
172.17.9.0      0.0.0.0         255.255.252.0   U     xscf#0-lan#0
0.0.0.0         172.17.9.175    0.0.0.0         UG    xscf#0-lan#0
0.0.0.0         192.168.1.1     0.0.0.0         UG    xscf#0-lan#0
The XSCFU is set to multiple default routers and can’t figure out which one to use.
 
In the above example, showroute command shows two default routes set.


HOW TO FIX:

I) Delete the bad route

xscf> setroute –c del –n 0.0.0.0 –m 0.0.0.0 –g 192.168.1.1 xscf#1-lan#0
xscf> applynetwork –y
xscf> rebootxscf -y

II) xscf> showroute -a -n

Destination     Gateway         Netmask         Flags Interface
172.17.9.0      0.0.0.0         255.255.252.0   U     xscf#0-lan#0
0.0.0.0         172.17.9.175    0.0.0.0         UG    xscf#0-lan#0

 

 

3) Always check that the default router has the netmask set to 0.0.0.0 and not to the same netmask as the XSCF’s network

xscf> showroute -a -n

Destination     Gateway         Netmask          Flags Interface
192.168.1.0     0.0.0.0         255.255.255.0    U     xscf#0-lan#0
0.0.0.0         192.168.1.1     255.255.255.0    UG    xscf#0-lan#0

With this configuration the XSCFU cannot route packages outside the 255.255.255.0 subnet. 

HOW TO FIX:

I) Add the correct default route

xscf> setroute –c add –n 0.0.0.0 –m 0.0.0.0 –g 192.168.1.1 xscf#1-lan#0
xscf> applynetwork –y
xscf> rebootxscf -y

II) Delete the bad route

xscf> setroute –c del –n 0.0.0.0 –m 255.255.255.0 –g 192.168.1.1 xscf#1-lan#0
xscf> applynetwork –y
xscf> rebootxscf -y

 
III)
xscf> showroute -a -n

Destination     Gateway          Netmask          Flags Interface
192.168.1.0     0.0.0.0          255.255.255.0    U     xscf#0-lan#0
0.0.0.0         192.168.1.1      0.0.0.0          UG    xscf#0-lan#0

 

4) Always check that the router destination field is set to 0.0.0.0

xscf> showroute -a -n

Destination     Gateway         Netmask         Flags Interface
192.168.1.0     0.0.0.0         255.255.255.0   U     xscf#0-lan#0
192.168.1.0     192.168.1.1     255.255.255.0   UG    xscf#0-lan#0

In this case there is only a static route set back to its own subnet. Note that in the showroute command both destination settings are set to 192.168.1.0. When configured this way, the XSCF doesn’t know about routes outside the local subnet and the XSCF will not route outside the local subnet.

HOW TO FIX:  

I) Add the correct default route

xscf> setroute –c add –n 0.0.0.0 –m 0.0.0.0 –g 192.168.1.1 xscf#1-lan#0
xscf> applynetwork –y
xscf> rebootxscf -y

II) Delete the bad route

xscf> setroute –c del –n 192.168.1.0 –m 0.0.0.0 –g 192.168.1.1 xscf#0-lan#0
xscf> applynetwork –y
xscf> rebootxscf -y

III) xscf> showroute -a -n

Destination     Gateway         Netmask         Flags Interface
192.168.1.0     0.0.0.0         255.255.255.0   U     xscf#0-lan#0
0.0.0.0         192.168.1.1     0.0.0.0         UG    xscf#0-lan#0

 

System Firmware Matrix:

Firmware: Sun SPARC[TM] Enterprise M3000, M4000, M5000, M8000, M9000 XSCF Control Package (XCP) Firmware Image Software Version Matrix Information (Doc ID 1002631.1)

TSC Engagement:

Support Alias: Support_M_Series_WW@oracle.com        
Mail Archives: http://mailfinder.us.oracle.com/group/Support_M_Series_WW_GRP
IM Room: sparc-enterprise                                            
Product area:
OPL Servers                     
CTC Messages: sparc-ctc                                                
PLA Support Group:
SN-SPARC: Mx000

 

 

 

 

References

<NOTE:1020259.1> - Sun[TM] SPARC Enterprise M3000/M4000/M5000/M8000/M9000: Troubleshooting LDAP client service on the XSCFU
<NOTE:1315765.1> - Sun SPARC(R) Enterprise M3000/M4000/M5000/M8000/M9000 (OPL) Servers: Current Issues Page
<NOTE:1387044.2> - Information Center: Troubleshooting M3000/M4000/M5000/M8000/M9000-32/M9000-64
<NOTE:1002631.1> - Sun SPARC[TM] Enterprise M3000, M4000, M5000, M8000, M9000 XSCF Control Package (XCP) Firmware Image Software Version Matrix Information
<NOTE:1008229.1> - Gathering diagnostic data for SPARC Enterprise M3000/M4000/M5000/M8000/M9000 (OPL) Servers
<NOTE:1340082.1> - Sun SPARC Enterprise M3000, M4000, M5000, M8000, M9000 Product Pages ( OPL )
<NOTE:1558849.1> - Sun SPARC Enterprise M3000/M4000/M5000/M8000/M9000 - How to enable packet filtering on the XSCF

Attachments
This solution has no attachment
  Copyright © 2018 Oracle, Inc.  All rights reserved.
 Feedback