Fabric Interconnect IO Module Shows State up/failed

Asset ID:	1-72-1952057.1
Update Date:	2018-04-03
Keywords:

Solution Type Problem Resolution Sure

Solution 1952057.1 : Fabric Interconnect IO Module Shows State up/failed

Applies to:

Oracle Fabric Interconnect F1-4 - Version All Versions to All Versions [Release All Releases]
Oracle Virtual Compute Appliance X4-2 Hardware - Version All Versions to All Versions [Release All Releases]
Oracle Virtual Compute Appliance X3-2 Hardware - Version All Versions to All Versions [Release All Releases]
Oracle Fabric Interconnect F1-15 - Version All Versions to All Versions [Release All Releases]
Oracle Fabric Interconnect F1-15
Information in this document applies to any platform.

Symptoms

IO Module shows from 'admin' CLI in state up/failed. Logging into the Fabric Interconnect as user 'root' and grepping the /var/log/user.log for "IOCARD=<slot#>" returns output similar to below which shows the IO Module tried to boot 3 times and failed. After failing to boot/come up 3 times, the system permanently set it down:

Command:

grep -i "IOCARD=4" /var/log/user.log

Example:

Oct 16 16:03:38 fpp chassisCtr[469]: [ALERT] chassisctr fpp-1 [chassis::iocardoff] [IOCARD=4] Powered off.
Oct 16 16:03:38 fpp chassisCtr[469]: [NOTICE] chassisctr fpp-1 [chassis::cardstatechange] [IOCARD=4] Operational state changed. OldState=operStateInitializing, NewState=operStateDown, Qualifier=default
Oct 16 16:03:38 fpp chassisCtr[469]: [NOTICE] chassisctr fpp-1 [IOCARD=4] Power Down
Oct 16 16:16:08 fpp chassisCtr[469]: [NOTICE] chassisctr fpp-1 [chassis::iocardadminstatechange] [IOCARD=4] (changeAdminState) New AdminState=adminStateUp, OperState=operStateDown, Qualifier=default
Oct 16 16:16:08 fpp chassisCtr[469]: [NOTICE] chassisctr fpp-1 [chassis::iocardon] [IOCARD=4] Powered on.
Oct 16 16:16:10 fpp chassisCtr[469]: [NOTICE] chassisctr fpp-1 [chassis::cardstatechange] [IOCARD=4] Operational state changed. OldState=operStateDown, NewState=operStateInitializing, Qualifier=default
Oct 16 16:31:10 fpp chassisCtr[470]: [ALERT] chassisctr fpp-1 [chassis::iocardstartupretry] [IOCARD=4] Failed to startup. Retrying, attempts=1
Oct 16 16:31:10 fpp chassisCtr[470]: [ALERT] chassisctr fpp-1 [chassis::iocardoff] [IOCARD=4] Powered off.
Oct 16 16:31:10 fpp chassisCtr[470]: [NOTICE] chassisctr fpp-1 [chassis::cardstatechange] [IOCARD=4] Operational state changed. OldState=operStateInitializing, NewState=operStateInitializing, Qualifier=default
Oct 16 16:31:10 fpp chassisCtr[470]: [NOTICE] chassisctr fpp-1 [IOCARD=4] Power Down
Oct 16 16:31:13 fpp chassisCtr[470]: [NOTICE] chassisctr fpp-1 [chassis::iocardon] [IOCARD=4] Powered on.
Oct 16 16:46:14 fpp chassisCtr[470]: [ALERT] chassisctr fpp-1 [chassis::iocardstartupretry] [IOCARD=4] Failed to startup. Retrying, attempts=2
Oct 16 16:46:14 fpp chassisCtr[470]: [ALERT] chassisctr fpp-1 [chassis::iocardoff] [IOCARD=4] Powered off.
Oct 16 16:46:14 fpp chassisCtr[470]: [NOTICE] chassisctr fpp-1 [chassis::cardstatechange] [IOCARD=4] Operational state changed. OldState=operStateInitializing, NewState=operStateInitializing, Qualifier=default
Oct 16 16:46:14 fpp chassisCtr[470]: [NOTICE] chassisctr fpp-1 [IOCARD=4] Power Down
Oct 16 16:46:17 fpp chassisCtr[470]: [NOTICE] chassisctr fpp-1 [chassis::iocardon] [IOCARD=4] Powered on.
Oct 16 17:01:18 fpp chassisCtr[470]: [ALERT] chassisctr fpp-1 [chassis::iocardstartupfailed] [IOCARD=4] Failed to startup after 3 attempts. Powering off card.
Oct 16 17:01:18 fpp chassisCtr[470]: [ALERT] chassisctr fpp-1 [chassis::iocardoff] [IOCARD=4] Powered off.

Changes

IO Card state changed to up/failed. No manual changes made to environment.

Cause

This is an IO Module hardware component failure, need to RMA the IO Module.

Solution

If information in /var/log/user.log shows the card failed to power on 3 times and was set down, RMA the IO Module. See the CAP Document:1518778.1 - How to Replace a Defective I/O Module on a Oracle Fabric Interconnect Chassis

References

<NOTE:1518778.1> - How to Replace a Defective I/O Module on a Oracle Fabric Interconnect Chassis?

Attachments

This solution has no attachment