Linux kernel Generating Messages "RDS/IB: recv completion <X.X.X.X,Y.Y.Y.Y,0> had status 5, disconnecting and reconnecting."

Asset ID:	1-72-1987091.1
Update Date:	2017-12-31
Keywords:

Solution Type Problem Resolution Sure

Solution 1987091.1 : Linux kernel Generating Messages "RDS/IB: recv completion had status 5, disconnecting and reconnecting."

Applies to:

Linux OS - Version Oracle Linux 5.5 and later
Exadata X4-2 Quarter Rack - Version All Versions to All Versions [Release All Releases]
Linux x86-64

Symptoms

Server is running without any outages or over workload reported
Following messages are logged in system log file /var/log/messages:

Feb 17 10:31:02 server kernel: RDS/IB: connection <x.x.x.x,y.y.y.y,0> dropped
Feb 17 10:31:02 server kernel: RDS/IB: connection <x.x.x.x,y.y.y.y,0> dropped
Feb 17 10:31:02 server kernel: RDS/IB: recv completion <x.x.x.x,y.y.y.y,0> had status 5, disconnecting and reconnecting
Feb 17 10:31:02 server kernel: RDS/IB: connection <x.x.x.x,y.y.y.y,0> dropped
Feb 17 10:31:02 server kernel: RDS/IB: recv completion <x.x.x.x,y.y.y.y.y,0> had status 4, disconnecting and reconnecting
Feb 17 10:31:02 server kernel: RDS: rds_conn_shutdown: failed to transition to state DOWN, current state is 4
Feb 17 10:31:02 server kernel: RDS/IB: connected <x.x.x.x,y.y.y.y,y,0> version 4.1
Feb 17 10:31:02 dsmid1 kernel: RDS/IB: connected <x.x.x.x,y.y.y.y,y,0> version 4.1

or

Feb 1 14:25:31 server kernel: RDS/IB: recv completion on 192.168.10.36 had status 4 (IB_WC_LOC_PROT_ERR), disconnecting and reconnecting
Feb 2 15:14:32 server kernel: mlx4_core 0000:19:00.0: command 0x1c failed: fw status = 0x10
Feb 2 15:14:32 server kernel: ib_modify_qp(IB_QP_STATE, RTS): err=-22
Feb 2 15:14:32 server kernel: rds_connect_complete: Cannot transition to state UP, current state is 4

Note: According to the latest kernel source code(ex. 2.6.39-400.294.4.el6uek), the message in this Note:
> RDS/IB: connection <10.60.7.241,10.60.7.232,0> dropped
has been changed to:
> RDS/IB: connection <10.60.7.241,10.60.7.232,0> dropped due to 'recv completion error'
adding detailed error information.

Cause

These messages are normal and recoverable. IB_WC_LOC_PROT_ERR can be caused by the HCA(IB card) accessing the MR (memory region) while it's in transition, for example, the MR is being accessed while it's partially or completely deregistered. This error is normal and recoverable.

Solution

You can safely ignore it as this message is normal and recoverable.

References

<BUG:14766056> - RDS/IB: RECV COMPLETION ON 192.168.10.36 HAD STATUS 4

Attachments

This solution has no attachment