Cannot See LUNS When Using ISCSI Over Iser with Sun ZFS Storage 7320

Asset ID:	1-72-2219852.1
Update Date:	2017-06-10
Keywords:

Solution Type Problem Resolution Sure

Solution 2219852.1 : Cannot See LUNS When Using ISCSI Over Iser with Sun ZFS Storage 7320

Applies to:

Sun ZFS Storage 7320 - Version All Versions and later
Linux OS - Version Oracle Linux 6.0 and later
Information in this document applies to any platform.

Symptoms

When configuring an Oracle ZFS Storage Appliance, changing iSCSI's transport mode from TCP to iSER, results in the LUNs from the storage no longer being available.

When trying to connect via iSER with UEK3, the following is seen on the client:
in /var/log/messages showed:

Jun 2 18:07:40 localhost iscsid: Kernel reported iSCSI connection 11:0 error (1011 - ISCSI_ERR_CONN_FAILED: iSCSI connection failed) state (3)
Jun 2 18:07:40 localhost iscsid: Kernel reported iSCSI connection 10:0 error (1011 - ISCSI_ERR_CONN_FAILED: iSCSI connection failed) state (3)
Jun 2 18:07:40 localhost iscsid: Kernel reported iSCSI connection 12:0 error (1011 - ISCSI_ERR_CONN_FAILED: iSCSI connection failed) state (3)
Jun 2 18:07:40 localhost iscsid: Kernel reported iSCSI connection 9:0 error (1011 - ISCSI_ERR_CONN_FAILED: iSCSI connection failed) state (3)

On the storage appliance, the following is seen:

Jun 2 08:07:40 zfs-appliance genunix: [ID 801312 kern.warning] WARNING: mcxnex0: CQE ERR: cqe fffff6046de140e0 QPN 500062 indx 7 status 0x13 vendor syndrome 88
Jun 2 08:07:40 zfs-appliance genunix: [ID 158346 kern.warning] WARNING: mcxnex0: CQE remote access error
Jun 2 08:07:40 zfs-appliance genunix: [ID 801312 kern.warning] WARNING: mcxnex0: CQE ERR: cqe fffff6046dce20e0 QPN 680061 indx 7 status 0x13 vendor syndrome 88
Jun 2 08:07:40 zfs-appliance genunix: [ID 158346 kern.warning] WARNING: mcxnex0: CQE remote access error
Jun 2 08:07:40 zfs-appliance genunix: [ID 801312 kern.warning] WARNING: mcxnex0: CQE ERR: cqe fffff6046d9e00e0 QPN c004e indx 7 status 0x13 vendor syndrome 88

On UEK4, when attempting to use iSER the following is seen:

root@localhost ~]# iscsiadm -m discoverydb -t st -p 172.30.251.250 --discover -I iser
iscsiadm: recv's end state machine bug?
iscsiadm: Could not perform SendTargets discovery: iSCSI PDU timed out

in var/log/messages:

Jul 23 09:34:15 localhost kernel: [ 2185.632750] scsi host16: iSCSI Initiator over iSER
Jul 23 09:34:15 localhost kernel: [ 2185.637190] iser: iser_handle_wc: wr id ffff880c3808c000 status 1 vend_err d7 <<<<<<<<<<<<<<
Jul 23 09:34:15 localhost kernel: [ 2185.637497] connection9:0: detected conn error (1011)

Cause

iSER is not supported for Linux clients using IPoIB connections to the Oracle ZFS Storage Appliance

Please refer to:
https://docs.oracle.com/cd/E26765_01/html/E26397/configuration__san__initiator_support.html
Linux is also not listed as supported in the following documentation:

Oracle ZFS Storage Appliance: Interoperability for OS8.6.0 Software Release Doc ID 2151588.1

Solution

Use TCP for connections instead of iSER.

References

<BUG:20761862> - CANNOT CONNECT/DISCOVER ISER DEVICES ON SUN ZS3-4 FROM SUN X4-2 / OL6.6
<BUG:24344027> - CAN NOT LOGIN ISCSI VOLUME THROUGH ISER
<NOTE:1435063.1> - Sun Storage 7000 Unified Storage System: How to Troubleshoot Infiniband Issues
<NOTE:1644939.1> - Oracle ZFS Storage: iSCSI Configuration Options for Windows, Linux, and Solaris
<NOTE:1401425.1> - Sun Storage 7000 Unified Storage System: iSCSI Troubleshooting - Cause Determination
<NOTE:2151588.1> - Oracle ZFS Storage Appliance: Interoperability for OS8.6.0 Software Release

Attachments

This solution has no attachment