democratic-csi: Unable to mount iSCSI targets when the portal is a hostname in certain situations

Buckle up, we are going down the rabbit hole!

Through an unfortunate series of events, I discovered that iSCSI portal discovery is completely broken on TrueNAS Scale when binding your portal to one or more IPv6 addresses. Unbeknownst to me at the time, it turned out that democratic-csi can cope with this and just skip iscsi portal discovery entirely and add targets to the discovery db on its own since it already knows what portal and IQNs the LUNs are accessible at through the PV resources and/or its config.

So, I debugged the TrueNAS problem and PR’d the fix here: https://github.com/truenas/middleware/pull/12664

This created a new problem. Now that iscsi portal discovery suddenly started working, democratic-csi started taking advantage of that fact and everything fell apart. I was now faced with this dreaded message when creating a pod depending on an iSCSI LUN: MountVolume.MountDevice failed for volume "pvc-4562489f2f2c4fb4" : rpc error: code = Unknown desc = unable to find iscsi session for iqn: iqn.2005-10.org.freenas.ctl:csi-pvc-4562489f2f2c4fb4-cluster, portal: spitfire.storage.235.tdude.co:3260.

I perused the source code to find out what was going on. So it turns out that despite logging into the iSCSI portal with hostname, the iSCSI Target is replying with literal IP addresses it is bound to (the hostname resolves to one of these addresses) in the discovery response. This causes a problem because democratic-csi is looking for the LUN from the hostname but the discovery db now only contains entries associated to the IP address.

The error occurs here because we can’t find the session associated with the host name.

https://github.com/democratic-csi/democratic-csi/blob/79f16a0cf636f376f394f9b10084d949004b1e62/src/driver/index.js#L887-L898

The information is sourced from iscsiadm -m session which is filled with a bunch of ip address literals after discovery/target login.

https://github.com/democratic-csi/democratic-csi/blob/79f16a0cf636f376f394f9b10084d949004b1e62/src/utils/iscsi.js#L203

Finally, the crux of the issue happens here:

https://github.com/democratic-csi/democratic-csi/blob/79f16a0cf636f376f394f9b10084d949004b1e62/src/utils/iscsi.js#L178-L196

Because the portal argument to the function will be the FQDN I set in the config and the sessions will all be identified by ip literals.

I feel like there are three solutions here:

• Get scst on TrueNAS to return a hostname from discovery (will probably never happen).
• Add some logic to resolve the portal hostname inside democratic-csi and perform the mapping in the getSession function.
• Add a config option to the driver to always skip iSCSI discovery.

Let me know what you think!