ISCSI Initiator Configuration and Mulitipathing Guide

Installation Instructions:

The initiator is actually a kernel module that is already available with the appropriate CentOS Linux installation. To use and manage the initiator, you need to install the iSCSI utilities.

Notes:

*Do not use the version that comes on the CentOS v5.0 install CDs. That version, iscsi-initiator-utils-6.2.0.742-0.5.el5 does not work with the EqualLogicarray. You can find targets but not connect to them. You need version iscsi-initiator-utils-6.2.0.742-0.6.el5 or greater.

*CentOS 5.0 requires at least one iSCSI HBA and one GbE NIC to do multipathing.
*You will need CentOS 5.1 or greater to take advantage of multipathing with GbE NICs. This requires version iscsi-initiator-utils-6.2.0.865-0.8.el5 or greater.

Once installed, run:

To verify that the iSCSI service will be started at boot time, the chkconfig command can be used as follows:

By default, the newly added iscsi initiator is not enabled at boot, which is the reason for each of the run levels listed to have the service set to off. To enable this at boot, again use the chkconfig command as follows:

The above two commands first checks to be sure there are the necessary scripts to start and stop the service, and then it sets this service to be enabled for the appropriate runlevels. Then check to be sure the changes took effect:

You also need to do the same for the Multipath daemon.

Discovering Targets:

Once you have the iscsi service running you will use the ‘iscsiadm’ utility to discover, login and logout of targets. To get a list of available targets type:

Example:

The example shows that the ‘centos-test’ volume has been found but not yet logged in.

Logging into Targets

Here are two ways to connect to iSCSI targets : ( #man iscsiadm for other methods)

Log into all targets.

Log into an individual target.

Example:

Checking Session Status:

To see the connection status run:

Sample output:

Logging off a target:

It’s the same as logging into a target, except, that you use -u instead of -l

Persistent Device naming:

Devices using the Red Hat software initiators do not have a persistent naming scheme, and do not guarantee that a device (i.e. /dev/sdc) will always have the same device node. Persistent Naming describes mechanisms where the system identifies devices without relying on the /dev node, and provides a reference point for it that does not change at reboot.

First you have to comment out the ‘Blacklist all devices’ section in /etc/multipath.conf file

Note:

If the example file, multipath.conf is not in /etc, copy it from /usr/share/doc/device-mapper- multipath-0.4.7/multipath.conf.synthetic and edit it as below

So it should look like this:

Then restart the multipathd daemon

Now check that dev-mapper has configured the volume.

The highlighted number is the UUID of the volume. That never changes. You can use that UUID to create a persistent, friendlier name. For example you can name it the same as you called the volume on the EQL array.

Again edit the /etc/multipath.conf file.

Uncomment the following section and change the defaults to match your UUID and set a friendly alias name.

Edit the /etc/multipath.conf file and uncomment out the following:

Change the number after wwid to the UUID for your volume. Change the alias to something more friendly or use the volume name from the array. Change the rr_min_io to 10.

Here’s an example showing how to do more than one volume.

Save the file, then run:

You should now have a persistent name to access that volume.

Now create a EXT3 filesystem on that device.

Example:

Mounting iSCSI Filesystems at Boot:

In order to mount a filesystem that exists on an iSCSI Volume connected through the Linux iSCSI Software initiator, you need to add a line to the /etc/fstab file. The format of this line is the same as any other device and filesystem with the exception being that you need to specify the _netdev mount option, and you want to have the last two numbers set to 0 (first is a dump parameter and the second is the fsck pass).

The _netdev option delays the mounting of the filesystem on the device listed until after the network has been started and also ensures that the filesystem is unmounted before stopping the network subsystem at shutdown.

An example of an /etc/fstab line for a filesystem to be mounted at boot that exists on an iSCSI Volume is as follows:

Configuring Mulitpath Connections:

To create the multiple logins needed for Linux dev-mapper to work you need to create an ‘interface’ file for each GbE interface you wish to use to connect to the array.

You will have to create the ‘iface.ethX’ file in /var/lib/iscsi/ifaces.

You then edit the file and add in the MAC address of the ETH0 adapter.

You can get the MAC address with the following command:

The MAC address in this case is: 00:50:56:97:70:70

Here’s an example of what the /var/lib/iscsi/ifaces/iface.eth0 looks like:

To bind by hardware address set the NIC’s MAC address to iface.hwaddress

Example:

Repeat for the other ports you wish to use. E.g. if you are going to use ETH1, then copy the /var/lib/iscsi/ifaces/iface.eth0 file to /var/lib/iscsi/ifaces/iface.eth1.

Then edit the MAC address to reflect the MAC address of the ETH1 adapter.

If you have already discovered your volumes, you now need to re-discover the target(s).

You should see an entry for each interface you specified.

You’ll need to log into the volume.

This shows that both adapters have connected to the array.

Verify that the multipathing is correctly configured.

In this example you see that there are two paths to each volume.

Linux NIC Configuration:

Flowcontrol

Be sure that the network interfaces are configured to use Flow Control and Jumbo Frames if supported. To do this, use the ethtool utility on Red Hat.

To check for Flow Control (RX/TX Pause) on your interface, use the -a option as follows:

Example:

Set Flow Control to on using the -A parameter with ethtool as follows:

Example:

For some NICs, using ethtool will not persistently set this variable. For these NICs, see the manufacturer of the NIC for steps to configure the Flow Control setting for the NIC, or simply add the ethtool command to the end of the /etc/rc.d/rc.local file to set Flow Control for rx and tx to on. The majority of GbE NICs will detect and enable flowcontrol properly

For Jumbo Frames, you can use the ifconfig utility to make the change on a running interface. Unfortunately, this change will revert back to the default on a system reboot.

First, show the current setting for the interface in question using the ifconfig command using the interface name as an argument:

The setting we are interested in is the MTU value. As we can see from the above output, this is currently set to 1500 which is not a Jumbo packet size. To change this, we need to set the mtu again using the ifconfig command as follows:

To make this setting persistent, you need to add the MTU=”” parameter to the end of the ifcfg startup scripts for your SAN interfaces. These are found in the /etc/sysconfig/network-scripts directory. The naming format of the cfg files for your interfaces is ifcfg-.

A sample output of one of these files after adding the MTU line to the file should be similar to the following

Once the above Network changes have been made, you should reboot the host to verify that they have been made correctly and that the settings are persistent.