Introduction to iSCSI in Solaris 10 8/07 (U4)
Tuesday Feb 19, 2008
After consulting with Jeff Victor a colleague of mine this week on the subject of iSCSI, I decide to do some reading and playing around myself with iSCSI. Jeff was actually testing iSCSI with LDOMS etc.. Look forward to seeing Jeff's work.
The Solaris 10 8/07 (U4) release provides support for iSCSI target devices, which can be disk or tape devices. Releases prior to Solaris 10 8/07 provided support for iSCSI initiators. 
So what does this mean? Well, It means that Solaris can act as both a “server” (target) and a “client” (initiator) for iSCSI.
The advantage of setting up Solaris iSCSI targets is you might have existing fibre-channel devices that can be connected to clients without the cost of fibre-channel HBAs. In addition, systems with dedicated arrays can now export replicated storage with ZFS or UFS file systems.
While experimenting with iSCSI you'll need to remember some basic commands.
You will use the iscsitadm(1M) command to set up and manage your iSCSI target devices. For the disk device that you select as your iSCSI target, you'll need to provide an equivalently sized ZFS or UFS file system as the backing store for the iSCSI daemon.
After the target device is set up, you'll use the iscsiadm(1M) command to identify your iSCSI targets, which will discover and use the iSCSI target device.
A good reference for you to start looking at when configuring iSCSI would be the Configuring Solaris iSCSI Targets and Initiators (Tasks), in the Systems Administration Guide: Devices and File Systems.
To assist you with iSCSI, I’ve jotted down some steps which will allow you to export some targets, play around with them and then destroy them afterwards.
The following hosts will be used in this exercise:
| Hostname | IP address | Description |
| sunrise1 | 192.168.1.25 | This machine has the zpool, called zpoolA for this exercise |
| spirit2 | 192.168.1.30 | This machine will be used to mount our iSCSI targets |
Our zpool, zpoolA, will be used for storage.
The goal of this exercise is to couple ZFS with the iSCSI target in Solaris. There are two new ZFS properties added to support this feature. For this exercise we will be only using the shareiscsi property. Note the other property is the iscsioptions.
shareiscsiLike the 'sharenfs' property, 'shareiscsi' indicates if a ZVOL should
be exported as an iSCSI target. The acceptable values for this property
are 'on', 'off', and 'direct'. In the future, we may support other
target types (for example, 'tape'). The default is 'off'. This property
may be set on filesystems, but has no direct effect; this is to allow
ZVOLs created under the ZFS hierarchy to inherit a default. For
example, an administrator may want ZVOLs to be shared by default, and
so set 'shareiscsi=on' for the pool.
iscsioptionsThis read-only property, which is hidden, is used by the iSCSI target
daemon to store persistent information, such as the IQN. It cannot be
viewed or modified using the zfs command. The contents are not intended
for external consumers.
First, we need to create some volumes to use. Since I already started with my zpoolA, we’ll use that. We’ll create three volumes, each 1Gbyte in
size using the zfs(1M) command.
sunrise1# zfs create -V 1G zpoolA/iscsi_luns/vol001
sunrise1# zfs create -V 1G zpoolA/iscsi_luns/vol002
sunrise1# zfs create -V 1G zpoolA/iscsi_luns/vol003
Now, you'll need to configure them as iSCSI targets
sunrise1# zfs set shareiscsi=on zpoolA/iscsi_luns/vol001
sunrise1# zfs set shareiscsi=on zpoolA/iscsi_luns/vol002
sunrise1# zfs set shareiscsi=on zpoolA/iscsi_luns/vol003
Walla! You've created three thinly provisioned 1GB volumes and then turned "shareiscsi" on.
It's that simple! Seeing is believe! Here's how you check!
sunrise1# svcs -a | grep -i iscsi
disabled 7:02:49 svc:/network/iscsi_initiator:default
online 19:21:01 svc:/system/iscsitgt:default
sunrise1# iscsitadm list target -v
Target: zpoolA/iscsi_luns/vol001
iSCSI Name: iqn.1986-03.com.sun:02:7536c2517-91e6-cf6c-d566-d48fb182e9f7
Alias: zpoolA/iscsi_luns/vol001
Connections: 0
ACL list:
TPGT list:
LUN information:
LUN: 0
GUID: 0x0
VID: SUN
PID: SOLARIS
Type: disk
Size: 1GB
Backing store: /dev/zvol/rdsk/zpoolA/iscsi_luns/vol001
Status: online
Target: zpoolA/iscsi_luns/vol002
iSCSI Name: iqn.1986-03.com.sun:02:78692333-1fb4-ee45-b547-ea49922ee538
Alias: zpoolA/iscsi_luns/vol002
Connections: 0
ACL list:
TPGT list:
LUN information:
LUN: 0
GUID: 0x0
VID: SUN
PID: SOLARIS
Type: disk
Size: 1GB
Backing store: /dev/zvol/rdsk/zpoolA/iscsi_luns/vol002
Status: online
Target: zpoolA/iscsi_luns/vol003
iSCSI Name: iqn.1986-03.com.sun:02:792405e0-a3fc-4ccf-f86d-b79f7b1ee006
Alias: zpoolA/iscsi_luns/vol003
Connections: 0
ACL list:
TPGT list:
LUN information:
LUN: 0
GUID: 0x0
VID: SUN
PID: SOLARIS
Type: disk
Size: 1GB
Backing store: /dev/zvol/rdsk/zpoolA/iscsi_luns/vol003
Status: online
Again make sure they’re visible by running the following command
sunrise1# iscsitadm list target
Target: zpoolA/iscsi_luns/vol001
iSCSI Name: iqn.1986-03.com.sun:02:7536c2517-91e6-cf6c-d566-d48fb182e9f7
Connections: 0
Target: zpoolA/iscsi_luns/vol002
iSCSI Name: iqn.1986-03.com.sun:02:78692333-1fb4-ee45-b547-ea49922ee538
Connections: 0
Target: zpoolA/iscsi_luns/vol003
iSCSI Name: iqn.1986-03.com.sun:02:792405e0-a3fc-4ccf-f86d-b79f7b1ee006
Connections: 0
Now, to start using the targets! We'll first need to tell our initiator where to look for the storage. Note: Keep in mind, that the iSCSI connection is not initiated until the discovery method is enabled. For the purpose here, we will be configuring the device for dynamical discovery (SendTargets). There are two new ZFS properties added to support this feature. For this exercise we will be only using the shareiscsi property. The other property is the iscsioptions.
shareiscsiLike the 'sharenfs' property, 'shareiscsi' indicates if a ZVOL should
be exported as an iSCSI target. The acceptable values for this property
are 'on', 'off', and 'direct'. In the future, we may support other
target types (for example, 'tape'). The default is 'off'. This property
may be set on filesystems, but has no direct effect; this is to allow
ZVOLs created under the ZFS hierarchy to inherit a default. For
example, an administrator may want ZVOLs to be shared by default, and
so set 'shareiscsi=on' for the pool.
iscsioptionsThis read-only property, which is hidden, is used by the iSCSI target
daemon to store persistent information, such as the IQN. It cannot be
viewed or modified using the zfs command. The contents are not intended
for external consumers.
spirit2# iscsiadm add discovery-address 192.168.1.25:3260
We will now need to tell our initiator how to look for our targets. Note the -t, this is the flag to enable the sendtargets. You can use --sendtargets flag if you prefer.
spirit2# iscsiadm modify discovery -t enable
It is now time to use devfsadm(1M) to find our new storage. Note the -C flag, This flag is the Cleanup mode. Prompts devfsadm to cleanup dangling /dev links that are not normally removed. This -i flag is for configuring only the devices for the named driver, driver_name.
spirit2# devfsadm -C -i iscsi
I've found if you run iostat -En you'll see that we have new storage, like this
spirit2# iostat -En
...
c10t2d0 Soft Errors: 2 Hard Errors: 0 Transport Errors: 0
Vendor: SUN Product: SOLARIS Revision: 1 Serial No:
Size: 1.07GB <1073741824 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
...
If you look at the syslog, you’ll probably notice messages complaining about a bad magic number. This is because our newly visible LUNs aren’t labelled yet. So all you'll need to do is run the format(1M) command and label them like any other disk.
Note: If you want to make the iSCSI drive available on reboot, create the file system, and add an entry to the /etc/vfstab file as you would with a UFS file system on a SCSI device.
Now once, that the LUN's are formatted we can create filesystems and mount them, for example:
spirit2# newfs /dev/rdsk/c10t2d0s0
spirit2# mkdir /mnt/iscsi_mytest
spirit2# mount /dev/dsk/c10t2d0s0 /mnt/iscsi_mytest
Once we’ve finished playing around with our volumes, you might want to return your hosts to a clean state so simply unmount any filesystems and remove the directory where it was mounted.
spirit2# umount /mnt/iscsi_mytest
spirit2# rmdir /mnt/iscsi_mytest
Now, If you created any zpools or SVM metadevices, clear them up as well. Once everything is clear: These optional procedure, assumes that you are logged in to the local system where access to an iSCSI target device has already been configured.
Note: After removing a discovery address, iSNS server, static-config, or disabling a discovery method, the associated targets are logged out. If these associated targets are still in use, for example, have mounted file systems, and you did not execute the previous commands, (umount, rmdir), the logout of these devices will fail and they will remain on the active target list.
spirit2# iscsiadm modify discovery -t disable
spirit2# iscsiadm remove discovery-address 192.168.1.25:3260
spirit2# devfsadm -C -i iscsi
and on the target host do the following
sunrise1# zfs destroy zpoolA/iscsi_luns/vol001
sunrise1# zfs destroy zpoolA/iscsi_luns/vol002
sunrise1# zfs destroy zpoolA/iscsi_luns/vol003
There you go! A simple exercise configuring using iSCSI
In conclusion, please note that because iSCSI target LUN's are shared network accessible block storage devices they are built on top of the zvols. The advantage of these zvols is that they come with all that ZFS has to offer filesystems including snapshots, replication, compression, etc...
Special Thanks to Jeff Victor & Scott Dickson for their help on the subject.
Well have fun with iSCSI... Until next time!!
Technocrati Tags:Sun Solaris ZFS iSCSI
|
|
|
|
For eval, can you share the targets over the loopb...