Out of Sync: Maddy's Blog

One of the features of the HAStoragePlus RT that is not available in HAStorage resource type is the ability to mount a file system locally.  i.e Only the node which owns the DG can have access to the FS while the other nodes don't.  This is the exact opposite of a global file system or PxFS where every node in the cluster can access to the file system irrespective of whether they have access to the storage disk or not.

Global file systems  make use of the global devices feature which has a global name space different from the OS naming convention of devices to ensure that all nodes see the shared devices irrespective of their access to it.  So in a globally mounted file system, the node which owns the DG does the read and write on behalf of the nodes even if some of them have direct access to the storage.  This means that it doesn't matter which node own the DG.  Hence there is no affinity with the DG ownership for  a Global FS.

On the contrary, for a locally mounted file system, since the node on which the file system is mounted alone can do any operations on it, it needs to have direct access to the storage devices like a stand alone box.  This means that the node has to own the DG to perform read/write operations.  Hence there is a strong affinity with DG ownership.  So the DG ownership is transfered to the node to which the HASP resource is switched or failed over to.

From the above conditions, one could conclude easily that scalable RG's HASP resource shouldn't have affinityon extension property of the HASP resource set to true.  Even if it is done, it is just ignored.

For a fail over RG with a global mount of the file system, it is better to set it to true.

For a fail over RG with a local mount of the file system, the affinityon value has to be true.

To change affinityon property, you can use the following command:

For 3.1u4 release, 

#scrgadm -cj <hasp-resource> -x affinityon=<false|true>

For 3.2 : 

#clrs set -p affinityon=<false|true> <hasp-resource> 

P.S:  note that the validation for affinityon checks the options column in /etc/vfstab file for "global" key word and hence change it accordingly failing which validation failures may be encountered.

One of the problems one could probably encounter  when adding a new node to a cluster is the failure of global devices to mount and you get a scary error message like this one:

...

...

Configuring DID devices

obtaining access to all attached disks



plift1 console login: _cladm: CL_GBLMNT_ENABLE: No such file or directory



WARNING - Unable to globally mount all filesystems.

Check logs for error messages and correct the problems.

After the problems are corrected, please clear the

maintenance flag on globaldevices by running the

following command:

/usr/sbin/svcadm clear svc:/system/cluster/globaldevices:default

While this appears very scary and one could think that there is a misconfiguration, it actually is a very simple problem.  The clue is verifying every single error message.  In this case, "_cladm: CL_GBLMNT_ENABLE: No such file or directory".

This problem occurs if there are global mounts in the cluster and the new node joining the cluster doesn't have the mount point present.  While HAStoragePlus creates a mount point if it is not already present when onlining a RG, in this case, the problem is mount of the file system at boot time. 

 Solution:

The solution is to create the mountpoint and clear the globaldevices service manifest! 

#mkdir  <global mountpoint> 

#/usr/sbin/svcadm clear svc:/system/cluster/globaldevices:default

Astalavista Baby!