People sometimes wonder what the difference is between a /dev/global and a /dev/did device in Sun Cluster.
Let us first discuss the /dev/did device. You can see the /dev/did device as just another name for a Solaris /dev/dsk/c#t#d#s# device, with the only difference that this name is guaranteed to be consistent throughout the cluster: The cluster nodes have synchronised their view on the storage and have ended up with the same name for a specific device. It is quite possible that a device is called /dev/dsk/c1t0d0s0 on nodea but /dev/dsk/c2t0d0s0 on nodeb. A common name for these disks comes in handy in the following situations:
-When assigning a Quorum Disk all nodes must know which disk I am talking about. If I would say /dev/dsk/c1t0d0s0, nodeb would be confused
-When adding disks to a diskset
-When creating datafiles on raw LUN partitions in Oracle RAC.
The last example is an interesting one and points us to an important difference between /dev/did and /dev/global devices. In Oracle RAC each instance on each node writes directly to the disks. Synchronisation of the I/O is done by Oracle RAC itself, the OS or cluster layer does not have to worry about that.
If you deploy Sun's HA Oracle agent, and you want to create datafiles on raw LUN partitions, you have to use the /dev/global devices. HA Oracle is a single instance Oracle which is cluster-unaware. This means when a node fails, the HAOracle scripts will take care of the Switchover of the instance to the other node. One of the important issues with such cluster-unaware applications that are made HA through scripts is that we must make sure that I/O to the disk always only happens from one node. During a failover from nodea to nodeb, we must prevent that nodea would still do I/O to the disks. This could happen if if wouldn't be aware of nodeb's takeover (eg in the case of a Split Brain). In Sun Cluster, the Device Configuration System (DCS) takes care of that. The DCS makes sure that any global device has only one primary node: only this primary node can do I/O to the disk. Other nodes, when they try to access the global device, will have their I/O rerouted through the private interconnect. When a failover of the global device occurs, the DCS will use fencing techniques to make sure that the old primary cannot accidentally access the device. /dev/global/dsk/d4s2 is a global device whereas /dev/did/dsk/d4s2 is not.

Let us take a look at how the device files look:

1) For a /dev/did device we see that it is a device file just like any other:
etc-1 # ls -l /dev/did/dsk/d4s2
lrwxrwxrwx   1 root     root          39 Jul 10  2003 /dev/did/dsk/d4s2 -> ../../../devices/pseudo/did@0:4,4s2,blk
etc-1 # ls -l /devices/pseudo/did@0:4,4s2,blk
brw-------   1 root     sys      228,130 Jul 11  2001 /devices/pseudo/did@0:4,4s2,blk


2) For the /dev/global access path of this device d4s2, we see that it is a bit more difficult:
etc-1 # ls -l /dev/global
lrwxrwxrwx   1 root     root          34 Apr 14 13:15 /dev/global -> /global/.devices/node@1/dev/global

So /dev/global/dsk/d4s2 is actually /global/.devices/node@1/dev/global/dsk/d4s2

etc-1 # ls -l /global/.devices/node@1/dev/global/dsk/d4s2
lrwxrwxrwx   1 root     root          39 Jan 23 21:03 /global/.devices/node@1/dev/global/dsk/d4s2 -> ../../../devices/pseudo/did@0:4,4s2,blk

This points to a physical device file /global/.devices/node@1/devices/pseudo/did@0:4,4s2,blk

The thing to notice here is that the file system /global/.devices/node@1 is actually a globally mounted, pxfs filesystem. If you access a device file in such a filesystem, the access is 'cluster aware', which means the actual request will be sent to the node that is the primary. Thus, although /dev/did/dsk/d4s2 and /dev/global/dsk/d4s2 point to the same device, their access path is different: non-cluster aware in the first case, cluster aware in the second case.