Tuesday July 03, 2007 Sun's Cluster, RAC and Interconnects
I have had to relay this little piece of information out a few times since Oracle does not support crossover cables for RAC interconnects regardless of the use of clustering software or not.These days RAC wants to manage the VIP, right down to the hardware level. If RAC sees a physical link loss, it wants to understand "why" and "where" so that it can evacuate that node.
With crossover cables Solaris basically reports that the entire link is down. And since Sun Cluster uses redundant interconnect paths (i.e "two" interconnects rather than "one); the Cluster nodes look at each other, shrug their shoulders, and continue communicating across the remaining path.
RAC gets all fussy and wants to exactly which NIC or link segment went away. With crossover cables, both links lights are down, and now what does RAC do?
But, with a switch in place, when a NIC fails, only the link light for the actual NIC goes down; RAC can then evaluate the situation and then evacuate the node.
So the tip for the day is how to figure out what NICs are connected without having to run over to the Data Center.
$ for i in 0 1 2 3 4 (and so forth)
> do
> echo "(interface_family)"$i
> kstat -n (interface_family)$i | grep -i link_up
>done
Where (interface_family) means the "ce," "ge," "e1000g" interface families.
The output to console will then tell you the status of all interfaces currently connected with a link light.
A grep of (interface_family) in /etc/path_to_inst will tell you all the installed network interfaces.
Oh yeah, you wouldn't believe how many times I've had to figure out what's going on (we can't tell you how we set up the cluster; that's privileged information).
( Jul 03 2007, 02:39:56 PM CDT ) Permalink Comments [1]