If you want high availability for MySQL databases on Solaris or Open Solaris consider Solaris Cluster / Open HA Cluster. Solaris Cluster supports all available releases for MySQ 5.1. The MySQL agent for Solaris Cluster is usable with the release candidates of MySQL 5.1, and definitely with any final release of the MySQL 5.1 enterprise server or the community server. In addition to 5.1 we support all the current versions up to 5.0.51a.

With Solaris Cluster, you can achieve high availability for the MySQL database server and other applications in the same cluster. Solaris Cluster provides you with a rich framework to orchestrate any start or restart dependencies between the MySQL database server and other applications. You can even combine master and slave databases within the same Solaris Cluster. So you can achieve highly available masters and highly available slaves.

One advantage of the MySQL acquisition is that we have now earlier access to to the MySQL enterprise releases and closer access to the MySQL engineering force. So we can qualify new versions earlier than before. The next advantage in the close relationship is a faster problem resolution, because only one organization is working on customer problems.

To enable the MySQL 5.1 support of the Sun Cluster 3.2 agent for MySQL, you have to apply the following patches:

    Solaris 9 Sparc: 126031-04 or higher

    Solaris 10 Sparc: 126032-04 or higher

    Solaris 10 X64: 126033-05 or higher

For more information about setting up MySQL in Solaris Cluster, consult Sun Cluster Data Service for MySQL Guide for Solaris OS

Detlef Ulherr
Solaris Cluster Engineering 

Perhaps you've noticed the usual stream of Solaris Cluster blogs has slowed a bit in recent weeks. In case you're wondering, we're not blogged out, and we haven't "shot our blog" and run out of things to tell you about. In fact we've been very busy with lots of big things -- so watch for an upcoming "flog" (flood of blogs) as we release several new Agents and tools over the next several weeks.

And be sure to MARK YOUR CALENDARS for September 22 - 25 -- this is your chance to come to Oracle Open World in San Francisco's Moscone Center and head straight to Sun's booth where you'll be able to see a great magic show and -- better yet -- a live demo of Solaris Cluster Geographic Edition managing several Oracle 11g databases in disaster-tolerant configurations. And one of those configurations will be a joint demo with one of Sun's key storage partners .. this is the good stuff!

But wait -- there's more! -- besides the demo you'll have a chance to meet some of Solaris Cluster's lead developers and architects, and learn about the benefits of using the Solaris Cluster product suite with applications such as Oracle RAC. See you there!

Burt Clouse
Solaris Engineering Manager

Folks, when late last year we announced support for Solaris Cluster in LDoms I/O domains on this blog entry , we also hinted about support for LDoms guest domains. It has taken a bit longer then we envisaged, but i am pleased to report that SC Marketing has just announced support for LDoms guest domains with Solaris Cluster!!

So, what exactly does "support" mean here? It means that you can create a LDoms guest domain running Solaris, and then treat that guest domain as a cluster node by installing SC software (specific version and patch information noted later in the blog) inside the guest domain and have the SC software work with the virtual devices in the guest domain. The technically inclined reader would, at this point, have several questions pop into his head... How exactly does SC work with virtual devices? What do i have to do to make SC recognize these devices? Are there any differences between how SC is configured in LDoms guest domains, vs non-virtualized environments? Read-on below for a high level summary of specifics:

This covers the high level overview of how SC is to be deployed inside the LDoms guest domains. Check out the SC Release notes for additional details, and some sample configurations. The whole virtualization space is evolving very rapidly and new developments are happening ever so quickly. Keep this blog page bookmarked and visit it frequently to find out how Solaris Cluster is evolving along with this space.

Cheers!

Ashutosh Tripathi
Solaris Cluster Engineering

If you are using Sun Cluster software you are using the Proxy file system (PxFS). Global devices are made possible by PxFS, and global devices are central to device management in a cluster. The source is out there and now is a good time to explain some of the PxFS magic. I will give an overview of PxFS architecture with source references. I will do so via multiple blog entries. In this entry I will introduce PxFS and explain global mounting.

PxFS is a protocol layer that distributes a disk-based file system in a POSIX-compliant and highly available manner among cluster nodes. POSIX-compliant simultaneous access from multiple nodes is possible without demanding file-level locking from applications. The only requirement from the administrator to do a global mount, is to make sure the mount point exists on all cluster nodes. After that add a "-g" to the mount command and your mount becomes global. The following blog entry explains the terminology.

First let me show how easy creating and mounting a UFS file system globally is, without even using a dedicated physical device. I will create a lofi device, format it as UFS, and mount it globally.

Note: Do not try this in Solaris 9, as that Solaris version has an lofs bug that can panic the system.

Let us mount the above lofi device cluster wide (make sure the target directory exists on all nodes).

Done! You can access /mnt on any node of the cluster and reach the UFS file system on the lofi device on node1 transparently.

We will now get into the details of global mounting. I will take the example of globally mounting a file system in shared storage. We have a three-node cluster, with node2 and node3 having direct connection to shared storage. The svm metadevice "/dev/md/mydg/dsk/d42" is being mounted globally on the directory "/global/answer" from node1.

For code reference, startup of PxFS services happens here.

The mount subsystem is an HA service. An HA service in cluster parlance means that the service has failover capability. Any HA service has one primary and one or more secondaries. Any of the secondaries can become a primary if the current primary dies. This promotion of secondary to primary is transparent to applications.

For any cluster setup, there is always only one mount-service primary. All other nodes will have mount-service secondaries. Every cluster node will also have a mount client created when global mounts are first enabled for the node.

The mount primary and secondary are two faces of the mount replica object, which is created when the node joins the cluster. This is the code that creates the mount replica server. The replica framework ensures that there is only one primary at a time and promotes a secondary to primary when needed.

Now for the sequence of operations while doing a global mount. Refer to the picture below. Various steps during the global mount are numbered in sequence. Pointing your mouse at the number will pop up a tooltip explaining the step, with links to corresponding code.

Here are the steps from the above image map for easy reading.

1. The global mount command, mount -g, can be issued from any cluster node. It gets into the kernel and a generic mount redirects the call to PxFS. At this point, the directory to be mounted on is locked.
   http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/syscall/mount.c#125 http://src.opensolaris.org/source/xref/ohac/ohac/usr/src/common/cl/pxfs/client/pxvfs.cc#594


2. The PxFS client tells the mount server about this global mount request via the mount client on that node. The mount client will have the server reference.
   http://src.opensolaris.org/source/xref/ohac/ohac/usr/src/common/cl/pxfs/client/pxvfs.cc#999


3. The mount server in turn asks every client except the originating node, in this case node1, to lock the mount point.
   http://src.opensolaris.org/source/xref/ohac/ohac/usr/src/common/cl/pxfs/mount/mount_server_impl.cc#1204


4. For shared devices, the mount server creates a PxFS primary and secondary. The node on which the device is primaried becomes the PxFS primary. For local devices, the mount is non-HA and an unreplicated PxFS server is created. The lofi device example above, will result in an unreplicated PxFS server being created on node1.
   
   The PxFS server does a hidden mount of the device. Details of the mount is contained in the PxFS server object.
   http://src.opensolaris.org/source/xref/ohac/ohac/usr/src/common/cl/pxfs/mount/mount_server_impl.cc#1237 http://src.opensolaris.org/source/xref/ohac/ohac/usr/src/common/cl/pxfs/server/repl_pxfs_server.cc#125


5. The mount server passes a reference to the newly created server to all mount clients and asks the clients to do a user-visible PxFS mount.
   http://src.opensolaris.org/source/xref/ohac/ohac/usr/src/common/cl/pxfs/mount/mount_server_impl.cc#1438 http://src.opensolaris.org/source/xref/ohac/ohac/usr/src/common/cl/pxfs/mount/mount_server_impl.cc#1684


6. The mount client creates and adds a vfs_t entry of the same type as the underlying file system.
   http://src.opensolaris.org/source/xref/ohac/ohac/usr/src/common/cl/pxfs/mount/mount_client_impl.cc#1804 http://src.opensolaris.org/source/xref/ohac/ohac/usr/src/common/cl/pxfs/mount/mount_client_impl.cc#1817

Now the mount is visible on all clients. There is some more magic the mount subsystem does, like starting an fs replica when a node joins the cluster, or creating a new PxFS secondary or primary when a node that is connected to storage joins the cluster etc. The next installment will be about how regular file access in PxFS works.

Thanks to Walter Zorn for the javascript library which made tooltips so much easier.

If you read the Solaris Cluster Express 6/08 available for download blog posting, you will have noticed that it mentioned that SCX Geographic Edition had been enhanced to support Oracle Data Guard replication. Now if you run Oracle 10g or 11g RAC in your data centers and need to integrate them into a disaster recovery framework that supports: Sun StorageTek Availability Suite (AVS), Hitachi TrueCopy and EMC SRDF then read on.

SCX Geographic Edition (SCXGE) is binary distribution of the open source version of Sun Cluster Geographic Edition (SCGE) that runs on Solaris Cluster Express (SCX) on top of Solaris Express Community Edition (SXCE) build 86. [We love our acronyms and abbreviations]. Trivia: did you know that SUN itself is an acronym that originally stood for Stanford University Networks?

Back to the plot... Services made highly available by SCX resource groups can grouped together in "protection groups". SCXGE then uses the robust start and stop methods of SCX to ensure that the protection group is migrated in a controlled fashion from a primary location to a standby site. This migration includes performing all the tasks needed to change the direction of data flow provided by the underlying replication technology. The advantage being that SCXGE hides the complexity of the individual replication methods behind a common command line interface. Thus switching a service from site A to site B is reduced to:

# geopg switchover -m newhost_cluster my-protection-pg

regardless of whether the replication was AVS, TC or SRDF based.

Support for Oracle Data Guard (ODG) extends SCXGE's capability beyond just software and storage based replication technologies. Internally, the ODG module uses the Oracle Data Guard broker interface which restricts its use to Oracle 10g and 11g RAC configurations only, i.e. no support for HA-Oracle.

So once you set up your Oracle RAC databases, configure ODG and create your Data Guard broker configuration, you can add that to your SCXGE configuration and have it monitored and controlled, just as you do for your other non-ODG protection groups. The module allows you to have multiple ODG protection groups, each with one or more ODG broker configuration in. Remember that each protection group can span any SCXGE partnership pair, so you can only have one primary and one standby database in each of these broker configurations.

If this has got you interested, feel free to download and install the SCX software bundle with the new ODG module and try it out. Just remember that this is the first build. I'm working on fixing all the bugs that my colleagues in QA discover, so keep an eye on this blog and the Open HA Cluster for news of any updates.

Although we don't offer support for this product, you can post questions in any of the usual forums (or is it fora ).

Tim Read