LDoms 1.0.3
Logical Domains (LDoms) 1.0.3 is now available. This release is mainly intended to enable many new features included in Solaris 10 5/08 (aka update 5). Jason did an excellent job with details & logistics in this blog entry, so there's no need in my repeating that info here.
One thing I do want to mention in terms of LDom Manager functionality is that with this release, the XML format produced by the
ldm ls-constraints -xcommand has changed. While the LDom Manager will continue to accept our previous, so-called v2 XML format, as of 1.0.3, it also accepts & produces the new v3 format. This format is designed to closely align with the schema defined as part of the draft Open Virtual Machine Format (OVF) specification.
This is just the tip of the iceberg. Coming in LDoms 1.1 (currently targeted for release in Q4CY08) will be a complete XML based control interface for monitoring & managing Logical Domains, based on this same v3 schema. In addition, it will utilize the XMPP transport, providing secure, standards-based XML messaging between client application & the LDom Manager.
This combination of a standards-based schema over a standard XML transport provides a rich control interface for creating management applications. More details about this new management interface, including detailed specifications, will be forthcoming.
Posted at 10:57AM May 19, 2008 by Eric Sharakan in Sun | Comments[0]
UltraSPARC T2 Plus & LDoms 1.0.2
Today we announced our T5140/T5240 platforms, based around the UltraSPARC T2 Plus processor. This is the first CMT platform which supports multiple processor chips (2), providing up to 128 hardware threads of CMT goodness! And to go along with these platforms, we're releasing Logical Domains version 1.0.2, which fully supports the new T5140/T5240 systems, including the ability to create up to 128 logical domains to match the number of available hardware threads.
Of course, most users probably won't want to run 128 single threaded logical domains, but the fact that you can is a testament to the scalability of the architecture.
Other features of LDoms 1.0.2 include:
- Support for up to 64 domains on UltraSPARC T2 based systems
- Libvirt management interface software included
- A slew of bug fixes
Of course, LDoms 1.0.2 supports all the platforms supported by LDoms 1.0.1 & LDoms 1.0, and like its predecessors, is available at no extra cost. In fact, LDoms 1.0.2 comes factory installed on all T5140/T5240 systems. The associated firmware & LDoms Manager packages are also available from the Sun Download Center.
Posted at 09:13PM Apr 09, 2008 by Eric Sharakan in Sun | Comments[0]
LDoms versions & compatibility
With the release of LDoms 1.0.1 last October, along with the release of the first members of our UltraSPARC T2 (US T2) based platform family, issues of version compatibility naturally arise. When I first posted about the availability of LDoms 1.0.1, I included the following caveats:
WARNING: There are two important caveats when upgrading from LDoms 1.0 to LDoms 1.0.1:
- Configurations saved to the service processor under 1.0 are not usable under 1.0.1. The LDoms 1.0.1 Administration Guide describes the upgrade procedure that needs to be applied to work around this. Part of this procedure needs to be carried out BEFORE performing the actual upgrade!
- You must upgrade both the firmware and LDom Manager components at the same time.
One of the pillars of Sun's products is our ability to maintain compatibility; we have explicit practices & processes in place to guarantee compatibility between product releases. Unfortunately, we did not apply those to the 1.0.1 LDoms upgrade. This was an omission we do not intend to repeat. So, when upgrading from LDoms 1.0 to any subsequent version, the caveats mentioned above do apply (the current LDoms release is still 1.0.1, but we are planning for two follow-on releases in CY 2008).
As we wrap up the development phase of our upcoming 1.0.2 release, we're re-verifying that existing configurations created under 1.0.1 do continue to work under 1.0.2, and that firmware & LDom Manager upgrades do not need to be synchronized. Of course, certain 1.0.2 capabilities might not be fully enabled until both the firmware & the LDom Manager are upgraded, but at least the system will continue to function with only one or the other upgraded.
UPDATE 4/29/2008: With the release of LDoms 1.0.2, I'm happy to report that neither of the caveats described in this entry apply when upgrading from LDoms 1.0.1 to LDoms 1.0.2. However, they do still apply if you're upgrading from LDoms 1.0 to LDoms 1.0.2 (or any subsequent release). In other words, these compatibility issues only come into play when upgrading from LDoms 1.0 to a subsequent release.
Posted at 11:22PM Dec 02, 2007 by Eric Sharakan in Sun | Comments[1]
Open xVM Launches
OpenxVM has been launched. Read Steve Wilson's blog entry for the details.
Posted at 05:57PM Nov 14, 2007 by Eric Sharakan in Sun | Comments[0]
Control domain reconfiguration in LDoms 1.0.1
This note explains how control domain reconfiguration works in LDoms 1.0.1, in contrast to how it functioned in LDoms 1.0. There were severe limitations placed on control domain reconfiguration in LDoms 1.0, which have been addressed in the 1.0.1 release:
- The control domain could only be reconfigured when running in the "factory-default" configuration. Once reconfigured, if subsequent changes were desired, one had to revert back to factory-default and re-apply the initial changes as well as any new ones.
- The only way to instantiate a newly reconfigured control domain was by downloading the new configuration to the SP, and then power-cycling the box.
When reconfiguring the control domain under LDoms 1.0.1, the LDom Manager enters "delayed reconfiguration" mode the first time it's asked to do something that can't be immediately instantiated (i.e. just about anything other than cpu DR and adding or removing disk volumes). Once in this mode, all subsequent operations are pended in the hypervisor until the control domain reboots. This delayed reconfiguration capability actually existed in LDoms 1.0, but could not be applied to the control domain because rebooting the control domain required a full powercycle of the system to make sure the I/O subsystem was properly reset, causing the loss of any pending operations queued up in the hypervisor.
LDoms 1.0.1 introduces the ability to soft reset the I/O subsystem, allowing the control domain (or any I/O domain) to reboot while the rest of the system stays up. This in turn allows delayed reconfiguration to work on the control domain.
Utilizing delayed reconfiguration mode for control domain reconfiguration also means the reconfiguration can take place at any time, not just when running in the factory-default configuration. This allows the control domain to be reconfigured as many times as needed without having to revert to the factory-default configuration and start over each time.
To facilitate control domain reconfiguration under LDoms 1.0, the LDom Manager ran in a special "config mode" when in the factory-default configuration. In this mode, all reconfiguration requests were simply queued up within the LDom Manager, so that the new config could be downloaded to the SP when ready, and instantiated by power-cycling the box. This mode is still utilized in Ldoms 1.0.1 on UltraSPARC T1 based platforms (when booted into its factory-default configuration). This is to support non-LDoms legacy compatibility mode, since these platforms initially shipped before the advent of LDoms technology. On these systems, the first control domain reconfiguration has to be done the same way it was for 1.0.
All subsequent control domain reconfigurations (and all control domain reconfigurations on T2 based and all future LDoms-supported platforms, as they do not utilize config mode) can be accomplished via delayed reconfig operations followed by a simple reboot of the control domain.
In summary, under LDoms 1.0.1, you can now reconfigure your control domain whenever you want, as many times as you need, without having to revert to factory-default and re-apply all your previous changes, without having to save the configuration to the SP, and without having to power-cycle the box (except for the first-ever reconfiguration on a T1 based platform).
The ability to reboot the control domain without the box power-cycling (aided with some magic I'll leave to Narayan to describe) deserves a little elaboration: it means you can truly reconfigure your control domain even with active guest domains running! The guest domains stay up during the ensuing control domain reboot; VIO services are pended & automatically re-established as the control domain comes back up.
One very important note about saving your domain configuration to the SP: just because you no longer _need_ to save the new configuration to the SP before rebooting the control domain to affect a reconfiguration, doesn't mean you _shouldn't_ save it; you absolutely should! It's _strongly_ recommend to always save any new configuration you create to the SP, otherwise if the system were to lose power, it would revert to a previously saved state (or to factory-default), which is almost certainly _not_ what you want. BTW, you can safely save your configuration even if there are delayed reconfig operations pending; in this case, the configuration that gets saved is the pending one.
Posted at 11:10PM Nov 01, 2007 by Eric Sharakan in Mac | Comments[0]
Leopard upgrade experience
I just upgraded my MacBook Pro laptop to Leopard (aka OS X 10.5). Here are my experiences so far:
- I ran a full backup first (doesn't everyone?), then performed the default upgrade procedure.
- On first Leopard boot, Spotlight started re-indexing both my startup drive, and my backup (external FireWire) drive, even though I had previously disabled Spotlight on the external volume. I had to disable it again. I then unmounted the external drive. Spotlight continued to index my internal drive, but starting reporting something ridiculous like -5234923 hours to go. Sure enough, next morning it had still not completed (it said 28% done by then - this is on a 120GB disk with 100GB used). I rebooted, and Spotlight finished indexing in about 10 minutes.
- My login keychain was messed up. Apparently it didn't like that my keychain had the name of my userid; it insisted on using the keychain named "login", which was empty. Running Keychain First Aid (under the Keychain Access menu) solved the problem, migrating all my keychain data over.
- 1passwd didn't work initially. Turns out I had an older version installed. When I upgraded to 2.5.0, it started working again. Others are apparently still having problems, see here.
- X11 is now installed by default. It noticed I had a custom ~/.xinitrc file and warned me about it, asking me what to do (I said to leave it alone). But there are problems. There is now an X11.app in /usr/X11/. The Tiger version was in /Applications/Utilities/, and remained there after the upgrade. There also seems to be some detritus left over in /usr: in addition to the directories /usr/X11/ and /usr/X11R6/ (a symlink to /usr/X11/), there is also a /usr/X11R6 1/, which contains its own include, lib & man subdirs.
When launching X11, I now wind up with _two_ X11 icons in my Dock. One of them blinks for a long time, then shows "Application Not Responding" when ctrl-clicking the dock icon; it can't be killed (though it goes away when I quit the original X11).
Plus there are two other mysteries: it takes two clicks of the dock icon to switch to an X11 window (and Cmd-Tab to X11 doesn't do the right thing either). Worst of all, three button mouse emulation doesn't work correctly. In Tiger, option-click emulated middle-click and Cmd-click emulated right-click. Now, option-click emulates right-click, and Cmd-click emulates Meta-middle-click. There seems to be no way to emulate a simple middle-click. Since that's how you paste in X11, this is a real problem. This works fine with a real mouse. - Mail Act-On & MailTags plugins are both incompatible with Leopard. Actually, Mail Act-On can be re-enabled, look here for details.
- The Cisco VPN client we use to login to Sun's network works fine under Leopard. This is the first time a major OS X upgrade has not also required a new VPN client. I have heard there are conflicts with the "Back to My Mac" functionality of .Mac, so I went ahead and disabled it as a precaution.
- Mail.app no longer breaks long URLs for other mail clients by adding gratuitous spaces. Hallelujah!!
- iCal now has a preference to set an alarm on new appointments by default. No more need for third party solutions like iCalFix.
- Quicksilver (web page currently not responding) always displays its icon in the dock, even if I tell it not to. Also, the two items it puts into the system-wide Services menu disappeared for quite some time (including through a couple reboots and rerunning of the Quicksilver setup), but they eventually returned.
UPDATE 1: An new version of quicksilver which addresses the dock icon issue can be found here.
- Spaces is great. I was tired of VirtueDesktop's memory leaks & crashing. I just wish it allowed more hotkey options for activating its multi-desktop view (one non-obvious tip: you can set a corner to activate Spaces, but you do it from the Exposé Pref. pane). I also wish I could name the spaces (at least for the menubar pull-down).
- I haven't played with Time Machine; it looks to be useless for me (and perhaps most laptop users), as it requires a second drive to be attached at all times.
- UPDATE 2: I can now with confidence state that the AirPort connection/stall problems plaguing me since upgrading to 10.4.10 are completely resolved in Leopard!! This has been a long, frustrating problem for many laptop users since 10.4.10 came out. Although I'm annoyed at Apple for how long they let this drag on, I'm glad it's over for me, and hope they release a fix for those 10.4.10 users not planning to upgrade.
One interesting change: now when I press the AirPort menubar icon (which is what used to be needed to get the AirPort connection to start up again with this problem in 10.4.10), instead of a delay before anything happens, the menu drops down immediately, with the top line initially saying: "AirPort: Scanning..." for a second or two (in a greyed-out font), before switching to "AirPort: On". - UPDATE 3: Salling Clicker stopped working for me; it said Bluetooth wasn't available. There is a preliminary 3.5.1 update available which solved the problem for me.
Posted at 12:03AM Oct 28, 2007 by Eric Sharakan in Mac | Comments[9]
Project Virginia? xVM? Whither LDoms?
There's been some understandable confusion regarding the re-branding of Sun's various virtualization technologies that's currently underway, specifically as it relates to our two Hypervisor based technologies; the one on our x64 systems (originally based on Xen); the other on our SPARC CMT systems (i.e. LDoms).
Part of the confusion stems from the fact that we recently started using the "xVM" moniker for our x86 based hypervisor technology, since we did not meet the requirements set out by Citrix to continue using the Xen brand. But then, as part of an effort to re-cast all our virtualization technologies under one systems management umbrella (I believe this is what Project Virginia is about), it was decided to expand the scope of technologies included under the xVM brand. The Sun xVM Product Family now encompasses both LDoms & our x86 based hypervisor. Read this blog entry from Marc Hamilton for more details.
Unfortunately, there is still plenty of collateral out there that either implies or explicitly correlates xVM solely with our x86 hypervisor technology. This requires our readers to carefully analyze any information describing xVM, and properly determine from the context whether it's a reference to the overall Sun xVM infrastructure product family, or to one of its members, be it one of the underlying virtualization technologies, or the systems management piece layered on top.
Speaking of systems management, one of the key pieces of the Sun xVM infrastructure is Sun xVM Ops Center, which is under active development by our SysNet organization. Its aim is to provide a systems management framework that truly encompasses all our virtualization technologies. Read more about it in Marc's blog.
So what does this mean for LDoms? Other than the planned support for it in Sun xVM Ops Center, absolutely nothing. Unchanged are the LDoms technology, product, roadmap or name. This includes our plans to support 3rd party management tools via SNMP, our planned extensions to libvirt, or through our own XML-based control interface currently under development. That's a topic for another posting...
Posted at 11:15AM Oct 24, 2007 by Eric Sharakan in Sun | Comments[1]
UltraSPARC T2 & LDoms 1.0.1
Sun is officially announcing the first products in its UltraSPARC T2 (US T2) based platform lineup today, the T5x20 series. You can read all about the details here and here. There are two big stories here related to our Logical Domains technology. For those of you who are new to my blog, Logical Domains (LDoms for short) is the name of Sun's virtualization technology for our SPARC CMT platforms that allows multiple operating systems to run concurrently on a single system.
These products represent the first of our CMT based platforms that are shipping with LDoms technology pre-installed from the factory. All future CMT servers from Sun will ship with the ability to run Logical Domains out of the box. This includes the LDoms-enabled hypervisor (our LDoms hypervisor runs on bare metal, and is embedded in the firmware of the platform), all the necessary Solaris components, and the LDoms Manager package (which is what my team works on).
This further represents the introduction of version 1.0.1 of LDoms technology. Besides support for the new US T2 based platforms (and a slew of bug fixes), this release supports the ability to reset any domain, even one which owns physical I/O devices, while all other domains continue to run. Even the control domain, i.e. the one on which the LDoms Manager runs, can reboot while all other domains stay up. This represents a major step forward in terms of RAS capability for LDoms.
As of today, LDoms version 1.0.1 is only available pre-installed on our newly announced US T2 based servers. Stay tuned here for information on the impending availability of this upgrade on our existing US T1 based platforms!
UPDATE: LDoms version 1.0.1 is now available for download here. This includes the firmware updates for US T1 platforms.
WARNING: There are two important caveats when upgrading from LDoms 1.0 to LDoms 1.0.1:
- Configurations saved to the service processor under 1.0 are not usable under 1.0.1. The LDoms 1.0.1 Administration Guide describes the upgrade procedure that needs to be applied to work around this. Part of this procedure needs to be carried out BEFORE performing the actual upgrade!
- You must upgrade both the firmware and LDom Manager components at the same time.
Posted at 05:18PM Oct 11, 2007 by Eric Sharakan in Sun | Comments[4]
VIO device renaming by LDom Manager
[This is the first in a series of entries I'll write about tips, tricks & other issues with the LDom Manager.]
The LDom Manager allows you to specify a name for each VIO client & server instance you configure. Currently (i.e. in LDoms 1.0 and the upcoming 1.0.1 releases), this information is not stored as part of the machine description (MD) for the associated guest domain. Instead, the device name to instance mapping is stored in the LDom Manager's private constraints database, which is itself persisted as a simple XML file in the control domain's filesystem.
There are cases where the information in the constraints database doesn't match that of the running system, and in those cases, the LDom Manager, on startup, will apply a canonical name to any VIO device(s) for which no name mapping is available. The two main reasons this can happen are:
- Loss of the constraints database file (as a result of an OS upgrade, for example)
- Reverting the system to a configuration stored on the SP containing a different set of VIO devices than the currently running Config.
When the LDom Manager first starts, if it can't find a mapping for a given VIO device in its constraints database, it applies a canonical name using the following heuristics:
For VIO clients: <type><instance #>, where <type> is either "vnet" or "vdisk", and the instance # is incremented for each additional device of type <type> encountered
For VIO servers: <domain-name>-<type><instance #>, where type is "vds", "vsw", or "vcc"
The Ldom Manager's renaming of VIO devices never affects the actual binding of VIO devices to instances in the OS, nor the binding of VIO clients to servers; everything continues to operate normally. The impact is in how the LDom Manager references VIO devices for display and reconfiguration by the user.
There is, however, one more serious problem to note: if a VIO device is configured using a name that matches a potential canonical name, and the LDom Manager subsequently attempts to use that same canonical name on another VIO device, it'll cause the LDom Manager to abort on startup, and eventually enter maintenance mode. This failure can be identified by this message appearing in the LDom Manager's log file:
Assertion failed: 0L != clientp->published_name, file vio_classes.c, line 2471
To work around this, so the LDom Manager can start, its constraints database (stored in /var/opt/SUNWldm/ldom-db.xml) must be hand-edited to rename the offending VIO device to one that doesn't collide with the canonical name namespace. There is a bug open in our bug tracking system for this problem; it's CR #6571091.
We plan to address these issues in an upcoming release (after the 1.0.1 release), by eliminating the need for the LDom Manager to rename VIO devices altogether.
Posted at 09:33AM Aug 21, 2007 by Eric Sharakan in Mac | Comments[0]
Updated Beginners Guide to LDoms
Tony Shoumack has just posted an updated version of the excellent Sun Blueprints™ document Beginners Guide to LDoms. This is the perfect resource and reference document for those folks new to Logical Domains. It provides both a conceptual background of the technology, and specific guidance on configuring LDoms. This represents a significant update to the previous version, incorporating feeedback from LDoms experts within Sun as well as our customers. It is also now current with the 1.0 release. If you had downloaded the previous version (which was targeted at our old, pre 1.0 release candidate builds of LDoms), please update to this latest version.
Posted at 07:55AM Jul 24, 2007 by Eric Sharakan in Sun | Comments[0]
LDoms 1.0 is out!
As Ash and Narayan have already announced, Logical Domains 1.0 is now a reality!
Posted at 11:57PM May 01, 2007 by Eric Sharakan in Sun | Comments[0]
Logical Domains 1.0 Early Access Now Available!
It's here! LDoms 1.0 Early Access has just been posted to the Sun Download Center. This 68+ MB tarball includes new firmware images for T1000, T2000, Netra T2000 Server & Netra CP3060 Blade systems; mandatory patches to Solaris 10 11/06 (aka Update 3); the unbundled LDoms Manager & security packages; documentation; and an install script. In addition, a Sun BluePrints article entitled Beginners Guide to LDoms: Understanding and Deploying Logical Domains is available separately.
This is Early Access software. As stated on the download page: "This is an unsupported technology preview of Logical Domains software, providing early access to those interested in the technology". In other words, there are known bugs, limitations & rough edges (all of which we're working hard to eliminate for our 1.0 General Availability release later this spring).
As the LDoms Manager team lead, I'm very excited to help make this technology available for folks to try out. Everyone on the LDoms team is thrilled to have reached this major milestone, and we all look forward to hearing about the experiences of all our early adopters. We welcome your feedback (ldoms-feedback AT sun DOT com) to help us improve the product, and you can participate in the discussion at the LDoms SysAdmin Hub on BigAdmin.
Posted at 08:53AM Mar 06, 2007 by Eric Sharakan in Sun | Comments[0]
I've been Tagged
So I've joined the ranks of those tagged. Thanks Jason. The rules are simple: I post five things most people don't know about me, and then I tag others to do the same. So here goes:
1) I am a licensed (but currently inactive) pilot. I passed my checkride in 1989. Unfortunately, lacking sufficient time or finances to maintain my proficiency, I stopped and have not flown in the past 15 or so years. I still yearn to get back up in the air.
2) I have played several instruments. I played french horn in my junior high school band. It didn't last long; the difficulty lugging the thing around meant I didn't do much practicing at home. I also took piano lessons as a kid, but never got very far with those either. More recently, I took an adult Ed. class to learn to play harmonica. Now there's a nice portable instrument! I still pick it up every once in a while and play a blues lick or two...
3) I saw Queen in concert in Buffalo, NY in 1979. My college roommate camped out for tickets, and we wound up with 8th row seats (plus the stage was jutting out four rows where we were sitting). It was one of the best concerts I can remember (there were a few I barely remember, but that's another story); Freddie Mercury was an amazing performer. RIP.
4) My nickname in college was Flounder. From the movie Animal House. Because my roommate decided to give me a nickname that bore no resemblance to my personality. This is the same roommate who got me 8th row Queen tickets, so I cut him enough slack.
5) I am a member of the Eric Conspiracy Secret Laboratories:
Okay, Ash, Narayan & Josh, consider yourselves tagged.
Posted at 09:21PM Jan 20, 2007 by Eric Sharakan in Personal | Comments[0]
Superhero Test
Which Superhero am I?[Read More]
Posted at 08:43AM Jan 04, 2007 by Eric Sharakan in Personal | Comments[0]