Tuesday November 11, 2008

As I mentioned, we released a simulator as part of the Sun Storage 7000 Unified Storage System (*) launch event yesterday. I geeked out and took it for a quick spin today to get a first-hand view of its capabilities. Short summary: this is a really cool way to play with one of these new storage appliances from the comfort of your existing desktop or laptop with no extra hardware required. It's all virtual, baby. Check this out...

The simulator (download it from here) is basically a VMware virtual machine that has been pre-loaded with the Unified Storage System software stack and configured with 15 virtual 2 GB disks. Here is what I did to try it out.

First I downloaded the simulator and booted the virtual machine using VMware Fusion on my MacBook Pro. The boot is straightforward with one tiny exception. At one point I was asked for a password, which stumped me since there is no mention in the instructions on the download page about supplying a password. As it turns out, this is where you specify the root password for the appliance. Pick something you will remember since you will need it to access the appliance's administrative interface later.

Once the appliance has booted it will display some helpful information about next steps, the most important of which is to access the appliance via its web interface to configure it for use. With the appliance running in a virtual machine on my Mac, I used the Safari web browser under Mac OS X to contact the appliance at the hostname (or IP address) supplied by the DHCP server when the appliance booted and using the port number shown in the documentation (port 215.)

I then logged into the appliance using the root password I had specified earlier. The BUI then walked me through a set of configuration steps that included networking, DNS, NTP, and name services. The process was simple and quick since the defaults were correct for most questions. Once I finished the basic configuration, I reached the following screen:

This is where it starts to get fun. This interface helps you choose which replication profile makes most sense for the storage that will be managed by your appliance. Each option is ranked by availability, performance, and capacity. The pie chart on the left illustrates how storage will be allocated under each scheme. In the case of Double Parity RAID, you can see that data and parity are placed on 14 disks and the last disk is held as a spare. In contrast, when I selected the "Striped" option, I saw this:

You can see that this strategy delivers maximum capacity and also great performance since I can get all those spindles working at once on my IO requests, but at the expense of low availability, which might be perfectly fine for a scratch file system. I opted for a Double Parity RAID scheme for my filesystem.

Once I configured the storage I visited the Shares tab and created an NFS filesystem called "ambertest." Again, this was straightforward. Straightforward enough that I forgot to take a screenshot of that step. Sorry about that.

I then mounted my new NFS filesystem under Mac OS X:

% mount -t nfs ip-address-or-hostname-of-the-virtual-storage-appliance:/export/ambertest /Volumes/amber

As a test, I copied several directory trees from my local Mac file system into the NFS filesystem exported by the virtual appliance and also ran several small test scripts to manipulate NFS files in various ways to generate load so I could play with the Analytics component of the appliance.

This is the part that should be illegal. Because the appliance stack is built on top of Solaris, DTrace is available for doing deep dives on all sorts of usage and performance information that would be interesting to an administrator of such a storage appliance. Here is one silly example that will give you a little flavor of what you can do with this capability. It is a much wider and deeper facility than this simple example shows.

Consider the following page:

I used the Analytics interface to graphically select a metric of interest--in this case, number of NFS v3 operations per second broken out by filename. The main graphical display shows how that metric varies over time and allows me to move backwards or forward in time, look at historical data, zoom in and out, pause data collection, etc. The lower pane in this case shows me the directories that are currently being touched within the NFS filesystem by ongoing operations. Individual files are listed in the small pane to the left of the timeline.

When I selected the pltestsuite line in the bottom pane, the timeline updated to show me exactly when in time the operations related to the files in that directory actually occurred. Since my test was a simple 'cp -r' of a directory tree into the NFS-mounted directory on the Mac, the display shows me when the files within the pltestsuite directory were cancelled and the NFS load generated by that part of the overall copy operation. I can easily see which file activity is contributing to load on the appliance--very useful for an administrator, for example.

In addition to examining NFS operations by filename, I can break down NFS statistics by type of operation, by client, by share, by project, by latency, by size, or by offset. I can do the same for CIFS or HTTP/WebDAV requests. NFS v4? No problem. Network traffic, disk operations, cache accesses, CPU utilization? It's all there in one easy-to-use, integrated web-based interface. To me, the Analytics are one of the coolest parts of the product since observability is often the first step to good performance and effective capacity planning.

If you are curious about the capabilities of the Sun Storage 7000 Unified Storage Server line, I do recommend trying the simulator. In addition to offering an effective way to explore the product without buying one (we expect you'll want to buy one after you finish :-) ), it is interesting to see how desktop virtualization neatly enabled us to create this simulator experience.

Monday November 10, 2008

While I'm mostly a "compute" guy, I know very well that at the end of the day servers are really just data manipulators and that for our customers storage plays an absolutely central role in their businesses. Which is why I've watched with fascination as some of Sun's best engineers banded together with the deliberate intention of rethinking how storage products are built, how they are priced, and what they can do. Today, Sun has announced the fruits of their labors--the Sun Storage 7000 Unified Storage Systems. Or better, check out Mike Shapiro's blog for his thoughts on what he and the rest of the team have accomplished. And be sure to check out the Sun Unified Storage Simulator Mike mentions if you want an easy way to see what all the hoopla is about.

View these products as an example of what Sun can do as a systems company when we bring our wide array of expertise and intellectual property together to create unique solutions that we believe customers will value and our competitors will loathe, especially those in the business of charging premium prices for capabilities we can now deliver at an entirely new price point.

In this case, we've integrated Sun server technology, flash memory and storage together under the control of a software stack based on OpenSolaris software innovations like ZFS and DTrace to create a set of NAS appliances that offer both extreme performance as well as built-in functionality other storage vendors either don't have or treat as expensive add-ons. We've talked a lot about Open Storage recently and this new product line is a poster child for what we've been talking about.

The capabilities of these new systems is amply documented elsewhere, so I will not delve into details here, except to mention that an enormous amount of engineering work and talent has been applied to this product line, the result being a system that offers a simple appliance-like installation and management experience; unmatched observability of overall system status as well as much more detailed drill-downs based on DTrace technology; high availability; and hybrid storage pools--a transparent combination of DRAM, FLASH and HDD that delivers significant read/write acceleration and some amazing performance numbers.

I could blather on about how great these products are, but thankfully you don't need to take my word for it. What matters is how these systems perform in particular deployment environments. These storage systems are available through Sun's Try and Buy program, so you can judge for yourself. We pay shipping, you put the system through its paces and make your own decision. You don't like it, we pay return shipping. You buy it, we give you a big discount. Seems like a deal to me.

Sunday November 02, 2008

My wife gave me an Amazon Kindle for my birthday yesterday, something I've been eyeing for awhile. While the device itself is nicely done, the Kindle experience has so far disappointed with its lack of interesting content. None of the random assortment of books I have in my "to read" queue are currently available. For example:

• House of Suns, Alastair Reynolds
• To Kill a Mockingbird, Harper Lee
• The Snow Leopard, Peter Matthiessen
• The Andromeda Strain, Michael Crichton
• The Great Train Robbery, Michael Crichton
• The Alan Lewrie Naval Adventure series, Dewey Lambdin

The Lambdin item highlights a particularly annoying problem. I found several additional series for which only a few books were available in Kindle format and often not the first in the series.

For a bookseller it should be a red flag when a person with poor impulse control regarding book purchases is stymied in his attempts to scratch his book itch. Amazon may not be worried about this since, after all, every single book mentioned above is available in hardcopy from Amazon. But, see, here's the thing. Amazon has disappointed me. Having invested in the reader and not finding the items I want available in that format, why should I then reward Amazon by buying the hardcopy versions from them anyway? They'd win either way and that isn't right. I should encourage them in whatever way I can to aggressively increase the content available for Kindle.

Which leaves me with the following book purchase algorithm. If the book is available for Kindle, buy it. If not and it can be found at another bookstore--Borders is my favorite-- then buy it there. Otherwise, if available in hardcopy from Amazon, buy it there. Following that logic, House of Suns is the only book I would buy from Amazon in hardcopy. I would buy the other dozen or so from Borders or another bookseller.

Amazon, are you listening?

Saturday October 25, 2008

Using flash memory cards for long-term photo storage is a really bad idea. I didn't realize just how bad until I heard a recent talk on flash technology, which included reliability statistics for the two main varieties of flash, MLC and SLC.

Before sharing the numbers, you need to know that flash memory reliability is generally measured by two quantities: endurance and retention. Endurance measures the number of write cycles each sector of the memory can handle before serious or fatal errors begin to occur. Retention measures how long a flash memory part can be expected to hold its data reliably before attempts to read the data will fail. Of course this is all statistical, but nonetheless the numbers will give you a rough idea of the lifetimes of these memory devices.

Multi Level Cell (MLC) is the consumer-grade version of flash memory. It has an endurance of about 1000 write cycles. Think about how often you fill and erase your memory cards to decide if this number bothers you. If you are like me, you tend to keep and use memory cards for a long time, which means 1000 cycles is not a very comforting number. As an aside, I've noticed that my Canon G9 lets me erase a memory card with either a standard formatting operation or what is called a "low-level formatting." Since the low-level operation takes considerably longer than the standard version, I suspect it erases every sector on the card, which imposes unnecessary wear, i.e. more cycles are consumed from the 1000-cycle budget for these memories. From now on I'll be using the standard formatting option, which I suspect only affects metadata blocks. If I recall correctly, my Canon 10D just offers one formatting option. It is very fast, so I am guessing it does not erase every data block on the card.

Let's now talk about data retention.

The characteristics of Single Level Cell (SLC) are considerably better, but you still probably should not use these devices for long-term storage. While endurance for SLC is about 100,000 cycles, data retention is about 10 years. A decade is good, but not for archival purposes. And don't count on 10 years if you have heavily used the card---in which case, retention time can be considerably shorter.

While clearly SLC is more appealing than MLC, it is about 4X more expensive than MLC and about half as dense. I am hoping that the so-called "professional" flash cards sold as digital film are built using SLC, but I have found no way to determine that from vendor websites. If anyone has any concrete information on the products made by the major flash memory card vendors, please share.

Photography aside, I think it is safe to assume those ubiquitous USB dongles that are given away at conferences and others events are made with MLC. Keep than in mind when deciding what you store on these devices and how long you need it to be recoverable.

For an interesting discussion of retention and endurance along with several examples of how to determine suitability for use in several non-photographic circumstances, see this document titled Practical Guide to Endurance and Data Retention [PDF].

And to find out who is working to create ultra-reliable SLC flash memory parts, read this short article.

Wednesday September 24, 2008

The cloud computing discussion at this week's High Performance on Wall Street conference stimulated some questions in my mind about the future of cloud computing.

Cloud Computing is currently at a very early stage in that clouds are just starting to appear, each with its own approach and with people now starting to explore how to use cloud infrastructures like Amazon's EC2, AT&T's Synaptic Hosting and others to advantage--both academic experimentation as well as a leveraging of these rented resources by startups, etc., as a fundamental part of their business infrastructure.

Most important, however, is the fact that cloud supply currently far exceeds cloud demand as one would expect in the early adoption stage of a highly-hyped concept. Because of the over-capacity of available resources, one does not currently need to worry about whether cloud resources will be available when you need them. But what happens when that changes?

Speakers at the High Performance on Wall Street conference were sure that as clouds became much more like commodities and resources became more constrained due to increased demand, then free market mechanisms like futures markets would evolve to mediate access to these resources.

If that is true, and it does sound reasonable, how will that change who uses the cloud and how they use it? Will startups be able to build their businesses with cloudy back ends if they must bid for access and utilization on an open market? It isn't clear. Right now, utilization costs for a startup using cloud resources will fluctuate as their business needs fluctuate--more customers, more business, generally more processing and higher rental costs on the cloud. The increased complexity and unpredictability of fluctuating cloud infrastructure costs in addition to fluctuations due to changing business demands may reduce the attractiveness of the cloud approach for these businesses.

Another question. As these shared resources become scarcer, might there be an increased risk that firms could use denial of cloud resources as a strategic weapon against competitors by pre-purchasing significant cloud resources in advance of a competitor's planned use of that resource? Perhaps somewhat farfetched...or perhaps not.

It will be interesting to see how cloud computing evolves as it matures and to see whether these kinds of problems will arise in practice or not. It does seem apparent, however, that the current nascent movement towards cloud computing is bound to get much more complicated in the relatively near future.

Wednesday September 03, 2008

The fifth meeting of NEOSUG (New England OpenSolaris User Group) will be held next Wednesday, September 10th at Sun's Burlington, Massachusetts site. The featured speaker will be Jim Mauro, who will talk about Solaris 10 and OpenSolaris Performance, Observability, and Debugging. Full details below.

The New England Open Solaris User Group (NEOSUG) Meeting

Topic for this meeting:

Solaris 10 and OpenSolaris Performance, Observability and Debugging (The Abridged Version)

Who should attend? : UNIX Developers, Solaris users, System Managers and System Administrators.

AGENDA:

New England OpenSolaris User Group Meeting (NEOSUG)
Sept 10, 2008 6:30-9:30 pm (registration opens @5:30)
Sun Microsystems
One Network Drive
Burlington, MA

5:30-6:30: Registration, Refreshments
6:30-6:40: Introductions, Peter Galvin
6:40-8:30: Solaris 10 and OpenSolaris Performance, Jim Mauro, Sun Microsystems
8:30-9:00: Questions and Discussion

Please RSVP at : https://www.suneventreg.com//cgi-bin/register.pl?EventID=2341

TALK DESCRIPTION:

The observability toolbox in Solaris 10 and OpenSolaris is loaded with powerful tools and utilities for analyzing applications and the underlying system. Solaris Dynamic Tracing (DTrace), allows you to connect the dots between the process and thread-centric tools, and the system utilization tools, and get a complete picture on what your applications are doing, how they are interacting with the kernel, and to what extent they are consuming hardware resources (CPU, Mem, etc).

This two hour talk walks through the tools, utilities and methods for analyzing workloads on your Solaris systems.

NEOSUG BIOs:

Peter Galvin : Chief Technologist, Corporate Technologies Inc.
Peter Baer Galvin is the Chief Technologist for Corporate Technologies, Inc., a systems integrator and VAR, and was the Systems Manager for Brown University’s Computer Science Department. He has written articles for Byte and other magazines. He wrote the Pete’s Wicked World and Pete’s Super Systems columns at SunWorld Magazine. He is currently contributing editor for SysAdmin Magazine, where he managed the Solaris Corner. Peter is co-author of the Operating Systems Concepts and Applied Operating Systems Concepts texbooks. Blog: http://pbgalvin.wordpress.com

Jim Mauro: Principal Engineer in the Systems Group, Sun Microsystems, Inc.
Jim Mauro works on improving delivered application performance on Sun hardware and Solaris. Jim's recent project work includes Solaris performance as a guest operating system on Xen and VMware virtual machines, Solaris large memory page performance, and Solaris performance on large SPARC systems. Jim co-authored Solaris Internals (1st Ed, Oct 2000), Solaris Internals (2nd Ed, June 2006) and Solaris Performance and Tools (1st Ed, June 2006).

ug-neosug mailing list: ug-neosug@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/ug-neosug

Sunday August 03, 2008

According to Wordle, here is what I've been talking about lately on the Navel...

Via Joan, Wordle is a cool cloud tool written by Jonathan Feinberg, a member of IBM Research’s Collaborative User Experience Group. Have fun changing colors, text directions, fonts, etc. Make your own beautiful word cloud.

Monday July 28, 2008

My friends Jamie and Lori left Sunday on their annual month-long sailing trip from Boston to the Canadian maritime provinces. As usual, the trip begins with an open ocean sail across the Gulf of Maine directly from Boston to Cape Sable, Nova Scotia.

This year they are carrying a Spot satellite messenger on board. This neat little device can report its location every ten minutes, allowing others to track their progress over the course of the trip. It can also transmit a 911 emergency message, if needed. It is quite a nifty device and surprisingly inexpensive given its capabilities. Jim Gray should have had one of these on his boat last year when he went missing. Of course, boating is only one application--I can imagine this would be useful in any number of situations in which people may need to be rescued.

Here is a screenshot I took this morning of their progress towards Nova Scotia:

You can also view the live interface here.

Monday July 14, 2008

Each year Sun holds two internal technical conferences that bring together all of Sun's most senior engineers (Principal Engineers, Distinguished Engineers, and Fellows) at an offsite location for two days of technical presentations and networking. The conferences alternate between a smaller and larger venue. The smaller conference (Technology@Sun) accommodates just the PEs, DEs, and Fellows. At the larger conference (Innovation@Sun) we are able to include approximately 50 other attendees from around Sun. This year, we will choose them based on the merits of their poster or demo proposals.

The deadline for submissions closed last Friday and we received a total of 227 technical proposals from around the company. The program committee will now review all proposals and decide who will be invited to attend. We are also busy planning the rest of the technical content and events for the conference, which promises to be at least as interesting and useful as prior events.

Sunday June 22, 2008

[WARNING: This blog entry is primarily about vomit.]

I flew home from Dresden via Frankfurt on Friday, boarding LH 422 for the eight-hour flight to Boston. Sitting just forward of me was a coed group of boisterous college-aged kids who mostly quieted down once they had stowed their gear and found their seats. With the exception of Oscar, who was sitting two seats in front of me across the aisle.

Oscar's behavior went beyond boisterous, well into the realm of obnoxious. He was loud, he was rude, he was up out of his seat repeatedly, unable to sit still. I'm a seasoned air traveler and very used to ignoring the various mis-behaviors of my fellow passengers, but there was something about Oscar I found particularly grating. He persisted in this behavior up until the first meal service, at which point some god out of some pantheon smote him but good and he threw up all over himself in spectacular fashion. When he stood up after being directed aft by an attendant, I saw that he was literally covered in vomit--all over his shirt and down his pant legs and in considerable quantity. And there was apparently enough left over to have covered the seat as well, since the attendant later placed a pillow on it to make the seat usable again.

Peace reigned for most of an hour while Oscar presumably cleaned himself up. He then reappeared--shirtless. Perhaps a little quieter, but still with plenty of swagger. Which I must say I viewed with some amusement since his cool demeanor did not jive with the fact that from the rear one could see that the entire crotch of his pants was completely packed with now-drying vomit which he had apparently missed in the clean-up effort. Ah, I thought to myself. This was schadenfreude.

Eventually one his of traveling companions gave Oscar a t-shirt and he fell asleep sitting on his vomit-laden pillow for most of the remainder of the flight. Later, mention of a $120 bar bill lead me to conclude that Oscar had had far too much to drink prior to boarding the flight.

Thursday June 05, 2008

[Thanks to Michal Bielicki for pointing out an error in the original post. The correct name of the Sun Studio package is "ss-dev". I have fixed the text and graphics below to reflect this.]

If you want to install either the Sun compilers and developer tools or the GNU developer tools onto OpenSolaris 2008.05, I summarize the process below. Currently, the procedure is somewhat less than obvious, hence this blog entry.

First, start the package manager: System -> Administration -> Package Manager. You will see the following:

Looking for the compilers, you might select Developer Tools. However, doing so will show the following:

Unfortunately, the compiler packages were not categorized correctly and therefore they do not show up under this package category. We will fix this, but the good news is that the packages are available, if you know where to look.

With the 'All' category selected, enter 'gcc' into the search field. You do not need to press return. The interface is slow, but it will eventually update to show the following:

To install the GNU tools, select the gcc-dev package and click on Install/Update in the toolbar. Once the download (about 120MB) and the installation complete, you can open a new terminal window and type 'gcc' to verify the software has been installed correctly. The installation script has created links from /usr/bin/gcc to /usr/sfw/bin/gcc as a convenience.

To find the Sun compilers and tools, type "ss-dev" ("ss" stands for Sun Studio, which is what we call our compiler and tools suite) and you will see the following:

Select the "ss-dev" package and then click on Install/Update on the toolbar. Once the download (over 600MB) and the installation complete, you can open a terminal window and try the 'cc' command. Don't panic when it fails. As you can see below, the compilers have been installed in /opt. You will need to either modify your startup files to include this directory on your execution path, or create the appropriate links from /usr/bin into this directory. I've been told we will fix this inconvenience soon. A wonderful benefit of network-based package management is that we can fix this relatively quickly and then subsequent downloaders of the package will see the new behavior automatically.

Wednesday June 04, 2008

I've just been playing with Goosh, a web-based, search-oriented command line interface to Google written by Stephan Grothkopp and available at http://www.goosh.org/. As Stephan points out, this is the unofficial google shell and not a google product.

Thanks to Monty for the pointer. Goosh looks like this:

Or this:

Friday May 30, 2008

The headline published by NASA Science News was "Strange Ring Found Circling Dead Star," a headline practically designed to set any true scifi fan's heart a-flutter. Could it be? Wouldn't it be wonderful if? Is it a...Ringworld?

Alas, it was not to be. The ring is gaseous and, while not explained, not nearly so interesting as Larry Niven's ring.

I've had a good desktop OS week.

First, I installed the first official release of OpenSolaris on my Apple MacBook Pro, which went flawlessly. It was better than the earlier release candidates and, for those only familiar with Solaris 10 installations, it is an entirely better, more modern and more enjoyable experience.

If you aren't sure you will use OpenSolaris, but are curious about it, then download the LiveCD image and play with OpenSolaris in memory with no commitment to write it to your local disk. Get the OpenSolaris 2008.05 LiveCD image here.

I opted to use OpenSolaris within VirtualBox on my Mac rather than create a separate Bootcamp partition for it. In the past, I've used both VMware Fusion and Parallels Desktop for my virtualized OS instances, but I've switched to VirtualBox. Partly because Sun now owns the technology, but also because it is free. I'm running V1.6 and have had no problems using it with OpenSolaris. Get VirtualBox here.

This was also the week that Apple finally released 10.5.3, a much-needed Leopard update that fixes lots of bugs. In my case I've been able to re-enable turn power management and allow my MBP to sleep again--it now seems to be working properly. Even better, closed-lid mode works again and I am typing this entry using my laptop with an external USB keyboard and the Sun 24" LCD monitor in my office. Apple's Vista versus Leopard TV advertisements are now funny again.

I have indeed had a good desktop OS week.

Thursday May 22, 2008

With President Bush's signature yesterday, the Genetic Nondiscrimination Act (GINA) has become law. This is excellent news for anyone considering genetic testing who might justifiably be worried about their future eligibility for health insurance being effected by the results of their testing.

Let me illustrate by personal example. My family has a history of Hypertrophic Cardiomyopathy (HCM) which causes thickening in the central wall of the heart and which can lead to complications, including sudden death. HCM has been linked to mutations in genes related to the formation of heart muscle. There are now genetic tests available that have a reasonably good chance of finding whether a person carries known mutations related to HCM. To use the test, one family member with known HCM is tested to see if any of the known HCM markers are present. If markers are found, it is easy and inexpensive to test additional family members for HCM. This is where GINA becomes important.

Because of the risk of sudden death with HCM and because HCM may not be initially detectable in children, using such a genetic test on my nieces and nephews would allow their parents to know for sure whether any of the kids have HCM or not and to take appropriate precautions. The problem has been, however, that if a child is found to have the mutations related to HCM, this may be considered a pre-existing condition by insurance companies later in the child's life which may affect their ability to obtain insurance. GINA improves the situation by making it unlawful for insurance companies to discriminate against people based on the results of genetic testing.

Full details on GINA are here.

(2008-05-22 10:37:38.0) Permalink Comments [0]