Neat blogs
Navigation
Editing
statcounter.com
clustrmaps.com
technorati.com
Thursday March 05, 2009 I got an email asking me what I didn't like about the Fishwork's CLI (see Impressions of Sun Unified Storage Simulator). I thought I would answer here.
I'd say the main complaint I had with it would be I couldn't issue 'ifconfig' or 'share'. I was trying to help someone over on a Linux mailing list with an issue (you can find the thread here -- Permission problem with NFSv4 mount and [Fwd: Re: Permission problem with NFSv4 mount]) and I wanted to just dive right in like it was any other OpenSolaris box.
It was actually easier for me to use the web interface than the CLI. Was the CLI that hard to use? No - just different. It was easy enough to figure out how to move about in with a couple of minutes of experimentation. I never pulled out any documentation.
So, it would be easiest for me if the CLI had normal unix commands. That doesn't mean that it would be easiest for everyone. This new product was the perfect time to change the CLI - you do not want local access on a appliance. Within pNFS, we are struggling with having local access for the MDS and DSes. Rob Thurlow is busy working on Proxy IO in case the MDS needs to read/write directly to the DSes (it also is a cheeky way to provide NFSv3 and CIFS access to the community).
So given that you want to deny local access, why not change the interface? And if you are going to change it, why not start off from scratch?
The whole GUI design reeks of object oriented design. Why not also have a CLI that does that?
Hmm, I'm not hitting specifics, which tells me a couple of things:
- I missed some common commands.
- I was quickly able to figure out how to use the new CLI.
- My lack of real frustration tells me that the ease of use outweighed familiarity.
I can't stress that I went into using the sim convinced I was going to hate it. I liked the NetApp sims, in part because I worked on it. But the NetApp sim was in no way as easy to configure or deploy. It took me a couple of minutes to get it up and running on the VMware network and only a couple of more to get it running across the physical wire. In the meantime, I had Debian Lenny not install on both a VMware machine and a mac mini. I finally got it working on an old laptop.
I walked away from using the simulator very happy with my experience. I think I was able to replicate the issue that a customer was seeing - I'm still waiting on a reply from him to confirm that. I'm confident that I can use it to troubleshoot other issues that might be reported on NFSv3 and NFSv4 for the Sun Storage 7000 Unified Storage Systems.
I also know that a lot of effort was put into observability for these boxes - just look at Bryan's entry on Eulogy for a benchmark or Brendan's entry on Unusual disk latency. My take away from both of these is that we can now measure things that other vendors can not. Before Dtrace, you had to systematically add trace points, recompile, QA, and then ship new bits to customer sites. And they may have had to QA the incoming bits. Now you can just start asking the interesting questions right away and getting answers.
Without delving too deeply, I expect all of this to be more integrated with the new UIs. I suspect there is a power here. I'm pretty sure I can quickly dig through this stuff on the web interface. And I'm just as confident that the CLI has the same power, but it just isn't my parent's '>' prompt...
I'll throw in my 2 cents. The Netapp CLI is great for a lot of things, except complex things like managing volume exports, or the editing of some files under vol0/etc. In those cases, I prefer the GUI.
For the Unified Storage product, I liked the GUI much better than the command line interface, and the visualization tools are far superior to Data Fabric Manager/Ops Manager. But, having a simple CLI interface can give me the ability to perform some functions by remote scripting.
Posted by Chris on March 05, 2009 at 09:39 AM CST #
Chris,
I suspect that Sun Support mandated that this new Storage product be a closed system. Otherwise, they expect people like you and I will get into the Solaris kernel and create configuration problems that take hours to detect and resolve.
IN the CLI enter "confirm shell" and you'll get a
"Big Boy Warranty" (you may violate your warranty.
I did that and tried to change the BUI management port from 215 to 80 to get around a nasty firewall. The system ended up changing my hack everytime I restarted Apache. So, I kept digging eventaully triggering a "maintenance" state that would not boot as storage... Oops. I installed a software update prior to hacking and rolled back to that version to keep working on my benchmarking. When update 2 came out it wrote over update 1 and the corrupt config was wpied from memory. I love OS snapshots.
Anyway, you can hack into Solaris. Just have a snaphos of a prior release and understand that they don't ship an install kernel. If the OS gets too corrupt they will replace the boot drives with a clean image. The ZFS pools will be intact and you'll need to re-config to fit your shop. So, 99.999% is NOT likely with such an approach.
It might be wise to add extra SAS drives to the controllers and take mirror images and put them aside for maintenance events and save your config.
Keep sharing your insights on the system. It's a game changer. I need to play with the remote replication features... I think that's the Killer App and NOT Analytics. I love the Analytics but it won't sell 100 units this year. Solid Remote Replication at no extra cost will. Every branch office backs up continously to Corporate and Corporate pushes updates downstream nightly. Amazing cost savings over all the other big players.
Posted by McD on March 05, 2009 at 12:56 PM CST #
Chris,
Volume exports I would do on the command line or as in the case of most of the stuff with a NetApp filer, via 'vi' over a NFS mount.
But I was the one who rewrote most of their exportfs code, so I would be happier with it than most.
What I was trying to say was that the NetApp CLI was easy for a Unix admin to dive right into. And their Filerview stuff was murder. (I'm recalling my introduction to it as a first time filer administrator.) I think in large part that was due to having to offer Filerview because they had just gone multiprotocol and all of the windows admins *hated* the NetApp CLI.
With the Unified Storage stuff, I found the CLI different than I expected and the web admin to be dead simple.
I find the Unified Storage model to be more consistent. The multiprotocol stuff is integrated and not an afterthought. The CLI *does* follow what I've come to expect in other appliance like layers. I.e., I've grown used to it in router CLIs and LOM CLIs.
Again, I found it intuitive to work with.
Tom
Posted by Thomas Haynes on March 05, 2009 at 01:59 PM CST #
Chris,
I just realized I couldn't tell if you thought the CLI allowed you to do some simple scripts or if you wished it would allow you to do so?
I.e., is there something you can't do here?
Thanks,
Tom
Posted by Thomas Haynes on March 05, 2009 at 02:00 PM CST #
The CLI can be scripted. For example:
#!/usr/bin/bash
#
#
SERVER="root@192.168.1.1"
# use ssh-genkey to avoid password requirement
#
#
# request a listing of analytics datasets
#
ssh -T $SERVER <<EOF
script
run('analytics');
run('datasets');
data = run('show');
printf('%-40s\n', data);
run('cd ..');
EOF
Posted by McD on March 05, 2009 at 04:07 PM CST #
Tom,
I haven't tried any scripting with the Unified Storage Product, but I figured it was possible to do a lot of things with it. My exposure to the CLI interface only amounted to a 2 minute demonstration. I think the salesperson probably focused more on the GUI because it comes across as a pretty nice accomplishment.
On the Netapp side, I've written remote scripts, and I've used their API. I think the most complex case I worked on was editing an /etc/rc file on a filer, but the only protocol I could use the easily access the file was ftp (or I suppose I could have used "wrfile" after cutting and pasting what I needed).
Posted by Chris on March 06, 2009 at 09:38 AM CST #
McD and Chris,
Yes, it is easy to automate the Unified Storage Product. I've spent the evening learning how to do that and having a blast. I love learning how to make new systems just work for me.
Thanks,
Tom
Posted by Thomas Haynes on March 09, 2009 at 12:02 AM CDT #
NetApp users are asking how a group of 7000's would be managed... I guess they have a Management Application that monitors and sends configuration messages to a grouping of systems, like OpenView for Storage systems.
I'd imagine early 7000 adopters would use a browser and multiple tabs and flip between system tabs.
The opportunity to create scripts that would post changes to a lot of systems seems like a first step and putting those scripts bethind a Web server with CGI would also start to move towards a Unified "Unified Storage" Console.
I'm sure Sun will build a higher level management application overtime. First they need to sell large installations... the Fishworks team seesm to have no loack of talent in the software development side of the equation. They just need to know what the sales barriers are for product adoption.
DISCLOSURE: I have Sun stock and am actively working to sell these products as a engineer for a Sun Reseller. I'm excited to see Sun take OpenStorage seriously as a business proposition.
Posted by McD on March 16, 2009 at 01:29 PM CDT #
Yes, they would have to ask how a group of 7000's are managed because they can't attack the ease of use of the BUI.
And yes, the ease of use for scripting makes writing a distributed manager that much easier.
You can talk a lot about what scares other vendors or what excites people about Sun's OpenStorage strategy, but for me, the number one thing is that the BUI/CLI is nimble and flexible. They are integrated well together and allow management to be stackable.
The other subtle point here is that these characteristics permeate the entire product. The developers are carefully picking where they are going to do their value add. E.g., replacing NVRAM with SSD.
Yeah, I'm excited as well.
Posted by Tom Haynes on March 16, 2009 at 01:47 PM CDT #