OpenSolaris community - an introduction

Deirdre just posted some great videos from the OpenSolaris developer summit. This was a great weekend and it was fun getting to see the faces behind the messages on opensolaris-discuss.

Thanks for the great videos Deirdre!


 

Developer Summit & Storage

Thanks Jesse for the great graphic for the upcoming developer summit. 

If you are interested in OpenSolaris storage and the latest happening, stop by our chat on Sunday afternoon. Unfortunately our 11 year old mascot will not be there, but I can bring autographed pictures.
 

OpenSolaris Storage - in their words

Today we are kicking off a new part of the OpenSolaris Storage Community. -- the Community Spotlight. The community spotlight is a place where we will feature the contributions and successes of OpenSolaris Storage.

Checkout out our first 2 interviews:

	Interview with Evan Powell, CEO Nexenta	http://nexenta.org
	Interview with Gregory Perry, CEO Liveammo	http://liveammo.com

Do you have a story to tell? I would be interested in hearing it...

Setting Up an OpenSolaris NAS Box: Father-Son Bonding

Over the winter break, my son and I spent some quality time together and we created a little home NAS server. We had a great time and we both learned a lot. I learned how much I love OpenSolaris (and how little I know about Windows) and my son got a glimpse of what I do at work. I had forgotten how important it is for 11 year olds to know what their dads do. My son also learned a little about computer hardware and how to use a UNIX command line.

This post details what we did to get our NAS box up and running as well as some of the challenges we faced.

What's the Problem?

We took on this challenge primarily as a learning experience, but we also wanted to solve a real problem. We have a few Windows XP machines at home and there is always some type of contention with the music and homework stored on one of the machines. Typically, the contention is that I'm working at home (on my OpenSolaris laptop) using the monitor attached to the “primary” Windows machine and the rest of the family is trying to use the Windows laptop to listen to music or get a homework file. A while back, I created a Windows Share on the “primary” desktop, but firewalls and other problems kept making the share go away. So, I decided that with the new CIFS server in OpenSolaris, it was a great time to use ZFS and CIFS to create a home NAS box and replace the Windows share machine.

Step 1: Getting the Hardware

To get started, I had to find hardware for the NAS box. The good thing about working for Sun is that there is always old hardware lying around. When I entered our lab, I quickly found a stack of unused multipacks. While they were pretty old and unexciting (6 x 40GB drives), they would provide a great way to use ZFS and I didn't need much storage, yet. The other thing I found was a recently retired W2100Z workstation. This is a great platform with dual 64-bit AMD processors, an internal SCSI drive, and a place to put 3 more drives. I also found some DIMMs for the W2100Z and an extra SCSI card. Sweet! This was everything we needed to get started.

Step 2: Assembling the Hardware

Since my son had never seen the inside of a computer, we started by opening up the W2100Z case and walking through the hardware components. “Here's a CPU, there's a controller, that's the power supply”... stuff like that. Also, I did a quick lesson about static electricity. There is a lot of static in January in Colorado , so I showed him how to “attach” himself to the computer with a static strip and get started with the hardware upgrade.

I explained we needed the SCSI card to attach to the multipack. I also explained that the extra memory would be used to help ZFS work better. The whole lesson and hardware installation took only 20 minutes. Then, it was time to install OpenSolaris.

Step 3: Installing OpenSolaris SXDE 1/08

Preparing to install OpenSolaris was quite simple and quick (5 minutes with explanations):

• Download and burn DVD

• Insert the DVD

• Boot the workstation

• Answer the questions

• Click the “ Finish” button
  
  

Then came the actual installation. My son didn't this part of the project, because he had to go to bed before it finished. The install took about 1.5 hours and our new server was up and running OpenSolaris when my son woke up the next morning. Now all we had to do was configure everything.

Step 4: Configuring OpenSolaris

I always try to be a good dad and teach my children the lessons that took me a while to learn. One of those lessons is that there are smart people who can be helpful to you, and you should build on their knowledge whenever possible. For this project, the smart people were the developers of the OpenSolaris CIFS server and ZFS features. The distillation of their knowledge is found in the CIFS Getting Started Guide. This guide was invaluable and provided us everything we needed to get started. So, I suggest you also start with this guide as well.

Create the Users

To begin with, we needed to create a few different users. At install time, we created our root user account and after this we needed to have a “privileged” regular user and a user that could own the files. We selected the userids admin and media for these purposes. To add the users, we used the “Users and Groups” action on the Administration section of the Start Menu.

Create a Storage Pool

To get our server up and serving data, we needed to configure the storage. To do this, we used the ZFS GUI by just pointing a browser to https://<hostname>:6789/zfs and logging in as a user with root permissions. This graphical interface provides the ability to see all of the storage attached to your system and how it is being used. We selected all of the drives on the multipack and created a single RAIDZ1 pool called mediapool.

Create the File Systems

After creating a ZFS pool with the ZFS GUI, we switched to a terminal window to check the status of the newly created pool:

# zfs list
NAME USED AVAIL REFER MOUNTPOINT
mediapool 144K 165G 29.9K /mediapool
  

Then, we created the file systems:

# zfs create -o casesensitivity=mixed mediapool/music
# zfs create -o casesensitivity=mixed mediapool/photos
# zfs create -o casesensitivity=mixed mediapool/movies
  

We decided to set the ZFS property casesensitivity=mixed to allow us to use mixed cases for the files. This is a very important step when sharing files with Windows. If you don't do this, all of the file names will get set to all upper case, which is very hard to read.

Configure CIFS

After creating the file systems, we started the CIFS service:

# svcadm enable -r smb/server
# smbadm join -w CENTRAL
  

Since we were working on our home systems and didn't have a fancy domain controller, we used the workgroup mode for sharing the files. I had previously set our home workgroup to CENTRAL, which is reflected in the join command. In most cases. you would keep the workgroup set to WORKGROUP.

Set Up Authentication

We also needed to update the OpenSolaris password system to configure the CIFS server to use PAM (pluggable authentication module) authentication. We did this using the following mystical command:

# echo "other password required pam_smb_passwd.so.1 nowarn" >> /etc/pam.conf
  

Additionally, after entering the above command, we had to reset the passwords for all users who would be using the CIFS authentication. For our situation, it was only one user:

# passwd media
  

That did the trick and we were ready to start sharing the file systems we created.

Share File Systems

We were getting closer to being able to use our new server. CIFS was enabled, the file systems were created, and we just needed to get them shared:

# zfs set sharesmb=name=Movies mediapool/movies
# zfs set sharesmb=name=Music mediapool/music
# zfs set sharesmb=name=photos mediapool/photos
  

We could have used a single set command with the sharesmb=on option, but we wanted useful names on our Windows' machines. So, to avoid any ambiguous or long share names, we explicitly set them. This was just one more of the little tricks and helpful tips from the CIFS Getting Started Guide and the discussion forums.

Set File System Permissions

For our last step, all of the files needed to be owned by our primary share user:

# chown -R media /mediapool
  

After this last step, we were done configuring OpenSolaris on our NAS box.

Step 5: Configuring Windows

Now, the real fun began. So far, it took only 2 hours to assemble the server hardware, install the OS, and configure the storage, and 90 minutes of that was the OpenSolaris installation. We were feeling pretty good about spending only 30 minutes to get our file system configured and shared. Then, we had to configure Windows.

I realized very quickly that while using a Windows XP system for e-mail and web browsing is pretty simple, there is some complexity in sharing files. I scoured web sites and discussion forums and learned that sharing files with CIFS is different than sharing file with NFS.

I'm used to having a filer export an NFS file system, which provides the ability to browse the share and then verify the permissions. Well, Windows is the same, but the permissions thing is different. Or, so I thought.

When mounting a CIFS share from a OpenSolaris box, you must authenticate to that server. This point is critical. The authentication also has to be from the perspective of the server and not the client.

To make the share on the windows client work, we needed to do the following:

1. Map a windows drive with the “Folder” path of \\< OpenSolaris_Server >\<sharename>

2. Select the Connect as different user name link and use the user name and password from the OpenSolaris server.

For our machine, the “Folder” path was \\PHRED\\Movies and the user name was media.

Making this simple discovery took about 3 days, but we finally had a way for all of our Windows machines to access a shared file system on our new OpenSolaris NAS box.

Variations on a Theme

My son and I had a great time with this exercise. We had a lot of fun playing with the hardware and we both learned a great deal. We are still playing with the server and are now looking at adding some new pieces:

• Larger drives – we'll be adding some 500GB SATA drives, using the onboard SATA controller

• Virus scanning – we're looking into using clamAV to do our onboard virus scanning. The directions on the Genunix wiki look pretty straight forward.

Thanks and References

This post would not be complete with out us taking the time to thank those who helped us out along the way. Here are a list of web sites, blogs, and discussion lists that were invaluable.

• http://opensolaris-DOT-org/os/project/cifs-server/

• http://www.genunix.org/wiki/index.php/Getting_Started_With_the_Solaris_CIFS_Service

• http://www.genunix.org/wiki/index.php/Solaris_CIFS_Service_Troubleshooting

• http://blogs.sun.com/dougm/

• storage-discuss-AT-opensolaris.org

• cifs-discuss-AT-opensolaris.org

Also, thanks to those who blazed the trail before us.

[Open]Solaris Filesystem Choices

There is a great article over at  discussing the various file system choices in Solaris 10. The first paragraph of the article is below and sets the tone for the rest of the article:

Solaris Filesystem Choices

posted by John Finigan on Mon 21st Apr 2008 19:00 UTC

When it comes to dealing with storage, Solaris 10 provides admins with more choices than any other operating system. Right out of the box, it offers two filesystems, two volume managers, an iscsi target and initiator, and, naturally, an NFS server. Add a couple of Sun packages and you have volume replication, a cluster filesystem, and a hierarchical storage manager. Trust your data to the still-in-development features found in OpenSolaris, and you can have a fibre channel target and an in-kernel CIFS server, among other things. True, some of these features can be found in any enterprise-ready UNIX OS. But Solaris 10 integrates all of them into one well-tested package.

 It is great to see such a great article showing up on OSNews. Well done John and thanks OSNews!

To complete the picture I think it would be important to highlight the other file systems that are being worked as part of OpenSolaris. These projects can all be found at the Storage Community projects page. Some of the file system work that should be noted include:

• Squashfs
• JFS: Journaled File System
• Ext3 file system support
• Fuse on Solaris
• and ZFS on disk encryption support

There is a growing excitement around storage and OpenSolaris, and these projects are a great expression of this energy.

How does OpenSolaris and open source play a roll in your storage solutions? Does it help or is it creating problems? Let me know I'm interested in hearing ...

Smashing ZFS ... literally

Jim Hughes  just posted a great video of how well data contained in ZFS survives physical destruction. Jim Walker also posted a few pictures to remember the event.

It is scary and fun to see what happens when engineers are give big hammers.

Thanks for the great demo guys!

PS: if you get a chance, Jim Hughes will be presenting at CommuniytOne, May 5th, it will be a great time. 

First time ...

We all have those moments in life when we just feel a bit ... wired, or awkward. For me, this is typically when I try something new and in front of other people. Sometimes we feel strange when we hear our recorded voice for the first time. The below video represents one of those awkward moments for me. My on-line video debut is the introduction from a recent Birds of a Feather session at the USENIX FAST conference. This was a great time of sharing the vision we have for the OpenSolaris Storage community and hearing what people are wanting from OpenSolaris.

Sit back and laugh as much as I did ... it's only 11 minutes.

For those interested in more videos of Jeff, you can tune in tonight as my son and I present our HomeNAS project. We will be presenting at the Front Range OpenSolaris user group meeting at 6:30pm.

[UPDATE: The video for the HomeNAS: So Simple a 5th Grader can do it, is now live. ]

FAST OpenSolaris storage

New project: Device Mapper

There is another new project in the OpenSolaris Storage community. Not that any one is keeping track but the 50% of the new OpenSolaris projects started in December and January were from the Storage community!

The latest addition is from one of the OpenSolaris Storage core contributors. He has recently taken on the challenge of investigating the use of the Linux device mapper in OpenSolaris. I look forward to hearing about the challenges and successes.

Have fun Cyril!

JFS coming to OpenSolaris

The interest in beafing up the OpenSolaris storage platform is growing!

The latest addition is the JFS project. The project is looking to re-implement IBM's Journaled File System (JFS) from the available specifications. A very exciting and potentially challenging project.

Welcome to OpenSolaris and we look forward to your success!

OpenSolaris Storage @ FAST

Just a quick note to let people know that the the Storage Community from OpenSolaris will be at the upcoming USENIX FAST conference. We will be hosting a BoF on Wednesday evening February 27th at 8pm in the California room.

I'll post more details as they are available. We look forward to meeting you there!

OpenSolaris is getting around....

No viruses on OpenSolaris ...

How to use OpenSolaris and ClamAV to scan file for viruses!

New OpenSolaris Project: SquashFS

The OpenSolaris storage community recently announced an other exciting project. The SquashFS project is getting started and looks quite interesting. 

 From the project page:

"Squashfs is a compressed read-only file system optimized for memory-constrained systems.

We plan to write a complete driver which would enable the kernel to boot from a Squashfs file system, thus enabling Live-CD/DVDs to use it.

It would also allow Solaris to access embedded devices using Squashfs, such as some Motorola phones."

Wouldn't that be cool to have OpenSolaris running on a cell phone? Actually I've heard that they guys that are working on Bellinix have something prototyped. Maybe they could comment?

Congratulations to Thomas on getting this project started and I wish you much success!

New OpenSolaris Project: VSCAN Service

Another project has gone live in the world of OpenSolaris Storage. This time it is the VSCAN service. The project is live and the first pass of the code is available. This project supports the IEFT ICAP protocol and allows for an off board virus scan engine to be called when ever a file is modified.

The combination of this project and the CIFS server project you can now safely store your Windows files on an  OpenSolaris server,  using ZFS, and have they scanned for viruses.

 
What will this team think up next? Maybe a way to backup this files in a standard way??