Paul Humphreys rambles on....

Last weekend we took the opportunity of a power down in our building to install one of these. It is also known as an Aspen switch. It has lots of new features and it was time to retire our old Black Diamond core switch after many years of faithful service. The sorts of things we will be doing with the new switch are ten gigabit networking and IPV6 routing etc.

Although the building was being powered down our production servers had to stay up as we need the information on them twenty four hours a day, seven days a week. We have a UPS/Diesel GenSet to accomplish this. So the first job was to have setup a temporary network to plug all the servers into. This project is a bit like doing a heart transplant. The most difficult bit was to then remove the old core switch and install the new one - both big and heavy. The new switch had already been setup with its vlans and other configuration information beforehand. Once bolted in and checked out we then transferred our servers back onto it. Of course having all the cables labelled was essential to this task...

The next stage will be to install 10G cards in our servers where we are sure the throughput is needed. For instance the links between our SunRay servers and the NFS server, also the NFS server with the source on it and the build boxes. AS we plan to change the two main NFS servers soon this will be easy to do.

Also on that day we installed the latest CPU's for the mid and high end servers. Inside Sun these are known as Panther cpu's. These CPU's have initially been installed in the SunRay servers running Solaris next , Nevada. One board of these speedy CPU's is enough for each of these machines with 64GB of memory on each board. These 1.8ghz CPU's have already won a lot of benchmarks against our competitors. As both of these servers were going to be physically powered off we popped in a ten 10G card in each ready to be enabled when each server takes their turn to get a new version Solaris next

For me my presence was only needed to help do the manual labour and provide drinks when people needed them. Two of this years students and our contractor were able to get some excellent experience helping us do this work. We have some post install work to do. The new core switch has many more one gigabit uplinks so we can now directly attach our edge switches to it rather than go via Summit 1i's. This will make our network simpler to manage and faster we reckon.

Comments:

Nice blog...thanks

Posted by manne on October 16, 2006 at 07:28 AM PDT #

Do you actually have (or moving to..) fast enough RAID arrays in your NFS servers to drive 10Gig-E, or are you getting some kind of latency improvement that makes all the difference?

Posted by Alex on October 17, 2006 at 12:00 PM PDT #

Well we are mving to new servers that themselves are going to have the cpu power to really drive the wire at 10G speeds. For the nfs servers we are using JBOD's not back end RAID disk devices and using ZFS to provide the redundancy and all the other clver stuff it does. Its more for us of using Sun's latest technology. It is so important for us to try these things out even if there is no tangible benefit.

Posted by Paul Humphreys on October 18, 2006 at 12:14 AM PDT #

Very cool :) I've just been reading up about ZFS now, I particularly like the ability to enable compression and the promised future support for encryption and secure deletion!

In some ways, it almost reminds me of Reiser4 - both are certainly revolutionary filesystems, although I'd wager while the plugin capability of Reiser4 makes it more extensible in the long term; at the moment things like the compression plugin make it comparatively unstable.

I currently use a hardware RAID controller to provide some redundancy (RAID-5) for my little NFS server at University - I'm going to experiment by moving stuff off the 8-port SATA card (Highpoint) and onto two 4-port cards supported by Solaris. Fun times :)

I couldn't find much information about the sorta timeframes for implementation of encryption/shredding in ZFS - is that info available at all?

Posted by Alex on October 18, 2006 at 11:18 AM PDT #

I am sorry but I do not have any dates for the stuff you are talking about for ZFS. I am sure as soon as it is ready you will find it in Solaris express; thats where the new stuff finds a home first. Thanks again for your comments!

Posted by Paul Humphreys on October 19, 2006 at 02:52 AM PDT #

For the nfs servers we are using JBOD's not back end RAID disk devices and using ZFS to provide the redundancy and all the other clver stuff it does.

Posted by Calo Bob on October 19, 2006 at 06:23 AM PDT #

Alex, visit opensolaris.org to stay tuned with all the latest going into solaris. There is an active user forum as well which you could pop questions. Ps. 10 gig is neat :) i wish i had fast enough disks to drive such thing.

Posted by Ioannis Psyllas on November 03, 2006 at 12:14 AM PST #