Random mumblings of an SSE Scott Howard's Weblog

The final patches for the Solaris 10 in.telnetd vulnerability are now available on SunSolve.

The patch numbers are : 

120068-02  :  Solaris 10 SPARC

120069-02  :  Solaris 10 x86/x64.

 Both patches are available from http://sunsolve.sun.com/

ISR (Interim Security Relief) patches are now available for the Solaris 10 in.telnetd vulnerability from http://sunsolve.sun.com/tpatches

 The Descriptions and readme links are currently empty, but they should be there shortly.

 The ISR's are :

IDR125456-01.zip  -  Solaris 10 SPARC
IDR125457-01.zip  -  Solaris 10 x86

Patch 120068-01 (SPARC) or 120069-01 (x86) are required patches, however none of these patches/ISR's require a reboot and they will take effect immediately.

When Dave Wischnowsky from the Chicago Tribune wrote about my 740 megapixel photo of Sydney By Night last August he called it "A skyline shot like no other", and made the comment that he hoped I would do one of Chicago in the future.

Little did he know at the time that I was already planning to be in Chicago a few months later, and had every intention of trying to produce a similar shot of Chicago!

Around a month ago I finally found the time to finish off Chicago By Night, which ended up being around the 1 gigapixel mark. When I showed it to Wischnowsky he loved it, and after a phone interview I was once again the subject of The Wisch List, this time under the heading of "Chicago's ultimage skyline shot".

Sun Explorer version 5.7 was released a few weeks ago, and is available for download from http://sunsolve.sun.com/explorer

 As well as the usual list of new features and bug fixes (full list available in the Release Notes) we've now officially added support for uploading via HTTPS through a proxy server, which we managed to miss when we first added HTTPS upload support in 5.5.

Unfortunately one bug did slip through - if you're using either the "scextended" or "alomextended" modules to connect to the system controller on an F3800->E6900 or V210/240/440 over the network, and the SC/ALOM is configured to use telnet rather than SSH then Explorer will fail to connect to the SC/ALOM.  The easiest workaround is simply to stay with Explorer 5.6 until 5.8 is released, or just configure your SC/ALOM to use SSH in stead of telnet.

The idea of being Solaris Flash Archives as a means of backing up a server is something that comes up fairly frequently - and is generally going to be a bad idea.  That's not to say that you can't make it work, it's just to say that it's normally going to be the wrong answer to the problem.

Flash archives are designed for quick deployment of like servers, where like is defined in terms of the software installed, and not (necessarily) the hardware involved.  By design what is "restored" from a flash archive is deliberately different to what was flashed, with many of the OS configuration files being deliberately deleted and/or re-created during the flash process. This includes things like /etc/hosts, /etc/hostname.*, /etc/netmasks, /etc/path_to_inst, the entire /dev and /devices trees and some others.

So in effect, what you put in is NOT what you get back out.

But the very definition of a backup/restore is that you get back exactly what you started with. ufsdump (or any commercial backup software) will give you a restored machine which looks exactly like what you started with.

Why does this matter?

Sure, it's possible to fix some of the files which are deliberately delete/changed (such as /etc/hosts) using post-install scripts, but to do this reliably means you need to actually take copies of these files whenever you do a flash archive, which begins to get messy.
Others like /dev and path_to_inst are a little more difficult.  If you've added storage to the machine post-install (such as if you didn't have any external storage installed during the initial install) then there's a very good chance that your controller numbers will be different after the flash "restore" than they were before.
Add in network interfaces and it gets very messy (the on-board NET0 on a V440 is ce0 if you build the machine without any extra CE cards in it, but will be something like CE4 or CE8 if you've installed some extra cards afterwards).

The whole concept of backup/restore is about getting your machine back exactly as it was when the backup was taken - Flash simply doesn't give you this (and wasn't designed to).

There are a few cases where Flash can replace backups, but it's pretty much the same cases where JumpStart alone can replace backups - such where you've got a pool of similar machines where details like controller numbers aren't important and network config can be easily re-created. Desktop workstations and pools of compute servers/web servers/etc spring to mind. General purpose servers normally do not fit into this category.

Keep in mind that you can do ufsdump over a network in much the same way as you can do a flarcreate. The restore side is a little more difficult to automate, but only if you don't take into account all the "extra" stuff you need to do for flash to get things back how you started.

ufsdump 0f backuphost:/backups/myhost-root.ufs /

Or even better, add in fssnap to make sure things are consistent :

ufsdump 0f backuphost:/backups/myhost-root.ufs `fssnap -o raw,bs=/export/home,unlink /`; fssnap -d /

I haven't seen it announced anywhere yet, but Solaris 10 11/06 (Update 3) appears to be out!

The list of new features from the "What's New" guide is impressive :

System Administration Features

• Storage Networking Industry Association Multipath Management API Support
• Sun Java Web Console Changes
• File-System Monitoring Tool
• ZFS Command Improvements and Changes

• Recursive ZFS snapshots
• Double Parity RAID-Z
• Hot-spares for ZFS storage pool devices
• Replacing a ZFS file system with a ZFS clone

System Resource Features

• Resource Pools Facility Service FMRIs

Solaris Zones Features

• Solaris Zones Renaming Feature
• Zones Move and Clone Features
• Migrating a Non-Global Zone From One Machine to Another
• Configurable Privileges for Non-Global Zones

Logical Domains Features

• Logical Domains (LDoms) 1.0 Software

Security Features

• Solaris Trusted Extensions
• Solaris Trusted Extensions for Printing
• Solaris Trusted Extensions File-System Labeling

Device Management Features

• Support for PCI Express (PCIe)
• x86: Sun Fire X4500 SATA Disk FMA
• SPARC: Transitioning SPARC-Based Systems From Ipge to E1000g Network Drivers
• Solaris Fibre Channel Host-Based Logical Unit Number Masking
• SPARC: Extended Message Signaled Interrupt Support for Fire-Based Platforms
• Improved Device in Use Error Checking

Desktop Features

• Default Desktop Session in dtlogin
• Adobe Flash Player Plugin for Solaris
• GNOME-VFS and Nautilus ACL Support
• Solaris Trusted Extensions Desktops

Installation Enhancements

• Secure By Default Network Profile
• Installing Solaris Trusted Extensions

System Performance Enhancements

• SPARC: Watchdog Timer for Sun4V

Networking Enhancements

• Sun Java System Message Queue 3.7 Update 1

New and Updated Drivers

• ST Driver Support for Quantum LTO-2 and LTO-3 Tape Drives
• CDB Length Capability

Language Support

• IIIMF and Language Engines

Of course, it's all available for free - just jump over to the Solaris Website and download it!

Lies, Damn Lies, and Statistics - that seems to be the best way to describe JetStar's (Qantas' "cut-price" airline) on-time performance figure.

Their website says they had an on-time performance of 84% in October (a bit above their normal of about 80%), but when you think about it that means that there's about a 1 in 3 chance that you'll have a delayed flight on a return trip - and of course "on-time" actually means more than 15 minutes - less than that and it's an on-time flight.

On Sunday I was booked on JQ209 - the 16:25 flight from the Gold Coast to Sydney. I did the right thing before I left for the airport and checked on their website to make sure that the flight was on-time.  The result?  A "Server Error" on the website when I tried to check that flight. Other flights before and after were ok, just not the one I was booked on.  Oh well, off to the airport and hope the flights on time...

When I made it to the airport I discovered the cause of the error - the magic word CANCELLED flashing on the monitors next to my flight number.

Welcome to the first rule of on-time calculations - Cancelled flights don't count!  That's right - if a flight leaves 20 minutes late then it counts as a hit against your on-time performance.  If it never leaves at all - no hit!

After around an hour in the check-in line I was finally moved to another flight - almost 4 hours and 3 flights to Sydney later than the one I had originally been booked on.  Of course I was compensated for the delay, in the form of an $8 refreshment voucher. Apparently my time it worth about $2/hour.

But it was what I saw in the 4 hours I was sitting in the departure lounge that amazed me. Every flight that departed whilst I was there (all 5 of them including my flight) departed late by more than 15 minutes.  But according to the monitors in the airport, and JetStar's website, all but 1 departed "on-time" (ie, no more than 15 minutes late).

 Presuming JetStar use the same numbers to calculate their on-time performance they are going to report that 80% of these departures were on-time, when in fact 0% of them were. That's a big difference!

Apparently a picture is worth 1000 words, so how about this (the top left corner is a zoom of the sign below it):

 That photo was taken at 19:35 and as the board says, the flight was "Closed", which should mean that everyone is on board. But at the point that photo was taken, not a single person had boarded - you can clearly see the passengers lining up waiting to be allowed on board.  They were finally allowed to start boarding about 5 minutes later at around 19:40.  According to JetStar, this flight departed at 19:45, only 10 minutes later than scheduled and thus within the magic 15 minutes for an on-time departure.  So somehow they apparently managed to get 177 passengers across the tarmac, into the plane, seated, and the door closed in about 5 minutes.  Somehow I think not.  Not surprisingly the flight landed almost 30 minutes later than scheduled.

 

The next flight out was  JQ229 - the flight I was transfered to. According to JetStar this flight departed only 15 minutes late at 20:20 - and 15 minutes late is "on-time".  This photo was taken at 20:16 :

Again you can see the line of people waiting to board the flight which is already closed.  Despite JetStar's claims that we departed at 20:20, we actually started boarding at about that time - I was still seated in the terminal at the time we apparently departed.

 It wasn't just Sydney flights this was occuring for either. At 20:15 Jetstar staff announced that the plane that was to run JQ283 to Melbourne was "On finals" to land, and that they would be boarding in about 10 minutes.  10 Minutes to land, taxi, unload, clean the plane, and then start boarding - not a hope.  Apparently JQ283 left at 20:35 (10 minutes late - "on-time"), but the stairs were clearly still attached when we taxied out at 20:41 - and it certainly takes more than 20 minutes to go from final approach to departure.

 

All up I managed to arrive in Sydney only minutes before the Sydney Airport 11pm cerfew kicked in, over 4 hours after I was originally scheduled to land. Now JetStar did at least acknowledge that this flight landed 30 minutes late (ie, not "on-time"), but the 4 hours and 15 minutes late I left Gold Coast is considered "On-Time"  - go figure!

 

This Wednesday at 10am PST/1pm EST (click here for your local timezone) Sun will be running the latest in it's Expert Exchange series, this time on using Live Upgrade to Upgrade and Patch your Solaris systems.

 Keeping systems running is critical to your business. How can you
maximize system availability - and minimize risk - while keeping your
systems up to date with the latest patches and upgrades? Simple - put
Solaris Live Upgrade to work - and this Sun Expert Exchange Q&A
forum is a great way to get started.
 

So that's the official blurb, but what can Live Upgrade really do for you?  One of the main complaints I hear from sysadmins is that patching is too difficult - primarily that it takes too long (often at 2am in the morning), and is too hard (and/or slow) to backout if there are any problems. If you're one of those Admins, then Live Upgrade is the answer.

What most people seem to miss is that Live Upgrade isn't just for upgrading between Solaris versions - it can also be used for patching. So instead of spending hours at 2am applying patches and rebooting your system - all during a multi-hour outage of course - you can carry out the patching to an "Alternate Boot Environment" anytime through the middle of the day, and then simply reboot the system to use the ABE during your outage window. If there's any problems with the patching, then it's simply another reboot to revert to the previous boot environment.

Yes, Live Upgrade does require a bit of planning beforehand, but the time it takes to configure a worthwhile Live Upgrade setup is generally far less than the time LU will save you in the long run.

So join us on Wednesday to find out how Live Upgrade can help you reduce your patching and upgrade outage windows. And don't let the marketing people on the panel fool you - the techies will be there as well to answer all of the hard questions!  To register, go to http://sun.com/expertexchange and enter your details.

A few months ago I found myself planning what to take for my 5 week trip to the USA and Peru.  Once the essentials were covered (the camera, the tripod and the like) I came to the age-old decision - to take my notebook or not.

Of course, being a geek, the obvious answer was "hell yes!", but when you're already carrying over 10kg (22 pounds) of camera gear, the thoughts of adding an additional few kilograms of fragile computer to the carry-on luggage isn't as tempting as it should be.

In the end I decided to compromise. Rather than taking my existing notebook, I'd buy a new, smaller, lighter one - a Sony UX17.  OK, so the truth was that I had pretty much wanted a UX17 from the day I had first seen a picture of one, and even more so from the day I first played with one at the local Sony store, so it was more a case of a solution looking for a problem - and I'd found my problem!

For those who haven't see the UX17 (or the UX280 in the US) it's a fully featured notebook which is roughly the same size as a typical paperback novel, and not that much heavier than one at a tiny 517 grams (about 18 ounces).

Of course, with such a small form factor there's compromises - the most obvious being the screen, which is less than 1/4 of the area of the normal "ultra-light" notebooks at only 4.5", and the keyboard which is even smaller than the screen. The other specs are at the low end of current systems, but still adequate for most tasks - a 1.2Ghz Intel Core Solo U1400 processor, 512Mb memory (UX280 is 1Gb), 30Gb hard disk (UX280 is 40Gb), 802.11a/b/g wifi, Bluetooth, two cameras (yes, two - it is a Sony after all), plus CF, MemoryStick and USB slots. Not bad for 517 grams!

 So after 5 weeks on the road, from the bright lights of Broadway, to the beaches of Miami, to the ruins of Machu Picchu and the Amazon Jungle I've got to say that this is the greatest travelling geek toy ever - at least for someone who takes a lot of photos.  The keyboard is a real limiting factor - it's fine for tapping out a basic email, but it's certainly not designed for the touch-typist. The screen is far less of a limiting fact that I first thought it would be.  Even for viewing and editing photos it's excellent, due mainly due to it's relatively high resolution (for it's size) of 1024 x 600 pixels, and the touch-screen and stylus which makes editing much easier than using a mouse.

For someone who takes far more photos than I should (my record this trip was over 600 in one day), and as a general rule hates writing long emails, this makes it the perfect on-the-road PC for me. The ability to view, edit and even upload photos whilst on the go was brilliant, and meant that I was able to keep my Flickr account up to date with photos as I went - in fact the longest time between any photo being taken and available on the web was about 4 days, and that's only because there's no WiFi on the Inca Trail!

 

 The Good :

• It's small - real small!
• It's fast - at least for it's size.
• The screen, though small, is clear and high-res at 1024x600 and has a very accurate touch-screen.
• Built-in everything - Wifi, Bluetooth, USB, Firewire (on the docking station only), Compact Flash, finger-print reader - just no optical drive. (The US version also has EDGE wireless built in)
  

The Bad :

• The keyboard. OK for web browsing or a quick email, but not much more.
• The "mouse". Slow to use accurately, and the texture wears off far to easily - although the touchscreen/stylus makes up for it.
• Battery life. The quoted 3.5" hours is optimistic as you'd expected, although I did managed to get almost 2 hours of Divx playback out of it. In general 2-3 hours seems to be the norm.
  

And then there's the outright silly. As well as a functional docking station, it comes with a tiny plastic stand that can be used to told it upright on a table. Between this stand, a USB keyboard (I have a small travel keyboard that is far more functional that the built in one) and the touchscreen/stylus it should be possible to turn this into a reasonably functional "desktop" setup, rather than hand-held as you'd normally use it. I say should because for some reason Sony decided to put the power connector on the bottom of the unit, which means that you can't use the stand whilst it's plugged into power - only whilst on batteries. I'm guessing that this is one of the compromises they had to make to keep the system so small/light, but it's a real limiting factor if you're trying to use it in a hotel room/office/etc for any period of time.

 Overall, as I said above, it's probably the best "geek toy" I've ever owned. Even if I only use it for 1 or 2 months of the year it was worth every cent that I paid for it! Sure, basically everything about it is a compromise in some form or other, but when you're "adventure travelling", a 517 gram compromise is a better option than either carrying 2+ kg of standard notebook, or taking no notebook at all. At least, it is when you're a geek! :)

 

Sometime within the coming week the Western Australian government will vote upon a bill to commence a 3 year trial of Daylight Savings in WA.

Whether you're a fan of daylight savings or not, the bill has one fatal flaw - it is due to commence at 2am on the 3rd of December, 2006. Less than 2 weeks from today, or about 10-12 days from when it goes before a vote of the upper house.

If it is passed, the implications of this change are significant and wide ranging. From an IT perspective it means that every computer in WA will need to be updated to be aware of the change - preferably by installing an updated timezone file beforehand, or at a minimum by manually modifying the time on the 3rd (although this will result in GMT/offset issues).

But it goes much further than that.  All flights into WA from other states/countries will arrive an hour later than expected (or leave an hour early, which is unlikely).  Flights out will most likely need to be rescheduled to allow for the late arrivals. Airlines normally build daylight savings changes into their schedules months in advance, but with less than 2 weeks notice they are not going to be able to do so.

The same goes for pretty much anything else that crosses the state border - call centre staff communicating with other states, other transport, B2B communications (especially in a state which hasn't had daylight savings for 15 years), etc. These are issues that occur in other states that already have daylight savings, but in those cases we've all had more than 10 days to plan for them!

Of course, this impact has either not occurred to the politicians, or they simply don't care. Given that the bill originally had a start date of the 1st of December (ie, Friday morning) which was changed to the 3rd they do obviously realise that there would be an impact, so...

If you happen to be in WA, or have Solaris systems that use the Australia/Perth or Australia/West timezones for some other reason, then you will need to take action if this legislation is passed. As we most likely won't have time to get a patch created and through Sun's patch testing process in less than the 2 weeks the government is going to give us we've instead created InfoDoc 87748 which describes a workaround to update the Solaris timezone files with the new details. This document will be updated as more details are available.

After the huge success of my 720 megapixel photo of Sydney earlier in the year (http://blogs.sun.com/Doc/entry/720_megapixel_photo_of_sydney) I decided I needed to go bigger - and where better to do it than the ancient Inca city of Machu Picchu in southern Peru.

The end result is a massive 1500 megapixel (that's 1.5 Gigapixel - 1.5 billion pixels!) image made from over 400 individual photos taken over the course of just over an hour.  Printed out it would be around 10 metres by 5 metres (at 150DPI),

The stiching was done using Autopano Pro and Smartblend on a dual processor Sun V40z, and took over 10 hours per render at full resolution.

To view the end result go to http://www.docbert.org/MP (or just click on the image below). It uses the Zoomify viewer so the bandwidth required to view and even zoom the image is minimal.

Version 5.6 of Sun's Explorer data collector has been released, and is available for download from http://sunsolve.sun.com/explorer

(It was actually released a few weeks ago whilst I was on holidays - sorry for the late notification!)

This version includes a total of 25 bug fixes and new features, which are detailed in the Release Notes if you happen to be interested.

 Although it's not normally critical to stay on the bleeding edge of Explorer versions, using a relatively current version can assist if you ever need to contact Sun Support - and having it pre-installed can save precious minutes when you do have a problem.

If you've got any comments or suggestions on how we can improve Explorer please let me know by using the "Comment" link below - I'm a member of the Explorer Change Control Board, and we're always open to suggestions on how to improve it.

It's been a fun 3 weeks, but my holiday in Peru is about to end.

There only so much you can do in 3 weeks, but it turns out that visiting half a dozen cities, staying on Lake Titicaca, walking the Inca Trail for 4 days, visiting Machu Picchu and the Amazon, and drinking a few too many Pisco Sours can be fit in if you try.

Thanks to the purchase of a Sony UX before I left home (a full PC which is smaller than a paperback novel and about the same weight - more about that later) I've been able to upload some of the pictures I've taken as we travelled - they are available at http://www.docbert.org/flickr

New York for a few days is next, before the long flight back to Sydney.

And if you ever happen to find yourself in Lima's airport you'll be pleased to know it's got the cheapest WiFi access of any airport I've seen - starting at only US$1 for 30 minues and US$3 for 4 hours!

They are two terms which are often used to refer to the same thing, but Redundant and Fault Tolerant are actually very different - and one certainly doesn't imply the other.

Redundant
As you'd pretty much expect, redundant means that you've got more of something than you need. The tyres on your car are redundant - you only need 4 to drive, but you have 5, including a spare.

Reduntant doesn't imply that there is no impact to service when a component fails, it simply means that you are able to recover the service - to at least a working (although possibly degraded) state - without the need for any external components.  If you get a flat tyre on your car you need to stop and replace it with the spare (redundant) tyre. This has an impact, but it still allows you to recover from the flat without needing any external assistance.


Fault Tolerant
As the name implies, Fault Tolerant refers to the ability to tolerate a fault. The exact definition of  Fault Tolerant will vary depending on who you ask, but generally it implies the ability for a service to continue running despite a fault. eg, "run flat" tyres on a car could be an example of fault tolerant - despite the failure you are able to continue driving without an "outage".


Redundant components, Fault Tolerant systems
Fault tolerant systems are usually designed by using redundant components. Probably the most common form of fault tolerance we are used to is "RAID" - Redundant Array of Inexpensive Disks - but why does RAID have "redundant" in it's name if it's actually fault tolerant?

The distinction comes down to the difference between the individual componets and the entire system itself.  Individually, the disks within a RAID array are redundant, but they are not fault tolerant - if a disk fails, then it is dead. However as a system, a RAID array is fault tolerant - if a disk fails the array, and your data, is able to continue without interuption (although probably with degregation).

The Component - the disk - is redundant.
The System - the array - is fault tolerant.


With a few historic exceptions, no Sun systems are completely "Fault Tolerant", although they frequently contain fault tolerant sub-systems, such as :
* Power Supplies
* Fans
* Disks (using RAID)
* RAM (using ECC - Error Checking and Correcting memory)

In most (all?) cases this fault tolerancy is achieved using redundancy - multiple power supplies, multiple fans, multiple disks in a configuration where the failure of any one can be transparently handled without an outage.

Some high-end Sun systems can go a step further and be configured to be completely Reundant. This still doesn't mean that they can transparently handle any failure, but much like a flat tyre on your car the system is able to re-configure itself to map out the failed component, and come back up in a (possibly) degraded configuration. Whilst there is obviously an impact in doing this, it's far better than the alternative of being non-redundant, and far far (far!) cheaper than the alternative of being fully fault tolerant in hardware.

100 kms - teams of 4 - 48 hours

 Think you could manage to walk 100km (around 62 miles), primarily through bushlands, with 3 other people, all within 48 hours?  How about 24 hours?

Well the winners of this years Oxfam Trailwalker Sydney managed to do it in just 12 hours and 37 minutes, slightly above the course record of 11 hours 59 minutes.

Trailwalker is an annual event held in a number of cities around the world as a fundraiser for Oxfam. The Sydney event this year is expected to raise around A$2 million dollars.

What's this got to do with Sun? Like many large companies Sun recognises the importance of supporting the community, and as such provides support to staff who want to Volunteer for such events. I managed to spend most of today (Friday) delivering supplies to the checkpoints and assisting with their setup - all with the full support of Sun.

For more information of Trailwalker, visit the website at http://www.oxfam.org.au/trailwalker/sydney/ and while you're there why not help out yourself by making a donation to one of the teams!