Thursday July 03, 2008

Welcome to the year 2038!

A lot of interesting things are going to happen in the next 30 years. One of them will be a big push to fix the so called "Year 2038 Problem" on Solaris and other Unix and C-based OSes (assuming there'll be any left), which will be similar to the "Year 2000 Problem".

The Year 2038 Problem

To understand the Year 2038 Problem, check out the definition of time_t in sys/time.h:

typedef long time_t; /* time of day in seconds */

To represent a date/time combination, most Unix OSes store the number of seconds since January 1st, 1970, 00:00:00 (UTC) in such a time_t variable. On 32-Bit systems, "long" is a signed integer between -2147483648 and 2147483647 (see types.h). This covers the range between December 13th, 1901, 20:45:52 (UTC) and January 19th, 2038, 03:14:07, which the fathers of C and Unix thought to be sufficient back then in the seventies.

On 64-Bit systems, time_t can be much bigger (or smaller), covering a range of several hundred thousands of years, but if you're 32-Bit in 2038 you'll be in trouble: A second after January 19th, 2038, 03:14:07 you'll travel back in time and immediately find yourself in the middle of December 13th, 1901, 20:45:52 with a major headache called "overflow".

More details about this problem can be found on its Wikipedia page.

2038 could be today...

Well, you might say, I'll most probably be retired in 2038 anyway and of course, there won't be any 32-Bit systems that far in the future, so who cares?

A customer of mine cared. They run a very big file server infrastructure, based on Solaris, ZFS and a number of Sun Fire X4500 machines. A big infrastructure like this also has a large number of clients in many variations. And some of their clients have a huge problem with time: They create files with a date after 2040.

Now, the NFS standard will happily accept dates outside the 32-Bit time_t range and so will ZFS. But any program compiled in 32-Bit mode (and there are many) will run into an overflow error as soon as it wants to handle such a file. Incidentally, most of the Solaris file utilities (you know, rm, cp, find, etc.) are still shipped in 32-Bit, so having files 30+ years in the future is a big problem if you can't administer them.

The 64-Bit solution

One simple solution is to recompile your favourite file utilities, say, from GNU coreutils in 64-Bit mode, then put them into your path and hello future! You can do this by saying something like:

CC=/opt/SUNWspro/bin/cc CFLAGS=-m64 ./configure --prefix=/opt/local; make

(Use /opt/SunStudioExpress if you're using Sun Studio Express).

Now, while trying to reproduce the problem and sending some of my own files into the future, I found out thanks to Chris and his short "what happes if I try" DTrace script, that OpenSolaris already has a way to deal with these problems: ufs and ZFS just won't accept any dates outside the 32-Bit range any more (check out lines 2416-2428 in zfs_vnops.c). Tmpfs will, so at least I could test there on my OpenSolaris 2008.05 laptop.

That's one way to deal with it, but shutting the doors doesn't help our poor disoriented client of the future. And it's also only available in OpenSolaris, not Solaris 10 (yet).

The DTrace solution

So, I followed Ulrich's helpful suggestions and Chris' example and started to hack together a DTrace script of my own that would print out who is trying to assign a date outside of 32-Bit-time_t to what file, and another one that would fix those dates so files can still be accepted and dealt with the way sysadmins expect.

The first script is called "showbigtimes" and it does just that:

constant@foeni:~/file/projects/futurefile$ pfexec ./showbigtimes_v1.1.d 
dtrace: script './showbigtimes_v1.1.d' matched 7 probes
CPU     ID                    FUNCTION:NAME
  0  18406                  futimesat:entry 
UID: 101, PID: 2826, program: touch, file: blah
  atime: 2071 Jun 23 12:00:00, mtime: 2071 Jun 23 12:00:00

^C

constant@foeni:~/file/projects/futurefile$ /usr/bin/amd64/ls -al /tmp/blah
-rw-r--r--   1 constant staff          0 Jun 23  2071 /tmp/blah
constant@foeni:~/file/projects/futurefile$

Of course, I ran "/opt/local/bin/touch -t 207106231200 /tmp/blah" in another terminal to trigger the probe in the script above (that was a 64-Bit touch compiled from GNU coreutils).

A couple of non-obvious hoops needed to be dealt with:

• To make the script thread-safe, all variables need to be prepended with self->.
• There are two system calls that can change a file's date: utimes(2) and futimesat(2) (let me know if you know of more). The former is very handy, because we can steal the filename from it's second argument, but the latter also allows to just give it a file descriptor. If we want to see the file names for futimesat(2) calls, then we may need to figure them out from the file descriptor. I decided to create my own fd-to-filename table by tapping into open(2) and close(2) calls because chasing down the thread data structures or calling pfiles from within a D script would have been more tedious/ugly.
• Depending on whether we are fired from utimes(2) or futimesat(2), the arguments we're interested in change place, i.e. the filename will come from arg0 in the case of utimes(2) or arg1 if futimesat(2) was used. To always get the right argument, we can use something like probefunc=="utimes"?argo:arg1.
• We can't directly access arguments to system calls and manipulate them, we have to use copyin().

I hope the comments inside the script are helpful. Be sure to check out the DTrace Documentation, which was very useful to me.

The second script is called correctbigtimes.d and it not only alerts us of files being sent into the future, it automatically corrects the dates to the current date/time in order to prevent any time-travel outside the bounds of 32-Bit time_t at all:

constant@foeni:~/file/projects/futurefile$ pfexec ./correctbigtimes_v1.1.d 
dtrace: script './correctbigtimes_v1.1.d' matched 2 probes
dtrace: allowing destructive actions
CPU     ID                    FUNCTION:NAME
  0  18406                  futimesat:entry 
UID: 101, PID: 2844, program: touch, fd: 0, file: 
  atime: 2071 Jun 23 12:00:00, mtime: 2071 Jun 23 12:00:00
Corrected atime and mtime to: 2008 Jul  3 16:23:25

^C

constant@foeni:~/file/projects/futurefile$ ls -al /tmp/blah
-rw-r--r-- 1 constant staff 0 2008-07-03 16:23 /tmp/blah
constant@foeni:~/file/projects/futurefile$

As you can see, we enabled DTrace's destructive mode (of course only for constructive purposes) which allows us to change the time values on the fly and ensure a stable time continuum.

This time, I left out the code that created the file descriptor-to-filename table, because this script may potentially be running for a long time and I didn't want to consume preciuous memory for just a convenience feature (Otherwise we'd kept an extra table of all open files for all running threads in the syste,!). If we get a filename string, we print it, otherwise a file descriptor needs to suffice, we can always look it up through pfiles(1).

The actual time modification takes place inside our local variables, which then get copied back into the system call through copyout().

I hope you liked this little excursion into the year 2038, which can happen sooner than we think for some. To me, this was a great opportunity to dig a little deeper into DTrace, a powerful tool that shows us what's going on while enabling us to fix stuff on the fly.

Update: Ulrich had some suggestions and found a bug, so I updated both scripts to version 1.2:

• It's better to use two individual probes for utimes(2) and futimesat(2) than placing if-then structures based on probefunc. Improves readability, less error-prone, more efficient.
• The predicates are now much simpler due to using struct timeval arrays there already.
• Introduced constants for LONG_MIN and LONG_MAX to improve readability of the predicates.
• The filename table doesn't account for one process opening a file in one thread, then pass the fd to another thread. Therefore, it's better to have a 2-dimensional array with fd and pid as index that is local.
• The +8 in the predicate to fetch was incorrect, +16 or sizeof(struct timeval) would have been correct. That's fixed by using the original structures right from the beginning at predicate time.

The new versions are already linked from above or available here: showbigtimes_v1.2.d, correctbigtimes_v1.2.d.

"Welcome to the year 2038!" has been brought to you by Constantin's Blooog.
This entry was created on 2008-07-03 08:18:35.0 PST and is associated with the following tags:
2038 bug dtrace problem script solaris
You're welcome to use this Permalink , add a comment below or send your feedback to constantin at sun dot com.
Comments [0]

Thursday June 26, 2008

ZFS and Mac OS X Time Machine: The Perfect Team

A few months ago, I wrote about "X4500 + Solaris ZFS + iSCSI = Perfect Video Editing Storage". And thanks to you, my readers, it became one of my most popular blog entries. Then I wrote about "VirtualBox and ZFS: The Perfect Team", which turned out to be another very popular blog article. Well, I'm glad to introduce you to another perfect team now: Solaris ZFS and Mac OS X Time Machine.

Actually, it began a long time ago: In December '06, Ben Rockwood wrote about the beauty of ZFS and iSCSI integration, and immediatley I thought "That's the perfect solution to back up my Mac OS X PowerBook!" No more strings attached, just back up over WLAN to a really good storage device that lives on Solaris ZFS, while still using the Mac OS X native file system peculiarities. But Apple didn't have an iSCSI initiator yet (they still don't have one now) and the only free iSCSI initiator I could find was buggy, unstable and didn't like Solaris targets at all.

Then, Apple announced their Time Machine technology. Many people thought that this was related to them supporting ZFS and in fact, it's easy to believe that Time Machine's travels back in time are supported by ZFS snapshots. But they aren't. In reality, it's just a clever use of hardlinks. And not a very efficient one, too: Whenever a file changes, the whole file gets backed up again, even if you just changed a little bit of it.

Last week, a colleague of mine told me that Studio Networks Solutions had updated their globalSAN iSCSI Initiator software for Mac OS X and that it now works well with Solaris iSCSI targets. I decided to give it another try. So, here are two ZFS ZVOLs sitting on my OpenSolaris 2008.05 home server:

Sun Microsystems Inc.   SunOS 5.11      snv_86  January 2008

-bash-3.2$ zfs list -rt volume 

NAME                             USED  AVAIL  REFER  MOUNTPOINT

santiago/volumes/aperturevault  6.50G   631G  6.50G  -

santiago/volumes/mbptm           193G   631G   193G  -

-bash-3.2$

They have both been shared as iSCSI targets through a single zfs set shareiscsi=on santiago/volumes command, thanks to ZFS' attribute inheritance:

-bash-3.2$ zfs get shareiscsi santiago/volumes

NAME              PROPERTY    VALUE             SOURCE

santiago/volumes  shareiscsi  on                local

-bash-3.2$ zfs get shareiscsi santiago/volumes/aperturevault

NAME                            PROPERTY    VALUE                           SOURCE

santiago/volumes/aperturevault  shareiscsi  on                              inherited from santiago/volumes

-bash-3.2$ zfs get shareiscsi santiago/volumes/mbptm

NAME                    PROPERTY    VALUE                   SOURCE

santiago/volumes/mbptm  shareiscsi  on                      inherited from santiago/volumes

-bash-3.2$ 

On the Mac side, they show up in the globalSAN GUI just nicely:

And Disk Utility can format them perfectly as if they were real disks:

Time Machine happily accepted one of the iSCSI disks and synced more than 190GB to it just fine and as I type these lines, Aperture is busy syncing more than 40GB of photos to the other iSCSI disk (it wouldn't accept a network share). Sometimes, they're busy working simultaneously :).

Of course, iSCSI performance heavily depends on network performance, so for larger transfers, a cable connection is mandatory. But the occasional Time Machine or Aperture sync in the background runs just fine over WLAN.

So finally, Solaris and Mac fans can have a Time Machine based on ZFS, with real data integrity, redundancy, robustness, two different ways of travelling through time (ZVOLs can be snapshotted just like regular ZFS file systems) and much more.

Many thanks to Christiano for letting me know and to the guys at Studio Network Solutions for making this possible. And of course to the ZFS team for a wonderful piece of open storage software!

"ZFS and Mac OS X Time Machine: The Perfect Team" has been brought to you by Constantin's Blooog.
This entry was created on 2008-06-26 14:40:55.0 PST and is associated with the following tags:
backup globalsan iscsi mac machine os storage time x zfs
You're welcome to use this Permalink , add a comment below or send your feedback to constantin at sun dot com.
Comments [4]

Tuesday May 27, 2008

OpenSolaris Home Server: ZFS and USB Disks

A couple of weeks ago, OpenSolaris 2008.05, project Indiana, saw its first official release. I've been looking forward to this moment so I can upgrade my home server and work laptop and start benefiting from the many cool features. If you're running a server at home, why not use the best server OS on the planet for it?

This is the first in a small series of articles about using OpenSolaris for home server use. I did a similar series some time ago and got a lot of good and encouraging feedback, so this is an update, or a remake, or home server 2.0, if you will.

I'm not much of a PC builder, but Simon has posted his experience with selecting hardware for his home server. I'm sure you'll find good tips there. In my case, I'm still using my trusty old Sun Java W1100z workstation, running in my basement. And for storing data, I like to use USB disks.

USB disk advantages

This is the moment where people start giving me that "Yeah, right" or "Are you serious?" looks. But USB disk storage has some cool advantages:

• It's cheap. About 90 Euros for half a TB of disk from a major brand. Can't complain about that.
• It's hot-pluggable. What happens if your server breaks and you want to access your data? With USB it's as easy as unplug from broken server, plug into laptop and you're back in business. And there's no need to shut down or open your server if you just want to add a new disk or change disk configuration.
• It scales. I have 7 disks running in my basement. All I needed to do to make them work with my server was to buy a cheap 15 EUR 4-port USB card to expand my existing 5 USB ports. I still have 3 PCI slots left, so I could add 12 disks more at full USB 2.0 speed if I wanted.
• It's fast enough. I measure about 10MB/s in write performance with a typical USB disk. That's about as fast as you can get over a 100 MBit/s LAN network which most people use at home. As long as the network remains the bottleneck, USB disk performance is not the problem.

ZFS and USB: A Great Team

But this is not enough. The beauty of USB disk storage lies in its combination with ZFS. When adding some ZFS magic to the above, you also get:

• Reliability. USB disks can be mirrored or used in a RAID-Z/Z2 configuration. Each disk may be unreliable (because they're cheap) individually, but thanks to ZFS' data integrity and self-healing properties, the data will be safe and FMA will issue a warning early enough so disks can be replaced before any real harm can happen.
• Flexibility. Thanks to pooled storage, there's no need to wonder what disks to use for what and how. Just build up a single pool with the disks you have, then assign filesystems to individual users, jobs, applications, etc. on an as-needed basis.
• Performance. Suppose you upgrade your home network to Gigabit Ethernet. No need to worry: The more disks you add to the pool, the better your performance will be. Even if the disks are cheap.

Together, USB disks and ZFS make a great team. Not enterprise class, but certainly an interesting option for a home server.

ZFS & USB Tips & Tricks

So here's a list of tips, tricks and hints you may want to consider when daring to use USB disks with OpenSolaris as a home server:

• Mirroring vs. RAID-Z/Z2: RAID-Z (or its more reliable cousin RAID-Z2) is tempting: You get more space for less money. In fact, my earlier versions of zpools at home were a combination of RAID-Z'ed leftover slices with the goal to squeeze as much space as possible at some reliability level out of my mixed disk collection.
  But say you have a 3+1 RAID-Z and want to add some more space. Would you buy 4 disks at once? Isn't that a bit big, granularity-wise?
  That's why I decided to keep it simple and just mirror. USB disks are cheap enough, no need to be even more cheap. My current zpool has a pair of 1 TB USB disks and a pair of 512 GB USB disks and works fine.
  Another advantage of this aproach is that you can organically modernize your pool: Wait until one of your disks starts showing some flakyness (FMA and ZFS will warn you as soon as the first broken data block has been repaired). Then replace the disk with a bigger one, then its mirror with the same, bigger size. That will give you more space without the complexity of too many disks and keep them young enough to not be a serious threat to your data. Use the replaced disks for scratch space or less important tasks.
• Instant replacement disk: A few weeks ago, one of my mirrored disks showed its first write error. It was a pair of 320GB disks, so I ordered a 512GB replacement (with the plan to order the second one later). But now, my mirror may be vulnerable: What if the second disk starts breaking before the replacement has arrived?
  That's why having a few old but functional disks around can be very valuable: In my case, took a 200GB and a 160GB disk and combined them into their own zpool:
  

  zpool create temppool c11t0d0 c12t0d0

  Then, I created a new ZVOL sitting on the new pool:
  

  zfs create -sV 320g temppool/tempvol

  Here's out temporary replacement disk! I then attached it to my vulnerable mirror:
  

  zfs attach santiago c10t0d0 /dev/zvol/dsk/temppool/tempvol

  And voilá, my precious production pool stated resilvering the new virtual disk. After the new disk arrived and has been resilvered, the temporary disk can be detached, destroyed and its space put to some other good use.
  Storage virtualization has never been so easy!
• Don't forget to scrub: Especially with cheap USB disks, regular scrubbing is important. Scrubbing will check each and every block of your data on disk and make sure it's still valid. If not, it will repair it (since we're mirroring or using RAID-Z/Z2) and tell you what disk had a broken block so you can decide whether it needs to be replaced or not just yet.
  How often you want to or should scrub depends on how much you trust your hardware and how much your data is being read out anyway (any data that is read out is automatically checked, so that particular portion of the data is already "scrubbed" if you will). I find scrubbing once every two weeks a useful cycle, othery may prefer once a month or once a week.
  But scrubbing is a process that needs to be initiated by the administrator. It doesn't happen by itself, so it is important that you think of issuing the "zpool scrub" command regularly, or better, set up a cronjob for it to happen automatically.
  As an example, the following line:
  

  23 01 1,15 * * for i in `zpool list -H -o name`; do zpool scrub $i; done

  in your crontab will start a scrub for each of your zpools twice a month on the 1st and the 15th at 01:23 AM.
• Snapshot often: Snapshots are cheap, but they can save the world if you accientally deleted that important file. Same rule as with scrubbing: Do it. Often enough. Automatically. Tim Foster did a great job of implementing an automatic ZFS snapshot service, so why don't you just install it now and set up a few snapshot schemes for your favourite ZFS filesystems?
  The home directories on my home server are snapshotted once a month (and all snapshots are kept), once a week (keeping 52 snapshots) and once a day (keeping 31 snapshots). This gives me a time-machine with daily, weekly and monthly granularities depending on how far back in time I want to travel through my snapshots.

So, USB disks aren't bad. In fact, thanks to ZFS, USB disks can be very useful building blocks for your own little cost-effective but reliable and integrity-checked data center.

Let me know what experiences you made while using USB storage at home, or with ZFS and what tips and tricks you have found to work well for you. Just enter a comment below or send me email!

"OpenSolaris Home Server: ZFS and USB Disks" has been brought to you by Constantin's Blooog.
This entry was created on 2008-05-27 13:40:54.0 PST and is associated with the following tags:
disks howto opensolaris server solaris usb zfs
You're welcome to use this Permalink , add a comment below or send your feedback to constantin at sun dot com.
Comments [11]

Monday May 26, 2008

HELDENFunk Podcast featured in "Blick über den Tellerrand"

Our HELDENFunk podcast, part of the german Systemhelden.com sysadmin portal (If you don't understand German, you may prefer systemheroes.co.uk) has been featured in episode #166 of Alex Wunschel's "Blick über den Tellerrand". Watch out after minute 21:50.

The "Blick" is a weekly podcast in german about the "Blogosphere, Podosphere, Web x.0 and user/corporate-generated knick-knack". If you understand german and are interested in how social media is conquering Germany, this podcast is a must-listen. The german saying "Blick über den Tellerrand" means "glance across the edge of the plate", which is the german version of "glancing beyond one's nose".

The hot topic discussed in this and the preceding episodes is about Germany's public broadcasting agencies. On one hand, they get money from everybody who owns a radio, TV or a computer (read: Everyone, like a tax) and they're supposed to use it to create high-quality programming. On the other hand, the current draft of their "Rundfunkstaatsvertrag" (broadcasting state contract) forbids them to use more than 5% of the budget for online media. Their stance in this dilemma is published in the form of a controversial documentary called "Quoten, Klicks und Kohle" which can be loosely translated to "Vieweing Figures, Clicks and Dough".

You and I, but not enough people apparently, know that all media is significantly moving towards online ways of distribution. In fact, according to a study made by Bonn University and IBM, classic TV is losing importance, in particular among the younger generations and may become less siginificant than online media quite soon.

As part of this discussion, Alex is receiving quite a lot of feedback via email, phone and as MP3 files, which is where the HELDENFunk podcast is being mentioned in the current episode.

But who is this "Kontainer Kalle" guy?

"HELDENFunk Podcast featured in "Blick über den Tellerrand"" has been brought to you by Constantin's Blooog.
This entry was created on 2008-05-26 01:49:23.0 PST and is associated with the following tags:
germany heldenfunk podcast podcasting podpimp systemhelden systemhelden.com tellerrand
You're welcome to use this Permalink , add a comment below or send your feedback to constantin at sun dot com.

Tuesday May 20, 2008

Geek Marketing

This morning, I listened to the "Blick über den Tellerrand" podcast, where Alex Wunschel gives us his thoughts and findings on the "Blogosphere, Podosphere, Web x.0 and User/Corporate-generated Knicknack". In one of his latest episodes, he interviewed Steve Rubel. 

Steve Rubel is a PR expert and author of one of the most read blogs called "Micropersuasion". In one of his articles, published through his employer Edelman Digital, he lists nine digital trends for the future. One of the trends that caught my attention is "Geek Marketing".

In a blog post, he further explains the concept: Technology is moving so fast, that marketing divisions are increasingly hiring geeks to help them understand developments in IT better. A geek marketer therefore is a link between technology and marketing people.

Cool. That's what I've been doing over the last year or so. I just didn't know there was a name for it! 

At Sun, we have a lot of geeks (they're mostly called "SEs" or "System Engineers") and we don't have much marketing (we'd rather spend the money on creating cool technology such as ZFS, UltraSPARC T2 or project Blackbox to name a few). But those few marketers we have, are really at the edge of the digital age. Starting from our long history of employee blogs, through our presence on Second Life to the Systemhelden.com portal (with the HELDENFunk podcast) - Our marketing people know what's hot and cool in the digital world, and how to engage the Sun geek communities to help them make cool stuff happen.

This fits very well with the book I'm currently reading: Dan Pink "A Whole New Mind". In this book, Pink argues that the virtues of the left half of the brain (typically associated with logical thinking) are not enough for today's global and networked world, in which left-brain work is increasingly outsourced to Asia and other emerging countries. In other words: If you want to keep your job, you better start thinking about your right half of your brain.

Right-brain work is quite interesting. Pink introduces six new aptitudes: Design, Story, Symphony, Empathy, Play, Meaning are senses that complement the Information Age worker's logical skills and help him be successful in the new "Conceptual Age".

So, without knowing, by participating in all this video, podcasting, blogging and Web 2.0 stuff, studying better ways of doing presentations and gaming on a Wii, DS or a Playstation, we're actually training the right half of our brains.

Who knew work could actually be that much fun?

Edit:

Added a link to the actual Tellerrand episode. Bummer!

P.S.: The photo shows me in front of a lot of technology. I find this quite fitting the geek theme. The technology is actually a nuclear fusion reactor in Garching. Yes, the kind of stuff you see in superhero movies right before someone gets a new superpower. Didn't seem to work on me, though. 

"Geek Marketing" has been brought to you by Constantin's Blooog.
This entry was created on 2008-05-20 14:10:28.0 PST and is associated with the following tags:
geek marketing podcasting web2.0
You're welcome to use this Permalink , add a comment below or send your feedback to constantin at sun dot com.
Comments [1]

Friday May 16, 2008

Detlef Drewanz on Virtualization in the POFACS Podcast

If you understand german, are interested in virtualization and listen to podcasts, don't miss the current episode of the POFACS podcast.

POFACS, the podcast for alternative computer systems is a german podcast that coveres everything non-mainstream in computing. From people running their business on a Commodore 64 to the state of the art Amiga OS to office packages that fit on a floppy disk or one of the many Linux variants.

There have been a few episodes covering Solaris related technologies, such as ZFS and Project Indiana. Today adds an interview with my colleague Detlef from Berlin about virtualization.

Actually, whenever I listen to one of the POFACS episodes about some crazy new operating system that's being developed somewhere, I've always liked to try it out and see how it is. The perfect way to do that of course is to use virtualization, so you don't have to re-install your machine again. Well, that's where Sun's VirtualBox comes in: It comes with a great range of supported operating systems so there's a good chance it will run even the strangest alternative OS just fine.

But now, let me download Detlef's interview myself and listen to is. Enjoy!

"Detlef Drewanz on Virtualization in the POFACS Podcast" has been brought to you by Constantin's Blooog.
This entry was created on 2008-05-16 07:52:28.0 PST and is associated with the following tags:
german podcast pofacs solaris virtualization
You're welcome to use this Permalink , add a comment below or send your feedback to constantin at sun dot com.

Friday May 02, 2008

Favourite Free Fun Geek Cartoons to Cheer you up!

World economy bad? Financial results lower than expected? Stock price down the toilet? Or just bad weather?

No need to worry, last time I checked, after rain, always came the Sun, and it was stronger than ever!

Meanwhile, let me cheer you up with some favourite geek cartoons of mine:

User Friendly

This strip depicts the life of the heroic employees of an ISP called "Columbia Internet". It's something like the Dilbert of sysadmins, if you will. Lots of fun references to geek culture. And if you travel often by plane, you'll enjoy the strip above, it's ah, so true... For more background, read the Wikipedia entry on User Friendly.

XKCD

Above you see the future of the Solaris Network Auto-Magic (NWAM) feature. How could the author know? xkcd is "a webcomic of romance, sarcasm, math, and language", and very funny. It became famous for it's depiction of online communities as a world map. Read more about xkcd in its about page.

Geek and Poke

...and now you know the real reason why Peter and I like to drive Enterprise 2.0 at Sun :). Geek and Poke is another self-published cartoon by a guy from Hamburg called Oliver Widder.

All of these cartoons come with a license to make them redistributable, so I'm glad I could put some of my favourite strips on this page and I hope they have cheered you up a bit :). If you want some more fun, it's easy: Just click on the cartoons above to see more. But be careful, they are addictive...

What other great geek cartoons did I miss? What are your favourites? Leave a comment!

"Favourite Free Fun Geek Cartoons to Cheer you up!" has been brought to you by Constantin's Blooog.
This entry was created on 2008-05-02 13:42:54.0 PST and is associated with the following tags:
comics fun geek humour
You're welcome to use this Permalink , add a comment below or send your feedback to constantin at sun dot com.

Tuesday April 29, 2008

Interview with the GSE Divas

The GSE Divas are Maria and Candace from New Jersey who work for Sun's GSE (Global System Engineering) organization as communication managers.

If you're into communication, you can hardly live without to blog, and so they keep the GSE Divas blog. One of their blog traditions is to interview GSE people and feature them as "Sun STARS" on a regular basis.

Today, I'm their victim number 15. Feel free to read their interview with me. Thanks!

This is probably the closest thing to an "about me" page for now. Every good blog seems to have an "about me" article somewhere, I should probably sit down and write such an article sometime soon.

Meanwhile, check out the other entries and Sun STARS in the GSE Divas blog, there are a lot of interesting people there, indeed!

P.S.: And yes, it's a pleasure to blogroll you :).

"Interview with the GSE Divas" has been brought to you by Constantin's Blooog.
This entry was created on 2008-04-29 00:54:46.0 PST and is associated with the following tags:
aboutme blogging blogs gse interview life
You're welcome to use this Permalink , add a comment below or send your feedback to constantin at sun dot com.
Comments [2]

Monday April 28, 2008

Presenting images and screenshots the cool, 3D, shiny way

If you give a presentation about hardware products, it is easy to make your slides look good: Remove boring text, add nice photos of your hardware and all is well.

But what if you have to present on software, some web service or give a Solaris training with lots of command line stuff?

Sure, you can do screenshots and hope that the GUI looks nice. Or use other photos (like the one to the left) that may or may not relate to the software you present about.

But screenshots and photos (to a lesser degree) are so, well, 2D. They look boring. Wouldn't it be nice to present your screenshots the way Apple presents its iTunes software? Like add some 3D depth to your slide-deck or website, with a nice, shiny, reflective underground?

Well, you don't need to spend thousands of dollars with art departments and graphics artists (they'd be glad to do something different for a change) or work long hours with Photoshop or the Gimp (a most excellent piece of software, BTW),  trying to create that stylish 3D look. Here's a script that can do this easily for you!

You're probably wondering why my daughter Amanda shows up at the top of this article. Well, she was volunteered to be a test subject for my new script. The script uses ImageMagick and POV-Ray in a similar way to my earlier photocube script that we now use to generate the animated cube of the HELDENFunk show notes. It places any image you give it into a 3D space and adds a nice, shiny reflection to it. Let's see how Amanda looks like after she's been through the featurepic.sh script:

-bash-3.00$ ./featurepic.sh -s 200 Amanda_small.jpg
Fetching and pre-processing file:///home/constant/software/featurepic/Amanda_small.jpg
Rendering image.
Writing image: featurepic.jpg
Ready.

The size (-s) parameter defines the length of either width or height of the result image, whichever is larger. In this case, we choose an image size of a maximum of 200x200 pixels, so the image can fit this blog. You can see the result to the right. Nice, eh?

As you can see, her picture has now been placed into a 3D scene, slightly rotated to the left, onto a shiny, white surface. More interesting than the usual flat picture on a blog, isn't it?

The script uses POV-Ray to place and rotate the photo in 3D and to generate the reflection. ImageMagick is used for pre- and post-processing the image. The reflection is not real, it is actually the same picture, flipped across the y axis and with a gradient transparency applied to it. That way, the reflection can be controlled much better. I tried the real thing and it didn't want to look artistic enough :).

The amount of rotation, the reflection intensity and the length of the reflective tail can be adjusted with command-line switches, so can the height of the camera. Here's an example that uses all of these parameters:

-bash-3.00$ ./featurepic.sh -h
Usage: ./featurepic.sh: [-a angle] [-c cameraheight] [-p] [-r reflection] [-s size] [-t taillength] image
-p creates a PNG image with transparency, otherwise a JPEG image is created.
Defaults: -a 15 -c 0.3 -r 0.3 -s 512 -t 0.3
-bash-3.00$ ./featurepic.sh -a 30 -c 0.1 -r 0.8 -s 200 -t 0.5 Amanda_small.jpg
Fetching and pre-processing file:///home/constant/software/featurepic/Amanda_small.jpg
Rendering image.
Writing image: featurepic.jpg
Ready.

The angle is in degrees and can be negative. One good thing about rotating the image into 3D is that you gain horizontal real estate to fit that slightly longer bullet point in. It helps you trade-off image width for height without losing too much detail. An angle value of up to 30 is still ok, I wouldn't recommend more than that.

The camera height (-c) value is relative to the picture: 0 is ground level, 1 is at the top edge. The camera will always look at the center of the image. Camera height values below 0.5 are good because a camera below the subject makes it look slightly more impressing. Values above 0.5 make you look down at the picture, making it a bit smaller and less significant.

The reflection intensity (-r) goes from 0 (no reflection) to 1 (perfect mirror) while the length of the reflection (the fade-off "tail", -t) goes from 0 (no tail) to 1 (same size as image). Smaller values for reflection and the tail length make the reflection more subtle and less distracting. I think the default values are very good for most cases.

Check out the -p option for a nicer way to integrate the resulting image into other graphical elements of your presentation. It creates a PNG image with a transparency channel. This means you can place it above other graphical elements (such as a different background color) and the reflection will still look right. See the next example to the right, where Amanda prefers a pink background. Keep in mind that the rendering step still assumes a white background, so drastic changes in background may or may not result in slight artifacts at the edges.

You can also use this script with some pictures of hardware to make them look more interesting, if the hardware shot is dead front and if it doesn't have any border at the bottom. Use an angle value of 0, this will place your hardware onto that virtual glossy carbon plastic that makes it look nicer. See below for an embellished Sun Fire T5440 Server, the new flagship in our line of Chip-Multi-Threading (CMT) servers.

This script should work on any unixoid OS, especially Solaris, that understands sh and where a reasonably recent (6.x.x) version of ImageMagick and POV-Ray are available.

You can get ImageMagick and POV-Ray from their websites. On Solaris, you can easily install them through Blastwave. The version of ImageMagick that is shipped with Solaris in /usr/sfw is not recent enough for the way I'm using it, so the Blastwave version is recommended at the moment.

The featurepic.sh script is free, open source, distributed under the CDDL and you don't have to attribute its use when using the resulting images in your own presentations, websites, or other derivative work.

It's free as in "free beer". Speaking of which, if you like this script, leave a comment or send me email at constantin at sun dot com telling me what you did with it, what other features you'd like to see in the script and where I can meet you for some beer :).

"Presenting images and screenshots the cool, 3D, shiny way" has been brought to you by Constantin's Blooog.
This entry was created on 2008-04-28 00:44:56.0 PST and is associated with the following tags:
3d cool effects howto imagemagick imaging photos povray presentations script
You're welcome to use this Permalink , add a comment below or send your feedback to constantin at sun dot com.
Comments [2]

Monday April 21, 2008

On Knowledge Management, Community Equity and Ontologies

Last week, I attended a meeting of the BITKOM Working Group for Knowledge Engineering & Management at the Sun Frankfurt office. The meeting was very nicely organized by Mr. Weber, Mr. Neuwirth and some colleagues from Sun in Germany (Hi Hansjörg, you should really blog!) and Peter Reiser from Sun in Switzerland. Therefore, I got to play host of the meeting without having to do too much work :).

Peter asked me to present his work on Community Equity (see also this interview with Shel Israel and this other one with Robert Scoble) and the CE 2.0 project to the group. The working group was very interested in how to encourage communities to participate and how Community Equity mechanisms can be used towards this goal. We had quite a few positive discussions during the breaks.

But, some people seem to be concerned with tracking community contribution and participation on an automatic basis, for example, see Mike's post on the subject and Alec's reaction to Peter's interview. These are all very valid thoughts, and indeed nobody wants to see their work or life be reduced into a couple of numbers.

As always, the threat is not in the technology, but in the way we use it:

• Measuring stuff is a good thing, if you know what you measure and how accurate that measurement is.
• Telling people how their work is being received is also a good thing. I always get a kick out of the HELDENFunk download statistics (We should probably start publishing them), or my own blog's metrics. This is a huge motivator.
• Telling people about how other people's work has been received is also a good thing. Nobody would put the kind of trust into eBay if it weren't for their rating system. How many books have you bought on Amazon based on other people's recommendations, stars, etc. on their site?
• Web 2.0 style commenting, crosslinking, social networking, tagging and rating is also a good thing. Much of the web 2.0 world today would be untrusted, unnavigationable and unuseful if it weren't for those mechanisms.
  
• The next step is to take these concepts, and apply them to an enterprise context. This is what Peter's Community Equity work is all about. The goal I see here is: If you do a good job, others should be able to notice (including, but not limited to, your manager). If you're looking for an expert on topic X, you should be able to find people that may be able to help you. If you are talking to person Y or if you run into that person as part of a team, you should be able to see what kind of work that person has contributed to the enterprise before and what others are saying about them. Think Amazon and eBay and LinkedIn ratings, recommendations, tags etc. as a tool to better navigate the social network and knowledge base of your enterprise.
  

Notice that the part where discussions become heated is not the technology one, it's the "what do we do with the numbers" part. That, of course, is where we need to be careful. We need to understand how the data is generated, how it has been processed (i.e. the exact rules and formular that is used to generate the Community Equity score) and what it does not tell us. You may trust your latest auction winner to transact with you on that particular sale, but you still don't know if she is actually a nice person or not :).

As long as the process is open, well-understood and transparent, using Web 2.0 mechanisms and Community Equity style metrics can be a very useful thing. You can generate a lot of useful information based on that kind of data: What are hot topics? Which documents are the most used, best rated, most re-used ones? Who are the company internal creators, connectors and consumers of knowledge? What topics have trouble to be picked up by the community? Sounds like fascinating stuff, if you're responsible for your company's knowledge...

Of course, this was only a small part of the BITKOM meeting. We heard presentations by other companies on different applications of knowledge management technologies in a customer service context. Interestingly, all of them (including CE 2.0) mentioned the term Ontology in one way or other. In a knowledge management context, an Ontology is the part of the system that relates "words" or other abstract data to real-world concepts and objects, resolving ambiguities, consolidating synonyms and clarifying user-errors. It's the part of the system that tries to bring in semantic knowledge as opposed to mere processing words.

Ontologies are very hard to do. That's why most of the times they are generated "by hand" which is very time and resource consuming. The holy grail of ontologies is when the system can automatically generate semantic meaning out of naked data by itself, without any help. Some of this systems are seeded with hand-made ontologies that can then expand somewhat automatically.

An interesting approach to generating ontologies might be to analyze web 2.0 style tagging data that has been created by users. An ontology system could then try to identify clusters of tags and assign them to a real world concept, then try to identify relationships between those concepts. As an example, the tags "LDAP", "Directory Server", "DS" all belong to the same concept and they are related to (but not the same as) "Identity Management", "IdM", and "Databases". A search engine then can use this data to find better matches for a user that is looking for "Identity Management and LDAP interoperability".

As you can see, even a seemingly dry and academic workshop on "Knowledge Engineering and Management", organized by an industry association can be an exciting topic, sometimes transcending the boundaries between technology, philosophy and anybody's daily web 2.0 style work.

"On Knowledge Management, Community Equity and Ontologies" has been brought to you by Constantin's Blooog.
This entry was created on 2008-04-21 04:10:00.0 PST and is associated with the following tags:
ce2.0 collaboration community equity knowledge management ontology web2.0
You're welcome to use this Permalink , add a comment below or send your feedback to constantin at sun dot com.
Comments [2]