Monday January 22, 2007

Using the OpenSolaris Mercurial repository
I finally decided to have a look at how you get OpenSolaris from the Mercurial repository I've been put off by the fact at the office we're behind a fire wall, so you have to pull it through a Socks proxy.

I turns out it's absolutely trivial to do (the instructions are all on the OpenSolaris WebSite.

If you're behind a firewall that requires you to have a proxy for ssh th first thing you need to do is set up ssh to use a proxy. I can't help you with that other than to say add the following line to your ssh config for opensolaris.org

$ cat ~/.ssh/config
Host *.opensolaris.org
ProxyCommand /usr/lib/ssh/ssh-socks5-proxy-connect -h [Proxy IP address] %h %p
Compression yes
(thanks to Erik and Stephen for pointing out the compression option)
Putting in the IP address of your socks proxy.

Now you can clone the repository

$ hg clone ssh://anon-AT-hg.opensolaris-DOT-org/hg/onnv/onnv-gate
adding changesets
adding manifests
adding file changes
added 3487 changesets with 67524 changes to 43099 files
39742 files updated, 0 files merged, 0 files removed, 0 files unresolved


It took only 37 minutes to my home machine which is much faster than a full bringover in teamware

Now to start playing with some of the build tools
Posted by cwb ( Jan 22 2007, 06:32:11 PM GMT ) Permalink Comments [5]

Monday January 08, 2007

Is it 2007 already?

Spider-Man 70% Superman 65% Robin 60% Green Lantern 60% Iron Man 55% Supergirl 53% Wonder Woman 43% The Flash 40% Catwoman 35% Hulk 35% Batman 35%

You are intelligent, witty, a bit geeky and have great power and responsibility.

Sunday September 24, 2006

EuroOSCON 2006 EuroOSCON 06

This week (18th - 21st September) I've had the opportunity to attend and be involved in EuroOSCON06. This was primarily to increase my understanding of opensource, but also to promote Open Solaris. This was my first OpenSource conference so first I'll make a few general observations before moving on to details about the sessions.

EuroOSCON06 was this year in Brussels, a city I'd not visited before. It's a surprisingly small city for the self proclaimed capital of europe. There are some very beautiful parts as well as some rather seedy parts, and being small the seedy and wonderful nestle uncomfortably together.

There were some other Sun employees (Martin Man Peter Dennis, Patrick Finch, Darren Kenney and Gary Pennington). I'd met a few of them before but we're from very different backgrounds so we had different reasons for wanting to attend EuroOSCON and promote OpenSolaris.

So I took the EuroStar from London (I was booking the trip just as the security scare happened last month so thought this would be easiest). Met with Peter Dennis on the train and worked through some demos we could show to people. We had a BOF and a Booth on Wednesday so thought we'd try and show some cool stuff.

The Demos

We decided we'd show how easy it was to set up and build OpenSolaris. I had a media kit with me on the train and by the time I was in Brussels had installed a build machine environment on my laptop and was happily building code, cool!. We also wanted to show some zfs features, and some zones features. There is a new facility in OpenSolaris to allow you to create a Zone on one system (preferably on a zfs file system, and then take a copy of it to create a new zone. If your using zfs it will sanpshot the filesystems rather than copying data meaning you get use the zfs snapshot facility meaning it is rather quick. This is done with zonecfg clone -s

You can then dettach that zone from your current system (using zoneadm dettach),and as the zpool was on an external disk we moved the USB disk to another laptop and imported the pool (zpool import ) and attach the zone to the new laptop (zoneadm -z attach -F -n ). It then just works as it did on the old system. I was amazed and can see how useful this is going to be.

Observations

Some things struck me. First - everyone uses Mac their. This surprised me as although it is based on an opensource OS, it is far from and OpenSource product. Second - There are a lot of people on the conference gravy train, they obviously go to a lot of these. That's fine, but it did distort the audience a bit. Finally Licensing both of Projects and data is still a big worry for this community.

It was well worth going this year to promote OpenSolaris. People were falling queing up at the booth to talk to us so on to some more detail

The Booth and the BOF

Wednesday had us setting up the booth for OpenSolaris and manning it. We had intended to take turns, but we were so busy for the whole of the morning that we all were talking pretty much all morning to various interested people. We had some "OpenSolaris Starter Kits" Containing the install media, the source and the compilers, along with a coupl of livedvd images. Overall they went like hot cakes everyone seemed excited by OpenSolaris. I even persuaded a guy from Google to take a look. The afternoon was a little quieter so we could go off to a few sessions, but Still a steady stream of interested people.

I gave more demos than I can remember - people were excited about how easy it was to use ZFS and how simple building opensolaris was. Dtrace is still a big thing and now it's in macOS the community can see am example of why they should be involved with OpenSolaris

The BOF was at the same time as one from Ubuntu's Mark Shuttleworth, so we weren't expecting many and we had a few, though not as many as I would have hoped. We talked through SMF, Zones, dtrace, ZFS among other things and the people seemed interested. The funny thing was that Mark Shuttleworth failed to turn up and the Ubuntu BOF got rather heated.

The Sessions

KeyNote Speakers: Tim Orielly, Tor Norretranders

Tim Orielly spoke about data, and licensing (as you'll see licensing seems to be a hotly discussed topic at the conference. It's not something that has ever interested me but it is clearly important. Tim was commenting that if you put it on Myspace but can't take it anywhere else is it really yours? Same goes for applications that run on the web server rather than in your own system, what is to stop the owner of the webserver doing what ever they want. I feeling he was blowing his own trumpet saying he'd pointed this out in 1998 or something but only now people are taking him seriously. He was trying to say (I think) that in the Web2.0 world where everyone contributes on line the things you put in your blog or on some one else collab site you may not own the rights to.

Tor gave a very funny speach about the motivations for participating in opensource. He stated it was about

Glow-> The nice feeling we get when interacting with someone else
Show -> Showing off and helping others gets you noticed (or laid as he put it)
Flow -> You are constantly changing.
2.0 -> We're getting back to a bartering type economy.

Certainly I enjoyed it and the first two points are clearly right, the rest felt a little forced, but then it was well made.

Industrial strength Email and Calendar: Flaorian von Kurnatowski

Without realising it I'd wandered in to the Products and Services track. Basically Opensource friendly companies promoting their products. That said he didn't push his company Scalix too much. What he observered was that you needed to have a true replacement for Outlook before people would be able to move away from Microsoft. It seems Outlook is very closely tied to all other Microsoft apps and if you remove them you loose a lot of functionality (and he said Outlook is 50% of the license fee too).

Until OpenOffice can provide that or has an equivalent it will not considered by many people. Also 90% of Admins have never done a migration of mail systems so they're scared of it, there need to be good migration tools.

Final point was that Calendar services do not have any standards which is why Calendar infrastructure is even harder to do than email.

Channeling OpenSourced in Europe: Ranga Tangachari

Back in the OpenSource world I was interested by this session. I'd assumed that this would be about getting the most out of OpenSource in Europe but instead it was a talk about how his company made money in Opensource by encouraging the Channel (resellers)

His assertion was that Communities provide innovations and companies provide Products (more the just projects, fully tests and supported things). In the middle are the Channel which adds value by things like locaization and training. Being a pool of deployment experts.

You need to encourage the Channel by giving them what they care about which is

1) Margin
2) Professional Services Opportunities
3) Maintenance (recurring revenue)

Think beyound Downloads they only mean one click, find examples of happy customers.

Big Data and the Open Warehouse: Roger Magoulas

This was a dissapointing presentation about what Orielly do about data storage and data mining. It was unfortunately simply a run down of tricks tips, products and techinques used by the Orielly guy in their data centre. There were some interesting things mentioned though which I will go and look at.

SecondLife and Opensource: Jim Purbrick

I'm intrigued by SecondLife, it's a game where the whole purpose is to make "Stuff" and "Hang Out" and generally share or sell what you do. I have looked at it and it is cool, but I haven't got my head round Why? yet.

SecondLife is not (yet) opensource but the Guy from LindenLab was explaining that the big difference with second life to other MMORPG games is that the players create the world. LindeLabs couldn't have provided enough content to keep people interested, but because it is created by the game community they reacon they get ~6500 man years of content development per year! (not even EA could manage that for one game I think) All of this is a course up for sale or copy depending on the desires of the community member

There are interesting aspects to the way LindenLab have architected their set up, like each now plot of land requires a new server so they're adding new servers at a huge rate.

Over all the crowd were excited by SecondLife. I didn't see them as engaged in any other session

Afternoon Keynotes: Steve Coast, Adrian Holovaty

Steve Coast: Open Data: An interesting view of "Good enough" data. He uses a gps held by volunteers as they go about there daily business to look for a plan of cities, making it available via OpenStreetMap. Most map data in the world is either govenement owned (like the Ordanance Survey) or not of great quality. Creating good enough Open Data which can be shared will be enough for some and will consequently bring down the price of Closed Data).

Adrian Holovaty: Journalism through Programming:

Facinating and slightly scarey view on providing access to the raw data used by journalists via web applications. He works for the Washington Post and is involved in a few projects one of which is Faces of the Fallen. Another one he quoted was to look at your MPs voting reacord TheyWorkForYou. Now I can see the point of this, and will certainly be checking on what my MP is doing, but I am slightly worried that this requires balance. We're asking the public to draw conclusions from only a small amount of the data (as the records published on line are incomplete), where as journalism is all about weighing up all the information and providing a balanced summary for consumption.

Open Useability: Jan Muehlig

A talk to encourage open source projects to include Useability eningeering in the project to make the user experience of Open Source Products as good or better than closed source, something we have often complained about. His greoup OpenUseability is working to promote this and it seems to boil down to publishing best practices, which noone does yet.

Making It Work: Louis Suarez-Potts

A talk about how to build succesful OpenSource projects. He comes from OpenOffice Which is both a succesful project and a really useful product so he should know what he's doing.

He talked about the two different approaches. The organic where a few friends start up with a common goal, and the Sponsored, where a corperate entity is driving towards a set of goals. Eitehr way you need to do the following

Pick the right license
Have a neutral environment (ie safe to contribute)
Have transparent governance and processes
Make decisions in public
Have clear decision paths
Use good communications tools (everyone liks IM these days
Have immediate gratification (easy and fast contribution)
Market your project
Have the right Product

I found this quite encouraging as I felt OpenSolaris has it about right

OpenSource and Freedom: Why Open Standards are crucial to protecting your linux investment: Jim Zemlin

This talk was aimed at promoting the LSB(Linux Standards Base To make sure applications will run on the largest number of Distros. LSB dictates the minimum number of components available within the Distro so your application can rely on them. This is to encourage growth over Microsoft. He quoted what happend in the Unix world when the standards fragmented and he is absolutely right

KeyNote: Florian Muller Roml Lefkowitz

Florian Muller Spoke about lobying in the European Parliament to limit the changes to Patent law which some companies are trying to tighten up to protect their IP, while OpenSource are trying to go the other way. I was left slightly disconcerted that someone with such a one sided view was having an effect on our laws.

Roml Lefkopwitz Spoke about the need to internationalize and localize the source code and languages used in opensource projects. Nice pie in the sky thinking, but misses the point that the source should not be the documentation, we need documentation before we can worry about such things.

Xgl and Compiz - New X11 features and the OpenGL Accelerated Desktop Matthias Hopf

Facinating talk about the future of desktop from Suze At last a talk with lots of technical details and a neat demo at the end demonstrating the desktop mapped on to a 3d cube running two movies and Quake 3 at the same time on different faces of the cube. All of this should soon also be possible in Solaris and I think it's vital we do it.

The End

If you got this far then well done It was a lot to read.

Technorati Tags: Solaris OpenSolaris EuroOSCON EuroOSCON06

Posted by cwb ( Sep 24 2006, 10:35:00 AM BST ) Permalink

Tuesday September 12, 2006

Building Opensolaris

This weekend I did something I've been meaning to do for a while. I've been putting it off due to lack of time to think about how to approach it without breaking anything.

Any way I finally tried downloading and installing Solaris Express the community edition (build 46) and downloaded all the build tools. I was amazed how easy it was. Within an hour offinishing the downloads, it was building opensolaris.

The main shock for me was the lack of an SCM (Source Code Management) system. Being fully entrenched in the world of ON (Os and Networking) for the last 10 years Teamware (the SCM we use) is just *what we do*. So I had to rethink how I'll manage the build. But then using opensolaris.sh from usr/src/tools/env did a good job.

So I know have my own build built from opensolaris running on my system, without anything from within Sun. Cool. Give it a go, it was easy and gives you the chance to play with how things work.

Next step for me - build the Xen bits from outside Sun

==== Nightly distributed build started: Mon Sep 11 11:27:18 BST 2006 ====
==== Nightly distributed build completed: Tue Sep 12 03:22:02 BST 2006 ====

==== Total build time ====

real 15:54:44

==== Nightly argument issues ====

Warning: the N option (do not run protocmp) is set; it probably shouldn't be

==== Build environment ====

/usr/bin/uname
SunOS osol-bld 5.11 snv_46 i86pc i386 i86pc

/opt/onbld/bin/nightly myopensolaris.sh
nightly.sh version 1.104 2006/08/29

/opt/SUNWspro/bin/dmake
dmake: Sun Distributed Make 7.7 2005/10/13
number of concurrent jobs = 4

32-bit compiler
/opt/onbld/bin/i386/cw -_cc
cw version 1.20
primary: /opt/SUNWspro/bin/cc
cc: Sun C 5.8 Patch 121016-02 2006/03/31
shadow: /usr/sfw/bin/gcc
gcc (GCC) 3.4.3 (csl-sol210-3_4-20050802)

64-bit compiler
/opt/onbld/bin/i386/cw -_cc
cw version 1.20
primary: /opt/SUNWspro/bin/cc
cc: Sun C 5.8 Patch 121016-02 2006/03/31
shadow: /usr/sfw/bin/gcc
gcc (GCC) 3.4.3 (csl-sol210-3_4-20050802)

/usr/java/bin/javac
java full version "1.5.0_08-b03"

/usr/ccs/bin/as
as: Sun Compiler Common 10 snv_46 08/03/2006

/usr/ccs/bin/ld
ld: Software Generation Utilities - Solaris Link Editors: 5.11-1.545

Build project: group.staff
Build taskid: 62

==== Build version ====

ws.opensolaris

==== Make clobber ERRORS ====

==== Make tools clobber ERRORS ====

==== Tools build errors ====

==== SCCS Noise (DEBUG) ====

==== Build errors (DEBUG) ====

==== Build warnings (DEBUG) ====

==== Elapsed build time (DEBUG) ====

real 11:42:57.3
user 4:41:00.3
sys 2:40:32.5

Technorati Tags:
Solaris OpenSolaris Xen

Saturday July 29, 2006

And back again Well The ride back yesterday took longer. 65:15. But then I rode back a very different way. One of my colleagues was riding as well so we went back via his house ~10 miles. Practicing drafting, which makes things much faster and easier. Only trouble was that he lives in Wokingham so I had to get back from there. So I started coming the direct route along Reading road, and quickly got fed up with that, so at Winersh headed in to Woodley and back through Sonning. Though this added a couple of miles it was probably nicer riding. So 19 miles in 65 minutes isn't so bad. Though I felt really tired afterwards, and hungry, and hot.... Well that makes 90 miles this week. Most I've ever managed. I'll have to see if I can do it next week Posted by cwb ( Jul 29 2006, 07:00:00 PM BST ) Permalink

Friday July 28, 2006

57:07 So cycled in today in 57:07. Must have been because I was happier in the traffic rather than riding faster. Posted by cwb ( Jul 28 2006, 11:00:00 AM BST ) Permalink

Wednesday July 26, 2006

And home again So the journey home wasn't too bad either. I measured from the cycle lockers home rather than the entrance to the campus (the speedbumps in the campus are so vicious you cannot get a decent speed up). So it was a bit further. 17.4 miles in 63:37. So averaging 16.36 mph. Could have done better really but The bit between shinfield and the M4 on the A327 was up hill and I felt my legs had no energy. (not surprising really) Posted by cwb ( Jul 26 2006, 07:00:00 PM BST ) Permalink Comments [1]

Rode in to work this morning I decided to try cycling to work. It's a 20 mile drive that takes around 45-50 minutes due to the traffic. Going by bike I avoid some of the faster rodes and take the more direct route which is almost exactly 17 miles to the entrance of the campus. So I checked mail from home and set off after that. Did the 17 miles in 59:09. Even taking a shower in to account that's not much slower than driving. Any way - I've just got to do the same on the way home now! Posted by cwb ( Jul 26 2006, 10:10:00 AM BST ) Permalink Comments [1]

Sunday July 23, 2006

What am I listening to Last week I was visiting our colleagues in Prague. Talking about Xen, Dtrace, FMA and other cool stuff, but also helping out with some problems they were having. Anyway I did have a nightmare journey back and was thankfull of two things. A good book (Use of Weapons - Iain M Banks: I love the detail he puts in the worlds he creates), and an IPod.

So what did I listen too? Well I found I had subscribed to a podcast that has some excellent music on it. IndustrialRadio. Plenty of interesting music I'd not heard before. Apprarently the detroit industrial scene is pretty active, which is more than can be said for most places.

Any way back in the real world, I rediscovered I like some Handel, Having bought Essential Handel I've remembered learning Violin as a child. May be I should take up some more.

What else. Well in complete contrast there's been Classic Euphoria Which is pretty much audio wall paper but great for getting stuff done too.

My eldest Son did a part in the school production of Oliver a couple of weeks a go. Today there was a local schools show case of the out of school activities that they've been doing. Naturally eldest was the best, but one other that stood out was another schools "Rock School". They have taken "Feel Good Inc" by Gorillaz and re recorded it using the instruments they can play (The group only sang to a backing track which I guess is fair enough given the size of the venue, and the fact they had no switch over time, Oh and hey're only 8-11). I only realised it wasn't to the real track when I heard recorders on it! Pretty impressive.

And this evening while watching the end of The Tour They had a butchered version of One Which made me put on the original which was much better.
Posted by cwb ( Jul 23 2006, 09:23:00 PM BST ) Permalink Comments [1]

Monday July 17, 2006

Xen for dummies - part1
I've been helping out the Xen team to try out the bits they've put on OpenSolaris over the last week or two. I've been impressed with how much they've got working so far, but as an experienced Solaris user who is new to Xen I've found it quite hard to get my head around what Xen does and how it works.

First off install it using the instructions on OpenSolaris and then you need to get some domains set up.

So boot up under Xen. You'll see from the grub menu (/boot/grub/menu.lst) some things have changed. Instead of booting a Solaris kernel, you boot xen which then loads a solaris kernel in the module line.

#Solaris on Xen 64bit
title Solaris on Xen 64-bit
kernel /boot/amd64/xen.gz dom0_mem=524288 console=com1 com1=9600,8n1
module /platform/i86xen/kernel/amd64/unix /platform/i86xen/kernel/amd64/unix -k
module /platform/i86pc/boot_archive


Note the option dom0_mem=524288. This assigne 512Mb to your Dom0 at startup.

Another thing to note is that even booting on metal (ie not booting the xen hypervisor first) we no longer use multiboot, but can boot our unix directley.

#---------- ADDED BY BOOTADM - DO NOT EDIT ----------
title Solaris Nevada snv_41 X86
kernel /platform/i86pc/kernel/amd64/unix
module /platform/i86pc/boot_archive
#---------------------END BOOTADM--------------------


Read Joe Bonasera's blog about why this is the case.

OK so you've got Solaris booted on Xen. This is refered to as Dom0 you need to set up some more instances of Solaris (or whatever Xen compatible OS you happen to want) and that is refered to as a DomU.

Each DomU has it's own full OS install so for solaris we set up a flar archive and use the vbdcfg script to help us convert that in to a OS instance Xen can boot as a DomU. This is all described here.

A couple of things to think about. Networking is bridged (ie it appears to be directly connected to the outside world) so you're going to need to give it a real IP address or use DHCP (this can be decided as the DomU boots up). Also you need to give it an ethernet address using the -e flag to vbdcfg. As this is a made up ethernet addess I don't know how you're supposed to create it, Dave Edmondson suggested making the first octet 0xaa and encoding the IP address in the rest of it.

So when you've got your DomU setup using vbdcfg what do you get?

Well in /export/xc/xvm you'll have a directory for your DomU domain

$ ls -l /export/xc/xvm/mydomU
total 10
-rw-r--r-- 1 root root 379 Jul 12 09:57 mydomU-64.py
-rw-r--r-- 1 root root 367 Jul 12 09:57 mydomU.py
drwxr-xr-x 4 root root 512 Jul 12 09:48 platform
-rw-r--r-- 1 root root 19 Jul 12 09:48 root.dev
drwxr-xr-x 2 root root 512 Jul 12 09:48 vmnt


the .py files are python scripts used to start the domain, platform is the directory where the kernel to boot is, root.dev contains the name of the root device and vmnt is where the domain can be mounted using

$ vbdcfg mountdomU domU-name


But you don't want to do that now. You want to start up your domain, So this is how you do it

$ xm create mydomU-64.py
(this starts mydomU in 64bit mode)
$ xm console mydomU
(puts you on the console)


or alternatively

$ xm create -c mydomU-64.py


which starts the domU and puts you directly on the console - useful sometimes to find out why your domain isn't coming up

So at that point you on the console of your domain and it's coming up just as if its a fresh install of Solaris.

In future "Xen for Dummies" installments I'll show how to configure the network, add disks, create more cpus in the domU than you have in the real box! and I'm sure I can think of more later.

Xen

Technorati Tags: Solaris OpenSolaris Xen
Posted by cwb ( Jul 17 2006, 10:31:10 AM BST ) Permalink

Friday July 07, 2006

Action Shot The new Pup still doesn't seem to stay still , but we can now take him for walks. As I was working from home today that's what we did at lunch time. He's pretty good at coming when called these days Posted by cwb ( Jul 07 2006, 06:00:00 PM BST ) Permalink Comments [1]

Thursday July 06, 2006

Another Dtrace Customer presentation I had another opportunity to talk to a customer (this time a law enforcement agency) about Dtrace last week. This time it was only a 1/2 hour slot and we had already over run by quite a way so the Presentation was a lot shorter You'll notice the similarity with the previous presentation (well there is no point in reinventing the wheel) In this case the customer was very interested in virtualization. They have a large farm of PCs which are only ~15% utilized. They felt something like Xen or vmware might be able to help bring that up. I'm working with the Xen team right now just getting up to speed on the technology. It looks pretty impressive. Go and check out the Xen Opensolaris community it's pretty interesting now and I'm sure it'll have a load more interesting stuff on it soon Technorati Tags: Solaris OpenSolaris dtrace Xen Posted by cwb ( Jul 06 2006, 11:00:00 AM BST ) Permalink

Wednesday June 21, 2006

Extreme Close Up The new Pup doesn't seem to stay still long enough for a photo. This is the best we've got this week Posted by cwb ( Jun 21 2006, 10:24:00 AM BST ) Permalink

Wednesday June 14, 2006

dtrace presentation I had the pleasant opportunity of visiting an exisiting Sun customer last week who was interested in dtrace and containers. I've been using dtrace since before it was in Solaris 10. Using the bits the development team were working on. In my line of work it's a great boon and I think the customer could see the benefits of the tool and appreciated the insights of someone who used it day in day out. Obviously in an hour or so presentation you can't cover anything in great detail, but I wanted to whet their appettites as to things dtrace could do for them. So I walked them through the reasons why it's a good solution, and the architecure. Then a few examples and demos. Then we talked about how it could solve some of the problems they've experienced. Over all a very good experience and I believe they found it useful. Any way I thought I'd put up the presentation here to remind me that I did it and so someone else might find it useful especially in now there is such a community on opensolaris.org Presentation Technorati Tags: Solaris OpenSolaris dtrace Posted by cwb ( Jun 14 2006, 08:00:00 AM BST ) Permalink

Monday June 12, 2006

New Pup After a 2 month wait our new pup finally arrived, He's a Cocker Spaniel in Chocolate Brown with Tan socks and beard. Very unusual. He settled in pretty well but the last couple of nights have been a bit disturbed. Any way - here are a few photos for good measure Posted by cwb ( Jun 12 2006, 09:15:00 AM BST ) Permalink Comments [2]