Last week, project ILB was finally pushed to the main Solaris (nevada) repository; even though I haven't been a full member of the project team for a few months (not because of ILB's fault, btw), I do take pride in this piece of work, and congratulations to the remaining team members, above all Sangeeta, who hung on during the last few nerve-wracking weeks.
The bits will appear on opensolaris.org with build 128.
See the wiki for more details.
A lot has happened since I wrote last -- among other things, I'm now with the MySQL organisation in the Enterprise Tools team, and one challenge this group is facing is the multitude of platforms being supported.
I needed to test a bug that to all appearences I had introduced, that was only manifesting itself on Linux; since (Open)Solaris is all I had at the time on the machines I use, and I couldn't find a ready-made box running Linux, I had two choices: (re-)install a test box somewhere in the lab or use Virtualbox. I decided to go with the latter, especially as I could choose from quite a few canned OS images for Virtualbox here.
I selected a Ubuntu 64-bit image; download and installation on the lab machine went well, and I had it up & running after a fairly short time. One of the notices that Virtualbox showed me when it booted the Ubuntu VM was something about "mouse integration" being present. I hadn't seen that active before, so I was pleasantly surprised that I didn't have to press the host key every time I wanted to "escape" from the VM, as it were.
A few minutes after logging in the first time, the update manager told me it had some updates ready, did I want them? I said "yes" and let it do its work, including "reboot" at the end. To my dismay, when the VM came back up, there was no mouse integration -- I had to go back to the old behaviour of pressing the host key... I had selected "don't display this warning again" sometime earlier, and initially thought that that may have been the (unexpected) cause.
As it turns out, selecting "install guest additions" from the devices menu and then running the appropriate script got mouse integration back.
I recently made my first attempt to merge our project gate with then-current bits from the Nevada gate; a change I wanted to benefit from had just been put back, and I was eager to get the bits.
Here's what I should have done:
If I'd done it this way, all would have been fine. And I did, except for one step: before the last 'push', I did something I shouldn't have: I issued a 'hg recommit'. If you're groaning now, there's probably no news for you in the rest of this piece ;-). If not, here's a little detail:
Whenever a change to the source is done (eg. a minor milestone or the fix for a bug), you can do 'hg commit' to document this in the your repository. Such an action is also reversible with hg-means; so it makes sense to commit fairly often. Note: commit doesn't change anything in the parent repository, rather, the change in the code is communicated to your working repository. That way, you accumulate several commits before pushing them to the parent repository (or "putting back" in teamware-speak).
When you compare your work with the repository you're planning to push this to, hg uses this history to find out which files actually changed and compare those that did.
Normally, when you want to push an incremental set of changes to your project gate, it's sufficient for these changes to appear as one changeset in the project history (esp if you're planning to integrate with Nevada anyway at a later point in time, where all those changes will again be collapsed into a single change-set). 'hg recommit' collapses all changes into one.
What was different in my situation: by pulling bits from onnv-clone, I'd imported Nevada's change-set history. Unless we'd been planning to integrate our project with Nevada right away (ie before (m)any more changes to Nevada), this history would have been essential for future merges.
By removing this history, I made that step very painful - I essentially declared all changes that Nevada had introduced since we'd started our project gate, and which I'd now incorporated into our gate, as "our" changes (and not as Nevada changes we just synced up with), so every future comparison with Nevada would not only show our project work as changed, but everything that had changed in Nevada since we'd created our gate.
Lesson learned: just because something makes sense in one set of circumstances, it does not necessarily do so when the circumstances change - even if the change is minimal.
I finally got Nevada history back into our repository with a lot of help from Mark J Nelson (and that was not trivial!), and if you want to buy him a beer the next time you see him, you're more than welcome to :-)
I recently had the chance to try a Toshiba Tecra M10 with OpenSolaris for a while; the idea was to find out whether I preferred it to my current one (a lenovo T60p) - its main advantage over the lenovo being that OpenSolaris supports suspend & resume, which the lenovo does not (due to its ATI graphics adapter).
I finally didn't keep it - the M10's screen resolution (1440x900) is less than the lenovo's (1680x1050), and for me the difference made the difference.
Since then, I've started looking around at the laptop market, more out of idle curiosity than anything else, and I came to realise that many vendors whose pages I looked at don't tell you up front about the screen resolution, but only about screen size - am I the only one who's interested in resolution?
here follows the text of an email that I just sent out to ilb-dev@opensolaris.org; if you feel inclined to respond, please do so on the same alias so the discussion can stay in one place. thx.
We need to nail down the semantics and the broad design of "ilbadm export"
and "ilbadm import" (note, this diverges from the spec as it's currently
posted, where we still talk about import-rules etc. This has been simplified).
In addition, there's what we have come to call "persistent configuration",
which is conceptually an extension of ilbd, basically a copy of ildb's
running configuration that persists across start/stop of ilbd (and reboot
of the server). (this has nothing to do with "session persistence", btw)
We have been investigating how closely these are related and what impact
this has; here's some points that we seek understanding and your input on:
* import/export: the initial requirement was to have a means to take the
current configuration of the load balancer ("ilbadm export <file>"),
transport it to another machine by whatever means and re-apply ("ilbadm
import <file>") it there.
Q1: Would people agree that this requirement makes sense?
Q1a: if yes, would you agree that the format we use here is of secondary
concern, ie. private?
Since we already have a parser in place for the CLI, it was suggested to
use that format for import/export (initially, anyway) to avoid having to
duplicate effort.
Q1b: does the above cause concern for anyone?
* persistent config: this was initially planned to be completely invisible
to users/admins, and exclusively maintained by ilbd. (As I understood it,)
this configuration file would be updated with every change the admin made
via ilbadm that was not explicitly designated temporary. Whenever ilbd
starts, it was supposed to clear all rules from the kernel (in case it, the
daemon, had died unexpectedly), re-read the persistent config and apply all
that to the kernel (ie all rules).
There is also discussion of a different usage model, namely, requiring
explicit admin action ("commit") to cause the running configuration to be
saved in the persistent config file. (There seems to be precedent for that
in the industry, but I couldn't find a reference right now)
There are tradeoffs for both models.
Q2: which of the above models (explicit commit vs implicit update) do you
think is more appropriate for us?
The argument that "we already have a parser for CLI" also seems attractive
here, so we're toying with the idea of using the same syntax we use for
import/export for persistent config as well; since SMF handles ilbd
restart, it would be easy to add an invocation of "ilbadm read-config" (or
sth. to indicate that persistent config is meant) to the service's start
method.
Q3: since persistent configuration is viewed as a private component of the
ilb framework, is it be acceptable from a privilege/authorisation POV to
expose any interaction with it via ilbadm?
If we went with "commit" model outlined above, a suitable subcommand could
be added to ilbadm, which would presumably perform the actual update to the
persistent config file (in line with the "we already have ..." argument).
This would have maybe even more severe implications than Q3 indicates, as
ilbadm would be *writing* persistent config.
Q4: would this cause security considerations?
TIA for your (timely ;-) thoughts.
as I announced here last week, I put back the first wad of code for the ilb project to an opensolaris.org - hosted repository. To repeat what I said in that email: this is not much more than a code drop, and things will break/go wrong/be missing. Nevertheless, if you're still interested, you're by all means welcome to give it a try and let us know how you fare. Please direct your comments to ilb-dev@opensolaris.org.
have fun!
Congratulations on the magnficent results of yesterday's elections! I'd like to present you with my wishlist for your first term:
I hope you can
If you manage that, you'll be a truly great president. For all that, my best wishes
Yours truly
I've been watching the preparations for tomorrow's elections with a mixture of fascination, horror, surprise and disbelief (to name but a few emotions) since the various candidates turned up and the tortuous process of primaries began ... I certainly could have paid more attention, but on the other hand, I could have paid a lot less.
Here now is one of the best articles I've read so far on this whole election drama - I can't judge whether this is that publication's usual style, but they certainly deserve kudos for this piece.
I just came across an interesting article that doesn't focus only on the candidates for the upcoming election, but also on the media's treatment of them ... and an interesting observation at the end.
Read it here.
Last weekend, I managed to make good on a promise I made to myself long ago (and got my wife to agree to ;-): I took part in a photography workshop. The instructor was Joe Decker (read more about him here). The whole group consisted of six people, including Joe, so we got good advice whenever we asked for it.
We started off in a parking lot just north of Golden Gate Bridge, and worked our way up Conzelman Road and all the way to Fort Barry. The weather was not quite unexpectedly misty, cold and generally miserable, which was ideal for shooting the battlements from WWII which now stand in disrepair. Not surprisingly, we weren't the only people around with photo gear.
After having seen and shot enough of ruins to last us quite some time, we proceeded to the beach next to Fort Cronkhite; the fog had increased in density, and although we were having a good time in general, it was getting a little damp, esp. close to the ocean with the wind blowing spray and mist right into our faces. Here's a picture which I believe captures the atmosphere of the day:
When and if I find time to properly process the pictures I took, I'll post them on one of my photo pages and drop a link here.
One of the requirements that need to be fulfilled by an offering in the load balancer space is he ability to periodically check the health of its' back end servers.
The health of a back end server can be defined in several ways:
in addition to the check, there must be an ability to report either the health status of a given server, or to report the change of status for a given server when this status changes (ie, when a server dies or comes back to life).
We examined a few open-source network monitoring tools (I think nagios was among those tools, as well as OpenNMS ... I wasn't too deeply involved in this part, so I don't know the details), but came to the conclusion that none was suited well enough for our purposes, so we
decided we'd need to build our own. We still need to finalise the design, but I think I can give a basic outline of what will be required for a health check subsystem within the ILB project, as well as some of the requirements on other parts of the ilb project to accomodate HC:
I just saw this - I'm glad to see this happen, and I'm looking forward to more student participation @Sun!
I consider blogs to be "work in progress", but this entry seems to be
even more so - and since it's also describing work in progress, somehow recursive :-)
One of the pieces still missing from (Open)Solaris is the capability to
forward IP incoming packets to a set of (more than one) hosts from within the
kernel, ie. to do load balancing.
The main benefit of an in-kernel load balancer vs. a userland-based one
is the much reduced traffic of networking data ("payload") through the
kernel/userland boundary. Traffic across this boundary is known to be expensive,
therefore the fact that we incur less of it means that - all other
things being equal - we can achieve better performance, both wrt
connections per second and wrt throughput.
To address this, we recently created a prototype with very basic load
balancing capabilities that we're hoping to put out on opensolaris.org
once all the formalities (read: legal stuff) have been completed. You
may have seen Sangeeta's email proposing this project for opensolaris: http://www.opensolaris.org/jive/thread.jspa?threadID=64639&tstart=0.
We're also going to be soliciting input from people who would like to
actively test this prototype.
We realise that a full product offering around a load balancer is
unlikely to be achievable within the time it would make sense for us to
do so, from the point of view of the addressable market, so we're going
to concentrate on providing the infrastructure necessary for developers
and OEMs to optimally exploit this capability we're introducing. (Plans
on *when* this is going to happen, and what exactly is going to be in
which delivery aren't quite finalised, so please bear with us ...)
Even before we release the code, I think I can present a short overview
of what the prototype consists of. We have:
- the in-kernel forwarding engine ("ilb" = internal load balancer, which
we also use as name for the whole project ...)
- the command-line utility ("lbadm").
Things like redundancy (ie. failover), backend server healthcheck etc.
were not implemented for the POC.
My task was and is to define the requirements for, and then design and
implement the CLI. While this sounds rather straightforward, the devil's
in the detail, as usual.
Here's some of the questions being asked of CLI as well as the
CLI/kernel module combo, as well as their answers:
some of the additions/modifications which will be implemented by this
project:
One time sometime in 1999 or 2000 (I think ...), when I was driving through Munich (or, more precisely, trying to do so) , I had to stop next to a construction site and a brick dropped down from several stories right onto the bonnet of my car and made a significant dent (and gave me a tremendous shock). I guess I got lucky that I'd had to stop where I was, otherwise it may have come in through the windshield and damaged things that were (and are) important to me. Again, luckily I had no passenger with me whom I would have had to console ...
I was lucky enough to find someone from the building company who actually took responsibility, so to get the insurance etc. sorted out was no problem, though it took it's time. Once I had the OK to have it fixed, I went about it, and here's a short piece I wrote about an experience I had while my car was being fixed ...
You can drive my baby car
I'd been planning to take my car to be repaired for a few weeks now - the dent in its bonnet was on its way to be something like Marilyn Monroe's beauty spot, something I just cannot have.
Naturally, we'd arranged for me to have a replacement during the time I had to leave my car at the garage, so when I'd taken care and leave of my meanwhile trusted friend, I was taken into custody by a representative of the rental car company, who with his three-piece suite was definitely much better dressed for the snow and the cold than I was with my coat, hat, gloves and once-fur-lined boots. Perhaps the amount of gel in his hair helped keep the cold out ... We took to the road (Frankfurter ring, full of cars but clear of snow) in a microscopic car and made our way to the rental company, where we'd do the paper work. Getting out of the car, I asked whether there were winter tires on the car, which produced a dubious look and a shake of the head. For safety, I repeated the request.
After we'd filled in the papers, my representative consulted a colleague about the tire-issue, and got a key from the locker. We then went out to the car park, where he unlocked a car the size of my hat that was painted a ghastly, poisonous lime green, about the colour of very unripe lemons. I almost threw up on the spot, which would have improved the colour no end. It did not get better after he'd cleaned the snow off. Nor where the winter tires much good.
(photo credit: Wikipedia.org - incidentally, this looks very much like the car I had)
Then I set off. As long as I was moving with the crowd, it wasn't so bad, apart from the moment when a fat BMW almost pushed me out of the way - the driver probably didn't even notice me, despite the colour. Then I reached the motorway. Stepping on the accelerator was an experience comparable to almost nothing, well, to nothing, actually, because that's exactly what happened for the first few seconds. Slowly the car would pick up speed, until at about 100 km/h, it would scream its little heart out trying to move faster.
I also noticed that almost the only thing that seems to be standardised apart from the steering wheel is the indicator switch (although this one needed extra strength - they probably think of big blond beefy guys with heavy-weight experience when designing these things); the wind-screen wiper switch works the wrong way (this one goes up for on, mine goes down), and they'd placed the rear-window heater switch next to the emergency signalling light switch, which is a very convenient thing for telling everybody that your rear window is misted up. The heating controls were at a level where I normally don't bend down to during driving, because I had to put my head on the passenger seat to reach it. Perhaps they intend you to get to know your female co-pilot better this way.
The switch to turn off the fresh air from outside thank-you-very-much only works when you're not defrosting the windscreen, which leaves you the choice, when following a big lorry, to either die of suffocation or crash into its rear because you can't see anything, but then, in this car, you'd probably go straight under the lorry and overtake it from below, so to speak.
The cute thing about this car is its fuel consumption. I got it with 7/8ths full, used another 8th, put in 8.85 litres and it was full. Marvellous. You could probably drive it on thimble-fulls.
I guess it's lucky this all happened in Winter. Imagine taking such a car to your favourite golf club, some ignorant dolt might put it down the next hole without a second thought.
----
(apologies to the Beatles for borrowing and mangling the famous song title ...)