Jyri Virkki

Endless Nights

How long is your night?

Or, how long does your nightly build take?

It's been about six months since my first article on Unconsolidating (as opposed to 'Consolidations', the very unique practice of the OpenSolaris organization of hosting the source code of all known applications in a single source tree instead of the more commonly accepted engineering practice of having individual applications in their own source repositories) in which I looked at the then-available packages in SFW and how long it took to build it (see the article for more).

Seems like a good time to do a refresh of the numbers and a sanity check on the predictions. Back in December I just assumed a linear increase in build time as package counts went up. Clearly there is a huge variation on build times of individual packages but I figured it would largely average out over a large number of packages.

Today (actually, last week when I looked), SFW is producing 205 packages and it took 3 hours 37 minutes to build (on the same dual Opteron w/2GB RAM which I used in December). That's not exactly on the line predicted in December, but it's actually closer than I thought it would be, so the linear approximation is not far off at all, so far.

So, once we have 20,000 packages we can expect the nightly build to take over two weeks. I guess we'll need to change the SFW release cycle since currently it pushes bits out every two weeks!

Even if we only get to 5000 packages in the foreseeable future, you'll only get to do one build per work week, so better make sure nobody broke anything ;-(

At least back in the days of batch processing and punchcards one could expect the results back the next day!

(Of course, from what I'm seeing, most would-be contributor to OpenSolaris are running within VirtualBox, so your builds will take much longer than this, I'm afraid.. I've been meaning to do a current SFW build within VirtualBox on my laptop, but I know it'll be painful so haven't found the time yet...)

Future problems aside, this is certainly a problem already. As the build time closes in on 4 hours, we can already only get a single build in during a work day, which tends to make even the simplest changes take multiple days instead of a few hours as they would in more normal circumstances.

Here's a graph showing current and predicted build times. The two lines are the predictions using the numbers from December'07 and from now, June'08. As you can see, they're identical for all practical purposes.

If we still haven't managed to solve this problem by December'08, I'll revisit the numbers and graph once again at the end of this year...

Posted at 02:25PM Jun 11, 2008 by jyri in WebStack  |  Comments[3]