Joseph D. Darcy's Sun Weblog

Friday March 02, 2007

Peak Code

Hubbert's theory on peak oil (and other geological resources) states that the maximum rate of production of the resource occurs when half of the reserve has been extracted. Initially, there is near exponential growth in the rate of extraction as lots of "easy oil" is found and pumped, but after the peak the rate of extraction decreases. The marginal cost of extraction goes up as less desirable source of oil are put into production; this cost should provide economic incentives to consider alternate energy sources.

I think the evolution of programming environments has some similarities to Hubbert's theory. A successful new programming environment enjoys an early inflationary phase where lots and lots of new code is being written in it. As the platform matures, the significant programs that are "easy" to write in that language get written, often code less practical to write elsewhere. As the platform grows, increasing interactions and compatibility constraints make adding new features more challenging. At some point, half of the total lines of code for that platform will be written. The further a platform gets past its peak, the less attractive adding new features to it becomes since there is less and less potential return on investment due to the dwindling number of programs that could benefit from the improvements.

However, there are some important differences between potential programs that can be written and natural resources. First, old programs are not consumed or made unavailable in the same way oil is when refined to fuel, made into plastic, and so on. Second, the constraints on the production of new code are very different from those in isolating a chosen natural resource; the primary limiting reagent to producing new programs is not a physical limit. In chemistry, a reaction takes places between fixed ratios of molecules so if you don't have enough of one kind of reagent molecule, say oxygen, the unavailability of that reagent limits the reaction no matter how much of the other reagents, like gasoline, are available. However, the amount of code that gets written will be capped by human interest or economics before hard physical resource limits come into play. Hubbert's peak applies to finite resources; in contrast, the potential amount of software is effectively infinite for the purposes of this kind of modeling. Additionally, there are difficulties in accurately measuring the rate of program production. These differences mean that it can be impractical to know the precise rate of new program development and even if that rate were known, the life cycle of the platform could follow a very different curve then Hubbert's. Therefore, if there is a peak, is is hard to know whether or not it has occurred and the timing and size of any peak can be influenced by changes in the platform.

As volume drives value, evolving a general-purpose programming platform should try to enable a wide range of applications to be written in a convenient and safe manner, expanding over time. Java's relative simplicity, garbage collection, and rich libraries were early advantages; more recently the language changes of generics and annotations have expanded the set of programs that are tractable to write on the platform. As the Java approaches its teenage years, we'll aim to keep improving the platform to support more kinds of programs in JDK 7 and beyond; perhaps closures will be the next major enhancement, allowing programmers to abstract over another axis.

Thanks to Alex for the conversation which led to this post and for providing feedback on earlier drafts.