The Observation Deck
 

The Observation Deck

Views on software from Bryan Cantrill's deck chair

« DTrace on ONTAP? | Main | DTrace on QNX! »
20071029 Monday October 29, 2007

 DTrace, Leopard, and the business of open source

If you haven't seen it, DTrace is now shipping in Mac OS X Leopard. This is very exciting for us here at Sun, but one could be forgiven for asking an obvious question: why? How is having our technology in Leopard (which, if Ars Technica is to be believed, is "perhaps the most significant change in the Leopard kernel") helping Sun? More bluntly, haven't we in Sun handed Apple a piece of technology that we could have sold them instead? The answer to these questions -- which are very similar in spirit to the questions that were asked over and over again internally as we prepared to open source Solaris -- is that they are simply the wrong questions.

The thrust of the business questions around open source should not be "how does this directly help me?" or "can't I sell them something instead?" but rather "how much does it cost me?" and "does it hurt me?" Why must one shift the bias of the questions? Because open source often helps in much more profound (and unanticipated) ways than just this quarter's numbers; one must look at open source as long term strategy rather than short term tactics. And as for the intense (and natural) desire to sell a technology instead of giving away the source code, one has to understand that the choice is not between "I give a customer my technology" and "I sell a customer my technology", but rather between "a customer that I never would have had uses my technology" and "a customer that I never would have had uses someone else's technology." When one thinks of open source in this way, the business case becomes much clearer -- but this still may be a bit abstract, so let's apply these questions to the specific case of DTrace in Leopard...

The first question is "how much did it cost Sun to get DTrace on Leopard?" The answer to this first question is that it cost Sun just about nothing. And not metaphorical nothing -- I'm talking actual, literal nothing: Adam, Mike and I had essentially one meeting with the Apple folks, answering some questions that we would have answered for anyone anyway. But answering questions doesn't ship product; how could the presence of our software in another product cost us nothing? This is possible because of that most beautiful property of software: it has no variable cost; the only meaningful costs associated with software are fixed costs, and those costs were all borne by Apple. Indeed, it has cost Sun more money in terms of my time to blog how this didn't cost anything to Sun than it did in fact cost Sun in the first place...

With that question answered, the second question is "does the presence of DTrace on Leopard hurt Sun?" The answer is that it's very hard to come up with a situation whereby this hurts Sun: one would have to invent a fictious customer who is happily buying Sun servers and support -- but only because they can't get DTrace on their beloved Mac OS X. In fact, this mythical customer apparently hates Sun (but paradoxically loves DTrace?) so much that they're willing to throw out all of their Sun and Solaris investment over a single technology -- and one that is present in both systems no less. Even leaving aside that Solaris and Mac OS X are not direct competitors, this just doesn't add up -- or at least, it adds up to such a strange, irrational customer that you'll find them in the ones and twos, not the thousands or millions.

But haven't we lost some competitive advantage to Apple? Doesn't that hurt Sun? The answer, again, is no. If you love DTrace (and again, that must be presupposed in the question -- if DTrace means nothing to you, then its presence in Mac OS X also means nothing to you), then you are that much more likely to look at (and embrace) other perhaps less ballyhooed Solaris technologies like SMF, FMA, Zones, least-privilege, etc. That is, the kind of technologist who appreciates DTrace is also likely to appreciate the competitive advantages of Solaris that run far, far beyond merely DTrace -- and that appreciation is not likely to be changed by the presence of DTrace in another system.

Okay, so this doesn't cost Sun anything, and it doesn't hurt Sun. Once one accepts that, one is open to a much more interesting and meaningful question: namely, does this help Sun? Does it help Sun to have our technology -- especially a revolutionary one -- present in other systems? The answer is "you bet!" There are of course some general, abstract ways that it helps -- it grows our DTrace community, it creates larger markets for our partners and ISVs that wish to offer DTrace-based solutions and services, etc. But there are also more specific, concrete ways: for example, how could it not help Solaris to have Ruby developers (the vast majority of whom develop on Mac OS X) become accustomed to using DTrace to debug their Rails app? Today, Rails apps are generally developed on Mac OS X and deployed on Linux -- but one can make a very, very plausible argument that getting Rails developers hooked on DTrace on the development side could well the change the dynamics on the deployment side. (After all, DTrace + Leopard + Ruby-on-Rails is crazy delicious!) This all serves as an object lesson of how unanticipatable the benefits of open source can be: despite extensive war-gaming, no one at Sun anticipated that open sourcing DTrace would allow it to be used to Sun's advantage on a hot web development platform running on a hip development system, neither of which originated at Sun.

And the DTrace/Leopard/Ruby triumvirate points to a more profound change: the presence of DTrace in other systems assures that it transcends a company or its products -- that it moves beyond a mere a feature, and becomes a technological advance. As such, you can be sure that systems that lack DTrace will become increasingly unacceptable over time. DTrace's shift from product to technological advance -- just like the shifts in NFS or Java before it -- is decidedly and indisputably in Sun's interest, and indeed it embodies the value proposition of the open systems vision that started our shop in the first place. So here's to DTrace on Leopard, long may it reign!

(2007-10-30 08:08:41.0/2007-10-29 11:25:23.0) Permalink Comments [12]

Comments:

I'm also happy to see DTrace on OS X for the reasons you mention. Apple has shown that it has the guts to buck the "not invented here" trend to adopting new technology with it's decision to use BSD a few years back. Today, their integration of DTrace and eventual integration (read-only for now) of ZFS shows they haven't wavered in that stance - bravo!

That said, I think Solaris is making massive progress on the desktop, to the extent that I'm not sure my 2002 iMac will be getting replaced with another Mac, but Apple, feel free to convince me otherwise.

Oh and speaking of the cost of software, there's a weird pricing model with Leopard here in Europe, hint $129 != 129EUR.

Posted by Tim Foster on October 29, 2007 at 01:40 PM PDT #

I think DTrace is so cool, it's a profound contribution to the world, a tool made by engineers for engineers. It kind of reminds me of when I first discovered oscilloscopes and, later, spectrum analyzers.

However I've always found the arguments for open source to be convoluted and in direct contravention of Occam's Razor, and yours is unfortunately no exception. The elephant in the room, of course, is the popularity of the Windows and Linux operating systems. Within that context it makes perfect sense that Sun as a systems company would open source its software. Loss lead the software and make money on the hardware and support services.

"As such, you can be sure that systems that lack DTrace will become increasingly unacceptable over time."

And what could be better than to pick up OpenSolaris with DTrace for free and run it on an x86 whitebox? Of course, if the customer happens to care about heat and power consumption per CPU cycle, as your CEO Jonathan is quite fond of talking about, then perhaps a Nigara 2 based system would be a better fit. And once you give a CIO some fast, cool, servers he's going to want some storage, perhaps a Thumper? And if you give a CIO some fast servers and lots of storage he'll want to optimize his investment so he'll fire up DTrace. And if you give a CIO DTrace he'll want some help interpreting the numbers, and where better to go than the inventor of all those technologies?

Posted by Andrew on October 29, 2007 at 05:06 PM PDT #

Hey Andrew,

I strongly agree with your first, third and fourth paragraphs. ;) And actually, I don't think that your second paragraph invalidates your other reasoning. It all comes down to the fact that we have confidence in our ability to deliver and support innovative systems, not just innovative bodies of code (though that too), so the presence of Windows and Linux doesn't really change our perception of the value of open source...

Posted by Bryan Cantrill on October 29, 2007 at 05:25 PM PDT #

The DTrace on linux concept needs a rest for the moment, but the concept of DTrace on MS-Windows.... *that* would prove interesting. If Microsoft ever ported DTrace - and released that port - then everybody whose OS of choice competes with them would have to really make a quantum leap in order to overtake MS.

Posted by James C. McPherson on October 30, 2007 at 07:09 PM PDT #

"...and becomes a technological advance. As such, you can be sure that systems that lack DTrace will become increasingly unacceptable over time"

Bryan,

Isn't that argument based on the assumption that DTrace will NOT be ported to Linux, at least in the near future, because of license incompatibility? What, in your opinion, would change if DTrace is ported to Linux or Windows in about one year? I know it is a long shot, but I wonder what your point of view about this hypothetical scenario is. By the way, how long did it take since your meeting with Apple engineers and OS X DTrace implementation?

Thanks,

Antonello

Posted by Antonello Cruz on October 31, 2007 at 05:46 AM PDT #

Antonello: I don't think BMC's assumption is that DTrace won't be ported to linux. If "the linux community" wants to do a serious DTrace port then I'm sure that Team DTrace would be very glad to help. I know that members of the DTrace community would like to see DTrace on linux. If DTrace was ported to linux that would be a good thing. Same goes for MS-Windows.

Posted by James C. McPherson on October 31, 2007 at 05:58 AM PDT #

Antonello,

Just what James said: I'm not assuming that it wouldn't be ported to Linux (as we have said quite a few times, we believe that such a port is both technically and legally possible, and that the license issue is with the GPL, not the CDDL), quite the contrary in fact: my point was that the presence of DTrace in a variety of systems allows users to start expecting it everywhere, and thus in the long term Linux will HAVE to port DTrace if they wish to remain competitive.

As for how long the port took, you would need to ask Apple: by the time we had our meeting with them, it was pretty clear that the port was well underway, and that they were just trying to clear up some niggling issues. Certainly, it wasn't long after that meeting that they announced to us that they had DTrace working and that they intended to ship it in Leopard...

Posted by Bryan Cantrill on October 31, 2007 at 07:09 AM PDT #

Having DTrace on MacOS X Leopard looks good indeed, for Solaris, MacOS X and developers using at least one of those, for the reasons stated here :)

However...
> and that the license issue is with the GPL, not the CDDL
Hmm. The CDDL came after the GPL(v2 at the time), and it's incompatible with it. Fact, as far as I've read on multiple places.
This does not necessarily mean that the CDDL was explicitely designed in order to be incompatible with the GPL(v2) (although many people think that it was, and I rather agree with that), but this at least means that the CDDL was not designed in order not to be incompatible with the GPL(v2).

And who benefits of the licensing trouble between GPL, CDDL (and BSD to a lesser extent), which delays/prevents progress in the open, for everyone's benefit ? Microsoft (R)(TM) Windows (R)(TM) and the ecosystem surrounding it, period.
Lowering the licensing barriers would be A Good Thing, and all parties involved (Sun, Linux kernel + Systemtap people) could do more for that to happen.

On the technical side, now:
I guess you developers of DTrace keep on adding features to DTrace ?
Judging from http://sources.redhat.com/systemtap/wiki/SystemtapDtraceComparison
Systemtap is supposed to have (and has partial or complete programming for) some features that DTrace doesn't have, especially in the scripting language itself. Some of these features don't look impossible to add in DTrace, don't they ?
(since Adam Leventhal edited the comparison, the information on the page should be correct enough)

Posted by Lionel Debroux on November 01, 2007 at 02:36 PM PDT #

Lionel,

The timing of the GPL vs. the CDDL is not terribly relevant -- this is software licensing, not calling shotgun. As for why we developed CDDL instead of GPL or BSD, the reasons are philosophical: unlike BSD, we believe in copylefting; unlike GPL, we believe in allowing proprietary derived works -- but we wanted to allow them in a way that didn't create a patent mess. Personally, I happen to think the CDDL is a very accurate reflection of the values of Solaris -- and I think many of the supposed license incompatibilities are invented as a cover for NIH.

As for SystemTap, the differences here, too, are philosophical: the lack of "full" control structures in DTrace, for example, actually represents our value of safety above all else. (Safety is considered a goal -- not a constraint -- of SystemTap.) As for their criterion "aggregate value readable by script", I can tell you that SystemTap's answer doesn't square with their assertion that they have scalable aggregations: in the DTrace architecture, anyway, it is the infinite scalability of aggregations (both in terms of concurrency and time) that makes it impossible to act on aggregation values in probe context.

Finally, while Adam was ultimately permitted to contribute to the wiki to correct its most glaring errors, the wiki remains very much a SystemTap construction; it is designed to make SystemTap look much more competitive with DTrace than it in fact is. More balanced comparisons can be found by googling "dtrace systemtap"...

Posted by Bryan Cantrill on November 01, 2007 at 03:05 PM PDT #

I haven't used DTrace yet but I understand the fundamental advantage it will give to developers coding on Leopard. I would just like to thank you (and the other DTrace developers) for creating DTrace in the first place. You make it sound like you didn't have a whole lot to do with the Apple port, but the fact that the software exists in the first place is a tribute to the efforts of the hard-working programmers at Sun. And without the original code, there could be no port.

Posted by David Portela on November 02, 2007 at 02:58 AM PDT #

I first saw DTrace used at Usenix by this guy named "Jim Mauro"; my first thought "this is sooo COOL!". With DTrace now in Leopard I can more easily spend time learning how to best use this awesome tool to make my work life more productive, Thanks SUN!

Posted by Rich Allen on November 02, 2007 at 05:06 PM PDT #

Brian,

OK, I understand your points :)

Posted by Lionel Debroux on November 03, 2007 at 01:53 PM PDT #

Post a Comment:

Comments are closed for this entry.

« May 2008
SunMonTueWedThuFriSat
    
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
       
Today


XML




Planet Solaris