[Trackback] Bryan Cantrill article "Whither USENIX?" I mentioned here last week got a lot of feedback....

Posted by arved's weblog on July 13, 2004 at 06:55 AM PDT #

I think you're falling into the same trap that you're saying academics do, the distinction being that you describe academic research as "irrelevant" while academics describe much industrial work as "uninteresting." I'd have to agree with anonymous (sorry, Adam); nohup is not really significant research, from my perspective. It's just a tiny example, something which I read about and said "Hunh, a neat trick." However, I found the agent LWP itself to be *very* interesting; after reading about it, I sat for about ten minutes thinking, "Wow, cool, you could really do some crazy stuff with it..." With a small amount of OS kernel support (e.g., it can only run when all other LWPs are halted), you have a mechanism to deal with a lot of problems and add a lot of functionality.

Posted by Philip Levis on July 13, 2004 at 11:15 AM PDT #

I think this is largely a philosophical question. The old-school definition of academically "significant" systems research is that it will allow *someone else* to build a better system that real people will use 5-10 years after the academic research is done. So it's work that's at the very earliest stages of the technology transfer cycle. That's why the premiere academic operating systems conference is called the Symposium on Operating Systems Principles -- the work is supposed to represent research on general principles of how to build systems, that applies across a wide range of actual operating systems or hardware platforms or whatever. Often a prototype of the idea will be produced, but it's more to prove the usefulness/practicality/applicability/benefit of the principles and--dare I say it--interfaces that are the real "meat" of the research. Because the artifacts are small, they can be produced by one or two graduate students even though they are designed to showcase a "big idea" with broad applicability. On the other hand, because the research is supposed to be general and 5-10 years out, it can appear irrelevant at the time it is done. [[ PRAGRAPH BREAK SHOULD GO HERE, BUT I CAN'T FIGURE OUT THE SYNTAX!! ]] Now, that's the theory. The reality is that a lot of (most?) systems research done in academia today is the same kind of work that could be (and often is) going on in industrial research labs and product groups. There are lots of forums for publishing this kind of work, and that is certainly the kind of work that could benefit from open-source Solaris, data (on workloads, failures, etc.) from companies, etc. I think the problem is that a lot of academic researchers today are doing very practically-oriented research without any real concern for its impact on real existing systems. And certainly I agree with you that if you are going to be doing reserach that you claim helps existing systems, you need to be using realistic existing systems and be able to make a good argument as to how the proposed ideas could be integrated into existing systems. I think the problem arises when academic researchers try to do short-term oriented research yet argue that the work doesn't have to solve a real problem using state-of-the-art systems, simply because they're academics. [[ PARAGRAPH BREAK SHOULD GO HERE, BUT I CAN'T FIGURE OUT THE SYNTAX! ]] Nonetheless, when trying to understand the thought process that academic researchers use when they pick problems, choose solutions to those problems, implement prototypes of the solutions, and write up their results, it's important to keep in mind that the kind of research I described in the first paragraph *is* considered the holy grail/gold standard. The closer your research, as an academic, is to that standard, the more highly valued it will be by the academic research community. Whether this is a good or bad thing, whether those ideas are ever applied to real systems, whether this *should* be the goal of academic research, etc., is yet another interesting topic in and of itself--and, unfortunately, one that many academics seem to consider to be too "practical" to be "interesting"!

Posted by David Oppenheimer on July 14, 2004 at 09:20 AM PDT #

I'm not sure I agree with the 5-10 years assertion, and would be really hard pressed to name any research that emerged as important in that time window. However, in a 1-3 year window, there are a lot of examples: Disco (VMWare) is one that immediately comes to mind. In the end, a lot of systems research is a craps shoot on what people think may be problems coming over the horizon, many of which turn out to not be that big a deal. Usually, the problems that academics look at are not identical to those industry faces, but there are enough similarities that ideas or basic approaches can be borrowed; divining exactly what future problems will be is hard. That being said, and I get the feeling this is a major component of Bryan's concern, academia has really moved away from hardcore kernel research: it's become systems research, rather than operating systems research. There's still always a few papers at each conference (e.g., Nooks at the recent SOSP, superpages at OSDI 2002, anticipatory scheduling at SOSP 2001) , but they're greatly outnumbered by papers about service composition, availability for distributed services, etc.

Posted by Philip Levis on July 14, 2004 at 02:23 PM PDT #

I can think of a few that were around the 5 year mark from the inception of the research project to the product becoming widespread (I admit that it is very difficult to think of successful technologies that took 10 years to make the transition). The first one that came to mind was the one that Phil mentioned, namely VMWare's roots in Disco, which was part of the FLASH project at Stanford. Another is the architecture and compiler research that went into the Itanium, particularly the VLIW research from the 1980s. For that matter I think a fair amount of the academic compiler research slowly made its way into compilers by SGI and others. As for the SMT work, which you cite and which is another good example, I never got the impression that academics felt it was too impractical, and certainly the latency from the work at UW and elsewhere to Intel's hyperthreading and companies like Tera was in this general timeframe. As for today, two of the hot research areas in academia today are sensor networks and distributed hash tables. Neither are widespread system building blocks today, but sensor networks seem likely to be widespread by 2005-2010, which is 5-10 years after the sensor research project began here at Berkeley, IIRC. (And if you talk to the DHT folks, they will tell you that their work will transform the Internet from a network that does address-based routing to one that does content-based routing, but that's definitely not going to happen within 5 years of ~2001.) Presumably the reserach going on in these areas in academia today is developing the principles and architectures that will be needed if/when these systems become deployed widely. Certainly there is lots of research going on in academia today in other areas as well--I was just expressing my opinion as to why much academic research may seem "impractical" and why many academics might not be looking at projects that are more immediately "practical." This is definitely not a comment on the value of one kind of project over another, and many projects with shorter-term horizons have had more impact than those with longer-term horizons. Indeed, as you correctly point out, RISC and RAID did not take 5 years to become widespread (OTOH I think there were some market forces that led those technologies to be exceptions (in particular IIRC RAID was concurrently under development at IBM and Berkeley, and Hennessy left Stanford for several years to start MIPS specifically to commercialize RISC)). [[ PARAGRAPH BREAK ]] To tie in to Phil's comment, I agree that there is not a lot of core OS research going on anymore, but I think that's due to a perception that OSes for the masses can't be substantially "improved" beyond their current state through more academic research (though I think the open sourcing of Solaris may help to change that, and is definitely a great thing). Areas like service composition and wide-area distributed services are still in their formative stages, and hence, I think, are perceived (perhaps incorrectly) as more subject to influence by academics than is core OS, which is viewed (again, perhaps incorrectly) both as an "old" field with no "low-hanging" fruit left, and as being controlled by the folks up in Redmond. OTOH, if you can come up with a great idea to improve operating systems, then your impact is likely to be greater, at least over the short term, than is research in distributed services or even sensor networks. Just my two cents on all of this; if I were truly perceptive and wise, I'd have my Ph.D. by now. :-)

Posted by David Oppenheimer on July 15, 2004 at 09:40 AM PDT #

I think that each field that evolves as much as ours has a number of early pioneers that feel that the art is gone. Where the art is associated with the simplicity of the early days. I have to admit that I also sometimes dream of the excitement I experienced when getting the pdp-11 to jump through hoops. I do think the comments to Bryan posting show exactly what Rob Pike means: if our only measure is how quickly research results get adopted, you get away from true innovation because you start doing market-oriented research, which can be valid research but always tends to be less spectacular than the stuff that is totally funky far-out principled work. The kind of work that when you read about it oozes something like 'these guys really believe in this and they really pushed the limits'. Remember that Rob's measurement for value is not industry adoption (otherwise he should consider his own work somewhat irrelevant). I don't completely agree with this conclusion, but I can see why there is less excitement than in the pioneering days. It will be fun to see what he is going to produce at Google. I have a less pessimistic view. I actually think a lot of good research is happening; it is just that SOSP is no longer the only measurement to use for validating results (easy for me to say, I already scored there :-)). My measurement is whether there is a paper in a conference where I think, wow I wish I had done that... I still have that at times, so some good stuff must be happening. Now whether I felt that about the DTrace paper is a whole different matter�

Posted by Werner Vogels on July 15, 2004 at 06:34 PM PDT #

An example comes to mind of a concept that was developed more than 10 years before its practical implementation in our science: hardware-accelerated radiosity. "Global" illumination had been in application for satellite heat sink design for about 20 years when ideas about how to really make it a broadly-applicable technology started coming out. It took another 10-15 years before these ideas could be tested. The concepts that go from idea to implementation in a few years come out with a bang and really grab our attention. We're hearing fewer and fewer bangs in the last few years and they're taking more and more effort to produce as time goes by.

As a science, we have enjoyed an explosive beginning. We've been spoiled. Small ideas were grandly innovative and we've spent our time being easily occupied - but I think this is changing. If you compare our world to one of the other hard sciences, we see a picture of where we may be going: how long does it take for new discoveries in physics and math to become of economic interest? How much dedication to pure research does it take to get to the point where you're contributing at that level? How long did it take for Maxwell's, Einstein's, Watson/Crick's work to go from the realm of research to the engineering of products?

If this becomes the case for software, then industry will let academics get on with its pie in the sky projects while industry continues with its "nohup -p"s. Some of what academia produces will be of use to engineers and some of it won't. As the science becomes more complex, the distinction between research and engineering becomes very important. Brian makes the point the point that nohup -p would not have been a likely product of academics, but neither would the proof of Fermat's Last Theorem be of interest to industry. Both sides are a necessary part of the whole.

Posted by Ron Barry on August 03, 2004 at 10:26 AM PDT #