John Rose @ Sun: John Rose's weblog at Sun Microsystems

Last week some Sun engineers and I invited our colleagues over to Sun Santa Clara to talk about the future of VMs and languages. We called the meeting the JVM Language Summit. It was a blast; I’d like to tell you why, and what I learned.

Pizza with extra MOP

Here are my top-level takeaways:

• The invokedynamic design is sound, but the exposition needs more work.
• The synergy of JSR 292 with Attila Szegedi’s MOP looks very promising.
• Interface injection is going to be helpful to a lot of people, and it is not hard to implement (on top of method handles).
• Tailcall and value types will never go away. We have to plan for them.
• Unless we do this sort of innovation on the JVM, crucial multicore research will move elsewhere.
• We have to do this again next year.

[Read More]

I have been working furiously this summer, patching the OpenJDK HotSpot JVM for the JSR 292 implementation of dynamic invocation.

In the wee hours of this morning, the JVM has for the first time processed a full bootstrap cycle for invokedynamic instructions, linking the constant pool entries, creating the reified call site object, finding and calling the per-class bootstrap method, linking the reified call site to a method handle, and then calling the linked call site 999 more times through the method handle, at full speed. The method names mentioned by the caller and the callee were different, though the signatures were the same. The linkage was done by random, hand-written Java code inside the bootstrap methdod.

The Email thread of the announcement is truly international, since Guillaume Laforge celebrated by sending virtual champagne.

The example code is included in the Email, and also posted (as a truly rebarbative test in a NetBeans project) with the patches. As for the JVM code, it only works on x86/32; the next step is to move the assembler code into the right files, and finish the support for x86/64 and SPARC.

Happy International Invokedynamic Day!

(And by a curious anagrammatic permutation of letters, it could also be International Davinci-Monkey Day. My co-workers, who watched me pounding on my keyboard all summer, claim to see some significance in this.)

Invited by some friends at Google, I went to Google I/O this week to find out about Android, and specifically their Java story. I went to a few talks and had some excellent chats with various colleagues.

Here are the top ten things I learned about Android and the Dalvik VM...[Read More]

After a successful meeting at JavaOne, the JSR 292 EG (expert group) has published its EDR (early draft review) for the invokedynamic instruction. This draft will be updated from time to time (in response to your comments), until August 17, which is the end of the 90-day review period. Check it out!

Update: There are some good conversations going on about the EDR at the jvm-languages Google group. Here are some rough changes to the EDR I will be proposing in response; the EG may choose to make these changes (or something like them) official with an EDR update pushed through the JCP (Java Community Process).

In the Java cosmos we can reckon time in terms of JavaOne conferences. For programming languages on the JVM, the just-finished epoch has seen much progress, and the next epoch looks even better. Here is some of the progress that I am excited about, after bouncing around at JavaOne...

In other news, today (May 10th) is National Train Day. On this day 139 years ago, a ceremonial golden spike was driven at Promontory Summit, Utah, joining the Central Pacific and Union Pacific railways into a single transcontinental line. The continent was very suddenly smaller, because people, goods, and mail could be moved more quickly from coast to coast, a task which was previously done with wagons, horses, and boats. In the years leading up to this watershed event, two railroads were built, with great difficulty and ingenuity, from each coast, to meet at Promontory Summit.

There is a Promontory Summit and a golden spike in our future also... [Read More]

For several years now, JSR 292 has promised an invokedynamic instruction in one form or another. The problem has been with picking the one form that simultaneously enables a good range of use cases, addresses several architectural challenges in the JVM, and can be optimized by a variety of commercial JVMs. It has been a restless search for “one bytecode to rule them all”.

The EG has decided to propose an answer, providing a hook which refers all important decisions at a dynamic call site out of the JVM and into Java code. This note builds on a previous blog entry, giving more concrete details and use cases. The current design makes heavy use of method handles, which greatly reduces complexity and clarifies the various roles of language implementors and the JVM. [Read More]

Hello, JSR 292 observers and language implementors!

The JSR 292 Expert Group met today at JavaOne.

There were representatives from three major JVMs and two dynamic languages (Groovy, Jython).

Here are some of my notes from that meeting. I hope you find them interesting...

[Read More]

“Self-modifying code...” used to be a phrase always uttered (by us hackers) with tones of both admiration and dread. Operating systems and VMs are required to support it (always, in the loader). Aspect oriented programming has made a cottage industry of it. I still fear it, and when I hear customers ask for an API to edit classes in the JVM, I always reach for an alternative, a prescription subtitute for the illegal substance. Inteface injection is a good substitute for a surprisingly wide range of use cases; perhaps it can handle your use case for self-modifying code also.

Interface injection (in the JVM) is the ability to modify old classes just enough for them to implement new interfaces which they have not encountered before. Here are the key design points, in brief... [Read More]

Or, how to finish a job twice.

Or, anything worth starting is anything worth starting is anything worth starting is anything worth starting is ...

A continuation, simply put, is a reference to the rest of some program P, as of some given point in the midst of P. It is an interesting design problem to introduce continuations into the JVM. I don't know of a full design for JVM continuations, yet, but it's possible to observe both the easy and the hard parts, and to survey some of the reasons we should care. [Read More]

The JVM prefers to interconnect methods via static reference or dispatch through a class or interface. The Core Reflection API lets programmers work with methods outside these constraints, but only through a simulation layer that imposes extra complexity and execution overhead. This note gives the essential outlines of a design for method handles, a way to name and interconnect methods without regard to method type or placement, and with full type safety and native execution speed. We will do this in three and a half swift movements...[Read More]

We just had a good conversation in the JVM Languages group about how memory is getting weird these days, and how microbenchmarks have always been weird. Please include in this weblog, by reference, my musings on memory and JVMs.

Kudos to Vladimir Sizikov for a nice collection of follow-up links. I was very glad for the introduction to Ulrich Drepper’s paper. He also refers to a new wiki on Hotspot implementation, inspired by and serving the OpenJDK, in which I and others are collecting information to help language and JVM implementors understand how Hotspot works, how to make it better, and how to byte-compile for performance.

Finally, if like me you’ve heard but never quite believed the story about how functional languages fix all the problems with side effects and you laugh at the funny papers, you’ll like this little fable about the perils of mutability, in programming languages. It provides an arresting, hard-hitting, and ultimately crushing new perspective on Hindley-Milner type checkers.

Hammering my way through a posting on the JVM and continuations, I realized again how oddly poetic are some of our terms of art. Some of them seem to have been with us from the dawn of the single-threaded stored-program computing machine. Terms like “continuation&rdquo, “closure&rdquo, “thunk&rdquo, even “call&rdquo and “loop&rdquo are metaphoric, evocative, polyvalent, elusive of final definition. What I mean is, they are poetic...[Read More]

Or, the headless object rides again.

Language-specific VMs often use pseudo-pointers with an immediate “payload” to represent a commonly used subset of numbers, and also “headerless” representations for small but ever-present types like Lisp’s cons cell. The JVM can support fixnums and other headerless objects for the sake of these languages, and even for Java. The idea is to make ordinary object pointers (sometimes called oops) coexist with more specialized formats for headerless objects, which we will call iops and xops. The techniques are mature and well-known, and the overheads (of extra tag checking before pointer usage) can be controlled by optimizations already used widely in JVMs.

We will examine low-level encoding techniques, and investigated how to integrate these concepts into the JVM. Although a language implementor might prefer to try these techniques in the closed environment of a from-scratch, specialized VM, it is probably more profitable in the end to incorporate them into an optimized, mature JIT and GC, such as the Open JDK project. [Read More]

Some of my colleagues have noticed the news flurry about home schooling and the sudden declaration of its illegality by a panel of federal judges in Los Angeles. The formal decision features a spicy stew of judicial threats to parents, in a section entitled “Consequences of Parental Denial of a Legal Education”.

That certainly got my attention and that of many friends, since (dare I now admit?) I've been home schooling my children since 1987. Two have finished with honors at good universities and are now productive taxpayers, two more are now making their way through college, and the rest are ahead of grade level and nicely socialized, thank you. Who knew my wife and I were guilty of Parental Denial of a Legal Education? (Gotta get some of that Legal Education. It must make you as wise as a Judge.) To those of us in the home schooling community, the general consensus is more adequately phrased in a San Francisco Chronicle Op-Ed: “What planet are those judges coming from?” I realize the education of one’s children is a culturally subversive thing to do, but since when is California suddenly shy of cultural deviancy?

One can only wince in wonder at the ideal California those judges are contemplating. The state has an interest in many children’s rights beyond mere education, such as nutrition. Perhaps we should require parents to be certified dieticians before they cook their children’s lunch. Or, let’s just go all the way and eliminate the inconvenient families, by requiring a parental license before the first child is brought to term. That would bring everything nicely under control, and our Wise Judges could rule a utopian, aristocratic Plato’s Republic—which is really a nice place to study, but a terrible home.

In my own home town of San Jose, I just noticed a reasonable Mercury News editorial on the subject. Common sense still rules in San Jose!

I make one key exception to the Merc.’s editorial position: All else being equal, I as a private citizen greatly prefer benign neglect to any form of regulation. But unlike us private citizens, editorial writers and politicians seem to have a professional rule: Never make ringing calls to do nothing. (And the corollary: Never be without a ringing call.) I am thankful that, somehow despite all the political fidgeting, life goes on anyway.

Also, I’m proud to say that the two debaters the Mercury mentions are from our group’s debate club. I think it is not too much to hope that, in their day as judges or other community leaders, they will write better opinions.

In the end, my advice to judges, and even to friendly editorialists and politicians, is: Leave parenting to us parents. It worked when all of us were growing up, and it works now.

August 2008 Update: The court has reversed its decision. Here is Governor Schwarzenegger's take on it:

This is a victory for California's students, parents and education community. This decision confirms the right every California child has to a quality education and the right parents have to decide what is best for their children," he said. "I hope the ruling settles this matter for parents and home-schooled children once and for all in California, but assure them that we, as elected officials, will continue to defend parents' rights.

And Superintendant Jack O'Connell says,

As head of California's public school system, it would be my wish that all children attend public school, but I understand that a traditional public school environment may not be the right setting for each and every child... I recognize and understand the consternation that the earlier court ruling caused for many parents and associations involved in home schooling. It is my hope that today's ruling will allay many of those fears and resolve much of the confusion.

(Source: LA Times.)

This week several of us from Sun attended the Lang.NET Symposium. The symposium was intensely technical and even downright inspiring. Our hosts kindly welcomed presentations from some of us at Sun; we introduced the Da Vinci Machine project. The centerpiece of the conference was the new Dynamic Language Runtime which factors out the reusable logic that glues dynamic languages on top of the CLR.

Why am I suddenly excited about Microsoft technology? Two or three reasons. First, the DLR (with IronPython and IronRuby) is another evidence that we are in some sort of renaissance or resurgence of programming language design. The second thing that excited me at Redmond was a striking case of parallel evolution between the DLR over the CLR on one hand and the Da Vinci Machine over the JVM on the other side... The final reason is that I am excited for the customers of the JVM, because they will also enjoy the new languages in an expansive open-source community, and on their choice of blazingly fast Java virtual machines. [Read More]