Java and security bits

Moving on

After more than six years at Sun working on the Java SE platform, I have decided to move on. Today was my last day. I still very much believe in all the technologies I was involved in and will leave it to my colleagues to continue the work. Thanks to OpenJDK I may not disappear completely and pop in every once in a while.

If you want to follow my future adventurers, see my new blog.

Posted at 17:19 Mar 14, 2008 by Andreas Sterbenz in About  | 

Crypto Code and OpenJDK

I just got a question about the status of the crypto code in OpenJDK that referred to the limitations mentioned in my blog post from last May. Back then some of the encryption code was not open source as we were working through some build and export control issues. Let me give a quick update on that: Brad spent quite a bit of time on this and eventually those issues were resolved, as announced in a message to the mailing lists. That means all the crypto code is available on OpenJDK now. There are a couple of checks (relating to signed providers) which are present in Sun's binaries but that are not present in OpenJDK, but that is simply because those checks are neither needed nor particularly appropriate for an open source project. Everything else is exactly the same as in Sun's releases.

Bottom line is that you can modify and build your own versions of the crypto framework and the crypto providers now. Have at it!

Posted at 18:08 Jan 15, 2008 by Andreas Sterbenz in Java  | 

JCP Early Draft of JSR 294 Now Available

As you have have already noticed, the JCP Early Draft specification of JSR 294 is now available for download from the JSR page.

In short, it consists of two chapters of the Java Language Specification that have been updated to cover Superpackages. There are change bars to highlight everything that has been touched. Then there are draft revisions for the Java Virtual Machine Specification. Finally, updates to the Java core reflection APIs. If you have followed the earlier explanations here or the expert group discussion, none of this should be too surprising.

If you have any comments about the draft specification, please send them to jsr-294-comments@jcp.org so that Alex and I can collect them and discuss them with the expert group. BTW, we are also working on the implementation, chiefly Jon for the javac changes. We will make the code available on the OpenJDK Modules project once it is ready.

Posted at 17:50 Nov 21, 2007 by Andreas Sterbenz in Modularity  | 

Source Code for the OpenJDK Modules Project

We just posted the first source drop of the OpenJDK Modules project on the project page. The Modules project covers the implementation of JSR 277 and JSR 294 and you now have a chance to get a really early look at it, try it out, and contribute to it.

If you want to know what it is all about and how it works, please read the docs on the project page, in particular the samples and the getting started pages. As the samples show, it is really easy to create and run a Module, even from a remote web server. It is fast too:

% time java -repository http://openjdk.java.net/projects/modules/samplerepo/ -module dep
Module 'dep' calling module 'hello'...
Hello world from module hello 1.0
0.34u 0.18s 0:00.81 64.1%

In other words, it takes less than a second to start the Java VM, download 2 minimal modules from a remote web server, and execute them [1]. (This is on a three year old Opteron 248 running Solaris; Java warm in the page cache) Note the disclaimer about untrusted code at the bottom of the samples page, though.

In general, I am really happy with the size and complexity (or lack thereof) of the JSR 277 implementation. The code that I call the core of the implementation - module initialization and class loading - is less than 1200 lines of code (see src/share/classes/sun/module/core). And that includes the big GPL copyright headers in all the files. Of course the implementation is not complete and will grow somewhat, but it already handles module initialization, including optional dependencies and reexport, versioning, dependency cycles, custom import policies, shallow validation, etc. In other words, all the core features.

Then we have the mailing lists:

• modules-dev@openjdk.java.net: Technical discussion about the implementation of the Modules project

• modules-discuss@openjdk.java.net: General discussion of the Modules project and how to use modules

If you are interested in the conversations of the developers working on the Modules project, in discussions about how to use JSR 277 and 294, or if you want to actively contribute, I encourage you to join.

Now time for a few random notes:

• this is a live development snapshot. It is not a beta. Nothing is complete and polished.

• it is source code for developers. No binaries at this time.

• not much of the JSR 294 implementation is included, because not much exists so far. This will change.

• the way to get the source is by downloading a ZIP file that includes all source code in the j2se workspace. That's why it is 68 MB, not because the Modules implementation is that large. We expect to get something more convenient - such as a Mercurial repository - before too long.

• all the new code in the Modules project is under the GPLv2 (plus classpath exception).

• regression and unit tests are included. More about that in a future post.

• before there is any confusion, the 1 second startup time does not mean we have made any fundamental changes to the JRE, that is done by another project. What the Modules sample program shows is just how fast Java is when you run small and efficient applications and systems.

Posted at 00:00 Jun 28, 2007 by Andreas Sterbenz in Modularity  |  Comments[5]

Nested Superpackages Restated

A few weeks ago I posted an entry that explained what superpackages are about. This is a continuation that deals with nested superpackages. If you have not read the first post, read it before continuing. As a reminder, the short version is: a superpackage is a language construct for information hiding [1].

With that in mind, let us explore what a nested superpackage is: a nested superpackage is a superpackage contained within another superpackage. Since a nested superpackage is still a superpackage, what I said about top level superpackages also applies to nested superpackages. In other words, a nested superpackage is a language construct for information hiding.

Let's look at what this definition says about what a nested superpackage is not: it is not a namespace construct. The Java source language does not recognize multiple types of the same fully qualified name today. Adding new access control mechanisms does not change that. Of course, the other things I mentioned which a superpackage is not also still apply: no changes to the type system or to the way you package your applications.

That states what nested superpackages are and what they are not, but it does not explain why they are useful. But that is pretty simply: they are an access control mechanism and as such allow you to separate code within one project (i.e. one top level superpackage) and to hide implementation details. A way to visualize it is to imagine a bubble representing your project and floating within it all your classes and interfaces. Often you will want to organize those classes into subsystems - a bubble within the bubble. If a subsystem is too large for a single package, you need something else. That is where nested superpackages come in. One example of a project where this structure will be very useful is the core of the JDK, as I explained in this message to the JSR 294 expert group, but there are certainly many other projects with similar needs.

The final thing to say about nested superpackages is that (as described above) they are a way to organize the internals of a component, not a way to aggregate code from multiple components. You would use a deployment module solution such as JSR 277 to do that. The guidance is that if a subsystem is tightly coupled with the rest of a project, it should be a nested superpackage. If it is effectively an independent unit and reusable, it should be its own component as a top level superpackage. Other components can import it using JSR 277. The distinction may seem arbitrary and impossible to make, but it is not much different than deciding whether to organize your code into one, two, or more packages. With a bit of experience, it comes natural to developers.

That was quite a bit of text, but the one thing to remember is that a nested superpackage is a superpackage contained within another superpackage. In other words, a nested superpackage is a language construct for information hiding. Really. Nothing more to it than that.

[1] If you don't like the term information hiding just replace it with access control in this entry. In this context it is a good approximation as information hiding is the concept and access control the mechanism to implement it.

Posted at 11:20 May 30, 2007 by Andreas Sterbenz in Modularity  | 

Security Unit and Regression Tests on OpenJDK

One thing I forgot to mention in my earlier entry is that apart from the implementation source code we also have a large number of unit and regression tests available on OpenJDK. For security alone, it is more than 400 separate tests, with each test often checking multiple conditions. In total there are more than 3500 unit and regression tests on OpenJDK today and the number is only growing as we write new tests and work to make more existing tests available.

These tests are in the j2se/test directory with a directory hierarchy that (usually) mirrors the source files being tested. That means for security most of the tests are in test/java/security and test/sun/security. You can run them using the jtreg harness, which predates JUnit & co.

I don't think I need to explain why tests are important, but I have to quote a sentence I first heard from Bill Moore: If it has not been not tested, it does not work. No ifs, buts, or maybes. Even if something happened to work initially, when there is no test, it will eventually get broken. So make sure to include a test when you decide to contribute code.

Posted at 13:43 May 29, 2007 by Andreas Sterbenz in Java  | 

Modules and Security on OpenJDK

I'd like to say a few words about the projects and code on OpenJDK that I am involved with. First off, if you are interested in reading the JDK source code and maybe even contributing, I suggest starting by downloading the OpenJDK sources as a ZIP file from the download page (who came up with that URL?). You can also browse it online using the Subversion interface, but it is rather painful to navigate (I believe we are getting an OpenGrok server soon).

On to specifics. Let's start with the easy part: there is now a Modules project on OpenJDK. It's easy to talk about because it does not have any real content yet. That will change sometime this summer, once the infrastructure is in place. Then it will host the implementation efforts related to modularity in OpenJDK. That means the JSR 277 implementation, the JSR 294 implementation, as well as related changes not defined by those JSRs, e.g. the java launcher. We will make the code available prior to integration into the official JDK 7 trunk, i.e. much earlier than you would have gotten access prior to open source.

Then there is the Security group, which along with the web page also has a (so far rather quiet) mailing list. A lot of code belongs to that group. Let me highlight the pieces I am somehow involved in.

SSL/TLS (JSSE). The framework lives in src/share/classes/javax/net/ssl. The implementation of the SunJSSE provider is more interesting, see src/share/classes/sun/security/ssl. Note that JSSE was not included in the regular JRL source snapshot drops of JDK 6 and JDK 7 due to legal issues, but it is all available on OpenJDK. For an incremental build to compile just the SSL code after the initial full JDK build, go to the make/sun/security/other and run GNU make.

PKCS#11. The SunPKCS11 provider consists of both Java and C code. The Java code lives in src/share/classes/sun/security/pkcs11 and the C code in src/share/native/sun/security/pkcs11. There is also a little bit of platform specific C code in src/solaris (for Solaris and Linux) and src/windows. For an incremental build, go to make/sun/security/pkcs11. One thing to note about SunPKCS11 is that the encryption interface code is not currently open source. We were working through some build and export control issues with all the code related to encryption, chiefly the JCE framework and SunJCE provider but also parts of SunPKCS11. I believe we cleared all the major hurdles so this should be resolved soon. But for now, when you build SunPKCS11 from OpenJDK, encryption will not be available in that binary.

Smart Card I/O. Although JSR 268 is not part of the Java SE 6 platform, as Sun's Java SE implementation bundles the JSR 268 reference implementation, it is also open source and available on OpenJDK now! You can find the source for the framework in src/share/classes/javax/smartcardio the the SunPCSC provider implementation in src/share/classes/sun/security/smartcardio and src/share/native/sun/security/smartcardio.

Authentication related crypto code, i.e. the SUN and SunRsaSign providers. Most of that is in src/share/classes/sun/security/provider and src/share/classes/sun/security/rsa. As I mentioned above, the SunJCE provider is not yet open source but hopefully will follow soon.

That's about it. To quote Neo: I didn't come here to tell you how this is going to end. I came here to tell you how it's going to begin. In other words: the code is out there now. What happens next is up to you. Exciting times.

One last word: remember that OpenJDK is all about code. Actual bits that do something. The opposite of politics. Of course, some entities are trying to politicize the project. Please ignore them. In particular if those same entities are refusing to commit their core technologies to open source. Just ask them for their bits.

Update: for a status update on the crypto code, see this entry.

Posted at 00:50 May 29, 2007 by Andreas Sterbenz in Java  |  Comments[4]

JavaOne Pen with USB Flash Drive

I finally went through the contents of my JavaOne backpack this weekend. Along with the usual promotional materials there was a small case with a pen - as far as I recall this was the speaker gift. I was about to dismiss it as another uninteresting pen when I noticed that you can pull out the top, which is in fact a 1 GB USB flash drive. Not huge, but at least potentially useful. So to all you JavaOne speakers: take a closer look before you throw it out. Here is an image of it:

Posted at 12:49 May 20, 2007 by Andreas Sterbenz in Java  |  Comments[2]

JavaOne Recap

Brief JavaOne recap now that I have had a weekend to recover:

Good announcements, in particular OpenJDK and JavaFX Script. Not a lot of hype about them, which is a good thing: it let's people work without distractions.

Good crowd. People who want to learn about the Java universe as well as people who want to shape its future. I got to meet a few of them: Dalibor Topic, who serves on the OpenJDK interim governance board; Eugene Kuleshov, who I worked with on the Smart Card I/O JSR; and many others.

Lots of interest in next generation modularity, both development time superpackages (JSR 294) and the deployment time infrastructure (JSR 277). Both of the sessions as well as the BOF were very well attended. At the Q&A sessions we got great feedback that points out the issues we need to work on, but nobody questioned the direction we are heading in. I also got the chance to meet Glyn Normington in person and to go over some of the issues we have been discussing on the JSR 294 expert group mailing list. Watch it for updates.

All around a great vibe at the conference. I was even presented with an OSGi Alliance hat after the superpackages talk :-)

PS: back with more technical content soon.

Posted at 23:29 May 13, 2007 by Andreas Sterbenz in Java  |  Comments[1]

Superpackages Restated

There seems to be some confusion about what superpackages are about. My one sentence explanation is: a superpackage is a language construct for information hiding. Let's examine that statement.

"Information hiding" is fancy way of saying "keeping implementation details private." In Java, we have access control modifiers to achieve that: private, default (aka package private), protected and public. But sometimes "public is too public".

I also said "language construct." That means something that is part of the Java language, defined in the Java Language Specification and enforced by all Java compilers (e.g. javac). It will also be part of the Java Virtual Machine Specification so that it is enforced at runtime by the JVM.

Let's look at all the things I did not say: no changes to the type system. No changes to the compile time and runtime access control architecture (just a small extension). No changes to how you must package your apps: if you had organized your application into 6 "components" before, each with its own JAR file, you most likely will do the same thing after migrating to superpackages (see note at the bottom).

In other words, a superpackage is a language construct for information hiding. Really. Nothing more to it than that.

But if that is still too abstract, how about this version: superpackages define an extension to the Java access control mechanism: a new access level for classes, which is wider than "package private" but narrower than "public". One could call the new access level "superpackage private", but that is potentially confusing so please don't use that term.

PS: a runtime infrastructure for component based applications is also desirable, see JSR 277. But that is very much orthogonal to the information hiding objective of superpackages.

Posted at 02:13 May 08, 2007 by Andreas Sterbenz in Modularity  |  Comments[8]

JavaOne 2007

JavaOne is upon us once again. Here is a list of the sessions and BOFs related to the work that I am involved in:

• Tuesday 9pm: BOF-2400: Modularity in the Next-Generation Java Platform, Standard Edition (Java SE): JSR 277 and JSR 294

• Tuesday 10pm: BOF-2899: Java Programming Language Features in JDK Release 7

• Wednesday 9.35am: TS-2594: Secure Coding Guidelines, Continued: Preventing Attacks and Avoiding Antipatterns

• Thursday 9.35am: TS-2401: Java Language Modularity with Superpackages

• Thursday 4.10pm: TS-2318: JSR 277: Java Module System

• Thursday 8.55pm: BOF-2516: Stump the (Security) Band

My main involvement is with the JSR 294 session that I am co-presenting with Alex as well as the JSR 277/294 BOF and the Security BOF, but I plan to attend all the others as well to help out with questions if needed. See you soon!

Posted at 00:58 May 07, 2007 by Andreas Sterbenz in Java  | 

Superpackage strawman and the JSR 294 mailing list

After a lot of internal discussion, Alex and I recently started the expert group discussion for JSR 294 (Improved Modularity Support in the Java Programming Language). Not an earth shattering event, but what may make this interesting to you is that we decided to make the expert group discussion publicly readable via a web archive and an observer list that anyone can subscribe to. You will not be able to post to the expert group list, but you can read all the discussions from now on. It's all out in the open.

Secondly and related to that, we also posted our strawman proposal for superpackages. The concepts will not be news to you if you read my earlier postings. Alex and I think that it is a straightforward way to address limitations in information hiding in the Java language and so far the expert group agrees.

Finally, we are presenting a session at JavaOne this May. It is called Java Language Modularity with Superpackages (abstract). Stanley will also present a session JSR 277: Java Module System plus we will all do a joint Modularity BOF that covers both JSR 277 and 294. We hope to see you all there.

Posted at 23:53 Apr 05, 2007 by Andreas Sterbenz in Modularity  |  Comments[9]

From No Charge To Free

I first became aware of Java in the fall of 1995 sometime after the Netscape announcement and long before I joined Sun. It was around the release of JDK 1.0 beta as I was working on a paper on Java security (published a year later). At the time I was amazed that Sun was making the technology available as a no-charge ($0) download. I could even get access to the complete source code for research purposes after just sending some signed legal agreement to Sun!

Things have sure changed since then. What was considered extremely open a decade ago seems like less than the bare minimum today. Developers have become used to getting software without charge, more often than not with the source code and the rights to change the code and redistribute those changes. In other words, open source. Java has also become more open, but has not fully followed the larger trend. Not any more!

As has been well covered absolutely everywhere, Sun's Java implementation is being released as free software under the Free Software Foundation's GPL open source license. That is the exactly the same license used by the GNU classpath project. For Java SE, the first two components are being released today with all of JDK 6 and JDK 7 following next year. For all the details and the code, please see the OpenJDK project and the FAQ.

This means all the JDK code that I am involved in will soon be open source. In particular, we plan to include all the security code in the full launch next year, even those parts that have been excluded from the current JRL source drops on java.net for legal reasons (JSSE and JCE). And since we are open sourcing the Sun JDK, that will also include JSR 268 Smart Card I/O API, even though it is not a part of the Java SE 6 platform specification. Just as importantly, because we are releasing the source code for the in-development JDK 7 release, you will get access to the implementations of JSR 277 modules and JSR 294 superpackages as soon as those pieces are integrated into the main source tree sometime next year. All that ready to use under the GPL license.

While not everybody will be happy with all the choices that have been made, a lot of people seem rather pleased. We have statements from many important players in the free software world, be it Tim O'Reilly, Red Hat, Trolltech, Intel, or Ubuntu. Having been able to line up all these people shows that Sun is really serious about open source Java. But if there is one quote that carries more weight than any other and is the least expected, it can only be that of Richard Stallman (video):

I think Sun has [well, with this contribution have] contributed more than any other company to the free software community in the form of software. It shows leadership. It’s an example I hope others will follow.

There is nothing I can add to that.

Posted at 12:34 Nov 13, 2006 by Andreas Sterbenz in Java  | 

Superpackages in JSR 294

The Java language modularity concepts proposed by JSR 294 have recently attracted some attention. I will attempt to clear up a few misconceptions and clarify the relationship of JSR 294 development time superpackages with JSR 277 deployment modules.

The basic ideas are described in the JSR proposal and Gilad's blog entry. Although these pages do not go into much detail, they capture the underlying concepts very well and I encourage you to read them. I will try to explain the background and fill in some of the blanks. Let me stress that this is just the initial proposal and as always the actual specification will be the result of the expert group discussion and may end up looking differently.

Background

JSR 294 deals with information hiding. Information hiding refers to the practice of keeping implementation details concealed while allowing access to published APIs. This prevents consumers from relying on implementation internals and makes it possible to change aspects of the implementation without affecting consumers.

In Java, we achieve information hiding by declaring types (classes and interfaces), methods, and fields with non-public access modifiers. For example, a class with default access (aka package private) or a private method. The access control rules are defined in the JLS and JVMS and are enforced by javac at compile time and the virtual machine at runtime. These low level mechanisms are very reliable and also serve as the foundation of Java platform security.

All this works fine as long as a project fits into a single package. Once the project grows beyond that, you may find yourself forced to make implementation classes public in order to access them from multiple packages in the project. That loses the benefits of information hiding, which is clearly suboptimal and a long-standing complaint that often comes up in language discussions in forums such as JavaOne.

Goals

When we decided to address these issues, we had the following goals in mind:

1. extend the access control model defined by the JLS and JVMS to accomodate projects larger than a single Java package

2. the extensions to the access control mechanisms should be simple and consistent with the existing model

3. access control information should be declared in Java source files so it can be processed and enforced by javac.

4. enable the new model to serve as the basis for deployment modules, but it should also be usable on its own without requiring modifications to the deployment mechanisms.

Proposal

The solution is a new entity tentatively called superpackage. A superpackage can be thought of as a named collection of types from one or more packages. Public types can be declared as exported to make them accessible outside the superpackage. Public types in a superpackage that are not exported are accessible only to other types in the superpackage.

Unlike packages, superpackages are explicitly declared in a Java source file. They are compiled by javac into a .module file (cf. class file). At runtime, the module file is loaded by the VM, which uses it to enforce access control. It is also reified via a reflective API similar to java.lang.Class.

As a rough analogy, you could think of a superpackage as a grouping of related packages similar to how a package is a grouping of related classes. Where today classes have either public or default (package private) access, for code that uses superpackages there will be exported public, non-exported public ("superpackage private"), and default (package private) access levels.

For practical purposes, attempting to access a non-exported class from outside its defining superpackage will result in the same behavior that is observed today when attempting to access a package private class from outside its defining package.

Deployment modules

JSR 294 deals with modularity at the Java language level. Deploying a module also requires consideration of aspects such as class loading, versioning, and packaging formats. Those issues are the topic of the deployment modules defined by JSR 277. That project uses the foundations provided by superpackages to build a system with the desired runtime semantics.

The reason that the development and deployment aspects are handled by two separate JSRs is rather simple: they are dealing with two different sets of problems. For example, the Java source language should not define the JAR file format and a set of libraries should not define VM level access control rules.

There are interactions between development and deployment, but to a large extent we are talking about independent issues. A divide-and-conquer approach that enables focused expert groups to concentrate on their individual problem space is preferable to a single large project. Of course, we want to coordinate between the development and deployment solutions, which is why both expert groups have several members in common.

Q&A

Do JSR 294 superpackages require deployment via JSR 277 modules?

No. Superpackages are independent of the deployment mechanism. They can be deployed in regular JAR files loaded by an existing ClassLoader such as a URLClassLoader, as a JSR 277 module, or using other mechanisms. In all cases the superpackage access control mechanisms are enforced by the Java VM. But the advanced deployment features of JSR 277 are only available if you package and deploy the superpackage as a JSR 277 module.

Why enforce access control at the VM level rather than hide implementation classes via some filtering mechanism in the ClassLoader?

The existing Java access control mechanisms operate at the VM level. That architecture has proved to be reliable and secure. Pushing access control decisions to a ClassLoader is inconsistent with that model. It is also susceptible to incorrect ClassLoader implementations and programming errors leading to accidental disclosure of ClassLoader objects. A deployment module system might still want to hide classes at the ClassLoader level, but the fundamental access control decisions must be made by the VM.

Why does the .module file list all the member and exported types?

An authoritative membership and export list is necessary for access control enforcement. It also eliminates the reliance on packaging provisions such as package sealing that are currently required for correctness in some configurations.

Can a class be a member of more than one superpackage?

No. Just like a class can only be a member of one package, it can also only be a member of one superpackage. At deployment time, an exported class can certainly be accessed from multiple contexts by setting up the ClassLoaders appropriately, but that is unrelated to development time concepts.

What about separate compilation, which is another area mentioned in the JSR 294 submission?

That is a separate topic that we should discuss another time. Stay tuned.

PS: don't let recent history fool you into believing that I will be posting an entry every day ;-) Most times it will be more like 2 or 3 entries a month.

Posted at 15:27 Nov 08, 2006 by Andreas Sterbenz in Modularity  |  Comments[4]

Aboard JSR 294

A short note to let people know that Alex Buckley and myself have assumed responsibility for JSR 294 (Improved Modularity Support in the Java Programming Language) and that we will be serving as co-spec leads. The direction for the JSR has not changed. I had already been working with Gilad on superpackage issues before his recent departure and we will continue on the course that he wisely laid out. You can read more about the concepts in this other posting.

I will also continue to be involved in the work on deployment modules being defined by JSR 277 (Java Module System) taking care of some of the core aspects of the implementation and helping out Stanley on a few spec issues as I mentioned earlier.

Posted at 15:26 Nov 08, 2006 by Andreas Sterbenz in About  |