A. Sundararajan's Weblog

Week-end fun with the java compiler source code

I downloaded java compiler (javac) source code from the JDK 7 site. I did not download entire JDK – I just downloaded compiler-7-ea-src-b15-05_jul_2007.zip I've installed JDK 6 and NetBeans 5.0

I extracted the source zip file into c:\javac directory. From NetBeans IDE, File->Open Project menu, I chose c:\javac\compiler directory. Then, I build the project – I scrolled the build output log to the end and I saw:

Building jar: C:\javac\compiler\dist\lib\javac.jar

build-bin.javac:

Copying 1 file to C:\javac\compiler\dist\bin

build:

BUILD SUCCESSFUL (total time: 8 seconds)

So, I tried to run the newly compiled java compiler. I attempted to compile a simple “Hello World” program. I got the following error:

C:\javac\compiler\dist\lib>java -jar javac.jar Hello.java

Exception in thread "main" java.lang.NoClassDefFoundError: com/sun/tools/javac/Main

What happened? I looked at the build log again. I missed the following lines – because I had seen only at the end!!

Copying 7 files to C:\javac\compiler\build\bootclasses

recompiling compiler with itself

javac: invalid flag: C:\javac\compiler\build\classes

Usage: javac <options> <source files>

use -help for a list of possible options

Java Result: 2

Copying 7 files to C:\javac\compiler\build\classes

Copying 1 file to C:\javac\compiler\build\classes\com\sun\tools\javac\resources

Building jar: C:\javac\compiler\dist\lib\javac.jar

build-bin.javac:

Copying 1 file to C:\javac\compiler\dist\bin

build:

BUILD SUCCESSFUL (total time: 8 seconds)

Looks like there is a build error. The compiler in built in two steps:

1. The sources are built with javac in JDK 6 (on which my NetBeans IDE ran)

2. Then, compiler sources are built again – but this time with the new compiler binary generated by step (1).

Looks we got error in the step (2) [see above: recompiling compiler with itself] . I searched the ant script used to build for “recompiling compiler with itself”. The following is the fragment after that:

<echo message="recompiling compiler with itself"/>

<pathconvert pathsep=" " property="src.javac.files">

<path>

<fileset dir="${src.classes}">

<patternset refid="src.javac"/>

</fileset>

</path>

</pathconvert>

<java fork="true" classpath="${build.bootclasses}" classname="com.sun.tools.javac.Main">

<arg value="-sourcepath"/>

<arg value=""/>

<arg value="-d"/>

<arg file="${build.classes}"/>

<arg value="-g:source,lines"/>

<arg line="${src.javac.files}"/>

</java>

The problem seems to be with “java” command above. Empty string is set as value for -sourcepath option. I changed that to the following:

<arg value="-sourcepath"/>

<arg value="${src.classes}"/>

When I re-built the compiler after the above change, there were no errors – yes, I scrolled the build output to check it And newly compiled javac could compile “Hello World” program.

Now, I wanted to make some to “interesting” but simple change to the compiler source. From a “doc” page, I came to know that there is a hidden javac option called “-printflat”. It appears that with -printflat option javac prints source code after doing transformations for generic types, inner classes, enhanced for-loops, assertions etc. It would be great to visualize the kind of transformations done by javac. So, I wanted to make “hidden” option available. I searched for “printflat” in the project. I got three hits:

1. JavaCompiler.java

2. RecognizedOptions.java

3. java.properties

As usual, I am impatient – wanted to enable printflat option always [regardless of what the command line is]. So, I changed the following line in JavaCompiler.java

printFlat = options.get("-printflat") != null;

to

printFlat = true; // options.get("-printflat") != null;

so that the secret option is enabled always. After rebuilding the compiler, I tried compiling my “Hello World” program. Surprise! I got the following error:

C:\javac\compiler\dist\lib>java -jar javac.jar Hello.java

Exception in thread "main" java.lang.NoClassDefFoundError: com/sun/tools/javac/Main

When I checked “javac.jar” by “jar tvf javac.jar”, I saw only “.java” files instead of “.class” files! Remember I mentioned that javac is recompiled by itself (step (2) above)? Apparently with “-printflat” option, javac just write transformed files but does not generate .class files! Because I had hardcoded printflat to be true always, during the second bootstrap compilation javac did not generate .class files. Looks like my lazy way does not work! I need to find how to really change the code to accept printflat command line option explicitly. I cut the story shot and just summarize the changes I made:

1. added a enum value to com.sun.tools.javac.main.OptionName – PRINTFLAT("-printflat");

2. In com.sun.tools.javac.main.RecognizedOptions class, I added PRINTFLAT to “static Set<OptionName> javacOptions” initialization value.

3. In public static Option[] getAll(final OptionHelper helper) method of RecognizedOptions class, I added “new HiddenOption(PRINTFLAT)” as an element in the returned Option[].

I managed to compile and run the compiler after the above changes! Now when I can pass “printflat” option!! I compiled the following simple Book.java:

class Book {
  private String name;
  public Book(String name) {
     this.name = name;
  }

  class Order {
     private int quantity;
     public Order(int quantity) {
        this.quantity = quantity;
     }
  }
}

c:\javac\compiler\dist\lib\>java -jar javac.jar -printflat c:\Book.java

Now, I can see the generated Book.java and Book$Order.java in the current directory where java compiler was run:

class Book {
    private String name;
    
    public Book(String name) {
        super();
        this.name = name;
    }
    {
    }
}

class Book$Order {
    /*synthetic*/ final Book this$0;
    private int quantity;
    
    public Book$Order(/*synthetic*/ final Book this$0, int quantity) {
        this.this$0 = this$0;
        super();
        this.quantity = quantity;
    }
}

Wow! I can see how java compiler generates a hidden synthetic parameter for the outer class object and so on. Note that the java compiler does not overwrite your original source files. You need to run the compiler in a different directory – compiler generates new files [which is good, you won't accidentally overwrite your original code with generics, inner classes and so on].

Now, you can experiment with constructs like asserts, inner class methods accessing outer's private methods/fields, anonymous/local classes, local class accessing final parameters/locals of enclosing method, generics, enhanced for-loop and so on and see how java compiler transforms those constructs to generate good-old “flat” classes without these features. Have fun!!

( Jul 14 2007, 11:28:29 PM IST ) Permalink Comments [1] | | | | |