Final Mercurial repositories of JDK7

Dec 06 2007, 10:52 MSK |  [  Java  ]
ivan@adagio:~/work % time hg fclone http://hg.openjdk.java.net/jdk7/jdk7/
[.]
requesting all changes
adding changesets
adding manifests
adding file changes
added 2 changesets with 26 changes to 26 files
26 files updated, 0 files merged, 0 files removed, 0 files unresolved

[corba]
requesting all changes
adding changesets
adding manifests
adding file changes
added 2 changesets with 1369 changes to 1369 files
1369 files updated, 0 files merged, 0 files removed, 0 files unresolved

[hotspot]
requesting all changes
adding changesets
adding manifests
adding file changes
added 2 changesets with 2895 changes to 2895 files
2895 files updated, 0 files merged, 0 files removed, 0 files unresolved

[jaxp]
requesting all changes
adding changesets
adding manifests
adding file changes
added 2 changesets with 1973 changes to 1973 files
1973 files updated, 0 files merged, 0 files removed, 0 files unresolved

[jaxws]
requesting all changes
adding changesets
adding manifests
adding file changes
added 2 changesets with 2307 changes to 2307 files
2307 files updated, 0 files merged, 0 files removed, 0 files unresolved

[jdk]
requesting all changes
adding changesets
adding manifests
adding file changes
added 2 changesets with 16527 changes to 16527 files
16527 files updated, 0 files merged, 0 files removed, 0 files unresolved

[langtools]
requesting all changes
adding changesets
adding manifests
adding file changes
added 2 changesets with 2974 changes to 2974 files
2974 files updated, 0 files merged, 0 files removed, 0 files unresolved

hg fclone http://hg.openjdk.java.net/jdk7/jdk7/  51.62s user 11.27s system 9% cpu 10:32.51 total

Yes, now you can really start working on your own fixes for JDK and be able to get them into the official repositories if they are that good. See also this post on Mark Reinhold's blog.

Enough said! Go get the JDK! :-)

OpenJDK Mercurial repositories are open

Nov 02 2007, 17:26 MSK |  [  Java  ]

Finally, we have the Mercurial repositories of JDK7 out in the open. They are still EXPERIMENTAL (yeah, in capitals), which means that these repositories (and their clones) are going to become invalid (totally unrelated to the repositories which would be used) in a couple of weeks, but you can still try to set up your environment and development process (if any) using them. Again, please don't do any real work on the clones of these repositories, the changesets you'd have probably would be useless when these repositories become officially open.

Here is a short instruction to get started with hacking the JDK. I assume that you have Mercurial set up. The build instructions here should work for Debian GNU/Linux (this is what I use on my laptop), however I don't promise anything :-) It should be relatively easy to do the same thing on Ubuntu (I doubt there are any differences other than the available package names/versions). It may require more work on other flavors of Linux. Still, if you have some problem building the JDK, you can ask me and I'll try to help you investigate (and, hopefully, fix) the problem.

Prerequisites

I'll assume that you have the following things set up and working:

  • JDK 6
  • Mercurial
  • GCC (>= 4.0)
  • Ant (>= 1.6.3)
  • FindBugs (>= 1.1)
  • GNU make
  • …and lots of other required libraries (see the README files for more info)

Getting the sources

Create the directory where you're going to have your repositories.

/home/ivan/work% mkdir openjdk
/home/ivan/work% cd openjdk

If you have the newer Mercurial with the forest extension, you can simply do

/home/ivan/work/openjdk% hg fclone http://hg.openjdk.java.net/jdk7/MASTER/
but if you don't, just follow the instructions below.

First, clone the master repository

/home/ivan/work/openjdk% hg clone http://hg.openjdk.java.net/jdk7/MASTER/

Clone all the other repositories into the MASTER repository:

/home/ivan/work/openjdk% cd MASTER
/home/ivan/work/openjdk/MASTER% for i in corba hotspot jaxp jaxws jdk langtools; hg clone http://hg.openjdk.java.net/jdk7/MASTER/$i/

After getting the sources, download the binary plugs from the OpenJDK download site and install them (run java -jar <downloaded-plugs-jar>) into some directory (I installed them in openjdk directory which I created above).

GCC 4.2 peculiarities

Now, if you are going to use GCC 4.2 or newer to build the JDK, you'll need to apply the patch to the HotSpot source before you continue. GCC 4.2 is more pedantic than the older ones and doesn't allow one to cast literal strings to char *, now you can only cast to const char * (which makes a lot of sense if you want to minimize the possibility of having nasty bugs which are hard to track). However the patch is a bit hackish: I use const_cast in a couple of places because it is the quickest and least intrusive fix I can provide there. This compilation issue should be addressed separately in JDK7 code-base (maybe in next build?).

First, download the patch, say, to /tmp/const_strings.patch. Now you have two choices: either use patch tool to apply the patch to the hotspot sources

/home/ivan/work/openjdk/MASTER/hotspot% patch -p1 < /tmp/const_strings.patch
or use the Mercurial Queues (the preferred way)
/home/ivan/work/openjdk/MASTER/hotspot% hg qinit
/home/ivan/work/openjdk/MASTER/hotspot% hg qimport /tmp/const_strings.patch
/home/ivan/work/openjdk/MASTER/hotspot% hg qpush const_strings.patch
You can read more about Mercurial Queues in this excellent tutorial (I suggest reading the other chapters too).

Setting up the environment

Build scripts use some environment variables to figure out what tools to use, where the JDK is, how many build threads should work concurrently, etc. You can either export these environment variables manually (like it is shown below) or create a script which sets the variables and executes the GNU make.

Here is what I did manually in shell:

% export LANG=C
% export ANT_HOME=/usr/share/ant 
% export FINDBUGS_HOME=~/work/findbugs-1.2.1 
% export ALT_BINARY_PLUGS_PATH=~/work/openjdk/openjdk-binary-plugs
% export ALT_BOOTDIR=/usr/local/java/jdk1.6.0
Fix the paths according to where you have the things installed. Then run make sanity and inspect the log. Fix the variables, run make sanity. Fix again, … I think you got it.

Although it isn't immediately clear from make sanity output and from the README files, you'd need to export one more variable: ALT_JDK_IMPORT_PATH, and according to the Glossary for JDK builds it should point to the path with a recent build of the same version of JDK we are building. Actually, it is a bug in the corba build files, and hopefully it would be fixed when the repositories are officially rolled out.

Building the JDK

Run make and go have a lunch :-)

Oh, you might want to have a tea first, just to be able to fix some minor problem that breaks the build in your case.

For reference: it took ~ 55 minutes to build the JDK on my 2Ghz laptop. The build directory size is 1.1Gb, although it may take more than that during the build (I didn't bother to measure).

Post-scriptum

Kudos to Kelly O'Hair and the JDK team, the build process now is very easy compared to what we had a little more than a year ago.

Try this at home: Building the JDK (on Debian GNU/Linux)

Aug 24 2006, 00:02 MSD |  [  Java  ]

One of my assignments here at Sun is to build the latest sources available at the Peabody site and provide some useful directions on how to do it. I use Debian GNU/Linux both at work and at home on my laptop, so I'll describe the build process on Debian. Probably someday I'll try to do it on another OS, say Windows or Mac OS X, when I get my hands on the computer with this OS installed. So, here we go...

Preparation: installing the necessary software

You will need to apt-get install the following software to build the JDK on Debian:

  • binutils (>= 2.11.93.0.2-11). I used 2.16.1.
  • gcc and g++ compilers (>= 3.2.2). I used gcc-4.0.2-2 which is stricter to the code than the 3.x versions, and probably some of the problems which I encountered appeared because of this. I plan to try 4.1.1 soon to see if there is a difference.
  • JDK installation to bootstrap some part of the build (>= 1.5). I used JDK 1.5.0_06 which I downloaded from Sun's JDK 5.0 download site at the time when Sun's JDK had not made its way into Debian and Ubuntu distributions yet. Now you can simply do
    apt-get install sun-java5-jdk
  • make (>= 3.78.1). I used 3.80-9.
  • zip (>= 2.2). I used 2.31-1.
  • libcupsys2-dev. I used 1.2.2-1.
  • libasound2-dev. I used 1.0.11-7.

Downloading the sources

Download the JDK 6 Source, JDK Binaries for Source Build 6, and Mozilla Binaries for Source Build 6 (Unix) from the page Java Platform, Standard Edition 6 Source Snapshot Releases. I did it a week ago, so I had a version b95. Some things may have changed since then.

You may also want to read the JDK 6 Readme and the Build Instructions.

Now you need to install the downloaded jars with sources and binaries, not unjar them:

  • $ java -jar jdk-6-rc-src-b95-jrl-10_aug_2006.jar
    • Read and accept the Java Research License.
    • Select the directory to install the sources to. Note: there isn't a single top directory in the archive, so make sure to create one beforehand.
  • $ java -jar jdk-6-rc-bin-b95-jrl-10_aug_2006.jar
    • Read and accept the Module License Agreement.
    • Select the directory to install the binaries (fonts, actually) to. Note: there isn't a single top directory in the archive. I installed the binaries into the same directory as sources and it worked out well.
  • $ java -jar jdk-6-rc-mozilla_headers-b95-unix-10_aug_2006.jar
    • Read and accept the Mozilla Public License.
    • Select the directory to install the sources to. Note: it doesn't have a single top directory in the archive. It can be safely installed into the same directory as sources.

Patching the source

GCC 4.x is stricter than the suggested GCC 3.2.2, so we need to fix some problems in order to build successfully. This part was very complicated in the days of the first beta (at least it had taken lots of my time in March, when I tried to guess how to fix those bugs). Now it is much easier, and you have the following cheat-sheet:

  • GCC compile-time errors "static declaration of ‘foo’ follows non-static declaration": to fix them you need to remove (or comment out) the "static" keyword in the following files:
    • j2se/src/solaris/hpi/native_threads/src/interrupt_md.c, line 115
    • j2se/src/solaris/native/sun/awt/awt_dnd.c, line 172
    • deploy/src/javaws/share/native/launchFile.c, line 66
    • deploy/src/javaws/share/native/xmlparser.c, line 53
  • GCC compile-time error: "PIC register ‘%ebx’ clobbered in ‘asm’" in file j2se/src/solaris/bin/java_md.c, line 1119. This one is very strange (I don't know what it means and whether the fix is safe), I'll try to contact the developer. To fix it, comment out ""%ebx"," in line 1140.
  • In JDK 6 b95 I had the following two problems which were caused by bugs recently introduced to the Makefiles (/usr/local/java/jdk1.5.0_06 is a path to my bootstrap JDK installation):
    • /bin/cp: cannot stat `/usr/local/java/jdk1.5.0_06/jre/plugin/i386/ns7-gcc29/ns7/libjavaplugin_oji.so': No such file or directory
      To fix it, apply the following patch to the file deploy/make/plugin/adapter/oji-gcc29-adapter/Makefile:
      26c26
      <       $(CP) $(ALT_GCC29_PLUGIN_LIB_PATH)/ns7/lib$(LIBRARY).so \
      ---
      >       $(CP) $(ALT_GCC29_PLUGIN_LIB_PATH)/lib$(LIBRARY).so \
    • /bin/cp: cannot stat `/usr/local/java/jdk1.5.0_06/jre/plugin/i386/ns7-gcc29/libjavaplugin_nscp_gcc29.so': No such file or directory
      To fix it, comment out lines 23-24 in file deploy/make/plugin/nscore/linux-gcc29/Makefile

Building the source

Create the build.sh file in the directory where you installed the sources, binaries and Mozilla headers, with the following contents:

#!/bin/bash
# Alter the paths as needed
# /home/ivan/projects/peabody/src is a directory where I installed sources, binaries and Mozilla headers

unset JAVA_HOME
export ALT_BOOTDIR=/usr/local/java/jdk1.5.0_06          # bootstrap JDK location
export ALT_OUTPUTDIR=/home/ivan/projects/peabody/build  # alter the path to which you want the JDK to be built
export ALT_DEVTOOLS_PATH=/usr/bin                       # path to 'make'
export BUILD_NUMBER=b95                                 # build number, you may want to change it
export ALT_MOZILLA_HEADERS_PATH=/home/ivan/projects/peabody/src/share/plugin  
export ALT_GCC29_PLUGIN_LIB_PATH=${ALT_BOOTDIR}/jre/plugin/i386/ns7-gcc29

export SKIP_FASTDEBUG_BUILD=true                        # after building the normal build scsl starts building
export SKIP_DEBUG_BUILD=true                            # fastdebug and debug builds which is probably a bug

cd control/make
make dev
Note: You may also need to set LC_ALL=C if you have some locale-related problems during the build.

Then do

$ chmod u+x build.sh
$ ./build.sh
cross your fingers and wait. If it bails out very quickly, you probably have some problems with paths which you set in build.sh. If you encounter some problems during the build, please write me, I'll do my best to find out how to fix them and then make the information available to the rest of the world!

Post Scriptum

  • Be sure to run
    $ bin/java -jar demo/jfc/Java2D/Java2Demo.jar
    in the build directory after the successful build to see if Java works.
  • If you are curious, you can run
    $ time ./build.sh
    in order to find out how much time it takes to build the JDK on your computer. Here is the time output from the build on my 1.6GHz laptop:
    real 36m46.351s
    user 26m56.408s
    sys  2m59.288s
  • There is a great blog post Honey, I built the JDK! (on Ubuntu) by Cay Horstmann, it saved me lots of time. Thanks, Cay!