Editor's Note: With this installment of The Open Road, author and Cafe Au Lait founder Elliotte Rusty Harold takes over our series on the ongoing development of Java SE 7. And with the goals and processes of the OpenJDK project having been covered in the first installment, Elliotte jumps right in with this guide to actually building JDK 7.

Before that, here's an update on the OpenJDK project, something we'll post at the top of each installment of The Open Road. The latest release is b23, posted October 30. This release addresses a small number of bugs and feature requests, as noted in its release notes, many relating to administrative issues such as top-level README files, such as white-space cleanup and removal of legacy "j2se" references. The previous build, b22, was more substantive, with 65 bug fixes and 14 features integrated, including the splitting out of CORBA, JAXP, and JAXWS into their own workspaces, Swing ThreadPool creation replaced by java.util.concurrent functionality, support for various time zone changes, and creation of an OpenJDK RenderingEngine plugin, meant "to provide a launching point for the Open JDK efforts to replace the Ductus library."

But how to get at all these new bits? That's what Elliotte addresses in this article, as he shows how to build the JDK from source.

Now that Sun's Java Development Kit is free-as-in-speech (modulo a few small pieces that are in the process of being replaced), it's time to start hacking. Whether your passion is optimization, experimentation, language design, debugging, or documentation, there's a ton of work to be done and many opportunities for coders of all skill levels. In future articles in this series, I'll explore many of the developing APIs that may appear in Java 7 and beyond. However, to play with these today, you'll need to be working on the bleeding edge, so get out your first-aid kits, tear off a tourniquet or two, and get ready to bleed a little. We're going to build the JDK.

System Requirements

The first thing you need to build the JDK is a supported operating system. That means Linux, Solaris, Windows XP, or Windows Vista. Mac OS X is not supported. Apple is responsible for porting the JDK to the Mac, and as usual they're years behind the curve. Mac users can, however, use Parallels, VMWare Fusion, or Boot Camp to run Windows or Linux and build the JDK there. Indeed, that's what I used to write this article when the Ethernet card in my PC died unexpectedly two days before the deadline. However, you still won't be able to run Java 7 (or 6) apps on Mac OS until Apple releases an up-to-date JDK. This could happen tomorrow, or it could happen next year, or it could happen never.

The second thing you'll need is the most recent Java 6 SDK. Java 7 cannot be compiled by Java 5. Personally, I like to design software so that it doesn't have such tight version dependencies, and this is especially important for open source software. However, the JDK is only recently open source and shows some legacies of the corporate IT "we can mandate everyone's build environment" attitude. Removing some of these dependencies is an ongoing task that will take years, but the process is underway. Just recently Sun switched their source repository from the proprietary Teamware to the open source Mercurial. Switching from a closed development model to an open one is a huge task, but it will result in a stronger, more flexible, and robust code base in the end.

The third thing you'll need is a C compiler. Some of the JDK is written in native code. It can't all be pure Java. For Linux, this means gcc4. For Windows, this means Microsoft Visual Studio .NET. For Solaris, this means Sun Studio 11.

On Linux, you'll probably also need to install or update a few libraries. Exactly which ones you'll have to install varies depending on your distribution and version. You'll likely also need to install a bunch of C header files for libraries you already have. In this article, I work from a stock Ubuntu 7.10 Gutsy Gibbon distro. Most other reasonably current distros should also work. If you find any that don't, it's worth figuring out why and filing a bug.

Finally, on Windows, you have build on top of an NTFS file system. You cannot build the JDK on FAT-32. You will also need to install Cygwin, because the JDK build on Windows is a weird mix of Windows and Unix utilities.

Getting the Source

About once a month, Sun posts a bundle of the complete JDK source to the OpenJDK Source Releases page. There are several different bundles here:

The OpenJDK source

The main source code for about 95 percent of the JDK 7.

A binary plug for your platform

Sun doesn't actually own all the code in the JDK, and they can't re-license what they don't own. Instead, a few pieces have to be provided as closed-source binaries. You'll need to download one of these for your platform. Linux, Solaris, and Windows are all supported in both 32- and 64-bit versions.

Jtreg test harness binary download

A test framework for the code. You don't actually have to use this to just build or hack on the code, but you should grab it anyway.

OpenJDK modules project

This covers the the new module system for Java 7. (I'll cover this in a future article in this series.) Eventually this will be rolled into the JDK, but for now you don't immediately need it.

Together these take up over 120 MB, and the download server isn't always that fast, so it may take a little while to get them all. One nice feature of open source is that there are no annoying licenses to click through. You just download what you want from an ordinary URL. This makes it much easier to use curl, wget, and similar tools. Set up a batch job to grab them and go get a cup of coffee. In fact, no-clickthroughs makes it easier to use a regular browser too. Click-through licenses should probably just be abolished. All they do is make more work for lawyers, and who besides lawyers needs that?

jars@jars-desktop:~/openjdk$ wget 
  http://www.java.net/download/openjdk/jdk7/promoted/b23/openjdk-7-ea-src-...
--18:02:02--  
http://www.java.net/download/openjdk/jdk7/promoted/b23/openjdk-7-ea-src-...
           => `openjdk-7-ea-src-b23-30_oct_2007.zip'
Resolving www.java.net... 64.125.132.37, 64.125.132.39
Connecting to www.java.net|64.125.132.37|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: 
  http://download.java.net/openjdk/jdk7/promoted/b23/openjdk-7-ea-src-b23-... 
 [following]
--18:02:02--
  http://download.java.net/openjdk/jdk7/promoted/b23/openjdk-7-ea-src-b23-...
           => `openjdk-7-ea-src-b23-30_oct_2007.zip'
Resolving download.java.net... 72.5.124.114
Connecting to download.java.net|72.5.124.114|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 84,617,174 (81M) [application/zip]

44% [===============>             ] 37,717,212    55.36K/s    ETA 12:19

For now, you only need the first two: the OpenJDK source and the binary plug. You'll probably want the other two later, though.

Mercurial

Sun has not yet opened up the JDK source code control repository to the general public. However, this will probably happen any week now. The system they have been using is a Sun proprietary product called Teamware. They are transitioning to the open source Mercurial (not Subversion or CVS). Once that's done, you too will be able to check out the absolute latest trunk version.

Ignore all references to Subversion on the OpenJDK sites. These are just standard parts of the java.net project templates. Sun is not actually using the java.net repository or Subversion to manage the OpenJDK. The only thing that's actually in Subversion is the HTML code for the openjdk.java.net website. The real JDK source code is not there.

Downloading a snapshot build gives you code that's more likely than not to build (though there have been a few snapshots that were just flat out broken). However, the code may be a month or more out of date. Checking the code out of Mercurial gives you the absolute latest code. However, it may be completely broken, and you can't be quite sure if a build failure is your fault or the code's. Once the repository has opened up, I recommend building from a known good snapshot first to work out all the kinks in the process, and then moving over to a more recent pull from version control when you're confident in your setup.

Compiling

Now that you've downloaded the bundles, unzip them:

$ unzip openjdk-7-ea-src-b23-30_oct_2007.zip
  inflating: openjdk/control/make/Makefile  
  inflating: openjdk/control/make/README  
  inflating: openjdk/control/make/jprt.config  
  inflating: openjdk/control/make/jprt.properties  
  inflating: openjdk/control/make/make/Defs-internal.gmk  
  inflating: openjdk/control/make/make/README.pre-components  
  ...

Next, move the binary plug JAR file into a convenient directory. The default on Linux is /opt/java/openjdk-binary-plugs. On Solaris, it's /usr/jdk/instances/openjdk-binary-plugs. On Windows, it's C:\openjdk-binary-plugs. You can put the JAR file somewhere else if you like, but you'll have to set the ALT_BINARY_PLUGS_PATH environment variable to point to your location before building.

It's silly to spread the binary plugs used just for building into such diverse places. Since this is an open source project, anyone can fix problems, so let's make this our first TODO:

TODO: Rewrite the build file so it looks in a standard openjdk/binary-plugs directory where the rest of the source code is first.

In fact, the binary plugs really aren't that big, so it would almost make sense to just distribute them all with the source bundle in the first place.

The unzipped openjdk directory contains a few readme files and directories for individual subprojects including jdk, hotspot, langtools, jaxws, and jaxp. These directories are supposed to be able to be built separately, but my efforts to do so were unsuccessful.

Generating Make Files

To build the JDK, you need to make the jdk_generic_profile.sh script executable and then run it. In the top openjdk directory, type:

$ chmod +x ./jdk/make/jdk_generic_profile.sh
$ ./jdk/make/jdk_generic_profile.sh

More likely than not this will fail. The first time I did this, I got these messages:

WARNING: Cannot access ALT_BOOTDIR=/opt/java/jdk1.6.0
WARNING: Missing ALT_BINARY_PLUGS_PATH: /opt/java/openjdk-binary-plugs

I had installed these, but not in the exact locations where the makefile expected to find them. The ALT_BINARY_PLUGS_PATH and ALT_BOOTDIR environment variables need to be set to the locations of your JDK install and your binary plugs directory, respectively. So I did:

$ export ALT_BOOTDIR=/usr/local/java;
$ export ALT_BINARY_PLUGS_PATH=~/plugs

Then the jdk_generic_profile script ran and created makefiles.

Sanity Checking

In the next source drop there may be a top-level makefile, but with b23 you need to now change into the control/make directory. From here, do a sanity check on your build environment with make sanity:

$ cd control/make
$ make sanity

This will warn you about some of the software you've forgotten to install and the environment variables you forgot to set. These are the results of my first sanity check:

$ make sanity
/bin/sh: /usr/bin/gawk: not found
/bin/sh: /usr/bin/gawk: not found
/bin/sh: /NOT-SET/devtools/share/ant/latest/bin/ant: not found
/bin/sh: /NOT-SET/devtools/share/findbugs/latest/bin/findbugs: not found
../make/common/shared/Sanity-Settings.gmk:121: WARNING: ANT_VER should not be empty [Sanity-Settings.gmk]
../make/common/shared/Sanity-Settings.gmk:122: WARNING: FINDBUGS_VER should not be empty [Sanity-Settings.gmk]
../make/common/shared/Sanity-Settings.gmk:191: WARNING: TEMP_FREE_SPACE should not be empty [Sanity-Settings.gmk]
../make/common/shared/Sanity-Settings.gmk:192: WARNING: FREE_SPACE should not be empty [Sanity-Settings.gmk]
../build/linux-i586/tmp/alsaversioncheck.c:1:28: error: alsa/asoundlib.h: No such file or directory
../build/linux-i586/tmp/alsaversioncheck.c: In function 'main':
../build/linux-i586/tmp/alsaversioncheck.c:3: warning: incompatible implicit declaration of built-in function 'printf'
../build/linux-i586/tmp/alsaversioncheck.c:3: error: 'SND_LIB_VERSION_STR' undeclared (first use in this function)
../build/linux-i586/tmp/alsaversioncheck.c:3: error: (Each undeclared identifier is reported only once
../build/linux-i586/tmp/alsaversioncheck.c:3: error: for each function it appears in.)
make: *** [../build/linux-i586/tmp/alsaversioncheck] Error 1

I was missing gawk, ant, findbugs, and ALSA. Go ahead and install whatever's missing and try again. After I installed these pieces (using Synaptic) the build still couldn't find Ant, although it was in my path:

$ make sanity
/bin/sh: /NOT-SET/devtools/share/ant/latest/bin/ant: not found
/bin/sh: /NOT-SET/devtools/share/findbugs/latest/bin/findbugs: not found
../make/common/shared/Sanity-Settings.gmk:121: WARNING: ANT_VER should not be empty [Sanity-Settings.gmk]
...
$ /usr/bin/ant -v
Apache Ant version 1.7.0 compiled on August 29 2007
Buildfile: build.xml does not exist!
Build failed

I suppose it would have made sense to try to set ANT_VER but I've worked enough with Ant that I suspected ANT_HOME might be what the make script really needed. I tried setting ANT_HOME and ran another sanity check:

$ export ANT_HOME=/usr/share/ant
$ make sanity
/bin/sh: /NOT-SET/devtools/share/findbugs/latest/bin/findbugs: not found
../make/common/shared/Sanity-Settings.gmk:122: WARNING: FINDBUGS_VER should not be empty [Sanity-Settings.gmk]
...

It no longer complained about Ant, but still wanted FindBugs. Personally, as much as I like FindBugs, I find making it a requirement for a base build to be questionable. We should be trying to remove dependencies at this stage, not introduce them. Nonetheless, the build script wants it, so I had to install it. This one wasn't available in Synaptic, so I had to install it manually. The error message was complaining that:

../make/common/shared/Sanity-Settings.gmk:122: WARNING: FINDBUGS_VER should not be empty [Sanity-Settings.gmk]

So I set FINDBUGS_VER to 1.3.0.

$ export FINDBUGS_VER=1.3.0

That didn't work, so I spent fifteen minutes hunting around in the build file and trying different values until I stumbled onto FINDBUGS_HOME. Setting this environment variable equal to the location of FindBugs fixed the problem.

$ export FINDBUGS_HOME=/opt/java/findbugs-1.3.0

TODO: If the problem is a missing FINDBUGS_HOME or ANT_HOME environment variable, then make the error message say that, not "FINDBUGS_VER/ANT_VER is empty." Better yet, remove the dependency on FindBugs completely.

The next problem seemed to be Freetype. I went back to Synaptic to install that. It turned out that although Freetype had been installed in Ubuntu by default, the libraries and header files for development hadn't been. Thus I needed to install the libfreetype6-dev package. This was going to become a common theme for the rest of the install. Possibly you'd have a little less trouble here if you started with an Ubuntu developer config instead of a base desktop config.

In any case, now I was getting somewhere. Not where I wanted to go, I admit, but somewhere:

$ make sanity
make[1]: Entering directory `/home/jars/openjdk/jdk/make/tools/freetypecheck'
make[1]: Nothing to be done for `all'.
make[1]: Leaving directory `/home/jars/openjdk/jdk/make/tools/freetypecheck'


Bootstrap Settings:
  BOOTDIR = /usr/local/java
    ALT_BOOTDIR = /usr/local/java
  BOOT_VER = 1.6 [requires at least 1.5]
  OUTPUTDIR = ./../build/linux-i586
    ALT_OUTPUTDIR = 
  ABS_OUTPUTDIR = /home/jars/openjdk/jdk/build/linux-i586
 
Build Tool Settings:
  SLASH_JAVA = /NOT-SET
    ALT_SLASH_JAVA = 
  VARIANT = OPT
  JDK_DEVTOOLS_DIR = /NOT-SET/devtools
    ALT_JDK_DEVTOOLS_DIR = 
  ANT_HOME = /usr/share/ant
  FINDBUGS_HOME = /home/jars/findbugs-1.3.0
  UNIXCOMMAND_PATH = /bin/
    ALT_UNIXCOMMAND_PATH = 
  COMPILER_PATH = /usr/bin/
    ALT_COMPILER_PATH = 
  DEVTOOLS_PATH = /usr/bin/
    ALT_DEVTOOLS_PATH = 
  UNIXCCS_PATH = /usr/ccs/bin/
    ALT_UNIXCCS_PATH = 
  USRBIN_PATH = /usr/bin/
    ALT_USRBIN_PATH = 
  COMPILER_NAME = GCC
  COMPILER_VERSION = 
  CC_VER = 4.1 [requires at least 3.2]
  ZIP_VER = 2.32 [requires at least 2.2]
  UNZIP_VER = 5.52 [requires at least 5.12]
  ANT_VER = 1.7 [requires at least 1.6.3]
  FINDBUGS_VER = 1.3 [requires at least 1.1]
  TEMPDIR = ./../build/linux-i586/tmp
 
Build Directives:
  OPENJDK = true
  USE_HOTSPOT_INTERPRETER_MODE = 
  PEDANTIC = 
  DEV_ONLY = 
  NO_DOCS = 
  NO_IMAGES = 
  TOOLS_ONLY = 
  INSANE = 
  COMPILE_APPROACH = parallel
  PARALLEL_COMPILE_JOBS = 2
    ALT_PARALLEL_COMPILE_JOBS = 
  FASTDEBUG = 
  COMPILER_WARNINGS_FATAL = false
  COMPILER_WARNING_LEVEL = 
  INCREMENTAL_BUILD = false
  CC_HIGHEST_OPT = -O3 
  CC_HIGHER_OPT = -O3 
  CC_LOWER_OPT = -O2 
  CXXFLAGS =  -O2  -fPIC -DCC_NOEX -W -Wall  -Wno-unused 
   -Wno-parentheses -fno-omit-frame-pointer -D_LITTLE_ENDIAN 
  CFLAGS =  -O2    -fno-strict-aliasing -fPIC -W -Wall  
   -Wno-unused -Wno-parentheses -fno-omit-frame-pointer -D_LITTLE_ENDIAN 
  BOOT_JAVA_CMD = /usr/local/java/bin/java  -client -Xmx375m -Xms128m 
   -XX:PermSize=32m -XX:MaxPermSize=160m
  BOOT_JAVAC_CMD = /usr/local/java/bin/javac  
   -J-XX:ThreadStackSize=768 -J-client -J-Xmx375m -J-Xms128m 
   -J-XX:PermSize=32m -J-XX:MaxPermSize=160m -encoding ascii
  BOOT_JAR_CMD = /usr/local/java/bin/jar
  BOOT_JARSIGNER_CMD = /usr/local/java/bin/jarsigner
  JAVAC_CMD = /NOT-SET/re/jdk/1.7.0/promoted/latest/binaries/linux-i586/bin/javac 
   -J-XX:ThreadStackSize=768 -J-client -J-Xmx375m -J-Xms128m 
   -J-XX:PermSize=32m -J-XX:MaxPermSize=160m  -source 1.5 -target 5 
   -encoding ascii -Xbootclasspath:./../build/linux-i586/classes 
  JAVAH_CMD = /NOT-SET/re/jdk/1.7.0/promoted/latest/binaries/linux-i586/bin/javah 
   -bootclasspath ./../build/linux-i586/classes
  JAVADOC_CMD = /NOT-SET/re/jdk/1.7.0/promoted/latest/binaries/linux-i586/bin/javadoc 
   -J-client -J-Xmx375m -J-Xms128m -J-XX:PermSize=32m -J-XX:MaxPermSize=160m
 
Build Platform Settings:
  USER = jars
  PLATFORM = linux
  ARCH = i586
  LIBARCH = i386
  ARCH_FAMILY = i586
  ARCH_DATA_MODEL = 32
  ARCHPROP = i386
  LINUX_VERSION = Unknown linux
  ALSA_VERSION = 1.0.14a
  OS_VERSION = 2.6.22-14-generic [requires at least 2.4.9-e.3]
  OS_NAME = linux
  TEMP_FREE_SPACE = 7515236
  FREE_SPACE = 7515236
  MB_OF_MEMORY = 503
 
GNU Make Settings:
  MAKE = make
  MAKE_VER = 3.81 [requires at least 3.78]
  MAKECMDGOALS = sanity
  MAKEFLAGS = 
  SHELL = /bin/sh
 
Target Build Versions:
  JDK_VERSION = 1.7.0
  MILESTONE = internal
  RELEASE = 1.7.0-internal
  FULL_VERSION = 1.7.0-internal-jars_17_nov_2007_21_32-b00
  BUILD_NUMBER = b00
 
External File/Binary Locations:
  USRJDKINSTANCES_PATH = /opt/java
  BUILD_JDK_IMPORT_PATH = /NOT-SET/re/jdk/1.7.0/promoted/latest/binaries
    ALT_BUILD_JDK_IMPORT_PATH = 
  JDK_IMPORT_PATH = /NOT-SET/re/jdk/1.7.0/promoted/latest/binaries/linux-i586
    ALT_JDK_IMPORT_PATH = 
  LANGTOOLS_DIST = 
    ALT_LANGTOOLS_DIST = 
  CORBA_DIST = 
    ALT_CORBA_DIST = 
  JAXP_DIST = 
    ALT_JAXP_DIST = 
  JAXWS_DIST = 
    ALT_JAXWS_DIST = 
  HOTSPOT_DOCS_IMPORT_PATH = /NO_DOCS_DIR
    ALT_HOTSPOT_DOCS_IMPORT_PATH = 
  HOTSPOT_IMPORT_PATH = /NOT-SET/re/jdk/1.7.0/promoted/latest/binaries/linux-i586
    ALT_HOTSPOT_IMPORT_PATH = 
  HOTSPOT_CLIENT_PATH = /NOT-SET/re/jdk/1.7.0/promoted/latest/binaries/linux-i586/jre/lib/i386/client
    ALT_HOTSPOT_CLIENT_PATH = 
  HOTSPOT_SERVER_PATH = /NOT-SET/re/jdk/1.7.0/promoted/latest/binaries/linux-i586/jre/lib/i386/server
    ALT_HOTSPOT_SERVER_PATH = 
  CACERTS_FILE = ./../src/share/lib/security/cacerts
    ALT_CACERTS_FILE = 
  CUPS_HEADERS_PATH = /usr/include
    ALT_CUPS_HEADERS_PATH = 
 
OpenJDK-specific settings:
  FREETYPE_HEADERS_PATH = /usr/include
    ALT_FREETYPE_HEADERS_PATH = 
  FREETYPE_LIB_PATH = /usr/lib
    ALT_FREETYPE_LIB_PATH = 
 
OPENJDK Import Binary Plug Settings:
  BINARY_PLUGS_JARFILE = /home/jars/plugs/jre/lib/rt-closed.jar
    ALT_BINARY_PLUGS_JARFILE = 
  BINARY_PLUGS_PATH = /home/jars/plugs
    ALT_BINARY_PLUGS_PATH = /home/jars/plugs
  BUILD_BINARY_PLUGS_PATH = /NOT-SET/re/jdk/1.7.0/promoted/latest/openjdk/binaryplugs
    ALT_BUILD_BINARY_PLUGS_PATH = 
  PLUG_LIBRARY_NAMES = 
 
Previous JDK Settings:
  PREVIOUS_RELEASE_PATH = /NOT-SET/re/jdk/1.6.0/archive/fcs/bundles/linux-i586
    ALT_PREVIOUS_RELEASE_PATH = 
  PREVIOUS_JDK_VERSION = 1.6.0
    ALT_PREVIOUS_JDK_VERSION = 
  PREVIOUS_JDK_FILE = jdk-6-linux-i586.tar.gz
    ALT_PREVIOUS_JDK_FILE = 
  PREVIOUS_JRE_FILE = jre-6-linux-i586.tar.gz
    ALT_PREVIOUS_JRE_FILE = 
  PREVIOUS_RELEASE_IMAGE = 
    ALT_PREVIOUS_RELEASE_IMAGE = 


WARNING: This machine appears to only have 503Mb of physical memory, 
         builds on this machine could be slow. 
 
WARNING: LANG has been set to en_US.UTF-8, this can cause build failures. 
         Try setting LANG to "C". 
 
WARNING: The directory HOTSPOT_IMPORT_PATH=/NOT-SET/re/jdk/1.7.0/promoted/latest/binaries/linux-i586 
       does not exist, check your value of ALT_HOTSPOT_IMPORT_PATH. 
 
WARNING: HOTSPOT_IMPORT_PATH does not contain the interface file jvmti.h. 
         Check your value of ALT_HOTSPOT_IMPORT_PATH. 
 
WARNING: File /NOT-SET/re/jdk/1.7.0/promoted/latest/binaries/linux-i586/lib/sa-jdi.jar 
         does not exist. The JDI binding for the Serviceability Agent will not be 
         included in the build.
         Please check your access to
           /NOT-SET/re/jdk/1.7.0/promoted/latest/binaries/linux-i586/lib/sa-jdi.jar
         and/or check your value of ALT_HOTSPOT_IMPORT_PATH.
 
ERROR: You do not have access to valid Cups header files. 
       Please check your access to 
           /usr/include/cups/cups.h 
       and/or check your value of ALT_CUPS_HEADERS_PATH, 
       CUPS is frequently pre-installed on many systems, 
       or may be downloaded from http://www.cups.org 
 
ERROR: HOTSPOT_CLIENT_PATH does not point to a valid HotSpot VM. 
       Please check your access to 
           /NOT-SET/re/jdk/1.7.0/promoted/latest/binaries/linux-i586/jre/lib/i386/client/libjvm.so 
       and/or check your value of ALT_HOTSPOT_CLIENT_PATH. 
 
ERROR: HOTSPOT_SERVER_PATH does not point to a valid HotSpot VM. 
       Please check your access to 
           /NOT-SET/re/jdk/1.7.0/promoted/latest/binaries/linux-i586/jre/lib/i386/server/libjvm.so 
       and/or check your value of ALT_HOTSPOT_SERVER_PATH. 
 
Exiting because of the above error(s). 

Continuing with the actual error messages in order, the next problem seemed to be that Ubuntu sets the LANG environment variable to en_US.UTF-8 by default, and the build script wants it to be C. Ubuntu is right here. In 2007, UTF-8 is the only default encoding anyone should use for anything. Nonetheless let's fix that and move on:

$ export LANG=C

TODO: Figure out why the build script insists on C as the LANG. I suspect two fundamentally different properties (natural language and programming language) are colliding over the same environment variable name here.

The next error was:

HOTSPOT_IMPORT_PATH=/NOT-SET/re/jdk/1.7.0/promoted/latest/binaries/linux-i586 does not exist, check your value of ALT_HOTSPOT_IMPORT_PATH.

What the heck is that supposed to be? The official build instructions don't mention it. It looks like it's actually supposed to be some Java 7 thing, not even part of the bootstrap JDK. Maybe I have to build HotSpot before I can build the JDK?

After another 15 minutes of reading various blogs, I discovered that the build instructions Sun publishes are wrong (big surprise). They talk about a "top-level Makefile" that doesn't yet exist. Instead I have to run the makefile in the control/make directory and that's supposed to build everything: openjdk, hotspot, everything. Let's try that:

$ make sanity
make[1]: Entering directory `/home/jars/openjdk/jdk/make'
make[2]: Entering directory `/home/jars/openjdk/jdk/make/tools/freetypecheck'
freetypecheck.c: In function 'main':
freetypecheck.c:45: warning: comparison is always false 
 due to limited range of data type
freetypecheck.c:54: warning: comparison is always false 
 due to limited range of data type
make[2]: Leaving directory `/home/jars/openjdk/jdk/make/tools/freetypecheck'
make[1]: Leaving directory `/home/jars/openjdk/jdk/make'

Damn, I thought I'd fixed the freetype problems. But looking closer, it seems these messages are just about a problem in the freetypecheck C program Sun includes with the build, not with freetype itself. The relevant lines are:

   if (strcmp(v, QUOTEMACRO(REQUIRED_FREETYPE_VERSION)) < 0) {
       printf("Failed: headers are too old.\n");
   }

and

   if (strcmp(v, QUOTEMACRO(REQUIRED_FREETYPE_VERSION)) < 0) {
      printf("Failed: too old library.\n");
   }

It looks like if this test always fails, freetype is good, so let's just ignore that one.

TODO: Figure out what's going on here and fix it.

Now I'm down to just one warning and one error:

WARNING: This machine appears to only have 503Mb of physical memory, 
         builds on this machine could be slow. 
 
ERROR: You do not have access to valid Cups header files. 
       Please check your access to 
           /usr/include/cups/cups.h 
       and/or check your value of ALT_CUPS_HEADERS_PATH, 
       CUPS is frequently pre-installed on many systems, 
       or may be downloaded from http://www.cups.org 

I've ordered four gigabytes of RAM for this laptop, but they haven't arrived yet. In the meantime, I'll just live with the slow build. However, CUPS could be a problem. Back to Synaptic. Looks like another dev install issue. It seems to be libcupsys2-dev that I need. Let's install that and try again:

$ make sanity
make[1]: Entering directory `/home/jars/openjdk/jdk/make'
make[2]: Entering directory `/home/jars/openjdk/jdk/make/tools/freetypecheck'
make[2]: Nothing to be done for `all'.
make[2]: Leaving directory `/home/jars/openjdk/jdk/make/tools/freetypecheck'
make[1]: Leaving directory `/home/jars/openjdk/jdk/make'

Build Machine Information:
   build machine = jars-desktop
...
Previous JDK Settings:
  PREVIOUS_RELEASE_PATH = /NOT-SET/re/jdk/1.6.0/archive/fcs/bundles/linux-i586
    ALT_PREVIOUS_RELEASE_PATH = 
  PREVIOUS_JDK_VERSION = 1.6.0
    ALT_PREVIOUS_JDK_VERSION = 
  PREVIOUS_JDK_FILE = jdk--linux-i586.tar.gz
    ALT_PREVIOUS_JDK_FILE = 
  PREVIOUS_JRE_FILE = jre--linux-i586.tar.gz
    ALT_PREVIOUS_JRE_FILE = 
  PREVIOUS_RELEASE_IMAGE = 
    ALT_PREVIOUS_RELEASE_IMAGE = 


WARNING: This machine appears to only have 503Mb of physical memory, 
         builds on this machine could be slow. 
 
Sanity check passed. 

Finally! About seven hours after I started, but the sanity check passes. Now to try to actually build it with make:

Making

$ make
linux i586 1.7.0-internal build started: 07-11-17 22:33
/bin/mkdir -p ../../control/build/linux-i586/j2sdk-image
/bin/mkdir -p /home/jars/openjdk/control/build/linux-i586/j2sdk-image
...
# Running javac:
Check_ALT_JDK_IMPORT_PATH/bin/javac -J-XX:ThreadStackSize=768 
  -J-client -J-Xmx375m -J-Xms128m -J-XX:PermSize=32m 
  -J-XX:MaxPermSize=160m -source 1.5 -target 5 -encoding ascii 
  -classpath /usr/local/java/lib/tools.jar -sourcepath   
/home/jars/openjdk/control/build/linux-i586/corba/gensrc:
../../../src/solaris/classes:
../../../src/share/classes -d 
/home/jars/openjdk/control/build/linux-i586/corba/classes 
@/home/jars/openjdk/control/build/linux-i586/corba/tmp/sun
/javax.transaction.xa/.classes.list
/bin/sh: Check_ALT_JDK_IMPORT_PATH/bin/javac: not found
make[3]: *** [.compile.classlist] Error 127
make[3]: Leaving directory `/home/jars/openjdk/corba/make/javax/xa'
make[2]: *** [build] Error 1
make[2]: Leaving directory `/home/jars/openjdk/corba/make/javax'
make[1]: *** [build] Error 1
make[1]: Leaving directory `/home/jars/openjdk/corba/make'
make: *** [corba-build] Error 2

Hmm, looks like it needs an ALT_JDK_IMPORT_PATH environment variable. The sanity check didn't catch this. It seems to be trying to load javac from there. I don't know why JAVA_HOME isn't good enough, but let's try just setting it to my regular JDK directory:

$ export ALT_JDK_IMPORT_PATH=/usr/local/java

Now the build seems to be moving. There are a lot of error messages of various types, like this one:

../../../../../../../src/share/classes/org/omg/CORBA/ORB.java:593: warning:
non-varargs call of varargs method with inexact argument type for last parameter;
cast to java.lang.Object for a varargs call
cast to java.lang.Object[] for a non-varargs call and to suppress this warning
            return (org.omg.CORBA.NVList)meth.invoke(this, argx);

It seems not all the code in the JDK is up to snuff with the latest Sun coding specs. However, these are just warnings. The code still builds.

TODO: Fix this to get a warning-free build.

The fan/hard drive on the build machine has now spun up, and is making whirring noises that are audible from across the room. This is only a laptop. I hope it doesn't burn out. At least I'm still under the 90-day warranty if it does. OK. It's quieted down a bit. Whew. Oh, wait a minute. It quieted down because it errored out:

/home/jars/openjdk/hotspot/agent/src/share/classes/sun/jvm
/hotspot/debugger/remote/x86/RemoteX86Thread.java:42: 
cannot find symbol
symbol  : class RemoteX86ThreadContext
location: class sun.jvm.hotspot.debugger.remote.x86.RemoteX86Thread
    RemoteX86ThreadContext context = new RemoteX86ThreadContext(debugger);
    ^
/home/jars/openjdk/hotspot/agent/src/share/classes/sun/jvm/hotspot
/debugger/remote/x86/RemoteX86Thread.java:42: cannot find symbol
symbol  : class RemoteX86ThreadContext
location: class sun.jvm.hotspot.debugger.remote.x86.RemoteX86Thread
    RemoteX86ThreadContext context = new RemoteX86ThreadContext(debugger);
                                         ^
Note: /home/jars/openjdk/hotspot/agent/src/share/classes/sun/jvm/hotspot/jdi/SAJDIClassLoader.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
2 errors
make[6]: *** [/home/jars/openjdk/control/build/linux-i586/hotspot/outputdir
/linux_i486_compiler2/product/../generated/sa-jdi.jar] Error 1
make[6]: Leaving directory 
`/home/jars/openjdk/control/build/linux-i586/hotspot/outputdir
/linux_i486_compiler2/product'
make[5]: *** [all] Error 2
make[5]: Leaving directory 
`/home/jars/openjdk/control/build/linux-i586/hotspot/outputdir
/linux_i486_compiler2/product'
make[4]: *** [sa_stuff] Error 2
make[4]: Leaving directory 
`/home/jars/openjdk/control/build/linux-i586/hotspot/outputdir
/linux_i486_compiler2/product'
make[3]: *** [product] Error 2
make[3]: Leaving directory 
`/home/jars/openjdk/control/build/linux-i586/hotspot/outputdir'
make[2]: *** [generic_build2] Error 2
make[2]: Leaving directory `/home/jars/openjdk/hotspot/make'
make[1]: *** [product] Error 2
make[1]: Leaving directory `/home/jars/openjdk/hotspot/make'
make: *** [hotspot-build] Error 2

Still, this is progress. This at least looks like a genuine Java bug, rather than an environmental problem or a C bug. The specific bug seems to be that there is no accessible RemoteThreadContext constructor that takes a RemoteDebuggerClient as an argument. Looking in RemoteX86Thread.java to see what may be going on, I don't see any problem, but when I open RemoteThreadContext.java the problem is obvious: the file is empty! I wonder what happened to it.

Perhaps something hiccuped when I unzipped or downloaded the original file. I grab a new copy, and it does seem to contain RemoteThreadContext.java, so I copy it over to where it should be and make again. This time I manage to play an entire round in Arathi Basin before it dies:

 Timing: 00000 seconds or 0s for make-java-jvm
<<<Finished Recursively making jvm all @ Sat Nov 17 23:54:14 CET 2007.
>>>Recursively making redist all @ Sat Nov 17 23:54:14 CET 2007 ...
make[3]: Entering directory `/home/jars/openjdk/jdk/make/java/redist'
BinaryPlugs import started: Sat Nov 17 23:54:14 CET 2007
BINARY_PLUGS_PATH=/home/jars/plugs
make[3]: *** No rule to make target `/home/jars/plugs/jre/lib/i386/libjsoundhs.so', needed by `/home/jars/openjdk/control/build/linux-i586/lib/i386/libjsoundhs.so'.  Stop.
make[3]: Leaving directory `/home/jars/openjdk/jdk/make/java/redist'
make[2]: *** [all] Error 1
make[2]: Leaving directory `/home/jars/openjdk/jdk/make/java'
make[1]: *** [all] Error 1
make[1]: Leaving directory `/home/jars/openjdk/jdk/make'
make: *** [jdk-build] Error 2

Maybe I didn't install the plug correctly? Maybe I have to unjar it? Nope. That doesn't do the trick. Wait: it's a self-running JAR:

$ java -jar jdk-7-ea-plug-b23-linux-i586-30_oct_2007.jar
Error: Install failed: java.awt.HeadlessException: 
No X11 DISPLAY variable was set, but this program performed 
an operation which requires it.
java.awt.HeadlessException: 
No X11 DISPLAY variable was set, but this program performed 
an operation which requires it.
   at java.awt.GraphicsEnvironment.checkHeadless(GraphicsEnvironment.java:159)
 ...

Hmm, guess I can't run it in the console. I'll have to wheel my chair over to the laptop and run it there.

TODO: Make the binary plug installer able to run headless.

OK. Now I've got the uncompressed binary plugs in /home/jars/plugs/openjdk-binary-plugs. I need to update the environment variable to match and make again:

$ export ALT_BINARY_PLUGS_PATH=/home/jars/openjdk-binary-plugs
$ make

Make runs again, but soon dies in a different location:

In file included from
 /home/jars/openjdk/jdk/src/share/native/sun/awt/../java2d/pipe/Region.h:34,
  from /home/jars/openjdk/jdk/src/share/native/sun/awt/../java2d/pipe/Region.c:30:
/home/jars/openjdk/jdk/src/solaris/native/sun/awt/utility/rect.h:31:22: error: 
X11/Xlib.h: No such file or directory
In file included from 
...
/home/jars/openjdk/jdk/src/solaris/native/sun/awt/img_util_md.h:32: 
  error: expected specifier-qualifier-list before 'XID'
/home/jars/openjdk/jdk/src/share/native/sun/awt/image/BufImgSurfaceData.c: 
  In function 'Java_sun_awt_image_BufImgSurfaceData_freeNativeICMData':
/home/jars/openjdk/jdk/src/share/native/sun/awt/image/BufImgSurfaceData.c:95: 
  warning: cast to pointer from integer of different size
make[4]: *** 
[/home/jars/openjdk/control/build/linux-i586/tmp/sun/sun.awt/awt/obj/BufImgSurfaceData.o] 
Error 1
make[4]: Leaving directory `/home/jars/openjdk/jdk/make/sun/awt'
make[3]: *** [library_parallel_compile] Error 2
make[3]: Leaving directory `/home/jars/openjdk/jdk/make/sun/awt'
make[2]: *** [all] Error 1
make[2]: Leaving directory `/home/jars/openjdk/jdk/make/sun'
make[1]: *** [all] Error 1
make[1]: Leaving directory `/home/jars/openjdk/jdk/make'
make: *** [jdk-build] Error 2

Maybe I'm missing some X11 development libraries? Back to Synaptic. Let's install libx11-dev and try again. Nope, that didn't do it. A little Googling and it looks like libxt-dev is the missing piece. This time I get a little bit further. Now a different file is missing:

/home/jars/openjdk/jdk/src/solaris/native/sun/awt/splashscreen/splashscreen_config.h:33:34: error: X11/extensions/shape.h: No such file or directory.

This time I go straight to Google and discover that this "include file is part of the Nonrectangular Window Shape Extension standard." It seems libext-dev is the package I need. Once more into the breach.

I think this may be the last one. I seem to have actually compiled everything. Of course, this being C and not Java, that doesn't mean I'm done. Now there are linker errors:

/usr/bin/ld: cannot find -lXtst

So I install the libxtst-dev headers.

This is getting boring. Let me just list the other libraries I had to install:

• libXi-dev

Hmm, that may have been it. It seems to be generating JavaDoc now. There are a lot of broken but easy-to-fix JavaDoc tags, but I think it's actually done. Now if I can only figure out where the build put everything. :-)

Installing

The output seems to be in openjdk/control/build/linux-i586/j2sdk-image. (Some other build products like a JRE with no dev tools are also in openjdk/control/build/linux-i586/.) Let's try copying that to /opt/java, setting it as JAVA_HOME, and adding it to the path:

$ sudo cp -R j2sdk-image /opt/java
$ export JAVA_HOME=/opt/java
$ export PATH=/opt/java/bin:$PATH

And now the moment of truth:

$ java -version
openjdk version "1.7.0-internal"
OpenJDK Runtime Environment (build 1.7.0-internal-jars_18_nov_2007_01_03-b00)
OpenJDK Client VM (build 12.0-b01, mixed mode)
$ javac -version
javac 1.7.0-internal

Success! It's 12:32 in the morning and I started around 10:00 (A.M., not P.M.); but it's done. Now that all the right libraries are installed maybe the next install will only take seven hours.

Easier Paths

Raw builds with make are essential for automation, testing, porting, and continuous integration. However, they aren't always the easiest way to do day-to-day development. If this just seems too fraught for you, there are alternatives, though these are not without similar issues of their own.

Prebuilt Binaries

If debugging makefiles isn't your idea of a good way to spend a Saturday, then you may just want to install a prebuilt binary from the JDK 7 binary snapshots page instead.

IDEs

I've focused on building from the command line because that's the most generic and interoperable approach. It also makes automation and testing a lot easier. Command-line builds should be supported by all good open source software. That said, sometimes an IDE can help. Sun actively encourages the use of NetBeans to build and hack on the JDK, and the openjdk downloads come with preconfigured NetBeans projects in openjdk/jdk/make/netbeans. Just open them up in NetBeans and go. More instructions for that are on the NetBeans website. Just be warned that those instructions aren't completely accurate, either, and you'll likely still need to do some debugging to get it all to build, even in NetBeans.

Summing Up

What have we learned? It is possible to build the JDK. It just takes a day or two the first time you do it, and familiarity with Unix and C libraries doesn't hurt. Hopefully it will take you less time than it took me. Starting from a stock Ubuntu desktop configuration, here's what you need to do:

1. Install the latest JDK 6 from Sun. (Ubuntu only ships with a JRE.)
2. Download the source code bundle from the OpenJDK Source Releases page and unzip it to create the openjdk directory.
3. Download the binary plugs JAR from the same page and run it to create the openjdk-binary-plugs directory.
4. Install these packages:
   • gawk
   • ant
   • findbugs
   • ALSA
   • libcupsys2-dev
   • libext-dev
   • libXi-dev
   • libxt-dev
   • libxtst-dev
5. Manually install FindBugs
6. Set the following environment variables:
   • ALT_BOOTDIR= where JDK 6 is installed
   • ALT_BINARY_PLUGS_PATH= wherever you installed the binary plugs
   • ANT_HOME= wherever you installed Ant
   • FINDBUGS_HOME= wherever you installed FindBugs
   • ALT_JDK_IMPORT_PATH= where JDK 6 is installed
   • LANG=C
7. Make openjdk/jdk/make/jdk_generic_profile.sh executable and run it.
8. Move into openjdk/control/make and type make sanity. Debug any problems that arise.
9. Once the sanity checks pass, type make. Go get a cup of coffee. This will take a while.
10. Once the build is complete, copy the directory openjdk/control/build/linux-i586/j2sdk-image to wherever you want to put your JDK. The directory /opt/java7 may be a nice choice.

On other Linux distros, you may discover you need to add a few more libraries that Ubuntu bundles by default, or you may not need to add all of the ones listed here. On other operating systems such as Solaris, you may need to specify gnumake instead of make (gnumake is the default make on Ubuntu). On Windows, good luck. I'm sure it can be done, but I don't have another full day to spend debugging that one. Maybe in another article.

Elliotte Rusty Harold is the author of numerous books including Java I/O, Java Network Programming, and the upcoming Refactoring HTML.