Skip to main content

The Open Road: Building the JDK

November 29, 2007

{cs.r.title}







Editor's Note: With this installment of "http://today.java.net/pub/ct/10">The Open Road, author and
Cafe Au Lait founder
Elliotte Rusty Harold takes over our series on the ongoing
development of Java SE 7. And with the goals and processes of the
OpenJDK project having
been covered in the "http://today.java.net/pub/a/today/2007/08/09/looking-ahead-to-java-7.html">
first installment
, Elliotte jumps right in with this guide to
actually building JDK 7.

Before that, here's an update on the OpenJDK project,
something we'll post at the top of each installment of The Open
Road. The latest "http://download.java.net/openjdk/jdk7/">release is b23, posted
October 30. This release addresses a small number of bugs and
feature requests, as noted in its "http://download.java.net/jdk7/changes/jdk7-b23.html">release
notes
, many relating to administrative issues such as top-level
README files, such as white-space cleanup and removal of legacy
"j2se" references. The previous build, "http://download.java.net/jdk7/changes/jdk7-b22.html">b22, was
more substantive, with 65 bug fixes and 14 features integrated,
including the "http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6609966">splitting
out of CORBA, JAXP, and JAXWS
into their own workspaces, Swing
ThreadPool creation "http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6480289">replaced
by java.util.concurrent functionality
, support for
various time zone changes, and "http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6610080">creation
of an OpenJDK RenderingEngine plugin
, meant "to provide a
launching point for the Open JDK efforts to replace the Ductus
library."

But how to get at all these new bits? That's what Elliotte
addresses in this article, as he shows how to build the JDK from
source.

Now that Sun's Java Development Kit is free-as-in-speech (modulo
a few small pieces that are in the process of being replaced), it's
time to start hacking. Whether your passion is optimization,
experimentation, language design, debugging, or documentation,
there's a ton of work to be done and many opportunities for coders
of all skill levels. In future articles in this series, I'll explore
many of the developing APIs that may appear in Java 7 and beyond.
However, to play with these today, you'll need to be working on the
bleeding edge, so get out your first-aid kits, tear off a
tourniquet or two, and get ready to bleed a little. We're
going to build the JDK.

System Requirements

The first thing you need to build the JDK is a supported
operating system. That means Linux, Solaris, Windows XP, or Windows
Vista. Mac OS X is not supported. Apple is responsible for porting
the JDK to the Mac, and as usual they're years behind the curve.
Mac users can, however, use "http://www.parallels.com/">Parallels, "http://www.vmware.com/products/fusion/">VMWare Fusion, or
Boot
Camp
to run Windows or Linux and build the JDK there. Indeed,
that's what I used to write this article when the Ethernet card in
my PC died unexpectedly two days before the deadline. However, you
still won't be able to run Java 7 (or 6) apps on Mac OS until Apple
releases an up-to-date JDK. This could happen tomorrow, or it could
happen next year, or it could happen never.

The second thing you'll need is the most recent Java 6 SDK. Java
7 cannot be compiled by Java 5. Personally, I like to design
software so that it doesn't have such tight version dependencies,
and this is especially important for open source software. However,
the JDK is only recently open source and shows some legacies of the
corporate IT "we can mandate everyone's build environment"
attitude. Removing some of these dependencies is an ongoing task
that will take years, but the process is underway. Just recently
Sun switched their source repository from the proprietary Teamware
to the open source "http://www.selenic.com/mercurial/">Mercurial. Switching from a
closed development model to an open one is a huge task, but it will
result in a stronger, more flexible, and robust code base in the
end.

The third thing you'll need is a C compiler. Some of the JDK is
written in native code. It can't all be pure Java. For Linux, this
means gcc4. For Windows, this means Microsoft Visual Studio .NET.
For Solaris, this means Sun Studio 11.

On Linux, you'll probably also need to install or update a few
libraries. Exactly which ones you'll have to install varies depending
on your distribution and version. You'll likely also need to
install a bunch of C header files for libraries you already have.
In this article, I work from a stock Ubuntu 7.10 Gutsy Gibbon
distro. Most other reasonably current distros should also work. If
you find any that don't, it's worth figuring out why and filing a
bug.

Finally, on Windows, you have build on top of an "NT File System">NTFS file system. You cannot build the
JDK on FAT-32. You
will also need to install "http://www.cygwin.com/">Cygwin, because the "Java Development Kit">JDK build on Windows is a weird
mix of Windows and Unix utilities.

Getting the Source

About once a month, Sun posts a bundle of the complete JDK
source to the "http://download.java.net/openjdk/jdk7/">OpenJDK Source Releases
page
. There are several different bundles here:

The OpenJDK source
The main source code for about 95 percent of the JDK 7.
A binary plug for your platform
Sun doesn't actually own all the code in the JDK, and they
can't re-license what they don't own. Instead, a few pieces have to
be provided as closed-source binaries. You'll need to download one
of these for your platform. Linux, Solaris, and Windows are all
supported in both 32- and 64-bit versions.
Jtreg test harness binary download
A test framework for the code. You don't actually have to use
this to just build or hack on the code, but you should grab it
anyway.
OpenJDK modules project
This covers the the new module system for Java 7. (I'll cover
this in a future article in this series.) Eventually this will be
rolled into the JDK, but for now you don't immediately need
it.

Together these take up over 120 MB, and the download server isn't
always that fast, so it may take a little while to get them all.
One nice feature of open source is that there are no annoying
licenses to click through. You just download what you want from an
ordinary URL. This makes it much easier to use curl,
wget, and similar tools. Set up a batch job to grab them
and go get a cup of coffee. In fact, no-clickthroughs makes it
easier to use a regular browser too. Click-through licenses should
probably just be abolished. All they do is make more work for
lawyers, and who besides lawyers needs that?

jars@jars-desktop:~/openjdk$ wget 
  http://www.java.net/download/openjdk/jdk7/promoted/b23/openjdk-7-ea-src-... 
http://www.java.net/download/openjdk/jdk7/promoted/b23/openjdk-7-ea-src-b23-30_oct_2007.zip
           => `openjdk-7-ea-src-b23-30_oct_2007.zip'
Resolving www.java.net... 64.125.132.37, 64.125.132.39
Connecting to www.java.net|64.125.132.37|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location:
  http://download.java.net/openjdk/jdk7/promoted/b23/openjdk-7-ea-src-b23-...
[following]
--18:02:02--
  http://download.java.net/openjdk/jdk7/promoted/b23/openjdk-7-ea-src-b23-...           => `openjdk-7-ea-src-b23-30_oct_2007.zip'
Resolving download.java.net... 72.5.124.114
Connecting to download.java.net|72.5.124.114|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 84,617,174 (81M) [application/zip]

44% [===============>             ] 37,717,212    55.36K/s    ETA 12:19

For now, you only need the first two: the OpenJDK source and the
binary plug. You'll probably want the other two later, though.

Mercurial

Sun has not yet opened up the JDK source code control repository
to the general public. However, this will probably happen any week
now. The system they have been using is a Sun proprietary product
called Teamware. They are transitioning to the open source "http://www.selenic.com/mercurial/wiki/">Mercurial (not
Subversion or CVS). Once that's done, you too will be able to check
out the absolute latest trunk version.

Ignore all references to Subversion on the OpenJDK
sites. These are just standard parts of the java.net project
templates. Sun is not actually using the java.net repository or
Subversion to manage the OpenJDK. The only thing that's actually in
Subversion is the HTML code for the openjdk.java.net website. The
real JDK source code is not there.

Downloading a snapshot build gives you code that's more likely
than not to build (though there have been a few snapshots that were
just flat out broken). However, the code may be a month or more out
of date. Checking the code out of Mercurial gives you the absolute
latest code. However, it may be completely broken, and you can't be
quite sure if a build failure is your fault or the code's. Once the
repository has opened up, I recommend building from a known good
snapshot first to work out all the kinks in the process, and then
moving over to a more recent pull from version control when you're
confident in your setup.

Compiling

Now that you've downloaded the bundles, unzip them:

$ unzip openjdk-7-ea-src-b23-30_oct_2007.zip
  inflating: openjdk/control/make/Makefile 
  inflating: openjdk/control/make/README 
  inflating: openjdk/control/make/jprt.config 
  inflating: openjdk/control/make/jprt.properties 
  inflating: openjdk/control/make/make/Defs-internal.gmk 
  inflating: openjdk/control/make/make/README.pre-components 
  ...

Next, move the binary plug JAR file into a convenient directory.
The default on Linux is /opt/java/openjdk-binary-plugs. On
Solaris, it's /usr/jdk/instances/openjdk-binary-plugs. On
Windows, it's C:\openjdk-binary-plugs. You can put the JAR
file somewhere else if you like, but you'll have to set the
ALT_BINARY_PLUGS_PATH environment variable to point to
your location before building.

It's silly to spread the binary plugs used just for building
into such diverse places. Since this is an open source project,
anyone can fix problems, so let's make this our first TODO:

TODO: Rewrite the build file so it looks in a standard
openjdk/binary-plugs directory where the rest of the source code is
first.

In fact, the binary plugs really aren't that big, so it would
almost make sense to just distribute them all with the source
bundle in the first place.

The unzipped openjdk directory contains a few readme files and
directories for individual subprojects including jdk,
hotspot, langtools, jaxws, and jaxp.
These directories are supposed to be able to be built separately,
but my efforts to do so were unsuccessful.

Generating Make Files

To build the JDK, you need to make the
jdk_generic_profile.sh script executable and then run it. In
the top openjdk directory, type:

$ chmod +x ./jdk/make/jdk_generic_profile.sh
$ ./jdk/make/jdk_generic_profile.sh

More likely than not this will fail. The first time I did this,
I got these messages:

WARNING: Cannot access ALT_BOOTDIR=/opt/java/jdk1.6.0
WARNING: Missing ALT_BINARY_PLUGS_PATH: /opt/java/openjdk-binary-plugs

I had installed these, but not in the exact locations where the
makefile expected to find them. The
ALT_BINARY_PLUGS_PATH and ALT_BOOTDIR
environment variables need to be set to the locations of your JDK
install and your binary plugs directory, respectively. So I did:

$ export ALT_BOOTDIR=/usr/local/java;
$ export ALT_BINARY_PLUGS_PATH=~/plugs

Then the jdk_generic_profile script ran and created
makefiles.

Sanity Checking

In the next source drop there may be a top-level makefile, but
with b23 you need to now change into the control/make directory.
From here, do a sanity check on your build environment with
make sanity:

$ cd control/make
$ make sanity

This will warn you about some of the software you've forgotten
to install and the environment variables you forgot to set. These
are the results of my first sanity check:

$ make sanity
/bin/sh: /usr/bin/gawk: not found
/bin/sh: /usr/bin/gawk: not found
/bin/sh: /NOT-SET/devtools/share/ant/latest/bin/ant: not found
/bin/sh: /NOT-SET/devtools/share/findbugs/latest/bin/findbugs: not found
../make/common/shared/Sanity-Settings.gmk:121: WARNING: ANT_VER should not be empty [Sanity-Settings.gmk]
../make/common/shared/Sanity-Settings.gmk:122: WARNING: FINDBUGS_VER should not be empty [Sanity-Settings.gmk]
../make/common/shared/Sanity-Settings.gmk:191: WARNING: TEMP_FREE_SPACE should not be empty [Sanity-Settings.gmk]
../make/common/shared/Sanity-Settings.gmk:192: WARNING: FREE_SPACE should not be empty [Sanity-Settings.gmk]
../build/linux-i586/tmp/alsaversioncheck.c:1:28: error: alsa/asoundlib.h: No such file or directory
../build/linux-i586/tmp/alsaversioncheck.c: In function 'main':
../build/linux-i586/tmp/alsaversioncheck.c:3: warning: incompatible implicit declaration of built-in function 'printf'
../build/linux-i586/tmp/alsaversioncheck.c:3: error: 'SND_LIB_VERSION_STR' undeclared (first use in this function)
../build/linux-i586/tmp/alsaversioncheck.c:3: error: (Each undeclared identifier is reported only once
../build/linux-i586/tmp/alsaversioncheck.c:3: error: for each function it appears in.)
make: *** [../build/linux-i586/tmp/alsaversioncheck] Error 1

I was missing gawk, ant,
findbugs, and ALSA. Go ahead and install whatever's
missing and try again. After I installed these pieces (using
Synaptic) the build still couldn't find Ant, although it was in my
path:

$ make sanity
/bin/sh: /NOT-SET/devtools/share/ant/latest/bin/ant: not found
/bin/sh: /NOT-SET/devtools/share/findbugs/latest/bin/findbugs: not found
../make/common/shared/Sanity-Settings.gmk:121: WARNING: ANT_VER should not be empty [Sanity-Settings.gmk]
...
$ /usr/bin/ant -v
Apache Ant version 1.7.0 compiled on August 29 2007
Buildfile: build.xml does not exist!
Build failed

I suppose it would have made sense to try to set
ANT_VER but I've worked enough with Ant that I
suspected ANT_HOME might be what the make script
really needed. I tried setting ANT_HOME and ran
another sanity check:

$ export ANT_HOME=/usr/share/ant
$ make sanity
/bin/sh: /NOT-SET/devtools/share/findbugs/latest/bin/findbugs: not found
../make/common/shared/Sanity-Settings.gmk:122: WARNING: FINDBUGS_VER should not be empty [Sanity-Settings.gmk]
...

It no longer complained about Ant, but still wanted "http://findbugs.sourceforge.net/">FindBugs. Personally, as
much as I like FindBugs, I find making it a requirement for a base
build to be questionable. We should be trying to remove
dependencies at this stage, not introduce them. Nonetheless, the
build script wants it, so I had to install it. This one wasn't
available in Synaptic, so I had to install it manually. The error
message was complaining that:

../make/common/shared/Sanity-Settings.gmk:122: WARNING: FINDBUGS_VER should not be empty [Sanity-Settings.gmk]

So I set FINDBUGS_VER to 1.3.0.

$ export FINDBUGS_VER=1.3.0

That didn't work, so I spent fifteen minutes hunting around in
the build file and trying different values until I stumbled onto
FINDBUGS_HOME. Setting this environment variable equal
to the location of FindBugs fixed the problem.

$ export FINDBUGS_HOME=/opt/java/findbugs-1.3.0

TODO: If the problem is a missing FINDBUGS_HOME or ANT_HOME
environment variable, then make the error message say that, not
"FINDBUGS_VER/ANT_VER is empty." Better yet, remove the dependency on
FindBugs completely.

The next problem seemed to be Freetype. I went back to Synaptic
to install that. It turned out that although Freetype had been
installed in Ubuntu by default, the libraries and header files for
development hadn't been. Thus I needed to install the
libfreetype6-dev package. This was going to become a
common theme for the rest of the install. Possibly you'd have a
little less trouble here if you started with an Ubuntu developer
config instead of a base desktop config.

In any case, now I was getting somewhere. Not where I wanted to
go, I admit, but somewhere:

$ make sanity
make[1]: Entering directory `/home/jars/openjdk/jdk/make/tools/freetypecheck'
make[1]: Nothing to be done for `all'.
make[1]: Leaving directory `/home/jars/openjdk/jdk/make/tools/freetypecheck'


Bootstrap Settings:
  BOOTDIR = /usr/local/java
    ALT_BOOTDIR = /usr/local/java
  BOOT_VER = 1.6 [requires at least 1.5]
  OUTPUTDIR = ./../build/linux-i586
    ALT_OUTPUTDIR =
  ABS_OUTPUTDIR = /home/jars/openjdk/jdk/build/linux-i586

Build Tool Settings:
  SLASH_JAVA = /NOT-SET
    ALT_SLASH_JAVA =
  VARIANT = OPT
  JDK_DEVTOOLS_DIR = /NOT-SET/devtools
    ALT_JDK_DEVTOOLS_DIR =
  ANT_HOME = /usr/share/ant
  FINDBUGS_HOME = /home/jars/findbugs-1.3.0
  UNIXCOMMAND_PATH = /bin/
    ALT_UNIXCOMMAND_PATH =
  COMPILER_PATH = /usr/bin/
    ALT_COMPILER_PATH =
  DEVTOOLS_PATH = /usr/bin/
    ALT_DEVTOOLS_PATH =
  UNIXCCS_PATH = /usr/ccs/bin/
    ALT_UNIXCCS_PATH =
  USRBIN_PATH = /usr/bin/
    ALT_USRBIN_PATH =
  COMPILER_NAME = GCC
  COMPILER_VERSION =
  CC_VER = 4.1 [requires at least 3.2]
  ZIP_VER = 2.32 [requires at least 2.2]
  UNZIP_VER = 5.52 [requires at least 5.12]
  ANT_VER = 1.7 [requires at least 1.6.3]
  FINDBUGS_VER = 1.3 [requires at least 1.1]
  TEMPDIR = ./../build/linux-i586/tmp

Build Directives:
  OPENJDK = true
  USE_HOTSPOT_INTERPRETER_MODE =
  PEDANTIC =
  DEV_ONLY =
  NO_DOCS =
  NO_IMAGES =
  TOOLS_ONLY =
  INSANE =
  COMPILE_APPROACH = parallel
  PARALLEL_COMPILE_JOBS = 2
    ALT_PARALLEL_COMPILE_JOBS =
  FASTDEBUG =
  COMPILER_WARNINGS_FATAL = false
  COMPILER_WARNING_LEVEL =
  INCREMENTAL_BUILD = false
  CC_HIGHEST_OPT = -O3
  CC_HIGHER_OPT = -O3
  CC_LOWER_OPT = -O2
  CXXFLAGS =  -O2  -fPIC -DCC_NOEX -W -Wall  -Wno-unused
   -Wno-parentheses -fno-omit-frame-pointer -D_LITTLE_ENDIAN
  CFLAGS =  -O2    -fno-strict-aliasing -fPIC -W -Wall 
   -Wno-unused -Wno-parentheses -fno-omit-frame-pointer -D_LITTLE_ENDIAN
  BOOT_JAVA_CMD = /usr/local/java/bin/java  -client -Xmx375m -Xms128m
   -XX:PermSize=32m -XX:MaxPermSize=160m
  BOOT_JAVAC_CMD = /usr/local/java/bin/javac 
   -J-XX:ThreadStackSize=768 -J-client -J-Xmx375m -J-Xms128m
   -J-XX:PermSize=32m -J-XX:MaxPermSize=160m -encoding ascii
  BOOT_JAR_CMD = /usr/local/java/bin/jar
  BOOT_JARSIGNER_CMD = /usr/local/java/bin/jarsigner
  JAVAC_CMD = /NOT-SET/re/jdk/1.7.0/promoted/latest/binaries/linux-i586/bin/javac
   -J-XX:ThreadStackSize=768 -J-client -J-Xmx375m -J-Xms128m
   -J-XX:PermSize=32m -J-XX:MaxPermSize=160m  -source 1.5 -target 5
   -encoding ascii -Xbootclasspath:./../build/linux-i586/classes
  JAVAH_CMD = /NOT-SET/re/jdk/1.7.0/promoted/latest/binaries/linux-i586/bin/javah
   -bootclasspath ./../build/linux-i586/classes
  JAVADOC_CMD = /NOT-SET/re/jdk/1.7.0/promoted/latest/binaries/linux-i586/bin/javadoc
   -J-client -J-Xmx375m -J-Xms128m -J-XX:PermSize=32m -J-XX:MaxPermSize=160m

Build Platform Settings:
  USER = jars
  PLATFORM = linux
  ARCH = i586
  LIBARCH = i386
  ARCH_FAMILY = i586
  ARCH_DATA_MODEL = 32
  ARCHPROP = i386
  LINUX_VERSION = Unknown linux
  ALSA_VERSION = 1.0.14a
  OS_VERSION = 2.6.22-14-generic [requires at least 2.4.9-e.3]
  OS_NAME = linux
  TEMP_FREE_SPACE = 7515236
  FREE_SPACE = 7515236
  MB_OF_MEMORY = 503

GNU Make Settings:
  MAKE = make
  MAKE_VER = 3.81 [requires at least 3.78]
  MAKECMDGOALS = sanity
  MAKEFLAGS =
  SHELL = /bin/sh

Target Build Versions:
  JDK_VERSION = 1.7.0
  MILESTONE = internal
  RELEASE = 1.7.0-internal
  FULL_VERSION = 1.7.0-internal-jars_17_nov_2007_21_32-b00
  BUILD_NUMBER = b00

External File/Binary Locations:
  USRJDKINSTANCES_PATH = /opt/java
  BUILD_JDK_IMPORT_PATH = /NOT-SET/re/jdk/1.7.0/promoted/latest/binaries
    ALT_BUILD_JDK_IMPORT_PATH =
  JDK_IMPORT_PATH = /NOT-SET/re/jdk/1.7.0/promoted/latest/binaries/linux-i586
    ALT_JDK_IMPORT_PATH =
  LANGTOOLS_DIST =
    ALT_LANGTOOLS_DIST =
  CORBA_DIST =
    ALT_CORBA_DIST =
  JAXP_DIST =
    ALT_JAXP_DIST =
  JAXWS_DIST =
    ALT_JAXWS_DIST =
  HOTSPOT_DOCS_IMPORT_PATH = /NO_DOCS_DIR
    ALT_HOTSPOT_DOCS_IMPORT_PATH =
  HOTSPOT_IMPORT_PATH = /NOT-SET/re/jdk/1.7.0/promoted/latest/binaries/linux-i586
    ALT_HOTSPOT_IMPORT_PATH =
  HOTSPOT_CLIENT_PATH = /NOT-SET/re/jdk/1.7.0/promoted/latest/binaries/linux-i586/jre/lib/i386/client
    ALT_HOTSPOT_CLIENT_PATH =
  HOTSPOT_SERVER_PATH = /NOT-SET/re/jdk/1.7.0/promoted/latest/binaries/linux-i586/jre/lib/i386/server
    ALT_HOTSPOT_SERVER_PATH =
  CACERTS_FILE = ./../src/share/lib/security/cacerts
    ALT_CACERTS_FILE =
  CUPS_HEADERS_PATH = /usr/include
    ALT_CUPS_HEADERS_PATH =

OpenJDK-specific settings:
  FREETYPE_HEADERS_PATH = /usr/include
    ALT_FREETYPE_HEADERS_PATH =
  FREETYPE_LIB_PATH = /usr/lib
    ALT_FREETYPE_LIB_PATH =

OPENJDK Import Binary Plug Settings:
  BINARY_PLUGS_JARFILE = /home/jars/plugs/jre/lib/rt-closed.jar
    ALT_BINARY_PLUGS_JARFILE =
  BINARY_PLUGS_PATH = /home/jars/plugs
    ALT_BINARY_PLUGS_PATH = /home/jars/plugs
  BUILD_BINARY_PLUGS_PATH = /NOT-SET/re/jdk/1.7.0/promoted/latest/openjdk/binaryplugs
    ALT_BUILD_BINARY_PLUGS_PATH =
  PLUG_LIBRARY_NAMES =

Previous JDK Settings:
  PREVIOUS_RELEASE_PATH = /NOT-SET/re/jdk/1.6.0/archive/fcs/bundles/linux-i586
    ALT_PREVIOUS_RELEASE_PATH =
  PREVIOUS_JDK_VERSION = 1.6.0
    ALT_PREVIOUS_JDK_VERSION =
  PREVIOUS_JDK_FILE = jdk-6-linux-i586.tar.gz
    ALT_PREVIOUS_JDK_FILE =
  PREVIOUS_JRE_FILE = jre-6-linux-i586.tar.gz
    ALT_PREVIOUS_JRE_FILE =
  PREVIOUS_RELEASE_IMAGE =
    ALT_PREVIOUS_RELEASE_IMAGE =


WARNING: This machine appears to only have 503Mb of physical memory,
         builds on this machine could be slow.

WARNING: LANG has been set to en_US.UTF-8, this can cause build failures.
         Try setting LANG to "C".

WARNING: The directory HOTSPOT_IMPORT_PATH=/NOT-SET/re/jdk/1.7.0/promoted/latest/binaries/linux-i586
       does not exist, check your value of ALT_HOTSPOT_IMPORT_PATH.

WARNING: HOTSPOT_IMPORT_PATH does not contain the interface file jvmti.h.
         Check your value of ALT_HOTSPOT_IMPORT_PATH.

WARNING: File /NOT-SET/re/jdk/1.7.0/promoted/latest/binaries/linux-i586/lib/sa-jdi.jar
         does not exist. The JDI binding for the Serviceability Agent will not be
         included in the build.
         Please check your access to
           /NOT-SET/re/jdk/1.7.0/promoted/latest/binaries/linux-i586/lib/sa-jdi.jar
         and/or check your value of ALT_HOTSPOT_IMPORT_PATH.

ERROR: You do not have access to valid Cups header files.
       Please check your access to
           /usr/include/cups/cups.h
       and/or check your value of ALT_CUPS_HEADERS_PATH,
       CUPS is frequently pre-installed on many systems,
       or may be downloaded from http://www.cups.org

ERROR: HOTSPOT_CLIENT_PATH does not point to a valid HotSpot VM.
       Please check your access to
           /NOT-SET/re/jdk/1.7.0/promoted/latest/binaries/linux-i586/jre/lib/i386/client/libjvm.so
       and/or check your value of ALT_HOTSPOT_CLIENT_PATH.

ERROR: HOTSPOT_SERVER_PATH does not point to a valid HotSpot VM.
       Please check your access to
           /NOT-SET/re/jdk/1.7.0/promoted/latest/binaries/linux-i586/jre/lib/i386/server/libjvm.so
       and/or check your value of ALT_HOTSPOT_SERVER_PATH.

Exiting because of the above error(s).

Continuing with the actual error messages in order, the next
problem seemed to be that Ubuntu sets the LANG
environment variable to en_US.UTF-8 by default, and the build
script wants it to be C. Ubuntu is right here. In 2007, "http://www-128.ibm.com/developerworks/xml/library/x-utf8/">UTF-8
is the only default encoding anyone should use for anything
.
Nonetheless let's fix that and move on:

$ export LANG=C

TODO: Figure out why the build script insists on C as the LANG.
I suspect two fundamentally different properties (natural language
and programming language) are colliding over the same environment
variable name here.

The next error was:

HOTSPOT_IMPORT_PATH=/NOT-SET/re/jdk/1.7.0/promoted/latest/binaries/linux-i586
does not exist, check your value of
ALT_HOTSPOT_IMPORT_PATH.

What the heck is that supposed to be? The "https://openjdk.dev.java.net/source/browse/*checkout*/openjdk/jdk/trunk/README-builds.html">
official build instructions
don't mention it. It looks like
it's actually supposed to be some Java 7 thing, not even part of
the bootstrap JDK. Maybe I have to build HotSpot before I can build
the JDK?

After another 15 minutes of reading various blogs, I discovered
that the build instructions Sun publishes are wrong (big surprise).
They talk about a "top-level Makefile" that doesn't yet exist.
Instead I have to run the makefile in the
control/make directory and that's supposed to build
everything: openjdk, hotspot, everything. Let's try that:

$ make sanity
make[1]: Entering directory `/home/jars/openjdk/jdk/make'
make[2]: Entering directory `/home/jars/openjdk/jdk/make/tools/freetypecheck'
freetypecheck.c: In function 'main':
freetypecheck.c:45: warning: comparison is always false
due to limited range of data type
freetypecheck.c:54: warning: comparison is always false
due to limited range of data type
make[2]: Leaving directory `/home/jars/openjdk/jdk/make/tools/freetypecheck'
make[1]: Leaving directory `/home/jars/openjdk/jdk/make'

Damn, I thought I'd fixed the freetype problems. But looking
closer, it seems these messages are just about a problem in the
freetypecheck C program Sun includes with the build, not with
freetype itself. The relevant lines are:


   if (strcmp(v, QUOTEMACRO(REQUIRED_FREETYPE_VERSION)) < 0) {
       printf("Failed: headers are too old.\n");
   }

and

   if (strcmp(v, QUOTEMACRO(REQUIRED_FREETYPE_VERSION)) < 0) {
      printf("Failed: too old library.\n");
   }

It looks like if this test always fails, freetype is good, so
let's just ignore that one.

TODO: Figure out what's going on here and fix it.

Now I'm down to just one warning and one error:

WARNING: This machine appears to only have 503Mb of physical memory, 
         builds on this machine could be slow.

ERROR: You do not have access to valid Cups header files.
       Please check your access to
           /usr/include/cups/cups.h
       and/or check your value of ALT_CUPS_HEADERS_PATH,
       CUPS is frequently pre-installed on many systems,
       or may be downloaded from http://www.cups.org

I've ordered four gigabytes of RAM for this laptop, but they
haven't arrived yet. In the meantime, I'll just live with the slow
build. However, CUPS could be a problem. Back to Synaptic. Looks
like another dev install issue. It seems to be
libcupsys2-dev that I need. Let's install that and try
again:

$ make sanity
make[1]: Entering directory `/home/jars/openjdk/jdk/make'
make[2]: Entering directory `/home/jars/openjdk/jdk/make/tools/freetypecheck'
make[2]: Nothing to be done for `all'.
make[2]: Leaving directory `/home/jars/openjdk/jdk/make/tools/freetypecheck'
make[1]: Leaving directory `/home/jars/openjdk/jdk/make'

Build Machine Information:
   build machine = jars-desktop
...
Previous JDK Settings:
  PREVIOUS_RELEASE_PATH = /NOT-SET/re/jdk/1.6.0/archive/fcs/bundles/linux-i586
    ALT_PREVIOUS_RELEASE_PATH =
  PREVIOUS_JDK_VERSION = 1.6.0
    ALT_PREVIOUS_JDK_VERSION =
  PREVIOUS_JDK_FILE = jdk--linux-i586.tar.gz
    ALT_PREVIOUS_JDK_FILE =
  PREVIOUS_JRE_FILE = jre--linux-i586.tar.gz
    ALT_PREVIOUS_JRE_FILE =
  PREVIOUS_RELEASE_IMAGE =
    ALT_PREVIOUS_RELEASE_IMAGE =


WARNING: This machine appears to only have 503Mb of physical memory,
         builds on this machine could be slow.

Sanity check passed.

Finally! About seven hours after I started, but the sanity check
passes. Now to try to actually build it with make:

Making

$ make
linux i586 1.7.0-internal build started: 07-11-17 22:33
/bin/mkdir -p ../../control/build/linux-i586/j2sdk-image
/bin/mkdir -p /home/jars/openjdk/control/build/linux-i586/j2sdk-image
...
# Running javac:
Check_ALT_JDK_IMPORT_PATH/bin/javac -J-XX:ThreadStackSize=768
  -J-client -J-Xmx375m -J-Xms128m -J-XX:PermSize=32m
  -J-XX:MaxPermSize=160m -source 1.5 -target 5 -encoding ascii
  -classpath /usr/local/java/lib/tools.jar -sourcepath  
/home/jars/openjdk/control/build/linux-i586/corba/gensrc:
../../../src/solaris/classes:
../../../src/share/classes -d
/home/jars/openjdk/control/build/linux-i586/corba/classes
@/home/jars/openjdk/control/build/linux-i586/corba/tmp/sun
/javax.transaction.xa/.classes.list
/bin/sh: Check_ALT_JDK_IMPORT_PATH/bin/javac: not found
make[3]: *** [.compile.classlist] Error 127
make[3]: Leaving directory `/home/jars/openjdk/corba/make/javax/xa'
make[2]: *** [build] Error 1
make[2]: Leaving directory `/home/jars/openjdk/corba/make/javax'
make[1]: *** [build] Error 1
make[1]: Leaving directory `/home/jars/openjdk/corba/make'
make: *** [corba-build] Error 2

Hmm, looks like it needs an ALT_JDK_IMPORT_PATH
environment variable. The sanity check didn't catch this. It seems
to be trying to load javac from there. I don't know why
JAVA_HOME isn't good enough, but let's try just
setting it to my regular JDK directory:

$ export ALT_JDK_IMPORT_PATH=/usr/local/java

Now the build seems to be moving. There are a lot of error
messages of various types, like this one:

../../../../../../../src/share/classes/org/omg/CORBA/ORB.java:593: warning:
non-varargs call of varargs method with inexact argument type for last parameter;
cast to java.lang.Object for a varargs call
cast to java.lang.Object[] for a non-varargs call and to suppress this warning
            return (org.omg.CORBA.NVList)meth.invoke(this, argx);

It seems not all the code in the JDK is up to snuff with the
latest Sun coding specs. However, these are just warnings. The code
still builds.

TODO: Fix this to get a warning-free build.

The fan/hard drive on the build machine has now spun up, and is
making whirring noises that are audible from across the room. This
is only a laptop. I hope it doesn't burn out. At least I'm still
under the 90-day warranty if it does. OK. It's quieted down a bit.
Whew. Oh, wait a minute. It quieted down because it errored
out:

/home/jars/openjdk/hotspot/agent/src/share/classes/sun/jvm
/hotspot/debugger/remote/x86/RemoteX86Thread.java:42:
cannot find symbol
symbol  : class RemoteX86ThreadContext
location: class sun.jvm.hotspot.debugger.remote.x86.RemoteX86Thread
    RemoteX86ThreadContext context = new RemoteX86ThreadContext(debugger);
    ^
/home/jars/openjdk/hotspot/agent/src/share/classes/sun/jvm/hotspot
/debugger/remote/x86/RemoteX86Thread.java:42: cannot find symbol
symbol  : class RemoteX86ThreadContext
location: class sun.jvm.hotspot.debugger.remote.x86.RemoteX86Thread
    RemoteX86ThreadContext context = new RemoteX86ThreadContext(debugger);
                                         ^
Note: /home/jars/openjdk/hotspot/agent/src/share/classes/sun/jvm/hotspot/jdi/SAJDIClassLoader.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
2 errors
make[6]: *** [/home/jars/openjdk/control/build/linux-i586/hotspot/outputdir
/linux_i486_compiler2/product/../generated/sa-jdi.jar] Error 1
make[6]: Leaving directory
`/home/jars/openjdk/control/build/linux-i586/hotspot/outputdir
/linux_i486_compiler2/product'
make[5]: *** [all] Error 2
make[5]: Leaving directory
`/home/jars/openjdk/control/build/linux-i586/hotspot/outputdir
/linux_i486_compiler2/product'
make[4]: *** [sa_stuff] Error 2
make[4]: Leaving directory
`/home/jars/openjdk/control/build/linux-i586/hotspot/outputdir
/linux_i486_compiler2/product'
make[3]: *** [product] Error 2
make[3]: Leaving directory
`/home/jars/openjdk/control/build/linux-i586/hotspot/outputdir'
make[2]: *** [generic_build2] Error 2
make[2]: Leaving directory `/home/jars/openjdk/hotspot/make'
make[1]: *** [product] Error 2
make[1]: Leaving directory `/home/jars/openjdk/hotspot/make'
make: *** [hotspot-build] Error 2

Still, this is progress. This at least looks like a genuine Java
bug, rather than an environmental problem or a C bug. The specific
bug seems to be that there is no accessible
RemoteThreadContext constructor that takes a
RemoteDebuggerClient as an argument. Looking in
RemoteX86Thread.java to see what may be going on, I don't see
any problem, but when I open RemoteThreadContext.java the
problem is obvious: the file is empty! I wonder what happened to
it.

Perhaps something hiccuped when I unzipped or downloaded the
original file. I grab a new copy, and it does seem to contain
RemoteThreadContext.java, so I copy it over to where it
should be and make again. This time I manage to play an entire
round in Arathi
Basin
before it dies:

 Timing: 00000 seconds or 0s for make-java-jvm
<<<Finished Recursively making jvm all @ Sat Nov 17 23:54:14 CET 2007.
>>>Recursively making redist all @ Sat Nov 17 23:54:14 CET 2007 ...
make[3]: Entering directory `/home/jars/openjdk/jdk/make/java/redist'
BinaryPlugs import started: Sat Nov 17 23:54:14 CET 2007
BINARY_PLUGS_PATH=/home/jars/plugs
make[3]: *** No rule to make target `/home/jars/plugs/jre/lib/i386/libjsoundhs.so', needed by `/home/jars/openjdk/control/build/linux-i586/lib/i386/libjsoundhs.so'.  Stop.
make[3]: Leaving directory `/home/jars/openjdk/jdk/make/java/redist'
make[2]: *** [all] Error 1
make[2]: Leaving directory `/home/jars/openjdk/jdk/make/java'
make[1]: *** [all] Error 1
make[1]: Leaving directory `/home/jars/openjdk/jdk/make'
make: *** [jdk-build] Error 2

Maybe I didn't install the plug correctly? Maybe I have to unjar
it? Nope. That doesn't do the trick. Wait: it's a self-running
JAR:

$ java -jar jdk-7-ea-plug-b23-linux-i586-30_oct_2007.jar
Error: Install failed: java.awt.HeadlessException:
No X11 DISPLAY variable was set, but this program performed
an operation which requires it.
java.awt.HeadlessException:
No X11 DISPLAY variable was set, but this program performed
an operation which requires it.
   at java.awt.GraphicsEnvironment.checkHeadless(GraphicsEnvironment.java:159)
...

Hmm, guess I can't run it in the console. I'll have to wheel my
chair over to the laptop and run it there.

TODO: Make the binary plug installer able to run headless.

OK. Now I've got the uncompressed binary plugs in
/home/jars/plugs/openjdk-binary-plugs. I need to update the
environment variable to match and make again:

$ export ALT_BINARY_PLUGS_PATH=/home/jars/openjdk-binary-plugs
$ make
Make runs again, but soon dies in a different location:

In file included from
/home/jars/openjdk/jdk/src/share/native/sun/awt/../java2d/pipe/Region.h:34,
  from /home/jars/openjdk/jdk/src/share/native/sun/awt/../java2d/pipe/Region.c:30:
/home/jars/openjdk/jdk/src/solaris/native/sun/awt/utility/rect.h:31:22: error:
X11/Xlib.h: No such file or directory
In file included from
...
/home/jars/openjdk/jdk/src/solaris/native/sun/awt/img_util_md.h:32:
  error: expected specifier-qualifier-list before 'XID'
/home/jars/openjdk/jdk/src/share/native/sun/awt/image/BufImgSurfaceData.c:
  In function 'Java_sun_awt_image_BufImgSurfaceData_freeNativeICMData':
/home/jars/openjdk/jdk/src/share/native/sun/awt/image/BufImgSurfaceData.c:95:
  warning: cast to pointer from integer of different size
make[4]: ***
[/home/jars/openjdk/control/build/linux-i586/tmp/sun/sun.awt/awt/obj/BufImgSurfaceData.o]
Error 1
make[4]: Leaving directory `/home/jars/openjdk/jdk/make/sun/awt'
make[3]: *** [library_parallel_compile] Error 2
make[3]: Leaving directory `/home/jars/openjdk/jdk/make/sun/awt'
make[2]: *** [all] Error 1
make[2]: Leaving directory `/home/jars/openjdk/jdk/make/sun'
make[1]: *** [all] Error 1
make[1]: Leaving directory `/home/jars/openjdk/jdk/make'
make: *** [jdk-build] Error 2

Maybe I'm missing some X11 development libraries? Back to
Synaptic. Let's install libx11-dev and try again.
Nope, that didn't do it. A little Googling and it looks like
libxt-dev is the missing piece. This time I get a
little bit further. Now a different file is missing:

/home/jars/openjdk/jdk/src/solaris/native/sun/awt/splashscreen/splashscreen_config.h:33:34:
error: X11/extensions/shape.h: No such file or
directory.

This time I go straight to Google and discover that this
"include file is part of the Nonrectangular Window Shape Extension
standard." It seems libext-dev is the package I need.
Once more into the breach.

I think this may be the last one. I seem to have actually
compiled everything. Of course, this being C and not Java, that
doesn't mean I'm done. Now there are linker errors:

/usr/bin/ld: cannot find -lXtst

So I install the libxtst-dev headers.

This is getting boring. Let me just list the other libraries I
had to install:

  • libXi-dev

Hmm, that may have been it. It seems to be generating JavaDoc
now. There are a lot of broken but easy-to-fix JavaDoc tags, but I
think it's actually done. Now if I can only figure out where the
build put everything. :-)

Installing

The output seems to be in
openjdk/control/build/linux-i586/j2sdk-image. (Some other
build products like a JRE with no dev tools are also in
openjdk/control/build/linux-i586/.) Let's try copying that
to /opt/java, setting it as JAVA_HOME, and
adding it to the path:

$ sudo cp -R j2sdk-image /opt/java
$ export JAVA_HOME=/opt/java
$ export PATH=/opt/java/bin:$PATH

And now the moment of truth:

$ java -version
openjdk version "1.7.0-internal"
OpenJDK Runtime Environment (build 1.7.0-internal-jars_18_nov_2007_01_03-b00)
OpenJDK Client VM (build 12.0-b01, mixed mode)
$ javac -version
javac 1.7.0-internal

Success! It's 12:32 in the morning and I started around 10:00
(A.M., not P.M.); but it's done. Now that all the right libraries
are installed maybe the next install will only take seven hours.

Easier Paths

Raw builds with make are essential for automation,
testing, porting, and continuous integration. However, they aren't
always the easiest way to do day-to-day development. If this just
seems too fraught for you, there are alternatives, though these are
not without similar issues of their own.

Prebuilt Binaries

If debugging makefiles isn't your idea of a good way to spend a
Saturday, then you may just want to install a prebuilt binary from
the JDK 7 binary
snapshots page
instead.

IDEs

I've focused on building from the command line because that's
the most generic and interoperable approach. It also makes
automation and testing a lot easier. Command-line builds should be
supported by all good open source software. That said, sometimes an
IDE can help. Sun actively encourages the use of NetBeans to build
and hack on the JDK, and the openjdk downloads come with
preconfigured NetBeans projects in
openjdk/jdk/make/netbeans. Just open them up in NetBeans and
go. More instructions for that are on the "http://nb-openjdk.netbeans.org/get-and-build.html">NetBeans website. Just be warned that those instructions aren't completely
accurate, either, and you'll likely still need to do some debugging
to get it all to build, even in NetBeans.

Summing Up

What have we learned? It is possible to build the JDK.
It just takes a day or two the first time you do it, and
familiarity with Unix and C libraries doesn't hurt. Hopefully it
will take you less time than it took me. Starting from a stock
Ubuntu desktop configuration, here's what you need to do:

  1. Install the latest JDK 6 from Sun. (Ubuntu only ships with a
    JRE.)
  2. Download the source code bundle from the "http://download.java.net/openjdk/jdk7/">OpenJDK Source Releases
    page
    and unzip it to create the openjdk directory.
  3. Download the binary plugs JAR from the same page and run
    it
    to create the openjdk-binary-plugs directory.
  4. Install these packages:
    • gawk
    • ant
    • findbugs
    • ALSA
    • libcupsys2-dev
    • libext-dev
    • libXi-dev
    • libxt-dev
    • libxtst-dev
  5. Manually install FindBugs
  6. Set the following environment variables:
    • ALT_BOOTDIR= where JDK 6 is installed
    • ALT_BINARY_PLUGS_PATH= wherever you installed the binary
      plugs
    • ANT_HOME= wherever you installed Ant
    • FINDBUGS_HOME= wherever you installed FindBugs
    • ALT_JDK_IMPORT_PATH= where JDK 6 is installed
    • LANG=C
  7. Make openjdk/jdk/make/jdk_generic_profile.sh executable
    and run it.
  8. Move into openjdk/control/make and type
    make
    sanity
    . Debug any problems that arise.
  9. Once the sanity checks pass, type make. Go get a cup
    of coffee. This will take a while.
  10. Once the build is complete, copy the directory
    openjdk/control/build/linux-i586/j2sdk-image to wherever you
    want to put your JDK. The directory /opt/java7 may be a nice choice.

On other Linux distros, you may discover you need to add a few
more libraries that Ubuntu bundles by default, or you may not need
to add all of the ones listed here. On other operating systems such
as Solaris, you may need to specify gnumake instead of
make (gnumake is the default
make on Ubuntu). On Windows, good luck. I'm sure it
can be done, but I don't have another full day to spend debugging
that one. Maybe in another article.


width="1" height="1" border="0" alt=" " />
Elliotte Rusty Harold is the author of numerous books including Java I/O, Java Network Programming, and the upcoming Refactoring HTML.
Related Topics >> Community   |   Linux   |   Research   |