Skip to main content

The Open Road: Superpackages

March 6, 2008

{cs.r.title}







In this installment of "http://today.java.net/pub/ct/10">The Open Road, Elliotte Rusty
Harold looks at how JSR 294 superpackages, intended for inclusion
in Java SE 7, will allow for the creation of hierarchical packages,
allowing you to organize your code more cleanly and
effectively.

First though, here's an update on the status of the OpenJDK
project. The latest JDK 7 release is "http://download.java.net/openjdk/jdk7/promoted/b24/">b24,
released December 4, 2007, which is no different from "http://download.java.net/openjdk/jdk7/promoted/b23/">b23,
except that b24 was built from Mercurial sources.

There are now several significant OpenJDK projects,
including the "http://openjdk.java.net/projects/jdk6/">OpenJDK 6 project, a
backport of the evolving GPL JDK 7 codebase to the JDK 6 spec. The
OpenJDK 6 project just released "http://download.java.net/openjdk/jdk6/promoted/b06/">b06. As
described in "http://blogs.sun.com/darcy/entry/openjdk_6_sources_for_b06">Joe
Darcy's blog
, imaging classes have "http://mail.openjdk.java.net/pipermail/discuss/2008-February/001078.html">
been moved from closed to open
, JAX-WS 2.1 "http://bugs.sun.com/view_bug.do?bug_id=6605798">has been
included
, and the SNMP portion of the build "http://bugs.sun.com/view_bug.do?bug_id=6661448">has been modified
to not fail when binary plugs are absent
. Several other
improvements are included.

Now, on to JDK 7 and our first feature preview:
superpackages. Here's Elliotte.

Since Java 1.0, packages have offered a convenient means of
organizing code into different namespaces at the source code level.
Classes in the same package can see non-public, non-protected,
and non-private methods and fields in other classes in the same package
that classes in other packages cannot see.

Unfortunately, there's no standard adjective for members that
have no explicit access specifier. The most common, though by no
means standard, name for these fields and methods seems to be
"default" access. They're also sometimes referred to as "package
protected" or "package private." The Java Language Specification
uses "default (package)," and that is what I'll use in this
article.

How Packages Affect Visibility

For example, consider the three classes in Examples 1, 2, and 3.
A cat can make a dog bark and a dog can make a cat meow, but
neither a cat nor a dog can get the bacon out of the refrigerator.
The refrigerator can make a dog salivate, but it can't make either
animal speak. This is because Cat and Dog
are in the same package as each other but different packages than
Refrigerator, while the speak() and
getBacon() methods have default (package) access.

Example 1. A Dog class in the
com.elharo.animals package
[prettify]package com.elharo.animals;

public class Dog {

    void speak() {
       System.out.println("bark"); 
    }

    public void salivate() {
        System.out.println("drool");
    }
    
}
[/prettify]
Example 2. A Cat class also in the
com.elharo.animals package
[prettify]package com.elharo.animals;

public class Cat {

    void speak() {
       System.out.println("meow"); 
       
    }
    
}
[/prettify]
Example 3. A class in a new package
[prettify]package org.cafeaulait.kitchen;

public class Refrigerator {

    String getBacon() {
      return "bacon";   
    }
    
}
[/prettify]

Hierarchical Names, Flat Namespaces

The package naming convention is hierarchical, but the namespace
division isn't. The package namespace structure is flat. A class in
java.util.zip has no more access to the internals of
java.util.HashMap than does a class in
org.apache.xerces. Even if the
Refrigerator class had been in the
com.elharo.animals.appliances subpackage, it still
wouldn't have been able to make the dog bark or the cat meow.
Despite the apparent hierarchy of the packages, there's no actual
hierarchy in the access protection.

Ninety percent or more of the time, this doesn't matter all that much, but
occasionally there are problems. For example, the "http://www.jdom.org/">JDOM XML API put its core model classes
like Element and Attribute in
org.jdom and its parser classes in
org.jdom.input. This meant the builder classes that
read XML documents and created objects from them could only use the
model classes' public API. However, this API duplicated checks the
underlying XML parser had already performed, thus doubling the
workload to build the in-memory representation of a document. The
core model classes could have provided special, non-verifying
methods that did not duplicate the work. However, because the
builder classes were in separate packages, these methods would have
to be public. Then anyone could use them to bypass the
checks, not just the known safe code from the input package.

Another common problem arises when writing unit tests. Test code
often needs to see parts of a class that the general public isn't
allowed to access. Sometimes test classes want to directly test
non-public methods. Other times they want to inject dependencies.
Usually you'd think test code belongs in a separate package.
However, many programmers put their tests in the same package as
the tested code precisely because they want to be able to access
the non-public parts. Alternatively, some programmers make methods
public just so they can be tested, even if that
pollutes the published API. Neither approach feels especially
palatable.

Workarounds

There are two common solutions to these problems in Java 6 and
earlier. The first is to make methods public that
really shouldn't be, and then document them as "non-published" and
"for internal use only." Of course, we know how much programmers
love to read documentation, and no developer anywhere would ever
use a method marked internal, and now that I've moved out of
Brooklyn, I've got a bridge for sale at the end of Flatbush Avenue,
cheap!

The other alternative is to place all related classes in the
same package. That's a little safer, and it's usually what I do--for instance, in my JDOM alternative library "http://www.xom.nu/">XOM, the input and output classes are in
the same package as the core classes precisely so they can see each
others' internal parts--but this loses the benefits of organizing
large code bases hierarchically.

Look! Up in the sky! It's a Bird! It's a Plane! No, It's
Superpackage!

JSR 294
proposes to improve the situation in Java 7 by introducing
superpackages. A superpackage is a new construct that,
like an interface or a class, has its own .java file. This
.java file lives in the usual file system hierarchy and is
always named super-package.java. For example, a superpackage
for com.elharo.pets would be defined by the file named
super-package.java in a com/elharo/pets directory.
This file would contain a list of all the members of the
superpackages and exported classes, as shown in Example 4.

Example 4. A superpackage for com.elharo.pets
[prettify]superpackage com.elharo.pets {

    // member packages
    member package com.elharo.pets;

    // member superpackages
    member superpackage com.elharo.pets.avian, com.elharo.pets.aquatic;

    // exported classes
    export com.elharo.pets.Dog;

   // exported superpackages
    export superpackage com.elharo.pets.aquatic;

}
[/prettify]

Once the superpackage is created, only exported classes are
visible outside the superpackage. For example, the
com.elharo.pets.Dog class is visible, but the
com.elharo.pets.Cat class is not. Classes in the
com.elharo.pets.aquatic package are visible, but
classes in com.elharo.pets.avian are not.

Inside the superpackage, everything is the same as before.
However, if a class is placed in a superpackage and is accessed
from outside the superpackage, everything changes. By default, all
access from outside the superpackage is blocked. It's as if all the
classes inside the superpackage had suddenly become private. The
superpackage barrier forms an event horizon from which information
can only escape if it's explicitly exported.

Note the difference between how packages are created and how
superpackages are created: packages are created internally when
classes declare themselves to be members of those packages.
Superpackages are created externally in a separate file.

Exporting and Hiding Methods

What has this bought us? Remember the issue is usually that you
have classes and methods you want to expose to other members of the
superpackage but not to the general public. For classes, the
problem is solved: make them public but don't export
them from the superpackage. For methods, the answer is a little
trickier and requires an additional level of indirection. In brief,
you have to tag the methods you want to publish within the
superpackage with default (package) access, just like you do now.
Then you define a new accessor class in the same package
that delegates to the default (package) methods. You make the
methods in this class public, but you don't export the
class from the superpackage.

For example, let's see how we'd use this to provide a test class
the ability to inject a set of mock system properties into a class.
Begin by assuming we have a class set up as in Example 5.

Example 5. A class that loads the system properties
[prettify]package com.elharo.examples;

import java.util.Properties;

public class Stubborn {

    private Properties props = System.getProperties();
    
    // rest of class...
    
    
}
[/prettify]

You want to test the stubborn class with various values for the
different system properties. However, that's going to be hard to do
because some of these properties, such as java.version
and java.vendor, are fixed by the virtual machine you
test with. They are not under your direct control.

To allow the test case in the
com.elharo.examples.test package to change the system
properties, you have to add an additional setter method for the
properties as shown in Example 6.

Example 6. A class that allows the system properties to be
injected
[prettify]package com.elharo.examples;

import java.util.Properties;

public class Stubborn {

    private Properties props = System.getProperties();
    
    // public dependency injection method
    public void setProperties(Properties props) {
        this.props = props;
    }
    
    // rest of class...
    
}
[/prettify]

Now the test code can inject the test properties directly into
the class. Unfortunately, so can everyone else, and that can be both
a security risk and a source of bugs. It also makes the API
needlessly complex and larger than it should be.

Instead, make the setProperties() method default
(package) as shown in Example 7.

Example 7. A class that allows the system properties to be
injected
[prettify]package com.elharo.examples;

import java.util.Properties;

public class Stubborn {

    private Properties props = System.getProperties();
    
    // no longer public
    void setProperties(Properties props) {
        this.props = props;
    }

    // rest of class...
    
    
}
[/prettify]

Then add a new public class in the same package that can see the
non-public setProperties() method, as shown in Example
8. The test class in the subpackage
com.elharo.examples.test can use this class to create
a Stubborn object with custom configured properties
rather than invoking the constructor directly. Note that this is
not the only way you could arrange this; there are several other
possibilities, but this seemed simplest.

Example 9. A class that allows the system properties to be
injected
[prettify]package com.elharo.examples;

import java.util.Properties;

public class StubbornBuilder {

    public static Stubborn build(Properties props) {
        Stubborn s = new Stubborn();
        s.setProperties(props);
        return s;
    }
    
}
[/prettify]

Finally, define the superpackage for
com.elharo.examples as shown in Example 9. Notice that
com.elharo.examples.test is a member, but is not
exported. StubbornBuilder has not been exported
either. This means that to all the classes outside the
superpackage, they effectively don't exist. Presumably, there won't
even be JavaDoc for these classes.

Example 9. A superpackage for com.elharo.examples
[prettify]superpackage com.elharo.examples {

    // member packages
    member package com.elharo.examples;

    // member superpackages
    member superpackage com.elharo.examples.test;

    // exported classes
    export com.elharo.examples.Stubborn;

}
[/prettify]

Voila! We now have a public API that exists only in the
superpackage. The compiler and the virtual machine will enforce
this. This does mean that neither source nor byte code that uses
these features will be processable with Java 6 and earlier tools.
However there are enough other changes coming in Java 7 that this
was likely to be true with or without superpackages.

Summing Up

Superpackages are one of the better ideas being floated for Java
7. They address a real pain point for many developers. However, they
don't introduce a lot of confusing new syntax, and if you don't
need them you can fairly safely ignore them.

The one change I'd really like to see is better support for
working at the method level rather than the class level. I'd like
to be able to tag individual methods as exported or not, rather
than working with entire classes at a time. The spec is still in
early draft review, so time remains for changes to be made and the
syntax to be improved, and I'm told a new access specifier to
accomplish this is indeed under consideration for the next
draft.

Nonetheless, even in its current imperfect state, superpackages
are a significant addition to the Java programmer's toolbox.

Resources

  • JSR 294:
    Improved Modularity Support in the Java Programming Language
  • The Open Road
    column on java.net


width="1" height="1" border="0" alt=" " />
Elliotte Rusty Harold is the author of numerous books including Java I/O, Java Network Programming, and the upcoming Refactoring HTML.
Related Topics >> JSR   |   Programming   |