Improve on Javadocs with ashkelon
Sun's Javadocs provide us with automatically generated, hyperlinked documentation of fields, methods, classes, interfaces, and inner classes. They are helpful in many ways, but you will soon develop a wish list that includes the ability to search for programming elements, the ability to access multiple APIs from one location, the ability to cross-reference relationships across APIs. The list goes on.
This article describes
ashkelon, an open source documentation system for Java that leverages Sun's Javadoc parser.
When developing a UI for, say, 50 APIs, 500 packages, 10,000 classes, and 100,000 methods, how do you present mounds of information in a readily accessible and intelligible way?
Patrick Chan's solution was to produce the Java Almanac, containing indices and cross-references to programming elements. But there are drawbacks to paper forms of referencing; for one thing, the Java Almanac covers only J2SE. And it's long and heavy.
The handful of techniques used repeatedly in
ashkelon's UI to address this problem are:
Hide information on the page using CSS
visibilityproperties, and toggle those properties dynamically (using various DHTML client-side "widgets" such as tabs, collapsible trees, and filtered tables).
Color code, for more natural filtering of information on the page.
Provide many ways to access the same information: via an index, browsing or searching, temporal navigation back to an element recently previewed, via a cross-reference, or another type of programmatic relationship such as a throws relationship.
A Database of Docs
One component of
ashkelon plugs in to Javadoc as a doclet to populate a relational database with Javadocs. The corresponding database schema for the Java language consists of tables with names such as
thrownexception. The schema can be updated and grow to include changes such as those introduced with J2SE 5.0 and aspects.
So what can you do with a database schema filled with Javadocs? For one thing, with
ashkelon you can populate multiple APIs into a single repository. You can do this incrementally, too. That is, you don't have to add all APIs in one run to "get your links." Let's look at how we might use
ashkelon to populate an API such as Hibernate into the database. Invoke the following command from the command line, in much the same way as you would use the Javadoc tool to produce HTML documentation:
~$ ashkelon add @hibernate.xml
To add another API (say
dom4j), you do this:
~$ ashkelon add @dom4j.xml
Now we can ask
ashkelon to list the APIs in its database:
~$ ashkelon list
which, with my local copy of
ashkelon running on my notebook computer today, produces:
Ashkelon: API: Ashkelon
Ashkelon: API: Hibernate
Ashkelon: API: J2SE
Ashkelon: API: JTidy
Ashkelon: API: JUnit
Ashkelon: API: JiBX
Ashkelon: API: Servlet 2.3 & JSP 1.2
Ashkelon: API: dom4j
I happen to be running my local copy of
ashkelon against PostgreSQL. The jdocs.org site chose to run
ashkelon with MySQL. (I've also recently added support for
mckoidb, a lightweight database written in Java.)
ashkelon's design decision to use a database gives it intimate knowledge of the code. It enables it to answer intelligent questions, such as: "list all methods that return objects of a certain type," "list all classes that implement a certain interface," "list all direct subclasses of a certain class," and so on.
More Than a Collection of Loosely Connected APIs
ashkelonmakes a major effort to truly integrate the multiple APIs that reside in its repository.
Each time you add an API, a cross-referencing step takes place after the "store" step to update inter-API references between programming elements. Here's an example. Assume you already have the W3C's DOM API populated and then decide to add Hibernate. Look up this Hibernate method:
public org.w3c.dom.Document toGenericDOM()
ashkelon's web-based UI, you'd find that the return type (
org.w3c.dom.Document) is hyperlinked.
Inter-API integration can also be seen in the cross-reference section of
ashkelon's UI. All classes and interfaces in the
ashkelon system have a cross-reference tab, shown in Figure 1.
Figure 1. GUI view of cross-linking
For example, you can ask
ashkelon to list all methods that return a
org.w3c.dom.Document. The results, shown in Figure 2, include methods from various APIs:
org.w3c.tidy, and of course,
Figure 2. Search for occurrences of a class
A third place where this integration is evident is in search results. A class search for "*Writer" will produce a listing that includes these classes or interfaces:
A Formal Definition for an API
Digging a little deeper, let's return to that
add command. We had to feed it an XML file: hibernate.xml or dom4j.xml. What's that all about?
With the original Javadoc tool, you typically feed it the name of a
file that contains the listing of packages that you want to document.
Being a multi-API repository,
ashkelon needs to invent one more level
up the containment hierarchy: the API.
These XML files are nothing more than information about the project you want to document. Those of you familiar with Maven are also familiar with its definition of a project XML file, known as the Project Object Model (POM). As of version 0.8.7,
ashkelon understands the Maven POM file format and can use these project.xml files as input for populating APIs into its database repository. Here's how one populates
dom4j's API using its project.xml file:
~$ ashkelon add @project.xml
This makes the population of APIs trivial and effortless for projects that use Maven. Alternatively,
ashkelon defines its own XML file format for APIs. Below are the contents of dom4j.xml.
<?xml version="1.0" ?>
<summarydescription>The flexible XML framework for Java</summarydescription>
<description>dom4j is an easy to use, open source
library for working with XML, XPath and XSLT on the Java
platform using the Java Collections Framework and with
full support for DOM, SAX and JAXP.</description>
There's no need to write this XML file by hand. For one thing,
ashkelon comes "out of the box" with approximately 32 pre-written API XML files for various popular open source projects including Ant, J2SE, Hibernate,
dom4j, and many more. Secondly,
ashkelon comes with a script, called apixml, that will automatically produce the XML file, directly from a Javadoc package-list file:
~/devel/dom4j-1.4/doc/javadoc $ apixml package-list
which produces this output (to
<?xml version="1.0" ?>
Finally, fill in the missing pieces (API name, description, publisher, etc.), which you can look up on the project's home page.
A Different User Interface?
There have been two user interfaces for Javadocs. The first was
released with JDK1.0 and used images (then in fashion) for page section
headers, such as "Classes" and "Interfaces." The veterans among you will
recall the convex yellow underlines adorning these titles. The second
UI was released with JDK1.1 seven years ago. That is, by large, the
same UI we use today.
ashkelon's DHTML user interface to address a number of usability issues that personally impeded my productivity as a Java developer.
Problem 1: Too much scrolling
To get from the top of
to the bottom, you must "page down" 117 times. Contrast this with
ashkelon's UI, which uses tabbed panes to lay out the information
horizontally across the page, thus minimizing scrolling. Furthermore,
ashkelonscrolls only those blocks of information that need to scroll
(not the entire page), so you don't lose sight of header, footer, and
sidebar sections that are the means of navigating to other pages in the
Figure 3. Compact layout
Problem 2: Monochromatic The class listing in Figure 4
below uses color and style to indicate whether a list item is an interface, an
exception, an abstract class, or a concrete class. The mind can digest
style and color information much faster than it can distinguish among
Figure 4. Color cues
Similar cues within a class method listing, as shown in Figure 5,
help you identify static methods (in bold), deprecated methods (with
strike-through), and abstract methods (italicized). How many times have
you made use of a deprecated method, only to discover after
implementation that you didn't read the Javadocs carefully enough?
Figure 5. Method styling
Problem 3: Static
ashkelon's UI is built dynamically using
JSPs. Yes, it also is dynamic in other ways: it sports a number of
client-side, DHTML features (as in a dynamically collapsible
inheritance tree, or dynamically filtered tables). But that's not the
point. With a dynamic (as in JSP or PHP) site, you can build a user
interface that is more flexible. You can implement search forms (direct
access to information). Figure 6 shows the results for searching for
all classes whose names begin with
ashkelonalso provides advanced search forms for either classes or methods.
Figure 6. Search Results
With a dynamic system, one can build a navigation trail to allow temporal access to information. A sample navigation trail is displayed in Figure 7, just below the menu bar (note that the color coding applies also to elements within the navigation trail).
Figure 7. Navigation trails
The schema can be extended to produce views that don't exist in
Javadocs. Figure 8 is one example of how you might display author
Figure 8. Author information
One can produce statistical information about code using SQL
statements. Figure 9 shows the results of a query that lists all of the
packages in alphabetical order. Next to each package is the number of
classes it contains, with all packages with more than fifty classes
highlighted in yellow.
Figure 9. Displaying statistical information
ashkeloncan benefit an individual developer, a team, and even the
The individual developer can run
ashkelonon his or her local box to
reference multiple APIs locally. More interestingly, the team can use
ashkelonto look up both internally developed code and APIs that the
team is using on their current project. The
ashkelondatabase can be updated
nightly with the latest snapshot of the code from CVS so that everyone
has a reference to the latest code. Lastly, a public web site hosting a
large number of APIs can create some terrific economies, and at the same
time capture community contributions and improve itself. I've long wanted to see something like Patrick Chan's examplets married with Javadocs. In the early days,
dbdocand was featured in the java.sun.com article
"dbdoc: Persistence Pays."
A public instance of
ashkelonran at dbdoc.org a few years ago, and
could be resurrected. We would need hosting as well as volunteers to
maintain the site, populate APIs, field users' requests for new APIs,
and so on.
There are so many ways you can get involved. Adopt
ashkelon in your work environment. Download
and, if you can, please
to the project and/or join the project. With a half-dozen talented
developers, this project might have what it takes to become an Apache
project. If you're a graphic designer, you could help significantly by
producing a formal project logo or graphics to enhance the
ashkelon UI. If you don't have time to become a developer, please
participate in the discussions. Join the ashkelon-users or
ashkelon-devs mailing lists.