Timing is Everything

Contents It's About Time Going Beyond the Built-In Timers Overview of the Timing Model Features in TimingController Introduction to the Code Future Work Appendix: Timing Resolution

Tick, Tick, Tick, Tick

Any time you introduce dynamic effects, animations, or time-based events
to a Java application, you find yourself
re-implementing the same functionality you have written for every application that
required timing or animation. The built-in APIs are powerful, but they require
that you write a fair amount of boilerplate code. This article considers the
current situation and what is needed in a timing framework. The article refers to
example code contained in a
project posted on java.net:
timingframework.dev.java.net.

There are so many different topics to discuss in this area, but for
the purposes of this article, I will follow the introductory material with a
look at these issues:

• Overview of the timing model: There are a few conceptual ideas to
  get across first, to make it easier to understand the terms and
  features that TimingController uses.
• Features in TimingController: This section covers the major
  interesting features in the TimingController utility.
• Introduction to the code: I actually went way out of my way to
  fully document the sample code, which I think is a much better way of
  explaining how things work than any walkthrough I could do in this
  article. Nevertheless, a high-level view of the files and functions
  involved might help, for starters.
• Future work: There is so much more to do here, but I wanted to
  get this code and framework out into the real world soon, to get
  feedback and to see what made sense for future efforts. But
  there are some obvious areas that I thought would be useful to get into
  eventually. This section describes those areas and proposes some
  possible solutions.
• Appendix: Timing Resolution: This section is not directly related to this
  article, which is mainly about utilities for effective timer usage, but
  it is a topic that arises eventually anytime you get into timing and
  animation, especially animation that depends on small time increments.
  I wanted to discuss some of the problems and issues around timer
  resolution somewhere, and this seemed as good a place as any.

It's About Time

I recently wrote some simple demo applications to show off a
particular point for a presentation on dynamic effects in Swing.
In the course of this, I once again
realized what a complete hassle it is dealing with timing and
animation issues.

For example, suppose I wanted to have a simple animation where I
move an image across the screen back and forth ten times, each time
taking 500 milliseconds. I'd like to update its position every 30
milliseconds (about 30 frames per second), I'd like to pause for 50ms
before starting, and finally, I'd like it to hold its final position
when it is finished. I should be able to write
code akin to the following:

        repeatCount = 10;         // do the animation 10 times
        duration = 500;           // each animation lasts 500 ms
        resolution = 30;          // update very 30 ms
        begin = 50;               // wait for 50 ms before starting
        repeatBehavior = Reverse; // reverse at the end of each animation
        endBehavior = Hold;       // hold final position
        Animation anim = new Animation(repeatCount, duration, resolution,
                                       begin, repeatBehavior, endBehavior); 

Unfortunately, here is the code I would have to write instead (using
javax.swing.Timer):

	class MyActionListener extends ActionListener {
            public void actionPerformed(ActionEvent e) {
	        // Here's where all the logic goes: check the event time and
                // see whether to reverse or end the animation based on the
                // time elapsed
            }
	}
	
        resolution = 30;          // update every 30 ms
        begin = 50;               // wait for 50 ms before starting
        ActionListener myActionListener = new MyActionListener();
        Timer timer = new Timer(resolution, myActionListener);
        timer.setInitialDelay(begin);

This snippet doesn't look too bad until you realize that
the real work is not yet done; implementing the insides of
myActionListener can be pretty tedious.

Going Beyond the Built-In Timers

To be fair, the timing utilities in Java, java.util.Timer
and javax.swing.Timer, are pretty powerful and can help
you schedule time-based events. These classes let you create one-time
and repeating events that will call into your application at scheduled
intervals so that you can cause things to happen at those times.

This is a standard way to handle time-based events, such as
animations; you are essentially telling the system "wake me up every n
milliseconds." When you are woken up, you perform the appropriate
action (change the appearance of an animating cursor, move an object on
the screen, poll the network, send mail to your boss telling her how
productive you are, whatever).

These are great utilities as far as they go, but I find that they don't
go far enough. There is always other time-based functionality that I
end up layering on top in order to deal with real application needs. For
example, I usually want to run a timing event (such as the fading in of a
component) for a set amount of time, but the Timer utilities assume
one-time or indefinite operation and must be manually stopped when you
are finished with them.

I thought it would be useful to codify some of the generic
functionality that I tend to need into a higher-level utility that
makes using timers a bit easier (at least for me). I've done this with
the TimingController and associated classes in the java.net project
TimingFramework.
Ideally, this framework could be
built upon to have more functionality that applications may need;
perhaps that topic is ripe for another article or project some day in
the future. But in the meantime, I hope that TimingController provides
the following things for developers:

• Example code: It could be that the basic functionality of the
  existing Timer classes do everything you need, but that it's not clear
  how to use them for your application. Hopefully, the sample code
  will help demonstrate how to use Timers in general.
• Framework: TimingController hopefully provides enough
  functionality that you can use it as is, or you can add to it for
  greater or more application-specific functionality.
• Discussion: I've solved my current hot items with
  TimingController, but I'm sure there are more common cases out there
  that we could add to the framework as appropriate. Also, I've taken
  some quick-and-dirty approaches to solving some of the problems so that
  I could get the code and this article out there in short order. Perhaps
  some feedback from the real world will help to improve the framework in
  general and provide a more robust and general solution that developers
  can use.
• Building block: I have much larger things in mind for animation
  and timing frameworks. I see TimingController as the first
  step toward that eventual goal of World Animation Domination; it rounds
  out the sharp edges of Timers and adds some important yet simple
  functionality. I would like to add more functionality to this framework to
  accomplish even great things eventually. But first things first.
• Stepping stone to ideal standard approach: Ideally, in the future
  there will be simple
  standards and libraries available to do this kind of stuff. In particular,
  the SMIL standard
  covers all of this type of functionality and more. SMIL is not currently
  available for ready use with J2SE, but we hope that it will be someday.
  This kind of technology should allow much easier and more powerful
  use of animations, both through use of tools that generate SMIL content
  (such as SVG tools) as well as
  language standards that allow you to write Java code that interacts with
  SMIL content. But until then, you might think of TimingController as
  the first step toward that goal of SMIL-based timing technology.
  So hopefully in the
  future, when everything is totally integrated and available, you'll be ready to dive in.

Overview of the Timing Model

The timing model can be pretty
confusing, especially when you try to incorporate ideas of simple timing
loops inside of larger frameworks, such as the animation example above, which
runs for 500ms but is repeated ten times inside a larger animation structure.
In an attempt to simplify this model, I use terms
in this article and in the code that describe these concepts in a
logical way:
cycle and envelope.

"Cycle" is the term I use to describe the lowest level of
timing; it is a timing event that runs for "duration" amount of time
with "resolution" amounts of time in between each timing event, as
depicted in this diagram:

The diagram shows time progressing from left to right, along the
arrow, with the beginning of the cycle at the far-left vertical line
and the end of the cycle at the far right vertical line. Each tick shows
a single timing event, which occurs with a frequency defined by the
resolution.

The idea of duration is useful. Typically, in the demos I
write, I want the animation to run for a specific amount of time and
then stop. For example, if I want a component to fade in
over the period of a second, I will want an animation to run
that performs incremental fade-in capability (by changing the
AlphaComposite used by the component's paint() method) and then I want
that animation to stop one second after it starts. This framework
allows you to specify that duration and the timer will stop
automatically.

This definition of a cycle is fine for very low-level simple
timing; we could build a utility that allows us to specify the duration
and resolution, and we could have a timer that performs appropriately.
This is basically just a simple layering of the duration concept over
the current Java timers.

But we could also expand upon this idea by having the concept of a
timing envelope, which I define to be a series of cycles. A simple diagram
of an envelope follows:

In this diagram, we once again see time marching forward from left
to right, but now we are seeing how cycles and their envelope
interact. Each envelope has the following:

• begin: The time when we will actually start the first Cycle.
• cycle(s): Some number of cycles, as defined above, which have
  duration and resolution associated with them.
• repeatCount: The number of cycles to run. This number
  determines the endpoint of the envelope in time; after the last cycle
  is finished, this envelope is also finished.

Some important things about envelopes that are not called out in the
diagram (because I found them difficult to draw; maybe if I had another
dimension or two at my disposal in my drawing tool I could manage
it) include:

• RepeatBehavior: Envelopes can choose to either Repeat or
  Reverse at the end of each cycle. This will cause the next
  cycle to either work forwards each time (Repeat) or to reverse
  direction each time (Reverse). This will become clearer in playing
  with the code and the SimpleAnimation example, but think about the
  animation example at the beginning of the article. We could
  choose to animate the image by moving it back and forth across the
  screen (Reverse) or by moving it always left-to-right, popping it back
  to the leftmost position whenever it finishes each left-to-right cycle
  (Repeat).
• EndBehavior: Timing events can either Hold the final value at
  the end of the envelope, or they can Reset the value to the initial
  value when the envelope began.

Features in TimingController

TimingController depends on the underlying Timer
capabilities of the
existing libraries, but layers on other pieces of functionality that I
have found useful in various applications. I'll highlight the main
points of the framework here, but looking at the code itself is
obviously the best way to understand how it all works. In particular,
the SimpleAnimation example uses all of the different features of
TimingController, so you can play with the features in the GUI and see
how they were implemented in the code.

Overview

TimingController uses the concepts of Cycle and Envelope, as
described above. Each TimingController is created with
a Cycle and an Envelope, like this:

	TimingController myTimer = new MyTimerSubclass(cycle, envelope);

The Cycle object is created with:

• duration: An integer value representing
  the number of milliseconds that
  each cycle should take. This value can also be
  TimingController.INFINITE to represent that the
  cycle will be unending.
• resolution: An integer value representing the number of
  milliseconds between each timing update (calls to
  timingEvent(), below). A developer can determine
  how often they would like to have their events processed (for
  example, how many times they want to update an animation), and can set
  this value appropriately.

The Envelope object is created with:

• repeatCount: A double value that represents the number of cycles that
  should be run before this Envelope is finished. This can be a
  fractional value, which would cause the final cycle to end somewhere in
  the middle. This value can also be TimingController.INFINITE, which
  means that the Envelope is unending and that the cycle in the envelope
  will continue repeating until the TimingController is stopped.
• begin: An integer value representing the number of milliseconds
  that should elapse before the first cycle begins.
• RepeatBehavior: FORWARD or REVERSE, which
  controls the direction of flow of each successive cycle. A
  FORWARD behavior will start each cycle at the beginning
  and continue to the end. A REVERSE behavior will start
  each successive cycle at the place where the previous cycle ended.
• EndBehavior: HOLD or RESET, which controls
  the final value taken at the end of the Envelope. A HOLD
  value will cause the Envelope to end sustaining the final value
  calculated, whereas a RESET value will cause the
  Envelope to end by resetting the final value to the initial value.

The TimingController object, after its start(),
repeatedly calls into the TimingController.timingEvent()
method with three values:

• cycleElapsedTime: The time that has elapsed in
  the current cycle.
• totalElapsedTime: The total time that has elapsed
  since starting the first cycle.
• fraction: A floating point fraction that represents the
  portion of the cycle that has elapsed.

Note that, at this point, TimingController puts the real work of
calculating values based on the timing events onto the application; the
framework does not interpolate values for you. However, given the
fraction of time that has elapsed in a cycle, it is usually
straightforward for an application to calculate values based on each
time event.

Note, also, that I use Cycle and Envelope in the code here to make it
easy to translate between the text in this article and the code in the
framework. In a real implementation of this framework, I might dispense with these
extra classes entirely. They do a nice job of documenting the timing
model structure, but
in reality, they add a level of indirection that real users might not want
to deal with constantly. But this more academic approach will do for now.

Timer Independence

There are various ways to use timers in your application, from
the existing Timer classes to running a thread of your own and doing
all of your own timer scheduling manually. I did not want to have any
visible dependencies on a particular scheme to limit what someone could
do with the framework. For example, the existing Timer classes have
certain implications for timing resolution; if you ask for a timer to
have a resolution of three milliseconds, but that timer uses a
low-resolution timer, then you may not get anywhere near that
resolution (see the section below on timing resolution for more
information on this). So instead of having a visible and obvious
dependency on one of these classes, this framework is independent of
the existing utilities. It may depend on these utilities internally,
but the behavior you get will not be constrained by these classes. In
the timing resolution example, we could write the framework to have a
fallback mechanism that works around the resolution constraints of any
particular Timer mechanisms.

The framework has a mechanism for firing time events that is
independent of existing Timer mechanisms. For example, the Swing Timer
utility calls an ActionListener with a timing event, which is not the
case with TimingController. When a timer event occurs in
TimingController, it will call its own timingEvent() method; your
application would subclass TimingController in order to receive this
event.

Animation Fraction

A very simple way to write animations is to simply increment or
decrement values by some amount every time a timing event occurs. For example, you might move an icon back and forth on the screen during
some animation. Every time the Timer event occurs, you increment or
decrement the x value of the icon position by one. This is fine for
simple demos, but the approach breaks down quickly for anything
complicated or where you need some dependable behavior.

For example, say you developed the application on some powerful
system with gobs of memory and a fast CPU. You set up a Timer to call
your application every five milliseconds, and your icon scoots across the
screen at a rate of about 200 pixels per second. Then you run that demo
on a system with little RAM, a slow CPU, and a very low-resolution
timer. Now your application may only get called every 50 milliseconds,
or may have very inconsistent performance and get called frequently
sometimes and infrequently at other times. Now your icon staggers
across the screen at a painful clip of 20 pixels per second, sometimes
running much slower than that due to system hiccups.

Video games were written this way many years ago; they looked great
on their target systems, but when they were run on systems much slower or
faster than the target systems, the games were unplayable. I once saw
an awesome graphics demo that spun around a wireframe piano model as it
played a song; the graphics of the piano keys were perfectly synchronized
with the sounds of the keys striking the strings. On the target system,
it looked and sounded great; a
very nice piece of work. But then I ran it again a few years later, when
graphics performance was much greater and it looked ridiculous;
the graphics finished rendering in about a tenth of the time as before,
while the music still played at the same speed. This demo was written
with the speed of the original system in mind. The movement
of the piano model was not based on realtime values, but rather on
the speed at which the runtime graphics system could draw the wireframe model.

The best way to handle animations so that they perform similarly
across a wide variety of platforms is to base the animations on elapsed
time instead of just the frequency of timing events. In the animating
icon example above, this means you would base the position of the icon
not on how many times your event was triggered, but on the total time
elapsed for any given timing event. For example, if you wanted your
icon to move at a constant rate of 200 pixels/second, then you would
calculate the number of seconds elapsed in the animation from left to
right and multiply that number by 200 to get the proper x value.

The existing Timer utilities send out events that are used simply
to signal the recipient that a timing event has occurred. It is up to
the client to choose what to do with that event. A typical thing to do in an
animation would be to figure out how much time has elapsed in either
the entire animation or since the last time an event fired and then to
react accordingly.

I thought it would be more useful to send out the information the
application actually needs here, instead of just a vanilla "timer
fired" event (which is the way the current Timer classes work).
Specifically, I wanted to be notified with the fraction of a
given cycle that has elapsed. So if we are halfway through the
duration of an animation, I would like to receive a notification that
means "you are halfway through." Then I can easily use that fraction to
calculate the correct value of my time-based animation.

TimingController helps out here by sending an event to timingEvent()
with the fraction of completion of the current cycle.
We can then use this fraction to calculate
the appropriate value that we are trying to vary.

The fraction is useful in both the Repeat and Reverse behavior
modes. In the simplest example, a fraction of .25 tells us that the
animation is one-quarter of the way through. But suppose we are using a
Reverse cycle and we are now running backwards; a
value of .25 tells us that we are one-quarter of the way through the
cycle, even if we got to that value by animating down from the end to
the beginning of the cycle. That is, the fraction always
represents the fraction between the beginning and end of a cycle, no
matter whether we are running the timing sequence the first time, or in
a Repeat cycle, or in a Reverse cycle.

Introduction to the Code

There should be little need for a full code walkthrough here; just check
out the code,
read the many comments, and play with the
SimpleAnimation sample program to understand how it all works. But I
will give some simple pointers to start you off.

The actual framework is in three classes in the timing package:

• Cycle: This is a simple structure that holds the
  resolution and
  duration parameters of each cycle.
• Envelope: This is also a simple structure that
  holds the
  repeatCount, begin, cycleReverse, and endBehavior parameters of the
  overall envelope.
• TimingController: This is where all of the
  functionality in the
  framework lies. The heart of the class is in the constructor (which
  sets up the parameters used in the animation, Cycle and Envelope), the
  start()
  method (which actually starts the animation), and the internal
  ActionListener
  class (which is used to field the javax.swing.Timer
  events and translate them into timingEvent()
  events). Your application would subclass TimingController and
  override timingEvent() in order to receive the timing events.

The other part of the sample code is SimpleAnimation, which
is a simple GUI-based animation application used to test TimingController.
The application allows you to specify the parameters for the Cycle and
Envelope used in the application, and then start the animation
running. Here's what you see when you start it:

Most of the code in SimpleAnimation is used to set up
and run the GUI. The core functionality that actually sets up
and runs the animation is in the following lines in
ControlPanel.actionPerformed():

	Cycle cycle = new Cycle(duration, resolution);
	Envelope envelope = new Envelope(repeatCount, begin, 
					 repeatBehavior, behavior);
	animation = new Animation(animationView, cycle, envelope);
	animation.start();

If you look at this code and then the first line of code at the
beginning of this article, and you squint a bit, you can see that
the approaches are pretty similar. Mere coincidence? I think not.

Future Work

My hope is that TimingController can become a building block in a
larger framework of general animation utilities. There are many
places to go from here, but there are a few key directions that seem
worth calling out here:

• Fixes and minor tweaks: I am not foolhardy enough to believe that
  the current incarnation of TimingController is set in stone. In
  fact, I just finished rewriting it completely over the last few
  weeks. I think it accurately represents my current thoughts on the
  basic functionality that I wanted to describe and implement, but there are
  many ways of going about this, and some may be significantly better
  than what I have done so far. So with feedback and more thought
  on the matter, the code may evolve to suit.
• Value interpolation: Currently, the application is responsible
  for interpolating and calculating target values for any time-based
  events. TimingController provides the events and the information
  necessary to calculate these values, but it does not interpolate values
  directly. This seems like a natural capability to add into the framework.
  Of course, many interpolations may be application-dependent, but there
  are some that seem like natural and general things that most
  applications would need, so we should put those capabilities into the
  shared framework. For example, the ability to linearly interpolate
  integer and floating-point values seems pretty basic and useful.
• Timelines: So far, the capabilities of TimingController with
  respect to global timelines or timing interaction is pretty basic. Your
  application controls when an envelope may start, but it has no easy way
  of synchronizing multiple envelopes or events. A more involved
  framework should include the idea of timelines and envelope
  synchronization so that we can more easily hook up multiple envelopes
  to synchronize their behavior.
  For example, suppose you want to animate
  GUI effects, such as fading in a component first and then repositioning
  it. Ideally, you would have one animation fire off the other one at an
  appropriate time, or you would have some master global timeline that
  fires off the appropriate animations at the right times. But currently,
  you would need to schedule and start these envelopes manually at the
  appropriate times. It can be done, but if this is a common thing that
  many applications need, it should be part of the framework.
• Standards: Once you have capabilities like those outlined above,
  it would be interesting to see if we could support standard animation
  specifications such as SMIL,
  which is a W3C standard for time-based animations. This language is
  used in other specifications such as href="http://www.w3.org/Graphics/SVG/">Scalable Vector Graphics (SVG),
  where it enables time-based animations of graphics.
  
  Or, better yet, there may be SMIL-based libraries available in the near future that
  may subsume much or all of the functionality of TimingController, in which
  case we should just use those libraries directly.
• Declarative animations, tools, and kitchen sinks: One you have
  the capabilities above, you could really go wild and start defining
  declarative syntaxes for Java-based animations, tools to help author
  the animations, and other more time-intensive and infinitely-recursive
  problems.

I hope that we can eventually accomplish much of what I've outlined
above. But it all starts with the basics, thus the TimingController
utility and sample code.

Appendix: Timing Resolution

I could write a whole article (or more) on this topic alone, but
I'll try to limit myself to the highlights here.

The issue here is that all platforms have different constraints on
the resolution of their default timing systems. Resolution here
means the amount of time between different time-based events. For
example, the amount of time that a call to Thread.sleep()
takes is dependent upon the resolution of some underlying timing
system, as is a call to System.currentTimeMillis().

For most applications, this topic is irrelevant. If you are a
static GUI application, then I don't even know why you have read this
far; you don't have any need of time-based events. If you do have some
simple animations or time-based events in your application, then
chances are good that the resolution you require is of far coarser
granularity than that provided by the default timers on most platforms,
so you don't need to worry about timing resolution in general.

This section is not about those applications.

This section is for developers of performance-sensitive
applications, including those with fast animations or where framerate
is very important (games come to mind). Control freaks, that's what
they are; developers that demand fine control over the scheduling of
their animations and events.

These applications may care that when they ask to sleep for 10ms,
they actually don't wake up for 16 milliseconds. Or when they ask a
Timer to call them 500 times per second, they actually only get
called 60 times per second.

The main problem is that the default timing systems on the native
platforms are, in general, of low resolution, by which I mean that
they do not have the granularity to deal with requests of two millisecond
wakeups. For example, in native code on Windows, the usual
way to find out how many
milliseconds have elapsed is the Win32 function GetTickCount(). This is
the native underlying utility used by System.currentTimeMillis()
on that platform. GetTickCount(), however, has a default resolution of
somewhere between 10 and 16 milliseconds, depending on the platform (for
example, my Windows XP laptop has a default resolution of 16ms). If
you call System.currentTimeMillis() on a platform with a
resolution of 16ms, then you will only get answers in 16ms
increments. So if you call it at one time, then call it one millisecond later, you
will either get an answer that is the same as what you had before or a
value that is 16ms greater (depending on whether you just crossed that 16ms boundary).

Similarly, Thread.sleep(long ms) is implemented using
the native platform's sleep() function, which again might
use a low-resolution timer by default. This may mean that when you ask
to sleep for, say, 10ms, you will actually sleep for 16ms instead.

There is much more to write on this subject (especially as to
workarounds, bugs filed, fixes, etc.), but the main reasons I wanted to
call it out in this article include:

• Awareness: You need to be aware of the implications of using the
  time-based commands in the JDK. If you have need of timing resolution
  that is greater than that provided by these methods by default, make
  sure you test your application on your target platforms and get the
  results you intended. And if you don't, make sure you find a way to
  work around the problem.
• System.nanoTime(): One of the new features in JDK
  1.5 is the new System.nanoTime() method. This uses, in
  general, a higher-resolution timing facility on the native platform. It
  returns a different number than System.currentTimeMillis()
  (for one thing, it's 1,000,000 larger since it's in nanoseconds; for
  another thing, it's a time value that is useful only when compared to
  other return values from that function). But if the constraints of this
  number meet your needs (as they did for my needs in TimingController),
  then it's a great way to get higher resolution time values.
• TimingController limitations: Currently, TimingController is
  layered on top of the existing javax.swing.Timer utility.
  I purposely hid that implementation detail inside of the class so that
  TimingController is not stuck with that approach. But the implication
  of using that Timer utility for now is that
  TimingController suffers from the same low-resolution constraints as
  that utility. For example, if you ask TimingController for a resolution
  that is less than that which javax.swing.Timer provides,
  you may be disappointed in the results. This can/should change in
  future iterations of TimingController, but it is an important thing to
  note in the current incarnation.

Note that the advice from the "Timing Fraction" section above
applies well here; even if you are stuck with a timer with a lower resolution
than you think you need, calculating your values based on
elapsed times should give you what you need. Maybe you
wanted your sprite to zip across the screen at 5000 times per second
and you're only getting callbacks 60 times per second. It's unfortunate
that it didn't match your ideal, but your icon will still look pretty
smoothly animated at that lower rate, and it should look to the user as if it is
moving at the same speed. It is just that the sprite's position will not be
updated as often as you wanted.

Conclusions

I hope, foremost, that this article helps you to understand how
timing works in Java in general.
I also hope that you will download the code, play with it,
understand it, and send in comments to help improve the
framework. It would be wonderful to build on what we have and to
create a more robust, functional, and general framework
for assisting in all kinds of time-based applications.

Reminder: check out the code for this article at:
timingframework.dev.java.net.