Skip to main content

Smooth Moves

February 23, 2006

{cs.r.title}







Are you interested in doing some animations in your Java applications, but find yourself plagued by results that seem stuttery and choppy? Want to figure out the problems and smooth out those animations to make them better and more seamless in your application? This article examines some of the factors that affect animation smoothness and things that you can do in your code to make your animations look better.

In my blog entry, I looked at the various problems contributing to an animation looking choppy. I found that much of the choppiness in my animations came from two general sources:

  • Color difference: The amount of color change per pixel needs to be minimized.
  • Vertical retrace: Artifacts coming from computer display technology.

Minimizing the Color Difference

The key idea here is to make our animations smoother by reducing the amount of color change for each frame of the animation.

How is this done? We have some particular object that we need to animate from here to there; we can't just alter the colors along the way to suit this approach, can we?

Yes and no. You may not be able to simply decide that you don't want a black space invader and you'd be happy with a light gray; sometimes that just doesn't work with the whole Evil motif. But there are things you can do to mitigate the strong contrast between the background color and the object color.

  • Object color: One approach is to change the color of the object. As noted above, it's not really a solution in all (or many--or any, really) cases, but it is interesting to see what effect that change has on perceived smoothness. An object that is closer to the background color causes the pixels to shift less as it moves around on that background; the background pixels do not need to shift as much in RGB space to represent the object color, and this difference in color shift results in smoother perceived movement for the object.
  • Anti-aliasing: Another approach is to use anti-aliasing. Instead of an object having hard edges that are at sharp contrast to the pixels the object will move over, you could soften those edges by fading the object color out translucently on the edges. This has the net effect of a smoother ramp up to the object color and back to the background color as the object moves around over the background.
  • Motion Blur: This is the effect of drawing ghost images of the object as it moves around. There are various ways to do motion blur, some more correct than others, and some more performant than others. I opted for a very simple approach of simply drawing trailing versions of the object translucently in the previous places occupied by the object. This is similar to the "cursor trails" available as an option on some operating systems. You draw the object in its real location in its true colors, and you draw some number of ghost images where the object used to be in some faded color (translucent, so that the background color shows through these ghost images). This effect does not impact the rate of color change for the pixels on the leading edge of the object movement; since we are still drawing the object in its true colors, the background pixels still have to shift immediately to that color as the object moves over those pixels. But blurring the past locations of the objects allows a smoother ramp back down to the background color from the object color. Romain Guy played around with other approaches to motion blur that allow for better ramp-up and ramp-down effects, by basically ghosting all locations of the image, including the leading one.
  • Hard, linear edges: One thing that you'll notice is that hard edge movement trips up your eyes more than irregular edges; the eye is pretty good at detecting artifacts in a column of pixels that marches along, but tracking artifacts in an irregular shape is more difficult, making artifacts easier to disguise with such shapes. For example, you could have a solid rectangle to represent your alien spaceship, which could make for some obvious rendering artifacts, but chances are you'll do better to have an irregular shape for the spaceship to smooth out the animation. Fortunately, most alien spaceships I've encountered tend to not be solid rectangles, so this meshes with reality fairly well.

Vertical Retrace

As discussed in my blog, vertical retrace artifacts come from rendering to the screen at the same time as the screen is being refreshed from video memory. Suppose we are trying to move our image between frame n and frame n+1 like so:

vsyncNoArtifact

Now suppose that the vertical retrace (represented by the red line below) is happening right in the middle of this area as we are doing this copy:

vsyncNoArtifact

There are various workarounds to this issue, mostly related to the same problems and solutions described above for color distance, including:

  • Minimizing color differences: The tearing artifact is made worse by high-contrast changes in an animation; the less distance there is between the background color and the object color, the less noticeable will be the tearing artifact. All of the approaches above that addressed this issue are applicable here.
  • Minimizing linear shapes: The artifact is particularly noticeable in objects like the rectangle above: the eye notices that what is supposed to be a straight line is no longer straight. If the object has an irregular shape instead, then tearing artifacts would be harder to spot and less disturbing.
  • Minimizing distance between frames: The further an object moves between frames, the more obvious will any tearing artifacts be. The picture above shows a pretty horrid tear that consumes nearly a third of the object. If that object instead moved only one pixel, the tear would be far less noticeable.

There is also a "fix" to the problem: avoid encountering the vertical retrace completely. This is done by waiting for the "vertical blank interval," which is a slot in time after the refresh has reached the bottom of the screen and before it starts again at the top. If you can successfully synchronize your application to be timed with this interval, then you can be fairly assured that such tearing operations will not happen because your application will update the screen only when it is safe to do so.

This fix is easy for anyone writing a fullscreen Java application; applications that use the FlipBufferStrategy get this for free. When that buffer strategy copies its contents to the screen from the back buffer, it specifically waits for the vertical blank interface, and thus avoids tearing completely.

The fix is not as easy for typical windowed (non-fullscreen) applications, because there is currently no way to tell Java to wait for this interval, and there is no way for your code to know when it is a good time to go ahead with the copy. We hope to address this in a future release (I just filed a bug on it last week!), but in the meantime there is no way to get this behavior.

Or is there?

After I hacked a prototype of the fix for the fix in the Java 2D implementation (which will be used in any real fix we provide in Java), I tried a similar approach with application code--and it worked!

Of course, the fix for application code is a bit different because there is no Java API to access this functionality. But there is native API available, at least on Windows, so with a very small amount of JNI code, I was able to successfully synchronize on the vertical retrace interval. I put this code into the SmoothAnimation application, discussed below, so you can try it out for yourself (assuming you are on Windows; I've only implemented a solution for that platform so far).

SmoothAnimation: Demonstrating the Problems and Solutions

Now we've talked about the problems and solutions; let's look at some code.

I wrote a sample application, SmoothAnimation, to play around with various factors and possible solutions to the choppy animation problem. The application renders two animations at the same rate: a fading animation that fades an object in and out between complete opacity and complete transparency, and a moving animation that shifts and object from left to right and back again at some steady rate. Running this application will show the problems we've discussed above pretty clearly; the fading animation looks quite smooth, while the moving animation looks quite choppy. The application also allows various flags to be toggled with keyboard options, to see the impact that different rendering approaches has on the perceived smoothness of the animations.

I'll put some snippets below as I discuss the relevant parts, but you are encouraged to run and download the whole application. Here are some ways you can use the sample code, in ascending order of detail and difficulty:

  • Run the applet : Go to the SmoothMoves demo page to run an applet version of the application. This applet demonstrates most of the functionality described below, although it skips out on the vertical retrace fix since that requires native code.
  • Run the application : Download the sample code file SmoothAnimationDemo.zip, unzip it, and run the application locally by cding into the dist/ directory and typing java -cp SmoothAnimation.jar com.sun.animation.SmoothAnimation
  • View the source: View the source for SmoothAnimation.java and VBLocker.cpp.
  • Download and build the source: Download SmoothAnimationDemo.zip and unzip it to get the source code in a buildable form.
1) Creating the graphics

The application first creates the graphics that it will use. In all cases, the application creates an image that will be copied later using drawImage() during the animation, but this image may be created from an actual image (I use my personal favorite, duke.gif) or from rendering commands (I use a solid rectangle, which contrasts sharply with the white background to better illustrate the points in this article). At first, the application uses a solid black rectangle, but this can be toggled during the application, as we'll see below.

Here is the image creation routine:

void createAnimationImage() {
    GraphicsConfiguration gc = GraphicsEnvironment.
        getLocalGraphicsEnvironment().
        getDefaultScreenDevice().getDefaultConfiguration();
    image = gc.createCompatibleImage(imageW, imageH, Transparency.TRANSLUCENT);
    Graphics2D gImg = (Graphics2D)image.getGraphics();
    if (useImage) {
        try {
            Image originalImage = ImageIO.read(new File("duke.gif"));
            gImg.drawImage(originalImage, 0, 0, imageW, imageH, null);
            gImg.dispose();
        } catch (Exception e) {}
    } else {
    // use graphics
    Color graphicsColor;
    graphicsColor = Color.black;
    gImg.setColor(graphicsColor);
    gImg.fillRect(0, 0, imageW, imageH);
    }
}
2) Running the Timer

The next step is to start a timer, which will run the animation loop. I do this by using my TimingFramework classes. You could do this with the simpler timer classes built into core, but TimingFramework has some additional facilities (like being able to reverse the animation at the end of each cycle) that make it easier for more involved animation usage. Here is the code to create and start the timer:

TimingController timer = new TimingController(
    new Cycle(1000, 30),
    new Envelope(TimingController.INFINITE, 500,
                 Envelope.RepeatBehavior.REVERSE,
                 Envelope.EndBehavior.HOLD),
    component);
timer.start();

I'll leave it as an exercise to the reader to check out TimingFramework for the details (there is an article linked from the project that describes the innards of the framework), but the basics here are:

  • Cycle: I want an animation that runs for 1000 milliseconds with a resolution of 30ms (which will give us a framerate of about 33 frames per second).
  • Envelope: An Envelope encapsulates one or more Cycles. In this case, I want my overall animation to run forever (or until I get tired of it and kill the app), with an initial delay of 500ms (to give the window frame time to realize itself on the screen). When the animation Cycle reaches its end, I want it to reverse back to where it started. The HOLD parameter, which causes the animation to hold its final value, is irrelevant in an animation like this that runs forever. The final parameter, component, is what the TimingFramework will call into with each timing event; this is where we will calculate the new positions and attributes of the objects based on the elapsed fraction of the animation and then force a repaint.
  • Timer.start(): Finally, we start the animation.

Now let's look at what happens as the animation runs. We will get callbacks into the timingEvent() method whenever the timing framework determines that it is time for another frame to be rendered:

public void timingEvent(long cycleElapsedTime,
        long totalElapsedTime,
        float fraction) {
opacity = fraction;
moveX = moveMinX + (int)(fraction * (float)(moveMaxX - moveMinX));
repaint();
}

Here, the cycleElapsedTime and totalElapsedTime parameters are not used; we only care about the fraction, which represents the fraction of animation elapsed between the start and endpoints of the total animation. We use this fraction to set the opacity value, used to calculate the translucency of our fading animation, and the moveX variable, used to position the moving object in our moving animation. After we've set these values, we call repaint() to force the application to render itself with these new values.

3) Rendering onto the Screen

Finally, let's look at the meat of the application, rendering the graphics onto the screen.

There are three main tasks in the paintComponent() method: erasing to the background color (I specifically wanted white, to contrast with the default black rectangle object), drawing a fading animation, and drawing a moving animation.

Erasing the background: This is simple; we just erase to white, like so:

g.setColor(Color.white);
g.fillRect(0, 0, getWidth(), getHeight());

Render the fading animation: In this step, we use the current value of opacity (calculated at every step of the animation, as seen later in timingEvent()) to create a new AlphaComposite object and set that on the Graphics2D object. Then we render our existing image using this Graphics2D object.

Graphics2D gFade = (Graphics2D)g.create();
AlphaComposite newComposite =
    AlphaComposite.getInstance(AlphaComposite.SRC_OVER,
                               opacity);
gFade.setComposite(newComposite);
gFade.drawImage(image, fadeX, fadeY, null);
gFade.dispose();

Note that I'm creating and disposing a new Graphics2D object here (cloned from the one passed into paintComponent()). This allows me to set the composite on the graphics object without having to worry about resetting it when I'm done (so I won't side-effect the rendering of other objects using the original graphics object). The opacity variable is set during the calls to timingEvent() (described above). fadeX and fadeY are just instance variables that declare where I want this thing to appear.

Render the moving animation: This part is simple:

g.drawImage(image, moveX, moveY, null);

We simply copy the image into the appropriate location, determined by the moveX and moveY parameters. moveY is static (we are only moving the object in the X direction), and moveX is set during each call to timingEvent().

4) Handling Vertical Retrace

The application also contains an optional piece to alleviate the problems with vertical retrace described above. I made this work only for the case where the application is running on Windows and DirectDraw is available, but even if you cannot get this part to work on your target system, it's interesting to see how you might go about doing something like this in general.

Here are the relevant bits that make this work:

  • Java code :
    • Initialization: We need to load the DLL created by our native code and then initialize it. This calls the native initialization code described above.
      static native void initVBLocker();
          static {
              System.loadLibrary("VBLocker");
              initVBLocker();
      }
    • Synchronization: During our paint routine, call the native code to synchronize on the blank interval:
      if (waitForVB) {
          // Do this right before finishing,
          // which is right before Swing's
          // copy of the back buffer to the screen
          vbLock();
      }
  • Native code :
    • Initialization: First, let's see what I did to perform the synchronization at the native layer. First, I needed to initialize DirectDraw by loading the ddraw DLL and calling the appropriate creation routines:
      HINSTANCE hLibDDraw = LoadLibrary(TEXT("ddraw.dll"));
      FnDDCreateExFunc ddCreateEx = (FnDDCreateExFunc)
              GetProcAddress(hLibDDraw, "DirectDrawCreateEx");
      if (ddCreateEx) {
          HRESULT ddResult = (*ddCreateEx)(NULL, (void**)&ddObject,
                                           IID_IDirectDraw7, NULL);
      }

      This gives us a handle to ddObject, which is used later to perform the synchronization operation.
    • Synchronization: The actual call to synchronize on the vertical blank interval is one simple function call: ddObject->WaitForVerticalBlank(DDWAITVB_BLOCKBEGIN, NULL);
  • Building : Incorporating JNI code into your project makes building the project a tad more complicated. You need to perform the following steps:
    • javah: You need to run this cool on your Java class, which will produce a header file that you need to include in the native source file you write.
    • Native compiling: You'll need to run the native compiler (VisualStudio98 or 2003 should do fine). You'll need to use appropriate include directories for both DirectX and JNI to make this work. You will also need to produce a DLL, which is an option to the VisualStudio linker.
    • Location, location, location: You'll need to put the DLL in a directory that will be found at runtime based on your Java library path (putting it in the directory you run the app from should be fine).
    • Building: The makefile and directory hierarchy I've included should work in most Windows situations. You'll need to modify a couple of variables in that makefile, but then you should be able to cd into that directory and type nmake (assuming you have an appropriate version of a Windows compiler installed and in your path).
  • Running: Finally, to run the app and see the effect, simply toggle V at runtime. This will toggle the waitForVB flag that is in the painting code that forces the synchronization to occur.
  • Caveat: It's important to note that this is a Windows-only solution. It's also important to note that it will only work in the (common) case where your application is running on the primary display (or where you have only one display). Multi-monitor would require a bit more work to detect which display you wanted to synchronize against.

That's it for the default behavior of the application; it creates the image, sets up the timer, and paints the two animations happily forever. But there's more; there are keyboard commands you can use while the application is running to try out different approaches to rendering to mitigate the choppiness and see the results:

  • I (Image) : This option toggles between rendering a solid rectangle as your image and an actual image (duke.gif). Using this option will give you irregular lines for your object, which make it harder to see the rendering artifacts that were so obvious with the solid black rectangle. The code to create this image is in createAnimationImage():
    if (useImage) {
        try {
            Image originalImage = ImageIO.read(new File("duke.gif"));
            gImg.drawImage(originalImage, 0, 0, imageW, imageH, null);
            gImg.dispose();
        } catch (Exception e) {
            System.out.println("Problems loading image file: " + e);
        }
    }
  • C (Color): This option toggles the color between black (the default) and light gray. You'll notice that the moving animation appears much smoother by doing this, just by decreasing the color distance between the object and the background color. The code to change the object color is in createAnimationImage():
    if (alterColor) {
        graphicsColor = new Color((int)(.75 * 255),
                (int)(.75 * 255), (int)(.75 * 255));
    }
  • B (Blur): This option toggles "motion blur," which causes the painting code to paint a trail of translucent ghost images where the moving object used to be. Notice how the artifacts on the trailing edge of the object are greatly decreased. The code to create this simple motion blur effect is in paintComponent(). First there is the setup code to create the arrays of blur values:
    if (prevMoveX == null) {
        // blur location array not yet created; create it now
        prevMoveX = new int[blurSize];
        prevMoveY = new int[blurSize];
        trailOpacity = new float[blurSize];
        float incrementalFactor = .2f / (blurSize + 1);
        for (int i = 0; i < blurSize; ++i) {
            // default values, act as flag to not render these
            // until they have real values
            prevMoveX[i] = -1;
            prevMoveY[i] = -1;
            // vary the translucency by the number of the ghost
            // image; the further away it is from the current one,
            // the more faded it will be
            trailOpacity[i] = (.2f - incrementalFactor) -
                    i * incrementalFactor;
        }
    }

    Next, there is the rendering of each blur image:

    for (int i = 0; i < blurSize; ++i) {
        if (prevMoveX[i] >= 0) {
            // Render each blur image with the appropriate
            // amount of translucency
            Graphics2D gTrail = (Graphics2D)g.create();
            AlphaComposite trailComposite =
                    AlphaComposite.getInstance(AlphaComposite.SRC_OVER, .1f);
            gTrail.setComposite(trailComposite);
            gTrail.drawImage(image, prevMoveX[i], prevMoveY[i], null);
        }
    }

    Finally, the ghost locations are updated at the end of the paintComponent method:

    if (motionBlur) {
        // shift the ghost positions to add the current position and
        // drop the oldest one
        for (int i = blurSize - 1; i > 0; --i) {
            prevMoveX[i] = prevMoveX[i - 1];
            prevMoveY[i] = prevMoveY[i - 1];
        }
        prevMoveX[0] = moveX;
        prevMoveY[0] = moveY;
    }
  • 1-9: These numbers toggle the length of the motion blur, from one ghost image to nine. The more images, the smoother the ramp down to background color and the less obvious the artifacts are, but with more ghost images comes a less realistic trail of motion behind the object. The code for this is shown above; this toggle affects the blurSize variable in the code above.
  • A (anti-aliasing): This option toggles the use of anti-aliasing on the solid rectangle image. The edges of the rectangle are now drawn in colors and translucency that gradually fade out to nothing. Just like the motion blur effect, this means a smoother ramp down to the background color, a smaller delta between object and background colors, and less noticeable artifacts. The approach I use is quite simply to simply render increasingly faded outlines at the edges of the rectangle, as shown in this code from createAnimationImage():
    if (useAA) {
        gImg.setComposite(AlphaComposite.Src);
        int channel = graphicsColor.getRed();
        gImg.setColor(new Color(channel, channel, channel, 50));
        gImg.drawRect(0, 0, imageW - 1, imageH - 1);
        gImg.setColor(new Color(channel, channel, channel, 100));
        gImg.drawRect(1, 1, imageW - 3, imageH - 3);
        gImg.setColor(new Color(channel, channel, channel, 150));
        gImg.drawRect(2, 2, imageW - 5, imageH - 5);
        gImg.setColor(new Color(channel, channel, channel, 200));
        gImg.drawRect(3, 3, imageW - 7, imageH - 7);
        gImg.setColor(new Color(channel, channel, channel, 225));
        gImg.drawRect(4, 4, imageW - 9, imageH - 9);
    }
  • Up/Down arrows: The animation starts off at a pretty awful frame rate, but you can increase or decrease the resolution of the timer (and thus worsen or improve the frame rate) by hitting the Up/Down arrows on your keyboard. Each click increments or decrements the resolution by 10ms.
  • V (vertical retrace): This option toggles synchronization with the vertical retrace interval. This only works if SmoothAnimation can find and load the VBLock DLL successfully. The option is disabled by default, so the first toggle will enable it. You should see the vertical retrace artifacts disappear completely when this works (although you may still see an occasional artifact, since there is some asynchronicity between the call to wait for the blank interval and the actual copying of bits to the screen).
  • L (linearity): This option toggles the interpolation mode of the animation. By default, the animation moves in a linear fashion. This toggle switches the animation to use a simple non-linear function instead (in this case, a simple trigonometry function). This gives a "bouncing" effect to the movement. Notice how this movement makes it more difficult to track individual frame discrepancies, as it is more difficult for the eye to predict exactly where the object is supposed to be, compared to the easily tracked linear movement.

To run the example, cd into the directory and compile everything. I've included a makefile to help out if you're going to try out the vertical retrace fix on Windows; you can either use the makefile directly (after changing a couple of variables in that file) or just see how it's doing the build. Otherwise, just compile the Java file com/sun/animation/SmoothAnimation.java and run it:

java com.sun.animation.SmoothAnimation

Conclusion

Check out the sample code, compile it, play with it, and see what you think. There are many more involved solutions than the ones I've toyed with here, but I think you'll see that anything that minimizes the rate of change of color helps in the overall problem.

Resources

width="1" height="1" border="0" alt=" " />
Chet Haase I'm a graphics geek. I worked at Sun, Adobe, and Google, always on the UI side of things.
Related Topics >> Programming   |   Research   |