Skip to main content

A Customized User Interface for Mobile Phones

June 5, 2008

{cs.r.title}







The StringItem class in "http://java.sun.com/javame/index.jsp">Java ME is used for
non-editable display of text. While it is very convenient and easy
to use, it has certain disadvantages too. Implementations of
StringItem are highly device-and theme-dependent. This
means the look-and-feel of an application can differ significantly
from one device to another. Also, some of these implementations
leave much to be desired. On one of my handsets, for example, a
StringItem based text display flickers so much during
scrolling that reading becomes extremely uncomfortable.

A text display based on the Canvas class overcomes
these drawbacks and offers advantages such as:

  • A highly uniform, device-independent and customizable user
    interface.
  • Text and display styles that can be more varied and can be
    changed dynamically.
  • Multilayer displays.

An example
Figure 1. An example.

Figure 1 shows such a display with different font styles and
with grouping for entries. Not only do such displays look virtually
the same on all phones, they are also very easy to implement.

In this article we shall examine a demo application showing the
step-by-step implementation of such a text display. The application
will use the following basic classes:

  • TextUI -- this is an abstract class that defines
    the main functionalities required for displaying text on a
    canvas.
  • MenuLayer -- this class is for creating a menu in
    the form of a Sprite.

In addition to the above, there will have to be a MIDlet --
TextUIDemo -- that will serve as the entry point.
However, the real work will be done by the classes listed above and
a subclass of TextUI.

In order to run the demo application you'll need to download
"http://java.sun.com/products/sjwtoolkit/download-2_5.html">the Sun
Java Wireless Toolkit
and install it on your computer.

The Display Parameters

Before we get into the programming details, let us establish the
broad visual aspects of what the user interface will look like. The
sketch in Figure 2 shows the overall structure and the associated
variables that have been used in the code to establish the
dimensions needed to draw the complete interface.

The basic layout
Figure 2. The basic
layout.

The dimensions of the area for rendering the text we want to
show can be easily visualized now. The height of this area,
represented by the variable height, will be:


height = totalheight - (2*topborder+titleheight+topmargin+
    2*bottomborder+bottomheight);

where

totalheight = height of available screen area
titleheight = height of title bar
topborder = thickness of borders above and below title bar
bottomheight = height of bottom bar
bottomborder = thickness of borders above and below bottom bar
topmargin = height of a margin above the text

Similarly:


width = totalwidth - (leftborder+leftmargin+rightborder);

where


totalwidth = width of available screen area
leftborder = width of the border at left
rightborder = width of the border at right
leftmargin = width of a margin at the left of the text

The TextUI Class

The base class for displaying text will subclass
Canvas and I've named it -- rather unimaginatively --
TextUI. This is an abstract class and must be
subclassed. The main function of TextUI is to break a
string into lines that can fit into the available display area. In
other words, width of a line must be less than or equal to
the variable width, whose value we calculated above. The
method used in this demo to perform this splitting is a very simple
and intuitive one. Also it looks only for line-breaks (CRLF) and
spaces. However, it is adequate for a demo and, if desired, a more
sophisticated and complete algorithm can be easily substituted.

The work of breaking down a string into lines is done by the
following three methods working together:

  • getFormatted -- makes sure that the entire
    String has been split into lines and returns an array
    of these lines.
  • stringSplitter -- generates one line at a
    time.
  • wordSplitter -- if there are words that are too
    long to be displayed in one line, this method breaks them into
    manageable strings.

Additionally, TextUI also provides support for
scrolling, stipulates the abstract methods that have to be
implemented by subclasses, and defines the variables required to
implement its functionalities. Some of these variables have to be
initialized by a subclass of TextUI.

As this article is about creating a customized text display, we
will not dwell on how to parse and tokenize a string. Instead we
will go straight on to our main topic.

The MenuLayer Class

The implementation of a menu, too, is highly platform-dependent.
On some devices, the menu always covers the entire screen. On others,
only a part of the screen is obscured. On some devices, there are no
system-enforced shortcut keys, while some have shortcut keys which
may conflict with those defined by the application. So it is
desirable to design one's own menu to ensure a uniform
look-and-feel. In this section, we shall create a simple menu which
will look the same and behave in the same way on all Java ME MIDP
2.0 compliant devices.

In our application the menu will extend Sprite. Our
menu will be designed to hold a number of options and the user will
be able to select one of them either through a cursor or by
clicking the designated shortcut key. Figure 3 shows what the menu
is going to look like.

The Menu
Figure 3. The Menu.

The main functions of the MenuLayer class are
described below.

Creating the Image for MenuLayer

The image is made up of the frames for the
Sprite. The only thing that changes in a menu is the
position of the cursor. So the number of frames will be equal to
the number of options and each frame will show the cursor on a
different option. The image is created by the
createMenuImage method:


//creates image for menu
public void createMenuImage()
{
    //this font is used to calculate menu dimensions
    Font font = Font.getFont(Font.FACE_SYSTEM, 
                Font.STYLE_PLAIN, Font.SIZE_SMALL);

    //number of options
    int length = options.length;

    int fontheight = font.getHeight();

    //index of the longest option string
    int longestindex = 0;

    //find the index of the longest option string
    for(int i = 0; i < length; i++)
    {
        longestindex = font.stringWidth(options[longestindex]) 
                > font.stringWidth(options[i]) ? 
                longestindex : i;
    }

    //width of longest option string
    int maxstringwidth = 
                font.stringWidth(options[longestindex]);

    //calculate menu size
    menuwidth = maxstringwidth + 20;
    menuheight = length*fontheight + 16;

    //get reference to the image to be drawn
    menuimage = Image.createImage(menuwidth * 3, menuheight);

    //draw the image

    //context for drawing image
    Graphics gm = menuimage.getGraphics();
    gm.setColor(backcolor);

    //draw the body of menu
    gm.fillRect(0, 0, menuwidth * 3, menuheight);

    gm.setColor(0x000000);//black

    //set the font that was used for size calculation
    gm.setFont(font);

    //draw the frames to be used for rendering the sprite
    for(int j = 0; j < length; j++)
    {
        //draw options for each frame
        for(int i = 0; i < length; i++)
        {
            //at the cursor
            if(i == j)
            {
                gm.setColor(cursorcolor);
                gm.fillRect(j*menuwidth, 8+fontheight*i, 
                        menuwidth, fontheight);
                gm.setColor(whitecolor);
                gm.drawString(options[i], j*menuwidth+10, 
                        8+fontheight*i, Graphics.TOP|Graphics.LEFT);
                gm.setColor(0x000000); //black again
            }
            //at other positions
            else
            {
                gm.drawString(options[i], j*menuwidth+10, 
                        8+fontheight*i, Graphics.TOP|Graphics.LEFT);
            }
        }
    }
}

Creating the Sprite

Now that the image is ready, we can set up the menu as a
Sprite. The Constructor of
MenuLayer first calls createMenuImage and
then instantiates a Sprite:


public MenuLayer(String[] options)
{
    this.options = options;

    //create the image for sprite that will be used as a menu
    createMenuImage();

    //make sprite with image created
    sprite = new Sprite(menuimage, menuwidth, menuheight);
}

Creating the Display

The subclass of TextUI that manages our display is
NewTextUI. This class converts text (the
textual matter that we want to display) into a
TiledLayer so that we can use a
LayerManager to show the text as well as the menu in a
co-ordinated manner. A LayerManager, as the "http://java.sun.com/javame/reference/apis/jsr118/javax/microedition/lcdui/game/LayerManager.html">
Java ME documentation
describes it, "simplifies the process of
rendering the Layers that have been added to it by automatically
rendering the correct regions of each Layer in the appropriate
order." Since both the text display and the menu are subclasses of
Layer, they can be added to a
LayerManager. We can then show only the text or the
text with the menu superimposed on it by calling the relevant
methods of LayerManager without getting involved in
the details of rendering them.

Let us see how the NewTextUI class performs the
tasks related to setting up the text display.

Creating the Image for the Text Layer

The image for a TiledLayer is composed of
individual tiles. We shall convert the entire text into a
single image so that there will be just one tile to consider. The
rendering of the image is handled by the
createTextImage method:


//create image for text layer
public void createTextImage()
{
    //adjust bottomindex for short text
    if(textsize <= numoflines)
    {
        bottomindex = topindex + textsize - 1;
    }

    //image for creating textlayer
    textimage = Image.createImage(width, textsize*fontht+2);

    //graphics context for rendering textimage
    Graphics gt = textimage.getGraphics();

    gt.setColor(0x000000); //black for writing text

    //for writing text set the font 
    //that has been used in line size calculation
    gt.setFont(f);

    //draw all the lines
    for(int line = 0;line < textsize;line++)
    {
        gt.drawString(contents[topindex+line], leftmargin, 
                topmargin+line*fontht, Graphics.TOP|Graphics.LEFT);
    }
}

Creating the TiledLayer

Now that the image is available, we can get the
TiledLayer by calling
getTextLayer:


//creates and returns a TiledLayer with one cell 
//and with text image as the only tile
public TiledLayer getTextLayer()
{
    //new Tiled layer with text image
    TiledLayer tl = new TiledLayer(1, 1, textimage, 
                width, textsize*fontht+2);

    //fill the only cell with the only tile
    tl.setCell(0, 0, 1);

    //make the layer visible
    tl.setVisible(true);
    return tl;
}

Note that our TiledLayer needs is only one
cell as we have just the one tile to display.

The LayerManager

The next step is to create a LayerManager and
insert the newly obtained TiledLayer into it. All the
tasks related to readying the TiledLayer and the
LayerManager are performed within the constructor of
NewTextUI:


public NewTextUI(String text, Display d, 
        TextUIDemo tuid)
{
    .
    .
    .
    createTextImage();
    textlayer = getTextLayer();
    manager = new LayerManager();

    //initialise the view window
    manager.setViewWindow(0, 0, width, 
                height+2*topborder+topmargin);

    //insert textlayer at the topmost position
    manager.insert(textlayer, 0);
    .
    .
    .
}

After the LayerManager has been instantiated, the
view window has to be initialized to display the text,
starting from the first line. The dimensions of the view window are
set to fit into the area designated for text.

Displaying the Text Within Its Frame

The paint method first paints the 'frame' around
the text display (Figure 2 above) and then
calls paint on manager (that is the
LayerManager) to render the layers:


//paint the frame, that is, the borders and the bars
//and, finally, ask manager to paint all the layers
public void paint(Graphics g)
{
    g.setColor(0xffffff);//white
    g.fillRect(0, 0, totalwidth, totalheight);

    //draw the borders, titlebar and bottombar
    g.setColor(0x30e030);//dark green
    g.fillRect(0, 0, totalwidth, titleheight);//titlebar

    g.fillRect(0, totalheight-bottomheight, totalwidth, 
                bottomheight);//bottombar

    g.setColor(0xffaaaa);//light red
    g.fillRect(0, 0, totalwidth, topborder);//top border
    g.fillRect(0, titleheight, totalwidth, 
                topborder);//border below titlebar

    g.fillRect(0, totalheight-bottomborder, totalwidth, 
                bottomborder);//bottom border
    g.fillRect(0, totalheight-bottomheight, totalwidth, 
                bottomborder);//border above bottombar

    g.fillRect(0, 0, leftborder, totalheight);//left border
    g.fillRect(totalwidth-rightborder, 0, rightborder, 
                totalheight);//rigt border

    g.setFont(tf);//set font for writing on the bars
    g.setColor(0x000000);//black for title
    g.drawString(title, width, 3, 
                Graphics.TOP|Graphics.RIGHT);//write title

    //if first line of display is first line of text 
    //then set white
    if(topindex==0)
    {
        g.setColor(0xffffff);//white
    }

    g.drawString(String.valueOf(topindex+1), orgx,3, 
                Graphics.TOP|Graphics.LEFT);//write top line index

    //if last line of display is last line of text 
    //then set white otherwise black
    g.setColor(bottomindex == textsize-1 ? 0xffffff : 0x000000);

    //write bottom line index
    g.drawString(String.valueOf(bottomindex+1), orgx, 
                totalheight-bottomheight+3, 
                Graphics.TOP|Graphics.LEFT);

    //paint all the layers
    manager.paint(g, leftborder, titleheight+topborder);
    flushGraphics();
}

When the application is launched the layer manager has only the
text layer. So the text is displayed within its frame as shown in
Figure 4.

Text display
Figure 4. Text
display.

The line number of the first line being shown is displayed on
the title bar, and the last line's line number is on the bottom
bar. When further scrolling in a given direction is not possible,
the corresponding line number turns white. A right-justified title
of the display is also written on the title bar. Conventionally the
numbers should have been written on the right and the title on the
left. This minor departure from convention is only to emphasize the
flexibility available to us.

Now that we have taken care of the visual aspects, let's see how
things work together by implementing the functionalities of the
text and the menu described below.

Scrolling the Text

The 'view window' of the LayerManager defines the
location and size its visible portion. Scrolling the text can be
achieved simply by moving the view window up or down. The unit
distance for movement is determined by the height of the font used
to render the text. The following methods in NewTextUI
perform this task:


//scroll up the text
protected void scrollTextUp()
{
    if(topindex > 0)
    {
        topindex--;
        .
        .
        .
        //topindex is decremented
        //so view window moves up
        manager.setViewWindow(0, topindex*fontht, width, 
            height+2*topborder+topmargin);
        updateTextScreen();
    }
}

//scroll down the text
protected void scrollTextDown()
{
    if(bottomindex < textsize - 1)
    {
        topindex++;
        .
        .
        .
        //topindex is incremented
        //so view window moves down
        manager.setViewWindow(0, topindex*fontht, width, 
            height+2*topborder+topmargin);
        updateTextScreen();
    }
}

Showing the Menu

When the Options command on the text display screen is
selected, the showMenu method in
NewTextUI is called, which performs the necessary
housekeeping tasks and then shows the menu:


//setup and show the menu
protected void showMenu()
{
    //remove commands from text canvas
    //because all user inputs now are for menu only
    tuid.removeCommands();

    //set flag so that key events can be routed properly
    menushown = true;

    //initialise cursor position
    menu.setCursorIndex(0);

    //remove existing second layer if any
    if(manager.getSize() == 2)
    {
        manager.remove(manager.getLayerAt(0));
    }

    //if menusprite hasn't already been obtained
    //then get it
    if(menusprite == null)
    {
        menusprite = menu.getSprite();
    }

    //select the first frame so that
    //cursor is shown on first option
    menusprite.setFrame(0);

    //set initial position of menu
    menusprite.setPosition(posx, posy);

    //insert sprite into layer manager as topmost layer
    manager.insert(menusprite, 0);

    //ask layer manager to paint the layers
    //as menusprite has been added as topmost layer
    //it is shown superimposed on text
    manager.paint(g, leftborder, titleheight+topborder);

    //flush to the screen
    flushGraphics();
}

Figure 5 shows the menu superimposed on the text display.

The menu is shown
Figure 5. The menu is
shown.

Scrolling the Cursor on the Menu

When the menu is visible on the screen and the Up or
the Down key is pressed, the keyPressed
method of NewTextUI catches the key event and calls
the appropriate method in MenuLayer. Since the sprite we use
for the menu has a frame for each position of the cursor, all we need to
do to 'animate' the cursor is move from one frame to the next or to
the previous one depending upon the direction of scrolling. The
code snippet below shows the method for scrolling up:


//scroll the cursor up on the menu
public void scrollUp()
{
    //cursor index decremented and
    //goes to highest value from zero
    cursorindex = cursorindex == 0? 
        options.length - 1 : --cursorindex;

    //show the previous frame (frame is circular)
    sprite.prevFrame();
}

Note that our job has been made very easy as the
Sprite class has two methods -- prevFrame
and nextFrame -- for traversing the sequence of
frames. In addition to changing the frame, scrollUp
updates the value of cursorindex to keep track of where
the cursor is positioned at any given time.

The method for scrolling down is scrollDown and it
operates in a similar fashion calling nextFrame to
show the next frame in sequence.

Selecting a Menu Option

Menu options, as we know, can be selected in two different ways.
The first is by positioning the cursor on the desired option and
clicking the selection key. This can be the middle button in a 4+1
navigation scheme and/or, in keeping with gaming convention, the
number key 5. In terms of the terminology used by
Canvas, we are interested in the
Canvas.FIRE key. The keyPressed method of
NewTextUI handles this key event when the menu is on
screen:


//when a key is pressed once
protected void keyPressed(int keycode)
{
        .
        .
        .
        //menu cursor control
        switch(getGameAction(keycode))
        {
                .
                .
                .

                case Canvas.FIRE :
                        menuAction(menu.getSelectedIndex());
                        break;
                .
                .
                .
        }
        .
        .
        .
}

The MenuLayer keeps track of the cursor position
and this value is returned when getSelectedIndex is
called. To handle menu item selection, the keyPressed
calls menuAction with the cursor index as parameter
and, depending on the value of cursor position, appropriate action
is taken.

The second method of selecting an option from the menu is to use
the respective shortcuts. The numbers in square brackets shown
against the options are the applicable shortcuts. To select Option
1, for instance, the number key 1 can be pressed. This key event
also is handled by keyPressed in
NewTextUI and menuAction is called with
the proper value of parameter:


//when a key is pressed once
protected void keyPressed(int keycode)
{
        .
        .
        .
        switch(keycode)
        {
                //menu action cases
                case Canvas.KEY_NUM1 :
                        menuAction(0);
                        break;

                case Canvas.KEY_NUM3 :
                        menuAction(1);
                        break;

                case Canvas.KEY_NUM0 :
                        menuAction(2);
                        break;
                .
                .
                .
        }
        .
        .
        .
}

Within the menuAction method, the required action is
taken and the menu is removed from the layer manager. Also the
commands that were removed when the menu was popped up are added
back and the menushown flag is cleared. This is shown
below:


//actions as per menu option selected
private void menuAction(int actioncode)
{
    Alert optionalert;

    switch(actioncode)
    {
        case 0 :
            menushown = false;//clear flag

            //remove menu from layer manager
            manager.remove(menusprite);

            tuid.addCommands();//add back commands

            //dummy action
            optionalert = new Alert("Text Canvas", 
                "Option 1 selected", null, AlertType.INFO);
            optionalert.setTimeout(2000);
            display.setCurrent(optionalert, this);

            updateTextScreen();
            break;

        case 1 :
            menushown = false;
            manager.remove(menusprite);
            tuid.addCommands();
            optionalert = new Alert("Text Canvas", 
                "Option 2 selected", null, AlertType.INFO);
            optionalert.setTimeout(2000);
            display.setCurrent(optionalert, this);
            updateTextScreen();
            break;

        case 2 :
            menushown = false;
            manager.remove(menusprite);
            tuid.addCommands();
            updateTextScreen();
    }
}

A Floating Menu

Sometimes, when I open a menu, I want to take a final look at
the original document before choosing an action. On mobile phones
this can be a problem as some menu implementations completely cover
the screen. Even on those that don't, the small display area means
that a menu is likely to obscure most of the screen. At such times
a menu that can be moved around would be very useful. In this
section we make our menu movable.

Before we look at the code, we need to choose the user action
required. The most obvious choice would be the navigation keys. But
we have already decided to use the Up and Down
keys to move the cursor on the menu. A popular practice for games
is to use the numeral keys 2, 4, 6 and 8 for movement -- 'up',
'left', 'right' and 'down' respectively. So our application will also use
these keys for menu movement.

The first thing that we have to do is listen for the events
corresponding to the keys that move the menu around. So the
following code is added to the keyPressed method in
NewTextUI:


.
.
.
//menu movement cases
case  Canvas.KEY_NUM2 :
menuUp();
break;

case  Canvas.KEY_NUM8 :
menuDown();
break;

case  Canvas.KEY_NUM4 :
menuLeft();
break;

case  Canvas.KEY_NUM6 :
menuRight();
break;
.
.
.

We also add a keyRepeated method to
NewTextUI so that holding one of the 'movement' keys
down will make the menu move continuously:


//when a key is held down
protected void keyRepeated(int keycode)
{
    //if menu is being shown then 2/8/4/6 keys 
    //refer to movement of menu
    if(menushown)
    {
        switch(keycode)
        {
            //menu movement cases
            case  Canvas.KEY_NUM2 :
                menuUp();
                break;

            case  Canvas.KEY_NUM8 :
                menuDown();
                break;

            case  Canvas.KEY_NUM4 :
                menuLeft();
                break;

            case  Canvas.KEY_NUM6 :
                menuRight();
        }
    }
    //otherwise they refer to the text
    //let super class handle it
    else
    {
        super.keyPressed(keycode);
    }
}

We now need to decide how much to move the menu for each key
press. This value corresponds to variables posdelta and
negdelta as defined in NewTextUI and, in this
example, both have been set to 2 pixels. Upward and leftward
movements will use negdelta while downward and rightward
movements will use posdelta. Changing the values of these
variables will change the granularity of movement.

The Layer class has a method which makes it very
convenient to control the menu's movement. Since a sprite is also a
layer,our menu movement methods in NewTextUI call this
method. Note that the move method of
Layer class takes two parameters - dx and
dy - that define, respectively, the horizontal and
vertical distances for movement. So, for moving the menu up
dy is negative and, for moving it down, dy is
positive. The motion of the menu is controlled by the following
methods:


//move menu up
private void menuUp()
{
    menusprite.move(0, negdelta);

    //repaint to show menu at new location
    updateTextScreen();
}

//move menu down
private void menuDown()
{
    menusprite.move(0, posdelta);

    //repaint to show menu at new location
    updateTextScreen();
}

//move menu left
private void menuLeft()
{
    menusprite.move(negdelta, 0);

    //repaint to show menu at new location
    updateTextScreen();
}

//move menu right
private void menuRight()
{
    menusprite.move(posdelta, 0);

    //repaint to show menu at new location
    updateTextScreen();
}

That's it. Now the menu will move around on the screen if one of
the "movement" keys (2/8/4/6) is pressed. Note that the menu will
keep moving until it goes out of the screen as there is no limit
check. If desired the movement can be easily stopped at a screen
edge by limiting the bounding co-ordinates of the sprite
appropriately. For example, to stop the menu at the upper edge of
the screen, we have to ensure that the vertical position of its
upper-left corner (as returned by getY method of
Layer) does not become negative.

Figure 6 shows the display with the menu having been moved to a
new position.

<br "Menu moved to a new location" />
Figure 6. Menu moved to a new
location.

Conclusion

We've seen one implementation of a custom UI on a mobile phone.
There are other ways to develop similar UIs and I hope you will
experiment and find the approach that best suits your requirements.
Remember that the technique shown here can be used for non-text
displays and UIs too. You can implement an 'arrow' cursor as
follows:

  • Make an image.
  • Create a sprite using the image.
  • Define a reference pixel. The tip would be the natural
    choice for an arrow.
  • Write methods to move the cursor.
  • To determine the position of the cursor use
    getRefPixelX and getRefPixelY of
    Sprite class. This will tell you what the cursor is
    pointing at.
  • Take desired action when the Canvas.FIRE key is pressed.

This year, at JavaOne, an ME equivalent of Swing was announced,
the "http://weblogs.java.net/blog/terrencebarr/archive/2008/04/coming_soon_swi.html">
Lightweight UI Toolkit
. No doubt LWUIT will provide the
platform for sophisticated and consistent user interfaces. However,
if you need a simple lightweight UI that needs to be different from
the standard lcdui screens, look and act the same on all
Java ME (MIDP 2.0) compatible phones and is easy to integrate into
your application, you will find the approach shown here worth going
with.

Acknowledgment

"http://www.apress.com/book/bookDisplay.html?bID=426">Beginning
J2ME: From Novice to Professional
by Sing Li and Jonathan
Knudsen has an excellent chapter on custom UIs. The design
described in this article is an extension of the basic scheme
outlined by Li and Knudsen.

Resources


width="1" height="1" border="0" alt=" " />
Biswajit Sarkar is an electrical engineer with specialization in Programmable Industrial Automation. Biswajit is the author of "LWUIT 1.1 for Java ME Developers" published by PACKT Publishing.
Related Topics >> Mobility   |   Programming   |