Measuring UI Responsiveness

Goal

Be able to measure exactly the time between invocation of a user action and finishing the action in the IDE. Typical user actions are

• invoking menu (pull-down, pop-up, or submenu),
• opening file,
• invoking dialog or window,
• expanding tree, etc.

It is desirable to have one general way to measure all kinds of actions. It should be simple to use and should not affect performance of the application.

Technique

User actions are triggered mostly by pressing a mouse button or a key. The action can be considered as finished after some painting is done and the program is responsive (idle).

So technically, we can watch for appropriate events to get the time when user action begins. Then we may watch the event queue (to see when it becomes idle) and painting events (whether some occurred during processing the event queue). Knowing the time between user event and event queue emptying, we have a good approximation of the response time. How can this be implemented?

This kind of measuring can be done easily by hacking the JDK classes directly (the sources are bundled with the JDK download). Full diffs are available here: for JDK 1.4.0, for JDK 1.4.1 and for JDK 1.4.2.

We start with java.awt.EventQueue, adding some fields:

    public static long firstEventTime;
    public static long userEventTime;
    public static boolean paintOccurred;
    private static int ncount;

Then we hack getNextEvent method. When the EventQueue starts dispatching events (after it was waiting on some), we remeber the current time:

    firstEventTime = System.currentTimeMillis();

To recognize user event, we may use the following code executed before getNextEvent returns:

    int id = eqi.event.getID();
    if (id == 501 || id == 502 || id == 401) {
        System.out.println("-> user event ["+id+"]");
        userEventTime = firstEventTime;
    }

(501 is whatever mouse down, 502 mouse up, 401 means pressed key.)

Note that we actually don't use the time of event creation, but the time of starting dispatching the events in EventQueue. More accurate would be to use e.g. InputEvent.getWhen() for getting the time, but it turned out it was not reliable on some systems. Using the event queue start dispatching time instead is a good approximation, differing just about in several milliseconds.

To watch whether some paint event occurred, we can simply hack JComponent.paint method by adding:

    java.awt.EventQueue.paintOccurred = true;

The rest is just to print results at moment the event queue is done and going to wait:

    if (paintOccurred && userEventTime > 0) {
        paintOccurred = false;
        long time = System.currentTimeMillis();
        long diff = time - userEventTime;
        if (diff > 40000)
            userEventTime = 0;
        else
            System.out.println(Integer.toString(++ncount)+". paint done "+diff);
    }

(Note we use a counter variable to number the painting events for better orientation in the output.)

To sum up, for any operation processed in event queue and painting something, we get the elapsed time from last user event.

There is actually one more thing to do: recognizing menu selection as user event. It is user event because it may invoke a submenu. But it may be triggered just by moving the mouse, not pressing anything -- so it is not covered by the code above. We may hack JMenu and JMenuItem by adding following code to their menuSelectionChanged method:

    if (java.awt.EventQueue.firstEventTime > java.awt.EventQueue.userEventTime) {
        java.awt.EventQueue.userEventTime = java.awt.EventQueue.firstEventTime;
        System.out.println("-> user event [menu]");
    }

(Note we must hack both, as JMenu does not call super.)

Usage

• Compile the four changed classes (get diffs for JDK 1.4.0 or for JDK 1.4.1).
• Pack the class files into a zip file, keeping the right package structure inside.
• Modify netbeans.conf file (located in etc directory of NetBeans) -- add the following line to it (without the < > brackets):
  -J-Xbootclasspath/p:<path_to_that_zip_file>
• Run the IDE.
• Try some operations with mouse and keyboard, check the console to get used to the output.

Note that for operations triggered by "mouse down" event (like selecting nodes, expanding trees, invoking pull-down main menus), you should keep the mouse pressed until the operation is finished (so the "mouse up" event does not kick in in the middle).

Please send any comments to nbdev.