RoboGeek

RoboGeek's (David Herron) Weblog: co-developer of Robot and several other things related to Java testing.

All | Corporate Blogging | ecological computing | General | Java | Podcast

« Testing Java GUI... | Main | PHP versus Java/JSP/... »
20050307 Monday March 07, 2005

The bottom level of GUI automation tools

My previous posting was probably too long and circuitous. Just to prove that I'm overly obsessed with GUI automation, I want to discuss the bottom layer. With the foundational principles of GUI automation covered, I ought to be able to discuss the higher level issue more freely. That's my plan anyway...

Seven years ago when I was hired by Sun, my manager said "Your job will be to test AWT. Oh, and if you want to avoid manual test execution, you'll find a way to automate the tests." Since the last thing I wanted to do was shift my career to being a tester, I spent more time looking for automation technology than writing tests. Please don't tell my manager that!

At the time there was nothing suitable. In the commercial arena we had WinRunner and X/Runner but I'd had a previous bad experience with X/Runner and didn't want to use it. Basically, X/Runner is specialized to testing Motif applications and the scuttlebut was it wasn't able to deal well with Java applications (even though AWT was based on Motif). My previous experience had been at Mainsoft where I'd spent two years working on a WIN32 implementation for X11, and we tried to use X/Runner for test automation. It didn't work terribly well because X/Runner had no clue where any of our components were, and could only work on absolute coordinates.

Sun had a product named JavaStar that I thought was peculiarly useless while being almost useful. It was unable to test AWT applications because it only sent fake events through the AWT/Swing EventQueue, and in my testing of that idea the AWT components would not react to those events. The AWT components would only react to events that arrived through the normal operating system channel. I did have a meeting with that team and was unable to get their attention on this flaw.

Now, I promised a bit of discussion of the fundamental level of GUI automation. Since I hadn't been able to find an existing tool, I had to study this fundamental level. This study eventually led to the java.awt.Robot class.

I created, on X11, a class I called NativeEvent which had around five methods.

  1. Mouse-Button press
  2. Mouse-Button release
  3. Key press
  4. Key release
  5. Mouse move (set-location)

I knew these were the fundamental operations of any GUI interaction, and that if I could perform those fundamental operations from Java, that they could be used to create any GUI interaction. It was an Alan Turing sort of moment, where I knew the fundamental state machine for any GUI interaction.

So with this implementation in hand, I presented it to my immediate team. They liked it enough, but wanted to see some help from the AWT developers. So I wrote a summary, and went to meet with the developers. In the hallway on the way there they met me and said "David, we have this idea for GUI automation and wonder what you think of it". It was one of those great minds think alike moments, because they had the same five methods written on a sheet of paper.

After a moment of "hey, we've got the same idea" ...well... We turned to collaboration, with Robi Khan doing the Windows implementation and I did the Solaris implementation.

I've written about this before, but we purposely kept Robot to those minimal methods. We didn't want to embed a GUI test framework in Java (due to size), we didn't want to force the development community to a specific method of testing, etc. We left it up to the community to develop this idea further, and they did.

I like to think of those five methods as the GUI automation equivalent of assembly code. You probably don't want to code your automation with those methods, but you can, and you can certainly build amazing contraptions with only those five methods. But you'll be more efficient if you use a higher level tool. (2005-03-07 10:26:33.0) Permalink Comments [1]

Trackback URL: http://blogs.sun.com/robogeek/entry/the_bottom_level_of_gui
Comments:

And I want to thank you for those 5 methods! ;) They make my automation with jfcUnit work *most of the time*. For example, on Windows, top level menus do not "stick" so that when you automate the clicking of a menu bar and the menu shows up, then you move the mouse down to click on a menu item, sometimes the menu disappears, causing a failure. I don't know if this is directly related to issues with the Robot class firing mousemoves incorrectly/inconsistently, but at least with jfcUnit, even when I am purposely moving the mouse straight down VERY Slowly, I can see the menu disappear sometimes, and sometimes it stays up. It's very odd, and sadly causes inconsistent behavior for running tests dependent on menu navigation. I've worked around it by creating a navigateMenu() mehtod that goes through each item one at a time and right before it moves to the item, it checks the popup menu (if any) and if it somehow is not visible, sets it to visible again. This works about 95% of the time, but I am still seeing the occasional closure. I think I just need to give the popup more time than 100msecs to show up. Usually it does, but maybe for some reason it takes more. At any rate, I have built a GUI harness that I am hoping to open source sooner than later, probably within a few months. It basically uses the premise of "parts" where you have a repository of reusable test parts, then you drag/drop these parts into test cases in the order of execution. Once dropped you can add resources (string key/value pairs) that the code can get ahold of so that each part, even if its the same part, can have it's own set of key/value resources (currently only string values are allowed, but have in the back burner to allow tokens from resource bundles). You can then right-click and run any testcase or group of testcases. The test suite (where you drop parts into testcases) allows the dynamic creation of folders in a tree to allow better organization of test cases into test suites. We also use TestLink and "mimic" the folder structure in the harness to those we use in TestLink to allow somewhat decent syncing of our TestCase descriptions to our actual automation testcases. When you run testcases, it automatically compiles the associated java source files for you, so that you do not have to restart the harness each time you make changes to the associated part java source files. Each part is really a java source file on disk. The harness also provides preferences you set. The location of where to find the repository parts (source files), the output directory to compile them to, and also the premise of a "golden files" directory and a "work" directory is in place, so that in your test code you make a call like getFile("someFile") and it checks the work directory first. If it's not there, it checks the golden directory. If it is there, it copies it to the work directory and returns you the work directory File object to the file. If it's not found at all, the test fails. Because the harness is a Swing UI, I have made it so I can embed it directly into our application. I simply build up test parts while our application is running. Without ever shutting down or restarting the app, because of the auto-compile in the harness of the associate source files, I can actually create Parts, then create their associated .java source files, then drag/drop those parts into a testcase, then run it. All without ever shutting down or restarting. If the test doesn't work right, I make a quick change and simply run the test again. The benefit is is saves huge time on having to restart the application and/or the harness. What do you think?

Posted by Kevin on March 08, 2005 at 10:25 PM PST #

Post a Comment:

Name:
E-Mail:
URL:

Your Comment:

HTML Syntax: NOT allowed