RoboGeek

Automation of web browsers and the java plugin I've been working on a tool meant to help the java plugin SQE team automate their testing.  I've finally "finished" the tool, and want to write a bit about it.

The first question is "why"?

Why do this?  I've covered it before, so here's a summary:

1. "Testing is never finished, only abandoned", which I take as "there's a given budget for testing, and automation will expand the testing you can do with that budget".
2. There's no existing tool for linux or solaris which will automate java applets and web browsers.  Especially not for the modern browsers (mozilla, firefox).
3. Automating the browser/applet scenarios are the biggest pain factor in the java quality team, hence we'd get the biggest payoff by resolving their pain.
4. The plugin has a different implementation on each platform, hence it has to be tested across platform.
   

We considered several routes, and selected the one that had me rummaging around the innards of Mozilla for a long time.  The routes were:

1. Use javascript inside mozilla/firefox to find out the GUI component tree.  Send the GUI component tree to a Java program.  Use Robot from the Java program to send events at the browser.
2. Develop a C/C++ extension to mozilla that does the same as Robot, and write something like Jemmy in Javascript.
3. Use the platform accessibility API's to discover the GUI components, get them into a Java program, and use Robot to send events.
4. Continue using the low level windowing system calls to get GUI components, but this gives us limited information (especially on X11).
   

Route (4) has been very painful mostly because of the limited that XQueryTree gives you on X11.  When you automate an interaction against some component, you need to know much more about the component tree than the x,y,width,height rectangle (which is all XQueryTree gives you).

Route (3) would be really cool, as it would work with any GUI application.  Unfortunately on the X11 side of the world, accessibility is a very mixed story.  The bright spot of X11 accessibility is that GNOME has an accessibility interface (developed by some Sun engineers), and that the KDE folk have apparently agreed to adopt the GNOME accessibility API.  Since Windows and Mac OS X already have good accessibility support, the flourishing of the GNOME accessibility API will eventually mean that accessibility is available "everywhere".  But "eventually" wasn't good enough for us.

Route (2) would be a more complex way to hook into Mozilla's innards.  So I chose route (2).

It took a lot of research, and along the way my eyes became opened to a major flaw in the Mozilla project.  Namely, DOCUMENTATION.

The source tree is millions upon millions of lines of code -- written in around 8 different programming languages (C, C++, javascript, XUL, XBL, HTML, CSS, IDL, ...?) -- there's somewhere around 0 internal documentation -- and somewhere around 1/2 of mozilla.org is stale out-of-date documentation.

All that goes together to make an extremely steep learning curve to being productive in understanding Mozilla.  Further, my project was not the typical use of javascript or any of the API's, and most of the documentation I found had limited use even if it was up-to-date.  There were two resources of great use.  First was an Sun engineer in the Beijing office who answered dozens of emails from me.  The second was the netscape.*.mozilla.* newsgroups.

For years before I had been looking at the DOM Inspector tool, noticing how it had screen x,y coordinates for everything in the Mozilla browser, and desparately wanting to be able to use that.  Well, it's the DOM Inspector tool which provided me some of the information.  Unfortunately the DOM Inspector is also a very tough piece of code to understand ... some of the crucial bits are buried in the object model for the tree, and it's tough, even now, unraveling how that object model works.

Would it be too much to ask for a few stinkin' comments????!?!?!?!

(2005-04-12 15:45:46.0) Permalink Comments [1]