RoboGeek

Test tools across platforms I must apologize for getting distracted with PHP.  Let me get back to the Automation mission I started with.

The nastiest problem we've been facing for years in J2SE SQE is automating Plugin and JaWS testing.  I am close to finishing a tool which handles this, though I don't know if I'll ever be able to release it.  Knock on wood and pray to the gods that be, let's hope this can be released through the Jemmy project.

Testing a Swing or AWT application is simple.  This is because we can browse through the AWT Component tree, discover Components, and get their location.  All of the Java GUI test tools use that same basic approach.

This approach works great when the GUI under test is contained within one JVM.  But as soon as you need to interact with a window outside the JVM, you quickly become blind as a bat.  To interact with a GUI component using Robot, you need to know it's location so you can click on it.  BTW, this holds true for some GUI features inside the current JVM, e.g. menu components don't export their location, and if you want to click on the arrow button of a scrollbar you're out of luck.

In what J2SE SQE tests the main area this issue shows up is, as I said, Plugin and JWS.  It also affects drag&drop testing as some test scenarios there involve D&D between applications.

Robot could click on anything, so long as it knows the location.  So the key clincher is to learn the location of the GUI components in other applications.

We tried using native windowing system API's (on Windows, EnumWindows and EnumChildren, while on X11 it's XQueryTree).  This worked fairly well, but has limitations, especially on X11 where XQueryTree only returns rectangles and no higher level information about the rectangles.  Further XQueryTree can only find Window objects, meaning that toolkits that don't use Window objects (such as Mozilla's GUI toolkit) won't have their internal structure visible to XQueryTree.

An approach I looked at approximately a year ago, and discarded at the time, was to become an Accessibility Tool.  The Accessibility support helps with the devices that disabled people use to facilitate their computer use.  Accessibility support means that some software tool is querying the system for nitty-gritty details about the components in all applications on the screen.  Among the data returned to an Accessibility tool is the screen location, GUI class, etc.  This is the perfect sort of information needed to solve the problem I outlined above.

Unfortunately it appeared the presence of Accessibility support was spotty.  It was available for Windows, but not for X11.  On the X11 side some Sun engineers are working with both the GNOME and Mozilla projects to implement Accessibility in both tools.  I'd also seen an announcement that KDE would be absorbing the Accessibility implementation from GNOME.  The latter is great news for compatibility, no doubt.  Unfortunately this Accessibility support was still in development, and wasn't shipping as part of the default/base installation on any system, but maybe that's changed since then.


(2005-03-11 16:44:32.0) Permalink Comments [1]