The results of the ~40 category 0 tests are not quite satisfying. Compared to other platforms the errors and warnings are huge.
The reason is currently not the quality of OpenOffice.org on MacOS, it is more an issue on how OpenOffice.org behaves on VCL TestTool.

All issues link.

Here we go:

Automated testing before cws aquavcl05 was even worse because of the issue:
i85769 testtool: Server Timeout because focus got lost... It got integrated into the master in DEV300m2.

i86736 testtool: AQUA: Slot is ignored, if 2 windows are open
How to reproduce:
Run the test qa/qatesttool/base/update/xforms_updt.bas. It will fail in tPropertyBrowser with: Edit : Window/Control could not be found. Which is a follow up from silently failing in openning a toolbar.
Research:
If you run just the testcase tPropertyBrowser, then it works if you start from the 'start center' aka backingwindow; But it fails if one (empty) writer document is already open on starting the testcase. It also works if I manually click into the window before opening the toolbar;
There are several tests with this behaviour.

i86746 testtool: AQUA: Font listbox seems to be empty for testtool
How to reproduce: qa/qatesttool/graphics/update/d_updt.bas Run just the testcase tiTextToolbar which results in: Schriftart : Entry number Select is too large in 3. Max. allowed is 0 and debugging with .getitemcount gives 0. Only way to work, is to open the listbox manually, before using it.

i86749 testtool: AQUA: Button state on dialog not updated until manually set focus to it
qa/qatesttool/spreadsheet/update/c_updt2.res Just run the testcase tDataDataPilot it fails with: Window does not exist:Datenfeld 58871. Which is because the option button is seen as enabled by testtool. It works correct if the dialog is clicked on before asking for button state.

Due to the start center behaviour on MacOS X there is something that can not be workarounded right now:
To be able to get always a window in OpenOffice.org, the start center/quick starter behaviour needs to get suppressed. This is done by API in the function: hDisableQuickstarterAPI. This makes sure, that if the last document was closed, there ist not just the menu for OpenOffice.org, but also a window - the backing window, or start center available, without the VCL TestTool would not work.
So in a case OpenOffice.org crashes and got restarted by VCL TestTool, it is no more controllable, as soon as the last window got closed; As seen in b_updt.bas, with lots of: Active : UNO URL "slot:5501" could not be executed: No ActiveFrame on desktop.
The quick way to solve this, would be: i86938 implement commandline switch -noquickstarter
The long term goal is to get: i86540 testtool: provide function trigger in cases testtool starts OOo on his own after a crash

i87290 StarDesktop.CurrentComponent fails when executed by macro started by testtool
When executing the test qa/qatesttool/framework/first/topten.bas function tSmokeTest, which loads the 10er-test document: qa/qatesttool/framework/first/input/10erTest_680.sxw. The call to StarDesktop.CurrentComponent returns null, if the start of the macro was triggered by VCL TestTool. Result: The macro stops execution. If the macro is executed manually it works.

i82427 Mac OS X (Aqua): Drawing function does not draw anything
 t_TB_Form in qa/qatesttool/writer/update/w_updt.bas

i54265 DocumentConverter: Button "Show Logfile" has no Help-ID
There was a workaround to use an accelerator, but that does not work on MacOS X right now...

There is one issue that can be workarounded:
i86137 testtool: AQUA: typekeys does not work in first start wizard
How to reproduce: Let VCL TestTool start OOo for the first time, so the first start wizard is used. It will break with: NextBtn : Window/Control could not be accessed. Disabled in master.inc line 369, since  LicenseText.TypeKeys "<MOD1 a>" , TRUE is ignored (CTRL+A), any other like <down>, too. Workaround is to use the 'Scroll Down' button, until it is disabled.

These are the most obvious ones, there are still some more, but they vanish if I rerun the test, and I hope they disappear, when the others are fixed ;-)

Other issues are related to not jet adapted test scripts for MacOS because some functions are not (jet) available:
i86992 aqua: No thesaurus available ...and no Hyphenation
i81545 File -> Print|Printer Settings: Properties button disabled

There is one annoying issues to the VCL TestTool itself: i86067 testtool: AQUA: horizontal scrollbar overlaps status area

List of working tests:
f_updt_spadmin.bas
f_updt_standardbar_1.bas
f_updt_standardbar_2.bas
f_updt_standardbar_4.bas
f_updt_standardbar_5.bas
f_updt_standardbar_6.bas
f_updt_templates.bas
f_updt_wizard_agenda.bas
f_updt_security.bas
f_updt_scriptnames.bas
f_basic_first.bas
f_scripting_basics.bas
f_updt_basic.bas
f_updt_basicorganizer.bas
f_updt_customize.bas
f_updt_help.bas
f_updt_optionitems.bas
f_updt_wizard_euroconverter.bas
f_updt_wizard_fax.bas
f_updt_wizard_firsttime.bas
f_updt_wizard_presentation.bas
f_updt_wizard_webpage.bas
f_lvl1_loadsave.bas
m_updt.bas
d_opt.bas
i_updt_1.bas
oasis_graphics_update.bas
ch2_updt_calc.bas
ch2_lvl1_wizard.bas

List of tests with errors due to MacOS X:
f_basic_spectemplate.bas i86997
topten.bas i87290
f_updt_wizard_documentconverter.bas i54265
f_updt_wizard_mailmerge.bas i81545
d_updt.bas i86746
i_updt_2.bas i86746
w_updt.bas i86736
c_updt1.bas not jet evaluated error with 'Font button'
c_updt2.bas i86746
xforms_updt.bas i86736
b_updt.bas i86938

List of tests failing because of general errors
f_updt_standardbar_3.bas i86276
f_updt_wizard_letter.bas i87245
b_updt_report_designer.bas issue also seen on other platforms

Test System: MacOS X 10.5 CWS aquavcl05 and DEV300 m3, Intel,
Testtool: from http://qa.openoffice.org/ooQAReloaded/AutomationTeamsite/ooQA-TeamAutomationBin.html
Test environment from cvs: HEAD

tags: automated_tests openoffice.org qa