RSS Feed
Bookmark to Delicious
Monday, 26 May 2008
accessibility
api
architecture
automated_tests
base
beta
build
calc
chart
code
community
database
development
directx
document
download
draw
eis
events
export
extensions
features
filter
framework
graphics
gsl
gsoc
gullfoss
import
impress
installation
iso26300
java
l10n
localization
mac
netbeans
odf
ooo
ooocon
ooocon2008
ooocon2009
open
opendocument
openoffice
openoffice.org
pdf
performance
porting
qa
quality
quaste
release
renaissance
security
snapshot
software
specification
spreadsheet
staroffice
statistics
sun
survey
testing
toolkit
tools
usability
user-experience
video
web
wiki
writer
xml
- show all articles
Monday, 26 May 2008
Thorsten Ziehm
As you still know I am the manager of the Quality Assurance Team at Sun Microsystems for StarOffice and OpenOffice.org and everyday I work and do my daily business with each developer milestone. I do the same with the ones of the OOo 3.0 code line. Why I do not have any objections? Why I am not afraid and why I am not a friend of writing 'please do not use the Beta for your daily work, you can destroy your documents'? Please read why I am so confident.
In the past years since OOo 2.0 Beta the development of OOo did a major step to increase the stability and quality of our product. Processes and guidelines were integrated to get a stable version every day. Some groups in the community say, these hinder the contribution to the project. I do understand this. But the great benefit for all of us is, that every contributor and each user of OOo can get stable releases every 3 months. And with the change to 'more frequent feature releases' every 6 months also new features are integrated in the new versions.
The number of users and interested people on OOo increased the past years dramatically. In some weeks more than 1 million downloads per week were registered. From these numbers the project was fare fare away when starting with OOo 2.0 Beta begin 2005. Beside the increased numbers of users of and contributors to OOo the number of incoming issues in IssueTracker decreased. In 2005 round about 1800 issues were reported each month. Last year the submission rate was at 1050 issues and in the past months before the announcement of OOo 2.4 and OOo 3.0 Beta the numbers decreased to 950 issues per month. Now with the announcement of OOo 3.0 Beta the numbers increased a little bit. But in general the numbers are fare fare away from the numbers at OOo 2.0 Beta release.
Since the release of OOo 3.0 Beta some more than 300 issues are reported on this version. But this does not increase the numbers for May in general. So I have to think the users switch to the Beta and report their issues on that version and do not report it on an older release. Currently less than 5 issues are identified as show stopper issue for the Beta Refresh end of June.
What was done to get the higher quality since OOo 2.0?
First of all the concentration on developing with Child Work Spaces (CWS) are very important. To bring in new features and changes only when they are tested in a separate build (the CWS). Only when it is finished the QA approval will be given and the CWS can be integrated into the master code line. This hinder to bring in unfinished features and perhaps broken functionality. Regressions are still there on the master code line. This couldn't be reduced to zero. But it could be minimized.
The second quality initiative was to concentrate on stability in OOo 2.0.3 and 2.0.4. With the Crash Reporter feature all most critical crashes were identified and fixed. This tooling is used for each release and all critical issues from the last releases are fixed in the next release. In 2.x time-frame nearly 400 issues reported over the crash reporter tooling by the user of OOo could be fixed. These 400 issues represent nearly 85% of all reported crashes.
Other quality initiatives in the OOo 2.x time frame were 'Warning Free Code' and check the code with Valgrind. This increased the general quality of the code. Memory leaks, programming errors, build errors etc. were identified and this represent the general stability of the product.
The most important point is, that OOo 3.0 based on the stable version of 2.4. More than 2.5 million downloads of that versions in the past 4 weeks shows a high credit in that version.
All these information – the number of incoming issues decrease, the incoming crash reports for newer versions decrease, the good processes for quality assurance on CWSs, the stable 2.x code line and ... – give me the confidence to do my daily business with every new build of OOo 3.0 (or sometimes with StarOffice).
I know that I have to use sometimes the latest version of OOo 2.x. This I do only when I have to create business critical documents and when I know the end users will not have an office which work with ODF 1.2 (like OOo 3.0). In some cases incompatibilities can occur with when features from 3.0 code-line are used, which aren't in 2.x. But you will get a warning in OOo 2.x, when you are opening a file with ODF 1.2.
When you are now more interested in using OOo 3.0 Beta, download it at http://download.openoffice.org/3.0beta/
If you are interested in general feedback on OOo 3.0 Beta, you can find it in OOo Wiki http://wiki.services.openoffice.org/wiki/OOo_3_Beta_feedback
If you think a critical bug should be fixed until Beta Refresh or the Final version of OOo write it on that page and let discuss it on the dev@qa.openoffice.org mailing list.
I understand that users think that OOo 3.0 isn't a big step forward when they are always using the newest versions of OOo 2.x. The feature set increased like each other Update and some more bug fixes are integrated – that's all. But with opening a new code line and integrated some major changes in the structure of OOo it will be possible to bring in the changes which are voted often on OOo mailing lists and also on the Beta Feedback side.
I am confident and looking promising to the updates on OOo 3.x code line.
tags: ooo openoffice.org qa quality statistics
Digg this
Comments
Dear Thorsten Ziehm
I am not sure, whether I got what the intention for the post was, just say hello and you are doing quality-assurance on 3.0 too, so this made it usable even at that stage of developement? Hm.
I have mailed that somewhere before and feel free to repeat: Do not look only at the crashes you count. Even seeing the increasing number of downloads and decreasing issues gives you the wrong impression, that things are well done sofar, for example you should not say, that 2.4.0 is a stable, while everybody is eagerly awaiting 2.4.1. And you should not say 3.0 is "no big step forward", while everybody hopes that OO.o matures now the way it had been expected for years ago.
You need to interpret the situation, check whether programming tools protect from haveing crashes but do not make performance better, features are still/are again not working and issues on these are a more or less tolerant complaints from userside, whether a decreasing number of reports is caused by the increasing quality or a decreasing support from the (user!)-community.
And - not the less - you need to consider, that quality is not something about counting errors only, but also about things that could have been done - and are not done.
I was not pleased with the quality of 2.4.0 when it was released "as stable", but things can be done better, so I like to have more often Beta's of 3.0 before released as final - and in that way you are right with the advise to use a beta - when the user knows what he is doing - and when he knows, that beside all other and more complex problems of development, that you checked the quality, stating it may be buggy but usable, and in no way harmfull to the users pc. This is the promise I liked to read from you posting.
rgds
Martin Heger
Posted by mhonline on May 26, 2008 at 03:16 PM CEST #
Hi Martin,
sorry to hear, that you have a different understanding of stable than I have. Numbers can be interpreted differently. I'm working for StarOffice and OOo for a very long time and you are right, I also miss something and I am angry about regressions we bring in from one release to the next one. But my opinion is still that the general quality increased over the time and that also the QA community increased and is increasing.
My intention of the blog is to give the user the confidence to work with OOo 3.0 Beta. If you think you cannot do your work with OOo 2.4 and also with the current 3.0 version, please write me your concerns in detail to my mail address. I need more background information about your concerns like "do not make performance better, features are still/are again not working".
Regards,
Thorsten
Posted by Thorsten Ziehm on May 26, 2008 at 03:40 PM CEST #
Hi Thorsten,
seems I need to agree to both of you, as OOo 3.0 beta is stable (I got no data loss or easy to reproduce crashes) - but Martin is right when speaking about missing or not working features. Just to name some:
- PDF import (one of the most eagerly awayted features) is still not ready for testing
- Sun Presenter Console has just been released for testing
- OOXML import has several flaws
- we will get a new Extension Manager interface with next beta
- localization is broken in some areas (not only for some languages but for all)
Of course, I would not report an isue in those areas of those as stopper for Beta Refresh or OOo 3.0 final. No single issue in thoses areas is a stopper - but this doesn't mean they don't need to be fixed before release. There is still a long way to go to be "ready to release".
Another point is, that my interpretation of the numbers is slightly different.
The good thing is, that our user base is increasing - the bad thing is, that our user base is changing. We gat more and more user that are "nothing but users" - many of them like OOo, because it is free (as in free Beer). Many of them do not see the need of giving something back to our community. And as it is still complicated to report issues, they are not likely to report issues. So - although we see much more downloads and have more users, the number of users reporting issues does not necessarily increase.
At the other hand we have several long time users who report bugs. Unfortunately some of them stopped to report bugs. Why? Because many of the issues do not get fixed. I know why this is so - and I do my work within the community anyway. But it is hard to convince people to report issues, if only half of them will be fixed.
Ok, nevertheless - you are right - quality of OOo 3.0 beta is much better than OOo 2.0 beta was :)
André
Posted by André Schnabel on May 26, 2008 at 06:56 PM CEST #
I agree to André that there are some missing features in the offical 3.0 beta that users (and testers) are waiting for. Relevant things I am missing are Issue 6087 (that already has a patch) and the possibility to handle dictionaries as extensions (to ease the adding and deleting and to make it permanent through updates).
Are newer developer releases than BEA300m2 available for testing? Does exist a DEV300m12 or BEA300m4 for download? I can't find anything newer than the official beta.
Regards,
Thomas
Posted by Thomas on May 26, 2008 at 08:31 PM CEST #
An old issue that is clearly solvable but has not been solved involves justification, or kerning (Issue 3243). This problem should have received the highest attention. Many Word/WordPerfect users won't switch to OOo because of this inadequacy.
Posted by W. Wayne Liauh on May 27, 2008 at 10:25 AM CEST #
Thomas,
the newest 3.0 developer snapshot is DEV300_m14. Just pick it from http://download.openoffice.org/680/index.html
Posted by Joost Andrae on May 30, 2008 at 11:52 AM CEST #