Monday, 25 May 2009
Monday, 25 May 2009
EIS just got a couple of new features.What´s new is that there are now a couple of tests that can be done on a OpenOffice.org ChildWorkspace which each have a status which is used to caclulate an overall status for the OpenOffice.org ChildWorkspace. Tests are either short test which are run automatically by EIS each time the user visits the new "Tests" tabpage of a ChildWorkspace or are longer running tests which are queued for execution on dedicated test computers. The short automatic tests do test such things like if all tasks at the ChildWorkspace are already fixed or if the release set at the ChildWorkspace is allowed for the MasterWorkspace the ChildWorkspace is based on. Longer running tests queued for execution on dedicated computers are AutomationCAT-0 tests, ConvWatch-Tests and Performance Test. Additionally also a couple of buildbots have been integrated which do now report their status of the build back to EIS. For tests being queued to dedicated computers to be run on this does mean that you might have to wait some time more for the results because tests on other ChildWorkspaces might be before yours in the queue. On the other hand using dedicated machines for the tests does make tests better comparable and does eliminate some problems we currently have with running automated tests on different hardware.
these tests are run on dedicated computes, they need approximately 8 hours and use the VCLTesttool with a couple of selected test scripts (called the CAT-0 tests) to check the quality of an OpenOffice.org installation. At the end of the test the status is in EIS is set to "finished needs review" and a link to QUASTe is being provided where the QA Rep can look at the details of the many testscript results and after review can finally set the status for the test in EIS to "ok" or "failed".
these tests are also run on dedicated computers, they need round about 15 minutes and do compare loading of some sample test documents and startup of OOo modules without opening a document for an OpenOffice.org of the current ChildWorkspace with the result of the same tests for an OpenOffice.org of the corresponding MasterWorkspace. At the end of the test the status is reported back to EIS and also an attachment has been save on the ChildWorkspace which contains details about the tests. This is done to find potential regressions in performance on a ChildWorkspace. The test has some tolerantable difference in performance that is allowed and could as well be cause by other factors than those in responsibility of the source code changes on the ChildWorkspace. AutomationCAT-0, ConvWatch and Performance tests currently need to be able to find installation sets for the CWS on Suns internal network. But if you are a Sun-external QARep of a ChildWorkspace owned by a Sun developer you can now also start these tests. We are in the process of considering how a integration of ChildWorkspaces where the owner is not a Sun developer could look like, stay tuned.
these test are also run on dedicated computers and need between 2 and a half and 3 hours to run. Here a couple of test documents are loaded, than printed to a postscript file and the postscript file is than converted to a graphics file. This is done for the OpenOffice.org of the ChildWorkspace and the corresponding MasterWorkspace. Graphics from the MasterWorkspace OpenOffice.org are than compared to those from the ChildWorkspace OpenOffice.org to find potential regressions in the layout engines and import filters.
As you can see in the picture below the new "Tests" tabpage in EIS contains the overall result and a table with results for individual tests and buttons to start tests.
there is an automatic test for wether a tool called cwslocalize has been called on a ChildWorkspace to calculate translation word count for the ChildWorkspace. This tool is not yet available to Sun external developers. This is being worked on.
The OOo Wiki contains some more details and there is also a "TestBots a first steps" announcement on the QA Mailinglist from Jörg Jahnke.
tags: automation cat0 development eis openoffice.org qa test testtool tools webapplication
Friday, 13 Feb 2009
When you consider that OOo runs fine on many different platforms (Windows, Unix and Mac), aims for good forwards/backwards/cross-compatibility and for many different scripts you may start to appreciate the work done in this area. A couple of years ago our office suite was just available in some languages for Latin scripts and didn't even have a chance to handle anything more complicated, because that part of the code was based on a lot of wrong assumptions. Replacing such a core part by a modern infrastructure which works natively on all platforms and is good enough for all scripts was a big effort. Gradually replacing the heart of a product while it was being released several times a year required an interesting kind of code refactoring. The success of OOo also in locales with CJK and CTL script requirements proves it was worth it.
We hope it will be even more successful now that a lot of long standing problems are being solved: In the newest snapshot OOO310_m1 you'll find better bidirectional text, text from complex scripts with vocalization marks looks better and users of arabic scripts will find that kashida-justified text has improved a lot. I'm very grateful to Henner Drewes for his insights into uniscribe issues and his outstanding work on kashida justification.
tags: development openoffice.org writer
Tuesday, 27 Jan 2009
There is a new master workspace (MWS) called OOO310. OOO310 is our branch for stabilizing towards the upcoming OpenOffice.org 3.1 release. Only childworkspaces accepted for OOo 3.1 may get integrated into it. No new features, no UI changes, only bug fixes. A first milestone is scheduled soon. Well know DEV300 representing svn trunk is now open again for ongoing development.
Technically the new code line is a branch, based on trunk as of DEV300 m40. BTW, it's our first release branch in subversion. You may find it at
svn://svn.services.openoffice.org/ooo/branches
tags: development openoffice.org release
Monday, 12 Jan 2009
It is my pleasure to announce the integration of the new overlining feature. I like to thank Martin Whitaker for his outstanding work on this. Let's have a look at what Martin did to get this feature done. First, he submitted a draft patch that implements this feature.
Just to give you an impression about the extend of code that had to be written or adjusted, this is the list of modules that were affected by the patch: vcl, svx, cppcanvas, drawinglayer, filter, officecfg, sfx2, svtools, reportdesign, chart2, sc, sd, sw, xmloff. Of course we need a specification for all new features. So Martin set up an I-Team consisting of Stefan Baltzer (QA), Uwe Fischer (Documentation) and Frank Loehmann (UX) and wrote this specification. But there was still some way to go: Since this new feature had to be added to ODF, Martin wrote a proposal email to the ODF office-comment mailing list. This proposal was discussed in the ODF TC and accepted to be added to ODF 1.2. Now it was time to finalize the cws. After a review of the code, the cws was built, handed over to Stefan for testing, and eventually integrated into DEV300m39. To sum this up: If you consider the amount and the quality of Martin's contribution, this definitely is a fantastic example of what is possible in an open source project.
tags: development openoffice.org writer
Wednesday, 27 Aug 2008
tags: development security users
Monday, 25 Aug 2008
The Tale of a Bugs Life (from OOo 2.3 to OOo 3.0):
A small code change of two lines and some glue code. Basically the change just added a call to a plain method in a high quality external library that is used thousands times a second on millions of desktops all over the world. The patch was from one of the most experienced non-Sun contributors to the OpenOffice.org project. A developer who deservedly earned and has my highest respect and that of the whole community. The patch was thoroughly tested by him and here. Patches don't get better than this, so it was bound to be approved with not too much of a second thought. Also the problem fixed by the patch was so exotic that it was only triggered rarely. So what could possibly go wrong?
There was nothing wrong with that at all. The change got tested even more and then it was integrated quickly. Everything worked happily ever since.
End of story...
...until it turned out to have introduced one of nastiest bugs OOo's X11 port had suffered from since OOo 2.3:
Some documents on some systems behaved in strange ways: In these cases OpenOffice.org displayed or printed some characters wrongly. E.g. an "a with a diaresis" seemingly got switched to a "per mille sign". Also the PDF export seemed to have problems like this too. This bug could have very serious implications, e.g. when a currency symbol is involved.
What made this bug so especially nasty was that it happened only when a set of certain documents were loaded in a specific order on specific systems It also made a difference which document pages were viewed in which order. So these reports were almost not reproducible on other systems, with other documents or even when the documents were loaded in a different order.
OpenOffice.org as a high profile open source project is built in many different ways from several different variants which are all based on
the source code from the openoffice.org domain. Certain non-100%-OOo builds have changes to code paths that could (with some very small probability) result in a problem like the one mentioned above. Unfortunately this Schroedinger-bug itself also had a similarly small probability overall. But the problem was apparently easily triggered for some users.
When I debugged into one of these mysterious problems I finally found the root cause. The function call in the patch, which helped to solve an exotic and rare problem (#i72129#) had a side effect that was neither to be expected nor documented. And this side effect resulted in a multitude of small but serious problems the sum of which almost had epic proportions.
Also the same bug not only happened for non-ASCII latin text, but in a totally different disguise it could show up as a many little problems in CJK contexts too. In case you are interested in more details of the bug you can find them here. The simple root cause that was absolutely deterministic on the inside turned into such a bad regression where affected scenarios might have a mind bending complexity.
OOo's "issuezilla" system currently tracks almost 93000 issues. Almost 2000 of these were unconfirmed or non-reproducable problem reports which mentioned "fonts" somewhere in a comment, were created after OOo2.3 and were potentially seen on X11 based platforms. Among them there finally was an issue with a description and an attached document where this exact symptom could be reproduced on a system in my reach. With this I had the chance to isolate it further, debug into it and to discover the root cause of the problem. Now that the root cause was understood with all its implications, an educated guess is that at least 5% of these almost 2000 issues may be caused by the side effect in the external library...
So how could this possibly go wrong? A process that could have prevented it would need to be so heavy, that the progress for a project with the size and complexity of OOo would be almost completely stalled by it. A really heavy process like this is only reasonable for the small, well defined and isolated tasks in very critical systems. OOo as the multi-purpose, multi-platform, universal productivity suite that tries to integrate as smoothly into the host system is quite the opposite. So problems like this are inevitably bound to happen once in a while. But that doesn't mean that things go wrong. Things are not exactly black or white. "Worse is better" is a common idiom in software development. There is a lot of insight in this play with words. It means that good code in time is better than perfect code that never gets to see the light of day: A software project flourishes if (1.) the changes to its code base help progress and (2.) if the resulting problems are found and fixed.
Unfortunately developers consider phase (1.) to be much more fun than phase (2.): Digging into countless heisenbugs, many of which are by definition often frustrating, long winded and tedious, to find reliable bug scenarios behind problem reports, their root causes and solving them is really hard work and requires lots of experience. Good problem reports and patches which don't just solve a symptom but the root cause are highly appreciated gems, because they help considerably with this work. Though, should you ever wonder why the reaction to a question like "the patch needs to get in ASAP. It tested well here. What could possibly go wrong?" occassionaly seems to be on the cautious side, you'll be able to understand it better now: Just consider it from the perspective of the few developers that also do phase (2.) of that work.
As a side note I'd like to mention that because phase (1.) has most of the fun and almost all of the CV-boosting fame, this is also exactly
the same reason why many developers prefer to work on libraries or applications that are not directly visible to end users. By targeting
it only to other developers they can avoid much of the usually thankless and hard work of phase (2.) as most of it gets done by these
other developers who try to use the code...
This story started as a fairy tale for developers and turned into a tragedy with many different side plots. I hope you enjoyed reading it and maybe also learned. We have to accept that stuff happens. If there are enough honest efforts to solve the real problems history shows that everything will turn out just fine.
So here comes the happy end: OpenOffice.org 3.0 will have the fix. Currently it is in the childworkspace named "vcl93". It also contains another fix that would deserve a blog entry almost as long as this one...
tags: development openoffice.org
Friday, 08 Aug 2008
Last week we had a face to face meeting of the “notes2” team in Hamburg. It was the first time where all members came together at one place – until then the work was completely done remotely in different parts of Germany. We started with a Google Summer Of Code project in 2007 and continued as a team that works together for quite a year now.
We think that our little project was very successful, but nevertheless we wanted to look back a little and talk about our “lessons learned”. Here's a short list of things that aren't new at all but that we consider to be important especially for remote team work:
While meeting agendas are a good thing to have always, they are indispensable for IRC meetings.
It's good to become acquainted with each other early and know more about everybody's background and motivations. This makes it easier to understand and trust each other. So if possible a face to face meeting in the beginning is desirable.
We should have used telephone conferences much more often; we used e-mail too much.
It was good not to plan for a fixed release and instead define the minimum feature and quality requirements for a first possible release.
Not all decisions can be made by the whole team, and so it shouldn't be tried. Instead of this the team members need common guidelines they have agreed to.
Use cases are a very useful way to define such guidelines. Personal preferences should be hold back as much as possible.
Discussions shouldn't be exhaustive in all cases, but it is important to point to technical limits and errors as soon as possible.
But of course the most important insight was that we all had a lot of fun working together and that we think that we created a considerable value for the product. That's what working in a community is about – and I hope that we can have more like this in the future. So if you know something you want to implement in Writer, but need some help – just let me know. :-)
Of course we also talked about future plans for the “notes” feature and the new notes sidebar we created for it. We want to start implementing them in OOo 3.1 and continue in the following releases. Here's what we came up with. You might be pleased to read that work on the first items is already in progress. :-)
As Christoph Noack happens to be the Co-Lead of the OOo User Experience team, we took the chance to meet with the User Experience team members Christian Jansen and Frank Löhmann in Hamburg also and made it a “ what we always wanted to discuss about Writer usability improvements” meeting. We identified some topics that we see important and that we want to include in the upcoming project review that is planned to happen in early September (right after my vacation ;-) ):
An "Insert page" function that frees users from dealing with several single steps just to e.g. insert a landscape page
"Insert page number" also as a one-step function
Improve other functions dealing with page styles (allow to describe which pages shall be influenced by the change etc.)
"Delete hyperlink" function in the context menu of text formatted as a hyperlink
“Accept/Reject Changes” in the context menu of changed text
Add (a) Landscape style(s)
Use transparent selection in Writer as in the other OOo applications
Add a Change Tracking option "don't show deleted text in track changes" and use it by default
Status bar improvements
The mentioned changes in the Change Tracking feature look even better if you add the planned “show change tracking comments in sidebar” feature from our list of “ notes” enhancements!
Here's a picture of the exhausted ;-) team after a long meeting:
From the left to the right: Christoph Noack (UX), Éric Savary (QA), Max Odendahl (Dev), Mathias Bauer (Dev)
Tuesday, 27 May 2008
How verbose should a build process be?
Some time ago we got asked to make openoffice.org's build process less verbose (issue 84497). Some people are bothered by the console output when building, especially by seeing complete compiler calls including all parameters. They proposed to make the build (more or less) silent. Others opposed, wanting to keep full information. What's considered best seems to depend on personal taste, experience, and habit. The idea came up to make build verbose level configurable. Unfortunately, even a configure option can not really make everyone happy, as it raises the question what behavior should become default (see discussion in issue 84263).
I the end I decided (that is the advantage of having no agreement - the one doing the work can decide himself) to create three different verbose levels for building openoffice.org. Default will be somewhat less verbose than what we had, but not as silent as proposed in issue 84497. Such quiet build can be achieved by setting an environment variable VERBOSE to FALSE. On the other hand, you can get full information by setting VERBOSE to TRUE. First steps toward this aim are implemented in childworkspace shutup2. I introduced the variable and did some very first adjustments regarding build verbosity. The main changes, especially those mentioned in issue 84497, are still open. More can be done independently on follow-up childworkspaces.
tags: development openoffice.org
Friday, 16 May 2008
The feedback we got for the Aqua port of OpenOffice.org 3 Beta was extremely encouraging. This is very motivating. Thank you!
There was some constructive criticisms about some aspects of the port too. I'm happy to note that many of the issues are already solved in two developer's child workspaces (aquavcl07 and aquabmpfix01): keyboard handling, graphics performance and some visual details will improve considerably soon in one of the next regular development snapshots.
tags: beta development mac openoffice.org
Thursday, 08 May 2008
tags: api architecture code development netbeans odf opendocument software specification sun xml