Wednesday August 20, 2008 Joseph D. Darcy's Sun Weblog
Joseph D. Darcy's Sun Weblog
Wednesday August 20, 2008 OpenJDK 6: Some more regression test results for b11
To address a configuration issue with the previously reported regression tests results for OpenJDK 6 b11, I reran the jdk tests with the display set to a virtual framebuffer and 77 more tests pass. In summary now:
JDK, 2,989 tests pass, 32 tests fail, 22 tests have errors.
Note that the jtdiff command, included as part of jtreg, can be used to compare the output of different regression test runs.
(2008-08-20 14:00:13.0) Permalink Comments [4]Post a Comment:
Comments are closed for this entry.
Calendar
| « November 2009 | ||||||
| Sun | Mon | Tue | Wed | Thu | Fri | Sat |
|---|---|---|---|---|---|---|
1 | 2 | 3 | 5 | 6 | 7 | |
8 | 9 | 10 | 11 | 12 | 13 | 14 |
15 | 16 | 17 | 18 | 19 | 20 | 21 |
22 | 23 | 24 | 25 | 26 | 27 | 28 |
29 | 30 | |||||
| Today | ||||||
RSS Feeds
All
/Annotation Processing
/General
/Java
/JavaOne
/Numerics
/OpenJDK
Search
Links
- Blogroll
-
- News
Navigation
Referers
Today's Page Hits: 243
3k tests seems low for the entire JDK. Curious, are code coverage metrics obtained during the run? It would be a great way to identify potential gaps that the community can address.
Posted by Abraham Tehrani on August 20, 2008 at 03:14 PM PDT #
Abraham,
Internally the full JDK code base is now split into a number of component workspaces/repositories: corba, hotspot, jaxp, jaxws, langtools (javac and friends), jdk, etc. The 3k tests are in the core jdk workspace; there are another 1,300 tests in langtools and a few regression tests in the other components. In addition, there are also closed regression tests that have not been open sourced for a variety of reasons as well as other test suites, like the JCK tests. There are no externally published code coverage numbers from running these tests, although I agree that would be useful information to have. Over time in developing the JDK, we've had increasing expectations of what it means to have acceptable regression tests so newer parts of the platform will generally have more extensive regression test coverage than older ones.
Posted by Joe Darcy on August 20, 2008 at 04:30 PM PDT #
How do you run the tests? It seems some of the errors you are seeing are because you run tests marked with @ignore. In IcedTea the default test run uses -v1 -a -ignore:quiet which on my local machine gives:
HOTSPOT: Tests that passed 5, Total: 5
LANGTOOLS: Tests that passed 1342, Tests that failed 1, Total: 1343
JDK: Tests that passed 3182, Tests that failed 50, Tests that had errors 3, Total: 3235
Posted by Mark Wielaard on August 21, 2008 at 04:24 AM PDT #
Mark,
For these runs I just used "-v1 -a", but it would be more informative to really ignore the @ignore tests as you've done using "-ignore:quiet" so I'll report the builds' test status using that option going forward.
Posted by Joe Darcy on August 21, 2008 at 02:11 PM PDT #