Re: Batik Test Suite

[Vincent Hennebert <vh...@gmail.com>]

Some updated update on this.

After the updates/fixes I’ve made to the tests in the past few weeks,
there are now 20 remaining tests that fail. I’ve posted the updated
Regard report on my public space:
http://people.apache.org/~vhennebert/batik-test-suite/testReport/html/regardReport.html

Attached is the updated list of variation images that can safely be
marked as acceptable.

Looks like we’re getting there...

Vincent


On 28/07/14 23:43, Vincent Hennebert wrote:
> Hi,
> 
> I thought I would post here my findings about running the Batik test
> suite.
> 
> The test suite can be run on the command line by using the ‘regard’
> target from build.xml. It produces an HTML report that lists the failing
> test cases.
> 
> To begin with, the Basic Effectivity test suite from the SVG Working
> Group will probably have to be removed. That’s because it’s not shipped
> with Batik (the code is pointing to a directory outside the project
> root), and I can no longer find any trace of it on the Internet. Unless
> I missed something, of course.
> 
> The rest of the test suite runs but results into many failures (nearly
> 500). Most of the tests probably just need to be updated (some of them
> are based on the comparison of a bitmap rendering against a reference
> image). But some of them may well point at regressions and I can see
> quite a few exceptions. Do we want to take the risk of releasing Batik
> with a failing test suite? Probably not.
> 
> This test suite looks like it was created before JUnit became the de
> facto standard for testing. It looks rather powerful and elegantly
> designed but, needless to say, it doesn’t quite have the same tooling
> support as JUnit (IDE integration, CI, code coverage, etc.). Also, it
> opens up several windows to render some SVG tests, which is quite
> disrupting. We might want to convert it to JUnit and reduce it to
> a subset that can run in the background.
> 
> But the most urgent probably is to go through the test results and
> investigate the failing ones.
> 
> Vincent