You are viewing a plain text version of this content. The canonical link for it is here.
Posted to fop-dev@xmlgraphics.apache.org by Glenn Adams <gl...@skynav.com> on 2011/11/17 00:47:05 UTC
insufficient output from LayoutEngineTestCase failures
With the recent transition to JUnit4 runner and changes to
LayoutEngineTestCase, it is now someone difficult to determine (from output
data alone) which layoutengine test file failed when a regression occurs.
In particular, the exception trace generated from EvalCheck, TrueCheck,
etc. does not include the test case input file name. So one ends up with
something like the following in
the TEST-org.apache.fop.layoutengine.LayoutEngineTestCase.txt file in
builds/test-reports:
Testcase: runTest[342] took 0.017 sec
Caused an ERROR
Expected XPath expression to evaluate to '2', but got '' (XPath:
//pageViewport[@nr=1]/page/regionViewport/regionBody/mainReference/span/flow/block/lineArea/inlineparent/text)
java.lang.RuntimeException: Expected XPath expression to evaluate to '2',
but got '' (XPath:
//pageViewport[@nr=1]/page/regionViewport/regionBody/mainReference/span/flow/block/lineArea/inlineparent/tex\
t)
at org.apache.fop.layoutengine.EvalCheck.doCheck(EvalCheck.java:86)
at org.apache.fop.layoutengine.EvalCheck.check(EvalCheck.java:60)
at
org.apache.fop.layoutengine.LayoutEngineTestCase.doATChecks(LayoutEngineTestCase.java:258)
at
org.apache.fop.layoutengine.LayoutEngineTestCase.checkAll(LayoutEngineTestCase.java:191)
at
org.apache.fop.layoutengine.LayoutEngineTestCase.runTest(LayoutEngineTestCase.java:172)
Testcase: runTest[343] took 0.012 sec
Unfortunately, there is no way to correlate runTest[342] with a specific
test case input file. It would be very useful (for AT and IF tests) to also
include the test case input file name/path in this output. Otherwise, one
is forced to run junit in a debugger with a breakpoint on EvalCheck,
TrueCheck, etc.
Re: insufficient output from LayoutEngineTestCase failures
Posted by Glenn Adams <gl...@skynav.com>.
s/doATTests/doATChecks/
On Fri, Nov 18, 2011 at 10:19 AM, Glenn Adams <gl...@skynav.com> wrote:
> one workaround for LayoutEngineTestCase.doATTests() would be to add the
> test file name/path as a accessible property of LayoutResult, which would
> then be accessible by check(LayoutResult);
>
> perhaps medhi can try this or something similar
>
> On Fri, Nov 18, 2011 at 3:26 AM, Vincent Hennebert <vh...@gmail.com>wrote:
>
>> Hi Glenn,
>>
>> On 16/11/11 23:47, Glenn Adams wrote:
>> > With the recent transition to JUnit4 runner and changes to
>> > LayoutEngineTestCase, it is now someone difficult to determine (from
>> output
>> > data alone) which layoutengine test file failed when a regression
>> occurs.
>> > In particular, the exception trace generated from EvalCheck, TrueCheck,
>> > etc. does not include the test case input file name. So one ends up with
>> > something like the following in
>> > the TEST-org.apache.fop.layoutengine.LayoutEngineTestCase.txt file in
>> > builds/test-reports:
>> >
>> > Testcase: runTest[342] took 0.017 sec
>> > Caused an ERROR
>>
>> This is a known issue:
>> https://github.com/KentBeck/junit/issues/44
>> And quite popular at that it seems:
>> https://github.com/KentBeck/junit/issues/search?q=parameterized+name
>>
>> In the meantime, I wrote the attached patch to IFParserTestCase. The
>> same could be done for LayoutEngineTestCase.
>>
>> <snip/>
>> >
>> > Unfortunately, there is no way to correlate runTest[342] with a specific
>> > test case input file. It would be very useful (for AT and IF tests) to
>> also
>> > include the test case input file name/path in this output. Otherwise,
>> one
>> > is forced to run junit in a debugger with a breakpoint on EvalCheck,
>> > TrueCheck, etc.
>>
>> Vincent
>>
>
>
Re: insufficient output from LayoutEngineTestCase failures
Posted by Glenn Adams <gl...@skynav.com>.
one workaround for LayoutEngineTestCase.doATTests() would be to add the
test file name/path as a accessible property of LayoutResult, which would
then be accessible by check(LayoutResult);
perhaps medhi can try this or something similar
On Fri, Nov 18, 2011 at 3:26 AM, Vincent Hennebert <vh...@gmail.com>wrote:
> Hi Glenn,
>
> On 16/11/11 23:47, Glenn Adams wrote:
> > With the recent transition to JUnit4 runner and changes to
> > LayoutEngineTestCase, it is now someone difficult to determine (from
> output
> > data alone) which layoutengine test file failed when a regression occurs.
> > In particular, the exception trace generated from EvalCheck, TrueCheck,
> > etc. does not include the test case input file name. So one ends up with
> > something like the following in
> > the TEST-org.apache.fop.layoutengine.LayoutEngineTestCase.txt file in
> > builds/test-reports:
> >
> > Testcase: runTest[342] took 0.017 sec
> > Caused an ERROR
>
> This is a known issue:
> https://github.com/KentBeck/junit/issues/44
> And quite popular at that it seems:
> https://github.com/KentBeck/junit/issues/search?q=parameterized+name
>
> In the meantime, I wrote the attached patch to IFParserTestCase. The
> same could be done for LayoutEngineTestCase.
>
> <snip/>
> >
> > Unfortunately, there is no way to correlate runTest[342] with a specific
> > test case input file. It would be very useful (for AT and IF tests) to
> also
> > include the test case input file name/path in this output. Otherwise, one
> > is forced to run junit in a debugger with a breakpoint on EvalCheck,
> > TrueCheck, etc.
>
> Vincent
>
Re: insufficient output from LayoutEngineTestCase failures
Posted by Vincent Hennebert <vh...@gmail.com>.
Hi Glenn,
On 16/11/11 23:47, Glenn Adams wrote:
> With the recent transition to JUnit4 runner and changes to
> LayoutEngineTestCase, it is now someone difficult to determine (from output
> data alone) which layoutengine test file failed when a regression occurs.
> In particular, the exception trace generated from EvalCheck, TrueCheck,
> etc. does not include the test case input file name. So one ends up with
> something like the following in
> the TEST-org.apache.fop.layoutengine.LayoutEngineTestCase.txt file in
> builds/test-reports:
>
> Testcase: runTest[342] took 0.017 sec
> Caused an ERROR
This is a known issue:
https://github.com/KentBeck/junit/issues/44
And quite popular at that it seems:
https://github.com/KentBeck/junit/issues/search?q=parameterized+name
In the meantime, I wrote the attached patch to IFParserTestCase. The
same could be done for LayoutEngineTestCase.
<snip/>
>
> Unfortunately, there is no way to correlate runTest[342] with a specific
> test case input file. It would be very useful (for AT and IF tests) to also
> include the test case input file name/path in this output. Otherwise, one
> is forced to run junit in a debugger with a breakpoint on EvalCheck,
> TrueCheck, etc.
Vincent
Re: insufficient output from LayoutEngineTestCase failures
Posted by mehdi houshmand <me...@gmail.com>.
Hi Glenn,
Yeah, JUnit4 doesn't allow one to parametrize test names this is a
known issue. I did spend some time looking into how it would be
possible, see https://issues.apache.org/bugzilla/show_bug.cgi?id=51928.
Hopefully it will be rectified soon.
Mehdi
On 16 November 2011 23:47, Glenn Adams <gl...@skynav.com> wrote:
> With the recent transition to JUnit4 runner and changes to
> LayoutEngineTestCase, it is now someone difficult to determine (from output
> data alone) which layoutengine test file failed when a regression occurs. In
> particular, the exception trace generated from EvalCheck, TrueCheck, etc.
> does not include the test case input file name. So one ends up with
> something like the following in
> the TEST-org.apache.fop.layoutengine.LayoutEngineTestCase.txt file in
> builds/test-reports:
> Testcase: runTest[342] took 0.017 sec
> Caused an ERROR
> Expected XPath expression to evaluate to '2', but got '' (XPath:
> //pageViewport[@nr=1]/page/regionViewport/regionBody/mainReference/span/flow/block/lineArea/inlineparent/text)
> java.lang.RuntimeException: Expected XPath expression to evaluate to '2',
> but got '' (XPath:
> //pageViewport[@nr=1]/page/regionViewport/regionBody/mainReference/span/flow/block/lineArea/inlineparent/tex\
> t)
> at org.apache.fop.layoutengine.EvalCheck.doCheck(EvalCheck.java:86)
> at org.apache.fop.layoutengine.EvalCheck.check(EvalCheck.java:60)
> at
> org.apache.fop.layoutengine.LayoutEngineTestCase.doATChecks(LayoutEngineTestCase.java:258)
> at
> org.apache.fop.layoutengine.LayoutEngineTestCase.checkAll(LayoutEngineTestCase.java:191)
> at
> org.apache.fop.layoutengine.LayoutEngineTestCase.runTest(LayoutEngineTestCase.java:172)
> Testcase: runTest[343] took 0.012 sec
> Unfortunately, there is no way to correlate runTest[342] with a specific
> test case input file. It would be very useful (for AT and IF tests) to also
> include the test case input file name/path in this output. Otherwise, one is
> forced to run junit in a debugger with a breakpoint on EvalCheck, TrueCheck,
> etc.
>
>