You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ant.apache.org by Bob Billings <Di...@Comcast.net> on 2011/07/28 15:05:20 UTC
junit task xml output omits failure message if it contains the word " more"?
I tried searching bugzilla to see if this had been noticed before but
couldn't find anything. My project is using Cruise Control to rebuild our
development database every night and send out unit test results in the
morning to the developers so in the morning they know whether the latest SQL
scripts they checked in worked properly, whether our test data loaded
correctly, etc. I was noticing that some of the test failures shown in the
emails were missing the actual reason for the test failure -- they just said
"at <class name, line number etc>".
I've tracked down the issue to the Ant junit task XML formatter output which
is used by the CruiseControl email. CruiseControl looks for the failure
message in the first line of the stacktrace in the text node child of the
<failure> tag, which normally looks something like
"junit.framework.AssertionFailedError: This data is more than 24 hours
old!." However, the Ant junit XML formatter strips off the first line with
the failure message in any test failure where the failure message contains
the word " more", resulting in just the "at ...." line remaining, which is
what shows up in CruiseControl's email.
Looking at the Ant 1.8.2 source code I tracked that down to the
org.apache.tools.ant.taskdefs.optional.junit.JUnitTaskRunner.getFilteredTrace
method. It removes lines from the stacktrace if they contain any of the
following values:
private static final String[] DEFAULT_TRACE_FILTERS = new String[] {
"junit.framework.TestCase",
"junit.framework.TestResult",
"junit.framework.TestSuite",
"junit.framework.Assert.", // don't filter AssertionFailure
"junit.swingui.TestRunner",
"junit.awtui.TestRunner",
"junit.textui.TestRunner",
"java.lang.reflect.Method.invoke(",
"sun.reflect.",
"org.apache.tools.ant.",
// JUnit 4 support:
"org.junit.",
"junit.framework.JUnit4TestAdapter",
" more",
};
As you can see, the last item is " more". Bingo! Presumably this was added
to remove the "... 48 more" line you see at the end of some stacktraces?
Anyway, I was wondering whether this qualifies as something worth fixing,
since:
- anybody using CruiseControl with Ant-run JUnit tests is fairly likely to
run into it (it should be fairly common to fail a test when there are "more
of X than there should be")
- it's unlikely that anybody running into this problem will suspect the word
"more" as the culprit
- and even if they do know, it's hard to implement a policy of "Okay
everybody, remember not to use the word 'more' in your failure messages."
How to fix it is another question entirely... maybe make it a regex and test
for "\.\.\. \d+ more"?
Bob
Re: junit task xml output omits failure message if it contains the word " more"?
Posted by Bob Billings <di...@gmail.com>.
Yeah, that would suit my purposes. Seems like the first line is the only one likely to contain a space before the word " more" anyway, aside from the line we want to strip.
Bob
On Aug 5, 2011, at 8:40 AM, Stefan Bodewig <bo...@apache.org> wrote:
> On 2011-08-04, Robert E. Billings III wrote:
>
>> OK, I'll fill out a bugzilla report. Thanks for verifying! :)
>
> Would it be enough if we simply started filtering after the first line
> of the stack trace? That fix would be easiest to implement.
>
> Stefan
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@ant.apache.org
> For additional commands, e-mail: user-help@ant.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@ant.apache.org
For additional commands, e-mail: user-help@ant.apache.org
Re: junit task xml output omits failure message if it contains the word " more"?
Posted by Stefan Bodewig <bo...@apache.org>.
On 2011-08-04, Robert E. Billings III wrote:
> OK, I'll fill out a bugzilla report. Thanks for verifying! :)
Would it be enough if we simply started filtering after the first line
of the stack trace? That fix would be easiest to implement.
Stefan
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@ant.apache.org
For additional commands, e-mail: user-help@ant.apache.org
Re: junit task xml output omits failure message if it contains the
word " more"?
Posted by "Robert E. Billings III" <bo...@tekassoc.com>.
OK, I'll fill out a bugzilla report. Thanks for verifying! :)
Bob
On Thu, Aug 4, 2011 at 11:05 AM, Stefan Bodewig <bo...@apache.org> wrote:
> On 2011-07-28, Bob Billings wrote:
>
> > However, the Ant junit XML formatter strips off the first line with
> > the failure message in any test failure where the failure message
> > contains the word " more", resulting in just the "at ...." line
> > remaining, which is what shows up in CruiseControl's email.
>
> Oh my, it must have taken quite some time to identify the word "more"
> has been the culprit. Sorry, really. Please open a bugzilla report for
> this so knowledge is preserved and others don't have to hunt down the
> same problem as well.
>
> > I was wondering whether this qualifies as something worth fixing,
> > since:
>
> > - anybody using CruiseControl with Ant-run JUnit tests is fairly likely
> to
> > run into it (it should be fairly common to fail a test when there are
> "more
> > of X than there should be")
> > - it's unlikely that anybody running into this problem will suspect the
> word
> > "more" as the culprit
> > - and even if they do know, it's hard to implement a policy of "Okay
> > everybody, remember not to use the word 'more' in your failure messages."
>
> I agree with you, in particular with the last point.
>
> To make things worse the manual page of the junit task does not list
> "more" as one of the strings that are filtered out.
>
> As a workaround you can set filtertrace to false, of course, but this is
> no real fix.
>
> Stefan
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@ant.apache.org
> For additional commands, e-mail: user-help@ant.apache.org
>
>
Re: junit task xml output omits failure message if it contains the word " more"?
Posted by Stefan Bodewig <bo...@apache.org>.
On 2011-07-28, Bob Billings wrote:
> However, the Ant junit XML formatter strips off the first line with
> the failure message in any test failure where the failure message
> contains the word " more", resulting in just the "at ...." line
> remaining, which is what shows up in CruiseControl's email.
Oh my, it must have taken quite some time to identify the word "more"
has been the culprit. Sorry, really. Please open a bugzilla report for
this so knowledge is preserved and others don't have to hunt down the
same problem as well.
> I was wondering whether this qualifies as something worth fixing,
> since:
> - anybody using CruiseControl with Ant-run JUnit tests is fairly likely to
> run into it (it should be fairly common to fail a test when there are "more
> of X than there should be")
> - it's unlikely that anybody running into this problem will suspect the word
> "more" as the culprit
> - and even if they do know, it's hard to implement a policy of "Okay
> everybody, remember not to use the word 'more' in your failure messages."
I agree with you, in particular with the last point.
To make things worse the manual page of the junit task does not list
"more" as one of the strings that are filtered out.
As a workaround you can set filtertrace to false, of course, but this is
no real fix.
Stefan
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@ant.apache.org
For additional commands, e-mail: user-help@ant.apache.org