Surefire: Success Percentage not handled correctly

[Steven Blundy <sb...@casenetllc.com>]

I have a wait test that I expect to fail on occasion because the test threads get too out of sync. So, using TestNG, I'm setting the successPercentage and invocationCount to handle the occasional acceptable failure. If the code's broken, none of the invocations will pass.

When I run the test in IntelliJ, everything's fine. But when I try and do a build in maven, I get the following

Tests run: 1407, Failures: 0, Errors: 1, Skipped: 0

[INFO] ------------------------------------------------------------------------
[ERROR] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] There are test failures.

Looking in restng-results.xml, I see this:

  <test-method status="SUCCESS_PERCENTAGE_FAILURE" signature="waits()" name="waits" duration-ms="1000" started-at="2012-03-02T13:51:58Z" finished-at="2012-03-02T13:51:59Z">

And in TEST-TestSuite.xml, I see:

  <testcase classname="com.casenet.integration.filesystem.TriggerFileListenerTest" time="1" name="waits">
    <error type="java.lang.AssertionError" message="expected:&lt;EXECUTE&gt; but was:&lt;TIMED_OUT&gt;">java.lang.AssertionError: expected:&lt;EXECUTE&gt; but was:&lt;TIMED_OUT&gt;

The rest of the testng generated files seem to think everything passed. But it looks like that surefire is getting confused by SUCCESS_PERCENTAGE_FAILURE or the <error/> element into thinking something's gone wrong when in fact nothing unexpected happened.

Anyone know if this is a known problem or if there's a solution available?


Steve


CONFIDENTIALITY NOTICE:  This e-mail and the information transmitted within including any attachments is only for the recipient(s) to which it is intended and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of; or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please send the e-mail back by replying to the sender and permanently delete the entire message and its attachments from all computers and network systems involved in its receipt.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org