[Bug 70175] New: JUnitLauncher task misses haltonerror attribute

[bu...@apache.org]

https://bz.apache.org/bugzilla/show_bug.cgi?id=70175

            Bug ID: 70175
           Summary: JUnitLauncher task misses haltonerror attribute
           Product: Ant
           Version: 1.10.17
          Hardware: PC
            Status: NEW
          Severity: normal
          Priority: P2
         Component: Optional Tasks
          Assignee: notifications@ant.apache.org
          Reporter: maarten_coene@yahoo.com
  Target Milestone: ---

We are upgrading from JUnit4 to JUnit5.

Our JUnit task looks like this:

<junit fork="yes" haltonerror="false" ...

We are facing difficulties here because the JUnitLauncher task doesn't seem to
have a 'haltonerror' attribute. We don't want to stop running the tests when
one of them did result in an unexpected error.

Would it be possible to add this attribute similar like the old JUnit task had?

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 70175] JUnitLauncher task misses haltonerror attribute

[bu...@apache.org]

https://bz.apache.org/bugzilla/show_bug.cgi?id=70175

Jaikiran Pai <ja...@apache.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 OS|                            |All

--- Comment #1 from Jaikiran Pai <ja...@apache.org> ---
Hello Maarten, the junitlauncher task has a "haltOnFailure" attribute
https://ant.apache.org/manual/Tasks/junitlauncher.html. It doesn't however
distinguish between error or failure since JUnit 5 itself doesn't have that
distinction anymore. Marc, whose one of the maintainers of JUnit project had
this to say about this recently
https://github.com/apache/ant/pull/221#issuecomment-2979280480.

Having said that, this isn't the first request for such an enhancement. So
maybe we should do something for the junitlauncher task. If we had to enhance
this in junitlauncher, maybe we consider Marc's suggestion of considering
"AssertionError" type exceptions as failures, and all other exception types as
errors. And then allow a way to control the build progress. Would that address
it for your project(s)?

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 70175] JUnitLauncher task misses haltonerror attribute

[bu...@apache.org]

https://bz.apache.org/bugzilla/show_bug.cgi?id=70175

--- Comment #2 from Maarten Coene <ma...@yahoo.com> ---
Hi Jaikiran,

we also use the haltonfailure property.

<junit fork="yes" haltonerror="false" haltonfailure="false" ...

became:

<junitlauncher haltonfailure="false">
  <testclasses ...>
    <fork forkMode="perTestClass" ...>
       ...
    </fork>
  </testclasses>
</junitlauncher>

I'm not sure we really distinguish between JUnit errors/failures ourselves:
they are all things that needs to be addressed.

The problem we run into is that with the JUnit task, the haltonerror='false'
makes the JUnit task continue with the next test, even if the current test
crashed (for instance a VM crash). With the JUnitLauncher Task, the task just
stops after such a crash. We want to be able to configure this JUnitLauncher
task to proceed to the next test in this case: the 'haltonerror' property seems
to be a good fit for this.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 70175] JUnitLauncher task misses haltonerror attribute

[bu...@apache.org]

https://bz.apache.org/bugzilla/show_bug.cgi?id=70175

--- Comment #3 from Sylwester Lachiewicz <sl...@apache.org> ---
Maarten's second comment moves this away from the error/failure distinction,
and the code seems to bear that out.

In JUnitLauncherTask the forked exit code is handled by a switch on
Constants.FORK_EXIT_CODE_*:

- TESTS_FAILED (2) and TIMED_OUT (3) both consult failureProperty and
haltOnFailure.
- EXCEPTION (1) throws a BuildException unconditionally. haltonfailure="false"
has no effect on it.
- There is no default branch, so any other exit code - a JVM killed by a
signal, say - matches nothing and the task carries on as though the fork had
succeeded.

So a forked JVM that dies either aborts the whole task with no way to opt out,
or is ignored silently, depending on which exit code it happens to leave
behind. Exit code 1 is ambiguous too: it is Ant's own "the launcher threw"
signal and also what a JVM that failed to start leaves behind.

If a haltonerror attribute were added, having it gate the EXCEPTION branch and
a new default branch the way haltOnFailure gates TESTS_FAILED would cover what
Maarten describes, and would keep the AssertionError question in 69687
separate. Defaulting it to true would preserve today's behaviour.

Happy to put a patch together if that shape sounds right.

This comment was created with AI assistance.

-- 
You are receiving this mail because:
You are the assignee for the bug.