[ANN] Maven Surefire Plugin 2.4 for Maven 2 Released

[Dan Fabulich <df...@apache.org>]

The Maven team is pleased to announce the release of the Maven Surefire 
Plugin, version 2.4

http://maven.apache.org/plugins/maven-surefire-plugin/

This version fixes numerous long-outstanding bugs, notably in TestNG 
support.

You can run mvn -up to get the latest version of the plugin, or specify 
the version in your project's plugin configuration:

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.4</version>
</plugin>

By default, Maven automatically consumes the latest version of Surefire, 
so we can reasonably expect some Maven users to begin using Surefire 2.4 
immediately, perhaps without their knowledge.  In at least some cases, 
this will result in tests failing that passed using Surefire 2.3.1.  If 
this happens to you, the first and easiest thing that you can do to get 
your build to work again is to add an explicit version reference to 
Surefire 2.3.1, like this:

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.3.1</version>
</plugin>

Especially watch out for these fixes, which may affect backwards 
compatibility:

1) "Abstract" test cases [SUREFIRE-167]: Surefire 2.3 would automatically 
skip tests whose names start with "Abstract."  In Surefire 2.4, we use 
reflection to detect abstract tests and avoid instantiating them. 
However, some people had come to rely on the old behavior, creating 
classes named "AbstractFooTest" without marking them abstract, and without 
putting any tests in them.  Surefire 2.4 will attempt to run these 
classes, and will register failures if they don't work.  Mark all abstract 
test cases with the "abstract" keyword.

2) Working directory [SUREFIRE-416]: By default, the working directory of 
the running tests will be the ${basedir} of the current project, which 
defaults to the "target" directory of the currently running project. 
Surefire 2.3's behavior was inconsistent, so this may cause problems for 
you if your tests somehow depend on the old behavior.  You can configure 
the test working directory using the "workingDirectory" configuration 
parameter.

3) System properties [SUREFIRE-416]: Surefire allows you to configure the 
system properties of the currently running test.  By default, Surefire 
will run tests in a separate process (forkMode=once); if process forking 
is disabled (forkMode=never), Surefire will set the Java system properties 
of the currently executing Maven process, and finally change them back. 
This may have unanticipated side effects if you're running Maven in 
embedded mode and process forking is disabled.  Embedders should not 
disable forking during Surefire testing.  [Tests themselves might do 
something even more dangerous, like calling System.exit().]

4) -DskipTests [SUREFIRE-417]: To skip tests, we now recommend using "mvn 
install -DskipTests", instead of the clumsier -Dmaven.test.skip or 
-Dmaven.test.skip.exec.  (Note that you don't need to add "=true".)  The 
older properties still do what they used to do: "maven.test.skip" will 
skip both compiling and running the tests, and "maven.test.skip.exec" will 
compile the tests without running them.  -DskipTests will compile tests 
without running them.  It is also easier to type.

5) -Dtest [SUREFIRE-350]: You can set the "test" property on the command 
line to run just a single test using Surefire (or a number of tests that 
match a * wildcard pattern).  In previous versions of Surefire, if you 
specified a non-existent test with -Dtest=MyTset, no tests would run, but 
the build would succeed.  In Surefire 2.4, the build will fail if no tests 
match the specified pattern.  Some people had been using -Dtest=0 to avoid 
having to type -Dmaven.test.skip.exec; use -DskipTests instead.


Release Notes - Maven Surefire - Version 2.4


** Bug
     * [SUREFIRE-42] - TestListenerInvocationHandler incorrectly assumes 
getName()
     * [SUREFIRE-47] - JUnit suite() method ends up being called twice when 
tests are run with Surefire
     * [SUREFIRE-52] - XML Reports include testcases from previous tests
     * [SUREFIRE-55] - Incorrect splitting of command line arguments in 
ForkConfiguration
     * [SUREFIRE-57] - Invalid characters in XML reports
     * [SUREFIRE-58] - Not compatible with TestNG 5.4: 
InstantiationException: 
org.testng.internal.annotations.JDK15AnnotationFinder
     * [SUREFIRE-112] - StringIndexOutOfBoundsException building reports 
from TestNG tests
     * [SUREFIRE-115] - Surefire-JUnit does not recognize "suite"-methods
     * [SUREFIRE-117] - ability to add dependency to jvm's classpath rather 
in surefirebooter classloader
     * [SUREFIRE-118] - Cannot override read-only parameter: 
classpathElements
     * [SUREFIRE-119] - With testng, incorrect test numbers are reported if 
setup method throws exception.
     * [SUREFIRE-121] - System properties set on the command line get 
clobbered
     * [SUREFIRE-157] - Surefire Plugin fails to handle exception thrown 
from TestNG @BeforeTest method
     * [SUREFIRE-158] - Web site has incorrect source repository 
information
     * [SUREFIRE-160] - Bug into xml report generation
     * [SUREFIRE-161] - Result message of Surefire TestNG run with invalid 
<suiteXmlFile> not logical.
     * [SUREFIRE-164] - Classpath in XML report is wrong
     * [SUREFIRE-165] - TestNG JDK1.4 JavaDoc annotated classes never run 
... and now I know why.
     * [SUREFIRE-166] - trimStackTrace=true trims "caused by:" sections 
complelely away
     * [SUREFIRE-167] - Non-Abstract TestCase not executed if name starts 
with Abstract
     * [SUREFIRE-168] - TestNG @BeforeMethod annotations not being 
processed
     * [SUREFIRE-169] - No tests detected when both TestNG and JUnit in 
classpath
     * [SUREFIRE-177] - Groups stipulated in the pom file get ignored when 
using a suiteXMLFile
     * [SUREFIRE-287] - Regression: org.testng.xml.Praser#parse() signature 
changed
     * [SUREFIRE-307] - Tests fail if in path with spaces
     * [SUREFIRE-313] - build fails with ClassNotFoundException, BUT the 
class IS there!
     * [SUREFIRE-316] - [M206] Test doesn't work anymore
     * [SUREFIRE-331] - Resources can't be found on classLoader
     * [SUREFIRE-334] - useSystemClassLoader introduces dependencies
     * [SUREFIRE-337] - Regression: NoSuchMethodException: 
org.apache.maven.surefire.testng.TestNGXmlTestSuite.<init>(java.io.File, 
java.lang.String)
     * [SUREFIRE-345] - TestNG test fail with IllegalARgumentExceptoin
     * [SUREFIRE-346] - build failure when there is any class with name 
matching the test pattern, but is not a testcase
     * [SUREFIRE-350] - if test parameter is provided, and no match is 
found, an error should occur, not a successful build with 0 tests
     * [SUREFIRE-361] - The maven-surefire-plugin fails with an NPE in 
TestNG 4.7
     * [SUREFIRE-362] - SurefireBooter does not recognise the Properties 
sent by SurefirePlugin
     * [SUREFIRE-363] - The maven-surefire-plugin fails with a 
NoSuchMethodException.
     * [SUREFIRE-364] - 2.4-SNAPSHOT of 27th Oct 2007 has invalid Plexus 
dependency
     * [SUREFIRE-366] - Out Of Memory exceptions in 2.4 SNAPSHOTs
     * [SUREFIRE-376] - TestNG @AfterSuite failures are ignored
     * [SUREFIRE-377] - When JUnit and TestNG tests are in same project, 
only one set gets run
     * [SUREFIRE-378] - junit-dep 4.4 isn't detected; tests are treated as 
POJO tests.
     * [SUREFIRE-379] - When an exception occurs in @BeforeMethod, the 
exception is not recorded.
     * [SUREFIRE-380] - TestNG 5.2 not recognized
     * [SUREFIRE-385] - Booter can't decode properties when <groups> 
contains commas
     * [SUREFIRE-388] - JUnit4 not activated when using junit-dep
     * [SUREFIRE-389] - IncompatibleClassChangeError when 
useSystemClassLoader=true
     * [SUREFIRE-390] - New ITs tend to duplicate class names in default 
package; this confuses IDEA
     * [SUREFIRE-391] - current trunk doesn't work on os whithout /bin/bash 
(as FreeBsd)
     * [SUREFIRE-392] - TestNG group-thread-parallel test fails 
non-deterministically
     * [SUREFIRE-398] - junit44-ignore test XML claims the test took a 
million seconds
     * [SUREFIRE-403] - TestNgPathWithSpaces integration test fails on Unix
     * [SUREFIRE-411] - -Dtest=SingleTest doesn't work in multi-project 
reactor
     * [SUREFIRE-412] - useSystemClassLoader warning when forkMode=never, 
but useSystemClassLoader isn't set
     * [SUREFIRE-416] - Forked process inherits user.dir of parent process, 
multimodule builds behave differently from single module

** Improvement
     * [SUREFIRE-59] - Not compatible with TestNG 5.2: 
java.lang.NoSuchMethodError: org.testng.xml.XmlSuite.setParallel(Z)V
     * [SUREFIRE-136] - The plugin does not use JAVA_HOME variable and 
launches default JVM
     * [SUREFIRE-137] - provide option to list all of the test cases which 
failed when running a build
     * [SUREFIRE-181] - allow 'test' argument to take fully qualified class 
names as well as the current short format, and document the current 
behaviour better
     * [SUREFIRE-183] - enhance maven.surefire.debug property to allow a 
full debug string (and take the current default if none given for 
backwards compat)
     * [SUREFIRE-186] - Make the use of 'test' env-var more 
Eclipse-friendly
     * [SUREFIRE-303] - Ignored/Skipped tests are not reported
     * [SUREFIRE-325] - when parsing excludedGroups config prop, trim 
leading and trailing whitespace off of group names
     * [SUREFIRE-352] - Get rid of hardcoded TestNG dependency name.
     * [SUREFIRE-370] - Make testing frameworks artifact names configurable 
in plugin
     * [SUREFIRE-384] - Allow TestNG to generate its native XML output
     * [SUREFIRE-400] - Add basic statement of JUnit version support
     * [SUREFIRE-417] - Make new "skipTests" parameter to replace skipExec

** New Feature
     * [SUREFIRE-268] - Aggragated surefire report from modules
     * [SUREFIRE-320] - Add a timeout property to the plugin

** Task
     * [SUREFIRE-50] - write comprehensive tests for surefire
     * [SUREFIRE-394] - Resolve snapshot dependencies in Surefire 2.4


** Wish
     * [SUREFIRE-139] - JUnit 4 integration

Enjoy,

-The Maven Team