You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@maven.apache.org by Dan Fabulich <da...@fabulich.com> on 2008/01/15 22:18:14 UTC
[ANN] Maven Surefire Plugin 2.4 for Maven 2 Released
The Maven team is pleased to announce the release of the Maven Surefire Plugin,
version 2.4
http://maven.apache.org/plugins/maven-surefire-plugin/
This version fixes numerous long-outstanding bugs, notably in TestNG support.
You can run mvn -up to get the latest version of the plugin, or specify the
version in your project's plugin configuration:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.4</version>
</plugin>
By default, Maven automatically consumes the latest version of Surefire, so we
can reasonably expect some Maven users to begin using Surefire 2.4 immediately,
perhaps without their knowledge. In at least some cases, this will result in
tests failing that passed using Surefire 2.3.1. If this happens to you, the
first and easiest thing that you can do to get your build to work again is to
add an explicit version reference to Surefire 2.3.1, like this:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.3.1</version>
</plugin>
Especially watch out for these fixes, which may affect backwards compatibility:
1) "Abstract" test cases [SUREFIRE-167]: Surefire 2.3 would automatically skip
tests whose names start with "Abstract." In Surefire 2.4, we use reflection to
detect abstract tests and avoid instantiating them. However, some people had
come to rely on the old behavior, creating classes named "AbstractFooTest"
without marking them abstract, and without putting any tests in them. Surefire
2.4 will attempt to run these classes, and will register failures if they don't
work. Mark all abstract test cases with the "abstract" keyword.
2) Working directory [SUREFIRE-416]: By default, the working directory of the
running tests will be the ${basedir} of the current project, which defaults to
the "target" directory of the currently running project. Surefire 2.3's
behavior was inconsistent, so this may cause problems for you if your tests
somehow depend on the old behavior. You can configure the test working
directory using the "workingDirectory" configuration parameter.
3) System properties [SUREFIRE-416]: Surefire allows you to configure the
system properties of the currently running test. By default, Surefire will run
tests in a separate process (forkMode=once); if process forking is disabled
(forkMode=never), Surefire will set the Java system properties of the currently
executing Maven process, and finally change them back. This may have
unanticipated side effects if you're running Maven in embedded mode and process
forking is disabled. Embedders should not disable forking during Surefire
testing. [Tests themselves might do something even more dangerous, like
calling System.exit().]
4) -DskipTests [SUREFIRE-417]: To skip tests, we now recommend using "mvn
install -DskipTests", instead of the clumsier -Dmaven.test.skip or
-Dmaven.test.skip.exec. (Note that you don't need to add "=true".) The older
properties still do what they used to do: "maven.test.skip" will skip both
compiling and running the tests, and "maven.test.skip.exec" will compile the
tests without running them. -DskipTests will compile tests without running
them. It is also easier to type.
5) -Dtest [SUREFIRE-350]: You can set the "test" property on the command line
to run just a single test using Surefire (or a number of tests that match a *
wildcard pattern). In previous versions of Surefire, if you specified a
non-existent test with -Dtest=MyTset, no tests would run, but the build would
succeed. In Surefire 2.4, the build will fail if no tests match the specified
pattern. Some people had been using -Dtest=0 to avoid having to type
-Dmaven.test.skip.exec; use -DskipTests instead.
Release Notes - Maven Surefire - Version 2.4
** Bug
* [SUREFIRE-42] - TestListenerInvocationHandler incorrectly assumes
getName()
* [SUREFIRE-47] - JUnit suite() method ends up being called twice when
tests are run with Surefire
* [SUREFIRE-52] - XML Reports include testcases from previous tests
* [SUREFIRE-55] - Incorrect splitting of command line arguments in
ForkConfiguration
* [SUREFIRE-57] - Invalid characters in XML reports
* [SUREFIRE-58] - Not compatible with TestNG 5.4: InstantiationException:
org.testng.internal.annotations.JDK15AnnotationFinder
* [SUREFIRE-112] - StringIndexOutOfBoundsException building reports from
TestNG tests
* [SUREFIRE-115] - Surefire-JUnit does not recognize "suite"-methods
* [SUREFIRE-117] - ability to add dependency to jvm's classpath rather in
surefirebooter classloader
* [SUREFIRE-118] - Cannot override read-only parameter: classpathElements
* [SUREFIRE-119] - With testng, incorrect test numbers are reported if
setup method throws exception.
* [SUREFIRE-121] - System properties set on the command line get clobbered
* [SUREFIRE-157] - Surefire Plugin fails to handle exception thrown from
TestNG @BeforeTest method
* [SUREFIRE-158] - Web site has incorrect source repository information
* [SUREFIRE-160] - Bug into xml report generation
* [SUREFIRE-161] - Result message of Surefire TestNG run with invalid
<suiteXmlFile> not logical.
* [SUREFIRE-164] - Classpath in XML report is wrong
* [SUREFIRE-165] - TestNG JDK1.4 JavaDoc annotated classes never run ...
and now I know why.
* [SUREFIRE-166] - trimStackTrace=true trims "caused by:" sections
complelely away
* [SUREFIRE-167] - Non-Abstract TestCase not executed if name starts with
Abstract
* [SUREFIRE-168] - TestNG @BeforeMethod annotations not being processed
* [SUREFIRE-169] - No tests detected when both TestNG and JUnit in
classpath
* [SUREFIRE-177] - Groups stipulated in the pom file get ignored when using
a suiteXMLFile
* [SUREFIRE-287] - Regression: org.testng.xml.Praser#parse() signature
changed
* [SUREFIRE-307] - Tests fail if in path with spaces
* [SUREFIRE-313] - build fails with ClassNotFoundException, BUT the class
IS there!
* [SUREFIRE-316] - [M206] Test doesn't work anymore
* [SUREFIRE-331] - Resources can't be found on classLoader
* [SUREFIRE-334] - useSystemClassLoader introduces dependencies
* [SUREFIRE-337] - Regression: NoSuchMethodException:
org.apache.maven.surefire.testng.TestNGXmlTestSuite.<init>(java.io.File,
java.lang.String)
* [SUREFIRE-345] - TestNG test fail with IllegalARgumentExceptoin
* [SUREFIRE-346] - build failure when there is any class with name matching
the test pattern, but is not a testcase
* [SUREFIRE-350] - if test parameter is provided, and no match is found, an
error should occur, not a successful build with 0 tests
* [SUREFIRE-361] - The maven-surefire-plugin fails with an NPE in TestNG
4.7
* [SUREFIRE-362] - SurefireBooter does not recognise the Properties sent by
SurefirePlugin
* [SUREFIRE-363] - The maven-surefire-plugin fails with a
NoSuchMethodException.
* [SUREFIRE-364] - 2.4-SNAPSHOT of 27th Oct 2007 has invalid Plexus
dependency
* [SUREFIRE-366] - Out Of Memory exceptions in 2.4 SNAPSHOTs
* [SUREFIRE-376] - TestNG @AfterSuite failures are ignored
* [SUREFIRE-377] - When JUnit and TestNG tests are in same project, only
one set gets run
* [SUREFIRE-378] - junit-dep 4.4 isn't detected; tests are treated as POJO
tests.
* [SUREFIRE-379] - When an exception occurs in @BeforeMethod, the exception
is not recorded.
* [SUREFIRE-380] - TestNG 5.2 not recognized
* [SUREFIRE-385] - Booter can't decode properties when <groups> contains
commas
* [SUREFIRE-388] - JUnit4 not activated when using junit-dep
* [SUREFIRE-389] - IncompatibleClassChangeError when
useSystemClassLoader=true
* [SUREFIRE-390] - New ITs tend to duplicate class names in default
package; this confuses IDEA
* [SUREFIRE-391] - current trunk doesn't work on os whithout /bin/bash (as
FreeBsd)
* [SUREFIRE-392] - TestNG group-thread-parallel test fails
non-deterministically
* [SUREFIRE-398] - junit44-ignore test XML claims the test took a million
seconds
* [SUREFIRE-403] - TestNgPathWithSpaces integration test fails on Unix
* [SUREFIRE-411] - -Dtest=SingleTest doesn't work in multi-project reactor
* [SUREFIRE-412] - useSystemClassLoader warning when forkMode=never, but
useSystemClassLoader isn't set
* [SUREFIRE-416] - Forked process inherits user.dir of parent process,
multimodule builds behave differently from single module
** Improvement
* [SUREFIRE-59] - Not compatible with TestNG 5.2:
java.lang.NoSuchMethodError: org.testng.xml.XmlSuite.setParallel(Z)V
* [SUREFIRE-136] - The plugin does not use JAVA_HOME variable and launches
default JVM
* [SUREFIRE-137] - provide option to list all of the test cases which
failed when running a build
* [SUREFIRE-181] - allow 'test' argument to take fully qualified class
names as well as the current short format, and document the current behaviour
better
* [SUREFIRE-183] - enhance maven.surefire.debug property to allow a full
debug string (and take the current default if none given for backwards compat)
* [SUREFIRE-186] - Make the use of 'test' env-var more Eclipse-friendly
* [SUREFIRE-303] - Ignored/Skipped tests are not reported
* [SUREFIRE-325] - when parsing excludedGroups config prop, trim leading
and trailing whitespace off of group names
* [SUREFIRE-352] - Get rid of hardcoded TestNG dependency name.
* [SUREFIRE-370] - Make testing frameworks artifact names configurable in
plugin
* [SUREFIRE-384] - Allow TestNG to generate its native XML output
* [SUREFIRE-400] - Add basic statement of JUnit version support
* [SUREFIRE-417] - Make new "skipTests" parameter to replace skipExec
** New Feature
* [SUREFIRE-268] - Aggragated surefire report from modules
* [SUREFIRE-320] - Add a timeout property to the plugin
** Task
* [SUREFIRE-50] - write comprehensive tests for surefire
* [SUREFIRE-394] - Resolve snapshot dependencies in Surefire 2.4
** Wish
* [SUREFIRE-139] - JUnit 4 integration
Enjoy,
-The Maven Team
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: [ANN] Maven Surefire Plugin 2.4 for Maven 2 Released
Posted by Dan Tran <da...@gmail.com>.
Dan,
I am so much impressed with the works you have put in for this plugin.
Thank you and thank you
-D
On Jan 15, 2008 1:18 PM, Dan Fabulich <da...@fabulich.com> wrote:
> The Maven team is pleased to announce the release of the Maven Surefire Plugin,
> version 2.4
>
> http://maven.apache.org/plugins/maven-surefire-plugin/
>
> This version fixes numerous long-outstanding bugs, notably in TestNG support.
>
> You can run mvn -up to get the latest version of the plugin, or specify the
> version in your project's plugin configuration:
>
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-surefire-plugin</artifactId>
> <version>2.4</version>
> </plugin>
>
> By default, Maven automatically consumes the latest version of Surefire, so we
> can reasonably expect some Maven users to begin using Surefire 2.4 immediately,
> perhaps without their knowledge. In at least some cases, this will result in
> tests failing that passed using Surefire 2.3.1. If this happens to you, the
> first and easiest thing that you can do to get your build to work again is to
> add an explicit version reference to Surefire 2.3.1, like this:
>
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-surefire-plugin</artifactId>
> <version>2.3.1</version>
> </plugin>
>
> Especially watch out for these fixes, which may affect backwards compatibility:
>
> 1) "Abstract" test cases [SUREFIRE-167]: Surefire 2.3 would automatically skip
> tests whose names start with "Abstract." In Surefire 2.4, we use reflection to
> detect abstract tests and avoid instantiating them. However, some people had
> come to rely on the old behavior, creating classes named "AbstractFooTest"
> without marking them abstract, and without putting any tests in them. Surefire
> 2.4 will attempt to run these classes, and will register failures if they don't
> work. Mark all abstract test cases with the "abstract" keyword.
>
> 2) Working directory [SUREFIRE-416]: By default, the working directory of the
> running tests will be the ${basedir} of the current project, which defaults to
> the "target" directory of the currently running project. Surefire 2.3's
> behavior was inconsistent, so this may cause problems for you if your tests
> somehow depend on the old behavior. You can configure the test working
> directory using the "workingDirectory" configuration parameter.
>
> 3) System properties [SUREFIRE-416]: Surefire allows you to configure the
> system properties of the currently running test. By default, Surefire will run
> tests in a separate process (forkMode=once); if process forking is disabled
> (forkMode=never), Surefire will set the Java system properties of the currently
> executing Maven process, and finally change them back. This may have
> unanticipated side effects if you're running Maven in embedded mode and process
> forking is disabled. Embedders should not disable forking during Surefire
> testing. [Tests themselves might do something even more dangerous, like
> calling System.exit().]
>
> 4) -DskipTests [SUREFIRE-417]: To skip tests, we now recommend using "mvn
> install -DskipTests", instead of the clumsier -Dmaven.test.skip or
> -Dmaven.test.skip.exec. (Note that you don't need to add "=true".) The older
> properties still do what they used to do: "maven.test.skip" will skip both
> compiling and running the tests, and "maven.test.skip.exec" will compile the
> tests without running them. -DskipTests will compile tests without running
> them. It is also easier to type.
>
> 5) -Dtest [SUREFIRE-350]: You can set the "test" property on the command line
> to run just a single test using Surefire (or a number of tests that match a *
> wildcard pattern). In previous versions of Surefire, if you specified a
> non-existent test with -Dtest=MyTset, no tests would run, but the build would
> succeed. In Surefire 2.4, the build will fail if no tests match the specified
> pattern. Some people had been using -Dtest=0 to avoid having to type
> -Dmaven.test.skip.exec; use -DskipTests instead.
>
>
> Release Notes - Maven Surefire - Version 2.4
>
>
> ** Bug
> * [SUREFIRE-42] - TestListenerInvocationHandler incorrectly assumes
> getName()
> * [SUREFIRE-47] - JUnit suite() method ends up being called twice when
> tests are run with Surefire
> * [SUREFIRE-52] - XML Reports include testcases from previous tests
> * [SUREFIRE-55] - Incorrect splitting of command line arguments in
> ForkConfiguration
> * [SUREFIRE-57] - Invalid characters in XML reports
> * [SUREFIRE-58] - Not compatible with TestNG 5.4: InstantiationException:
> org.testng.internal.annotations.JDK15AnnotationFinder
> * [SUREFIRE-112] - StringIndexOutOfBoundsException building reports from
> TestNG tests
> * [SUREFIRE-115] - Surefire-JUnit does not recognize "suite"-methods
> * [SUREFIRE-117] - ability to add dependency to jvm's classpath rather in
> surefirebooter classloader
> * [SUREFIRE-118] - Cannot override read-only parameter: classpathElements
> * [SUREFIRE-119] - With testng, incorrect test numbers are reported if
> setup method throws exception.
> * [SUREFIRE-121] - System properties set on the command line get clobbered
> * [SUREFIRE-157] - Surefire Plugin fails to handle exception thrown from
> TestNG @BeforeTest method
> * [SUREFIRE-158] - Web site has incorrect source repository information
> * [SUREFIRE-160] - Bug into xml report generation
> * [SUREFIRE-161] - Result message of Surefire TestNG run with invalid
> <suiteXmlFile> not logical.
> * [SUREFIRE-164] - Classpath in XML report is wrong
> * [SUREFIRE-165] - TestNG JDK1.4 JavaDoc annotated classes never run ...
> and now I know why.
> * [SUREFIRE-166] - trimStackTrace=true trims "caused by:" sections
> complelely away
> * [SUREFIRE-167] - Non-Abstract TestCase not executed if name starts with
> Abstract
> * [SUREFIRE-168] - TestNG @BeforeMethod annotations not being processed
> * [SUREFIRE-169] - No tests detected when both TestNG and JUnit in
> classpath
> * [SUREFIRE-177] - Groups stipulated in the pom file get ignored when using
> a suiteXMLFile
> * [SUREFIRE-287] - Regression: org.testng.xml.Praser#parse() signature
> changed
> * [SUREFIRE-307] - Tests fail if in path with spaces
> * [SUREFIRE-313] - build fails with ClassNotFoundException, BUT the class
> IS there!
> * [SUREFIRE-316] - [M206] Test doesn't work anymore
> * [SUREFIRE-331] - Resources can't be found on classLoader
> * [SUREFIRE-334] - useSystemClassLoader introduces dependencies
> * [SUREFIRE-337] - Regression: NoSuchMethodException:
> org.apache.maven.surefire.testng.TestNGXmlTestSuite.<init>(java.io.File,
> java.lang.String)
> * [SUREFIRE-345] - TestNG test fail with IllegalARgumentExceptoin
> * [SUREFIRE-346] - build failure when there is any class with name matching
> the test pattern, but is not a testcase
> * [SUREFIRE-350] - if test parameter is provided, and no match is found, an
> error should occur, not a successful build with 0 tests
> * [SUREFIRE-361] - The maven-surefire-plugin fails with an NPE in TestNG
> 4.7
> * [SUREFIRE-362] - SurefireBooter does not recognise the Properties sent by
> SurefirePlugin
> * [SUREFIRE-363] - The maven-surefire-plugin fails with a
> NoSuchMethodException.
> * [SUREFIRE-364] - 2.4-SNAPSHOT of 27th Oct 2007 has invalid Plexus
> dependency
> * [SUREFIRE-366] - Out Of Memory exceptions in 2.4 SNAPSHOTs
> * [SUREFIRE-376] - TestNG @AfterSuite failures are ignored
> * [SUREFIRE-377] - When JUnit and TestNG tests are in same project, only
> one set gets run
> * [SUREFIRE-378] - junit-dep 4.4 isn't detected; tests are treated as POJO
> tests.
> * [SUREFIRE-379] - When an exception occurs in @BeforeMethod, the exception
> is not recorded.
> * [SUREFIRE-380] - TestNG 5.2 not recognized
> * [SUREFIRE-385] - Booter can't decode properties when <groups> contains
> commas
> * [SUREFIRE-388] - JUnit4 not activated when using junit-dep
> * [SUREFIRE-389] - IncompatibleClassChangeError when
> useSystemClassLoader=true
> * [SUREFIRE-390] - New ITs tend to duplicate class names in default
> package; this confuses IDEA
> * [SUREFIRE-391] - current trunk doesn't work on os whithout /bin/bash (as
> FreeBsd)
> * [SUREFIRE-392] - TestNG group-thread-parallel test fails
> non-deterministically
> * [SUREFIRE-398] - junit44-ignore test XML claims the test took a million
> seconds
> * [SUREFIRE-403] - TestNgPathWithSpaces integration test fails on Unix
> * [SUREFIRE-411] - -Dtest=SingleTest doesn't work in multi-project reactor
> * [SUREFIRE-412] - useSystemClassLoader warning when forkMode=never, but
> useSystemClassLoader isn't set
> * [SUREFIRE-416] - Forked process inherits user.dir of parent process,
> multimodule builds behave differently from single module
>
> ** Improvement
> * [SUREFIRE-59] - Not compatible with TestNG 5.2:
> java.lang.NoSuchMethodError: org.testng.xml.XmlSuite.setParallel(Z)V
> * [SUREFIRE-136] - The plugin does not use JAVA_HOME variable and launches
> default JVM
> * [SUREFIRE-137] - provide option to list all of the test cases which
> failed when running a build
> * [SUREFIRE-181] - allow 'test' argument to take fully qualified class
> names as well as the current short format, and document the current behaviour
> better
> * [SUREFIRE-183] - enhance maven.surefire.debug property to allow a full
> debug string (and take the current default if none given for backwards compat)
> * [SUREFIRE-186] - Make the use of 'test' env-var more Eclipse-friendly
> * [SUREFIRE-303] - Ignored/Skipped tests are not reported
> * [SUREFIRE-325] - when parsing excludedGroups config prop, trim leading
> and trailing whitespace off of group names
> * [SUREFIRE-352] - Get rid of hardcoded TestNG dependency name.
> * [SUREFIRE-370] - Make testing frameworks artifact names configurable in
> plugin
> * [SUREFIRE-384] - Allow TestNG to generate its native XML output
> * [SUREFIRE-400] - Add basic statement of JUnit version support
> * [SUREFIRE-417] - Make new "skipTests" parameter to replace skipExec
>
> ** New Feature
> * [SUREFIRE-268] - Aggragated surefire report from modules
> * [SUREFIRE-320] - Add a timeout property to the plugin
>
> ** Task
> * [SUREFIRE-50] - write comprehensive tests for surefire
> * [SUREFIRE-394] - Resolve snapshot dependencies in Surefire 2.4
>
>
> ** Wish
> * [SUREFIRE-139] - JUnit 4 integration
>
> Enjoy,
>
> -The Maven Team
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org