You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@maven.apache.org by "Elias Ross (JIRA)" <ji...@codehaus.org> on 2007/11/29 01:47:57 UTC
[jira] Created: (SUREFIRE-396) 2.4-SNAPSHOT does not execute TestNG
suite/test in correct order, collab does
2.4-SNAPSHOT does not execute TestNG suite/test in correct order, collab does
-----------------------------------------------------------------------------
Key: SUREFIRE-396
URL: http://jira.codehaus.org/browse/SUREFIRE-396
Project: Maven Surefire
Issue Type: Bug
Reporter: Elias Ross
Attachments: testsf-testng.zip
Please view and the attached .zip file containing two TestNG tests and one "Base" TestNG test, as well as the pom.xml etc.
Here is the output with surefire plugin 2.4-SNAPSHOT:
$ mvn test
-------------------------------------------------------
T E S T S
-------------------------------------------------------
[Parser] Running:
Command line suite
Running Command line suite
beforeSuite
beforeTest com.autodesk.testsf.SF2Test@14b7453
SF2Test.testTrue
SFTest.testTrue
afterTest com.autodesk.testsf.SF2Test@14b7453
afterSuite
Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.125 sec <<< FAILURE!
With 2.4-collab-SNAPSHOT:
-------------------------------------------------------
T E S T S
-------------------------------------------------------
[Parser] Running:
Command line suite
beforeSuite
Running com.autodesk.testsf.SFTest
beforeTest com.autodesk.testsf.SFTest@1690726
SFTest.testTrue
afterTest com.autodesk.testsf.SFTest@1690726
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.015 sec
Running com.autodesk.testsf.SF2Test
beforeTest com.autodesk.testsf.SF2Test@1820dda
SF2Test.testTrue
afterTest com.autodesk.testsf.SF2Test@1820dda
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.016 sec
afterSuite
Results :
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
SFTest and SF2Test both extend from BaseTest. Here are the classes, for your reference:
public abstract class BaseTest {
protected boolean test = false;
private static boolean suite = false;
@BeforeSuite
public static void beforeSuite() throws Exception {
System.out.println("beforeSuite");
suite = true;
}
@AfterSuite
public static void afterSuite() throws Exception {
System.out.println("afterSuite");
if (!suite)
throw new IllegalStateException();
suite = false;
}
@BeforeTest
public void beforeTest() throws Exception {
System.out.println("beforeTest " + this);
if (test)
throw new IllegalStateException();
if (!suite)
throw new IllegalStateException();
test = true;
}
@AfterTest
public void afterTest() {
System.out.println("afterTest " + this);
test = false;
}
}
@Test
public class SFTest extends BaseTest {
@Test
public void testTrue() {
System.out.println("SFTest.testTrue");
assertEquals(this.test, true);
}
}
@Test
public class SF2Test extends BaseTest {
@Test
public void testTrue() {
System.out.println("SF2Test.testTrue");
assertEquals(this.test, true);
}
}
... Can somebody explain the discrepancy? If not, this appears to be a bug
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Closed: (SUREFIRE-396) 2.4-SNAPSHOT does not execute TestNG
suite/test in correct order, collab does
Posted by "Dan Fabulich (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/SUREFIRE-396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dan Fabulich closed SUREFIRE-396.
---------------------------------
Resolution: Won't Fix
I believe collab is failing to run your suite file at all. When I run your tests using just TestNG from the command line (no Maven), TestNG generates the same single failure in SF2Test. I get the same thing running your tests in IntelliJ IDEA.
More generally, in my experience, when you put a @BeforeTest or @BeforeSuite annotation on a base class in TestNG, it is not called once for every instantiated subclass; it just just called once on the first subclass. This is arguably a TestNG bug, but it's definitely not a Surefire problem.
In collab, when we ignore your suite file, the individual test classes are each treated as their own separate tests, which is not what you asked for, but it's what you actually wanted.
> 2.4-SNAPSHOT does not execute TestNG suite/test in correct order, collab does
> -----------------------------------------------------------------------------
>
> Key: SUREFIRE-396
> URL: http://jira.codehaus.org/browse/SUREFIRE-396
> Project: Maven Surefire
> Issue Type: Bug
> Reporter: Elias Ross
> Attachments: testsf-testng.zip
>
>
> Please view and the attached .zip file containing two TestNG tests and one "Base" TestNG test, as well as the pom.xml etc.
> Here is the output with surefire plugin 2.4-SNAPSHOT:
> $ mvn test
> -------------------------------------------------------
> T E S T S
> -------------------------------------------------------
> [Parser] Running:
> Command line suite
> Running Command line suite
> beforeSuite
> beforeTest com.autodesk.testsf.SF2Test@14b7453
> SF2Test.testTrue
> SFTest.testTrue
> afterTest com.autodesk.testsf.SF2Test@14b7453
> afterSuite
> Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.125 sec <<< FAILURE!
> With 2.4-collab-SNAPSHOT:
> -------------------------------------------------------
> T E S T S
> -------------------------------------------------------
> [Parser] Running:
> Command line suite
> beforeSuite
> Running com.autodesk.testsf.SFTest
> beforeTest com.autodesk.testsf.SFTest@1690726
> SFTest.testTrue
> afterTest com.autodesk.testsf.SFTest@1690726
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.015 sec
> Running com.autodesk.testsf.SF2Test
> beforeTest com.autodesk.testsf.SF2Test@1820dda
> SF2Test.testTrue
> afterTest com.autodesk.testsf.SF2Test@1820dda
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.016 sec
> afterSuite
> Results :
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
> SFTest and SF2Test both extend from BaseTest. Here are the classes, for your reference:
> public abstract class BaseTest {
> protected boolean test = false;
> private static boolean suite = false;
> @BeforeSuite
> public static void beforeSuite() throws Exception {
> System.out.println("beforeSuite");
> suite = true;
> }
>
> @AfterSuite
> public static void afterSuite() throws Exception {
> System.out.println("afterSuite");
> if (!suite)
> throw new IllegalStateException();
> suite = false;
> }
>
> @BeforeTest
> public void beforeTest() throws Exception {
> System.out.println("beforeTest " + this);
> if (test)
> throw new IllegalStateException();
> if (!suite)
> throw new IllegalStateException();
> test = true;
> }
> @AfterTest
> public void afterTest() {
> System.out.println("afterTest " + this);
> test = false;
> }
> }
> @Test
> public class SFTest extends BaseTest {
>
> @Test
> public void testTrue() {
> System.out.println("SFTest.testTrue");
> assertEquals(this.test, true);
> }
> }
> @Test
> public class SF2Test extends BaseTest {
>
> @Test
> public void testTrue() {
> System.out.println("SF2Test.testTrue");
> assertEquals(this.test, true);
> }
> }
> ... Can somebody explain the discrepancy? If not, this appears to be a bug
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira