You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@maven.apache.org by "Kenney Westerhof (JIRA)" <ji...@codehaus.org> on 2005/07/14 22:32:57 UTC
[jira] Created: (MNG-591) Integration tests lifecycle
Integration tests lifecycle
---------------------------
Key: MNG-591
URL: http://jira.codehaus.org/browse/MNG-591
Project: Maven 2
Type: Improvement
Reporter: Kenney Westerhof
I'm trying to do an integration test that depends on a war/ear to be deployed.
What i'm missing is:
- integration-test-compile stage and/or:
- a way to specify an integrationTestSourceDirectory or multiple testSourceDirectories in the pom
I can't put the test sources in src/test/java because then surefire will run them in the test stage, when
there's no artifact to deploy yet.
[Btw, I'm doing this while creating a cactus plugin, for the moment using cargo in the TestSuite itself to deploy.]
The idea is that the integration test sources go in src/itest/*; that there be a integration-test-compile,
integration-test-package and/or integration-test-appdeploy[or something] and that surefire
is also bound to integration-test.
Maybe something can be done using the src/test/project/some-project/.... approach seen in
maven-javadoc-plugin, maven-site-plugin and maven-eclipse-plugin (i'd like to see some of that
standardized anyway to allow plugin testing generally - which can also be seen as integration testing).
Thoughts, comments, approaches?
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: Integration vs acceptance tests [was (MNG-591) Integration tests lifecycle]
Posted by John Fallows <jo...@gmail.com>.
On 7/25/05, Rafal Krzewski <Ra...@caltha.pl> wrote:
> John Fallows wrote:
>
> > Is it always considered a build failure if integration tests do not pass 100%?
>
> I did you mean acceptance tests?
I'm glad you raised this, Rafal. I agree with your description below.
My main point is to question the need for separation in source
directory structure between unit tests and integration tests, if they
are both required to pass at 100%, as you have explained.
Further, it would seem to make sense to separate out acceptance tests
into their own source directory so that they could easily be isolated,
and therefore not be required to pass at 100% success.
> In my understanding integration tests should pass 100% each time they
> are run. The difference between unit and integration tests is only in
> granularity. Unit tests focus on each component (one or few classes) in
> separation, and integration tests focus on many components wired
> together into a larger artifact - a GUI or web application for example.
>
> Acceptance tests were (I believe) first introduced in eXtreme
> Programming methodology. For each milestone (iteration) of application
> development a number of features is scheduled, and a tests that verify
> if the application provides the feature are created. At the beginning of
> an iteration all of it's acceptance tests are expected to fail, and at
> the end all are expected to pass - any failing tests indicate features
> that must be rescheduled for further iterations. After the iteration is
> completed the acceptance tests become regression tests that guard
> developed features from being broken further down the road.
>
> An acceptance test could be both an unit test or integration test,
> depending on the type application. Usually the customer cares about
> features that are accessible from GUI or web front end and therefore
> acceptance tests will be integration tests, but when developing an
> utility library unit tests could be valid acceptance tests.
>
> As I understand it, acceptance test has the following features:
>
> 1) it expresses a contract item between the customer and the developer,
> and therefore it should have an associated bugtracker item / XP story
> card id for reference.
> 2) belongs to a specific milestone - should have a version associated
> with it.
> 3) it's failure should not cause a SNAPSHOT build to fail.
> 4) it's failure should cause a release build to fail iff release version
> = tests milestone.
>
> I think acceptance tests support would be an interesting enhancement to
> maven testing framework.
> Probably the simplest way of dealing with it would be adding a JavaDoc
> annotation to the test methods, and using XDoclet to extract the
> information:
>
> /**
> * Test adding a book to the catalogue.
> * @acceptance 1.0 LIBRARY-9
> */
> public void testAddBook() {
> // ...
> }
>
> The information could be used to display an useful report combined with
> versions information from the POM: total/completed features per version,
> and as dashboard data source.
>
> Rafal
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
Kind Regards,
John Fallows.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Integration vs acceptance tests [was (MNG-591) Integration tests
lifecycle]
Posted by Rafal Krzewski <Ra...@caltha.pl>.
John Fallows wrote:
> Is it always considered a build failure if integration tests do not pass 100%?
I did you mean acceptance tests?
In my understanding integration tests should pass 100% each time they
are run. The difference between unit and integration tests is only in
granularity. Unit tests focus on each component (one or few classes) in
separation, and integration tests focus on many components wired
together into a larger artifact - a GUI or web application for example.
Acceptance tests were (I believe) first introduced in eXtreme
Programming methodology. For each milestone (iteration) of application
development a number of features is scheduled, and a tests that verify
if the application provides the feature are created. At the beginning of
an iteration all of it's acceptance tests are expected to fail, and at
the end all are expected to pass - any failing tests indicate features
that must be rescheduled for further iterations. After the iteration is
completed the acceptance tests become regression tests that guard
developed features from being broken further down the road.
An acceptance test could be both an unit test or integration test,
depending on the type application. Usually the customer cares about
features that are accessible from GUI or web front end and therefore
acceptance tests will be integration tests, but when developing an
utility library unit tests could be valid acceptance tests.
As I understand it, acceptance test has the following features:
1) it expresses a contract item between the customer and the developer,
and therefore it should have an associated bugtracker item / XP story
card id for reference.
2) belongs to a specific milestone - should have a version associated
with it.
3) it's failure should not cause a SNAPSHOT build to fail.
4) it's failure should cause a release build to fail iff release version
= tests milestone.
I think acceptance tests support would be an interesting enhancement to
maven testing framework.
Probably the simplest way of dealing with it would be adding a JavaDoc
annotation to the test methods, and using XDoclet to extract the
information:
/**
* Test adding a book to the catalogue.
* @acceptance 1.0 LIBRARY-9
*/
public void testAddBook() {
// ...
}
The information could be used to display an useful report combined with
versions information from the POM: total/completed features per version,
and as dashboard data source.
Rafal
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: [jira] Created: (MNG-591) Integration tests lifecycle
Posted by John Fallows <jo...@gmail.com>.
On 7/15/05, Vincent Massol <vm...@pivolis.com> wrote:
> Hi Kenney,
>
> > -----Original Message-----
> > From: Kenney Westerhof (JIRA) [mailto:jira@codehaus.org]
> > Sent: jeudi 14 juillet 2005 22:33
> > To: dev@maven.apache.org
> > Subject: [jira] Created: (MNG-591) Integration tests lifecycle
>
> [snip]
>
> > [Btw, I'm doing this while creating a cactus plugin, for the moment using
> > cargo in the TestSuite itself to deploy.]
>
> Cool :-)
>
> > The idea is that the integration test sources go in src/itest/*;
>
> Don't you mean src/test/itest/*?
Is it always considered a build failure if integration tests do not pass 100%?
If so, then src/test/* is the right place for this stuff, otherwise
integration tests should be separated out into something like
src/itest/*.
M2 has done a great job of reducing the need for pattern-based
customization of directory contents processing. Take java vs
resources source directories as an example.
The same rule of thumb would seem to apply here, assuming that
integration tests are not required to pass at 100% on every build.
> I think I'd prefer src/test/cactus actually because there can possibly be
> different types of integration tests for the same project and each would
> need to be run with its own testing framework (which might not be junit
> BTW).
That's a good point about having multiple possible execution engines
for integration testing, including junit.
According to the above feedback then, the cactus integration tests
would reside in either src/test/cactus or src/itest/cactus.
[snip]
Kind Regards,
John Fallows.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[jira] Updated: (MNG-591) Integration tests lifecycle
Posted by "Brett Porter (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MNG-591?page=all ]
Brett Porter updated MNG-591:
-----------------------------
Assign To: (was: Brett Porter)
> Integration tests lifecycle
> ---------------------------
>
> Key: MNG-591
> URL: http://jira.codehaus.org/browse/MNG-591
> Project: Maven 2
> Type: Improvement
> Components: Plugins and Lifecycle
> Reporter: Kenney Westerhof
> Priority: Blocker
> Fix For: 2.1
>
> Original Estimate: 8 hours
> Time Spent: 45 minutes
> Remaining: 7 hours, 15 minutes
>
> I'm trying to do an integration test that depends on a war/ear to be deployed.
> What i'm missing is:
> - integration-test-compile stage and/or:
> - a way to specify an integrationTestSourceDirectory or multiple testSourceDirectories in the pom
> I can't put the test sources in src/test/java because then surefire will run them in the test stage, when
> there's no artifact to deploy yet.
> [Btw, I'm doing this while creating a cactus plugin, for the moment using cargo in the TestSuite itself to deploy.]
> The idea is that the integration test sources go in src/itest/*; that there be a integration-test-compile,
> integration-test-package and/or integration-test-appdeploy[or something] and that surefire
> is also bound to integration-test.
> Maybe something can be done using the src/test/project/some-project/.... approach seen in
> maven-javadoc-plugin, maven-site-plugin and maven-eclipse-plugin (i'd like to see some of that
> standardized anyway to allow plugin testing generally - which can also be seen as integration testing).
> Thoughts, comments, approaches?
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[jira] Commented: (MNG-591) Integration tests lifecycle
Posted by "Dave Sag (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MNG-591?page=comments#action_59690 ]
Dave Sag commented on MNG-591:
------------------------------
I would like to split out some of my heavier tests as integration tests. i'd kinda hoped that if I put my src files into src/itest/java/.. and added a simple
<plugin>
<artifactId>maven-it-plugin</artifactId>
</plugin>
then maven2 would have magically worked. alas not.
i'm not at work now so can access svn (hooray) - but can't find where to check out the sandbox plugins.
I was able to check out http://svn.apache.org/repos/asf/maven/components/trunk/ and build it fine, and some googling turned up http://svn.apache.org/repos/asf/maven/plugins/trunk/ too but i can't find, nor guess the sandbox url. could someone perhaps post that here, or even a url of a page detailing where to find such things.
cheers dave
> Integration tests lifecycle
> ---------------------------
>
> Key: MNG-591
> URL: http://jira.codehaus.org/browse/MNG-591
> Project: Maven 2
> Type: Improvement
> Components: Plugins and Lifecycle
> Reporter: Kenney Westerhof
> Priority: Blocker
> Fix For: 2.1
>
> Original Estimate: 8 hours
> Time Spent: 45 minutes
> Remaining: 7 hours, 15 minutes
>
> I'm trying to do an integration test that depends on a war/ear to be deployed.
> What i'm missing is:
> - integration-test-compile stage and/or:
> - a way to specify an integrationTestSourceDirectory or multiple testSourceDirectories in the pom
> I can't put the test sources in src/test/java because then surefire will run them in the test stage, when
> there's no artifact to deploy yet.
> [Btw, I'm doing this while creating a cactus plugin, for the moment using cargo in the TestSuite itself to deploy.]
> The idea is that the integration test sources go in src/itest/*; that there be a integration-test-compile,
> integration-test-package and/or integration-test-appdeploy[or something] and that surefire
> is also bound to integration-test.
> Maybe something can be done using the src/test/project/some-project/.... approach seen in
> maven-javadoc-plugin, maven-site-plugin and maven-eclipse-plugin (i'd like to see some of that
> standardized anyway to allow plugin testing generally - which can also be seen as integration testing).
> Thoughts, comments, approaches?
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[jira] Commented: (MNG-591) Integration tests lifecycle
Posted by "Brett Porter (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MNG-591?page=comments#action_44921 ]
Brett Porter commented on MNG-591:
----------------------------------
I'm taking a look at this. Just to answer about dependencies and scopes - I think per plugin dependencies are a better idea than going to phase over scope. I originally proposed phase instead of scope, but now see the relationships not mapping that way, as we discussed in March.
So I think test deps go to all tests, and anything specific to a certain type of test goes to that plugin.
> Integration tests lifecycle
> ---------------------------
>
> Key: MNG-591
> URL: http://jira.codehaus.org/browse/MNG-591
> Project: Maven 2
> Type: Improvement
> Reporter: Kenney Westerhof
> Assignee: Brett Porter
> Priority: Blocker
> Fix For: 2.0-beta-1
>
> Original Estimate: 1 day
> Remaining: 1 day
>
> I'm trying to do an integration test that depends on a war/ear to be deployed.
> What i'm missing is:
> - integration-test-compile stage and/or:
> - a way to specify an integrationTestSourceDirectory or multiple testSourceDirectories in the pom
> I can't put the test sources in src/test/java because then surefire will run them in the test stage, when
> there's no artifact to deploy yet.
> [Btw, I'm doing this while creating a cactus plugin, for the moment using cargo in the TestSuite itself to deploy.]
> The idea is that the integration test sources go in src/itest/*; that there be a integration-test-compile,
> integration-test-package and/or integration-test-appdeploy[or something] and that surefire
> is also bound to integration-test.
> Maybe something can be done using the src/test/project/some-project/.... approach seen in
> maven-javadoc-plugin, maven-site-plugin and maven-eclipse-plugin (i'd like to see some of that
> standardized anyway to allow plugin testing generally - which can also be seen as integration testing).
> Thoughts, comments, approaches?
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[jira] Commented: (MNG-591) Integration tests lifecycle
Posted by "Kenney Westerhof (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MNG-591?page=comments#action_43635 ]
Kenney Westerhof commented on MNG-591:
--------------------------------------
:) That really wasn't necessary! I should have filed it.
As long as we're being complete, I'll add another reason this is nice to have and some more food for thought:
The xdoclet plugin currently has to have dependencies on all plugins for xdoclet itself (container / environment specific),
because there's no way to add those to the plugin execution environment, except now using <extensions> which contaminates
the root classloader. So basically by allowing this you can have plugins for the plugins.
However, what we need to consider is the need to add some libraries to higher level than the plugin classloader, for instance to
a lifecycle phase. Currently artifacts with scope=provided aren't being resolved in the test scope:
(ScopeArtifactFilter.java:
else if ( DefaultArtifact.SCOPE_TEST.equals( scope ) )
{
providedScope = false;
compileScope = true;
runtimeScope = true;
testScope = true;
}
)
This poses a problem for the eclipse plugin. Enabling this will fix that, but they stil aren't included in the plugin
classpath - the plugin will need to create it's own classloader for that. However, sometimes just an -api artifact is
depended upon, and during tests an -impl version should be provided - ideally this artifact is present in the classpath during all mojo
executions in a particular phase. (This may be a bad example since there' s only one test plugin - does anybody have more examples?)
> Integration tests lifecycle
> ---------------------------
>
> Key: MNG-591
> URL: http://jira.codehaus.org/browse/MNG-591
> Project: Maven 2
> Type: Improvement
> Reporter: Kenney Westerhof
> Priority: Blocker
> Fix For: 2.0-beta-1
>
>
> I'm trying to do an integration test that depends on a war/ear to be deployed.
> What i'm missing is:
> - integration-test-compile stage and/or:
> - a way to specify an integrationTestSourceDirectory or multiple testSourceDirectories in the pom
> I can't put the test sources in src/test/java because then surefire will run them in the test stage, when
> there's no artifact to deploy yet.
> [Btw, I'm doing this while creating a cactus plugin, for the moment using cargo in the TestSuite itself to deploy.]
> The idea is that the integration test sources go in src/itest/*; that there be a integration-test-compile,
> integration-test-package and/or integration-test-appdeploy[or something] and that surefire
> is also bound to integration-test.
> Maybe something can be done using the src/test/project/some-project/.... approach seen in
> maven-javadoc-plugin, maven-site-plugin and maven-eclipse-plugin (i'd like to see some of that
> standardized anyway to allow plugin testing generally - which can also be seen as integration testing).
> Thoughts, comments, approaches?
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[jira] Updated: (MNG-591) Integration tests lifecycle
Posted by "Brett Porter (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MNG-591?page=all ]
Brett Porter updated MNG-591:
-----------------------------
Fix Version: (was: 2.0-beta-1)
2.0-beta-2
> Integration tests lifecycle
> ---------------------------
>
> Key: MNG-591
> URL: http://jira.codehaus.org/browse/MNG-591
> Project: Maven 2
> Type: Improvement
> Reporter: Kenney Westerhof
> Assignee: Brett Porter
> Priority: Blocker
> Fix For: 2.0-beta-2
>
> Original Estimate: 8 hours
> Time Spent: 45 minutes
> Remaining: 7 hours, 15 minutes
>
> I'm trying to do an integration test that depends on a war/ear to be deployed.
> What i'm missing is:
> - integration-test-compile stage and/or:
> - a way to specify an integrationTestSourceDirectory or multiple testSourceDirectories in the pom
> I can't put the test sources in src/test/java because then surefire will run them in the test stage, when
> there's no artifact to deploy yet.
> [Btw, I'm doing this while creating a cactus plugin, for the moment using cargo in the TestSuite itself to deploy.]
> The idea is that the integration test sources go in src/itest/*; that there be a integration-test-compile,
> integration-test-package and/or integration-test-appdeploy[or something] and that surefire
> is also bound to integration-test.
> Maybe something can be done using the src/test/project/some-project/.... approach seen in
> maven-javadoc-plugin, maven-site-plugin and maven-eclipse-plugin (i'd like to see some of that
> standardized anyway to allow plugin testing generally - which can also be seen as integration testing).
> Thoughts, comments, approaches?
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[jira] Commented: (MNG-591) Integration tests lifecycle
Posted by "John Casey (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MNG-591?page=comments#action_44291 ]
John Casey commented on MNG-591:
--------------------------------
in the xdoclet example above, would it make more sense to emulate the build-extension pattern and allow plugin-extensions that are loaded into the plugin container? I know this is a repackaging of the plugin-dependency argument, but I believe calling them plugin-extensions would be more "correct".
Also, it would remain up to the plugin's underlying technology (xdoclet, f.e.) to actually discover the optional components contained in the extension artifacts...we'd just make them available.
> Integration tests lifecycle
> ---------------------------
>
> Key: MNG-591
> URL: http://jira.codehaus.org/browse/MNG-591
> Project: Maven 2
> Type: Improvement
> Reporter: Kenney Westerhof
> Assignee: John Casey
> Priority: Blocker
> Fix For: 2.0-beta-1
>
>
> I'm trying to do an integration test that depends on a war/ear to be deployed.
> What i'm missing is:
> - integration-test-compile stage and/or:
> - a way to specify an integrationTestSourceDirectory or multiple testSourceDirectories in the pom
> I can't put the test sources in src/test/java because then surefire will run them in the test stage, when
> there's no artifact to deploy yet.
> [Btw, I'm doing this while creating a cactus plugin, for the moment using cargo in the TestSuite itself to deploy.]
> The idea is that the integration test sources go in src/itest/*; that there be a integration-test-compile,
> integration-test-package and/or integration-test-appdeploy[or something] and that surefire
> is also bound to integration-test.
> Maybe something can be done using the src/test/project/some-project/.... approach seen in
> maven-javadoc-plugin, maven-site-plugin and maven-eclipse-plugin (i'd like to see some of that
> standardized anyway to allow plugin testing generally - which can also be seen as integration testing).
> Thoughts, comments, approaches?
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[jira] Commented: (MNG-591) Integration tests lifecycle
Posted by "John Casey (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MNG-591?page=comments#action_43628 ]
John Casey commented on MNG-591:
--------------------------------
maybe we need to add dependencies that are included in the execution context of a plugin ONLY...that way, we can have cactus-specific dependencies which are only relevant for our project (i.e. not needed for every cactus execution everywhere...they'd be plugin-deps in this case).
Something like:
<build>
<plugins>
<plugin>
<!-- g:a:v -->
<dependencies>
<dependency>
<groupId>com.myorg.project</groupId>
<artifactId>myproject-testharness</artifactId>
<version>1.0</version>
</dependency>
</dependencies>
<configuration/>
</plugin>
</plugins>
</build>
It's a little awkward, but at least it's immediately apparent where these deps will be used...
> Integration tests lifecycle
> ---------------------------
>
> Key: MNG-591
> URL: http://jira.codehaus.org/browse/MNG-591
> Project: Maven 2
> Type: Improvement
> Reporter: Kenney Westerhof
> Priority: Blocker
> Fix For: 2.0-beta-1
>
>
> I'm trying to do an integration test that depends on a war/ear to be deployed.
> What i'm missing is:
> - integration-test-compile stage and/or:
> - a way to specify an integrationTestSourceDirectory or multiple testSourceDirectories in the pom
> I can't put the test sources in src/test/java because then surefire will run them in the test stage, when
> there's no artifact to deploy yet.
> [Btw, I'm doing this while creating a cactus plugin, for the moment using cargo in the TestSuite itself to deploy.]
> The idea is that the integration test sources go in src/itest/*; that there be a integration-test-compile,
> integration-test-package and/or integration-test-appdeploy[or something] and that surefire
> is also bound to integration-test.
> Maybe something can be done using the src/test/project/some-project/.... approach seen in
> maven-javadoc-plugin, maven-site-plugin and maven-eclipse-plugin (i'd like to see some of that
> standardized anyway to allow plugin testing generally - which can also be seen as integration testing).
> Thoughts, comments, approaches?
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[jira] Commented: (MNG-591) Integration tests lifecycle
Posted by "John Casey (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MNG-591?page=comments#action_43634 ]
John Casey commented on MNG-591:
--------------------------------
the previous comment was actually kenney's idea, I just thought we need to keep it in the discussion... ;-)
wanna keep the attribution straight here...
> Integration tests lifecycle
> ---------------------------
>
> Key: MNG-591
> URL: http://jira.codehaus.org/browse/MNG-591
> Project: Maven 2
> Type: Improvement
> Reporter: Kenney Westerhof
> Priority: Blocker
> Fix For: 2.0-beta-1
>
>
> I'm trying to do an integration test that depends on a war/ear to be deployed.
> What i'm missing is:
> - integration-test-compile stage and/or:
> - a way to specify an integrationTestSourceDirectory or multiple testSourceDirectories in the pom
> I can't put the test sources in src/test/java because then surefire will run them in the test stage, when
> there's no artifact to deploy yet.
> [Btw, I'm doing this while creating a cactus plugin, for the moment using cargo in the TestSuite itself to deploy.]
> The idea is that the integration test sources go in src/itest/*; that there be a integration-test-compile,
> integration-test-package and/or integration-test-appdeploy[or something] and that surefire
> is also bound to integration-test.
> Maybe something can be done using the src/test/project/some-project/.... approach seen in
> maven-javadoc-plugin, maven-site-plugin and maven-eclipse-plugin (i'd like to see some of that
> standardized anyway to allow plugin testing generally - which can also be seen as integration testing).
> Thoughts, comments, approaches?
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[jira] Commented: (MNG-591) Integration tests lifecycle
Posted by "Brett Porter (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MNG-591?page=comments#action_42970 ]
Brett Porter commented on MNG-591:
----------------------------------
John's point in MNG-475 is relevant to the above:
" how is providing integration-test phase dependencies any different than providing an <integrationTestSourceDirectory/>?
In other words, if we're leaving the source directory to be specified by cactus, etc. (because they may need different sources, or none), then why can we assume that all integration tests will have the same dependency needs?"
> Integration tests lifecycle
> ---------------------------
>
> Key: MNG-591
> URL: http://jira.codehaus.org/browse/MNG-591
> Project: Maven 2
> Type: Improvement
> Reporter: Kenney Westerhof
> Fix For: 2.0-beta-1
>
>
> I'm trying to do an integration test that depends on a war/ear to be deployed.
> What i'm missing is:
> - integration-test-compile stage and/or:
> - a way to specify an integrationTestSourceDirectory or multiple testSourceDirectories in the pom
> I can't put the test sources in src/test/java because then surefire will run them in the test stage, when
> there's no artifact to deploy yet.
> [Btw, I'm doing this while creating a cactus plugin, for the moment using cargo in the TestSuite itself to deploy.]
> The idea is that the integration test sources go in src/itest/*; that there be a integration-test-compile,
> integration-test-package and/or integration-test-appdeploy[or something] and that surefire
> is also bound to integration-test.
> Maybe something can be done using the src/test/project/some-project/.... approach seen in
> maven-javadoc-plugin, maven-site-plugin and maven-eclipse-plugin (i'd like to see some of that
> standardized anyway to allow plugin testing generally - which can also be seen as integration testing).
> Thoughts, comments, approaches?
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[jira] Updated: (MNG-591) Integration tests lifecycle
Posted by "Brett Porter (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MNG-591?page=all ]
Brett Porter updated MNG-591:
-----------------------------
Assign To: Brett Porter (was: John Casey)
Remaining Estimate: 1 day
Original Estimate: 86400
> Integration tests lifecycle
> ---------------------------
>
> Key: MNG-591
> URL: http://jira.codehaus.org/browse/MNG-591
> Project: Maven 2
> Type: Improvement
> Reporter: Kenney Westerhof
> Assignee: Brett Porter
> Priority: Blocker
> Fix For: 2.0-beta-1
>
> Original Estimate: 1 day
> Remaining: 1 day
>
> I'm trying to do an integration test that depends on a war/ear to be deployed.
> What i'm missing is:
> - integration-test-compile stage and/or:
> - a way to specify an integrationTestSourceDirectory or multiple testSourceDirectories in the pom
> I can't put the test sources in src/test/java because then surefire will run them in the test stage, when
> there's no artifact to deploy yet.
> [Btw, I'm doing this while creating a cactus plugin, for the moment using cargo in the TestSuite itself to deploy.]
> The idea is that the integration test sources go in src/itest/*; that there be a integration-test-compile,
> integration-test-package and/or integration-test-appdeploy[or something] and that surefire
> is also bound to integration-test.
> Maybe something can be done using the src/test/project/some-project/.... approach seen in
> maven-javadoc-plugin, maven-site-plugin and maven-eclipse-plugin (i'd like to see some of that
> standardized anyway to allow plugin testing generally - which can also be seen as integration testing).
> Thoughts, comments, approaches?
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[jira] Commented: (MNG-591) Integration tests lifecycle
Posted by "Brett Porter (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MNG-591?page=comments#action_45005 ]
Brett Porter commented on MNG-591:
----------------------------------
we should have a setup and teardown phase around the actual test goal
> Integration tests lifecycle
> ---------------------------
>
> Key: MNG-591
> URL: http://jira.codehaus.org/browse/MNG-591
> Project: Maven 2
> Type: Improvement
> Reporter: Kenney Westerhof
> Assignee: Brett Porter
> Priority: Blocker
> Fix For: 2.0-beta-1
>
> Original Estimate: 8 hours
> Time Spent: 15 minutes
> Remaining: 7 hours, 45 minutes
>
> I'm trying to do an integration test that depends on a war/ear to be deployed.
> What i'm missing is:
> - integration-test-compile stage and/or:
> - a way to specify an integrationTestSourceDirectory or multiple testSourceDirectories in the pom
> I can't put the test sources in src/test/java because then surefire will run them in the test stage, when
> there's no artifact to deploy yet.
> [Btw, I'm doing this while creating a cactus plugin, for the moment using cargo in the TestSuite itself to deploy.]
> The idea is that the integration test sources go in src/itest/*; that there be a integration-test-compile,
> integration-test-package and/or integration-test-appdeploy[or something] and that surefire
> is also bound to integration-test.
> Maybe something can be done using the src/test/project/some-project/.... approach seen in
> maven-javadoc-plugin, maven-site-plugin and maven-eclipse-plugin (i'd like to see some of that
> standardized anyway to allow plugin testing generally - which can also be seen as integration testing).
> Thoughts, comments, approaches?
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[jira] Commented: (MNG-591) Integration tests lifecycle
Posted by "Kenney Westerhof (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MNG-591?page=comments#action_45936 ]
Kenney Westerhof commented on MNG-591:
--------------------------------------
I just committed sandbox/maven-it-plugin which has the ability to fork maven (sort of)
to run all projects found in src/it/. Use like this:
<plugin>
<artifactId>maven-it-plugin</artifactId>
<executions>
<execution>
<phase>integration-test</phase>
<goals>
<goal>fork</goal>
</goals>
</execution>
</executions>
</plugin>
Brett, can you see if this is a suitable solution for this issue? All setup/teardown/deployment/whatever
can now be done in the integration test projects. Test projects can depend on the parent project
(which could be added at runtime) so that they have access to the generated artifact (and test against it).
> Integration tests lifecycle
> ---------------------------
>
> Key: MNG-591
> URL: http://jira.codehaus.org/browse/MNG-591
> Project: Maven 2
> Type: Improvement
> Reporter: Kenney Westerhof
> Assignee: Brett Porter
> Priority: Blocker
> Fix For: 2.0-beta-2
>
> Original Estimate: 8 hours
> Time Spent: 45 minutes
> Remaining: 7 hours, 15 minutes
>
> I'm trying to do an integration test that depends on a war/ear to be deployed.
> What i'm missing is:
> - integration-test-compile stage and/or:
> - a way to specify an integrationTestSourceDirectory or multiple testSourceDirectories in the pom
> I can't put the test sources in src/test/java because then surefire will run them in the test stage, when
> there's no artifact to deploy yet.
> [Btw, I'm doing this while creating a cactus plugin, for the moment using cargo in the TestSuite itself to deploy.]
> The idea is that the integration test sources go in src/itest/*; that there be a integration-test-compile,
> integration-test-package and/or integration-test-appdeploy[or something] and that surefire
> is also bound to integration-test.
> Maybe something can be done using the src/test/project/some-project/.... approach seen in
> maven-javadoc-plugin, maven-site-plugin and maven-eclipse-plugin (i'd like to see some of that
> standardized anyway to allow plugin testing generally - which can also be seen as integration testing).
> Thoughts, comments, approaches?
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[jira] Updated: (MNG-591) Integration tests lifecycle
Posted by "Brett Porter (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MNG-591?page=all ]
Brett Porter updated MNG-591:
-----------------------------
Fix Version: (was: 2.0-beta-2)
2.1
Kenney's plugin should be sufficient for 2.0
> Integration tests lifecycle
> ---------------------------
>
> Key: MNG-591
> URL: http://jira.codehaus.org/browse/MNG-591
> Project: Maven 2
> Type: Improvement
> Components: maven-core
> Reporter: Kenney Westerhof
> Assignee: Brett Porter
> Priority: Blocker
> Fix For: 2.1
>
> Original Estimate: 8 hours
> Time Spent: 45 minutes
> Remaining: 7 hours, 15 minutes
>
> I'm trying to do an integration test that depends on a war/ear to be deployed.
> What i'm missing is:
> - integration-test-compile stage and/or:
> - a way to specify an integrationTestSourceDirectory or multiple testSourceDirectories in the pom
> I can't put the test sources in src/test/java because then surefire will run them in the test stage, when
> there's no artifact to deploy yet.
> [Btw, I'm doing this while creating a cactus plugin, for the moment using cargo in the TestSuite itself to deploy.]
> The idea is that the integration test sources go in src/itest/*; that there be a integration-test-compile,
> integration-test-package and/or integration-test-appdeploy[or something] and that surefire
> is also bound to integration-test.
> Maybe something can be done using the src/test/project/some-project/.... approach seen in
> maven-javadoc-plugin, maven-site-plugin and maven-eclipse-plugin (i'd like to see some of that
> standardized anyway to allow plugin testing generally - which can also be seen as integration testing).
> Thoughts, comments, approaches?
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[jira] Updated: (MNG-591) Integration tests lifecycle
Posted by "Brett Porter (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MNG-591?page=all ]
Brett Porter updated MNG-591:
-----------------------------
Remaining Estimate: 8 hours (was: 1 day)
Original Estimate: 28800 (was: 86400)
> Integration tests lifecycle
> ---------------------------
>
> Key: MNG-591
> URL: http://jira.codehaus.org/browse/MNG-591
> Project: Maven 2
> Type: Improvement
> Reporter: Kenney Westerhof
> Assignee: Brett Porter
> Priority: Blocker
> Fix For: 2.0-beta-1
>
> Original Estimate: 8 hours
> Remaining: 8 hours
>
> I'm trying to do an integration test that depends on a war/ear to be deployed.
> What i'm missing is:
> - integration-test-compile stage and/or:
> - a way to specify an integrationTestSourceDirectory or multiple testSourceDirectories in the pom
> I can't put the test sources in src/test/java because then surefire will run them in the test stage, when
> there's no artifact to deploy yet.
> [Btw, I'm doing this while creating a cactus plugin, for the moment using cargo in the TestSuite itself to deploy.]
> The idea is that the integration test sources go in src/itest/*; that there be a integration-test-compile,
> integration-test-package and/or integration-test-appdeploy[or something] and that surefire
> is also bound to integration-test.
> Maybe something can be done using the src/test/project/some-project/.... approach seen in
> maven-javadoc-plugin, maven-site-plugin and maven-eclipse-plugin (i'd like to see some of that
> standardized anyway to allow plugin testing generally - which can also be seen as integration testing).
> Thoughts, comments, approaches?
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[jira] Commented: (MNG-591) Integration tests lifecycle
Posted by "Brett Porter (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MNG-591?page=comments#action_44943 ]
Brett Porter commented on MNG-591:
----------------------------------
various things we need:
- plugin integration tests (c.f. eclipse plugin)
- report integration tests (see Vincent S's work - htmlunit)
- cactus plugin
- geronimo itest plugin
- need to be able to compile additional sources, have own dependencies (basically reapply the whole test lifecycle again - given that this also matches the main one, perhaps that can be reusable/forked)
separate project is ideal
+ fits most things by dealing with them in the same way
- a bit unnatural, more strung out
- doesn't fit IDE well
While a separate project might be something we recommend, I think we need to support them within the same project.
I'll put together a solution tomorrow.
> Integration tests lifecycle
> ---------------------------
>
> Key: MNG-591
> URL: http://jira.codehaus.org/browse/MNG-591
> Project: Maven 2
> Type: Improvement
> Reporter: Kenney Westerhof
> Assignee: Brett Porter
> Priority: Blocker
> Fix For: 2.0-beta-1
>
> Original Estimate: 1 day
> Remaining: 1 day
>
> I'm trying to do an integration test that depends on a war/ear to be deployed.
> What i'm missing is:
> - integration-test-compile stage and/or:
> - a way to specify an integrationTestSourceDirectory or multiple testSourceDirectories in the pom
> I can't put the test sources in src/test/java because then surefire will run them in the test stage, when
> there's no artifact to deploy yet.
> [Btw, I'm doing this while creating a cactus plugin, for the moment using cargo in the TestSuite itself to deploy.]
> The idea is that the integration test sources go in src/itest/*; that there be a integration-test-compile,
> integration-test-package and/or integration-test-appdeploy[or something] and that surefire
> is also bound to integration-test.
> Maybe something can be done using the src/test/project/some-project/.... approach seen in
> maven-javadoc-plugin, maven-site-plugin and maven-eclipse-plugin (i'd like to see some of that
> standardized anyway to allow plugin testing generally - which can also be seen as integration testing).
> Thoughts, comments, approaches?
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[jira] Commented: (MNG-591) Integration tests lifecycle
Posted by "Kenney Westerhof (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MNG-591?page=comments#action_42866 ]
Kenney Westerhof commented on MNG-591:
--------------------------------------
Yes, the entire thing could be just a single plugin. I also talked to jdcasey, and we liked the following approach:
src/itest/projects/project-a/ [pom.xml, etc]
src/itest/projects/project-b/ [pom.xml. etc]
Each project would be executed (an 'm2 install' forked somehow) in a new environment containing
the artifact[s] of the project (the one the src/itest dir is a part of).
This way plugins can be easily tested - you still need to specify that you want to use the plugin
in the project-a's pom, of course.
The same goes for 'normal' projects, like those with packaging war/ear. The nested test
project could use normal tests to test the artifact, for instance using cargo in a Suite to deploy
the artifact and then run tests.
The artifact here is from the project being integration-tested. It can be referenced by defining
a dependency in the nested test project, and a different ArtifactFactory will be used in the forked
lifecycle, so that the packaged (but not installed) artifact gets picked up from the target/ dir.
So basically, the integration-test could stay unmodified, as long as it's possible to fork an m2 lifecycle on
a different project (than a clone of the current one). This would mean exposing more of the lifecycle to plugins.
Even better, per packaging a different integration-test plugin/mojo could be used, because plugins / war / ear / ejb / etc
may need different test circumstances. But this could also be specified in the nested test projects.
> Integration tests lifecycle
> ---------------------------
>
> Key: MNG-591
> URL: http://jira.codehaus.org/browse/MNG-591
> Project: Maven 2
> Type: Improvement
> Reporter: Kenney Westerhof
>
>
> I'm trying to do an integration test that depends on a war/ear to be deployed.
> What i'm missing is:
> - integration-test-compile stage and/or:
> - a way to specify an integrationTestSourceDirectory or multiple testSourceDirectories in the pom
> I can't put the test sources in src/test/java because then surefire will run them in the test stage, when
> there's no artifact to deploy yet.
> [Btw, I'm doing this while creating a cactus plugin, for the moment using cargo in the TestSuite itself to deploy.]
> The idea is that the integration test sources go in src/itest/*; that there be a integration-test-compile,
> integration-test-package and/or integration-test-appdeploy[or something] and that surefire
> is also bound to integration-test.
> Maybe something can be done using the src/test/project/some-project/.... approach seen in
> maven-javadoc-plugin, maven-site-plugin and maven-eclipse-plugin (i'd like to see some of that
> standardized anyway to allow plugin testing generally - which can also be seen as integration testing).
> Thoughts, comments, approaches?
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[jira] Commented: (MNG-591) Integration tests lifecycle
Posted by "Brett Porter (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MNG-591?page=comments#action_44939 ]
Brett Porter commented on MNG-591:
----------------------------------
types of tests:
- unit tests - run before package, want 100% success before moving on generally
- integration tests - require the artifact, so run after package but before install. generally want 100% before moving on. May use junit or other plugin like cactus, including deploying to a server
- acceptance/regression/functional tests - not run every time, probably a report and/or part of a profile. Regression tests (acceptance tests from previous milestones) should be at 100%, whereas acceptance (also called functional) tests only need to get to 100% at release time.
- performance/stress tests - another form of acceptance test, really.
> Integration tests lifecycle
> ---------------------------
>
> Key: MNG-591
> URL: http://jira.codehaus.org/browse/MNG-591
> Project: Maven 2
> Type: Improvement
> Reporter: Kenney Westerhof
> Assignee: Brett Porter
> Priority: Blocker
> Fix For: 2.0-beta-1
>
> Original Estimate: 1 day
> Remaining: 1 day
>
> I'm trying to do an integration test that depends on a war/ear to be deployed.
> What i'm missing is:
> - integration-test-compile stage and/or:
> - a way to specify an integrationTestSourceDirectory or multiple testSourceDirectories in the pom
> I can't put the test sources in src/test/java because then surefire will run them in the test stage, when
> there's no artifact to deploy yet.
> [Btw, I'm doing this while creating a cactus plugin, for the moment using cargo in the TestSuite itself to deploy.]
> The idea is that the integration test sources go in src/itest/*; that there be a integration-test-compile,
> integration-test-package and/or integration-test-appdeploy[or something] and that surefire
> is also bound to integration-test.
> Maybe something can be done using the src/test/project/some-project/.... approach seen in
> maven-javadoc-plugin, maven-site-plugin and maven-eclipse-plugin (i'd like to see some of that
> standardized anyway to allow plugin testing generally - which can also be seen as integration testing).
> Thoughts, comments, approaches?
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[jira] Commented: (MNG-591) Integration tests lifecycle
Posted by "Emmanuel Venisse (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MNG-591?page=comments#action_42863 ]
Emmanuel Venisse commented on MNG-591:
--------------------------------------
we chatted with jdcasey for add some new phases.
For integration test source directory, I think it must be defined as a plugin parameter because a project can have multiple type of integration tests (tests for war/ear, test for ui, test for performance...). Each test type are specific to a plugin.
> Integration tests lifecycle
> ---------------------------
>
> Key: MNG-591
> URL: http://jira.codehaus.org/browse/MNG-591
> Project: Maven 2
> Type: Improvement
> Reporter: Kenney Westerhof
>
>
> I'm trying to do an integration test that depends on a war/ear to be deployed.
> What i'm missing is:
> - integration-test-compile stage and/or:
> - a way to specify an integrationTestSourceDirectory or multiple testSourceDirectories in the pom
> I can't put the test sources in src/test/java because then surefire will run them in the test stage, when
> there's no artifact to deploy yet.
> [Btw, I'm doing this while creating a cactus plugin, for the moment using cargo in the TestSuite itself to deploy.]
> The idea is that the integration test sources go in src/itest/*; that there be a integration-test-compile,
> integration-test-package and/or integration-test-appdeploy[or something] and that surefire
> is also bound to integration-test.
> Maybe something can be done using the src/test/project/some-project/.... approach seen in
> maven-javadoc-plugin, maven-site-plugin and maven-eclipse-plugin (i'd like to see some of that
> standardized anyway to allow plugin testing generally - which can also be seen as integration testing).
> Thoughts, comments, approaches?
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[jira] Updated: (MNG-591) Integration tests lifecycle
Posted by "Brett Porter (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MNG-591?page=all ]
Brett Porter updated MNG-591:
-----------------------------
Priority: Blocker (was: Major)
> Integration tests lifecycle
> ---------------------------
>
> Key: MNG-591
> URL: http://jira.codehaus.org/browse/MNG-591
> Project: Maven 2
> Type: Improvement
> Reporter: Kenney Westerhof
> Priority: Blocker
> Fix For: 2.0-beta-1
>
>
> I'm trying to do an integration test that depends on a war/ear to be deployed.
> What i'm missing is:
> - integration-test-compile stage and/or:
> - a way to specify an integrationTestSourceDirectory or multiple testSourceDirectories in the pom
> I can't put the test sources in src/test/java because then surefire will run them in the test stage, when
> there's no artifact to deploy yet.
> [Btw, I'm doing this while creating a cactus plugin, for the moment using cargo in the TestSuite itself to deploy.]
> The idea is that the integration test sources go in src/itest/*; that there be a integration-test-compile,
> integration-test-package and/or integration-test-appdeploy[or something] and that surefire
> is also bound to integration-test.
> Maybe something can be done using the src/test/project/some-project/.... approach seen in
> maven-javadoc-plugin, maven-site-plugin and maven-eclipse-plugin (i'd like to see some of that
> standardized anyway to allow plugin testing generally - which can also be seen as integration testing).
> Thoughts, comments, approaches?
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[jira] Updated: (MNG-591) Integration tests lifecycle
Posted by "Brett Porter (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MNG-591?page=all ]
Brett Porter updated MNG-591:
-----------------------------
Component: maven-core
> Integration tests lifecycle
> ---------------------------
>
> Key: MNG-591
> URL: http://jira.codehaus.org/browse/MNG-591
> Project: Maven 2
> Type: Improvement
> Components: maven-core
> Reporter: Kenney Westerhof
> Assignee: Brett Porter
> Priority: Blocker
> Fix For: 2.0-beta-2
>
> Original Estimate: 8 hours
> Time Spent: 45 minutes
> Remaining: 7 hours, 15 minutes
>
> I'm trying to do an integration test that depends on a war/ear to be deployed.
> What i'm missing is:
> - integration-test-compile stage and/or:
> - a way to specify an integrationTestSourceDirectory or multiple testSourceDirectories in the pom
> I can't put the test sources in src/test/java because then surefire will run them in the test stage, when
> there's no artifact to deploy yet.
> [Btw, I'm doing this while creating a cactus plugin, for the moment using cargo in the TestSuite itself to deploy.]
> The idea is that the integration test sources go in src/itest/*; that there be a integration-test-compile,
> integration-test-package and/or integration-test-appdeploy[or something] and that surefire
> is also bound to integration-test.
> Maybe something can be done using the src/test/project/some-project/.... approach seen in
> maven-javadoc-plugin, maven-site-plugin and maven-eclipse-plugin (i'd like to see some of that
> standardized anyway to allow plugin testing generally - which can also be seen as integration testing).
> Thoughts, comments, approaches?
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[jira] Commented: (MNG-591) Integration tests lifecycle
Posted by "Barry Kaplan (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MNG-591?page=comments#action_49625 ]
Barry Kaplan commented on MNG-591:
----------------------------------
After reading all this, I'm a bit confused. What is the recommended approach in maven2 v1.x for handling different test types?
I pretty much need test types to exist in different directories so I can run them independently in eclipse. I don't want to use packages to segregate the tests, because I need the unit tests to exist the same directory as the CUT. Is seperate projects the only solution for maven2 v1.x? And how does this relate to http://jira.codehaus.org/browse/SUREFIRE-7?
> Integration tests lifecycle
> ---------------------------
>
> Key: MNG-591
> URL: http://jira.codehaus.org/browse/MNG-591
> Project: Maven 2
> Type: Improvement
> Components: maven-core
> Reporter: Kenney Westerhof
> Assignee: Brett Porter
> Priority: Blocker
> Fix For: 2.1
>
> Original Estimate: 8 hours
> Time Spent: 45 minutes
> Remaining: 7 hours, 15 minutes
>
> I'm trying to do an integration test that depends on a war/ear to be deployed.
> What i'm missing is:
> - integration-test-compile stage and/or:
> - a way to specify an integrationTestSourceDirectory or multiple testSourceDirectories in the pom
> I can't put the test sources in src/test/java because then surefire will run them in the test stage, when
> there's no artifact to deploy yet.
> [Btw, I'm doing this while creating a cactus plugin, for the moment using cargo in the TestSuite itself to deploy.]
> The idea is that the integration test sources go in src/itest/*; that there be a integration-test-compile,
> integration-test-package and/or integration-test-appdeploy[or something] and that surefire
> is also bound to integration-test.
> Maybe something can be done using the src/test/project/some-project/.... approach seen in
> maven-javadoc-plugin, maven-site-plugin and maven-eclipse-plugin (i'd like to see some of that
> standardized anyway to allow plugin testing generally - which can also be seen as integration testing).
> Thoughts, comments, approaches?
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
RE: [jira] Created: (MNG-591) Integration tests lifecycle
Posted by Vincent Massol <vm...@pivolis.com>.
Hi Kenney,
> -----Original Message-----
> From: Kenney Westerhof (JIRA) [mailto:jira@codehaus.org]
> Sent: jeudi 14 juillet 2005 22:33
> To: dev@maven.apache.org
> Subject: [jira] Created: (MNG-591) Integration tests lifecycle
[snip]
> [Btw, I'm doing this while creating a cactus plugin, for the moment using
> cargo in the TestSuite itself to deploy.]
Cool :-)
> The idea is that the integration test sources go in src/itest/*;
Don't you mean src/test/itest/*?
I think I'd prefer src/test/cactus actually because there can possibly be
different types of integration tests for the same project and each would
need to be run with its own testing framework (which might not be junit
BTW).
[snip]
> that
> there be a integration-test-compile,
> integration-test-package and/or integration-test-appdeploy[or something]
> and that surefire
> is also bound to integration-test.
I'm not sure about the surefire thing. Again when it comes to integration
testing there can be several test frameworks. I think it should be left to
the different integration test plugins to add themselves to the lifecycle.
Of course the surefire plugin could possibly also be an integration test
plugin and add itself to the integration test phases but it shouldn't be
considered the only one I think.
> Maybe something can be done using the src/test/project/some-project/....
> approach seen in
> maven-javadoc-plugin, maven-site-plugin and maven-eclipse-plugin (i'd like
> to see some of that
> standardized anyway to allow plugin testing generally - which can also be
> seen as integration testing).
For plugin testing, we have had a discussion thread on that on the list. We
agreed that the way forward would be to have a helper class (and possibly a
plugin later on if required) to embed a m2 instance in the test.
-Vincent
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[jira] Commented: (MNG-591) Integration tests lifecycle
Posted by "John Casey (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MNG-591?page=comments#action_44292 ]
John Casey commented on MNG-591:
--------------------------------
the problem with adding integration-test dependency scope and srcdir is that this is a slippery slope. Can we really define all of the types of tests that are out there, and if we do, do we really want <integrationTestSourceDirectory/> and <functionalTestSourceDirectory/> and ...?
I think it's reasonable to expect that integration- or functional- or ... tests will need their own set of dependencies, and that it's unreasonable to expect that these can be satisfied using <scope>test</scope>. The source dir is easier, although we'd still have to provide a utility to help these plugins compile it.
This got shot down once before, but I'll toss it out there again, just to see if it'll help generate some discussion/thinking:
what about allowing dependencies to be tied to a particular lifecycle phase?
While this would help by allowing <phase>integration-test</phase> in the dependency, it could conceivably muddy the waters of scope vs. bindings for dependencies. Calculating inheritedScope is easy on this...if it's not runtime || compile, it's not included transitively.
I think it might not be such a great thing to ship maven 2.0 with the disclaimer "We can't handle all of your testing needs yet." However, I'm not sure what the way forward is. I'd really like to get started on this issue, though, since it looks like the highest-risk issue left for -beta-1.
> Integration tests lifecycle
> ---------------------------
>
> Key: MNG-591
> URL: http://jira.codehaus.org/browse/MNG-591
> Project: Maven 2
> Type: Improvement
> Reporter: Kenney Westerhof
> Assignee: John Casey
> Priority: Blocker
> Fix For: 2.0-beta-1
>
>
> I'm trying to do an integration test that depends on a war/ear to be deployed.
> What i'm missing is:
> - integration-test-compile stage and/or:
> - a way to specify an integrationTestSourceDirectory or multiple testSourceDirectories in the pom
> I can't put the test sources in src/test/java because then surefire will run them in the test stage, when
> there's no artifact to deploy yet.
> [Btw, I'm doing this while creating a cactus plugin, for the moment using cargo in the TestSuite itself to deploy.]
> The idea is that the integration test sources go in src/itest/*; that there be a integration-test-compile,
> integration-test-package and/or integration-test-appdeploy[or something] and that surefire
> is also bound to integration-test.
> Maybe something can be done using the src/test/project/some-project/.... approach seen in
> maven-javadoc-plugin, maven-site-plugin and maven-eclipse-plugin (i'd like to see some of that
> standardized anyway to allow plugin testing generally - which can also be seen as integration testing).
> Thoughts, comments, approaches?
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[jira] Commented: (MNG-591) Integration tests lifecycle
Posted by "Brett Porter (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MNG-591?page=comments#action_42969 ]
Brett Porter commented on MNG-591:
----------------------------------
Currently, we have "test" which exists before package, and has its own source code and resources, but its classes are not processed like a normal build. Then we have integration-test and verify which occur after packaging, but before installation, and don't have any sources of their own.
We are also missing the case where you might have intensive tests that you don't want to run during development. I still think these belong before install, and can be enabled in an integration environment using profiles instead.
Can we think of better ways to specify test sets than this?
What I'm thinking is:
- we have unit-test, functional-test (both before package), and integration-test
- each has an identical lifecycle to main (generate/process-sources/resources, compile, process-classes)
- it is up to the user to use profiles to apply things they want in dev or not
I think the itest plugin IDEA above is good (could we use the verifier plugin?) However, I don't think we should have to rely on it being installed before it is tested. Maybe that's not a big deal, esp. since the reactor should be able to take care of it, however I feel there is some merit to keeping it all together, especially as IDEs might deal with it better that way if they have plugins for the testing framework. Of course it is still possible to have a separate test suite, using dependencies to get the code to test.
I'm still a little uncertain how we might do something like cactus in the same as m1. Perhaps it could fork it's own lifecycle to compile it's special sources, then package them up, deploy, and run tests? It could map flat to the integration-test-* stuff, but then if you have >1 the order is whacky and you also end up having a cactus:compile goal that has to extend AbstractCompilerMojo which is not ideal.
> Integration tests lifecycle
> ---------------------------
>
> Key: MNG-591
> URL: http://jira.codehaus.org/browse/MNG-591
> Project: Maven 2
> Type: Improvement
> Reporter: Kenney Westerhof
> Fix For: 2.0-beta-1
>
>
> I'm trying to do an integration test that depends on a war/ear to be deployed.
> What i'm missing is:
> - integration-test-compile stage and/or:
> - a way to specify an integrationTestSourceDirectory or multiple testSourceDirectories in the pom
> I can't put the test sources in src/test/java because then surefire will run them in the test stage, when
> there's no artifact to deploy yet.
> [Btw, I'm doing this while creating a cactus plugin, for the moment using cargo in the TestSuite itself to deploy.]
> The idea is that the integration test sources go in src/itest/*; that there be a integration-test-compile,
> integration-test-package and/or integration-test-appdeploy[or something] and that surefire
> is also bound to integration-test.
> Maybe something can be done using the src/test/project/some-project/.... approach seen in
> maven-javadoc-plugin, maven-site-plugin and maven-eclipse-plugin (i'd like to see some of that
> standardized anyway to allow plugin testing generally - which can also be seen as integration testing).
> Thoughts, comments, approaches?
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[jira] Updated: (MNG-591) Integration tests lifecycle
Posted by "Brett Porter (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MNG-591?page=all ]
Brett Porter updated MNG-591:
-----------------------------
Fix Version: 2.0-beta-1
> Integration tests lifecycle
> ---------------------------
>
> Key: MNG-591
> URL: http://jira.codehaus.org/browse/MNG-591
> Project: Maven 2
> Type: Improvement
> Reporter: Kenney Westerhof
> Fix For: 2.0-beta-1
>
>
> I'm trying to do an integration test that depends on a war/ear to be deployed.
> What i'm missing is:
> - integration-test-compile stage and/or:
> - a way to specify an integrationTestSourceDirectory or multiple testSourceDirectories in the pom
> I can't put the test sources in src/test/java because then surefire will run them in the test stage, when
> there's no artifact to deploy yet.
> [Btw, I'm doing this while creating a cactus plugin, for the moment using cargo in the TestSuite itself to deploy.]
> The idea is that the integration test sources go in src/itest/*; that there be a integration-test-compile,
> integration-test-package and/or integration-test-appdeploy[or something] and that surefire
> is also bound to integration-test.
> Maybe something can be done using the src/test/project/some-project/.... approach seen in
> maven-javadoc-plugin, maven-site-plugin and maven-eclipse-plugin (i'd like to see some of that
> standardized anyway to allow plugin testing generally - which can also be seen as integration testing).
> Thoughts, comments, approaches?
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org