You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@maven.apache.org by nicolas de loof <ni...@apache.org> on 2008/11/16 15:08:50 UTC
RFC : integration tests
Hi,
First of all, please don't blame me for my horrible english. I'm french, and
french guys are known for beeing poor foreign language speackers...
maven default lifecycle defines integration-test phase, but AFAIK there is
no consensus (yet) on how to use it. Some wiki doc help to setup a
maven/surefire hack to split test into unit / integration based on naming
conventions, but nothing better.
I'd like to suggest some convention and maybe plugins enhancement to support
integration tests in a common way.
*What's an IT test ?
*Just to keep things clear, I thing "intergation-test" is a far too large
word. Some of us consider tests that require a container (typically, war
deployed and selenium test), some other consider tests that require host or
network resources (DataBase-related tests) and others simply tests that
require complex setup and not just "unit" scope, like GWTTestCase or
Spring-related ones that start an ApplicationContext.
Based on this, I don't thing we can define a 100% consensus on how to define
and run IT tests with maven. I'll focus on the simpliest use case : tests
written in Java that you don't want / can't run before packaging. I suppose
the pre-integration test phase is enough to setup all required
infrastructure to get IT test running.
*What's wrong with the existing solutions ?
*Even it works in many cases, I don't like the Test naming convention hack
to use a shared target/test-classes for all tests
- there is no way to distinguish unit and IT resources
- there is no nice distinction between unit and IT tests.This doesn't
help newbee developpers to understand the role of such code
- hey, this really looks like an ugly maven hack ! We should support IT
tests with no configuration as we support compilation and unit tests with a
minimal POM.
*What is missing in maven ?*
- there is no convention for an IT-tests source directory.
- there is no maven-compiler-plugin Mojo to handle such java sources,
with a dedicated output folder
- there is no maven-resources-plugin Mojo to handle resources dedicated
to IT tests
- there is no maven-surefire-plugin Mojo to run them
*What can we do ?*
Define a dedicated directory for IT tests. As a new XML element in POM would
require to change the ModelVersion with compatibility issues, I'd suggest to
define a standard property project.build.integrationTestSourceDirectory (the
same way we did for encoding), with default value to
src/integration-test/java. Same for IT resources
Either define new Mojos in compiler, resources and surefire plugins (that
would require 3 new releases), or create a new
maven-integration-tests-plugin to share code from those 3 plugins and handle
the whole process :
compile integrationTestSourceDirectory to target/integration-tests-classes
copy IT resources to target/integration-tests-classes
run tests found in target/integration-tests-classes, with target/classes and
target/test-classes in classpath.
Any suggestion, feedback or link to an existing solution is welcome
Cheers,
Nicolas
Re: RFC : integration tests
Posted by Rémy Sanlaville <re...@gmail.com>.
Hi Nicolas,
I agree that integration-test is not enough support with maven.
I thought that src/it/ and target/it was the original convention (even if it
is not really used because of the lack of integration with maven 2.0.x).
It would be really nice to have a better and standard support for
integration-tests and a document that describes how to manage
integration-tests with maven 2.
Rémy
Re: RFC : integration tests
Posted by Brett Porter <br...@apache.org>.
I agree, for a future version of Maven. What I think is needed is
reusable test lifecycles so you get the source directory, compilation
and resources sequence, and then the run.
You might also like to look at the slides from my talk at ApacheCon
that cover the two alternatives that are available today: http://www.us.apachecon.com/c/acus2008/sessions/50
Cheers,
Brett
On 17/11/2008, at 1:08 AM, nicolas de loof wrote:
> Hi,
>
> First of all, please don't blame me for my horrible english. I'm
> french, and
> french guys are known for beeing poor foreign language speackers...
>
> maven default lifecycle defines integration-test phase, but AFAIK
> there is
> no consensus (yet) on how to use it. Some wiki doc help to setup a
> maven/surefire hack to split test into unit / integration based on
> naming
> conventions, but nothing better.
>
> I'd like to suggest some convention and maybe plugins enhancement to
> support
> integration tests in a common way.
>
> *What's an IT test ?
> *Just to keep things clear, I thing "intergation-test" is a far too
> large
> word. Some of us consider tests that require a container (typically,
> war
> deployed and selenium test), some other consider tests that require
> host or
> network resources (DataBase-related tests) and others simply tests
> that
> require complex setup and not just "unit" scope, like GWTTestCase or
> Spring-related ones that start an ApplicationContext.
>
> Based on this, I don't thing we can define a 100% consensus on how
> to define
> and run IT tests with maven. I'll focus on the simpliest use case :
> tests
> written in Java that you don't want / can't run before packaging. I
> suppose
> the pre-integration test phase is enough to setup all required
> infrastructure to get IT test running.
>
> *What's wrong with the existing solutions ?
> *Even it works in many cases, I don't like the Test naming
> convention hack
> to use a shared target/test-classes for all tests
>
> - there is no way to distinguish unit and IT resources
> - there is no nice distinction between unit and IT tests.This
> doesn't
> help newbee developpers to understand the role of such code
> - hey, this really looks like an ugly maven hack ! We should
> support IT
> tests with no configuration as we support compilation and unit
> tests with a
> minimal POM.
>
>
> *What is missing in maven ?*
>
> - there is no convention for an IT-tests source directory.
> - there is no maven-compiler-plugin Mojo to handle such java
> sources,
> with a dedicated output folder
> - there is no maven-resources-plugin Mojo to handle resources
> dedicated
> to IT tests
> - there is no maven-surefire-plugin Mojo to run them
>
> *What can we do ?*
>
> Define a dedicated directory for IT tests. As a new XML element in
> POM would
> require to change the ModelVersion with compatibility issues, I'd
> suggest to
> define a standard property
> project.build.integrationTestSourceDirectory (the
> same way we did for encoding), with default value to
> src/integration-test/java. Same for IT resources
>
> Either define new Mojos in compiler, resources and surefire plugins
> (that
> would require 3 new releases), or create a new
> maven-integration-tests-plugin to share code from those 3 plugins
> and handle
> the whole process :
> compile integrationTestSourceDirectory to target/integration-tests-
> classes
> copy IT resources to target/integration-tests-classes
> run tests found in target/integration-tests-classes, with target/
> classes and
> target/test-classes in classpath.
>
>
> Any suggestion, feedback or link to an existing solution is welcome
>
> Cheers,
> Nicolas
--
Brett Porter
brett@apache.org
http://blogs.exist.com/bporter/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: RFC : integration tests
Posted by Benjamin Bentmann <be...@udo.edu>.
nicolas de loof wrote:
> maven default lifecycle defines integration-test phase, but AFAIK there is
> no consensus (yet) on how to use it.
Just to mention a related Wiki article where people have previously
collected some ideas:
http://docs.codehaus.org/display/MAVEN/best+practices+-+testing+strategies
> I'd like to suggest some convention and maybe plugins enhancement to support
> integration tests in a common way.
+1, we only need someone to implement all this ;-)
Benjamin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org