You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@maven.apache.org by hongbin ma <ma...@apache.org> on 2015/05/25 16:06:12 UTC

what is the difference between failsafe's integration test and verify?

hi, I'm new to failsafe,

i'm starting to move a test case from unit test to integration test, and
i'm using

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-failsafe-plugin</artifactId>
    <executions>
        <execution>
            <id>integration-tests</id>
            <phase>integration-test</phase>
            <goals>
                <goal>integration-test</goal>
                <goal>verify</goal>
            </goals>
        </execution>
    </executions>

​in my pom.

When I run "mvn verify", I ​found my new integration test ran twice.
I realized that both the "integration-test" and "verify" goal may
contribute to it.
Should I remove the line "<goal>verify</goal>" in pom?
But It seems most failsafe tutorials keeps both goals in their poms.

The question is, what are these two goals for?
The official explanation:

*integration-test for running the integration tests.*
*verify for checking the results of the integration tests.*

seems so vague to me.
Can any expert give me a concrete example?

thanks!

-- 
Regards,

*Bin Mahone | 马洪宾*
Apache Kylin: http://kylin.io
Github: https://github.com/binmahone

Re: what is the difference between failsafe's integration test and verify?

Posted by hongbin ma <ma...@apache.org>.
To put the question in another way, what is being "verified" at all?​




-- 
Regards,

*Bin Mahone | 马洪宾*
Apache Kylin: http://kylin.io
Github: https://github.com/binmahone

Re: what is the difference between failsafe's integration test and verify?

Posted by Karl Heinz Marbaise <kh...@gmx.de>.
Hi,

On 5/25/15 4:06 PM, hongbin ma wrote:
> hi, I'm new to failsafe,
>
> i'm starting to move a test case from unit test to integration test, and
> i'm using

So you realized that your "unit test" are in reality integration tests...

>
> <plugin>
>      <groupId>org.apache.maven.plugins</groupId>
>      <artifactId>maven-failsafe-plugin</artifactId>
>      <executions>
>          <execution>
>              <id>integration-tests</id>
>              <phase>integration-test</phase>
>              <goals>
>                  <goal>integration-test</goal>
>                  <goal>verify</goal>
>              </goals>
>          </execution>
>      </executions>
>
> ​in my pom.
>
> When I run "mvn verify", I ​found my new integration test ran twice.
> I realized that both the "integration-test" and "verify" goal may
> contribute to it.
> Should I remove the line "<goal>verify</goal>" in pom?
> But It seems most failsafe tutorials keeps both goals in their poms.

How have you named your integration tests?

Have you correctly followed the naming schema for integration tests in 
relationship with maven-failsafe-plugin


WhatEverIT.java

means *IT.java, *ITCase.java, IT*.java

see into the docs:

https://maven.apache.org/surefire/maven-failsafe-plugin/integration-test-mojo.html#includes

I assume you have configured the maven-failsafe-plugin different and so 
the integration tests will be picked up by maven-surefire-plugin (naming 
schema!) as well as from the maven-failsafe-plugin...




>
> The question is, what are these two goals for?
> The official explanation:
>
> *integration-test for running the integration tests.*
> *verify for checking the results of the integration tests.*

It might as already mentioned that your integration tests needed 
something to setup like an application server or servlet engine to run 
the integration test...For those cases you have the pre-integration-test 
phase where you can setup things...
After running the integration tests (integration-test-phase) you have to 
shutdown the app server or the servlet engine within the 
(post-integration-test) and in the verify phase you have to check if the 
results were successful or having failed integration tests...


>
> seems so vague to me.
> Can any expert give me a concrete example?
>
> thanks!
>

Kind regards
Karl Heinz Marbaise

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org


RE: what is the difference between failsafe's integration test and verify?

Posted by Sander Verhagen <sa...@sanderverhagen.net>.
I think it's important to realize there is four phases involved:

pre-integration-test
integration-test
post-integration-test
verify

During "post-integration-test" you have an opportunity to clean up all the complicated setup that you did in order to run the integration tests, before "verify" halts the build (if there is test failures).

This rigor is only needed for tests with some moving parts. If you need to setup a database, and an application server with your WAR, which are then exercised by your test code (which would still often be in JUnit), just to give an example, this may be for you.

If this is "just" a larger kind of JUnit test, that can still run self-contained, but perhaps happens to violate the "a unit test tests only a single class" paradigm, and that may have IntegrationTest in its name, that may then not be an integration test the way Maven means it.



Sander Verhagen
[  sander@sanderverhagen.net  ]

NOTICE: my e-mail address has changed. You may still e-mail me at Verhagen@Sander.com but you will see me using Sander@SanderVerhagen.net from now on. Feel free to update your address book.


-------- Original message --------
From: Stephen Connolly <st...@gmail.com>
Date:25/05/2015 07:25 (GMT-08:00)
To: Maven Users List <us...@maven.apache.org>
Cc:
Subject: Re: what is the difference between failsafe's integration test and verify?

failsafe:integration-test runs the tests and stores whether the tests
passed or failed in a file

failsafe:verify reads back that file and then if there were failing tests
it fails the build.



On 25 May 2015 at 15:06, hongbin ma <ma...@apache.org> wrote:

> hi, I'm new to failsafe,
>
> i'm starting to move a test case from unit test to integration test, and
> i'm using
>
> <plugin>
>     <groupId>org.apache.maven.plugins</groupId>
>     <artifactId>maven-failsafe-plugin</artifactId>
>     <executions>
>         <execution>
>             <id>integration-tests</id>
>             <phase>integration-test</phase>
>             <goals>
>                 <goal>integration-test</goal>
>                 <goal>verify</goal>
>             </goals>
>         </execution>
>     </executions>
>
> ​in my pom.
>
> When I run "mvn verify", I ​found my new integration test ran twice.
> I realized that both the "integration-test" and "verify" goal may
> contribute to it.
> Should I remove the line "<goal>verify</goal>" in pom?
> But It seems most failsafe tutorials keeps both goals in their poms.
>
> The question is, what are these two goals for?
> The official explanation:
>
> *integration-test for running the integration tests.*
> *verify for checking the results of the integration tests.*
>
> seems so vague to me.
> Can any expert give me a concrete example?
>
> thanks!
>
> --
> Regards,
>
> *Bin Mahone | 马洪宾*
> Apache Kylin: http://kylin.io
> Github: https://github.com/binmahone
>

Re: what is the difference between failsafe's integration test and verify?

Posted by Stephen Connolly <st...@gmail.com>.
failsafe:integration-test runs the tests and stores whether the tests
passed or failed in a file

failsafe:verify reads back that file and then if there were failing tests
it fails the build.



On 25 May 2015 at 15:06, hongbin ma <ma...@apache.org> wrote:

> hi, I'm new to failsafe,
>
> i'm starting to move a test case from unit test to integration test, and
> i'm using
>
> <plugin>
>     <groupId>org.apache.maven.plugins</groupId>
>     <artifactId>maven-failsafe-plugin</artifactId>
>     <executions>
>         <execution>
>             <id>integration-tests</id>
>             <phase>integration-test</phase>
>             <goals>
>                 <goal>integration-test</goal>
>                 <goal>verify</goal>
>             </goals>
>         </execution>
>     </executions>
>
> ​in my pom.
>
> When I run "mvn verify", I ​found my new integration test ran twice.
> I realized that both the "integration-test" and "verify" goal may
> contribute to it.
> Should I remove the line "<goal>verify</goal>" in pom?
> But It seems most failsafe tutorials keeps both goals in their poms.
>
> The question is, what are these two goals for?
> The official explanation:
>
> *integration-test for running the integration tests.*
> *verify for checking the results of the integration tests.*
>
> seems so vague to me.
> Can any expert give me a concrete example?
>
> thanks!
>
> --
> Regards,
>
> *Bin Mahone | 马洪宾*
> Apache Kylin: http://kylin.io
> Github: https://github.com/binmahone
>