You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@maven.apache.org by Nigel Jones <ni...@gmail.com> on 2020/07/02 16:37:42 UTC
maven failsafe plugin & POJO tests
I have some 'integration' tests that have been written as POJOs. They run manually
I'm looking at refactoring to automate, making use of the failsafe plugin, together with a few other plugins to launch a server process, and configure the server against which the integration tests run.
I'm struggling to get failsafe to call any tests...
We have a 300+ multi module maven project. A significant amount of those projects contribute to a spring boot application - this is all built/assembled through maven.
When the pojo tests were written, they were added as a regular maven module. As such the source is in the usual source folders, not test ... so I have added configuration like
<testSourceDirectory>${project.build.sourceDirectory}</testSourceDirectory>
<testClassesDirectory>${project.build.outputDirectory}</testClassesDirectory>
The classes didn't follow the failsafe naming convention, so I added:
<includes>
<include>**/*.java</include>
<include>**/*.java</include>
</includes>
This isn't where I finally want to get to (refactoring to naming conventions will be clearer for the future) but as a demonstration/first pass, I thought this would be a sensible evolutionary approach.
The tests themselves are not written using any framework like junit/testNG, and require input/parms, but in a class which begins:
package org.odpi.openmetadata.accessservices.subjectarea.fvt;
// imports
public class GlossaryFVT {
I could add a method like:
public static void testGlossary()
{
try {
runIt("https://localhost:10443", "fvtserver", "garygeeke");
} catch (Exception e) {
// Test method - so ok to do this
Thread.dumpStack();
}
}
(here runIt is an existing method, which is also called via a main() entry point - but that's not of interest in migrating to failsafe)
Not only is the method not reported by failsafe, but the class isn't either. For the method, I also tried prepending/appending with 'test' , 'Test', 'IT' but couldn't get any combination working. The code on github (for surefire) hinted at Test, but the docs refer to IT
In the actual source directory I have a number of classes:
CategoryFVT.java GlossaryFVT.java RunAllFVTNow.java
CategoryHierarchyFVT.java GraphFVT.java SubjectAreaDefinitionCategoryFVT.java
EffectiveDatesFVT.java ProjectFVT.java SubjectAreaFVTCheckedException.java
FVTConstants.java RelationshipsFVT.java TermFVT.java
FVTUtils.java RunAllFVT.java
and when I run I actually get some of these classes picked up by surefire for example:
[INFO] --- maven-failsafe-plugin:3.0.0-M5:integration-test (local) @ subject-area-fvt ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.odpi.openmetadata.accessservices.subjectarea.fvt.FVTConstants
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.004 s - in org.odpi.openmetadata.accessservices.subjectarea.fvt.FVTConstants
[INFO] Running org.odpi.openmetadata.accessservices.subjectarea.fvt.RunAllFVT
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s - in org.odpi.openmetadata.accessservices.subjectarea.fvt.RunAllFVT
[INFO] Running org.odpi.openmetadata.accessservices.subjectarea.fvt.RunAllFVTNow
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 s - in org.odpi.openmetadata.accessservices.subjectarea.fvt.RunAllFVTNow
[INFO] Running org.odpi.openmetadata.accessservices.subjectarea.fvt.FVTUtils
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 s - in org.odpi.openmetadata.accessservices.subjectarea.fvt.FVTUtils
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
Interesting ... in fact I changed the includes to '**/*.FVT' as these represent the classes I actually will modify for testing (as opposed to utils/runners) - but then I had NONE of the classes listed above. Hence I went back to * for purpose of explanation .
If we look at one of those picked up -
$ cat src/main/java/org/odpi/openmetadata/accessservices/subjectarea/fvt/FVTConstants.java
/* SPDX-License-Identifier: Apache-2.0 */
/* Copyright Contributors to the ODPi Egeria project. */
package org.odpi.openmetadata.accessservices.subjectarea.fvt;
public class FVTConstants
{
public static final String USERID = "erinoverview";
public static final String DEFAULT_URL = "https://localhost:9443";
public static final String SERVER_NAME1 = "cocoMDS4";
public static final String SERVER_NAME2 = "cocoMDS1";
}
There's nothing of interest there..
So, I am confused over the status of POJO testing, how to specify classes to test, and which methods will be called.
Can anyone explain?
Are pojo tests supported?
What methods are counted as tests?
Can I override the method names used
Why aren't my other classes even picked up when figuring out classes to use? let alone methods?
I should add I'm using the latest 3.0.0-M5 plugin, though also tried backing off to 2.x - same results. Currently java 14 locally, though java 8 is the base build env. MacOS
I can do more refactoring - ie move to test directories, refactor code, follow method naming conventions (once known), and if needed just do more refactoring and move to JUnit etc, but was confused why my first step wasn't working?
Many thanks
Nigel Jones
ODPi Egeria maintainer
https://github.com/odpi/egeria
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: maven failsafe plugin & POJO tests
Posted by Nigel Jones <ni...@gmail.com>.
Thanks for all the replies
I did try the test project from @tibordigana which worked well. I was able to change names etc and still have the tests working, but oddly could still not figure out why my main project was failing to find all the correct classes & methods.
I ended up abandoning trying to get the pojos working & just created some JUnit5 tests to wrap the existing tests. Not perfect as it's a small handful of large tests and should be decomposed, but that is independent of the framework. The key thing was with JUnit5 there was no issue locating and running the tests - and in any case it's a better model for future tests.
Regards
Nigel.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: maven failsafe plugin & POJO tests
Posted by Nigel Jones <ni...@gmail.com>.
On 2020/07/02 17:10:08, Bernd Eckenfels <ec...@zusammenkunft.net> wrote:
> The failsafe IntegrationTestMojo essentially is the surefire plugin (but with a different error lifecycle handled by the verifyMojo) and a different default include filter.
>
> https://github.com/apache/maven-surefire/blob/master/maven-failsafe-plugin/src/main/java/org/apache/maven/plugin/failsafe/IntegrationTestMojo.java#L223
Thanks -- I did look at that file earlier - and saw nothing there that might change the name used for Pojo tests which seems to remain as including 'Test'
> I would have expected it only supports Junit4/5 and TestNG conventions (I.e. @Test annotations)
I'm thinking that the pojo support isn't something most people use, and I only really looked at it to avoid too much refactoring of code to prove a concept.
> BTW one option would be to deliver a single regular JUnit annotated Test which dynamiclly introspective or calls the spring test wrapper - I don't know much about spring testing but I would asume such a thing already exists.
Thanks - though am not so concerned with spring - I perhaps caused some confusion as this is how the server is built. We do have other tests (ie unit tests) but with failsafe I'm looking at integration tests that depend on a remote server (accessed either via REST or a simple client). So spring isn't really a factor for these particular tests
Thanks for the reply. My current thought is to step away from pojo and go down the JUnit5 route as it seems a more natural, and commonly used fit.
That being said the surefire docs do reference POJO tests at https://maven.apache.org/surefire/maven-failsafe-plugin/usage.html hence wondering if there were any useful tips to get that working :-)
Thanks
nigel
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: maven failsafe plugin & POJO tests
Posted by Bernd Eckenfels <ec...@zusammenkunft.net>.
Hello,
The failsafe IntegrationTestMojo essentially is the surefire plugin (but with a different error lifecycle handled by the verifyMojo) and a different default include filter.
https://github.com/apache/maven-surefire/blob/master/maven-failsafe-plugin/src/main/java/org/apache/maven/plugin/failsafe/IntegrationTestMojo.java#L223
I would have expected it only supports Junit4/5 and TestNG conventions (I.e. @Test annotations)
BTW one option would be to deliver a single regular JUnit annotated Test which dynamiclly introspective or calls the spring test wrapper - I don't know much about spring testing but I would asume such a thing already exists.
Gruss
Bernd
--
http://bernd.eckenfels.net
________________________________
Von: Nigel Jones <ni...@gmail.com>
Gesendet: Thursday, July 2, 2020 6:37:42 PM
An: users@maven.apache.org <us...@maven.apache.org>
Betreff: maven failsafe plugin & POJO tests
I have some 'integration' tests that have been written as POJOs. They run manually
I'm looking at refactoring to automate, making use of the failsafe plugin, together with a few other plugins to launch a server process, and configure the server against which the integration tests run.
I'm struggling to get failsafe to call any tests...
We have a 300+ multi module maven project. A significant amount of those projects contribute to a spring boot application - this is all built/assembled through maven.
When the pojo tests were written, they were added as a regular maven module. As such the source is in the usual source folders, not test ... so I have added configuration like
<testSourceDirectory>${project.build.sourceDirectory}</testSourceDirectory>
<testClassesDirectory>${project.build.outputDirectory}</testClassesDirectory>
The classes didn't follow the failsafe naming convention, so I added:
<includes>
<include>**/*.java</include>
<include>**/*.java</include>
</includes>
This isn't where I finally want to get to (refactoring to naming conventions will be clearer for the future) but as a demonstration/first pass, I thought this would be a sensible evolutionary approach.
The tests themselves are not written using any framework like junit/testNG, and require input/parms, but in a class which begins:
package org.odpi.openmetadata.accessservices.subjectarea.fvt;
// imports
public class GlossaryFVT {
I could add a method like:
public static void testGlossary()
{
try {
runIt("https://localhost:10443", "fvtserver", "garygeeke");
} catch (Exception e) {
// Test method - so ok to do this
Thread.dumpStack();
}
}
(here runIt is an existing method, which is also called via a main() entry point - but that's not of interest in migrating to failsafe)
Not only is the method not reported by failsafe, but the class isn't either. For the method, I also tried prepending/appending with 'test' , 'Test', 'IT' but couldn't get any combination working. The code on github (for surefire) hinted at Test, but the docs refer to IT
In the actual source directory I have a number of classes:
CategoryFVT.java GlossaryFVT.java RunAllFVTNow.java
CategoryHierarchyFVT.java GraphFVT.java SubjectAreaDefinitionCategoryFVT.java
EffectiveDatesFVT.java ProjectFVT.java SubjectAreaFVTCheckedException.java
FVTConstants.java RelationshipsFVT.java TermFVT.java
FVTUtils.java RunAllFVT.java
and when I run I actually get some of these classes picked up by surefire for example:
[INFO] --- maven-failsafe-plugin:3.0.0-M5:integration-test (local) @ subject-area-fvt ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.odpi.openmetadata.accessservices.subjectarea.fvt.FVTConstants
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.004 s - in org.odpi.openmetadata.accessservices.subjectarea.fvt.FVTConstants
[INFO] Running org.odpi.openmetadata.accessservices.subjectarea.fvt.RunAllFVT
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s - in org.odpi.openmetadata.accessservices.subjectarea.fvt.RunAllFVT
[INFO] Running org.odpi.openmetadata.accessservices.subjectarea.fvt.RunAllFVTNow
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 s - in org.odpi.openmetadata.accessservices.subjectarea.fvt.RunAllFVTNow
[INFO] Running org.odpi.openmetadata.accessservices.subjectarea.fvt.FVTUtils
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 s - in org.odpi.openmetadata.accessservices.subjectarea.fvt.FVTUtils
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
Interesting ... in fact I changed the includes to '**/*.FVT' as these represent the classes I actually will modify for testing (as opposed to utils/runners) - but then I had NONE of the classes listed above. Hence I went back to * for purpose of explanation .
If we look at one of those picked up -
$ cat src/main/java/org/odpi/openmetadata/accessservices/subjectarea/fvt/FVTConstants.java
/* SPDX-License-Identifier: Apache-2.0 */
/* Copyright Contributors to the ODPi Egeria project. */
package org.odpi.openmetadata.accessservices.subjectarea.fvt;
public class FVTConstants
{
public static final String USERID = "erinoverview";
public static final String DEFAULT_URL = "https://localhost:9443";
public static final String SERVER_NAME1 = "cocoMDS4";
public static final String SERVER_NAME2 = "cocoMDS1";
}
There's nothing of interest there..
So, I am confused over the status of POJO testing, how to specify classes to test, and which methods will be called.
Can anyone explain?
Are pojo tests supported?
What methods are counted as tests?
Can I override the method names used
Why aren't my other classes even picked up when figuring out classes to use? let alone methods?
I should add I'm using the latest 3.0.0-M5 plugin, though also tried backing off to 2.x - same results. Currently java 14 locally, though java 8 is the base build env. MacOS
I can do more refactoring - ie move to test directories, refactor code, follow method naming conventions (once known), and if needed just do more refactoring and move to JUnit etc, but was confused why my first step wasn't working?
Many thanks
Nigel Jones
ODPi Egeria maintainer
https://github.com/odpi/egeria
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: maven failsafe plugin & POJO tests
Posted by Nigel Jones <ni...@gmail.com>.
Just to close out on this issue -- we have tests working now. The intent was always to refactor into Junit tests in any case -- and these are getting fired correctly.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: maven failsafe plugin & POJO tests
Posted by Tibor Digana <ti...@apache.org>.
Hi Nigel,
I wrote a project for you
https://github.com/Tibor17/pojo-testing
It is related to the JUnit POJO testing with Maven Failsafe Plugin.
I guess you did not use the expected postfix. That's why there were 0 tests
to run.
Additionally, README file shows the command line example with filtering the
test.
btw, The postfix can be customized by the configuration.
Cheers
Tibor
On Thu, Jul 2, 2020 at 9:44 PM Nigel Jones <ni...@gmail.com> wrote:
>
>
> On 2020/07/02 19:08:26, Nigel Jones <ni...@gmail.com> wrote:
>
> > What is odd though is that the class 'GlossaryFVT' itself isn't showing
> up in the Tests run line -- even with the includes in place. Yet other
> files like 'FVTContants' are. This I don't understand - before I get to the
> methods (though once found, the static methods observation is very
> valuable. )
>
> From a quick test, it does seem that if I convert to a JUnit5 test I don't
> have an issue in getting tests run - seems specific to the pojo tests. I
> may work on that tomorrow.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>
Re: maven failsafe plugin & POJO tests
Posted by Nigel Jones <ni...@gmail.com>.
On 2020/07/02 19:08:26, Nigel Jones <ni...@gmail.com> wrote:
> What is odd though is that the class 'GlossaryFVT' itself isn't showing up in the Tests run line -- even with the includes in place. Yet other files like 'FVTContants' are. This I don't understand - before I get to the methods (though once found, the static methods observation is very valuable. )
From a quick test, it does seem that if I convert to a JUnit5 test I don't have an issue in getting tests run - seems specific to the pojo tests. I may work on that tomorrow.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: maven failsafe plugin & POJO tests
Posted by Nigel Jones <ni...@gmail.com>.
On 2020/07/02 18:23:13, Piotr Żygieło <pi...@gmail.com> wrote:
> In plugin I only define executions (i.e. have no redefined includes),
> and test classes follow convention *IT (like SimpleIT).
Thanks
> When I change my method to static I get similar result to your log:
>
> > [INFO] Running org.odpi.openmetadata.accessservices.subjectarea.fvt.FVTConstants
> > [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.004 s - in org.odpi.openmetadata.accessservices.subjectarea.fvt.FVTConstants
>
> - class is considered by failsafe but no test is discovered.
What is odd though is that the class 'GlossaryFVT' itself isn't showing up in the Tests run line -- even with the includes in place. Yet other files like 'FVTContants' are. This I don't understand - before I get to the methods (though once found, the static methods observation is very valuable. )
I can also rename that method as GlossaryIT - I did this (after the point below) and the class isn't listed in the output - even with 0 tests (which is what I'd expect if just the method name was wrong)
> Did you try to make test method in GlossaryFVT not static and run
> maven `verify -Dit.test=GlossaryFVT`? It shall not need/ignore
> includes configured.
Yes I did, but the run just includes
[INFO] Results:
[INFO]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
So no mention of that class at all
> Note, that per http://maven.apache.org/surefire/maven-surefire-plugin/examples/pojo-test.html
> - test method has to be named test* and it works fine.
Thanks, I did try test*, *test, IT*, and *IT ... since I know the default class names were changed from surefire -> failsafe.. wasn't sure about the methods
However thanks -- I'll look through that example as I'd missed it..|
I also thought to check 'mvn help:effective-pom' but, as expected, I'm not picking up any other failsafe related configuration.
> Piotrek
Thanks for trying :-)
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: maven failsafe plugin & POJO tests
Posted by Piotr Żygieło <pi...@gmail.com>.
> > > public static void testGlossary()
> > > {
> >
> > POJO ITs work in my set up, but my test methods are not static.
>
> Thanks. Can I check what naming convention you've used for your test methods -- and/or configuration to override this? How about your classes?
In plugin I only define executions (i.e. have no redefined includes),
and test classes follow convention *IT (like SimpleIT).
When I change my method to static I get similar result to your log:
> [INFO] Running org.odpi.openmetadata.accessservices.subjectarea.fvt.FVTConstants
> [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.004 s - in org.odpi.openmetadata.accessservices.subjectarea.fvt.FVTConstants
- class is considered by failsafe but no test is discovered.
Did you try to make test method in GlossaryFVT not static and run
maven `verify -Dit.test=GlossaryFVT`? It shall not need/ignore
includes configured.
> For the method, I also tried prepending/appending with 'test' , 'Test', 'IT' but couldn't get any combination working.
> [...]
> naming convention you've used for your test methods
Note, that per http://maven.apache.org/surefire/maven-surefire-plugin/examples/pojo-test.html
- test method has to be named test* and it works fine.
--
Piotrek
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: maven failsafe plugin & POJO tests
Posted by Nigel Jones <ni...@gmail.com>.
On 2020/07/02 17:39:25, Piotr Żygieło <pi...@gmail.com> wrote:
> >
> > public static void testGlossary()
> > {
>
> POJO ITs work in my set up, but my test methods are not static.
Thanks. Can I check what naming convention you've used for your test methods -- and/or configuration to override this? How about your classes?
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: maven failsafe plugin & POJO tests
Posted by Piotr Żygieło <pi...@gmail.com>.
>
> public static void testGlossary()
> {
POJO ITs work in my set up, but my test methods are not static.
--
Piotrek
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org