You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@maven.apache.org by Kristian Rosenvold <kr...@gmail.com> on 2012/12/02 14:48:33 UTC
Modernizing IT's in core (and elsewhere ?)
Some time ago, I modernized the surefire IT's using a builder on top
of Verifier. I think they are much simpler and more concise than they
used to be.
Typical samples can be seen here;
https://git-wip-us.apache.org/repos/asf?p=maven-surefire.git;a=blob;f=surefire-integration-tests/src/test/java/org/apache/maven/surefire/its/jiras/Surefire907PerThreadWithoutThreadCountIT.java
https://git-wip-us.apache.org/repos/asf?p=maven-surefire.git;a=blob;f=surefire-integration-tests/src/test/java/org/apache/maven/surefire/its/jiras/Surefire628ConsoleOutputBeforeAndAfterClassIT.java
https://git-wip-us.apache.org/repos/asf?p=maven-surefire.git;a=blob;f=surefire-integration-tests/src/test/java/org/apache/maven/surefire/its/jiras/Surefire809GroupExpressionsIT.java
Now I just worked on an IT for MNG-5208 and I must admit I felt the
step "back" to old-school verifier tests feels painful.
The test can be seen at
https://git-wip-us.apache.org/repos/asf?p=maven-integration-testing.git;a=blob;f=core-it-suite/src/test/java/org/apache/maven/it/MavenITmng5208EventSpyParallelTest.java
I would like to do "something" about the core it's, maybe define a
simpler standard for newer tests. I'm really just seeking input on
this ;)
I think there are a number of issues even with my revised tests. As
can be seen the signal to noise ratio becomes worse than ever;
while the test can be precisely expressed as a one-liner, we
consistently add 35 lines of java boiler-plate including license
header.
The "builder" I made for surefire could be generalized and moved into
verifier; I'm really interested in both feedback and other
pointers to "nice" ways this has been solved.. Maybe I'm missing
something big ;)
Kristian
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: Modernizing IT's in core (and elsewhere ?)
Posted by Jason van Zyl <ja...@tesla.io>.
I think trying anything new that's easier is good. The only thing we need to be cognizant of is making sure the body of ITs work to test all the versions of Maven.
But I think project self-containment is a good goal. I just liked the setup where there was a project and a file that described the output so you could just run it with doing anything else other than setting up the project and declaratively stating what the results should be.
On Dec 2, 2012, at 9:47 AM, Kristian Rosenvold <kr...@gmail.com> wrote:
> 2012/12/2 Jason van Zyl <ja...@tesla.io>:
>> I think the way the ITs used to work was better. Where a single project was entirely self-contained and there was a declarative file that described the desired outputs.
>
> Well I think this is one of the design goals I *almost* managed to
> achieve when I refactored the surefire IT's; the supplied test project
> can run with "mvn clean install" straight out of the box to
> demonstrate the problem (as described here
> http://maven.apache.org/surefire/maven-surefire-plugin/developing.html).
> (This is where I think invoker fails miserably). In surefire I did
> this by using the system properties for the plugin parameters as well
> as defining standard "properties" in the poms for all the props
> without predefined -D values. Writing the actual IT to do the
> assertion is a walk in the park, and there's a quite high percentage
> of surefire patches that actually contain well-functioning IT's. (This
> may be related to personal beliefs of anyone
> submitting patches to a test framework ;)
>
> The core IT's are probably the most complex ITs in all of maven. They
> are still not *that* much more complex than the
> surefire IT's used to be, where the simplification seems to have paid
> off well. The effort of rewriting may not be worth it (although it was
> in surefire), but establishing a clear norm for "nice" tests that can
> be documented is always a good thing.
>
> I'm no big fan of BDD frameworks like jbehave or cucumber; I think
> simple "java" based DSL's do the job better; especially with what is
> essentially a very limited domain like this. I like runnable junit
> tests that apply to "real" runnable projects that can be made by end
> users. I'm a programmer dammit ;)
>
> Kristian
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
Thanks,
Jason
----------------------------------------------------------
Jason van Zyl
Founder & CTO, Sonatype
Founder, Apache Maven
http://twitter.com/jvanzyl
---------------------------------------------------------
First, the taking in of scattered particulars under one Idea,
so that everyone understands what is being talked about ... Second,
the separation of the Idea into parts, by dividing it at the joints,
as nature directs, not breaking any limb in half as a bad carver might.
-- Plato, Phaedrus (Notes on the Synthesis of Form by C. Alexander)
Re: Modernizing IT's in core (and elsewhere ?)
Posted by Kristian Rosenvold <kr...@gmail.com>.
2012/12/2 Jason van Zyl <ja...@tesla.io>:
> I think the way the ITs used to work was better. Where a single project was entirely self-contained and there was a declarative file that described the desired outputs.
Well I think this is one of the design goals I *almost* managed to
achieve when I refactored the surefire IT's; the supplied test project
can run with "mvn clean install" straight out of the box to
demonstrate the problem (as described here
http://maven.apache.org/surefire/maven-surefire-plugin/developing.html).
(This is where I think invoker fails miserably). In surefire I did
this by using the system properties for the plugin parameters as well
as defining standard "properties" in the poms for all the props
without predefined -D values. Writing the actual IT to do the
assertion is a walk in the park, and there's a quite high percentage
of surefire patches that actually contain well-functioning IT's. (This
may be related to personal beliefs of anyone
submitting patches to a test framework ;)
The core IT's are probably the most complex ITs in all of maven. They
are still not *that* much more complex than the
surefire IT's used to be, where the simplification seems to have paid
off well. The effort of rewriting may not be worth it (although it was
in surefire), but establishing a clear norm for "nice" tests that can
be documented is always a good thing.
I'm no big fan of BDD frameworks like jbehave or cucumber; I think
simple "java" based DSL's do the job better; especially with what is
essentially a very limited domain like this. I like runnable junit
tests that apply to "real" runnable projects that can be made by end
users. I'm a programmer dammit ;)
Kristian
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: Modernizing IT's in core (and elsewhere ?)
Posted by Jason van Zyl <ja...@tesla.io>.
On Dec 2, 2012, at 5:48 AM, Kristian Rosenvold <kr...@gmail.com> wrote:
> Some time ago, I modernized the surefire IT's using a builder on top
> of Verifier. I think they are much simpler and more concise than they
> used to be.
>
> Typical samples can be seen here;
>
> https://git-wip-us.apache.org/repos/asf?p=maven-surefire.git;a=blob;f=surefire-integration-tests/src/test/java/org/apache/maven/surefire/its/jiras/Surefire907PerThreadWithoutThreadCountIT.java
> https://git-wip-us.apache.org/repos/asf?p=maven-surefire.git;a=blob;f=surefire-integration-tests/src/test/java/org/apache/maven/surefire/its/jiras/Surefire628ConsoleOutputBeforeAndAfterClassIT.java
> https://git-wip-us.apache.org/repos/asf?p=maven-surefire.git;a=blob;f=surefire-integration-tests/src/test/java/org/apache/maven/surefire/its/jiras/Surefire809GroupExpressionsIT.java
>
> Now I just worked on an IT for MNG-5208 and I must admit I felt the
> step "back" to old-school verifier tests feels painful.
>
> The test can be seen at
>
> https://git-wip-us.apache.org/repos/asf?p=maven-integration-testing.git;a=blob;f=core-it-suite/src/test/java/org/apache/maven/it/MavenITmng5208EventSpyParallelTest.java
>
> I would like to do "something" about the core it's, maybe define a
> simpler standard for newer tests. I'm really just seeking input on
> this ;)
>
> I think there are a number of issues even with my revised tests. As
> can be seen the signal to noise ratio becomes worse than ever;
> while the test can be precisely expressed as a one-liner, we
> consistently add 35 lines of java boiler-plate including license
> header.
>
> The "builder" I made for surefire could be generalized and moved into
> verifier; I'm really interested in both feedback and other
> pointers to "nice" ways this has been solved.. Maybe I'm missing
> something big ;)
I think the way the ITs used to work was better. Where a single project was entirely self-contained and there was a declarative file that described the desired outputs. They couldn't not be run in an IDE but I think that could be fixed. The problem with the current setup is that they are impenetrable for anyone who might want to contribute. I can count on one hand the number of ITs contributed by outsiders. Imagine if a single unit test was a completely self contained project and we used something like JBehave so that someone could describe the desired output given the POM for the test project?
I still think that we need to be able to debug and run from the IDE but we can probably make a custom test running.
I would not change any of the existing tests, but if you wanted to use your builder and it makes the test more succinct go for it.
>
> Kristian
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
Thanks,
Jason
----------------------------------------------------------
Jason van Zyl
Founder & CTO, Sonatype
Founder, Apache Maven
http://twitter.com/jvanzyl
---------------------------------------------------------
We all have problems. How we deal with them is a measure of our worth.
-- Unknown