You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@maven.apache.org by Jesse McConnell <je...@gmail.com> on 2006/02/17 20:39:46 UTC
plugin testing
brett asked me to look into the idea of a plugin testing framework and
having mulled it over a bit and talked to some folks about it I wanted to
spill out my thoughts here and a couple of stabs at breaking the nut
cleanly. Also, in the interests of having people read this and not have it
drag on I'll jump right to the chase.
We need someway to be able to gather code coverage metrics from plugin
testing so we have at least a modicum of belief that changes didn't break
anything, and since plugins can be so amazingly complex in many cases and
just downright trivial in others there can't be a one size fits all
solution....so
I propose that we break the problem down into two neat little parts. First
would be actual plugin unit testing were we have a simple way to
instantiating the plugin and any of the pretty normal variables that get
injected, collections, strings, bool, etc. then we have basic junit type
testing on these objects, either through careful use of the execute method
on the mojo's or just being able to test the supporting methods like special
filename manipulation, downloading a wsdl from a website, perhaps even
generating some code. Basically, anything and everything that you can
massage on the plugin that doesn't stray into complex objects like
MavenProject, or ArtifactManagers or anything.
Now, some of plugins can be completely tested by this mechanism while others
might not actually fit too tell into this lower level testing. That is
where the integration testing comes into more of a play.
I talked to john about this and we were kind of a mind that stubbing and
mocking up these project objects and whatnot don't represent reality very
well and perhaps the best solution would be to follow the route of the
integration plugin where we can craft little projects in the plugin
directory that represent real life usages of the plugin, then we verify that
the right thing was done. This verification could take the form of
validating the resulting directory structure matched an outcome, certain
files matched an outcome, or the state of objects in the project matched the
template.
This all seems pretty obvious, but I wanted to at least start a conversation
about how we ought to test these plugins. I like this clear delineation in
that it draws a line in the sand as to what can get tested where between
src/test/java and src/it/*. It also encourages a clean plugin design as the
unit testing should make it simple for you to easily test the checking of
variable existence and states and helper methods. Although, I know vincent
has had some success with mocking up some of the more complex objects so I
would be interested in his thoughts on the matter...and bretts as well on if
I am barking up the wrong tree...
so, thoughts?
jesse
--
jesse mcconnell
jesseDOTmcconnellATgmailDOTcom
Re: plugin testing
Posted by Jesse McConnell <je...@gmail.com>.
wow, great feedback guys :)
vmassol: I'll read up on mocks tonight, I really need to learn more about
them before I talk about them at any lvl beyond what i have...if I am going
to try and build something out it ought to be useful :P Hopefully you and I
can chat a bit this week about all this once I am more up to speed on
mocking..
jesse
On 2/18/06, Vincent Massol <vm...@pivolis.com> wrote:
>
> Hi Carlos,
>
> Just wanted to say that I'm not against providing some stubs. I think this
> is very useful. I guess the only thing I'm saying is that unit testing of
> plugins is possible right now using mocks. If I find some time in the
> future, I'll document it.
>
> Thanks
> -Vincent
>
> > -----Original Message-----
> > From: carlossg@gmail.com [mailto:carlossg@gmail.com] On Behalf Of Carlos
> > Sanchez
> > Sent: samedi 18 février 2006 21:32
> > To: Maven Developers List
> > Subject: Re: plugin testing
> >
> > On 2/18/06, Vincent Massol <vm...@pivolis.com> wrote:
> > >
> > >
> > > > -----Original Message-----
> > > > From: carlossg@gmail.com [mailto:carlossg@gmail.com] On Behalf Of
> > Carlos
> > > > Sanchez
> > > > Sent: samedi 18 février 2006 20:14
> > > > To: Maven Developers List
> > > > Subject: Re: plugin testing
> > > >
> > > > I thought more about static mocks vs. dinamic mocks, i think this is
> > > > what you call stubs vs. mocks.
> > >
> > > Not quite. There are 2 types of mocks: static and dynamic. They are
> > > different from stubs. I could point you at JUnit in Action where I've
> > tried
> > > to explain the difference ;-) but here's another explanation:
> > > http://martinfowler.com/bliki/TestDouble.html
> > >
> >
> > Thanks for the pointer. After reading it, what I'd like to have are
> stubs
> >
> >
> > > > What I found is that while jmock is great for certain cases, like
> > > > throwing an exception as you say, for other common tasks static
> mocks
> > > > are easier and require less test code.
> > >
> > > I don't quite agree... :-)
> > >
> > > When this happens this is a code smell. As I said either you're
> mocking
> > too
> > > deep or your code has a design issue. Here's a very good article on
> the
> > > topic of using mocks:
> > > http://www.jmock.org/oopsla2004.pdf
> > >
> >
> > Creating a mock with jMock takes more lines than creating a Stub
> > (castings, setting expectations,... vs. just "new X()"), that's why
> > I'd like to have stubs for the common objects.
> >
> >
> > > I do agree though that there are some very simple cases when you don't
> > need
> > > a mock at all. For example when you have a data object there's usually
> > no
> > > point in mocking it.
> >
> > Sure
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > For additional commands, e-mail: dev-help@maven.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
--
jesse mcconnell
jesseDOTmcconnellATgmailDOTcom
RE: plugin testing
Posted by Vincent Massol <vm...@pivolis.com>.
Hi Carlos,
Just wanted to say that I'm not against providing some stubs. I think this
is very useful. I guess the only thing I'm saying is that unit testing of
plugins is possible right now using mocks. If I find some time in the
future, I'll document it.
Thanks
-Vincent
> -----Original Message-----
> From: carlossg@gmail.com [mailto:carlossg@gmail.com] On Behalf Of Carlos
> Sanchez
> Sent: samedi 18 février 2006 21:32
> To: Maven Developers List
> Subject: Re: plugin testing
>
> On 2/18/06, Vincent Massol <vm...@pivolis.com> wrote:
> >
> >
> > > -----Original Message-----
> > > From: carlossg@gmail.com [mailto:carlossg@gmail.com] On Behalf Of
> Carlos
> > > Sanchez
> > > Sent: samedi 18 février 2006 20:14
> > > To: Maven Developers List
> > > Subject: Re: plugin testing
> > >
> > > I thought more about static mocks vs. dinamic mocks, i think this is
> > > what you call stubs vs. mocks.
> >
> > Not quite. There are 2 types of mocks: static and dynamic. They are
> > different from stubs. I could point you at JUnit in Action where I've
> tried
> > to explain the difference ;-) but here's another explanation:
> > http://martinfowler.com/bliki/TestDouble.html
> >
>
> Thanks for the pointer. After reading it, what I'd like to have are stubs
>
>
> > > What I found is that while jmock is great for certain cases, like
> > > throwing an exception as you say, for other common tasks static mocks
> > > are easier and require less test code.
> >
> > I don't quite agree... :-)
> >
> > When this happens this is a code smell. As I said either you're mocking
> too
> > deep or your code has a design issue. Here's a very good article on the
> > topic of using mocks:
> > http://www.jmock.org/oopsla2004.pdf
> >
>
> Creating a mock with jMock takes more lines than creating a Stub
> (castings, setting expectations,... vs. just "new X()"), that's why
> I'd like to have stubs for the common objects.
>
>
> > I do agree though that there are some very simple cases when you don't
> need
> > a mock at all. For example when you have a data object there's usually
> no
> > point in mocking it.
>
> Sure
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: plugin testing
Posted by Carlos Sanchez <ca...@apache.org>.
On 2/18/06, Vincent Massol <vm...@pivolis.com> wrote:
>
>
> > -----Original Message-----
> > From: carlossg@gmail.com [mailto:carlossg@gmail.com] On Behalf Of Carlos
> > Sanchez
> > Sent: samedi 18 février 2006 20:14
> > To: Maven Developers List
> > Subject: Re: plugin testing
> >
> > I thought more about static mocks vs. dinamic mocks, i think this is
> > what you call stubs vs. mocks.
>
> Not quite. There are 2 types of mocks: static and dynamic. They are
> different from stubs. I could point you at JUnit in Action where I've tried
> to explain the difference ;-) but here's another explanation:
> http://martinfowler.com/bliki/TestDouble.html
>
Thanks for the pointer. After reading it, what I'd like to have are stubs
> > What I found is that while jmock is great for certain cases, like
> > throwing an exception as you say, for other common tasks static mocks
> > are easier and require less test code.
>
> I don't quite agree... :-)
>
> When this happens this is a code smell. As I said either you're mocking too
> deep or your code has a design issue. Here's a very good article on the
> topic of using mocks:
> http://www.jmock.org/oopsla2004.pdf
>
Creating a mock with jMock takes more lines than creating a Stub
(castings, setting expectations,... vs. just "new X()"), that's why
I'd like to have stubs for the common objects.
> I do agree though that there are some very simple cases when you don't need
> a mock at all. For example when you have a data object there's usually no
> point in mocking it.
Sure
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
RE: plugin testing
Posted by Vincent Massol <vm...@pivolis.com>.
> -----Original Message-----
> From: carlossg@gmail.com [mailto:carlossg@gmail.com] On Behalf Of Carlos
> Sanchez
> Sent: samedi 18 février 2006 20:14
> To: Maven Developers List
> Subject: Re: plugin testing
>
> I thought more about static mocks vs. dinamic mocks, i think this is
> what you call stubs vs. mocks.
Not quite. There are 2 types of mocks: static and dynamic. They are
different from stubs. I could point you at JUnit in Action where I've tried
to explain the difference ;-) but here's another explanation:
http://martinfowler.com/bliki/TestDouble.html
> What I found is that while jmock is great for certain cases, like
> throwing an exception as you say, for other common tasks static mocks
> are easier and require less test code.
I don't quite agree... :-)
When this happens this is a code smell. As I said either you're mocking too
deep or your code has a design issue. Here's a very good article on the
topic of using mocks:
http://www.jmock.org/oopsla2004.pdf
I do agree though that there are some very simple cases when you don't need
a mock at all. For example when you have a data object there's usually no
point in mocking it.
Anyway, YMMV. Mocks are great but are also difficult to understand and use
properly. This is probably the reason people are not using them more. I
think using mocks purely for testing is an error. They should be used in
conjunction with the desire to improve the design of the code they are
testing.
-Vincent
> On 2/18/06, Vincent Massol <vm...@pivolis.com> wrote:
> >
> >
> > > -----Original Message-----
> > > From: Brett Porter [mailto:brett@apache.org]
> > > Sent: vendredi 17 février 2006 23:59
> > > To: Maven Developers List
> > > Subject: Re: plugin testing
> > >
> > >
> > > I think this is the right separation. Unit test to get coverage, add
> and
> > > use setters like Vincent suggested. Integration test to verify things
> > > like the lifecycle intereactions, etc. controlled by the annotations
> at
> > > the class level. We can already do both.
> > >
> > > As Carlos mentioned, getting the objects is difficult. We do need a
> mock
> > > MavenProject instance, and can use PlexusTestCase as the base to be
> able
> > > to lookup components to set.
> > >
> > > Eventually I'd suggest creating a new module with an abstract test
> case
> > > that extends PlexusTestCase. This can be used to lookup components to
> > > set (and possibly automatically wire them up which would be much more
> > > convenient):
> > >
> > > setUp() {
> > > mojo = lookup( Mojo.ROLE, "goal name" );
> > > }
> > >
> > > In this case we also need to have an expression evaluator replacement
> > > that will provide replacements for ${project} and ${settings}.
> > >
> > > I'd start by manually injecting those values though and see how it
> goes.
> >
> > I think what you're describing is a stub but not a mock. The advantage
> of a
> > dynamic mock is that you don't need to code any method. It's the user of
> the
> > mock which says what behavior it should have for the methods it calls on
> the
> > mock. This seems to me the best possible thing because:
> >
> > - it makes it easy for the user to have several tests with different
> values
> > coming from the mock whereas with a stub it's not always going to be
> easy to
> > configure everything you want the way you want it.
> >
> > - it's hard to test exception in a stub whereas with a mock you can say
> that
> > for this test you want such method to throw such exception.
> >
> > My belief is that people do not know mock objects (e.g. jmock) and this
> is
> > why they think it's difficult (in most cases they think mock = stubs). I
> > have the feeling that providing samples on the web site of how to use
> jmock
> > to test a plugin would solve a huge percentage of unit testing cases.
> >
> > Last, when your unit test has to use too many mocks and define too many
> > behaviors it means 2 things:
> >
> > - you're not mocking at the right level (you're going too deep)
> > - the API is bad and needs refactoring
> >
> > [snip]
> >
> > Thanks
> > -Vincent
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > For additional commands, e-mail: dev-help@maven.apache.org
> >
> >
>
>
> --
> I could give you my word as a Spaniard.
> No good. I've known too many Spaniards.
> -- The Princess Bride
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: plugin testing
Posted by Carlos Sanchez <ca...@apache.org>.
I thought more about static mocks vs. dinamic mocks, i think this is
what you call stubs vs. mocks.
What I found is that while jmock is great for certain cases, like
throwing an exception as you say, for other common tasks static mocks
are easier and require less test code.
On 2/18/06, Vincent Massol <vm...@pivolis.com> wrote:
>
>
> > -----Original Message-----
> > From: Brett Porter [mailto:brett@apache.org]
> > Sent: vendredi 17 février 2006 23:59
> > To: Maven Developers List
> > Subject: Re: plugin testing
> >
> >
> > I think this is the right separation. Unit test to get coverage, add and
> > use setters like Vincent suggested. Integration test to verify things
> > like the lifecycle intereactions, etc. controlled by the annotations at
> > the class level. We can already do both.
> >
> > As Carlos mentioned, getting the objects is difficult. We do need a mock
> > MavenProject instance, and can use PlexusTestCase as the base to be able
> > to lookup components to set.
> >
> > Eventually I'd suggest creating a new module with an abstract test case
> > that extends PlexusTestCase. This can be used to lookup components to
> > set (and possibly automatically wire them up which would be much more
> > convenient):
> >
> > setUp() {
> > mojo = lookup( Mojo.ROLE, "goal name" );
> > }
> >
> > In this case we also need to have an expression evaluator replacement
> > that will provide replacements for ${project} and ${settings}.
> >
> > I'd start by manually injecting those values though and see how it goes.
>
> I think what you're describing is a stub but not a mock. The advantage of a
> dynamic mock is that you don't need to code any method. It's the user of the
> mock which says what behavior it should have for the methods it calls on the
> mock. This seems to me the best possible thing because:
>
> - it makes it easy for the user to have several tests with different values
> coming from the mock whereas with a stub it's not always going to be easy to
> configure everything you want the way you want it.
>
> - it's hard to test exception in a stub whereas with a mock you can say that
> for this test you want such method to throw such exception.
>
> My belief is that people do not know mock objects (e.g. jmock) and this is
> why they think it's difficult (in most cases they think mock = stubs). I
> have the feeling that providing samples on the web site of how to use jmock
> to test a plugin would solve a huge percentage of unit testing cases.
>
> Last, when your unit test has to use too many mocks and define too many
> behaviors it means 2 things:
>
> - you're not mocking at the right level (you're going too deep)
> - the API is bad and needs refactoring
>
> [snip]
>
> Thanks
> -Vincent
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
--
I could give you my word as a Spaniard.
No good. I've known too many Spaniards.
-- The Princess Bride
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: plugin testing
Posted by Jesse McConnell <je...@gmail.com>.
http://jira.codehaus.org/browse/MNG-2093
for the regular test case patch and then the plexified approach as well
On 2/20/06, Jesse McConnell <je...@gmail.com> wrote:
>
> ok, I fiddled around a bit today with the clean plugin as a basic test
> case...
>
> I really don't like the idea of putting setters on the mojo...not sure
> why, but it bugs me, probably because it is really only putting them there
> for testing purposes which is generally a no-no.
>
> 1) so I did it once with just a normal junit test class and put the
> setters on the mojo. Very little new code had to be added to get it working
> and it was trivial to take that plugin up to about 85% coverage...the
> remainder being some workaround for a windows jdk shortcoming.
>
> 2) then I redid it using the approach that Trygvis was doing in his
> deb-maven-plugin where you make the mojo a basic adapter to the
> implementation which is put together in a standard plexus layout....this
> forced the generation of a fair bit more code and a couple of additional
> classes, but it certainly felt more true to the nature of what a mojo
> is...at least my perception of what a mojo is :)
>
> So I am curious as to what people think about that idea of having the
> mojo's setup in that way? It seems a bit heavywieght for the clean pluginbut how important is it to not put those setters on the mojo? I have always
> been a bit of a fan for the correct conceptual way as opposed to the cheap
> and easy...though the cheap and easy can be faster :)
>
> Another thought I had on this is that with this adapter approach it is
> real easy to show the template for the plugin and how to use it...and it
> offers a real easy stepping stone to understanding how maven is built deeper
> in...and how much is that worth?
>
> so I am not really proposing anything with this followup mail...I'll
> generate a patch with my first test case setup for the clean plugin and
> put that in jira... I am however wondering if people generally like the
> plexified approach for the plugin implementation? I wouldn't be adverse
> to converting a couple more over to see how they look....and it does make
> them feel a bit more modular and useful outside of maven for a toolset like
> axistools, sablecc, or javacc since normal people don't have a facility for
> injecting those params..
>
> jesse
>
>
> On 2/19/06, Jesse McConnell <je...@gmail.com> wrote:
> >
> > after reading up on mocks from the links that vincent posted, I am going
> > to take a stab at putting together a minor set of these to work with for a
> > couple of the plugins...just to see how it would work out. Hopefully I can
> > get with vincent a bit tomorrow to make sure I get it close to right the
> > first time
> >
> > jesse
> >
> > On 2/18/06, Brett Porter < brett@apache.org> wrote:
> > >
> > > Vincent Massol wrote:
> > > > I think what you're describing is a stub but not a mock. The
> > > advantage of a
> > > > dynamic mock is that you don't need to code any method. It's the
> > > user of the
> > > > mock which says what behavior it should have for the methods it
> > > calls on the
> > > > mock.
> > >
> > > You're right, I've always referred to stubs incorrectly as mocks. I
> > > meant a stub. I think it's in our interest to produce these to make
> > > testing easier and more consistent for everyone.
> > >
> > > I'm interested to see your thoughts on the mocks eventually though -
> > > I've never really done anything with them since I was reading JiA
> > > (which
> > > I don't have any more :(
> > >
> > > - Brett
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > For additional commands, e-mail: dev-help@maven.apache.org
> > >
> > >
> >
> >
> > --
> > jesse mcconnell
> > jesseDOTmcconnellATgmailDOTcom
>
>
>
>
> --
> jesse mcconnell
> jesseDOTmcconnellATgmailDOTcom
--
jesse mcconnell
jesseDOTmcconnellATgmailDOTcom
Re: plugin testing
Posted by Jesse McConnell <je...@gmail.com>.
I reread through some of this thread this morning which prompted me poke
around the ReflectionUtils classin plexus-utils and added some functionality
to it as well as the AbstractMojoTestCase..
the test cases extending the harness now have the ability to get and set
values on the mojo and internal variables using reflectionutils...this
should address concerns about wanting to be able to do testing entirely in
java, using the plugin configuration xml route, or a combination of both.
I added example test cases to the simple mojo test that comes with the
harness and added a test class to plexus utils to test all that
functionality and it all seems to be a go.
http://docs.codehaus.org/display/MAVEN/Maven+Plugin+Harness
I also updated the documentation at the link above so people can see how to
use this new mechanism (and a couple of warnings :)
with this ability to now mine the state of the mojo and its internals with
impunity I am getting pretty happy with the way the harness is shaping up in
terms of actually being useful...now I just need some more people to give it
a spin and adjust accordingly.
feedback on both the usage of the harness and perhaps
issues/concerns/comments on the guide linked above would be much
appreciated!
cheers,
jesse
p.s. I might actually let this thread die someday :P
On 3/14/06, Jesse McConnell <je...@gmail.com> wrote:
>
> its back, the thread from heck and gone...
>
> ok, so now that we have a moderately working plugin harness that I have
> been able to get working for a few plugins, what should be the next step on
> this?
>
> Talking it over with jdcasey we both loathe the idea of just moving the
> plugin harness directly over to the plugin trunk. One idea that we both
> kinda liked was pulling a branch and further developing the plugin harness
> and the test cases for each plugin there and then when we are happy just
> doing a wholesale merge to the trunk. This shouldn't be bad as I'll
> illustrate next :)
>
> The nature of the harness means that having to modify the actual plugin
> source in any way is an indication that we missed something (IMO at least)
> so we shouldn't have to worry about breaking plugins at least up to the test
> phase...Adding tests to the plugin should only be a matter of preparing a
> *MojoTest and the various plugin-pom.xml files for each test case in a
> directory structure.
>
> For the examples I was working with I did:
>
> src/test/resources/unit/<shortDesc>-test/plugin-pom.xml
>
> then in the test case when I looked up my mojo I used the location that
> the file would have been copied to in:
>
> target/test-classes/unit/<shortDesc>-test/plugin-pom.xml
>
> an other files that were needed were nested below the plugin-pom.xml in
> src/main/foo or whatever.
>
> so...any ideas on how this ought to go from here?
>
> oh, and the jira issue on this and related confluence documentation is:
>
> http://jira.codehaus.org/browse/MNG-32
>
> http://docs.codehaus.org/display/MAVEN/Maven+Plugin+Harness
>
>
> Jesse
>
>
>
> On 3/7/06, Vincent Massol < vmassol@pivolis.com> wrote:
>
> > +1 to all that! Exactly my sentiment.
> >
> > Thanks
> > -Vincent
> >
> > > -----Original Message-----
> > > From: Brett Porter [mailto:brett@apache.org]
> > > Sent: mardi 7 mars 2006 22:37
> > > To: Maven Developers List
> > > Subject: Re: plugin testing
> > >
> > > Summarising this thread...
> > >
> > > Terminology I mean when I mention things:
> > > - unit testing, intended to test this module only, independent of its
> > > interaction with Maven and dependencies
> > > - integration testing, in this context, is intended to test that the
> > > plugin works in Maven, that it's metadata is bound correctly,
> > lifecycle
> > > interactions, etc. Done via the embedder.
> > >
> > > I saw this as being the unit testing side, and the reason we needed
> > > stubs is because we weren't using real objects.
> > >
> > > It sounds like what we have now is just great as it is, and I'd like
> > to
> > > start with that and see how far we get. My only issue remains the
> > > confusing POM-that-isn't-a-POM that you were going to fix anyway.
> > >
> > > I was concerned with the perceived direction towards using more real
> > > Maven stuff. I think there are several levels to this:
> > >
> > > * artifact code. I think it's quite reasonable to use this via
> > > maven-artifact as it is fairly pojo like anyway. No new stubs needed.
> > >
> > > * project builder. If you need a whole pom with interpolation and
> > > inheritance, you build up a local repo and use the project builder. I
> > > think this indicates a smell in the mojo (see more details below), but
> > > we might have to live with it in the current setup. The framework
> > > shouldn't care - it just takes in a MavenProject, and someone has
> > > already either instantiated a stub, or built with the project builder
> > > component. Still, we should start to consider the embedder here.
> > >
> > > * artifact code that sets up the whole environment through
> > > maven-artifact-manager. I dislike using this for mojos, though I set
> > up
> > > maven-artifact-test for it. Primarily for use by other libraries
> > > (modello's tests, maybe artifact-ant). I think by this point we should
> > > be thinking about the embedder anyway.
> > >
> > > * the embedder should be used for anything that really relies on the
> > > Maven environment, lifecycle handling, or any testing of how
> > expressions
> > > are set.
> > >
> > > While the embedder certainly has its purposes, it should be used when
> > > unit tests can't do something on their own.
> > >
> > > I don't know if anyone has built the eclipse plugin lately, but here
> > are
> > > some figures:
> > > * time to build skipping tests: 5 seconds
> > > * time to build running tests: 25 seconds
> > > * number of tests: 11
> > > * number of (non-svn) files in src/test: 241
> > > * code covered: ~65%
> > >
> > > This really isn't scalable. More than 20 files per test and 2-3
> > seconds
> > > each.
> > >
> > > We now have 100 integration tests for Maven itself which take maybe
> > 5-6
> > > seconds each, much that could instead be done through unit tests. It's
> >
> > > just too slow to write and to run, and starting to defeat the purpose.
> > > Bootstrapping is something you have to walk away from and grab a
> > coffee
> > > again.
> > >
> > > So that's my summary of the situation. I'll review the framework and
> > > tests in place and see where we go forward from here.
> > >
> > > Specific responses....
> > >
> > > Jesse McConnell wrote:
> > > > the repo will need to be real at some point, some plugins needs to
> > > interact
> > > > with the real thing, for getting jars for packaging or for just
> > looking
> > > into
> > > > for resources or something..so the artifact repo seems to be a
> > required
> > > part
> > > > of actual unit tests..
> > >
> > > Probably, though it should never be a remote repo. There are already
> > > some tools in maven-artifact-test for this.
> > >
> > > I'm still very cautious about this though. It's really pushing into
> > the
> > > realms of integration testing.
> > >
> > > > simple, I'll take care of this today... <plugin>-config.xml maybe?
> > >
> > > I think it can be called whatever you like and passed in as a File,
> > > right? If you look at some of the model tests, there is
> > > pom-that-includes-something.xml, pom-that-doesnt.xml, etc.
> > >
> > > > I think we can get away with all this without actually needing to
> > add
> > > > getters and setters, personally I think adding setters somewhat
> > clouds
> > > the
> > > > interaction with mojo's, have a combo of setters and private field
> > > injection
> > > > seems prone to some kind of abuse
> > >
> > > Well, plexus is using the setters if they are there anyway, so there's
> > > no abuse. I'm fine with leaving that to the discretion of the Mojo
> > > author. As long as its possible.
> > >
> > > > I was getting visions of people just making one test pom and either
> > > reusing
> > > > or copying and modifying a bit.
> > >
> > > So say there are two fields with six possible values each and I need
> > to
> > > test them all with each other for some reason. Do I create 36 poms, or
> >
> > > do I create one and then call the Java setters to modify those values?
> > >
> > > An example you can refer to here are the maven-project tests that test
> > > scopes work from the POM. I felt like that was much harder to work
> > with
> > > than the equivalent unit tests later put into maven-artifact to do the
> > > same thing.
> > >
> > > > I agree with you here, it is disjointed to have the configuration
> > and
> > > the
> > > > asserts in another place...but if we are going to route of a new
> > file
> > > for
> > > > configuring these things, how about just building out an assert
> > setup in
> > > the
> > > > configuration xml?
> > > >
> > > > <test>
> > > > <configuration>
> > > > ...
> > > > </configuration>
> > > > <asserts>
> > > > <assertTrue>expression</
> > > > ..
> > > > ..
> > >
> > > Well, now you are really starting a separate testing framework and the
> >
> > > Java won't be required at all. That's a lot of work. Something I'd
> > love
> > > to investigate, but a lot of work. We probably should be looking at
> > > using a scripting language to write the tests, perhaps.
> > >
> > > Anyway... more responses.
> > >
> > > Jesse McConnell wrote:
> > > > Mojo's need some way of being tested and having coverage determined.
> > >
> > > This actually works if the embedder is used via surefire. Not sure
> > about
> > > the IT plugin. The purpose is more about being tested easily, and
> > quickly.
> > >
> > > Jason van Zyl wrote:
> > > > Brett Porter wrote:
> > > >> A stub of the maven project builder, or the real one? The real one
> > uses
> > > >> the repo, and that's not really conducive to unit testing.
> > > >
> > > > The harness somewhat blurs the lines but once you execute the Mojo I
> > > > don't really consider it a unit test anymore.
> > >
> > > I don't see why it can't be. What happened to the OJO in MOJO?
> > >
> > > > Even with the simple clean
> > > > plugin example that you started with executed the Mojo. I think
> > that's
> > > > what's useful for testing and many Mojos need a local repository so
> > I
> > > > was suggesting using the real project builder passing in a local
> > > > repository created for testing purposes.
> > >
> > > The clean example is quirky because it basically does nothing outside
> > of
> > > what should be in external libraries, and has very little
> > configuration.
> > > If all of our plugins were like this, it would be great, and none of
> > > this would be necessary, but there is plenty of Maven specific
> > > functionality that needs to go into these if you look at plugins like
> > > jar, eclipse, etc.
> > >
> > > My problem with using a real project is that it is a really
> > complicated
> > > piece of code, that requires a local repository. Setting up that repo
> > is
> > > fine, but its really quite tedious. All of that complicated code runs
> > > before the mojo is instantiated - so the mojo tests have no business
> > > caring what it does. All we need is a prepopulated project object.
> > Now,
> > > we can certainly use the project builder for this, if it is simpler to
> > > do so. I have no objections to it. But I like the idea of having a
> > stub
> > > that is a simple bean I can just call .setFoo() and .getFoo() on as
> > well
> > > on too.
> > >
> > > Another reason I dislike the project builder is that we really should
> > > not be using ${project} on its own in the mojos. That binds you to a
> > > non-trivial library in Maven core that limits the ability to change
> > the
> > > version. More use of ${project.field} instead would be welcomed by me,
> > > and I feel is more in the original spirit of Mojos.
> > >
> > > As before, I think we need to start doing the tests and see where it
> > > leads us. And definitely need to criticially review the tests and try
> > > and find ways to make them simpler in code, and faster to write.
> > >
> > > >> At one point, we were doing away with private field injection and
> > > >> recommending setters, so it is how they'd be used.
> > > >
> > > > Hasn't happened yet. Not sure if that will happen as I'm not sure
> > how
> > > > many Ant tasks are actually used outside of Ant. Given what we do in
> >
> > > > promoting the creation of components that get wrapped by a Mojo to
> > > > expose them in Maven I'm not sure if there's much point to
> > properties.
> > > > If that's what we decide then we can do that. I'm not fussed one way
> > or
> > > > the other.
> > >
> > > Ok, likewise. I think this is tied into the dependence on Maven
> > objects
> > > instead of making them simple POJOs. Maybe it was putting the M in
> > POJO
> > > that was the problem, rather than a lack of OJO in MOJO :)
> > >
> > > > But there are plugins that require a MavenProject and other parts of
> > the
> > > > POM not in the configuration for the plugin.
> > > > The IDEA plugin is one example that needs a POM so having the
> > pom.xml
> > > > file actually be the source of a MavenProject I think would work.
> > There
> > > > is also the resources plugin which requires elements in the <build/>
> > > > element. I don't think we'll get away with just the <configuration/>
> >
> > > > element for the plugin.
> > >
> > > If it needs to be a POM, it should be a POM, loaded with the project
> > > loader. If it is not a POM, it should not look like a POM, and just be
> > > the XML configuration fragment inserted into the instantiated Mojo. Or
> >
> > > it could all be done in java code with a stub project and setters on
> > the
> > > mojo. I'd like the alternative for both.
> > >
> > > > We've had this discussion before :-) They are not properties. We
> > happen
> > > > to abuse the field name being the same as the configuration element.
> > But
> > > > with a property the configuration element can be different then the
> > > > field name as the setter would intervene.
> > >
> > > Ok, I get confused on the terminology, sorry. But isn't it the case
> > > right now that if I add a setter to a mojo, that gets called instead?
> > Or
> > > is that only if you add the property element to the configuration?
> > >
> > > I think this needs to be written down in the documentation, as I
> > suspect
> > > nobody other than yourself knows how the setters truly work right now
> > > and it's probably clouding our perception of how to deal with them
> > here.
> > > property="..." is not listed on
> > > http://maven.apache.org/developers/mojo-api-specification.html.
> > >
> > > > In how many cases in a Mojo can you only set one field and actually
> > make
> > > > it do anything?
> > >
> > > surefirePlugin.setForkMode( "once" );
> > > surefirePlugin.setForkMode( "pertest" );
> > > surefirePlugin.setForkMode( "none" );
> > >
> > > I think there are plenty of examples.
> > >
> > > > I'm all for a Java way, but I think the pom.xml will be
> > > > the most thorough and actually correspond most accurately to real
> > use.
> > > > Whatever kind of testing that actually is.
> > >
> > > That's fine. Let's allow both, and find out through experience.
> > >
> > > > In SuiteRunner the input and assertions were in the same file. Sure,
> > it
> > > > can't hurt to have both methods and I think we need a method in Java
> >
> > > > simply because we will probably find holes in the declarative
> > structure
> > > > and that shouldn't stop you from testing.
> > >
> > > Ok, so we are generally in agreement. Summary at the top.
> > >
> > > Cheers,
> > > Brett
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > For additional commands, e-mail: dev-help@maven.apache.org
> >
> >
> >
> >
> >
> >
> > ___________________________________________________________________________
> >
> > Nouveau : t�l�phonez moins cher avec Yahoo! Messenger ! D�couvez les
> > tarifs exceptionnels pour appeler la France et l'international.
> > T�l�chargez sur http://fr.messenger.yahoo.com
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > For additional commands, e-mail: dev-help@maven.apache.org
> >
> >
>
>
> --
> jesse mcconnell
> jesseDOTmcconnellATgmailDOTcom
>
--
jesse mcconnell
jesseDOTmcconnellATgmailDOTcom
Re: plugin testing
Posted by Jesse McConnell <je...@gmail.com>.
its back, the thread from heck and gone...
ok, so now that we have a moderately working plugin harness that I have been
able to get working for a few plugins, what should be the next step on this?
Talking it over with jdcasey we both loathe the idea of just moving the
plugin harness directly over to the plugin trunk. One idea that we both
kinda liked was pulling a branch and further developing the plugin harness
and the test cases for each plugin there and then when we are happy just
doing a wholesale merge to the trunk. This shouldn't be bad as I'll
illustrate next :)
The nature of the harness means that having to modify the actual plugin
source in any way is an indication that we missed something (IMO at least)
so we shouldn't have to worry about breaking plugins at least up to the test
phase...Adding tests to the plugin should only be a matter of preparing a
*MojoTest and the various plugin-pom.xml files for each test case in a
directory structure.
For the examples I was working with I did:
src/test/resources/unit/<shortDesc>-test/plugin-pom.xml
then in the test case when I looked up my mojo I used the location that the
file would have been copied to in:
target/test-classes/unit/<shortDesc>-test/plugin-pom.xml
an other files that were needed were nested below the plugin-pom.xml in
src/main/foo or whatever.
so...any ideas on how this ought to go from here?
oh, and the jira issue on this and related confluence documentation is:
http://jira.codehaus.org/browse/MNG-32
http://docs.codehaus.org/display/MAVEN/Maven+Plugin+Harness
Jesse
On 3/7/06, Vincent Massol <vm...@pivolis.com> wrote:
>
> +1 to all that! Exactly my sentiment.
>
> Thanks
> -Vincent
>
> > -----Original Message-----
> > From: Brett Porter [mailto:brett@apache.org]
> > Sent: mardi 7 mars 2006 22:37
> > To: Maven Developers List
> > Subject: Re: plugin testing
> >
> > Summarising this thread...
> >
> > Terminology I mean when I mention things:
> > - unit testing, intended to test this module only, independent of its
> > interaction with Maven and dependencies
> > - integration testing, in this context, is intended to test that the
> > plugin works in Maven, that it's metadata is bound correctly, lifecycle
> > interactions, etc. Done via the embedder.
> >
> > I saw this as being the unit testing side, and the reason we needed
> > stubs is because we weren't using real objects.
> >
> > It sounds like what we have now is just great as it is, and I'd like to
> > start with that and see how far we get. My only issue remains the
> > confusing POM-that-isn't-a-POM that you were going to fix anyway.
> >
> > I was concerned with the perceived direction towards using more real
> > Maven stuff. I think there are several levels to this:
> >
> > * artifact code. I think it's quite reasonable to use this via
> > maven-artifact as it is fairly pojo like anyway. No new stubs needed.
> >
> > * project builder. If you need a whole pom with interpolation and
> > inheritance, you build up a local repo and use the project builder. I
> > think this indicates a smell in the mojo (see more details below), but
> > we might have to live with it in the current setup. The framework
> > shouldn't care - it just takes in a MavenProject, and someone has
> > already either instantiated a stub, or built with the project builder
> > component. Still, we should start to consider the embedder here.
> >
> > * artifact code that sets up the whole environment through
> > maven-artifact-manager. I dislike using this for mojos, though I set up
> > maven-artifact-test for it. Primarily for use by other libraries
> > (modello's tests, maybe artifact-ant). I think by this point we should
> > be thinking about the embedder anyway.
> >
> > * the embedder should be used for anything that really relies on the
> > Maven environment, lifecycle handling, or any testing of how expressions
> > are set.
> >
> > While the embedder certainly has its purposes, it should be used when
> > unit tests can't do something on their own.
> >
> > I don't know if anyone has built the eclipse plugin lately, but here are
> > some figures:
> > * time to build skipping tests: 5 seconds
> > * time to build running tests: 25 seconds
> > * number of tests: 11
> > * number of (non-svn) files in src/test: 241
> > * code covered: ~65%
> >
> > This really isn't scalable. More than 20 files per test and 2-3 seconds
> > each.
> >
> > We now have 100 integration tests for Maven itself which take maybe 5-6
> > seconds each, much that could instead be done through unit tests. It's
> > just too slow to write and to run, and starting to defeat the purpose.
> > Bootstrapping is something you have to walk away from and grab a coffee
> > again.
> >
> > So that's my summary of the situation. I'll review the framework and
> > tests in place and see where we go forward from here.
> >
> > Specific responses....
> >
> > Jesse McConnell wrote:
> > > the repo will need to be real at some point, some plugins needs to
> > interact
> > > with the real thing, for getting jars for packaging or for just
> looking
> > into
> > > for resources or something..so the artifact repo seems to be a
> required
> > part
> > > of actual unit tests..
> >
> > Probably, though it should never be a remote repo. There are already
> > some tools in maven-artifact-test for this.
> >
> > I'm still very cautious about this though. It's really pushing into the
> > realms of integration testing.
> >
> > > simple, I'll take care of this today... <plugin>-config.xml maybe?
> >
> > I think it can be called whatever you like and passed in as a File,
> > right? If you look at some of the model tests, there is
> > pom-that-includes-something.xml, pom-that-doesnt.xml, etc.
> >
> > > I think we can get away with all this without actually needing to add
> > > getters and setters, personally I think adding setters somewhat clouds
> > the
> > > interaction with mojo's, have a combo of setters and private field
> > injection
> > > seems prone to some kind of abuse
> >
> > Well, plexus is using the setters if they are there anyway, so there's
> > no abuse. I'm fine with leaving that to the discretion of the Mojo
> > author. As long as its possible.
> >
> > > I was getting visions of people just making one test pom and either
> > reusing
> > > or copying and modifying a bit.
> >
> > So say there are two fields with six possible values each and I need to
> > test them all with each other for some reason. Do I create 36 poms, or
> > do I create one and then call the Java setters to modify those values?
> >
> > An example you can refer to here are the maven-project tests that test
> > scopes work from the POM. I felt like that was much harder to work with
> > than the equivalent unit tests later put into maven-artifact to do the
> > same thing.
> >
> > > I agree with you here, it is disjointed to have the configuration and
> > the
> > > asserts in another place...but if we are going to route of a new file
> > for
> > > configuring these things, how about just building out an assert setup
> in
> > the
> > > configuration xml?
> > >
> > > <test>
> > > <configuration>
> > > ...
> > > </configuration>
> > > <asserts>
> > > <assertTrue>expression</
> > > ..
> > > ..
> >
> > Well, now you are really starting a separate testing framework and the
> > Java won't be required at all. That's a lot of work. Something I'd love
> > to investigate, but a lot of work. We probably should be looking at
> > using a scripting language to write the tests, perhaps.
> >
> > Anyway... more responses.
> >
> > Jesse McConnell wrote:
> > > Mojo's need some way of being tested and having coverage determined.
> >
> > This actually works if the embedder is used via surefire. Not sure about
> > the IT plugin. The purpose is more about being tested easily, and
> quickly.
> >
> > Jason van Zyl wrote:
> > > Brett Porter wrote:
> > >> A stub of the maven project builder, or the real one? The real one
> uses
> > >> the repo, and that's not really conducive to unit testing.
> > >
> > > The harness somewhat blurs the lines but once you execute the Mojo I
> > > don't really consider it a unit test anymore.
> >
> > I don't see why it can't be. What happened to the OJO in MOJO?
> >
> > > Even with the simple clean
> > > plugin example that you started with executed the Mojo. I think that's
> > > what's useful for testing and many Mojos need a local repository so I
> > > was suggesting using the real project builder passing in a local
> > > repository created for testing purposes.
> >
> > The clean example is quirky because it basically does nothing outside of
> > what should be in external libraries, and has very little configuration.
> > If all of our plugins were like this, it would be great, and none of
> > this would be necessary, but there is plenty of Maven specific
> > functionality that needs to go into these if you look at plugins like
> > jar, eclipse, etc.
> >
> > My problem with using a real project is that it is a really complicated
> > piece of code, that requires a local repository. Setting up that repo is
> > fine, but its really quite tedious. All of that complicated code runs
> > before the mojo is instantiated - so the mojo tests have no business
> > caring what it does. All we need is a prepopulated project object. Now,
> > we can certainly use the project builder for this, if it is simpler to
> > do so. I have no objections to it. But I like the idea of having a stub
> > that is a simple bean I can just call .setFoo() and .getFoo() on as well
> > on too.
> >
> > Another reason I dislike the project builder is that we really should
> > not be using ${project} on its own in the mojos. That binds you to a
> > non-trivial library in Maven core that limits the ability to change the
> > version. More use of ${project.field} instead would be welcomed by me,
> > and I feel is more in the original spirit of Mojos.
> >
> > As before, I think we need to start doing the tests and see where it
> > leads us. And definitely need to criticially review the tests and try
> > and find ways to make them simpler in code, and faster to write.
> >
> > >> At one point, we were doing away with private field injection and
> > >> recommending setters, so it is how they'd be used.
> > >
> > > Hasn't happened yet. Not sure if that will happen as I'm not sure how
> > > many Ant tasks are actually used outside of Ant. Given what we do in
> > > promoting the creation of components that get wrapped by a Mojo to
> > > expose them in Maven I'm not sure if there's much point to properties.
> > > If that's what we decide then we can do that. I'm not fussed one way
> or
> > > the other.
> >
> > Ok, likewise. I think this is tied into the dependence on Maven objects
> > instead of making them simple POJOs. Maybe it was putting the M in POJO
> > that was the problem, rather than a lack of OJO in MOJO :)
> >
> > > But there are plugins that require a MavenProject and other parts of
> the
> > > POM not in the configuration for the plugin.
> > > The IDEA plugin is one example that needs a POM so having the pom.xml
> > > file actually be the source of a MavenProject I think would work.
> There
> > > is also the resources plugin which requires elements in the <build/>
> > > element. I don't think we'll get away with just the <configuration/>
> > > element for the plugin.
> >
> > If it needs to be a POM, it should be a POM, loaded with the project
> > loader. If it is not a POM, it should not look like a POM, and just be
> > the XML configuration fragment inserted into the instantiated Mojo. Or
> > it could all be done in java code with a stub project and setters on the
> > mojo. I'd like the alternative for both.
> >
> > > We've had this discussion before :-) They are not properties. We
> happen
> > > to abuse the field name being the same as the configuration element.
> But
> > > with a property the configuration element can be different then the
> > > field name as the setter would intervene.
> >
> > Ok, I get confused on the terminology, sorry. But isn't it the case
> > right now that if I add a setter to a mojo, that gets called instead? Or
> > is that only if you add the property element to the configuration?
> >
> > I think this needs to be written down in the documentation, as I suspect
> > nobody other than yourself knows how the setters truly work right now
> > and it's probably clouding our perception of how to deal with them here.
> > property="..." is not listed on
> > http://maven.apache.org/developers/mojo-api-specification.html.
> >
> > > In how many cases in a Mojo can you only set one field and actually
> make
> > > it do anything?
> >
> > surefirePlugin.setForkMode( "once" );
> > surefirePlugin.setForkMode( "pertest" );
> > surefirePlugin.setForkMode( "none" );
> >
> > I think there are plenty of examples.
> >
> > > I'm all for a Java way, but I think the pom.xml will be
> > > the most thorough and actually correspond most accurately to real use.
> > > Whatever kind of testing that actually is.
> >
> > That's fine. Let's allow both, and find out through experience.
> >
> > > In SuiteRunner the input and assertions were in the same file. Sure,
> it
> > > can't hurt to have both methods and I think we need a method in Java
> > > simply because we will probably find holes in the declarative
> structure
> > > and that shouldn't stop you from testing.
> >
> > Ok, so we are generally in agreement. Summary at the top.
> >
> > Cheers,
> > Brett
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > For additional commands, e-mail: dev-help@maven.apache.org
>
>
>
>
>
>
>
> ___________________________________________________________________________
> Nouveau : t�l�phonez moins cher avec Yahoo! Messenger ! D�couvez les
> tarifs exceptionnels pour appeler la France et l'international.
> T�l�chargez sur http://fr.messenger.yahoo.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
--
jesse mcconnell
jesseDOTmcconnellATgmailDOTcom
RE: plugin testing
Posted by Vincent Massol <vm...@pivolis.com>.
+1 to all that! Exactly my sentiment.
Thanks
-Vincent
> -----Original Message-----
> From: Brett Porter [mailto:brett@apache.org]
> Sent: mardi 7 mars 2006 22:37
> To: Maven Developers List
> Subject: Re: plugin testing
>
> Summarising this thread...
>
> Terminology I mean when I mention things:
> - unit testing, intended to test this module only, independent of its
> interaction with Maven and dependencies
> - integration testing, in this context, is intended to test that the
> plugin works in Maven, that it's metadata is bound correctly, lifecycle
> interactions, etc. Done via the embedder.
>
> I saw this as being the unit testing side, and the reason we needed
> stubs is because we weren't using real objects.
>
> It sounds like what we have now is just great as it is, and I'd like to
> start with that and see how far we get. My only issue remains the
> confusing POM-that-isn't-a-POM that you were going to fix anyway.
>
> I was concerned with the perceived direction towards using more real
> Maven stuff. I think there are several levels to this:
>
> * artifact code. I think it's quite reasonable to use this via
> maven-artifact as it is fairly pojo like anyway. No new stubs needed.
>
> * project builder. If you need a whole pom with interpolation and
> inheritance, you build up a local repo and use the project builder. I
> think this indicates a smell in the mojo (see more details below), but
> we might have to live with it in the current setup. The framework
> shouldn't care - it just takes in a MavenProject, and someone has
> already either instantiated a stub, or built with the project builder
> component. Still, we should start to consider the embedder here.
>
> * artifact code that sets up the whole environment through
> maven-artifact-manager. I dislike using this for mojos, though I set up
> maven-artifact-test for it. Primarily for use by other libraries
> (modello's tests, maybe artifact-ant). I think by this point we should
> be thinking about the embedder anyway.
>
> * the embedder should be used for anything that really relies on the
> Maven environment, lifecycle handling, or any testing of how expressions
> are set.
>
> While the embedder certainly has its purposes, it should be used when
> unit tests can't do something on their own.
>
> I don't know if anyone has built the eclipse plugin lately, but here are
> some figures:
> * time to build skipping tests: 5 seconds
> * time to build running tests: 25 seconds
> * number of tests: 11
> * number of (non-svn) files in src/test: 241
> * code covered: ~65%
>
> This really isn't scalable. More than 20 files per test and 2-3 seconds
> each.
>
> We now have 100 integration tests for Maven itself which take maybe 5-6
> seconds each, much that could instead be done through unit tests. It's
> just too slow to write and to run, and starting to defeat the purpose.
> Bootstrapping is something you have to walk away from and grab a coffee
> again.
>
> So that's my summary of the situation. I'll review the framework and
> tests in place and see where we go forward from here.
>
> Specific responses....
>
> Jesse McConnell wrote:
> > the repo will need to be real at some point, some plugins needs to
> interact
> > with the real thing, for getting jars for packaging or for just looking
> into
> > for resources or something..so the artifact repo seems to be a required
> part
> > of actual unit tests..
>
> Probably, though it should never be a remote repo. There are already
> some tools in maven-artifact-test for this.
>
> I'm still very cautious about this though. It's really pushing into the
> realms of integration testing.
>
> > simple, I'll take care of this today... <plugin>-config.xml maybe?
>
> I think it can be called whatever you like and passed in as a File,
> right? If you look at some of the model tests, there is
> pom-that-includes-something.xml, pom-that-doesnt.xml, etc.
>
> > I think we can get away with all this without actually needing to add
> > getters and setters, personally I think adding setters somewhat clouds
> the
> > interaction with mojo's, have a combo of setters and private field
> injection
> > seems prone to some kind of abuse
>
> Well, plexus is using the setters if they are there anyway, so there's
> no abuse. I'm fine with leaving that to the discretion of the Mojo
> author. As long as its possible.
>
> > I was getting visions of people just making one test pom and either
> reusing
> > or copying and modifying a bit.
>
> So say there are two fields with six possible values each and I need to
> test them all with each other for some reason. Do I create 36 poms, or
> do I create one and then call the Java setters to modify those values?
>
> An example you can refer to here are the maven-project tests that test
> scopes work from the POM. I felt like that was much harder to work with
> than the equivalent unit tests later put into maven-artifact to do the
> same thing.
>
> > I agree with you here, it is disjointed to have the configuration and
> the
> > asserts in another place...but if we are going to route of a new file
> for
> > configuring these things, how about just building out an assert setup in
> the
> > configuration xml?
> >
> > <test>
> > <configuration>
> > ...
> > </configuration>
> > <asserts>
> > <assertTrue>expression</
> > ..
> > ..
>
> Well, now you are really starting a separate testing framework and the
> Java won't be required at all. That's a lot of work. Something I'd love
> to investigate, but a lot of work. We probably should be looking at
> using a scripting language to write the tests, perhaps.
>
> Anyway... more responses.
>
> Jesse McConnell wrote:
> > Mojo's need some way of being tested and having coverage determined.
>
> This actually works if the embedder is used via surefire. Not sure about
> the IT plugin. The purpose is more about being tested easily, and quickly.
>
> Jason van Zyl wrote:
> > Brett Porter wrote:
> >> A stub of the maven project builder, or the real one? The real one uses
> >> the repo, and that's not really conducive to unit testing.
> >
> > The harness somewhat blurs the lines but once you execute the Mojo I
> > don't really consider it a unit test anymore.
>
> I don't see why it can't be. What happened to the OJO in MOJO?
>
> > Even with the simple clean
> > plugin example that you started with executed the Mojo. I think that's
> > what's useful for testing and many Mojos need a local repository so I
> > was suggesting using the real project builder passing in a local
> > repository created for testing purposes.
>
> The clean example is quirky because it basically does nothing outside of
> what should be in external libraries, and has very little configuration.
> If all of our plugins were like this, it would be great, and none of
> this would be necessary, but there is plenty of Maven specific
> functionality that needs to go into these if you look at plugins like
> jar, eclipse, etc.
>
> My problem with using a real project is that it is a really complicated
> piece of code, that requires a local repository. Setting up that repo is
> fine, but its really quite tedious. All of that complicated code runs
> before the mojo is instantiated - so the mojo tests have no business
> caring what it does. All we need is a prepopulated project object. Now,
> we can certainly use the project builder for this, if it is simpler to
> do so. I have no objections to it. But I like the idea of having a stub
> that is a simple bean I can just call .setFoo() and .getFoo() on as well
> on too.
>
> Another reason I dislike the project builder is that we really should
> not be using ${project} on its own in the mojos. That binds you to a
> non-trivial library in Maven core that limits the ability to change the
> version. More use of ${project.field} instead would be welcomed by me,
> and I feel is more in the original spirit of Mojos.
>
> As before, I think we need to start doing the tests and see where it
> leads us. And definitely need to criticially review the tests and try
> and find ways to make them simpler in code, and faster to write.
>
> >> At one point, we were doing away with private field injection and
> >> recommending setters, so it is how they'd be used.
> >
> > Hasn't happened yet. Not sure if that will happen as I'm not sure how
> > many Ant tasks are actually used outside of Ant. Given what we do in
> > promoting the creation of components that get wrapped by a Mojo to
> > expose them in Maven I'm not sure if there's much point to properties.
> > If that's what we decide then we can do that. I'm not fussed one way or
> > the other.
>
> Ok, likewise. I think this is tied into the dependence on Maven objects
> instead of making them simple POJOs. Maybe it was putting the M in POJO
> that was the problem, rather than a lack of OJO in MOJO :)
>
> > But there are plugins that require a MavenProject and other parts of the
> > POM not in the configuration for the plugin.
> > The IDEA plugin is one example that needs a POM so having the pom.xml
> > file actually be the source of a MavenProject I think would work. There
> > is also the resources plugin which requires elements in the <build/>
> > element. I don't think we'll get away with just the <configuration/>
> > element for the plugin.
>
> If it needs to be a POM, it should be a POM, loaded with the project
> loader. If it is not a POM, it should not look like a POM, and just be
> the XML configuration fragment inserted into the instantiated Mojo. Or
> it could all be done in java code with a stub project and setters on the
> mojo. I'd like the alternative for both.
>
> > We've had this discussion before :-) They are not properties. We happen
> > to abuse the field name being the same as the configuration element. But
> > with a property the configuration element can be different then the
> > field name as the setter would intervene.
>
> Ok, I get confused on the terminology, sorry. But isn't it the case
> right now that if I add a setter to a mojo, that gets called instead? Or
> is that only if you add the property element to the configuration?
>
> I think this needs to be written down in the documentation, as I suspect
> nobody other than yourself knows how the setters truly work right now
> and it's probably clouding our perception of how to deal with them here.
> property="..." is not listed on
> http://maven.apache.org/developers/mojo-api-specification.html.
>
> > In how many cases in a Mojo can you only set one field and actually make
> > it do anything?
>
> surefirePlugin.setForkMode( "once" );
> surefirePlugin.setForkMode( "pertest" );
> surefirePlugin.setForkMode( "none" );
>
> I think there are plenty of examples.
>
> > I'm all for a Java way, but I think the pom.xml will be
> > the most thorough and actually correspond most accurately to real use.
> > Whatever kind of testing that actually is.
>
> That's fine. Let's allow both, and find out through experience.
>
> > In SuiteRunner the input and assertions were in the same file. Sure, it
> > can't hurt to have both methods and I think we need a method in Java
> > simply because we will probably find holes in the declarative structure
> > and that shouldn't stop you from testing.
>
> Ok, so we are generally in agreement. Summary at the top.
>
> Cheers,
> Brett
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
___________________________________________________________________________
Nouveau : t�l�phonez moins cher avec Yahoo! Messenger ! D�couvez les tarifs exceptionnels pour appeler la France et l'international.
T�l�chargez sur http://fr.messenger.yahoo.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: plugin testing
Posted by Brett Porter <br...@apache.org>.
Summarising this thread...
Terminology I mean when I mention things:
- unit testing, intended to test this module only, independent of its
interaction with Maven and dependencies
- integration testing, in this context, is intended to test that the
plugin works in Maven, that it's metadata is bound correctly, lifecycle
interactions, etc. Done via the embedder.
I saw this as being the unit testing side, and the reason we needed
stubs is because we weren't using real objects.
It sounds like what we have now is just great as it is, and I'd like to
start with that and see how far we get. My only issue remains the
confusing POM-that-isn't-a-POM that you were going to fix anyway.
I was concerned with the perceived direction towards using more real
Maven stuff. I think there are several levels to this:
* artifact code. I think it's quite reasonable to use this via
maven-artifact as it is fairly pojo like anyway. No new stubs needed.
* project builder. If you need a whole pom with interpolation and
inheritance, you build up a local repo and use the project builder. I
think this indicates a smell in the mojo (see more details below), but
we might have to live with it in the current setup. The framework
shouldn't care - it just takes in a MavenProject, and someone has
already either instantiated a stub, or built with the project builder
component. Still, we should start to consider the embedder here.
* artifact code that sets up the whole environment through
maven-artifact-manager. I dislike using this for mojos, though I set up
maven-artifact-test for it. Primarily for use by other libraries
(modello's tests, maybe artifact-ant). I think by this point we should
be thinking about the embedder anyway.
* the embedder should be used for anything that really relies on the
Maven environment, lifecycle handling, or any testing of how expressions
are set.
While the embedder certainly has its purposes, it should be used when
unit tests can't do something on their own.
I don't know if anyone has built the eclipse plugin lately, but here are
some figures:
* time to build skipping tests: 5 seconds
* time to build running tests: 25 seconds
* number of tests: 11
* number of (non-svn) files in src/test: 241
* code covered: ~65%
This really isn't scalable. More than 20 files per test and 2-3 seconds
each.
We now have 100 integration tests for Maven itself which take maybe 5-6
seconds each, much that could instead be done through unit tests. It's
just too slow to write and to run, and starting to defeat the purpose.
Bootstrapping is something you have to walk away from and grab a coffee
again.
So that's my summary of the situation. I'll review the framework and
tests in place and see where we go forward from here.
Specific responses....
Jesse McConnell wrote:
> the repo will need to be real at some point, some plugins needs to interact
> with the real thing, for getting jars for packaging or for just looking into
> for resources or something..so the artifact repo seems to be a required part
> of actual unit tests..
Probably, though it should never be a remote repo. There are already
some tools in maven-artifact-test for this.
I'm still very cautious about this though. It's really pushing into the
realms of integration testing.
> simple, I'll take care of this today... <plugin>-config.xml maybe?
I think it can be called whatever you like and passed in as a File,
right? If you look at some of the model tests, there is
pom-that-includes-something.xml, pom-that-doesnt.xml, etc.
> I think we can get away with all this without actually needing to add
> getters and setters, personally I think adding setters somewhat clouds the
> interaction with mojo's, have a combo of setters and private field injection
> seems prone to some kind of abuse
Well, plexus is using the setters if they are there anyway, so there's
no abuse. I'm fine with leaving that to the discretion of the Mojo
author. As long as its possible.
> I was getting visions of people just making one test pom and either reusing
> or copying and modifying a bit.
So say there are two fields with six possible values each and I need to
test them all with each other for some reason. Do I create 36 poms, or
do I create one and then call the Java setters to modify those values?
An example you can refer to here are the maven-project tests that test
scopes work from the POM. I felt like that was much harder to work with
than the equivalent unit tests later put into maven-artifact to do the
same thing.
> I agree with you here, it is disjointed to have the configuration and the
> asserts in another place...but if we are going to route of a new file for
> configuring these things, how about just building out an assert setup in the
> configuration xml?
>
> <test>
> <configuration>
> ...
> </configuration>
> <asserts>
> <assertTrue>expression</
> ..
> ..
Well, now you are really starting a separate testing framework and the
Java won't be required at all. That's a lot of work. Something I'd love
to investigate, but a lot of work. We probably should be looking at
using a scripting language to write the tests, perhaps.
Anyway... more responses.
Jesse McConnell wrote:
> Mojo's need some way of being tested and having coverage determined.
This actually works if the embedder is used via surefire. Not sure about
the IT plugin. The purpose is more about being tested easily, and quickly.
Jason van Zyl wrote:
> Brett Porter wrote:
>> A stub of the maven project builder, or the real one? The real one uses
>> the repo, and that's not really conducive to unit testing.
>
> The harness somewhat blurs the lines but once you execute the Mojo I
> don't really consider it a unit test anymore.
I don't see why it can't be. What happened to the OJO in MOJO?
> Even with the simple clean
> plugin example that you started with executed the Mojo. I think that's
> what's useful for testing and many Mojos need a local repository so I
> was suggesting using the real project builder passing in a local
> repository created for testing purposes.
The clean example is quirky because it basically does nothing outside of
what should be in external libraries, and has very little configuration.
If all of our plugins were like this, it would be great, and none of
this would be necessary, but there is plenty of Maven specific
functionality that needs to go into these if you look at plugins like
jar, eclipse, etc.
My problem with using a real project is that it is a really complicated
piece of code, that requires a local repository. Setting up that repo is
fine, but its really quite tedious. All of that complicated code runs
before the mojo is instantiated - so the mojo tests have no business
caring what it does. All we need is a prepopulated project object. Now,
we can certainly use the project builder for this, if it is simpler to
do so. I have no objections to it. But I like the idea of having a stub
that is a simple bean I can just call .setFoo() and .getFoo() on as well
on too.
Another reason I dislike the project builder is that we really should
not be using ${project} on its own in the mojos. That binds you to a
non-trivial library in Maven core that limits the ability to change the
version. More use of ${project.field} instead would be welcomed by me,
and I feel is more in the original spirit of Mojos.
As before, I think we need to start doing the tests and see where it
leads us. And definitely need to criticially review the tests and try
and find ways to make them simpler in code, and faster to write.
>> At one point, we were doing away with private field injection and
>> recommending setters, so it is how they'd be used.
>
> Hasn't happened yet. Not sure if that will happen as I'm not sure how
> many Ant tasks are actually used outside of Ant. Given what we do in
> promoting the creation of components that get wrapped by a Mojo to
> expose them in Maven I'm not sure if there's much point to properties.
> If that's what we decide then we can do that. I'm not fussed one way or
> the other.
Ok, likewise. I think this is tied into the dependence on Maven objects
instead of making them simple POJOs. Maybe it was putting the M in POJO
that was the problem, rather than a lack of OJO in MOJO :)
> But there are plugins that require a MavenProject and other parts of the
> POM not in the configuration for the plugin.
> The IDEA plugin is one example that needs a POM so having the pom.xml
> file actually be the source of a MavenProject I think would work. There
> is also the resources plugin which requires elements in the <build/>
> element. I don't think we'll get away with just the <configuration/>
> element for the plugin.
If it needs to be a POM, it should be a POM, loaded with the project
loader. If it is not a POM, it should not look like a POM, and just be
the XML configuration fragment inserted into the instantiated Mojo. Or
it could all be done in java code with a stub project and setters on the
mojo. I'd like the alternative for both.
> We've had this discussion before :-) They are not properties. We happen
> to abuse the field name being the same as the configuration element. But
> with a property the configuration element can be different then the
> field name as the setter would intervene.
Ok, I get confused on the terminology, sorry. But isn't it the case
right now that if I add a setter to a mojo, that gets called instead? Or
is that only if you add the property element to the configuration?
I think this needs to be written down in the documentation, as I suspect
nobody other than yourself knows how the setters truly work right now
and it's probably clouding our perception of how to deal with them here.
property="..." is not listed on
http://maven.apache.org/developers/mojo-api-specification.html.
> In how many cases in a Mojo can you only set one field and actually make
> it do anything?
surefirePlugin.setForkMode( "once" );
surefirePlugin.setForkMode( "pertest" );
surefirePlugin.setForkMode( "none" );
I think there are plenty of examples.
> I'm all for a Java way, but I think the pom.xml will be
> the most thorough and actually correspond most accurately to real use.
> Whatever kind of testing that actually is.
That's fine. Let's allow both, and find out through experience.
> In SuiteRunner the input and assertions were in the same file. Sure, it
> can't hurt to have both methods and I think we need a method in Java
> simply because we will probably find holes in the declarative structure
> and that shouldn't stop you from testing.
Ok, so we are generally in agreement. Summary at the top.
Cheers,
Brett
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: plugin testing
Posted by Jesse McConnell <je...@gmail.com>.
well, what I have been working with uses precious little of maven, and jason
set up the project at mojo so I could do commits on it and we could
collaborate.
Here is the nut that I am trying to crack:
Mojo's need some way of being tested and having coverage determined. This
is complicated by the fact that mojo's have a couple of different ways of
being initialized, either though private variable injection or in some cases
getters and setters. There is also the little problem Maven objects that
need to be stubbed out and to act correctly so the mojo code can be tested
against them outside of the maven execution engine.
There hasn't been any resolution on mojos using private variable
injection/getters and setters/or both issue so we need some mechanism of
populating the mojo's which is where the use of plexus comes in.
way back up in the thread there were two different concepts thrown around, 1
was 'unit' testing the mojo's which would be getting an instance of a mojo
populated and then the massage it through different means to increase
coverage.
the other was more of an integration with maven idea where it would tie into
maven itself and each plugin could have what would amount to an battery of
integration tests available.
hope that kinda summaries things up to this point.
jesse
On 3/7/06, Vincent Massol <vm...@pivolis.com> wrote:
>
> I guess the terminology is still an issue but what the it plugin does and
> what Jesse is implementing are for me integration tests and not unit
> tests.
>
> Unit testing a plugin would be, for me, testing it in java, in isolation
> from the execution engine (ie. Maven).
>
> I know I'm not answering you question though... Still think that it's
> important we have a common terminology WRT plugin testing.
>
> -Vincent
>
> > -----Original Message-----
> > From: Kenney Westerhof [mailto:kenney@apache.org]
> > Sent: mardi 7 mars 2006 13:14
> > To: Maven Developers List
> > Subject: Re: plugin testing
> >
> > On Mon, 6 Mar 2006, Jesse McConnell wrote:
> >
> > Hi,
> >
> > (It took me a while to find the code - dev@maven and the heave use of
> > maven internals suggested it was in the maven sandbox :))
> >
> > I'm catching up to this thread, and I'm wondering what the exact goal
> > of this effort is.
> >
> > Specifically, wheter it is meant for unit tests, and wheter it is
> > possible to somehow merge this with the maven-it-plugin, binding
> > some test to the (pre-/ and post)integration-test phases. That way,
> > you can rely on the Embedder to do all the configuring and setting
> > up the environment, which really seems what the bulk of the
> > maven-plugin-testing-harness is trying to accomplish.
> >
> > Custom configuration of other project aspects like a local repository,
> > stubs and whatnot could be put in a mojo that runs prior to the
> > maven-it-plugin's main mojo (ForkMojo); the pre-integration-test phase
> can
> > be used to set up the environment (or integrated more in the
> > ForkMojo on a per-project basis), and the post-integration test phase
> > could contain the unit tests to validate the plugin execution.
> >
> > I'm just tossing this in, since it seems that this effort overlaps
> > with my initial ideas for the maven-it-plugin (basically,
> > running unit tests for plugins withing a maven environment). I'd like
> > to understand why this approach was taken (philosophy mismatch/the it
> > plugin still being in the sandbox? :))
> >
> > -- Kenney
> >
> >
> > > On 3/6/06, Brett Porter <br...@apache.org> wrote:
> > > >
> > > > Jason van Zyl wrote:
> > > > > This was just the first stage. I just whacked in the DOM reader
> > because
> > > > > we need some more testing code to create a local repository for
> the
> > > > > MavenProjectBuilder. The MavenProjectBuilder should be used.
> > > >
> > > > A stub of the maven project builder, or the real one? The real one
> > uses
> > > > the repo, and that's not really conducive to unit testing.
> > >
> > >
> > >
> > > the repo will need to be real at some point, some plugins needs to
> > interact
> > > with the real thing, for getting jars for packaging or for just
> looking
> > into
> > > for resources or something..so the artifact repo seems to be a
> required
> > part
> > > of actual unit tests..
> > >
> > >
> > > > Why? I don't think you should have to add methods in order to be
> able
> > to
> > > > > test the mojos. It's not how they would ever be used, so you're
> not
> > > > > really testing how they would be used in the field by adding
> > properties.
> > > >
> > > > At one point, we were doing away with private field injection and
> > > > recommending setters, so it is how they'd be used.
> > > >
> > > > >
> > > > > I think the POM style testing will be most familiar to people
> > writing
> > > > > plugins and ultimately allows for completely declarative testing
> of
> > > > > Mojos which I think would be excellent. In almost all cases you
> have
> > to
> > > > > execute the mojo in order to test it and in order to execute it
> you
> > need
> > > > > to populate the configuration elements. You could very clearly
> > outline
> > > > > different test scenerios using different POMs. You could do this
> in
> > Java
> > > > > but does it really matter if the inputs come via Java or another
> > > > > mechanism? I honestly don't think so.
> > > >
> > > > No, neither do I. I said having both is a good idea, but that
> calling
> > it
> > > > a POM is misleading, and to cut it back to just a configuration.
> > >
> > >
> > > simple, I'll take care of this today... <plugin>-config.xml maybe?
> > >
> > > >
> > > > > At any rate I don't think you should alter classes for testing.
> > > > > Especially in this case as we know Mojos work just fine without
> > > > properties.
> > > >
> > > > But they *are* properties, it's just a matter of whether it is
> > injected
> > > > using private fields, setters or constructors. Adding getters is
> > > > probably a bit more lame, but I think that's unavoidable in either
> > case?
> > >
> > >
> > > I think we can get away with all this without actually needing to add
> > > getters and setters, personally I think adding setters somewhat clouds
> > the
> > > interaction with mojo's, have a combo of setters and private field
> > injection
> > > seems prone to some kind of abuse
> > >
> > >
> > > > I think the Artifact/MavenProject population could use a little
> > tweaking
> > > > > but we could probably do that with a special expression evaluator.
> > > >
> > > > Cool. I have to look at the actual code yet...
> > > >
> > > > > I think people writing tests will like this declarative method of
> > > > > writing tests. It will be less error prone and take less effort
> and
> > > > > provide a standard way for writing Mojo tests.
> > > >
> > > > In some cases, but when you are only setting one field, its a pain,
> > and
> > > > when you are doing the same thing over and over and changing one
> > field,
> > > > it's a pain (though you could combine the two techniques for that).
> > >
> > >
> > > I was getting visions of people just making one test pom and either
> > reusing
> > > or copying and modifying a bit.
> > >
> > >
> > > > I think we can easily provide some Java code for those that want to
> do
> > > > > the Java thing but I imagine the desire to do that would fade
> given
> > the
> > > > > declarative option. But you still shouldn't have to add properties
> > to a
> > > > > mojo to make that work unless that's what you wanted to do to make
> > it
> > > > > reusable.
> > > >
> > > > As long as the assertions are still in Java, I don't think its a
> > > > declarative option, and I'd prefer to have my pre-conditions next to
> > the
> > > > post-conditions to make the tests more readable, myself.
> > >
> > >
> > >
> > > I agree with you here, it is disjointed to have the configuration and
> > the
> > > asserts in another place...but if we are going to route of a new file
> > for
> > > configuring these things, how about just building out an assert setup
> in
> > the
> > > configuration xml?
> > >
> > > <test>
> > > <configuration>
> > > ...
> > > </configuration>
> > > <asserts>
> > > <assertTrue>expression</
> > > ..
> > > ..
> > >
> > > The expression could be fed into the test case somehow...then they are
> > bound
> > > together
> > >
> > > jesse
> > >
> > > --
> > > jesse mcconnell
> > > jesseDOTmcconnellATgmailDOTcom
> > >
> >
> > --
> > Kenney Westerhof
> > http://www.neonics.com
> > GPG public key: http://www.gods.nl/~forge/kenneyw.key
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > For additional commands, e-mail: dev-help@maven.apache.org
>
>
>
>
>
>
>
> ___________________________________________________________________________
> Nouveau : téléphonez moins cher avec Yahoo! Messenger ! Découvez les
> tarifs exceptionnels pour appeler la France et l'international.
> Téléchargez sur http://fr.messenger.yahoo.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
--
jesse mcconnell
jesseDOTmcconnellATgmailDOTcom
RE: plugin testing
Posted by Vincent Massol <vm...@pivolis.com>.
I guess the terminology is still an issue but what the it plugin does and
what Jesse is implementing are for me integration tests and not unit tests.
Unit testing a plugin would be, for me, testing it in java, in isolation
from the execution engine (ie. Maven).
I know I'm not answering you question though... Still think that it's
important we have a common terminology WRT plugin testing.
-Vincent
> -----Original Message-----
> From: Kenney Westerhof [mailto:kenney@apache.org]
> Sent: mardi 7 mars 2006 13:14
> To: Maven Developers List
> Subject: Re: plugin testing
>
> On Mon, 6 Mar 2006, Jesse McConnell wrote:
>
> Hi,
>
> (It took me a while to find the code - dev@maven and the heave use of
> maven internals suggested it was in the maven sandbox :))
>
> I'm catching up to this thread, and I'm wondering what the exact goal
> of this effort is.
>
> Specifically, wheter it is meant for unit tests, and wheter it is
> possible to somehow merge this with the maven-it-plugin, binding
> some test to the (pre-/ and post)integration-test phases. That way,
> you can rely on the Embedder to do all the configuring and setting
> up the environment, which really seems what the bulk of the
> maven-plugin-testing-harness is trying to accomplish.
>
> Custom configuration of other project aspects like a local repository,
> stubs and whatnot could be put in a mojo that runs prior to the
> maven-it-plugin's main mojo (ForkMojo); the pre-integration-test phase can
> be used to set up the environment (or integrated more in the
> ForkMojo on a per-project basis), and the post-integration test phase
> could contain the unit tests to validate the plugin execution.
>
> I'm just tossing this in, since it seems that this effort overlaps
> with my initial ideas for the maven-it-plugin (basically,
> running unit tests for plugins withing a maven environment). I'd like
> to understand why this approach was taken (philosophy mismatch/the it
> plugin still being in the sandbox? :))
>
> -- Kenney
>
>
> > On 3/6/06, Brett Porter <br...@apache.org> wrote:
> > >
> > > Jason van Zyl wrote:
> > > > This was just the first stage. I just whacked in the DOM reader
> because
> > > > we need some more testing code to create a local repository for the
> > > > MavenProjectBuilder. The MavenProjectBuilder should be used.
> > >
> > > A stub of the maven project builder, or the real one? The real one
> uses
> > > the repo, and that's not really conducive to unit testing.
> >
> >
> >
> > the repo will need to be real at some point, some plugins needs to
> interact
> > with the real thing, for getting jars for packaging or for just looking
> into
> > for resources or something..so the artifact repo seems to be a required
> part
> > of actual unit tests..
> >
> >
> > > Why? I don't think you should have to add methods in order to be able
> to
> > > > test the mojos. It's not how they would ever be used, so you're not
> > > > really testing how they would be used in the field by adding
> properties.
> > >
> > > At one point, we were doing away with private field injection and
> > > recommending setters, so it is how they'd be used.
> > >
> > > >
> > > > I think the POM style testing will be most familiar to people
> writing
> > > > plugins and ultimately allows for completely declarative testing of
> > > > Mojos which I think would be excellent. In almost all cases you have
> to
> > > > execute the mojo in order to test it and in order to execute it you
> need
> > > > to populate the configuration elements. You could very clearly
> outline
> > > > different test scenerios using different POMs. You could do this in
> Java
> > > > but does it really matter if the inputs come via Java or another
> > > > mechanism? I honestly don't think so.
> > >
> > > No, neither do I. I said having both is a good idea, but that calling
> it
> > > a POM is misleading, and to cut it back to just a configuration.
> >
> >
> > simple, I'll take care of this today... <plugin>-config.xml maybe?
> >
> > >
> > > > At any rate I don't think you should alter classes for testing.
> > > > Especially in this case as we know Mojos work just fine without
> > > properties.
> > >
> > > But they *are* properties, it's just a matter of whether it is
> injected
> > > using private fields, setters or constructors. Adding getters is
> > > probably a bit more lame, but I think that's unavoidable in either
> case?
> >
> >
> > I think we can get away with all this without actually needing to add
> > getters and setters, personally I think adding setters somewhat clouds
> the
> > interaction with mojo's, have a combo of setters and private field
> injection
> > seems prone to some kind of abuse
> >
> >
> > > I think the Artifact/MavenProject population could use a little
> tweaking
> > > > but we could probably do that with a special expression evaluator.
> > >
> > > Cool. I have to look at the actual code yet...
> > >
> > > > I think people writing tests will like this declarative method of
> > > > writing tests. It will be less error prone and take less effort and
> > > > provide a standard way for writing Mojo tests.
> > >
> > > In some cases, but when you are only setting one field, its a pain,
> and
> > > when you are doing the same thing over and over and changing one
> field,
> > > it's a pain (though you could combine the two techniques for that).
> >
> >
> > I was getting visions of people just making one test pom and either
> reusing
> > or copying and modifying a bit.
> >
> >
> > > I think we can easily provide some Java code for those that want to do
> > > > the Java thing but I imagine the desire to do that would fade given
> the
> > > > declarative option. But you still shouldn't have to add properties
> to a
> > > > mojo to make that work unless that's what you wanted to do to make
> it
> > > > reusable.
> > >
> > > As long as the assertions are still in Java, I don't think its a
> > > declarative option, and I'd prefer to have my pre-conditions next to
> the
> > > post-conditions to make the tests more readable, myself.
> >
> >
> >
> > I agree with you here, it is disjointed to have the configuration and
> the
> > asserts in another place...but if we are going to route of a new file
> for
> > configuring these things, how about just building out an assert setup in
> the
> > configuration xml?
> >
> > <test>
> > <configuration>
> > ...
> > </configuration>
> > <asserts>
> > <assertTrue>expression</
> > ..
> > ..
> >
> > The expression could be fed into the test case somehow...then they are
> bound
> > together
> >
> > jesse
> >
> > --
> > jesse mcconnell
> > jesseDOTmcconnellATgmailDOTcom
> >
>
> --
> Kenney Westerhof
> http://www.neonics.com
> GPG public key: http://www.gods.nl/~forge/kenneyw.key
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
___________________________________________________________________________
Nouveau : t�l�phonez moins cher avec Yahoo! Messenger ! D�couvez les tarifs exceptionnels pour appeler la France et l'international.
T�l�chargez sur http://fr.messenger.yahoo.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: plugin testing
Posted by Kenney Westerhof <ke...@apache.org>.
On Mon, 6 Mar 2006, Jesse McConnell wrote:
Hi,
(It took me a while to find the code - dev@maven and the heave use of
maven internals suggested it was in the maven sandbox :))
I'm catching up to this thread, and I'm wondering what the exact goal
of this effort is.
Specifically, wheter it is meant for unit tests, and wheter it is
possible to somehow merge this with the maven-it-plugin, binding
some test to the (pre-/ and post)integration-test phases. That way,
you can rely on the Embedder to do all the configuring and setting
up the environment, which really seems what the bulk of the
maven-plugin-testing-harness is trying to accomplish.
Custom configuration of other project aspects like a local repository,
stubs and whatnot could be put in a mojo that runs prior to the
maven-it-plugin's main mojo (ForkMojo); the pre-integration-test phase can
be used to set up the environment (or integrated more in the
ForkMojo on a per-project basis), and the post-integration test phase
could contain the unit tests to validate the plugin execution.
I'm just tossing this in, since it seems that this effort overlaps
with my initial ideas for the maven-it-plugin (basically,
running unit tests for plugins withing a maven environment). I'd like
to understand why this approach was taken (philosophy mismatch/the it
plugin still being in the sandbox? :))
-- Kenney
> On 3/6/06, Brett Porter <br...@apache.org> wrote:
> >
> > Jason van Zyl wrote:
> > > This was just the first stage. I just whacked in the DOM reader because
> > > we need some more testing code to create a local repository for the
> > > MavenProjectBuilder. The MavenProjectBuilder should be used.
> >
> > A stub of the maven project builder, or the real one? The real one uses
> > the repo, and that's not really conducive to unit testing.
>
>
>
> the repo will need to be real at some point, some plugins needs to interact
> with the real thing, for getting jars for packaging or for just looking into
> for resources or something..so the artifact repo seems to be a required part
> of actual unit tests..
>
>
> > Why? I don't think you should have to add methods in order to be able to
> > > test the mojos. It's not how they would ever be used, so you're not
> > > really testing how they would be used in the field by adding properties.
> >
> > At one point, we were doing away with private field injection and
> > recommending setters, so it is how they'd be used.
> >
> > >
> > > I think the POM style testing will be most familiar to people writing
> > > plugins and ultimately allows for completely declarative testing of
> > > Mojos which I think would be excellent. In almost all cases you have to
> > > execute the mojo in order to test it and in order to execute it you need
> > > to populate the configuration elements. You could very clearly outline
> > > different test scenerios using different POMs. You could do this in Java
> > > but does it really matter if the inputs come via Java or another
> > > mechanism? I honestly don't think so.
> >
> > No, neither do I. I said having both is a good idea, but that calling it
> > a POM is misleading, and to cut it back to just a configuration.
>
>
> simple, I'll take care of this today... <plugin>-config.xml maybe?
>
> >
> > > At any rate I don't think you should alter classes for testing.
> > > Especially in this case as we know Mojos work just fine without
> > properties.
> >
> > But they *are* properties, it's just a matter of whether it is injected
> > using private fields, setters or constructors. Adding getters is
> > probably a bit more lame, but I think that's unavoidable in either case?
>
>
> I think we can get away with all this without actually needing to add
> getters and setters, personally I think adding setters somewhat clouds the
> interaction with mojo's, have a combo of setters and private field injection
> seems prone to some kind of abuse
>
>
> > I think the Artifact/MavenProject population could use a little tweaking
> > > but we could probably do that with a special expression evaluator.
> >
> > Cool. I have to look at the actual code yet...
> >
> > > I think people writing tests will like this declarative method of
> > > writing tests. It will be less error prone and take less effort and
> > > provide a standard way for writing Mojo tests.
> >
> > In some cases, but when you are only setting one field, its a pain, and
> > when you are doing the same thing over and over and changing one field,
> > it's a pain (though you could combine the two techniques for that).
>
>
> I was getting visions of people just making one test pom and either reusing
> or copying and modifying a bit.
>
>
> > I think we can easily provide some Java code for those that want to do
> > > the Java thing but I imagine the desire to do that would fade given the
> > > declarative option. But you still shouldn't have to add properties to a
> > > mojo to make that work unless that's what you wanted to do to make it
> > > reusable.
> >
> > As long as the assertions are still in Java, I don't think its a
> > declarative option, and I'd prefer to have my pre-conditions next to the
> > post-conditions to make the tests more readable, myself.
>
>
>
> I agree with you here, it is disjointed to have the configuration and the
> asserts in another place...but if we are going to route of a new file for
> configuring these things, how about just building out an assert setup in the
> configuration xml?
>
> <test>
> <configuration>
> ...
> </configuration>
> <asserts>
> <assertTrue>expression</
> ..
> ..
>
> The expression could be fed into the test case somehow...then they are bound
> together
>
> jesse
>
> --
> jesse mcconnell
> jesseDOTmcconnellATgmailDOTcom
>
--
Kenney Westerhof
http://www.neonics.com
GPG public key: http://www.gods.nl/~forge/kenneyw.key
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: plugin testing
Posted by Jesse McConnell <je...@gmail.com>.
On 3/6/06, Brett Porter <br...@apache.org> wrote:
>
> Jason van Zyl wrote:
> > This was just the first stage. I just whacked in the DOM reader because
> > we need some more testing code to create a local repository for the
> > MavenProjectBuilder. The MavenProjectBuilder should be used.
>
> A stub of the maven project builder, or the real one? The real one uses
> the repo, and that's not really conducive to unit testing.
the repo will need to be real at some point, some plugins needs to interact
with the real thing, for getting jars for packaging or for just looking into
for resources or something..so the artifact repo seems to be a required part
of actual unit tests..
> Why? I don't think you should have to add methods in order to be able to
> > test the mojos. It's not how they would ever be used, so you're not
> > really testing how they would be used in the field by adding properties.
>
> At one point, we were doing away with private field injection and
> recommending setters, so it is how they'd be used.
>
> >
> > I think the POM style testing will be most familiar to people writing
> > plugins and ultimately allows for completely declarative testing of
> > Mojos which I think would be excellent. In almost all cases you have to
> > execute the mojo in order to test it and in order to execute it you need
> > to populate the configuration elements. You could very clearly outline
> > different test scenerios using different POMs. You could do this in Java
> > but does it really matter if the inputs come via Java or another
> > mechanism? I honestly don't think so.
>
> No, neither do I. I said having both is a good idea, but that calling it
> a POM is misleading, and to cut it back to just a configuration.
simple, I'll take care of this today... <plugin>-config.xml maybe?
>
> > At any rate I don't think you should alter classes for testing.
> > Especially in this case as we know Mojos work just fine without
> properties.
>
> But they *are* properties, it's just a matter of whether it is injected
> using private fields, setters or constructors. Adding getters is
> probably a bit more lame, but I think that's unavoidable in either case?
I think we can get away with all this without actually needing to add
getters and setters, personally I think adding setters somewhat clouds the
interaction with mojo's, have a combo of setters and private field injection
seems prone to some kind of abuse
> I think the Artifact/MavenProject population could use a little tweaking
> > but we could probably do that with a special expression evaluator.
>
> Cool. I have to look at the actual code yet...
>
> > I think people writing tests will like this declarative method of
> > writing tests. It will be less error prone and take less effort and
> > provide a standard way for writing Mojo tests.
>
> In some cases, but when you are only setting one field, its a pain, and
> when you are doing the same thing over and over and changing one field,
> it's a pain (though you could combine the two techniques for that).
I was getting visions of people just making one test pom and either reusing
or copying and modifying a bit.
> I think we can easily provide some Java code for those that want to do
> > the Java thing but I imagine the desire to do that would fade given the
> > declarative option. But you still shouldn't have to add properties to a
> > mojo to make that work unless that's what you wanted to do to make it
> > reusable.
>
> As long as the assertions are still in Java, I don't think its a
> declarative option, and I'd prefer to have my pre-conditions next to the
> post-conditions to make the tests more readable, myself.
I agree with you here, it is disjointed to have the configuration and the
asserts in another place...but if we are going to route of a new file for
configuring these things, how about just building out an assert setup in the
configuration xml?
<test>
<configuration>
...
</configuration>
<asserts>
<assertTrue>expression</
..
..
The expression could be fed into the test case somehow...then they are bound
together
jesse
--
jesse mcconnell
jesseDOTmcconnellATgmailDOTcom
Re: plugin testing
Posted by Jason van Zyl <ja...@maven.org>.
Brett Porter wrote:
> Jason van Zyl wrote:
>> This was just the first stage. I just whacked in the DOM reader because
>> we need some more testing code to create a local repository for the
>> MavenProjectBuilder. The MavenProjectBuilder should be used.
>
> A stub of the maven project builder, or the real one? The real one uses
> the repo, and that's not really conducive to unit testing.
The harness somewhat blurs the lines but once you execute the Mojo I
don't really consider it a unit test anymore. Even with the simple clean
plugin example that you started with executed the Mojo. I think that's
what's useful for testing and many Mojos need a local repository so I
was suggesting using the real project builder passing in a local
repository created for testing purposes.
>> Why? I don't think you should have to add methods in order to be able to
>> test the mojos. It's not how they would ever be used, so you're not
>> really testing how they would be used in the field by adding properties.
>
> At one point, we were doing away with private field injection and
> recommending setters, so it is how they'd be used.
Hasn't happened yet. Not sure if that will happen as I'm not sure how
many Ant tasks are actually used outside of Ant. Given what we do in
promoting the creation of components that get wrapped by a Mojo to
expose them in Maven I'm not sure if there's much point to properties.
If that's what we decide then we can do that. I'm not fussed one way or
the other.
>> I think the POM style testing will be most familiar to people writing
>> plugins and ultimately allows for completely declarative testing of
>> Mojos which I think would be excellent. In almost all cases you have to
>> execute the mojo in order to test it and in order to execute it you need
>> to populate the configuration elements. You could very clearly outline
>> different test scenerios using different POMs. You could do this in Java
>> but does it really matter if the inputs come via Java or another
>> mechanism? I honestly don't think so.
>
> No, neither do I. I said having both is a good idea, but that calling it
> a POM is misleading, and to cut it back to just a configuration.
But there are plugins that require a MavenProject and other parts of the
POM not in the configuration for the plugin.
The IDEA plugin is one example that needs a POM so having the pom.xml
file actually be the source of a MavenProject I think would work. There
is also the resources plugin which requires elements in the <build/>
element. I don't think we'll get away with just the <configuration/>
element for the plugin.
>> At any rate I don't think you should alter classes for testing.
>> Especially in this case as we know Mojos work just fine without properties.
>
> But they *are* properties, it's just a matter of whether it is injected
> using private fields, setters or constructors. Adding getters is
> probably a bit more lame, but I think that's unavoidable in either case.
We've had this discussion before :-) They are not properties. We happen
to abuse the field name being the same as the configuration element. But
with a property the configuration element can be different then the
field name as the setter would intervene.
>> I think the Artifact/MavenProject population could use a little tweaking
>> but we could probably do that with a special expression evaluator.
>
> Cool. I have to look at the actual code yet...
>
>> I think people writing tests will like this declarative method of
>> writing tests. It will be less error prone and take less effort and
>> provide a standard way for writing Mojo tests.
>
> In some cases, but when you are only setting one field, its a pain, and
> when you are doing the same thing over and over and changing one field,
> it's a pain (though you could combine the two techniques for that).
In how many cases in a Mojo can you only set one field and actually make
it do anything? I'm all for a Java way, but I think the pom.xml will be
the most thorough and actually correspond most accurately to real use.
Whatever kind of testing that actually is.
>> I think we can easily provide some Java code for those that want to do
>> the Java thing but I imagine the desire to do that would fade given the
>> declarative option. But you still shouldn't have to add properties to a
>> mojo to make that work unless that's what you wanted to do to make it
>> reusable.
>
> As long as the assertions are still in Java, I don't think its a
> declarative option, and I'd prefer to have my pre-conditions next to the
> post-conditions to make the tests more readable, myself.
>
> But, we can do both, and learn from experience which works, right?
In SuiteRunner the input and assertions were in the same file. Sure, it
can't hurt to have both methods and I think we need a method in Java
simply because we will probably find holes in the declarative structure
and that shouldn't stop you from testing.
> - Brett
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: plugin testing
Posted by Brett Porter <br...@apache.org>.
Jason van Zyl wrote:
> This was just the first stage. I just whacked in the DOM reader because
> we need some more testing code to create a local repository for the
> MavenProjectBuilder. The MavenProjectBuilder should be used.
A stub of the maven project builder, or the real one? The real one uses
the repo, and that's not really conducive to unit testing.
> Why? I don't think you should have to add methods in order to be able to
> test the mojos. It's not how they would ever be used, so you're not
> really testing how they would be used in the field by adding properties.
At one point, we were doing away with private field injection and
recommending setters, so it is how they'd be used.
>
> I think the POM style testing will be most familiar to people writing
> plugins and ultimately allows for completely declarative testing of
> Mojos which I think would be excellent. In almost all cases you have to
> execute the mojo in order to test it and in order to execute it you need
> to populate the configuration elements. You could very clearly outline
> different test scenerios using different POMs. You could do this in Java
> but does it really matter if the inputs come via Java or another
> mechanism? I honestly don't think so.
No, neither do I. I said having both is a good idea, but that calling it
a POM is misleading, and to cut it back to just a configuration.
>
> At any rate I don't think you should alter classes for testing.
> Especially in this case as we know Mojos work just fine without properties.
But they *are* properties, it's just a matter of whether it is injected
using private fields, setters or constructors. Adding getters is
probably a bit more lame, but I think that's unavoidable in either case.
> I think the Artifact/MavenProject population could use a little tweaking
> but we could probably do that with a special expression evaluator.
Cool. I have to look at the actual code yet...
> I think people writing tests will like this declarative method of
> writing tests. It will be less error prone and take less effort and
> provide a standard way for writing Mojo tests.
In some cases, but when you are only setting one field, its a pain, and
when you are doing the same thing over and over and changing one field,
it's a pain (though you could combine the two techniques for that).
> I think we can easily provide some Java code for those that want to do
> the Java thing but I imagine the desire to do that would fade given the
> declarative option. But you still shouldn't have to add properties to a
> mojo to make that work unless that's what you wanted to do to make it
> reusable.
As long as the assertions are still in Java, I don't think its a
declarative option, and I'd prefer to have my pre-conditions next to the
post-conditions to make the tests more readable, myself.
But, we can do both, and learn from experience which works, right?
- Brett
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: plugin testing
Posted by Jason van Zyl <ja...@maven.org>.
Brett Porter wrote:
> Just some things for the record:
>
> - I don't really like the <artifact>...</artifact> element here as it
> isn't normal to a POM.
> - The POM doesn't do the full inheritence thing, so it might not behave
> as expected. This could be confusing
>
> I think if the file is not really a POM, it shouldn't be construed as
> one for the sake of clarity. Maybe we can just use the normal Xpp3Dom
> configuration lookup, and that can be created from xpp3 dom builder from
> a file with just the config element.
This was just the first stage. I just whacked in the DOM reader because
we need some more testing code to create a local repository for the
MavenProjectBuilder. The MavenProjectBuilder should be used.
> I assume if there are project elements to be populated that are not part
> of a plugin, they would be set on the stub project housed in the stub
> expression evaluator, which could be done in java code?
>
> Also:
> - I think we should be adding getters/setters and being able to
> construct a normal mojo and set fields by hand (components and
> expressions should still be prepopulated in the lookup).
Why? I don't think you should have to add methods in order to be able to
test the mojos. It's not how they would ever be used, so you're not
really testing how they would be used in the field by adding properties.
I think the POM style testing will be most familiar to people writing
plugins and ultimately allows for completely declarative testing of
Mojos which I think would be excellent. In almost all cases you have to
execute the mojo in order to test it and in order to execute it you need
to populate the configuration elements. You could very clearly outline
different test scenerios using different POMs. You could do this in Java
but does it really matter if the inputs come via Java or another
mechanism? I honestly don't think so.
At any rate I don't think you should alter classes for testing.
Especially in this case as we know Mojos work just fine without properties.
I think the Artifact/MavenProject population could use a little tweaking
but we could probably do that with a special expression evaluator.
I think people writing tests will like this declarative method of
writing tests. It will be less error prone and take less effort and
provide a standard way for writing Mojo tests.
That was what I thought was beautiful about SuiteRunner, the same
notions I pushed into Surefire, where test invocation can come from
anywhere: Java, XML, text files. The same idea used in Fitnesse where
users can write tests for webapps using simple markup.
I think we can easily provide some Java code for those that want to do
the Java thing but I imagine the desire to do that would fade given the
declarative option. But you still shouldn't have to add properties to a
mojo to make that work unless that's what you wanted to do to make it
reusable.
> It seems like most of this is in place, just issues of clarity and
> perhaps preference.
>
> Thanks again!
>
> - Brett
>
> Jesse McConnell wrote:
>> we can now inject StubArtifacts into the mojo's :)
>>
>> <project>
>> <build>
>> <plugins>
>> <plugin>
>> <artifactId>maven-compiler-plugin</artifactId>
>> <configuration>
>> <compileSourceRoots>
>> <compileSourceRoot>src/main/java</compileSourceRoot>
>> </compileSourceRoots>
>> <compilerId>javac</compilerId>
>> <outputDirectory>target/classes</outputDirectory>
>> <artifact>org.apache.maven.artifact.Artifact</artifact>
>> </configuration>
>> </plugin>
>> </plugins>
>> </build>
>> </project>
>>
>> for the time being I am sticking all of the Stub* maven artifacts in the
>> maven-plugin-testing-harness and they are all referenced in the
>> components.xml file in the same project. I am not sure if we want them to
>> ultimately reside next to the Default* implementations or not...anyone have
>> thoughts on that? I could go either way on that.
>>
>> I am pretty however pretty happy with the way this is fleshing out :)
>>
>> very useful and by the time more unit tests are written we should have a
>> pretty decent set of stub's for mojo's to work with.
>>
>> Not everything will have to be a stub either here, jason is working on the
>> ArtifactRepository and aggregating some things together but I can see the
>> need where we are going to need a real one or more of those since some
>> plugins make use of it...the axistools plugin for one :)
>>
>> cheers,
>> jesse
>>
>> On 2/23/06, Jesse McConnell <je...@gmail.com> wrote:
>>> updating this thread again
>>>
>>> Jason and I talked over the above idea and he took a few minutes and put
>>> together the basic jist of the concept and shoved it into the mojo-sandbox
>>> so I could work with it.
>>>
>>> mojo-sandbox/maven-plugin-testing/maven-plugin-testing-harness
>>>
>>> that has version 1.0-SNAPSHOT of the AbstractMojoTestCase which is what
>>> the test cases can subclass from to get the ability to dynamically load the
>>> mojo based on an arbitary pom.
>>>
>>> I modified the maven-clean-plugin again to make use of this and it turned
>>> out quite well I'd say.
>>>
>>> /**
>>> * tests the ability of the plugin to clean out a set of directories
>>> *
>>> * @throws Exception
>>> */
>>> public void testClean() throws Exception {
>>>
>>> CleanMojo mojo = (CleanMojo) lookupMojo ("clean", testPom );
>>>
>>> assertNotNull( mojo );
>>>
>>> mojo.execute();
>>>
>>> assertFalse ( FileUtils.fileExists(
>>> "target/test/testDirectoryStructure/buildDirectory" ) );
>>> assertFalse ( FileUtils.fileExists(
>>> "target/test/testDirectoryStructure/buildOutputDirectory" ) );
>>> assertFalse ( FileUtils.fileExists(
>>> "target/test/testDirectoryStructure/buildTestDirectory" ) );
>>> }
>>>
>>>
>>> This method in the CleanMojoTest class coupled with this pom.xml @
>>> src/test/resources/unit/clean/pom.xml execute the unit test very easily.
>>>
>>> <project>
>>> <build>
>>> <plugins>
>>> <plugin>
>>> <artifactId>maven-clean-plugin</artifactId>
>>> <configuration>
>>>
>>> <directory>target/test/testDirectoryStructure/buildDirectory</directory>
>>>
>>> <outputDirectory>target/test/testDirectoryStructure/buildOutputDirectory</outputDirectory>
>>>
>>>
>>> <testOutputDirectory>target/test/testDirectoryStructure/buildTestDirectory</testOutputDirectory>
>>> </configuration>
>>> </plugin>
>>> </plugins>
>>> </build>
>>> </project>
>>>
>>> I am going to take this approach and work on testing another plugin in
>>> maven now, get a couple of plugins under my belt with this and the abstract
>>> base class ought to be pretty tight.
>>>
>>> jesse
>>>
>>> --
>>> jesse mcconnell
>>> jesseDOTmcconnellATgmailDOTcom
>>
>>
>>
>> --
>> jesse mcconnell
>> jesseDOTmcconnellATgmailDOTcom
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: plugin testing
Posted by Jesse McConnell <je...@gmail.com>.
works for things like the artifact, but MavenProject isn't a component and I
can't use a role lookup on it, that was my first direction where I used
components.xml for "stub" role-hint versions of things..
${basedir} works the same as before, I ended up pulling that part out and
putting it into the StubExpressionEvaluator that I made since we don't have
project instances and things to initialize the maven expression evaluator
with.
jesse
On 3/6/06, Brett Porter <br...@apache.org> wrote:
>
> Cool. so I was thinking we could drop all the wrapping elements to make
> it:
>
> Jesse McConnell wrote:
> > <configuration>
> > <compileSourceRoots>
> >
> >
> <compileSourceRoot>{$basedir}target/classes/unit/compiler-basic-test/src/main/java</compileSourceRoot>
> > </compileSourceRoots>
> > <compilerId>javac</compilerId>
> >
> >
> <outputDirectory>{$basedir}/target/test/unit/compiler-basic-test/target</outputDirectory>
> > <artifact implementation="
> > org.apache.maven.plugin.testing.stubs.StubArtifact"/>
> > </configuration>
>
> This raises another question:
>
> > <artifact implementation="
> > org.apache.maven.plugin.testing.stubs.StubArtifact"/>
>
> Do these have default implementations when the harness is in play?
> Trying to think if that is even possible...
>
> Having extended PlexusTestCase, you can already do that by:
>
> TestCaseName.xml:
>
> [...]
> <component>
> <role>o.a.m.a.Artifact</role>
> <implementation>o.a.m.p.t.s.StubArtifact</implemnetation>
> </component>
> [...]
>
> Not sure I really prefer that, of course. Still like to write Java
> tests, but alternatives are good :)
>
> Also, I assume its ${basedir} and you haven't changed the behaviour of
> the expression eval? :)
>
> Cheers,
> Brett
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
--
jesse mcconnell
jesseDOTmcconnellATgmailDOTcom
Re: plugin testing
Posted by Brett Porter <br...@apache.org>.
Cool. so I was thinking we could drop all the wrapping elements to make it:
Jesse McConnell wrote:
> <configuration>
> <compileSourceRoots>
>
> <compileSourceRoot>{$basedir}target/classes/unit/compiler-basic-test/src/main/java</compileSourceRoot>
> </compileSourceRoots>
> <compilerId>javac</compilerId>
>
> <outputDirectory>{$basedir}/target/test/unit/compiler-basic-test/target</outputDirectory>
> <artifact implementation="
> org.apache.maven.plugin.testing.stubs.StubArtifact"/>
> </configuration>
This raises another question:
> <artifact implementation="
> org.apache.maven.plugin.testing.stubs.StubArtifact"/>
Do these have default implementations when the harness is in play?
Trying to think if that is even possible...
Having extended PlexusTestCase, you can already do that by:
TestCaseName.xml:
[...]
<component>
<role>o.a.m.a.Artifact</role>
<implementation>o.a.m.p.t.s.StubArtifact</implemnetation>
</component>
[...]
Not sure I really prefer that, of course. Still like to write Java
tests, but alternatives are good :)
Also, I assume its ${basedir} and you haven't changed the behaviour of
the expression eval? :)
Cheers,
Brett
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: plugin testing
Posted by Jesse McConnell <je...@gmail.com>.
<project>
<build>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<compileSourceRoots>
<compileSourceRoot>{$basedir}target/classes/unit/compiler-basic-test/src/main/java</compileSourceRoot>
</compileSourceRoots>
<compilerId>javac</compilerId>
<outputDirectory>{$basedir}/target/test/unit/compiler-basic-test/target</outputDirectory>
<artifact implementation="
org.apache.maven.plugin.testing.stubs.StubArtifact"/>
</configuration>
</plugin>
</plugins>
</build>
</project>
I should have sent this eariler, but this is actually what that test file is
looking like atm.
jesse
On 3/6/06, Brett Porter <br...@apache.org> wrote:
>
> Just some things for the record:
>
> - I don't really like the <artifact>...</artifact> element here as it
> isn't normal to a POM.
> - The POM doesn't do the full inheritence thing, so it might not behave
> as expected. This could be confusing
>
> I think if the file is not really a POM, it shouldn't be construed as
> one for the sake of clarity. Maybe we can just use the normal Xpp3Dom
> configuration lookup, and that can be created from xpp3 dom builder from
> a file with just the config element.
>
> I assume if there are project elements to be populated that are not part
> of a plugin, they would be set on the stub project housed in the stub
> expression evaluator, which could be done in java code?
>
> Also:
> - I think we should be adding getters/setters and being able to
> construct a normal mojo and set fields by hand (components and
> expressions should still be prepopulated in the lookup).
>
> It seems like most of this is in place, just issues of clarity and
> perhaps preference.
>
> Thanks again!
>
> - Brett
>
> Jesse McConnell wrote:
> > we can now inject StubArtifacts into the mojo's :)
> >
> > <project>
> > <build>
> > <plugins>
> > <plugin>
> > <artifactId>maven-compiler-plugin</artifactId>
> > <configuration>
> > <compileSourceRoots>
> > <compileSourceRoot>src/main/java</compileSourceRoot>
> > </compileSourceRoots>
> > <compilerId>javac</compilerId>
> > <outputDirectory>target/classes</outputDirectory>
> > <artifact>org.apache.maven.artifact.Artifact</artifact>
> > </configuration>
> > </plugin>
> > </plugins>
> > </build>
> > </project>
> >
> > for the time being I am sticking all of the Stub* maven artifacts in the
> > maven-plugin-testing-harness and they are all referenced in the
> > components.xml file in the same project. I am not sure if we want them
> to
> > ultimately reside next to the Default* implementations or not...anyone
> have
> > thoughts on that? I could go either way on that.
> >
> > I am pretty however pretty happy with the way this is fleshing out :)
> >
> > very useful and by the time more unit tests are written we should have a
> > pretty decent set of stub's for mojo's to work with.
> >
> > Not everything will have to be a stub either here, jason is working on
> the
> > ArtifactRepository and aggregating some things together but I can see
> the
> > need where we are going to need a real one or more of those since some
> > plugins make use of it...the axistools plugin for one :)
> >
> > cheers,
> > jesse
> >
> > On 2/23/06, Jesse McConnell <je...@gmail.com> wrote:
> >> updating this thread again
> >>
> >> Jason and I talked over the above idea and he took a few minutes and
> put
> >> together the basic jist of the concept and shoved it into the
> mojo-sandbox
> >> so I could work with it.
> >>
> >> mojo-sandbox/maven-plugin-testing/maven-plugin-testing-harness
> >>
> >> that has version 1.0-SNAPSHOT of the AbstractMojoTestCase which is what
> >> the test cases can subclass from to get the ability to dynamically load
> the
> >> mojo based on an arbitary pom.
> >>
> >> I modified the maven-clean-plugin again to make use of this and it
> turned
> >> out quite well I'd say.
> >>
> >> /**
> >> * tests the ability of the plugin to clean out a set of
> directories
> >> *
> >> * @throws Exception
> >> */
> >> public void testClean() throws Exception {
> >>
> >> CleanMojo mojo = (CleanMojo) lookupMojo ("clean", testPom );
> >>
> >> assertNotNull( mojo );
> >>
> >> mojo.execute();
> >>
> >> assertFalse ( FileUtils.fileExists(
> >> "target/test/testDirectoryStructure/buildDirectory" ) );
> >> assertFalse ( FileUtils.fileExists(
> >> "target/test/testDirectoryStructure/buildOutputDirectory" ) );
> >> assertFalse ( FileUtils.fileExists(
> >> "target/test/testDirectoryStructure/buildTestDirectory" ) );
> >> }
> >>
> >>
> >> This method in the CleanMojoTest class coupled with this pom.xml @
> >> src/test/resources/unit/clean/pom.xml execute the unit test very
> easily.
> >>
> >> <project>
> >> <build>
> >> <plugins>
> >> <plugin>
> >> <artifactId>maven-clean-plugin</artifactId>
> >> <configuration>
> >>
> >>
> <directory>target/test/testDirectoryStructure/buildDirectory</directory>
> >>
> >>
> <outputDirectory>target/test/testDirectoryStructure/buildOutputDirectory</outputDirectory>
> >>
> >>
> >>
> <testOutputDirectory>target/test/testDirectoryStructure/buildTestDirectory</testOutputDirectory>
> >> </configuration>
> >> </plugin>
> >> </plugins>
> >> </build>
> >> </project>
> >>
> >> I am going to take this approach and work on testing another plugin in
> >> maven now, get a couple of plugins under my belt with this and the
> abstract
> >> base class ought to be pretty tight.
> >>
> >> jesse
> >>
> >> --
> >> jesse mcconnell
> >> jesseDOTmcconnellATgmailDOTcom
> >
> >
> >
> >
> > --
> > jesse mcconnell
> > jesseDOTmcconnellATgmailDOTcom
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
--
jesse mcconnell
jesseDOTmcconnellATgmailDOTcom
Re: plugin testing
Posted by Brett Porter <br...@apache.org>.
Just some things for the record:
- I don't really like the <artifact>...</artifact> element here as it
isn't normal to a POM.
- The POM doesn't do the full inheritence thing, so it might not behave
as expected. This could be confusing
I think if the file is not really a POM, it shouldn't be construed as
one for the sake of clarity. Maybe we can just use the normal Xpp3Dom
configuration lookup, and that can be created from xpp3 dom builder from
a file with just the config element.
I assume if there are project elements to be populated that are not part
of a plugin, they would be set on the stub project housed in the stub
expression evaluator, which could be done in java code?
Also:
- I think we should be adding getters/setters and being able to
construct a normal mojo and set fields by hand (components and
expressions should still be prepopulated in the lookup).
It seems like most of this is in place, just issues of clarity and
perhaps preference.
Thanks again!
- Brett
Jesse McConnell wrote:
> we can now inject StubArtifacts into the mojo's :)
>
> <project>
> <build>
> <plugins>
> <plugin>
> <artifactId>maven-compiler-plugin</artifactId>
> <configuration>
> <compileSourceRoots>
> <compileSourceRoot>src/main/java</compileSourceRoot>
> </compileSourceRoots>
> <compilerId>javac</compilerId>
> <outputDirectory>target/classes</outputDirectory>
> <artifact>org.apache.maven.artifact.Artifact</artifact>
> </configuration>
> </plugin>
> </plugins>
> </build>
> </project>
>
> for the time being I am sticking all of the Stub* maven artifacts in the
> maven-plugin-testing-harness and they are all referenced in the
> components.xml file in the same project. I am not sure if we want them to
> ultimately reside next to the Default* implementations or not...anyone have
> thoughts on that? I could go either way on that.
>
> I am pretty however pretty happy with the way this is fleshing out :)
>
> very useful and by the time more unit tests are written we should have a
> pretty decent set of stub's for mojo's to work with.
>
> Not everything will have to be a stub either here, jason is working on the
> ArtifactRepository and aggregating some things together but I can see the
> need where we are going to need a real one or more of those since some
> plugins make use of it...the axistools plugin for one :)
>
> cheers,
> jesse
>
> On 2/23/06, Jesse McConnell <je...@gmail.com> wrote:
>> updating this thread again
>>
>> Jason and I talked over the above idea and he took a few minutes and put
>> together the basic jist of the concept and shoved it into the mojo-sandbox
>> so I could work with it.
>>
>> mojo-sandbox/maven-plugin-testing/maven-plugin-testing-harness
>>
>> that has version 1.0-SNAPSHOT of the AbstractMojoTestCase which is what
>> the test cases can subclass from to get the ability to dynamically load the
>> mojo based on an arbitary pom.
>>
>> I modified the maven-clean-plugin again to make use of this and it turned
>> out quite well I'd say.
>>
>> /**
>> * tests the ability of the plugin to clean out a set of directories
>> *
>> * @throws Exception
>> */
>> public void testClean() throws Exception {
>>
>> CleanMojo mojo = (CleanMojo) lookupMojo ("clean", testPom );
>>
>> assertNotNull( mojo );
>>
>> mojo.execute();
>>
>> assertFalse ( FileUtils.fileExists(
>> "target/test/testDirectoryStructure/buildDirectory" ) );
>> assertFalse ( FileUtils.fileExists(
>> "target/test/testDirectoryStructure/buildOutputDirectory" ) );
>> assertFalse ( FileUtils.fileExists(
>> "target/test/testDirectoryStructure/buildTestDirectory" ) );
>> }
>>
>>
>> This method in the CleanMojoTest class coupled with this pom.xml @
>> src/test/resources/unit/clean/pom.xml execute the unit test very easily.
>>
>> <project>
>> <build>
>> <plugins>
>> <plugin>
>> <artifactId>maven-clean-plugin</artifactId>
>> <configuration>
>>
>> <directory>target/test/testDirectoryStructure/buildDirectory</directory>
>>
>> <outputDirectory>target/test/testDirectoryStructure/buildOutputDirectory</outputDirectory>
>>
>>
>> <testOutputDirectory>target/test/testDirectoryStructure/buildTestDirectory</testOutputDirectory>
>> </configuration>
>> </plugin>
>> </plugins>
>> </build>
>> </project>
>>
>> I am going to take this approach and work on testing another plugin in
>> maven now, get a couple of plugins under my belt with this and the abstract
>> base class ought to be pretty tight.
>>
>> jesse
>>
>> --
>> jesse mcconnell
>> jesseDOTmcconnellATgmailDOTcom
>
>
>
>
> --
> jesse mcconnell
> jesseDOTmcconnellATgmailDOTcom
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: plugin testing
Posted by Jesse McConnell <je...@gmail.com>.
we can now inject StubArtifacts into the mojo's :)
<project>
<build>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<compileSourceRoots>
<compileSourceRoot>src/main/java</compileSourceRoot>
</compileSourceRoots>
<compilerId>javac</compilerId>
<outputDirectory>target/classes</outputDirectory>
<artifact>org.apache.maven.artifact.Artifact</artifact>
</configuration>
</plugin>
</plugins>
</build>
</project>
for the time being I am sticking all of the Stub* maven artifacts in the
maven-plugin-testing-harness and they are all referenced in the
components.xml file in the same project. I am not sure if we want them to
ultimately reside next to the Default* implementations or not...anyone have
thoughts on that? I could go either way on that.
I am pretty however pretty happy with the way this is fleshing out :)
very useful and by the time more unit tests are written we should have a
pretty decent set of stub's for mojo's to work with.
Not everything will have to be a stub either here, jason is working on the
ArtifactRepository and aggregating some things together but I can see the
need where we are going to need a real one or more of those since some
plugins make use of it...the axistools plugin for one :)
cheers,
jesse
On 2/23/06, Jesse McConnell <je...@gmail.com> wrote:
>
> updating this thread again
>
> Jason and I talked over the above idea and he took a few minutes and put
> together the basic jist of the concept and shoved it into the mojo-sandbox
> so I could work with it.
>
> mojo-sandbox/maven-plugin-testing/maven-plugin-testing-harness
>
> that has version 1.0-SNAPSHOT of the AbstractMojoTestCase which is what
> the test cases can subclass from to get the ability to dynamically load the
> mojo based on an arbitary pom.
>
> I modified the maven-clean-plugin again to make use of this and it turned
> out quite well I'd say.
>
> /**
> * tests the ability of the plugin to clean out a set of directories
> *
> * @throws Exception
> */
> public void testClean() throws Exception {
>
> CleanMojo mojo = (CleanMojo) lookupMojo ("clean", testPom );
>
> assertNotNull( mojo );
>
> mojo.execute();
>
> assertFalse ( FileUtils.fileExists(
> "target/test/testDirectoryStructure/buildDirectory" ) );
> assertFalse ( FileUtils.fileExists(
> "target/test/testDirectoryStructure/buildOutputDirectory" ) );
> assertFalse ( FileUtils.fileExists(
> "target/test/testDirectoryStructure/buildTestDirectory" ) );
> }
>
>
> This method in the CleanMojoTest class coupled with this pom.xml @
> src/test/resources/unit/clean/pom.xml execute the unit test very easily.
>
> <project>
> <build>
> <plugins>
> <plugin>
> <artifactId>maven-clean-plugin</artifactId>
> <configuration>
>
> <directory>target/test/testDirectoryStructure/buildDirectory</directory>
>
> <outputDirectory>target/test/testDirectoryStructure/buildOutputDirectory</outputDirectory>
>
>
> <testOutputDirectory>target/test/testDirectoryStructure/buildTestDirectory</testOutputDirectory>
> </configuration>
> </plugin>
> </plugins>
> </build>
> </project>
>
> I am going to take this approach and work on testing another plugin in
> maven now, get a couple of plugins under my belt with this and the abstract
> base class ought to be pretty tight.
>
> jesse
>
> --
> jesse mcconnell
> jesseDOTmcconnellATgmailDOTcom
--
jesse mcconnell
jesseDOTmcconnellATgmailDOTcom
Re: plugin testing
Posted by Jesse McConnell <je...@gmail.com>.
updating this thread again
Jason and I talked over the above idea and he took a few minutes and put
together the basic jist of the concept and shoved it into the mojo-sandbox
so I could work with it.
mojo-sandbox/maven-plugin-testing/maven-plugin-testing-harness
that has version 1.0-SNAPSHOT of the AbstractMojoTestCase which is what the
test cases can subclass from to get the ability to dynamically load the mojo
based on an arbitary pom.
I modified the maven-clean-plugin again to make use of this and it turned
out quite well I'd say.
/**
* tests the ability of the plugin to clean out a set of directories
*
* @throws Exception
*/
public void testClean() throws Exception {
CleanMojo mojo = (CleanMojo) lookupMojo ("clean", testPom );
assertNotNull( mojo );
mojo.execute();
assertFalse ( FileUtils.fileExists(
"target/test/testDirectoryStructure/buildDirectory" ) );
assertFalse ( FileUtils.fileExists(
"target/test/testDirectoryStructure/buildOutputDirectory" ) );
assertFalse ( FileUtils.fileExists(
"target/test/testDirectoryStructure/buildTestDirectory" ) );
}
This method in the CleanMojoTest class coupled with this pom.xml @
src/test/resources/unit/clean/pom.xml execute the unit test very easily.
<project>
<build>
<plugins>
<plugin>
<artifactId>maven-clean-plugin</artifactId>
<configuration>
<directory>target/test/testDirectoryStructure/buildDirectory</directory>
<outputDirectory>target/test/testDirectoryStructure/buildOutputDirectory</outputDirectory>
<testOutputDirectory>target/test/testDirectoryStructure/buildTestDirectory</testOutputDirectory>
</configuration>
</plugin>
</plugins>
</build>
</project>
I am going to take this approach and work on testing another plugin in maven
now, get a couple of plugins under my belt with this and the abstract base
class ought to be pretty tight.
jesse
--
jesse mcconnell
jesseDOTmcconnellATgmailDOTcom
Re: plugin testing
Posted by Jesse McConnell <je...@gmail.com>.
jason and I had a nice chat about this on #plexus today when I asked a
question about getting the right logger instance inside of an object built
out by lookup(ROLE)..
Here is the framework that we worked out...there are two lvls to it but in a
nutshell it is a nice approach I think, very much along the lines of what
brett was talking about.
First off we build out a AbstractMojoTestCase that is a child of
PlexusTestCase and in this we add a method that can be used like this:
MyMojo mojo = (MyMojo) lookup ( Mojo.ROLE, path_to_pom );
now, the way this works is that in the src/test/resources (or whatever)
directory we start building out a number of directories for different test
cases and in those directories we put pom.xml files that contain the plugin
declaration and the configuration parameters...
these poms can be used in one of two different ways:
1) they can be used to instantiate the mojo through the above lookup method
inside of java unit tests, giving you the fully instantiated mojo that you
can execute all of the unit tests you like
2) they can be iterated over in a testing battery where we can do
verification on the output of the plugin, a la the integration tests
the lookup() method above can use the default plugin manager for a lot of
the work I guess. Some of this gets into murky territory for me since I
haven't dug into that area of maven, but it all seems like it ought to work
pretty smooth.
We would probably want to split out the directory structure for the 1) and
2) points above so we can distingish between them in the test running
harness but...or perhaps even just use the embedder/integration plugin for
much/all of the second one, depends on what kind of verification system we
want to use. But the interesting part would be the ability to use the mojo
lookup much like bret was mentioning above.
so if people are comfortable with this approach I'll work on hammering out a
moderately working implementation so we can make sure the api is clean and
what we are looking for. jason was very keen on the something along the
lines of lookup(mojo, configuration) but we started getting bogged down in
details on what that object would look like since it would need to seed the
mojo for an arbitrary number and type of private variables...
the nice part of using a pom is that we get a pretty good build out of the
whole deal, dependencies and all if need be...and it is a _very_ natural
extension for maven and mojo users. We can provide a few examples of
building out the test cases and it should flow very naturally...and what I
like, expose people to a bit more to the way plexus is used to build objects
in maven...
hm...and the same deal might work if you were wanting to put a inner class
extending abstract mojo test case on the mojo for testing private
methods...kinda interesting
anyway thoughts?
jesse
On 2/21/06, Brett Porter <br...@apache.org> wrote:
>
> Thanks for this!
>
> Jesse McConnell wrote:
> > 1) so I did it once with just a normal junit test class and put the
> setters
> > on the mojo. Very little new code had to be added to get it working and
> it
> > was trivial to take that plugin up to about 85% coverage...the remainder
> > being some workaround for a windows jdk shortcoming.
>
> I almost get 100% on Windows :)
>
> Just needs a test that if you try and delete something that's not
> allowed to be, it throws an exception. And test John's changes that came
> after this :)
>
> > 2) then I redid it using the approach that Trygvis was doing in his
> > deb-maven-plugin where you make the mojo a basic adapter to the
> > implementation which is put together in a standard plexus layout....this
> > forced the generation of a fair bit more code and a couple of additional
> > classes, but it certainly felt more true to the nature of what a mojo
> > is...at least my perception of what a mojo is :)
>
> http://cvs.codehaus.org/changelog/mojo?cs=1478&@csTruncateDiffs=false#
>
> I recently did the separated content and found it extremely tedious to
> implement, so I moved back to not doing that - but it depends on the
> situation as we generally do what you are suggesitng anyway.
>
> The original idea was for a mojo was that it could be reused anywhere
> (automatically generating an ant task wrapper, for example). I think
> we've steered away from that practically though idealogically would
> still like them to be the reusable components. We steered away mostly
> because:
> 1) the plugin api dependency and extending AbstractMojo
> 2) the amount of extra work it is to populate the mojo variables,
> especially when it comes to wiring up the plexus components which is so
> wasy as is.
>
> I think its good to abstract things out like you've done to a degree,
> but I think the right level for clean is pretty much where it is:
> deleteDirectory. Some of this needs to move to plexus utils, and the
> clean plugin should really be delete target, delete output, delete test
> output. WDYT?
>
> So far the approach to factoring has actually taken components out of
> the plugins themselves - plexus-compiler, plexus-archiver, plexus-utils,
> etc. I'm actually happy to continue that way for now to clearly separate
> the responsibilities and build libraries that are reusable outside of
> Maven entirely. A lot of these are being pushed to Jakarta commons since
> they originated from code in Ant that is also being developed there.
> Others go in Maven and its sandbox (the issue tracking stuff, the SCM
> library, the JXR library, ...)
>
> As far as "plexifying" goes, that's probably not the right word, as
> Plexus isn't being used to instantiate the CleanPlugin object.
>
> This is where the testing toolkit might come in - it would do the plexus
> component instantiation of the plugin, so rather than requiring:
> plugin = new Plugin();
> plugin.setComplexObject( constructComplexObject() );
> You'd have:
> plugin = (Plugin) lookup( Mojo.ROLE, "goal-name" );
> with everything wired up, just like it is done in the plugin manager.
> And then you use the setters to add the customised stuff. This will
> probably be more obvious when it comes to the site plugin.
>
> I think the unit tests are a great start though, and we can probably
> tackle some other plugins and see how the other issues go (something
> that has fields with more expressions or @component references).
>
> I'll apply the first patch.
>
> Cheers,
> Brett
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
--
jesse mcconnell
jesseDOTmcconnellATgmailDOTcom
Re: plugin testing
Posted by Brett Porter <br...@apache.org>.
Thanks for this!
Jesse McConnell wrote:
> 1) so I did it once with just a normal junit test class and put the setters
> on the mojo. Very little new code had to be added to get it working and it
> was trivial to take that plugin up to about 85% coverage...the remainder
> being some workaround for a windows jdk shortcoming.
I almost get 100% on Windows :)
Just needs a test that if you try and delete something that's not
allowed to be, it throws an exception. And test John's changes that came
after this :)
> 2) then I redid it using the approach that Trygvis was doing in his
> deb-maven-plugin where you make the mojo a basic adapter to the
> implementation which is put together in a standard plexus layout....this
> forced the generation of a fair bit more code and a couple of additional
> classes, but it certainly felt more true to the nature of what a mojo
> is...at least my perception of what a mojo is :)
http://cvs.codehaus.org/changelog/mojo?cs=1478&@csTruncateDiffs=false#
I recently did the separated content and found it extremely tedious to
implement, so I moved back to not doing that - but it depends on the
situation as we generally do what you are suggesitng anyway.
The original idea was for a mojo was that it could be reused anywhere
(automatically generating an ant task wrapper, for example). I think
we've steered away from that practically though idealogically would
still like them to be the reusable components. We steered away mostly
because:
1) the plugin api dependency and extending AbstractMojo
2) the amount of extra work it is to populate the mojo variables,
especially when it comes to wiring up the plexus components which is so
wasy as is.
I think its good to abstract things out like you've done to a degree,
but I think the right level for clean is pretty much where it is:
deleteDirectory. Some of this needs to move to plexus utils, and the
clean plugin should really be delete target, delete output, delete test
output. WDYT?
So far the approach to factoring has actually taken components out of
the plugins themselves - plexus-compiler, plexus-archiver, plexus-utils,
etc. I'm actually happy to continue that way for now to clearly separate
the responsibilities and build libraries that are reusable outside of
Maven entirely. A lot of these are being pushed to Jakarta commons since
they originated from code in Ant that is also being developed there.
Others go in Maven and its sandbox (the issue tracking stuff, the SCM
library, the JXR library, ...)
As far as "plexifying" goes, that's probably not the right word, as
Plexus isn't being used to instantiate the CleanPlugin object.
This is where the testing toolkit might come in - it would do the plexus
component instantiation of the plugin, so rather than requiring:
plugin = new Plugin();
plugin.setComplexObject( constructComplexObject() );
You'd have:
plugin = (Plugin) lookup( Mojo.ROLE, "goal-name" );
with everything wired up, just like it is done in the plugin manager.
And then you use the setters to add the customised stuff. This will
probably be more obvious when it comes to the site plugin.
I think the unit tests are a great start though, and we can probably
tackle some other plugins and see how the other issues go (something
that has fields with more expressions or @component references).
I'll apply the first patch.
Cheers,
Brett
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
RE: plugin testing
Posted by Vincent Massol <vm...@pivolis.com>.
> -----Original Message-----
> From: Jesse McConnell [mailto:jesse.mcconnell@gmail.com]
> Sent: mardi 21 février 2006 00:45
> To: Maven Developers List
> Subject: Re: plugin testing
>
> ok, I fiddled around a bit today with the clean plugin as a basic test
> case...
>
> I really don't like the idea of putting setters on the mojo...not sure
> why,
> but it bugs me, probably because it is really only putting them there for
> testing purposes which is generally a no-no.
Funnily it's quite the opposite for me. When a unit test wants something
that is not in the main source files, I consider this as a code smell. It
usually means my code is not flexible enough or has a design issue and needs
refactoring.
In addition, I personally believe private field injection is a bit too
"magical" to my taste. We had discussed allowing setters in the past.
I guess you're not yet part of the unit testing addicts who consider that
unit testing is a design activity and that tests are a side-effect of
this... ;-)
[...]
> So I am curious as to what people think about that idea of having the
> mojo's
> setup in that way? It seems a bit heavywieght for the clean plugin but
> how
> important is it to not put those setters on the mojo? I have always been
> a
> bit of a fan for the correct conceptual way as opposed to the cheap and
> easy...though the cheap and easy can be faster :)
For me the conceptually correct one is almost always the mock solution
because it forces you to refactor to offer a flexible and open architecture.
However sometime you don't have the time for this and you go for stubs. But
then you risk adding to your technical debt
(http://www.martinfowler.com/bliki/TechnicalDebt.html). Now I agree it's not
always that black and white...
BTW, I'm playing the devil's advocate here so take my words with a grain of
salt. I'm just answering to fuel the conversation! :-)
[...]
> so I am not really proposing anything with this followup mail...I'll
> generate a patch with my first test case setup for the clean plugin and
> put
> that in jira... I am however wondering if people generally like the
> plexified approach for the plugin implementation? I wouldn't be adverse
> to
> converting a couple more over to see how they look....and it does make
> them
> feel a bit more modular and useful outside of maven for a toolset like
> axistools, sablecc, or javacc since normal people don't have a facility
> for
> injecting those params..
I think you're right and the next step is probably to see some tests in
action. BTW it could be interesting to see tests for the same plugin coded
with both approaches (after refactoring to suit the mock approach, if need
be ;-)).
-Vincent
> On 2/19/06, Jesse McConnell <je...@gmail.com> wrote:
> >
> > after reading up on mocks from the links that vincent posted, I am going
> > to take a stab at putting together a minor set of these to work with for
> a
> > couple of the plugins...just to see how it would work out. Hopefully I
> can
> > get with vincent a bit tomorrow to make sure I get it close to right the
> > first time
> >
> > jesse
> >
> > On 2/18/06, Brett Porter <br...@apache.org> wrote:
> > >
> > > Vincent Massol wrote:
> > > > I think what you're describing is a stub but not a mock. The
> advantage
> > > of a
> > > > dynamic mock is that you don't need to code any method. It's the
> user
> > > of the
> > > > mock which says what behavior it should have for the methods it
> calls
> > > on the
> > > > mock.
> > >
> > > You're right, I've always referred to stubs incorrectly as mocks. I
> > > meant a stub. I think it's in our interest to produce these to make
> > > testing easier and more consistent for everyone.
> > >
> > > I'm interested to see your thoughts on the mocks eventually though -
> > > I've never really done anything with them since I was reading JiA
> (which
> > > I don't have any more :(
> > >
> > > - Brett
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > For additional commands, e-mail: dev-help@maven.apache.org
> > >
> > >
> >
> >
> > --
> > jesse mcconnell
> > jesseDOTmcconnellATgmailDOTcom
>
>
>
>
> --
> jesse mcconnell
> jesseDOTmcconnellATgmailDOTcom
___________________________________________________________________________
Nouveau : téléphonez moins cher avec Yahoo! Messenger ! Découvez les tarifs exceptionnels pour appeler la France et l'international.
Téléchargez sur http://fr.messenger.yahoo.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: plugin testing
Posted by Jesse McConnell <je...@gmail.com>.
ok, I fiddled around a bit today with the clean plugin as a basic test
case...
I really don't like the idea of putting setters on the mojo...not sure why,
but it bugs me, probably because it is really only putting them there for
testing purposes which is generally a no-no.
1) so I did it once with just a normal junit test class and put the setters
on the mojo. Very little new code had to be added to get it working and it
was trivial to take that plugin up to about 85% coverage...the remainder
being some workaround for a windows jdk shortcoming.
2) then I redid it using the approach that Trygvis was doing in his
deb-maven-plugin where you make the mojo a basic adapter to the
implementation which is put together in a standard plexus layout....this
forced the generation of a fair bit more code and a couple of additional
classes, but it certainly felt more true to the nature of what a mojo
is...at least my perception of what a mojo is :)
So I am curious as to what people think about that idea of having the mojo's
setup in that way? It seems a bit heavywieght for the clean plugin but how
important is it to not put those setters on the mojo? I have always been a
bit of a fan for the correct conceptual way as opposed to the cheap and
easy...though the cheap and easy can be faster :)
Another thought I had on this is that with this adapter approach it is real
easy to show the template for the plugin and how to use it...and it offers a
real easy stepping stone to understanding how maven is built deeper in...and
how much is that worth?
so I am not really proposing anything with this followup mail...I'll
generate a patch with my first test case setup for the clean plugin and put
that in jira... I am however wondering if people generally like the
plexified approach for the plugin implementation? I wouldn't be adverse to
converting a couple more over to see how they look....and it does make them
feel a bit more modular and useful outside of maven for a toolset like
axistools, sablecc, or javacc since normal people don't have a facility for
injecting those params..
jesse
On 2/19/06, Jesse McConnell <je...@gmail.com> wrote:
>
> after reading up on mocks from the links that vincent posted, I am going
> to take a stab at putting together a minor set of these to work with for a
> couple of the plugins...just to see how it would work out. Hopefully I can
> get with vincent a bit tomorrow to make sure I get it close to right the
> first time
>
> jesse
>
> On 2/18/06, Brett Porter <br...@apache.org> wrote:
> >
> > Vincent Massol wrote:
> > > I think what you're describing is a stub but not a mock. The advantage
> > of a
> > > dynamic mock is that you don't need to code any method. It's the user
> > of the
> > > mock which says what behavior it should have for the methods it calls
> > on the
> > > mock.
> >
> > You're right, I've always referred to stubs incorrectly as mocks. I
> > meant a stub. I think it's in our interest to produce these to make
> > testing easier and more consistent for everyone.
> >
> > I'm interested to see your thoughts on the mocks eventually though -
> > I've never really done anything with them since I was reading JiA (which
> > I don't have any more :(
> >
> > - Brett
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > For additional commands, e-mail: dev-help@maven.apache.org
> >
> >
>
>
> --
> jesse mcconnell
> jesseDOTmcconnellATgmailDOTcom
--
jesse mcconnell
jesseDOTmcconnellATgmailDOTcom
Re: plugin testing
Posted by Jesse McConnell <je...@gmail.com>.
after reading up on mocks from the links that vincent posted, I am going to
take a stab at putting together a minor set of these to work with for a
couple of the plugins...just to see how it would work out. Hopefully I can
get with vincent a bit tomorrow to make sure I get it close to right the
first time
jesse
On 2/18/06, Brett Porter <br...@apache.org> wrote:
>
> Vincent Massol wrote:
> > I think what you're describing is a stub but not a mock. The advantage
> of a
> > dynamic mock is that you don't need to code any method. It's the user of
> the
> > mock which says what behavior it should have for the methods it calls on
> the
> > mock.
>
> You're right, I've always referred to stubs incorrectly as mocks. I
> meant a stub. I think it's in our interest to produce these to make
> testing easier and more consistent for everyone.
>
> I'm interested to see your thoughts on the mocks eventually though -
> I've never really done anything with them since I was reading JiA (which
> I don't have any more :(
>
> - Brett
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
--
jesse mcconnell
jesseDOTmcconnellATgmailDOTcom
Re: plugin testing
Posted by Brett Porter <br...@apache.org>.
Vincent Massol wrote:
> I think what you're describing is a stub but not a mock. The advantage of a
> dynamic mock is that you don't need to code any method. It's the user of the
> mock which says what behavior it should have for the methods it calls on the
> mock.
You're right, I've always referred to stubs incorrectly as mocks. I
meant a stub. I think it's in our interest to produce these to make
testing easier and more consistent for everyone.
I'm interested to see your thoughts on the mocks eventually though -
I've never really done anything with them since I was reading JiA (which
I don't have any more :(
- Brett
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
RE: plugin testing
Posted by Vincent Massol <vm...@pivolis.com>.
> -----Original Message-----
> From: Brett Porter [mailto:brett@apache.org]
> Sent: vendredi 17 février 2006 23:59
> To: Maven Developers List
> Subject: Re: plugin testing
>
>
> I think this is the right separation. Unit test to get coverage, add and
> use setters like Vincent suggested. Integration test to verify things
> like the lifecycle intereactions, etc. controlled by the annotations at
> the class level. We can already do both.
>
> As Carlos mentioned, getting the objects is difficult. We do need a mock
> MavenProject instance, and can use PlexusTestCase as the base to be able
> to lookup components to set.
>
> Eventually I'd suggest creating a new module with an abstract test case
> that extends PlexusTestCase. This can be used to lookup components to
> set (and possibly automatically wire them up which would be much more
> convenient):
>
> setUp() {
> mojo = lookup( Mojo.ROLE, "goal name" );
> }
>
> In this case we also need to have an expression evaluator replacement
> that will provide replacements for ${project} and ${settings}.
>
> I'd start by manually injecting those values though and see how it goes.
I think what you're describing is a stub but not a mock. The advantage of a
dynamic mock is that you don't need to code any method. It's the user of the
mock which says what behavior it should have for the methods it calls on the
mock. This seems to me the best possible thing because:
- it makes it easy for the user to have several tests with different values
coming from the mock whereas with a stub it's not always going to be easy to
configure everything you want the way you want it.
- it's hard to test exception in a stub whereas with a mock you can say that
for this test you want such method to throw such exception.
My belief is that people do not know mock objects (e.g. jmock) and this is
why they think it's difficult (in most cases they think mock = stubs). I
have the feeling that providing samples on the web site of how to use jmock
to test a plugin would solve a huge percentage of unit testing cases.
Last, when your unit test has to use too many mocks and define too many
behaviors it means 2 things:
- you're not mocking at the right level (you're going too deep)
- the API is bad and needs refactoring
[snip]
Thanks
-Vincent
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: plugin testing
Posted by Brett Porter <br...@apache.org>.
I think this is the right separation. Unit test to get coverage, add and
use setters like Vincent suggested. Integration test to verify things
like the lifecycle intereactions, etc. controlled by the annotations at
the class level. We can already do both.
As Carlos mentioned, getting the objects is difficult. We do need a mock
MavenProject instance, and can use PlexusTestCase as the base to be able
to lookup components to set.
Eventually I'd suggest creating a new module with an abstract test case
that extends PlexusTestCase. This can be used to lookup components to
set (and possibly automatically wire them up which would be much more
convenient):
setUp() {
mojo = lookup( Mojo.ROLE, "goal name" );
}
In this case we also need to have an expression evaluator replacement
that will provide replacements for ${project} and ${settings}.
I'd start by manually injecting those values though and see how it goes.
- Brett
Jesse McConnell wrote:
> brett asked me to look into the idea of a plugin testing framework and
> having mulled it over a bit and talked to some folks about it I wanted to
> spill out my thoughts here and a couple of stabs at breaking the nut
> cleanly. Also, in the interests of having people read this and not have it
> drag on I'll jump right to the chase.
>
> We need someway to be able to gather code coverage metrics from plugin
> testing so we have at least a modicum of belief that changes didn't break
> anything, and since plugins can be so amazingly complex in many cases and
> just downright trivial in others there can't be a one size fits all
> solution....so
>
> I propose that we break the problem down into two neat little parts. First
> would be actual plugin unit testing were we have a simple way to
> instantiating the plugin and any of the pretty normal variables that get
> injected, collections, strings, bool, etc. then we have basic junit type
> testing on these objects, either through careful use of the execute method
> on the mojo's or just being able to test the supporting methods like special
> filename manipulation, downloading a wsdl from a website, perhaps even
> generating some code. Basically, anything and everything that you can
> massage on the plugin that doesn't stray into complex objects like
> MavenProject, or ArtifactManagers or anything.
>
> Now, some of plugins can be completely tested by this mechanism while others
> might not actually fit too tell into this lower level testing. That is
> where the integration testing comes into more of a play.
>
> I talked to john about this and we were kind of a mind that stubbing and
> mocking up these project objects and whatnot don't represent reality very
> well and perhaps the best solution would be to follow the route of the
> integration plugin where we can craft little projects in the plugin
> directory that represent real life usages of the plugin, then we verify that
> the right thing was done. This verification could take the form of
> validating the resulting directory structure matched an outcome, certain
> files matched an outcome, or the state of objects in the project matched the
> template.
>
> This all seems pretty obvious, but I wanted to at least start a conversation
> about how we ought to test these plugins. I like this clear delineation in
> that it draws a line in the sand as to what can get tested where between
> src/test/java and src/it/*. It also encourages a clean plugin design as the
> unit testing should make it simple for you to easily test the checking of
> variable existence and states and helper methods. Although, I know vincent
> has had some success with mocking up some of the more complex objects so I
> would be interested in his thoughts on the matter...and bretts as well on if
> I am barking up the wrong tree...
>
> so, thoughts?
>
> jesse
>
> --
> jesse mcconnell
> jesseDOTmcconnellATgmailDOTcom
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: plugin testing
Posted by Carlos Sanchez <ca...@apache.org>.
I agree that we need both of unit and integration tests
- unit tests are currently painful because a lot of core objects don't
have a constructor without arguments or don't have constructor and
require caling a factory method. We need to agree where to put all
that staff (I sent a mail about this not long ago)
- it tests can be done with something like the it-core does for maven.
We need to agree where should they be in the directory structure.
As we're not providing the support in core people is wasting effort in
each plugin to do this, eg in the aspectj there's a unit test that
calls the MavenEmbedder to run the mojo in an it test, and also a mock
of Artifact.
On 2/17/06, Jesse McConnell <je...@gmail.com> wrote:
> brett asked me to look into the idea of a plugin testing framework and
> having mulled it over a bit and talked to some folks about it I wanted to
> spill out my thoughts here and a couple of stabs at breaking the nut
> cleanly. Also, in the interests of having people read this and not have it
> drag on I'll jump right to the chase.
>
> We need someway to be able to gather code coverage metrics from plugin
> testing so we have at least a modicum of belief that changes didn't break
> anything, and since plugins can be so amazingly complex in many cases and
> just downright trivial in others there can't be a one size fits all
> solution....so
>
> I propose that we break the problem down into two neat little parts. First
> would be actual plugin unit testing were we have a simple way to
> instantiating the plugin and any of the pretty normal variables that get
> injected, collections, strings, bool, etc. then we have basic junit type
> testing on these objects, either through careful use of the execute method
> on the mojo's or just being able to test the supporting methods like special
> filename manipulation, downloading a wsdl from a website, perhaps even
> generating some code. Basically, anything and everything that you can
> massage on the plugin that doesn't stray into complex objects like
> MavenProject, or ArtifactManagers or anything.
>
> Now, some of plugins can be completely tested by this mechanism while others
> might not actually fit too tell into this lower level testing. That is
> where the integration testing comes into more of a play.
>
> I talked to john about this and we were kind of a mind that stubbing and
> mocking up these project objects and whatnot don't represent reality very
> well and perhaps the best solution would be to follow the route of the
> integration plugin where we can craft little projects in the plugin
> directory that represent real life usages of the plugin, then we verify that
> the right thing was done. This verification could take the form of
> validating the resulting directory structure matched an outcome, certain
> files matched an outcome, or the state of objects in the project matched the
> template.
>
> This all seems pretty obvious, but I wanted to at least start a conversation
> about how we ought to test these plugins. I like this clear delineation in
> that it draws a line in the sand as to what can get tested where between
> src/test/java and src/it/*. It also encourages a clean plugin design as the
> unit testing should make it simple for you to easily test the checking of
> variable existence and states and helper methods. Although, I know vincent
> has had some success with mocking up some of the more complex objects so I
> would be interested in his thoughts on the matter...and bretts as well on if
> I am barking up the wrong tree...
>
> so, thoughts?
>
> jesse
>
> --
> jesse mcconnell
> jesseDOTmcconnellATgmailDOTcom
>
>
--
I could give you my word as a Spaniard.
No good. I've known too many Spaniards.
-- The Princess Bride
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: plugin testing
Posted by John Casey <jd...@yahoo.com>.
Hi Vincent,
Vincent Massol wrote:
> Hi Jesse,
>
[snip]
>> Now, some of plugins can be completely tested by this mechanism while
>> others
>> might not actually fit too tell into this lower level testing. That is
>> where the integration testing comes into more of a play.
>>
>> I talked to john about this and we were kind of a mind that stubbing and
>> mocking up these project objects and whatnot don't represent reality very
>> well and perhaps the best solution would be to follow the route of the
>> integration plugin where we can craft little projects in the plugin
>> directory that represent real life usages of the plugin, then we verify
>> that
>> the right thing was done. This verification could take the form of
>> validating the resulting directory structure matched an outcome, certain
>> files matched an outcome, or the state of objects in the project matched
>> the
>> template.
>
> I agree about not mocking anything for integration testing. How is that
> different from what the it and verifier plugin currently do?
>
It's possible that we already have everything we need for executing and
verifying integration tests, though I'd really like to see options for
setup and tear-down of the environment as well. However, more to the
point, I think we need to provide some sort of archetype or something to
bring all of these myriad solution-parts together into something people
can easily hook into for testing...with minimal fuss.
Thanks for starting this discussion, Jesse. It's really important that
we get this sorted out soon, so we can bring plugin development out of
the stone age! :-)
My 2c.
-john
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
RE: plugin testing
Posted by Vincent Massol <vm...@pivolis.com>.
Hi Jesse,
> -----Original Message-----
> From: Jesse McConnell [mailto:jesse.mcconnell@gmail.com]
> Sent: vendredi 17 février 2006 20:40
> To: Maven Developers List
> Subject: plugin testing
>
> brett asked me to look into the idea of a plugin testing framework and
> having mulled it over a bit and talked to some folks about it I wanted to
> spill out my thoughts here and a couple of stabs at breaking the nut
> cleanly. Also, in the interests of having people read this and not have
> it
> drag on I'll jump right to the chase.
Hey thanks for taking this up! :-)
> We need someway to be able to gather code coverage metrics from plugin
> testing so we have at least a modicum of belief that changes didn't break
> anything, and since plugins can be so amazingly complex in many cases and
> just downright trivial in others there can't be a one size fits all
> solution....so
>
> I propose that we break the problem down into two neat little parts.
> First
> would be actual plugin unit testing were we have a simple way to
> instantiating the plugin
For unit testing, the way I do is simple: new MyPlugin()!
> and any of the pretty normal variables that get
> injected, collections, strings, bool, etc.
I add setters to my plugins for this. BTW I have an example of unit testing
a plugin here: http://tinyurl.com/a2cmh
[snip]
> Now, some of plugins can be completely tested by this mechanism while
> others
> might not actually fit too tell into this lower level testing. That is
> where the integration testing comes into more of a play.
>
> I talked to john about this and we were kind of a mind that stubbing and
> mocking up these project objects and whatnot don't represent reality very
> well and perhaps the best solution would be to follow the route of the
> integration plugin where we can craft little projects in the plugin
> directory that represent real life usages of the plugin, then we verify
> that
> the right thing was done. This verification could take the form of
> validating the resulting directory structure matched an outcome, certain
> files matched an outcome, or the state of objects in the project matched
> the
> template.
I agree about not mocking anything for integration testing. How is that
different from what the it and verifier plugin currently do?
[snip]
Thanks!
-Vincent
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org