note for profiles

[Jason van Zyl <ja...@maven.org>]

Howdy,

Just making a note here so that I don't forget:

I'm building all the plexus components and the plexus-compile-jikes
component needs the jikes on the cli and if someone running the reactor
didn't have jikes installed the build would fail even though there is
nothing wrong with the component. So how to capture this in a profile so
that random joe can build successfully when they don't have a jikes
installed. So it boils down to being able to capture information
provided by the user to say they can execute a build. So a user might be
able to state in a profile that the jikes resource is available and
therefore can attempt a build that requires it.

-- 
jvz.

Jason van Zyl
jason at maven.org
http://maven.apache.org

Simplex sigillum veri. (Simplicity is the seal of truth.)


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org

Re: note for profiles

[Brett Porter <br...@apache.org>]

Jason van Zyl wrote:

>On Sun, 2005-05-29 at 10:47 +1000, Brett Porter wrote:
>  
>
>>I believe that should be an integration test, firstly - 
>>    
>>
>
>Is the criterion is that it uses an external tool?
>  
>
Maybe it should be called a functional test in some cases, I may be 
abusing the word. But I don't believe a unit test should
- take a long time
- have external dependencies

The first here is usually related to the second more than anything (need 
for a network, external process, etc).

>I want to test that the component actually works isolation so I'm not
>sure I would consider that an integration test. 
>  
>
But the success depends on something outside of the component - that's 
more what I was getting at.

>some of the other compilers use Java code so it's easier to actually
>compile something in a unit test. Would you consider those all to be
>integration tests? Not sure if I would buy that.
>  
>
I'm not really sure of the naming, but I think it is. For the Java 
compiler, it depends on the tools.jar from the JDK which is somewhat 
external. The eclipse one is much more normal, but I'd still consider it 
separate. Actually compiling the source file goes beyond what I'd expect 
to need to test. There shouldn't be any need to test whether the 
compiler itself works to check the component works.

>If I'm testing, say, the i18n component and I want to make sure it works
>what's the point at which you would consider it an integration test? I
>just want to know the component itself works. When in relation to other
>components I would consider that an integration test.
>  
>
I'm not sure what you mean by the i18n example, but as you say - testing 
among other components. I consider the compiler libraries above at the 
same level as "other components", I think.

>I would consider something like starting up a container, or some other
>application context (like our core-it) and look and verifying operation
>as an integration test.
>  
>
Personally I'd still consider anything that requires some specific 
environment, a test that crosses the boundaries of components/libraries 
to be an integration test.

>
>
>I didn't go to any create lengths to use mock objects there and I just
>found whatever servers I could find but they still are just trying to
>verify that the component itself works. Trying to download something
>from m2 I would consider an integration test for Wagon.
>  
>
I'd consider than an integration test between m2 and wagon, yes. Wagon 
and SCM are probably different - wagon is more functional tests, SCM is 
probably integration tests. So maybe I'm just using the wrong term, but 
waiting the length of time we do for those test cases to run (SCM in 
particular) is insane.  Those sorts of tests should only need to be run 
in the CI and QA environments.

I'm no big fan of mock objects - they have their uses, but in this case 
I think the unit tests should just be testing that, for example, the SCM 
command lines are constructed as expected (perhaps needs to be different 
for any pure Java SCM interfaces we come up with, but that won't affect 
the TCK).

>
>Would the general rule be the involvement of external resources
>classifies it as an integration test? Or is there a degree? Would using
>a file make a test an integration test? Question becomes what's
>external?
>  
>
No, I don't think we should be running ashcroft :) That's a bit too 
extreme. But it annoys me no end that tests are dependant on a basedir 
sys property. My IDE doesn't set that for me.

That's a minor thing I think we should avoid. I think the criteria is 
not so much what is an integration test, but what is not a unit test 
(and hence not run in general development). Anything that
- relies on a particular environment setup (eg external network 
resources, paths, external apps, sys props)
- takes a long time to run through repetitive processing, spending a lot 
of time in external libraries, or using external apps or network resources
- has multiple tests that only test external libraries (eg if I pass a 
parameter as A and then as B that follows the exact same code path as 
long as it is in your own code).

>Maybe a differentiation between an integration test and a functional
>test? Where the scm and wagon tests are functional tests because there
>aren't really being integrated with anything per se but those tests have
>them hopping out on to a network to make sure they actually work.
>  
>
I think that is right. Not sure whether I'd consider SCM integration or 
not, though it seems mostly definitional. The reason I say that is Wagon 
tests using its own libraries so it isn't really variable though it is 
doing more than just testing itself. SCM, on the other hand, has been 
known to break on:
- different CVS, SVN executables
- different operating systems
- different locales
These all indicate that the integration points with the external apps 
are affecting the tests, which is not what I think should be the case 
for unit tests.

- Brett


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org

Re: note for profiles

[Jason van Zyl <ja...@maven.org>]

On Sun, 2005-05-29 at 10:47 +1000, Brett Porter wrote:
> I believe that should be an integration test, firstly - 

Is the criterion is that it uses an external tool?

I want to test that the component actually works isolation so I'm not
sure I would consider that an integration test. 

some of the other compilers use Java code so it's easier to actually
compile something in a unit test. Would you consider those all to be
integration tests? Not sure if I would buy that.

If I'm testing, say, the i18n component and I want to make sure it works
what's the point at which you would consider it an integration test? I
just want to know the component itself works. When in relation to other
components I would consider that an integration test.

I would consider something like starting up a container, or some other
application context (like our core-it) and look and verifying operation
as an integration test.

> only executed 
> when an env is present and not part of the normal jar build. Same with 
> all those wagon and scm tests. For me, I'm fine if they get run as a 
> separate step after install.

I didn't go to any create lengths to use mock objects there and I just
found whatever servers I could find but they still are just trying to
verify that the component itself works. Trying to download something
from m2 I would consider an integration test for Wagon.

> I think when we discssued it on IRC (it should be in the logs on 
> confluence) with regards to profiles, I was leaning towards the 
> integration test plugin having this capability... if no jikesExecutable 
> is configured for the plugin, then skip those tests, and jikesExecutable 
> would be specified in the profile and default to "jikes" so the path 
> would be checked. I'm not sure if we'd build that directly into the 
> plugin - maybe just pass it through as a sysprop and the suite() method 
> can be used in junit to determine what tests to run..

Would the general rule be the involvement of external resources
classifies it as an integration test? Or is there a degree? Would using
a file make a test an integration test? Question becomes what's
external?

Maybe a differentiation between an integration test and a functional
test? Where the scm and wagon tests are functional tests because there
aren't really being integrated with anything per se but those tests have
them hopping out on to a network to make sure they actually work.

> - Brett
> 
> Jason van Zyl wrote:
> 
> >Howdy,
> >
> >Just making a note here so that I don't forget:
> >
> >I'm building all the plexus components and the plexus-compile-jikes
> >component needs the jikes on the cli and if someone running the reactor
> >didn't have jikes installed the build would fail even though there is
> >nothing wrong with the component. So how to capture this in a profile so
> >that random joe can build successfully when they don't have a jikes
> >installed. So it boils down to being able to capture information
> >provided by the user to say they can execute a build. So a user might be
> >able to state in a profile that the jikes resource is available and
> >therefore can attempt a build that requires it.
> >
> >  
> >
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
> 
> 
-- 
jvz.

Jason van Zyl
jason at maven.org
http://maven.apache.org

A party which is not afraid of letting culture,
business, and welfare go to ruin completely can
be omnipotent for a while.

  -- Jakob Burckhardt


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org

Re: note for profiles

[Brett Porter <br...@apache.org>]

I believe that should be an integration test, firstly - only executed 
when an env is present and not part of the normal jar build. Same with 
all those wagon and scm tests. For me, I'm fine if they get run as a 
separate step after install.

I think when we discssued it on IRC (it should be in the logs on 
confluence) with regards to profiles, I was leaning towards the 
integration test plugin having this capability... if no jikesExecutable 
is configured for the plugin, then skip those tests, and jikesExecutable 
would be specified in the profile and default to "jikes" so the path 
would be checked. I'm not sure if we'd build that directly into the 
plugin - maybe just pass it through as a sysprop and the suite() method 
can be used in junit to determine what tests to run..

- Brett

Jason van Zyl wrote:

>Howdy,
>
>Just making a note here so that I don't forget:
>
>I'm building all the plexus components and the plexus-compile-jikes
>component needs the jikes on the cli and if someone running the reactor
>didn't have jikes installed the build would fail even though there is
>nothing wrong with the component. So how to capture this in a profile so
>that random joe can build successfully when they don't have a jikes
>installed. So it boils down to being able to capture information
>provided by the user to say they can execute a build. So a user might be
>able to state in a profile that the jikes resource is available and
>therefore can attempt a build that requires it.
>
>  
>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org