Unit Tests pretty slow?

[Isabel Drost <is...@apache.org>]

Hello,

is it just my current machine, or do we have some pretty slow
running/large tests in our unit test suite?

What do you think about splitting the tests into short (sort of
pre-commit) and longish (sort of integration) tests during
build-time? That should be relatively easy to set up with maven.

Are there tests that could benefit from a little optimization to run
faster?

Isabel

Re: Unit Tests pretty slow?

[Jeff Eastman <jd...@windwardsolutions.com>]

Some of the clustering unit tests that test the Hadoop jobs take a while 
to run through their iterations. This is on the order of a minute or two 
in some cases. I think testing the jobs should still be done in the 
pre-commit batch, since the commits really need them to pass successfully.

Jeff

Isabel Drost wrote:
> Hello,
>
> is it just my current machine, or do we have some pretty slow
> running/large tests in our unit test suite?
>
> What do you think about splitting the tests into short (sort of
> pre-commit) and longish (sort of integration) tests during
> build-time? That should be relatively easy to set up with maven.
>
> Are there tests that could benefit from a little optimization to run
> faster?
>
> Isabel
>
>
>
>   

Re: Unit Tests pretty slow?

[Isabel Drost <is...@apache.org>]

On Fri, 18 Sep 2009 07:28:47 -0700
Grant Ingersoll <gs...@apache.org> wrote:

> Is it possible to setup the tests to run in parallel?  I know Ant
> has this option.  This could speed things up.

That sounds like an interesting option.


> As for integration vs units.  Are you suggesting then, that most of  
> the time you'd only run the unit tests?  In the end, the only way
> you have assurance that things are still working is that all the
> tests pass.  In other words, I don't quite understand the benefit of
> moving things around.

I am not speaking of moving out those tests that are testing
functionality but things like the load test mentioned by Owen.


> FWIW, I usually just run the individual test in my IDE and then
> before committing run all the tests.

Same here.


Isabel

Re: Unit Tests pretty slow?

[Isabel Drost <is...@apache.org>]

On Fri, 18 Sep 2009 08:19:48 -0700
Grant Ingersoll <gs...@apache.org> wrote:

> I usually just kick 'em off and go fill in docs or do something else.

Like answering questions on the mailing list ;)

Cheers,
Isabel

Re: Unit Tests pretty slow?

[Grant Ingersoll <gs...@apache.org>]

On Sep 18, 2009, at 7:43 AM, Sean Owen wrote:
>
> Or we take a moment to figure out parallelization and speed up a few  
> key
> tests. The time doesn't bother me much yet. Takes maybe 5 mins? When  
> that is
> 30, I will complain!

I usually just kick 'em off and go fill in docs or do something else.

Re: Unit Tests pretty slow?

[Sean Owen <sr...@gmail.com>]

Yeah, don't know how to do it in Ant. There are decorator implementations of
TestSuite that can parallelize, thought it means maintaining the suites sort
of manually in code. Indeed it is one line in Ant.

In the end you have to run all tests before a release. There is a useful
distinction between stuff that tests whether a unit is correct and stuff
that tests whether units work together, or under load, or with external
entities. One should test units all the time and maybe integration tests
less so. And happily it is usually quick to run unit tests. So, common to
run at every checkin and practical too.

I suppose the question is do we really have integration tests, or just some
fat unit tests? My LoadTest falls I think in my mental category of
integration test I think. I can picture Hadoop blackbox tests being the same
(theory being perhaps there are also unit tests for the mapper in isolation
- we don't have these though)

Or we take a moment to figure out parallelization and speed up a few key
tests. The time doesn't bother me much yet. Takes maybe 5 mins? When that is
30, I will complain!

Sean

On Sep 18, 2009 3:29 PM, "Grant Ingersoll" <gs...@apache.org> wrote:

Is it possible to setup the tests to run in parallel?  I know Ant has this
option.  This could speed things up.

As for integration vs units.  Are you suggesting then, that most of the time
you'd only run the unit tests?  In the end, the only way you have assurance
that things are still working is that all the tests pass.  In other words, I
don't quite understand the benefit of moving things around.

FWIW, I usually just run the individual test in my IDE and then before
committing run all the tests.

On Sep 18, 2009, at 4:04 AM, Isabel Drost wrote: > On Fri, 18 Sep 2009
11:36:40 +0100 > Sean Owen...

Re: Unit Tests pretty slow?

[Grant Ingersoll <gs...@apache.org>]

Is it possible to setup the tests to run in parallel?  I know Ant has  
this option.  This could speed things up.

As for integration vs units.  Are you suggesting then, that most of  
the time you'd only run the unit tests?  In the end, the only way you  
have assurance that things are still working is that all the tests  
pass.  In other words, I don't quite understand the benefit of moving  
things around.

FWIW, I usually just run the individual test in my IDE and then before  
committing run all the tests.

On Sep 18, 2009, at 4:04 AM, Isabel Drost wrote:

> On Fri, 18 Sep 2009 11:36:40 +0100
> Sean Owen <sr...@gmail.com> wrote:
>> - *IntegrationTest for long-running tests, or
>
> +1
>
> Isabel

Re: Unit Tests pretty slow?

[Isabel Drost <is...@apache.org>]

On Fri, 18 Sep 2009 11:36:40 +0100
Sean Owen <sr...@gmail.com> wrote:
> - *IntegrationTest for long-running tests, or

+1

Isabel

Re: Unit Tests pretty slow?

[Sean Owen <sr...@gmail.com>]

In the interest of limiting the change necessary, let me suggest the
following complete naming scheme:

First I think the standard naming convention is:
- *TestCase for abstract superclasses of tests, not for test classes themselves
- *Test for tests

(Many of the matrix tests and all clustering tests are named Test*
now. If it's just those... may be worth the effort to standardize
IMHO. Even if it's purely naming convention stuff.)

So most tests follow that. Then we could add:

- *IntegrationTest for long-running tests, or
- *.integration.*Test as a convention, for putting these in their own package

I am indifferent between these two.

On Fri, Sep 18, 2009 at 11:33 AM, Isabel Drost <is...@apache.org> wrote:
> One way would be to agree on a common naming schema (something like
> "any test class <name_of_class>ITest" should go into the integration
> suite and "any test class <name_of_class>UTest" should go into the unit
> test suite.
>
> Then in the surefire plugin you can define separate includes/ excludes
> based on pattern matching on these names for execution phases utest and
> integration-test.

Re: Unit Tests pretty slow?

[Isabel Drost <is...@apache.org>]

On Fri, 18 Sep 2009 10:47:44 +0100
Sean Owen <sr...@gmail.com> wrote:

> Is there a clean way to separate these in Maven? I support
> separating those out, as well as the Hadoop ones.

One way would be to agree on a common naming schema (something like
"any test class <name_of_class>ITest" should go into the integration
suite and "any test class <name_of_class>UTest" should go into the unit
test suite.

Then in the surefire plugin you can define separate includes/ excludes
based on pattern matching on these names for execution phases utest and
integration-test.

Isabel

Re: Unit Tests pretty slow?

[Sean Owen <sr...@gmail.com>]

>From my neck of the woods -- there is a test called LoadTest that is
certainly long-running by design. The rest ought to be quick. Is there
a clean way to separate these in Maven? I support separating those
out, as well as the Hadoop ones.



On Fri, Sep 18, 2009 at 10:39 AM, Isabel Drost <is...@apache.org> wrote:
>
> Hello,
>
> is it just my current machine, or do we have some pretty slow
> running/large tests in our unit test suite?
>
> What do you think about splitting the tests into short (sort of
> pre-commit) and longish (sort of integration) tests during
> build-time? That should be relatively easy to set up with maven.
>
> Are there tests that could benefit from a little optimization to run
> faster?
>
> Isabel
>
>

Re : Unit Tests pretty slow?

[deneche abdelhakim <a_...@yahoo.fr>]

> Are there tests that could benefit from a little
> optimization to run faster?

in my machine, the examples tests are very (very...) slow. Those tests are related to watchmaker integration (my code =P ). I wrote them a year ago and I think that they can be made faster

--- En date de : Ven 18.9.09, Isabel Drost <is...@apache.org> a écrit :

> De: Isabel Drost <is...@apache.org>
> Objet: Unit Tests pretty slow?
> À: mahout-dev@lucene.apache.org
> Date: Vendredi 18 Septembre 2009, 11h39
> 
> Hello,
> 
> is it just my current machine, or do we have some pretty
> slow
> running/large tests in our unit test suite?
> 
> What do you think about splitting the tests into short
> (sort of
> pre-commit) and longish (sort of integration) tests during
> build-time? That should be relatively easy to set up with
> maven.
> 
> Are there tests that could benefit from a little
> optimization to run
> faster?
> 
> Isabel
> 
>