Failing Integration Tests

[Andrei Savu <sa...@gmail.com>]

I'm moving this to the email list.

It's strange we are seeing this failure because before submitting the
final patch for WHIRR-199 I have been able to run all the tests by
hand.

I believe that it would be helpful for now to at least have a CI
server that would run integration tests for every patch that gets
committed to the trunk (even if we are doing this by hand before
committing - it seems like it's not enough). It's better to know
sooner rather than later that you have committed something broken.

Tibor Kiss commented on WHIRR-55:
---------------------------------

Thx.
Last time I have made successful integration tests it was when I
applied my WHIRR-167 patch to the 1059503 revision on the trunk.
Now it we are at 1065812 revision on the trunk. Somewhere in between
these revision numbers has been introduced the problem. I'm sure you
or somebody else can further reduce the interval of search.

Even if you have CI server, sometimes is inefficient to run
integration tests on each patch you apply. Maybe a few patches
together it worst, then if fails further dividing can be applied. The
problem to automate the integration test while merging the patches is
caused by the fact that sometimes you want to run the integration test
before you are commiting. I don't know how this can be solved, or you
just commit merge the patches into trunk, then one by one you will
wait for the results on CI server? (sorry for offtopic).

--
Andrei Savu -- andreisavu.ro

Re: Failing Integration Tests

[Tom White <to...@gmail.com>]

On Wed, Feb 2, 2011 at 6:23 AM, Andrei Savu <sa...@gmail.com> wrote:
> I'm moving this to the email list.

Thanks for raising this, Andrei.

>
> It's strange we are seeing this failure because before submitting the
> final patch for WHIRR-199 I have been able to run all the tests by
> hand.

This should be fixed now. See my comment on
https://issues.apache.org/jira/browse/WHIRR-227 for more detail.

>
> I believe that it would be helpful for now to at least have a CI
> server that would run integration tests for every patch that gets
> committed to the trunk (even if we are doing this by hand before
> committing - it seems like it's not enough). It's better to know
> sooner rather than later that you have committed something broken.

I agree - I've opened https://issues.apache.org/jira/browse/WHIRR-228
to discuss how we might do this.

Cheers,
Tom

>
> Tibor Kiss commented on WHIRR-55:
> ---------------------------------
>
> Thx.
> Last time I have made successful integration tests it was when I
> applied my WHIRR-167 patch to the 1059503 revision on the trunk.
> Now it we are at 1065812 revision on the trunk. Somewhere in between
> these revision numbers has been introduced the problem. I'm sure you
> or somebody else can further reduce the interval of search.
>
> Even if you have CI server, sometimes is inefficient to run
> integration tests on each patch you apply. Maybe a few patches
> together it worst, then if fails further dividing can be applied. The
> problem to automate the integration test while merging the patches is
> caused by the fact that sometimes you want to run the integration test
> before you are commiting. I don't know how this can be solved, or you
> just commit merge the patches into trunk, then one by one you will
> wait for the results on CI server? (sorry for offtopic).
>
> --
> Andrei Savu -- andreisavu.ro
>