Running tests that failed during last run

[Antoine Toulme <an...@lunar-ocean.com>]

Is it possible as of today to have Buildr run the tests that failed during
the latest run (and only them) ?

If not, would that be an interesting contribution ?

Thanks,

Antoine

Re: Running tests that failed during last run

[Daniel Spiewak <dj...@gmail.com>]

That's certainly a contribution I would be interested in.  As I recall,
there is currently a way to fake it, but I can't remember exactly how we do
it right now.

Daniel

On Thu, Jan 14, 2010 at 11:37 AM, Antoine Toulme <an...@lunar-ocean.com>wrote:

> Is it possible as of today to have Buildr run the tests that failed during
> the latest run (and only them) ?
>
> If not, would that be an interesting contribution ?
>
> Thanks,
>
> Antoine
>

Re: Running tests that failed during last run

[Antoine Toulme <an...@lunar-ocean.com>]

The weekend ? Hey you come back here, it's only Friday noon! :)

On Fri, Jan 15, 2010 at 11:35, Alex Boisvert <al...@gmail.com>wrote:

> Sweet!!  I'm off for the weekend... will take a look when I'm back.
>
> alex
>
>
>
> On Fri, Jan 15, 2010 at 10:59 AM, Antoine Toulme <an...@lunar-ocean.com>wrote:
>
>> See BUILDR-363, and my patch there.
>>
>> On Thu, Jan 14, 2010 at 10:42, Antoine Toulme <antoine@lunar-ocean.com
>> >wrote:
>>
>> > OK, I'll use that. Thanks!
>> >
>> >
>> > On Thu, Jan 14, 2010 at 10:38, Alex Boisvert <alex.boisvert@gmail.com
>> >wrote:
>> >
>> >> On Thu, Jan 14, 2010 at 10:22 AM, Daniel Spiewak <djspiewak@gmail.com
>> >> >wrote:
>> >>
>> >> > I don't think that JUnit has anything like this.  If I were you, I
>> would
>> >> > look to rspec for inspiration.  When we finish running tests for a
>> >> project,
>> >> > grab the list of test names which failed and save it to a file (maybe
>> >> > target/failed-tests?).  Then have your run-failed-tests task (maybe
>> >> > test:failed?) read that file and only run those tests, rather than
>> usin
>> >> the
>> >> > test provider to determine the set of all tests in that project.
>>  Even
>> >> > better, you could still grab the list of all tests and then intersect
>> it
>> >> > with the set of failed tests.  That way, you avoid running a
>> >> > previously-failed test which no longer exists.
>> >> >
>> >>
>> >> Yep, I'd take the same approach too.
>> >>
>> >> This way, the re-run task can be generic across test frameworks.
>> >>
>> >> alex
>> >>
>> >
>> >
>>
>
>

Re: Running tests that failed during last run

[Alex Boisvert <al...@gmail.com>]

Sweet!!  I'm off for the weekend... will take a look when I'm back.

alex


On Fri, Jan 15, 2010 at 10:59 AM, Antoine Toulme <an...@lunar-ocean.com>wrote:

> See BUILDR-363, and my patch there.
>
> On Thu, Jan 14, 2010 at 10:42, Antoine Toulme <antoine@lunar-ocean.com
> >wrote:
>
> > OK, I'll use that. Thanks!
> >
> >
> > On Thu, Jan 14, 2010 at 10:38, Alex Boisvert <alex.boisvert@gmail.com
> >wrote:
> >
> >> On Thu, Jan 14, 2010 at 10:22 AM, Daniel Spiewak <djspiewak@gmail.com
> >> >wrote:
> >>
> >> > I don't think that JUnit has anything like this.  If I were you, I
> would
> >> > look to rspec for inspiration.  When we finish running tests for a
> >> project,
> >> > grab the list of test names which failed and save it to a file (maybe
> >> > target/failed-tests?).  Then have your run-failed-tests task (maybe
> >> > test:failed?) read that file and only run those tests, rather than
> usin
> >> the
> >> > test provider to determine the set of all tests in that project.  Even
> >> > better, you could still grab the list of all tests and then intersect
> it
> >> > with the set of failed tests.  That way, you avoid running a
> >> > previously-failed test which no longer exists.
> >> >
> >>
> >> Yep, I'd take the same approach too.
> >>
> >> This way, the re-run task can be generic across test frameworks.
> >>
> >> alex
> >>
> >
> >
>

Re: Running tests that failed during last run

[Antoine Toulme <an...@lunar-ocean.com>]

See BUILDR-363, and my patch there.

On Thu, Jan 14, 2010 at 10:42, Antoine Toulme <an...@lunar-ocean.com>wrote:

> OK, I'll use that. Thanks!
>
>
> On Thu, Jan 14, 2010 at 10:38, Alex Boisvert <al...@gmail.com>wrote:
>
>> On Thu, Jan 14, 2010 at 10:22 AM, Daniel Spiewak <djspiewak@gmail.com
>> >wrote:
>>
>> > I don't think that JUnit has anything like this.  If I were you, I would
>> > look to rspec for inspiration.  When we finish running tests for a
>> project,
>> > grab the list of test names which failed and save it to a file (maybe
>> > target/failed-tests?).  Then have your run-failed-tests task (maybe
>> > test:failed?) read that file and only run those tests, rather than usin
>> the
>> > test provider to determine the set of all tests in that project.  Even
>> > better, you could still grab the list of all tests and then intersect it
>> > with the set of failed tests.  That way, you avoid running a
>> > previously-failed test which no longer exists.
>> >
>>
>> Yep, I'd take the same approach too.
>>
>> This way, the re-run task can be generic across test frameworks.
>>
>> alex
>>
>
>

Re: Running tests that failed during last run

[Antoine Toulme <an...@lunar-ocean.com>]

OK, I'll use that. Thanks!

On Thu, Jan 14, 2010 at 10:38, Alex Boisvert <al...@gmail.com>wrote:

> On Thu, Jan 14, 2010 at 10:22 AM, Daniel Spiewak <djspiewak@gmail.com
> >wrote:
>
> > I don't think that JUnit has anything like this.  If I were you, I would
> > look to rspec for inspiration.  When we finish running tests for a
> project,
> > grab the list of test names which failed and save it to a file (maybe
> > target/failed-tests?).  Then have your run-failed-tests task (maybe
> > test:failed?) read that file and only run those tests, rather than usin
> the
> > test provider to determine the set of all tests in that project.  Even
> > better, you could still grab the list of all tests and then intersect it
> > with the set of failed tests.  That way, you avoid running a
> > previously-failed test which no longer exists.
> >
>
> Yep, I'd take the same approach too.
>
> This way, the re-run task can be generic across test frameworks.
>
> alex
>

Re: Running tests that failed during last run

[Alex Boisvert <al...@gmail.com>]

On Thu, Jan 14, 2010 at 10:22 AM, Daniel Spiewak <dj...@gmail.com>wrote:

> I don't think that JUnit has anything like this.  If I were you, I would
> look to rspec for inspiration.  When we finish running tests for a project,
> grab the list of test names which failed and save it to a file (maybe
> target/failed-tests?).  Then have your run-failed-tests task (maybe
> test:failed?) read that file and only run those tests, rather than usin the
> test provider to determine the set of all tests in that project.  Even
> better, you could still grab the list of all tests and then intersect it
> with the set of failed tests.  That way, you avoid running a
> previously-failed test which no longer exists.
>

Yep, I'd take the same approach too.

This way, the re-run task can be generic across test frameworks.

alex

Re: Running tests that failed during last run

[Antoine Toulme <an...@lunar-ocean.com>]

Yeah, I know that's what buildr does (rake failed) for its own tests.
Exactly what I want, but for junit.
I might be able to reuse the reports *.xml and get the failures from there
without using a file for it.

On Thu, Jan 14, 2010 at 10:22, Daniel Spiewak <dj...@gmail.com> wrote:

> I don't think that JUnit has anything like this.  If I were you, I would
> look to rspec for inspiration.  When we finish running tests for a project,
> grab the list of test names which failed and save it to a file (maybe
> target/failed-tests?).  Then have your run-failed-tests task (maybe
> test:failed?) read that file and only run those tests, rather than usin the
> test provider to determine the set of all tests in that project.  Even
> better, you could still grab the list of all tests and then intersect it
> with the set of failed tests.  That way, you avoid running a
> previously-failed test which no longer exists.
>
> At least, that's what I would do for this task.  Feel free to take a
> different route.
>
> Danie
>
>
> On Jan 14, 2010, at 12:14 PM, "Antoine Toulme" <an...@lunar-ocean.com>
> wrote:
>
>  I was thinking there was maybe something bundled in junit to do that and
>> that we just need to press the right buttons.
>>
>> Totally ignorant of other test frameworks, but I can try with junit first
>> and we could expand.
>>
>> On Thu, Jan 14, 2010 at 09:56, Alex Boisvert <alex.boisvert@gmail.com
>> >wrote:
>>
>>  Yes, yes, yes :)
>>>
>>>
>>> On Thu, Jan 14, 2010 at 9:37 AM, Antoine Toulme <antoine@lunar-ocean.com
>>> >wrote:
>>>
>>>  Is it possible as of today to have Buildr run the tests that failed
>>>> during
>>>> the latest run (and only them) ?
>>>>
>>>> If not, would that be an interesting contribution ?
>>>>
>>>> Thanks,
>>>>
>>>> Antoine
>>>>
>>>>
>>>
>>>

Re: Running tests that failed during last run

[Daniel Spiewak <dj...@gmail.com>]

I don't think that JUnit has anything like this.  If I were you, I  
would look to rspec for inspiration.  When we finish running tests for  
a project, grab the list of test names which failed and save it to a  
file (maybe target/failed-tests?).  Then have your run-failed-tests  
task (maybe test:failed?) read that file and only run those tests,  
rather than usin the test provider to determine the set of all tests  
in that project.  Even better, you could still grab the list of all  
tests and then intersect it with the set of failed tests.  That way,  
you avoid running a previously-failed test which no longer exists.

At least, that's what I would do for this task.  Feel free to take a  
different route.

Danie

On Jan 14, 2010, at 12:14 PM, "Antoine Toulme" <antoine@lunar- 
ocean.com> wrote:

> I was thinking there was maybe something bundled in junit to do that  
> and
> that we just need to press the right buttons.
>
> Totally ignorant of other test frameworks, but I can try with junit  
> first
> and we could expand.
>
> On Thu, Jan 14, 2010 at 09:56, Alex Boisvert  
> <al...@gmail.com>wrote:
>
>> Yes, yes, yes :)
>>
>>
>> On Thu, Jan 14, 2010 at 9:37 AM, Antoine Toulme <antoine@lunar-ocean.com 
>> >wrote:
>>
>>> Is it possible as of today to have Buildr run the tests that  
>>> failed during
>>> the latest run (and only them) ?
>>>
>>> If not, would that be an interesting contribution ?
>>>
>>> Thanks,
>>>
>>> Antoine
>>>
>>
>>

Re: Running tests that failed during last run

[Antoine Toulme <an...@lunar-ocean.com>]

I was thinking there was maybe something bundled in junit to do that and
that we just need to press the right buttons.

Totally ignorant of other test frameworks, but I can try with junit first
and we could expand.

On Thu, Jan 14, 2010 at 09:56, Alex Boisvert <al...@gmail.com>wrote:

> Yes, yes, yes :)
>
>
> On Thu, Jan 14, 2010 at 9:37 AM, Antoine Toulme <an...@lunar-ocean.com>wrote:
>
>> Is it possible as of today to have Buildr run the tests that failed during
>> the latest run (and only them) ?
>>
>> If not, would that be an interesting contribution ?
>>
>> Thanks,
>>
>> Antoine
>>
>
>

Re: Running tests that failed during last run

[Alex Boisvert <al...@gmail.com>]

Yes, yes, yes :)

On Thu, Jan 14, 2010 at 9:37 AM, Antoine Toulme <an...@lunar-ocean.com>wrote:

> Is it possible as of today to have Buildr run the tests that failed during
> the latest run (and only them) ?
>
> If not, would that be an interesting contribution ?
>
> Thanks,
>
> Antoine
>