Re: Maven surefire question

[Kalle Korhonen <ka...@gmail.com>]

I was hoping for a failfast behavior for surefire as well. My reason is that
we are running a set of functional tests that take a long time to execute
(say 2 hours), we require all of them to succeed but only know at the end if
it did. I wonder if this feature ever made it to surefire (can't see any
argument like that on surefire's documentation). For us, it'd work if
surefire would take mvn argument -ff (--fail-fast) into account (though as I
understand it, is really meant for reactor builds only).

Kalle


On Fri, Apr 27, 2007 at 2:32 PM, Barrie Treloar <ba...@gmail.com> wrote:

> On 4/28/07, Si'mon <si...@purpleblade.net> wrote:
>
>>
>> When a JUnit test fails I would like Maven:surefire to exit and not run
>> additional tests.  Anyone know how to do this?  It keeps going for some
>> time
>> and then stops but I want it to come to a screeching halt right then.
>>
>
> As Wayne suggests this sounds reasonable, but what is your reasoning behind
> it?
>
> It's most likely because running surefire-report:report re-runs the
> lifecycle again, which is damn annoying.  There is a patch which
> creates surefire-report:report-only which only generates the reports,
> you must have already run surefire before.
>
> I'm looking at putting a release of surefire-report together soonish
> so that this patch can get included, at the moment you have to install
> it locally yourself.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>

Re: Maven surefire question

[Kalle Korhonen <ka...@gmail.com>]

Not sure. How do I configure tests to use one with surefire & junit 3.8?

Kalle


On Wed, Jan 28, 2009 at 12:14 PM, Stephen Connolly <
stephen.alan.connolly@gmail.com> wrote:

> Could you not use a custom test runner?
>
> 2009/1/28 Kalle Korhonen <ka...@gmail.com>:
> > Absolutely agree, shouldn't be a default. Our tests are all independent
> but
> > we'd just like to know right away even if one test fails (no matter which
> > one) - our biggest issue is the time it takes to run the whole suite
> (these
> > are complex feature functional tests, not simple unit tests). So I
> suppose
> > this is not possible at the moment and there's no JIRA for it. I'll add
> an
> > enhancement request.
> >
> > Kalle
> >
> >
> > On Wed, Jan 28, 2009 at 11:22 AM, Baptiste MATHUS <ml...@batmat.net> wrote:
> >
> >> Well, the thing is: I'm not sure running tests in a fail-fast way does
> make
> >> a lot of sense since they must be independent. In fact, if a test
> crashes,
> >> there should never have any impact on other tests (if they're not
> defined
> >> in
> >> the same chain as you can do with testng for example). When ran inside a
> >> continuous integration server, if I have say 50 tests that are failing,
> I
> >> don't want to spend the whole week or more on fixing those bugs
> (imagining
> >> it would only happen in the CI process).
> >>
> >> Adding this option for debugging test fails could be an interesting
> option,
> >> I admit.
> >> However, I really think it is a very good thing it's not the default.
> >>
> >> Cheers.
> >>
> >> 2009/1/28 Kalle Korhonen <ka...@gmail.com>
> >>
> >> > I was hoping for a failfast behavior for surefire as well. My reason
> is
> >> > that
> >> > we are running a set of functional tests that take a long time to
> execute
> >> > (say 2 hours), we require all of them to succeed but only know at the
> end
> >> > if
> >> > it did. I wonder if this feature ever made it to surefire (can't see
> any
> >> > argument like that on surefire's documentation). For us, it'd work if
> >> > surefire would take mvn argument -ff (--fail-fast) into account
> (though
> >> as
> >> > I
> >> > understand it, is really meant for reactor builds only).
> >> >
> >> > Kalle
> >> >
> >> >
> >> > On Fri, Apr 27, 2007 at 2:32 PM, Barrie Treloar <ba...@gmail.com>
> >> > wrote:
> >> >
> >> > > On 4/28/07, Si'mon <si...@purpleblade.net> wrote:
> >> > >
> >> > >>
> >> > >> When a JUnit test fails I would like Maven:surefire to exit and not
> >> run
> >> > >> additional tests.  Anyone know how to do this?  It keeps going for
> >> some
> >> > >> time
> >> > >> and then stops but I want it to come to a screeching halt right
> then.
> >> > >>
> >> > >
> >> > > As Wayne suggests this sounds reasonable, but what is your reasoning
> >> > behind
> >> > > it?
> >> > >
> >> > > It's most likely because running surefire-report:report re-runs the
> >> > > lifecycle again, which is damn annoying.  There is a patch which
> >> > > creates surefire-report:report-only which only generates the
> reports,
> >> > > you must have already run surefire before.
> >> > >
> >> > > I'm looking at putting a release of surefire-report together soonish
> >> > > so that this patch can get included, at the moment you have to
> install
> >> > > it locally yourself.
> >> > >
> >> > >
> >> > >
> ---------------------------------------------------------------------
> >> > > To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> >> > > For additional commands, e-mail: users-help@maven.apache.org
> >> > >
> >> > >
> >> >
> >>
> >>
> >>
> >> --
> >> Baptiste <Batmat> MATHUS - http://batmat.net
> >> Sauvez un arbre,
> >> Mangez un castor !
> >>
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>

Re: Maven surefire question

[Stephen Connolly <st...@gmail.com>]

Could you not use a custom test runner?

2009/1/28 Kalle Korhonen <ka...@gmail.com>:
> Absolutely agree, shouldn't be a default. Our tests are all independent but
> we'd just like to know right away even if one test fails (no matter which
> one) - our biggest issue is the time it takes to run the whole suite (these
> are complex feature functional tests, not simple unit tests). So I suppose
> this is not possible at the moment and there's no JIRA for it. I'll add an
> enhancement request.
>
> Kalle
>
>
> On Wed, Jan 28, 2009 at 11:22 AM, Baptiste MATHUS <ml...@batmat.net> wrote:
>
>> Well, the thing is: I'm not sure running tests in a fail-fast way does make
>> a lot of sense since they must be independent. In fact, if a test crashes,
>> there should never have any impact on other tests (if they're not defined
>> in
>> the same chain as you can do with testng for example). When ran inside a
>> continuous integration server, if I have say 50 tests that are failing, I
>> don't want to spend the whole week or more on fixing those bugs (imagining
>> it would only happen in the CI process).
>>
>> Adding this option for debugging test fails could be an interesting option,
>> I admit.
>> However, I really think it is a very good thing it's not the default.
>>
>> Cheers.
>>
>> 2009/1/28 Kalle Korhonen <ka...@gmail.com>
>>
>> > I was hoping for a failfast behavior for surefire as well. My reason is
>> > that
>> > we are running a set of functional tests that take a long time to execute
>> > (say 2 hours), we require all of them to succeed but only know at the end
>> > if
>> > it did. I wonder if this feature ever made it to surefire (can't see any
>> > argument like that on surefire's documentation). For us, it'd work if
>> > surefire would take mvn argument -ff (--fail-fast) into account (though
>> as
>> > I
>> > understand it, is really meant for reactor builds only).
>> >
>> > Kalle
>> >
>> >
>> > On Fri, Apr 27, 2007 at 2:32 PM, Barrie Treloar <ba...@gmail.com>
>> > wrote:
>> >
>> > > On 4/28/07, Si'mon <si...@purpleblade.net> wrote:
>> > >
>> > >>
>> > >> When a JUnit test fails I would like Maven:surefire to exit and not
>> run
>> > >> additional tests.  Anyone know how to do this?  It keeps going for
>> some
>> > >> time
>> > >> and then stops but I want it to come to a screeching halt right then.
>> > >>
>> > >
>> > > As Wayne suggests this sounds reasonable, but what is your reasoning
>> > behind
>> > > it?
>> > >
>> > > It's most likely because running surefire-report:report re-runs the
>> > > lifecycle again, which is damn annoying.  There is a patch which
>> > > creates surefire-report:report-only which only generates the reports,
>> > > you must have already run surefire before.
>> > >
>> > > I'm looking at putting a release of surefire-report together soonish
>> > > so that this patch can get included, at the moment you have to install
>> > > it locally yourself.
>> > >
>> > >
>> > > ---------------------------------------------------------------------
>> > > To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
>> > > For additional commands, e-mail: users-help@maven.apache.org
>> > >
>> > >
>> >
>>
>>
>>
>> --
>> Baptiste <Batmat> MATHUS - http://batmat.net
>> Sauvez un arbre,
>> Mangez un castor !
>>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org

Re: Maven surefire question

[Kalle Korhonen <ka...@gmail.com>]

Absolutely agree, shouldn't be a default. Our tests are all independent but
we'd just like to know right away even if one test fails (no matter which
one) - our biggest issue is the time it takes to run the whole suite (these
are complex feature functional tests, not simple unit tests). So I suppose
this is not possible at the moment and there's no JIRA for it. I'll add an
enhancement request.

Kalle


On Wed, Jan 28, 2009 at 11:22 AM, Baptiste MATHUS <ml...@batmat.net> wrote:

> Well, the thing is: I'm not sure running tests in a fail-fast way does make
> a lot of sense since they must be independent. In fact, if a test crashes,
> there should never have any impact on other tests (if they're not defined
> in
> the same chain as you can do with testng for example). When ran inside a
> continuous integration server, if I have say 50 tests that are failing, I
> don't want to spend the whole week or more on fixing those bugs (imagining
> it would only happen in the CI process).
>
> Adding this option for debugging test fails could be an interesting option,
> I admit.
> However, I really think it is a very good thing it's not the default.
>
> Cheers.
>
> 2009/1/28 Kalle Korhonen <ka...@gmail.com>
>
> > I was hoping for a failfast behavior for surefire as well. My reason is
> > that
> > we are running a set of functional tests that take a long time to execute
> > (say 2 hours), we require all of them to succeed but only know at the end
> > if
> > it did. I wonder if this feature ever made it to surefire (can't see any
> > argument like that on surefire's documentation). For us, it'd work if
> > surefire would take mvn argument -ff (--fail-fast) into account (though
> as
> > I
> > understand it, is really meant for reactor builds only).
> >
> > Kalle
> >
> >
> > On Fri, Apr 27, 2007 at 2:32 PM, Barrie Treloar <ba...@gmail.com>
> > wrote:
> >
> > > On 4/28/07, Si'mon <si...@purpleblade.net> wrote:
> > >
> > >>
> > >> When a JUnit test fails I would like Maven:surefire to exit and not
> run
> > >> additional tests.  Anyone know how to do this?  It keeps going for
> some
> > >> time
> > >> and then stops but I want it to come to a screeching halt right then.
> > >>
> > >
> > > As Wayne suggests this sounds reasonable, but what is your reasoning
> > behind
> > > it?
> > >
> > > It's most likely because running surefire-report:report re-runs the
> > > lifecycle again, which is damn annoying.  There is a patch which
> > > creates surefire-report:report-only which only generates the reports,
> > > you must have already run surefire before.
> > >
> > > I'm looking at putting a release of surefire-report together soonish
> > > so that this patch can get included, at the moment you have to install
> > > it locally yourself.
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> > > For additional commands, e-mail: users-help@maven.apache.org
> > >
> > >
> >
>
>
>
> --
> Baptiste <Batmat> MATHUS - http://batmat.net
> Sauvez un arbre,
> Mangez un castor !
>

Re: Maven surefire question

[Baptiste MATHUS <ml...@batmat.net>]

Well, the thing is: I'm not sure running tests in a fail-fast way does make
a lot of sense since they must be independent. In fact, if a test crashes,
there should never have any impact on other tests (if they're not defined in
the same chain as you can do with testng for example). When ran inside a
continuous integration server, if I have say 50 tests that are failing, I
don't want to spend the whole week or more on fixing those bugs (imagining
it would only happen in the CI process).

Adding this option for debugging test fails could be an interesting option,
I admit.
However, I really think it is a very good thing it's not the default.

Cheers.

2009/1/28 Kalle Korhonen <ka...@gmail.com>

> I was hoping for a failfast behavior for surefire as well. My reason is
> that
> we are running a set of functional tests that take a long time to execute
> (say 2 hours), we require all of them to succeed but only know at the end
> if
> it did. I wonder if this feature ever made it to surefire (can't see any
> argument like that on surefire's documentation). For us, it'd work if
> surefire would take mvn argument -ff (--fail-fast) into account (though as
> I
> understand it, is really meant for reactor builds only).
>
> Kalle
>
>
> On Fri, Apr 27, 2007 at 2:32 PM, Barrie Treloar <ba...@gmail.com>
> wrote:
>
> > On 4/28/07, Si'mon <si...@purpleblade.net> wrote:
> >
> >>
> >> When a JUnit test fails I would like Maven:surefire to exit and not run
> >> additional tests.  Anyone know how to do this?  It keeps going for some
> >> time
> >> and then stops but I want it to come to a screeching halt right then.
> >>
> >
> > As Wayne suggests this sounds reasonable, but what is your reasoning
> behind
> > it?
> >
> > It's most likely because running surefire-report:report re-runs the
> > lifecycle again, which is damn annoying.  There is a patch which
> > creates surefire-report:report-only which only generates the reports,
> > you must have already run surefire before.
> >
> > I'm looking at putting a release of surefire-report together soonish
> > so that this patch can get included, at the moment you have to install
> > it locally yourself.
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> > For additional commands, e-mail: users-help@maven.apache.org
> >
> >
>



-- 
Baptiste <Batmat> MATHUS - http://batmat.net
Sauvez un arbre,
Mangez un castor !