You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Andrew Purtell <ap...@apache.org> on 2012/11/08 01:26:15 UTC
[DISCUSSION] Policy proposal for JIRAs opened for unit test failures
without patches attached
There has been a recent uptick in JIRAs opened for unit test failures
without patches attached. Since these merely duplicate information readily
available on our Jenkins, we should institute a policy of closing them as
Invalid if a patch is not attached to the JIRA in a timely manner (within
hours). Simply pointing out a failing test is not
a consequential contribution. We should also update the How To Contribute
documentation accordingly.
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: [DISCUSSION] Policy proposal for JIRAs opened for unit test
failures without patches attached
Posted by Jonathan Hsieh <jo...@cloudera.com>.
I've talked about updating the jenkins builds in the past to break
them up into a flaky and non-flakey sets. (it is something I've setup
on an internal jenkins instance). If we do that the the flakey tests
should have jiras until they get fixed.
I can't do this right now but could make some time to work on in
sometime next week.
Jon.
On Thu, Nov 8, 2012 at 12:09 PM, lars hofhansl <lh...@yahoo.com> wrote:
> I'd say we keep the pressure up for failing tests... I.e. we file jiras.
> IMHO, a failing test should either be fixed or disabled, otherwise it just adds noise.
>
>
> (This is true for even occasionally failing tests. We have > 1000 tests, if we have many tests that fail just once/100 runs, we get frequent build failures.)
>
> Just my $0.02.
>
>
> -- Lars
>
>
>
> ________________________________
> From: Stack <st...@duboce.net>
> To: HBase Dev List <de...@hbase.apache.org>
> Sent: Thursday, November 8, 2012 11:26 AM
> Subject: Re: [DISCUSSION] Policy proposal for JIRAs opened for unit test failures without patches attached
>
> On Wed, Nov 7, 2012 at 4:26 PM, Andrew Purtell <ap...@apache.org> wrote:
>> There has been a recent uptick in JIRAs opened for unit test failures
>> without patches attached. Since these merely duplicate information readily
>> available on our Jenkins, we should institute a policy of closing them as
>> Invalid if a patch is not attached to the JIRA in a timely manner (within
>> hours). Simply pointing out a failing test is not
>> a consequential contribution. We should also update the How To Contribute
>> documentation accordingly.
>>
>
> I can go either way.
>
> On the one hand our JIRA has loads of issues opened against failing
> tests that we need to clear up as now as either fixed, invalid, or
> still pertinent. Would be better if failing tests were just addressed
> near immediately.
>
> On the other hand, one day we'll be in a situation where we'll want to
> look at tests that failed in the past but that are currently not
> failing so it'd be good to keep record of the old test in JIRA.
>
> I suppose I'd lean toward no special 'unit test' rule that precludes
> creating issues for failing tests mostly because if a new user, it'd
> be hard to explain the rule they'd be violating.
> St.Ack
--
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
// jon@cloudera.com
Re: [DISCUSSION] Policy proposal for JIRAs opened for unit test failures without patches attached
Posted by lars hofhansl <lh...@yahoo.com>.
Oh yeah. That's definitely something to communicate.
I think your initial point was more nuanced, and I agree that filing a jira for a failing test is only usefully if somebody actually plans to work on it.
-- Lars
________________________________
From: Andrew Purtell <ap...@apache.org>
To: "dev@hbase.apache.org" <de...@hbase.apache.org>; lars hofhansl <lh...@yahoo.com>
Sent: Thursday, November 8, 2012 12:21 PM
Subject: Re: [DISCUSSION] Policy proposal for JIRAs opened for unit test failures without patches attached
> I'd say we keep the pressure up for failing tests... I.e. we file jiras.
I'm not advocating against filing JIRAs for failing tests.
I'm advocating for encouraging *users* to not put up JIRAs for failing tests without a patch for the problem.
On Thu, Nov 8, 2012 at 12:09 PM, lars hofhansl <lh...@yahoo.com> wrote:
I'd say we keep the pressure up for failing tests... I.e. we file jiras.
>IMHO, a failing test should either be fixed or disabled, otherwise it just adds noise.
>
>
>(This is true for even occasionally failing tests. We have > 1000 tests, if we have many tests that fail just once/100 runs, we get frequent build failures.)
>
>Just my $0.02.
>
>
>-- Lars
>
>
>
>________________________________
> From: Stack <st...@duboce.net>
>To: HBase Dev List <de...@hbase.apache.org>
>Sent: Thursday, November 8, 2012 11:26 AM
>Subject: Re: [DISCUSSION] Policy proposal for JIRAs opened for unit test failures without patches attached
>
>
>On Wed, Nov 7, 2012 at 4:26 PM, Andrew Purtell <ap...@apache.org> wrote:
>> There has been a recent uptick in JIRAs opened for unit test failures
>> without patches attached. Since these merely duplicate information readily
>> available on our Jenkins, we should institute a policy of closing them as
>> Invalid if a patch is not attached to the JIRA in a timely manner (within
>> hours). Simply pointing out a failing test is not
>> a consequential contribution. We should also update the How To Contribute
>> documentation accordingly.
>>
>
>I can go either way.
>
>On the one hand our JIRA has loads of issues opened against failing
>tests that we need to clear up as now as either fixed, invalid, or
>still pertinent. Would be better if failing tests were just addressed
>near immediately.
>
>On the other hand, one day we'll be in a situation where we'll want to
>look at tests that failed in the past but that are currently not
>failing so it'd be good to keep record of the old test in JIRA.
>
>I suppose I'd lean toward no special 'unit test' rule that precludes
>creating issues for failing tests mostly because if a new user, it'd
>be hard to explain the rule they'd be violating.
>St.Ack
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
Re: [DISCUSSION] Policy proposal for JIRAs opened for unit test
failures without patches attached
Posted by Andrew Purtell <ap...@apache.org>.
> I'd say we keep the pressure up for failing tests... I.e. we file jiras.
I'm not advocating against filing JIRAs for failing tests.
I'm advocating for encouraging *users* to not put up JIRAs for failing
tests without a patch for the problem.
On Thu, Nov 8, 2012 at 12:09 PM, lars hofhansl <lh...@yahoo.com> wrote:
> I'd say we keep the pressure up for failing tests... I.e. we file jiras.
> IMHO, a failing test should either be fixed or disabled, otherwise it just
> adds noise.
>
>
> (This is true for even occasionally failing tests. We have > 1000 tests,
> if we have many tests that fail just once/100 runs, we get frequent build
> failures.)
>
> Just my $0.02.
>
>
> -- Lars
>
>
>
> ________________________________
> From: Stack <st...@duboce.net>
> To: HBase Dev List <de...@hbase.apache.org>
> Sent: Thursday, November 8, 2012 11:26 AM
> Subject: Re: [DISCUSSION] Policy proposal for JIRAs opened for unit test
> failures without patches attached
>
> On Wed, Nov 7, 2012 at 4:26 PM, Andrew Purtell <ap...@apache.org>
> wrote:
> > There has been a recent uptick in JIRAs opened for unit test failures
> > without patches attached. Since these merely duplicate information
> readily
> > available on our Jenkins, we should institute a policy of closing them as
> > Invalid if a patch is not attached to the JIRA in a timely manner (within
> > hours). Simply pointing out a failing test is not
> > a consequential contribution. We should also update the How To Contribute
> > documentation accordingly.
> >
>
> I can go either way.
>
> On the one hand our JIRA has loads of issues opened against failing
> tests that we need to clear up as now as either fixed, invalid, or
> still pertinent. Would be better if failing tests were just addressed
> near immediately.
>
> On the other hand, one day we'll be in a situation where we'll want to
> look at tests that failed in the past but that are currently not
> failing so it'd be good to keep record of the old test in JIRA.
>
> I suppose I'd lean toward no special 'unit test' rule that precludes
> creating issues for failing tests mostly because if a new user, it'd
> be hard to explain the rule they'd be violating.
> St.Ack
>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: [DISCUSSION] Policy proposal for JIRAs opened for unit test failures without patches attached
Posted by lars hofhansl <lh...@yahoo.com>.
I'd say we keep the pressure up for failing tests... I.e. we file jiras.
IMHO, a failing test should either be fixed or disabled, otherwise it just adds noise.
(This is true for even occasionally failing tests. We have > 1000 tests, if we have many tests that fail just once/100 runs, we get frequent build failures.)
Just my $0.02.
-- Lars
________________________________
From: Stack <st...@duboce.net>
To: HBase Dev List <de...@hbase.apache.org>
Sent: Thursday, November 8, 2012 11:26 AM
Subject: Re: [DISCUSSION] Policy proposal for JIRAs opened for unit test failures without patches attached
On Wed, Nov 7, 2012 at 4:26 PM, Andrew Purtell <ap...@apache.org> wrote:
> There has been a recent uptick in JIRAs opened for unit test failures
> without patches attached. Since these merely duplicate information readily
> available on our Jenkins, we should institute a policy of closing them as
> Invalid if a patch is not attached to the JIRA in a timely manner (within
> hours). Simply pointing out a failing test is not
> a consequential contribution. We should also update the How To Contribute
> documentation accordingly.
>
I can go either way.
On the one hand our JIRA has loads of issues opened against failing
tests that we need to clear up as now as either fixed, invalid, or
still pertinent. Would be better if failing tests were just addressed
near immediately.
On the other hand, one day we'll be in a situation where we'll want to
look at tests that failed in the past but that are currently not
failing so it'd be good to keep record of the old test in JIRA.
I suppose I'd lean toward no special 'unit test' rule that precludes
creating issues for failing tests mostly because if a new user, it'd
be hard to explain the rule they'd be violating.
St.Ack
Re: [DISCUSSION] Policy proposal for JIRAs opened for unit test
failures without patches attached
Posted by Stack <st...@duboce.net>.
On Wed, Nov 7, 2012 at 4:26 PM, Andrew Purtell <ap...@apache.org> wrote:
> There has been a recent uptick in JIRAs opened for unit test failures
> without patches attached. Since these merely duplicate information readily
> available on our Jenkins, we should institute a policy of closing them as
> Invalid if a patch is not attached to the JIRA in a timely manner (within
> hours). Simply pointing out a failing test is not
> a consequential contribution. We should also update the How To Contribute
> documentation accordingly.
>
I can go either way.
On the one hand our JIRA has loads of issues opened against failing
tests that we need to clear up as now as either fixed, invalid, or
still pertinent. Would be better if failing tests were just addressed
near immediately.
On the other hand, one day we'll be in a situation where we'll want to
look at tests that failed in the past but that are currently not
failing so it'd be good to keep record of the old test in JIRA.
I suppose I'd lean toward no special 'unit test' rule that precludes
creating issues for failing tests mostly because if a new user, it'd
be hard to explain the rule they'd be violating.
St.Ack
Re: [DISCUSSION] Policy proposal for JIRAs opened for unit test
failures without patches attached
Posted by Andrew Purtell <ap...@apache.org>.
I don't disagree disagree :-) but there's a difference between committers
doing this and users dropping by with unit test failures on their local box
with 0.90 or something like that. So maybe we can make an exception for
committers but still put something up to dissuade users from putting JIRAs
up like that unless they also have a patch contribution? This could be
making too much of this but it seems reasonable to have this discussion.
On Wed, Nov 7, 2012 at 4:32 PM, Todd Lipcon <to...@cloudera.com> wrote:
> I actually disagree -- jenkins doesn't maintain old logs forever. So, if
> you open a JIRA for a failing test, please do attach the log of the failing
> test, and hopefully the particular revision or git hash where it failed.
>
> On our HDFS team here, we've started devoting days once a quarter or so as
> "fixit days" -- where we all pick a small fixup task to work on and
> hopefully knock down. Flaky tests are good ones for this - they don't
> usually get prioritized during normal development, but when you batch them
> together it's usually easy to burn them down. Having the JIRAs open and
> with logs available makes it much easier to do so.
>
> -Todd
>
> On Wed, Nov 7, 2012 at 4:26 PM, Andrew Purtell <ap...@apache.org>
> wrote:
>
> > There has been a recent uptick in JIRAs opened for unit test failures
> > without patches attached. Since these merely duplicate information
> readily
> > available on our Jenkins, we should institute a policy of closing them as
> > Invalid if a patch is not attached to the JIRA in a timely manner (within
> > hours). Simply pointing out a failing test is not
> > a consequential contribution. We should also update the How To Contribute
> > documentation accordingly.
> >
> > --
> > Best regards,
> >
> > - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
> >
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: [DISCUSSION] Policy proposal for JIRAs opened for unit test
failures without patches attached
Posted by Todd Lipcon <to...@cloudera.com>.
I actually disagree -- jenkins doesn't maintain old logs forever. So, if
you open a JIRA for a failing test, please do attach the log of the failing
test, and hopefully the particular revision or git hash where it failed.
On our HDFS team here, we've started devoting days once a quarter or so as
"fixit days" -- where we all pick a small fixup task to work on and
hopefully knock down. Flaky tests are good ones for this - they don't
usually get prioritized during normal development, but when you batch them
together it's usually easy to burn them down. Having the JIRAs open and
with logs available makes it much easier to do so.
-Todd
On Wed, Nov 7, 2012 at 4:26 PM, Andrew Purtell <ap...@apache.org> wrote:
> There has been a recent uptick in JIRAs opened for unit test failures
> without patches attached. Since these merely duplicate information readily
> available on our Jenkins, we should institute a policy of closing them as
> Invalid if a patch is not attached to the JIRA in a timely manner (within
> hours). Simply pointing out a failing test is not
> a consequential contribution. We should also update the How To Contribute
> documentation accordingly.
>
> --
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>
--
Todd Lipcon
Software Engineer, Cloudera