Are BadApple reliable when 7x tests are included?

[Erick Erickson <er...@gmail.com>]

Kevin Risden asked me whether the failures for HDFS tests were on
master and/or 8x and/or 7x. That led me to realize that the current
way BadApple annotations are working has a fundamental flaw since I
don't know what versions of Solr the tests fail on.

The rollups I'm using from Hoss' work aggregate tests that _include_
7x.  So it's unreliable to annotate tests that have failures for every
one of the last 4 weeks since 7x tests probably won't be fixed. And
the raw work involved in trying to pull out individual test results is
too much effort.

It's still reasonable to _unannotate_ tests that haven't failed on any
version of Solr in the last 4 weeks. I'll do that on Thursday since
there are a bunch of them. But there also aren't all that many tests
that are BadApple'd, after Thursday there'll only be 24.

I can pick this up again when we stop running Jenkins jobs for 7x as
part of the normal process.

Unless there are other ideas?

Erick

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org