You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@solr.apache.org by David Smiley <ds...@apache.org> on 2023/03/23 00:39:15 UTC
Notice of tests failing often
A public service announcement of tests failing a lot (>5%):
as reported by from
http://fucit.org/solr-jenkins-reports/failure-report.html
1. ShardSplitTest. 29%
2. JWTAuthPluginIntegrationTest: 29%
3. TestLeaderElectionZkExpiry: 14%
4. TestReplicationHandler: 14%
(The above are out of only 7 runs whereas everything else is out of
hundreds. Weird)
~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley
Re: Notice of tests failing often
Posted by Houston Putman <ho...@apache.org>.
Sorry, I think I mixed up BadApples and Nightly.
- Houston
On Thu, Mar 23, 2023 at 11:05 AM Jan Høydahl <ja...@cominvent.com> wrote:
> Actually, the JWTAuthPluginIntegrationTest is @BadApple so runs even less
> often I believe -
> https://ci-builds.apache.org/job/Solr/job/Solr-BadApples-Tests-main/941/
>
> Jan
>
> > 23. mar. 2023 kl. 16:56 skrev Houston Putman <ho...@apache.org>:
> >
> > Thanks for the PSA David!
> >
> > Overall having only 13 entries above 1% failure is such an improvement to
> > the state we were in a year ago, so thank you so much to everyone that
> has
> > put in a lot of hard work on that (shout out to Kevin and Mike)!
> >
> > (The above are out of only 7 runs whereas everything else is out of
> >> hundreds. Weird)
> >
> >
> > This is because those tests are @nightly tests, and thus only run on one
> > jenkins job that runs daily, so 7 runs per week. The other tests run on
> > main, 9x, the release branch, the policeman jenkins, and others, so they
> > get a ton more iterations.
> >
> > Since the tests are in such a good place, it might be a good time to look
> > into the nightly tests and trying to move some of them to the general
> runs.
> > Though, this would require making some of them faster since Mike sped up
> > the tests by moving the really slow ones to @nightly.
> >
> > - Houston
> >
> > On Wed, Mar 22, 2023 at 7:39 PM David Smiley <ds...@apache.org> wrote:
> >
> >> A public service announcement of tests failing a lot (>5%):
> >> as reported by from
> >> http://fucit.org/solr-jenkins-reports/failure-report.html
> >> 1. ShardSplitTest. 29%
> >> 2. JWTAuthPluginIntegrationTest: 29%
> >> 3. TestLeaderElectionZkExpiry: 14%
> >> 4. TestReplicationHandler: 14%
> >>
> >> (The above are out of only 7 runs whereas everything else is out of
> >> hundreds. Weird)
> >>
> >> ~ David Smiley
> >> Apache Lucene/Solr Search Developer
> >> http://www.linkedin.com/in/davidwsmiley
> >>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> For additional commands, e-mail: dev-help@solr.apache.org
>
>
Re: Notice of tests failing often
Posted by Jan Høydahl <ja...@cominvent.com>.
Actually, the JWTAuthPluginIntegrationTest is @BadApple so runs even less often I believe - https://ci-builds.apache.org/job/Solr/job/Solr-BadApples-Tests-main/941/
Jan
> 23. mar. 2023 kl. 16:56 skrev Houston Putman <ho...@apache.org>:
>
> Thanks for the PSA David!
>
> Overall having only 13 entries above 1% failure is such an improvement to
> the state we were in a year ago, so thank you so much to everyone that has
> put in a lot of hard work on that (shout out to Kevin and Mike)!
>
> (The above are out of only 7 runs whereas everything else is out of
>> hundreds. Weird)
>
>
> This is because those tests are @nightly tests, and thus only run on one
> jenkins job that runs daily, so 7 runs per week. The other tests run on
> main, 9x, the release branch, the policeman jenkins, and others, so they
> get a ton more iterations.
>
> Since the tests are in such a good place, it might be a good time to look
> into the nightly tests and trying to move some of them to the general runs.
> Though, this would require making some of them faster since Mike sped up
> the tests by moving the really slow ones to @nightly.
>
> - Houston
>
> On Wed, Mar 22, 2023 at 7:39 PM David Smiley <ds...@apache.org> wrote:
>
>> A public service announcement of tests failing a lot (>5%):
>> as reported by from
>> http://fucit.org/solr-jenkins-reports/failure-report.html
>> 1. ShardSplitTest. 29%
>> 2. JWTAuthPluginIntegrationTest: 29%
>> 3. TestLeaderElectionZkExpiry: 14%
>> 4. TestReplicationHandler: 14%
>>
>> (The above are out of only 7 runs whereas everything else is out of
>> hundreds. Weird)
>>
>> ~ David Smiley
>> Apache Lucene/Solr Search Developer
>> http://www.linkedin.com/in/davidwsmiley
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
For additional commands, e-mail: dev-help@solr.apache.org
Re: Notice of tests failing often
Posted by Houston Putman <ho...@apache.org>.
Thanks for the PSA David!
Overall having only 13 entries above 1% failure is such an improvement to
the state we were in a year ago, so thank you so much to everyone that has
put in a lot of hard work on that (shout out to Kevin and Mike)!
(The above are out of only 7 runs whereas everything else is out of
> hundreds. Weird)
This is because those tests are @nightly tests, and thus only run on one
jenkins job that runs daily, so 7 runs per week. The other tests run on
main, 9x, the release branch, the policeman jenkins, and others, so they
get a ton more iterations.
Since the tests are in such a good place, it might be a good time to look
into the nightly tests and trying to move some of them to the general runs.
Though, this would require making some of them faster since Mike sped up
the tests by moving the really slow ones to @nightly.
- Houston
On Wed, Mar 22, 2023 at 7:39 PM David Smiley <ds...@apache.org> wrote:
> A public service announcement of tests failing a lot (>5%):
> as reported by from
> http://fucit.org/solr-jenkins-reports/failure-report.html
> 1. ShardSplitTest. 29%
> 2. JWTAuthPluginIntegrationTest: 29%
> 3. TestLeaderElectionZkExpiry: 14%
> 4. TestReplicationHandler: 14%
>
> (The above are out of only 7 runs whereas everything else is out of
> hundreds. Weird)
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>