You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Mark Miller <ma...@gmail.com> on 2015/06/09 17:40:14 UTC
BadApple Nightly?
I'd like to work on getting the nightly Solr runs to pass more frequently -
they have never been in very good shape. There are a few problem tests that
don't have such persistent issues on non nightly runs. Is it possible to
ignore specified tests as BadApples or AwaitsFix but only for nightly runs?
It would be great if we could pare out the troublesome tests without losing
non nightly coverage and then work on those issues to get those tests back
on for nightly.
- Mark
--
- Mark
about.me/markrmiller
Re: BadApple Nightly?
Posted by Mark Miller <ma...@gmail.com>.
Ah, of course - thanks again!
- Mark
On Thu, Jun 11, 2015 at 2:43 AM Dawid Weiss <da...@gmail.com> wrote:
> There has to be some distinction between nightly and regular runs --
> for jenkins, we pass tests.nightly anyway to make this distinction
> clear. You'd have to configure an appropriate filter for your jenkins
> job.
>
> > Is it possible to ignore specified tests as BadApples or AwaitsFix but
> only for nightly runs?
>
> You can do it manually by adding @BeforeClass
> assumeTrue(!"true".equals(System.getProperty("tests.nightly")))?
>
> Dawid
>
> On Wed, Jun 10, 2015 at 9:40 PM, Mark Miller <ma...@gmail.com>
> wrote:
> > Hmm...is there any way to get this behavior somehow via just annotations
> > though? It doesn't really work for my use case if it has to be specified
> > from the command line.
> >
> > On Tue, Jun 9, 2015 at 2:46 PM Mark Miller <ma...@gmail.com>
> wrote:
> >>
> >> Thanks!
> >>
> >> On Tue, Jun 9, 2015 at 2:28 PM Dawid Weiss <da...@gmail.com>
> wrote:
> >>>
> >>> > don't have such persistent issues on non nightly runs. Is it possible
> >>> > to
> >>> > ignore specified tests as BadApples or AwaitsFix but only for nightly
> >>> > runs?
> >>>
> >>> Sure, that's what test group filtering was added for. It's interesting
> >>> that test-help doesn't show anything... ES has a more verbose info:
> >>>
> >>> You can also filter tests by certain annotations ie:
> >>>
> >>> * `@Slow` - tests that are know to take a long time to execute
> >>> * `@Nightly` - tests that only run in nightly builds (disabled by
> >>> default)
> >>> * `@Integration` - integration tests
> >>> * `@Backwards` - backwards compatibility tests (disabled by default)
> >>> * `@AwaitsFix` - tests that are waiting for a bugfix (disabled by
> >>> default)
> >>> * `@BadApple` - tests that are known to fail randomly (disabled by
> >>> default)
> >>>
> >>> Those annotation names can be combined into a filter expression like:
> >>>
> >>> ------------------------------------------------
> >>> mvn test -Dtests.filter="@nightly and not @slow"
> >>> ------------------------------------------------
> >>>
> >>> to run all nightly test but not the ones that are slow. `tests.filter`
> >>> supports
> >>> the boolean operators `and, or, not` and grouping ie:
> >>>
> >>> ---------------------------------------------------------------
> >>> mvn test -Dtests.filter="@nightly and not(@slow or @backwards)"
> >>> ---------------------------------------------------------------
> >>>
> >>> The same works for Lucene (-Dtests.filter=...), try it.
> >>>
> >>> Dawid
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> >>> For additional commands, e-mail: dev-help@lucene.apache.org
> >>>
> >> --
> >> - Mark
> >> about.me/markrmiller
> >
> > --
> > - Mark
> > about.me/markrmiller
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
> --
- Mark
about.me/markrmiller
Re: BadApple Nightly?
Posted by Dawid Weiss <da...@gmail.com>.
There has to be some distinction between nightly and regular runs --
for jenkins, we pass tests.nightly anyway to make this distinction
clear. You'd have to configure an appropriate filter for your jenkins
job.
> Is it possible to ignore specified tests as BadApples or AwaitsFix but only for nightly runs?
You can do it manually by adding @BeforeClass
assumeTrue(!"true".equals(System.getProperty("tests.nightly")))?
Dawid
On Wed, Jun 10, 2015 at 9:40 PM, Mark Miller <ma...@gmail.com> wrote:
> Hmm...is there any way to get this behavior somehow via just annotations
> though? It doesn't really work for my use case if it has to be specified
> from the command line.
>
> On Tue, Jun 9, 2015 at 2:46 PM Mark Miller <ma...@gmail.com> wrote:
>>
>> Thanks!
>>
>> On Tue, Jun 9, 2015 at 2:28 PM Dawid Weiss <da...@gmail.com> wrote:
>>>
>>> > don't have such persistent issues on non nightly runs. Is it possible
>>> > to
>>> > ignore specified tests as BadApples or AwaitsFix but only for nightly
>>> > runs?
>>>
>>> Sure, that's what test group filtering was added for. It's interesting
>>> that test-help doesn't show anything... ES has a more verbose info:
>>>
>>> You can also filter tests by certain annotations ie:
>>>
>>> * `@Slow` - tests that are know to take a long time to execute
>>> * `@Nightly` - tests that only run in nightly builds (disabled by
>>> default)
>>> * `@Integration` - integration tests
>>> * `@Backwards` - backwards compatibility tests (disabled by default)
>>> * `@AwaitsFix` - tests that are waiting for a bugfix (disabled by
>>> default)
>>> * `@BadApple` - tests that are known to fail randomly (disabled by
>>> default)
>>>
>>> Those annotation names can be combined into a filter expression like:
>>>
>>> ------------------------------------------------
>>> mvn test -Dtests.filter="@nightly and not @slow"
>>> ------------------------------------------------
>>>
>>> to run all nightly test but not the ones that are slow. `tests.filter`
>>> supports
>>> the boolean operators `and, or, not` and grouping ie:
>>>
>>> ---------------------------------------------------------------
>>> mvn test -Dtests.filter="@nightly and not(@slow or @backwards)"
>>> ---------------------------------------------------------------
>>>
>>> The same works for Lucene (-Dtests.filter=...), try it.
>>>
>>> Dawid
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>
>> --
>> - Mark
>> about.me/markrmiller
>
> --
> - Mark
> about.me/markrmiller
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: BadApple Nightly?
Posted by Mark Miller <ma...@gmail.com>.
Hmm...is there any way to get this behavior somehow via just annotations
though? It doesn't really work for my use case if it has to be specified
from the command line.
On Tue, Jun 9, 2015 at 2:46 PM Mark Miller <ma...@gmail.com> wrote:
> Thanks!
>
> On Tue, Jun 9, 2015 at 2:28 PM Dawid Weiss <da...@gmail.com> wrote:
>
>> > don't have such persistent issues on non nightly runs. Is it possible to
>> > ignore specified tests as BadApples or AwaitsFix but only for nightly
>> runs?
>>
>> Sure, that's what test group filtering was added for. It's interesting
>> that test-help doesn't show anything... ES has a more verbose info:
>>
>> You can also filter tests by certain annotations ie:
>>
>> * `@Slow` - tests that are know to take a long time to execute
>> * `@Nightly` - tests that only run in nightly builds (disabled by
>> default)
>> * `@Integration` - integration tests
>> * `@Backwards` - backwards compatibility tests (disabled by default)
>> * `@AwaitsFix` - tests that are waiting for a bugfix (disabled by
>> default)
>> * `@BadApple` - tests that are known to fail randomly (disabled by
>> default)
>>
>> Those annotation names can be combined into a filter expression like:
>>
>> ------------------------------------------------
>> mvn test -Dtests.filter="@nightly and not @slow"
>> ------------------------------------------------
>>
>> to run all nightly test but not the ones that are slow. `tests.filter`
>> supports
>> the boolean operators `and, or, not` and grouping ie:
>>
>> ---------------------------------------------------------------
>> mvn test -Dtests.filter="@nightly and not(@slow or @backwards)"
>> ---------------------------------------------------------------
>>
>> The same works for Lucene (-Dtests.filter=...), try it.
>>
>> Dawid
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>> --
> - Mark
> about.me/markrmiller
>
--
- Mark
about.me/markrmiller
Re: BadApple Nightly?
Posted by Mark Miller <ma...@gmail.com>.
Thanks!
On Tue, Jun 9, 2015 at 2:28 PM Dawid Weiss <da...@gmail.com> wrote:
> > don't have such persistent issues on non nightly runs. Is it possible to
> > ignore specified tests as BadApples or AwaitsFix but only for nightly
> runs?
>
> Sure, that's what test group filtering was added for. It's interesting
> that test-help doesn't show anything... ES has a more verbose info:
>
> You can also filter tests by certain annotations ie:
>
> * `@Slow` - tests that are know to take a long time to execute
> * `@Nightly` - tests that only run in nightly builds (disabled by
> default)
> * `@Integration` - integration tests
> * `@Backwards` - backwards compatibility tests (disabled by default)
> * `@AwaitsFix` - tests that are waiting for a bugfix (disabled by
> default)
> * `@BadApple` - tests that are known to fail randomly (disabled by
> default)
>
> Those annotation names can be combined into a filter expression like:
>
> ------------------------------------------------
> mvn test -Dtests.filter="@nightly and not @slow"
> ------------------------------------------------
>
> to run all nightly test but not the ones that are slow. `tests.filter`
> supports
> the boolean operators `and, or, not` and grouping ie:
>
> ---------------------------------------------------------------
> mvn test -Dtests.filter="@nightly and not(@slow or @backwards)"
> ---------------------------------------------------------------
>
> The same works for Lucene (-Dtests.filter=...), try it.
>
> Dawid
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
> --
- Mark
about.me/markrmiller
Re: BadApple Nightly?
Posted by Dawid Weiss <da...@gmail.com>.
> don't have such persistent issues on non nightly runs. Is it possible to
> ignore specified tests as BadApples or AwaitsFix but only for nightly runs?
Sure, that's what test group filtering was added for. It's interesting
that test-help doesn't show anything... ES has a more verbose info:
You can also filter tests by certain annotations ie:
* `@Slow` - tests that are know to take a long time to execute
* `@Nightly` - tests that only run in nightly builds (disabled by default)
* `@Integration` - integration tests
* `@Backwards` - backwards compatibility tests (disabled by default)
* `@AwaitsFix` - tests that are waiting for a bugfix (disabled by default)
* `@BadApple` - tests that are known to fail randomly (disabled by default)
Those annotation names can be combined into a filter expression like:
------------------------------------------------
mvn test -Dtests.filter="@nightly and not @slow"
------------------------------------------------
to run all nightly test but not the ones that are slow. `tests.filter` supports
the boolean operators `and, or, not` and grouping ie:
---------------------------------------------------------------
mvn test -Dtests.filter="@nightly and not(@slow or @backwards)"
---------------------------------------------------------------
The same works for Lucene (-Dtests.filter=...), try it.
Dawid
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org