You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Shawn Heisey <ap...@elyograg.org> on 2015/09/11 15:06:34 UTC
test suite counter skipping numbers?
While running tests, I noticed this:
[junit4] Suite: org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest
[junit4] IGNOR/A 0.00s J1 | HdfsChaosMonkeySafeLeaderTest.test
[junit4] > Assumption #1: 'nightly' test group is disabled
(@Nightly(value=))
[junit4] Completed [515/545] on J1 in 0.00s, 1 test, 1 skipped
[junit4]
[junit4] Suite: org.apache.solr.internal.csv.writer.CSVWriterTest
[junit4] Completed [516/545] on J0 in 0.00s, 2 tests
[junit4]
[junit4] Suite: org.apache.solr.cloud.CdcrReplicationDistributedZkTest
[junit4] IGNOR/A 0.00s J1 | CdcrReplicationDistributedZkTest.doTests
[junit4] > Assumption #1: 'nightly' test group is disabled
(@Nightly())
[junit4] Completed [517/545] on J1 in 0.00s, 1 test, 1 skipped
[junit4]
[junit4] Suite: org.apache.solr.internal.csv.writer.CSVFieldTest
[junit4] Completed [520/545] on J1 in 0.00s, 2 tests
[junit4]
[junit4] Suite: org.apache.solr.TestCrossCoreJoin
[junit4] Completed [522/545] on J1 in 1.38s, 3 tests
Suite numbers 518, 519,and 521 did not display.
Any idea what's up there?
Thanks,
Shawn
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: test suite counter skipping numbers?
Posted by Dawid Weiss <da...@gmail.com>.
Done, Hoss.
https://issues.apache.org/jira/browse/LUCENE-6924
D.
On Mon, Nov 16, 2015 at 9:13 AM, Dawid Weiss <da...@gmail.com> wrote:
> Yeah... it'd be perfect to make it somehow configurable so that tweaks
> like this can be done without changing the code. I'll see what I can
> do.
>
> https://github.com/randomizedtesting/randomizedtesting/issues/220
>
> By the way -- if any of you guys is using argument factory methods
> I've added a nicer way to provide their names without the need to wrap
> the argument in a custom class, etc.
>
> https://github.com/randomizedtesting/randomizedtesting/releases/tag/release%2F2.3.0
>
> Dawid
>
> On Sun, Nov 15, 2015 at 10:21 PM, Chris Hostetter
> <ho...@fucit.org> wrote:
>>
>> : > Tests summary: 100 suites (2 ignored), xx tests, xx ignored (xx assumptions).
>> :
>> : Will do it in the next version. Thanks for letting me know.
>>
>> Dawid: that reminds me, if you are tweaking the logging, it would also be
>> helpful if the "Completed [X/Y] ..." line after each test also indicated
>> if there had been any failures so far, so you can see at a glance in your
>> terminal if any thing has failed up to this point (w/o needing to scroll
>> back, or wait for the ant command to complete.
>>
>> for example...
>>
>>>> [junit4] Completed [1/545] on J1 in 17.77s, 3 tests, 1 skipped
>>
>> ..just started, everything is all good..
>>
>>>> [junit4] Completed [25/545 (1!)] on J3 in 7.35s, 5 tests, 5 skipped
>>
>> ..a suite just had failure, every "Completed" line after that will include
>> "(1!)" until another quite has a failure...
>>
>>>> [junit4] Completed [26/545 (1!)] on J0 in 1.34s, 42 tests, 1 skipped
>>>> [junit4] Completed [27/545 (1!)] on J1 in 3.45s, 31 tests, 4 skipped
>>
>> ... if you glance at your terminal now, you know to scroll up because
>> something failed earlier...
>>
>> If another test fails later the output changes a bit...
>>
>>>> [junit4] Completed [145/545 (2!)] on J1 in 13.73s, 42 tests, 1 skipped
>>
>> ...etc...
>>
>>
>>
>> -Hoss
>> http://www.lucidworks.com/
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: test suite counter skipping numbers?
Posted by Dawid Weiss <da...@gmail.com>.
Yeah... it'd be perfect to make it somehow configurable so that tweaks
like this can be done without changing the code. I'll see what I can
do.
https://github.com/randomizedtesting/randomizedtesting/issues/220
By the way -- if any of you guys is using argument factory methods
I've added a nicer way to provide their names without the need to wrap
the argument in a custom class, etc.
https://github.com/randomizedtesting/randomizedtesting/releases/tag/release%2F2.3.0
Dawid
On Sun, Nov 15, 2015 at 10:21 PM, Chris Hostetter
<ho...@fucit.org> wrote:
>
> : > Tests summary: 100 suites (2 ignored), xx tests, xx ignored (xx assumptions).
> :
> : Will do it in the next version. Thanks for letting me know.
>
> Dawid: that reminds me, if you are tweaking the logging, it would also be
> helpful if the "Completed [X/Y] ..." line after each test also indicated
> if there had been any failures so far, so you can see at a glance in your
> terminal if any thing has failed up to this point (w/o needing to scroll
> back, or wait for the ant command to complete.
>
> for example...
>
>>> [junit4] Completed [1/545] on J1 in 17.77s, 3 tests, 1 skipped
>
> ..just started, everything is all good..
>
>>> [junit4] Completed [25/545 (1!)] on J3 in 7.35s, 5 tests, 5 skipped
>
> ..a suite just had failure, every "Completed" line after that will include
> "(1!)" until another quite has a failure...
>
>>> [junit4] Completed [26/545 (1!)] on J0 in 1.34s, 42 tests, 1 skipped
>>> [junit4] Completed [27/545 (1!)] on J1 in 3.45s, 31 tests, 4 skipped
>
> ... if you glance at your terminal now, you know to scroll up because
> something failed earlier...
>
> If another test fails later the output changes a bit...
>
>>> [junit4] Completed [145/545 (2!)] on J1 in 13.73s, 42 tests, 1 skipped
>
> ...etc...
>
>
>
> -Hoss
> http://www.lucidworks.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: test suite counter skipping numbers?
Posted by Chris Hostetter <ho...@fucit.org>.
: > Tests summary: 100 suites (2 ignored), xx tests, xx ignored (xx assumptions).
:
: Will do it in the next version. Thanks for letting me know.
Dawid: that reminds me, if you are tweaking the logging, it would also be
helpful if the "Completed [X/Y] ..." line after each test also indicated
if there had been any failures so far, so you can see at a glance in your
terminal if any thing has failed up to this point (w/o needing to scroll
back, or wait for the ant command to complete.
for example...
>> [junit4] Completed [1/545] on J1 in 17.77s, 3 tests, 1 skipped
..just started, everything is all good..
>> [junit4] Completed [25/545 (1!)] on J3 in 7.35s, 5 tests, 5 skipped
..a suite just had failure, every "Completed" line after that will include
"(1!)" until another quite has a failure...
>> [junit4] Completed [26/545 (1!)] on J0 in 1.34s, 42 tests, 1 skipped
>> [junit4] Completed [27/545 (1!)] on J1 in 3.45s, 31 tests, 4 skipped
... if you glance at your terminal now, you know to scroll up because
something failed earlier...
If another test fails later the output changes a bit...
>> [junit4] Completed [145/545 (2!)] on J1 in 13.73s, 42 tests, 1 skipped
...etc...
-Hoss
http://www.lucidworks.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: test suite counter skipping numbers?
Posted by Dawid Weiss <da...@gmail.com>.
Hi Shawn,
Suites which didn't have failures (in setup/teardown) and had zero
tests are omitted from reporting. I think it'd make more sense to
report the total number of suites and the actual number of suites
separately in the summary, leaving the counter without gaps, for
example:
> Tests summary: 100 suites (2 ignored), xx tests, xx ignored (xx assumptions).
Will do it in the next version. Thanks for letting me know.
Dawid
On Fri, Sep 11, 2015 at 3:26 PM, Shawn Heisey <ap...@elyograg.org> wrote:
> On 9/11/2015 7:06 AM, Shawn Heisey wrote:
>> [junit4] Suite: org.apache.solr.TestCrossCoreJoin
>> [junit4] Completed [522/545] on J1 in 1.38s, 3 tests
>>
>> Suite numbers 518, 519,and 521 did not display.
>>
>> Any idea what's up there?
>
> I did not check all 545 results to see if there were more than those
> three missing. The ones I know are missing did not not seem to affect
> the overall count. From the end of that test run:
>
> [junit4] Suite: org.apache.solr.util.TestSolrCLIRunExample
> [junit4] Completed [545/545] on J1 in 17.77s, 3 tests, 1 skipped
> [junit4]
> [junit4] JVM J0: 1.26 .. 3060.03 = 3058.78s
> [junit4] JVM J1: 1.46 .. 3068.68 = 3067.22s
> [junit4] Execution time total: 51 minutes 8 seconds
> [junit4] Tests summary: 545 suites, 2143 tests, 47 ignored (35
> assumptions)
>
> Thanks,
> Shawn
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: test suite counter skipping numbers?
Posted by Shawn Heisey <ap...@elyograg.org>.
On 9/11/2015 7:06 AM, Shawn Heisey wrote:
> [junit4] Suite: org.apache.solr.TestCrossCoreJoin
> [junit4] Completed [522/545] on J1 in 1.38s, 3 tests
>
> Suite numbers 518, 519,and 521 did not display.
>
> Any idea what's up there?
I did not check all 545 results to see if there were more than those
three missing. The ones I know are missing did not not seem to affect
the overall count. From the end of that test run:
[junit4] Suite: org.apache.solr.util.TestSolrCLIRunExample
[junit4] Completed [545/545] on J1 in 17.77s, 3 tests, 1 skipped
[junit4]
[junit4] JVM J0: 1.26 .. 3060.03 = 3058.78s
[junit4] JVM J1: 1.46 .. 3068.68 = 3067.22s
[junit4] Execution time total: 51 minutes 8 seconds
[junit4] Tests summary: 545 suites, 2143 tests, 47 ignored (35
assumptions)
Thanks,
Shawn
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: test suite counter skipping numbers?
Posted by Dawid Weiss <da...@gmail.com>.
I am on vacation this week but I will check what's going on. Either the
message is incorrect or (more likely) certain suites are skipped entirely
because of their annotations ( ignored group). Will look into it.
On Sep 11, 2015 4:13 PM, "Shawn Heisey" <ap...@elyograg.org> wrote:
> On 9/11/2015 7:08 AM, Dawid Weiss wrote:
> >
> > These are tests, not suites. A suite is a class - a collection of many
> > tests.
>
> I admit that the following is silly nitpicking on my part, and not
> terribly relevant to a discussion about why some of the output seems to
> be missing: Other parts of the test output would seem to disagree with
> that statement. Further back in the session:
>
> -test:
> [mkdir] Created dir: /home/elyograg/asf/trunk/solr/build/solr-core/test
> [mkdir] Created dir:
> /home/elyograg/asf/trunk/solr/build/solr-core/test/temp
> [junit4] <JUnit4> says ciao! Master seed: F3E8A9FC6DA9AEF7
> [junit4] Executing 545 suites with 2 JVMs.
>
> Thanks,
> Shawn
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
Re: test suite counter skipping numbers?
Posted by Shawn Heisey <ap...@elyograg.org>.
On 9/11/2015 7:08 AM, Dawid Weiss wrote:
>
> These are tests, not suites. A suite is a class - a collection of many
> tests.
I admit that the following is silly nitpicking on my part, and not
terribly relevant to a discussion about why some of the output seems to
be missing: Other parts of the test output would seem to disagree with
that statement. Further back in the session:
-test:
[mkdir] Created dir: /home/elyograg/asf/trunk/solr/build/solr-core/test
[mkdir] Created dir:
/home/elyograg/asf/trunk/solr/build/solr-core/test/temp
[junit4] <JUnit4> says ciao! Master seed: F3E8A9FC6DA9AEF7
[junit4] Executing 545 suites with 2 JVMs.
Thanks,
Shawn
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: test suite counter skipping numbers?
Posted by Dawid Weiss <da...@gmail.com>.
These are tests, not suites. A suite is a class - a collection of many
tests.
On Sep 11, 2015 4:06 PM, "Shawn Heisey" <ap...@elyograg.org> wrote:
> While running tests, I noticed this:
>
> [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest
> [junit4] IGNOR/A 0.00s J1 | HdfsChaosMonkeySafeLeaderTest.test
> [junit4] > Assumption #1: 'nightly' test group is disabled
> (@Nightly(value=))
> [junit4] Completed [515/545] on J1 in 0.00s, 1 test, 1 skipped
> [junit4]
> [junit4] Suite: org.apache.solr.internal.csv.writer.CSVWriterTest
> [junit4] Completed [516/545] on J0 in 0.00s, 2 tests
> [junit4]
> [junit4] Suite: org.apache.solr.cloud.CdcrReplicationDistributedZkTest
> [junit4] IGNOR/A 0.00s J1 | CdcrReplicationDistributedZkTest.doTests
> [junit4] > Assumption #1: 'nightly' test group is disabled
> (@Nightly())
> [junit4] Completed [517/545] on J1 in 0.00s, 1 test, 1 skipped
> [junit4]
> [junit4] Suite: org.apache.solr.internal.csv.writer.CSVFieldTest
> [junit4] Completed [520/545] on J1 in 0.00s, 2 tests
> [junit4]
> [junit4] Suite: org.apache.solr.TestCrossCoreJoin
> [junit4] Completed [522/545] on J1 in 1.38s, 3 tests
>
> Suite numbers 518, 519,and 521 did not display.
>
> Any idea what's up there?
>
> Thanks,
> Shawn
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>