You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Dawid Weiss <da...@gmail.com> on 2012/03/03 09:21:11 UTC
The two non-deterministic solr tests (BasicZkTest, BasicDistributedZkTest)
Guys, are these tests broken or is it something with the timing
(races)? They do fail for me pretty consistently although they are not
reproducible for any particular seed. Is anybody working on fixing
this? I realize distributed stuff is among the hardest to debug/ fix,
but I feel tempted to make a test group called
@KnownIssue(issue="URL") which would be disabled by default (and
different than plain @Ignore) because these two are really annoying.
Dawid
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: The two non-deterministic solr tests (BasicZkTest, BasicDistributedZkTest)
Posted by Mark Miller <ma...@gmail.com>.
Thanks Steve - I guess for some reason the Maven build is either more intensive or often runs when something else more intensive runs. Will keep an eye on this one to make sure we nailed it.
On Mar 3, 2012, at 2:03 PM, Steven A Rowe wrote:
> BasicDistributedZkTest just succeeded for the first time ever under the trunk Maven build! : https://builds.apache.org/job/Lucene-Solr-Maven-trunk/411/console
>
> (The build as a whole failed because Jenkins couldn't find test results, but that's a different problem.)
>
> Steve
>
> -----Original Message-----
> From: Mark Miller [mailto:markrmiller@gmail.com]
> Sent: Saturday, March 03, 2012 11:30 AM
> To: dev@lucene.apache.org
> Subject: Re: The two non-deterministic solr tests (BasicZkTest, BasicDistributedZkTest)
>
> Thanks Steve - went back in the history a bit and found the fail reports - will help to have it enabled so that I can try / receive feedback.
>
> On Mar 3, 2012, at 11:03 AM, Steven A Rowe wrote:
>
>> BasicDistributedZkTest is disabled on the Maven build (via a perl script inserting an @Ignore annotation in the source before compilation), because it has *never* succeeded there.
>>
>> I'll re-enable it.
>>
>> Steve
>>
>> -----Original Message-----
>> From: Mark Miller [mailto:markrmiller@gmail.com]
>> Sent: Saturday, March 03, 2012 10:52 AM
>> To: dev@lucene.apache.org
>> Subject: Re: The two non-deterministic solr tests (BasicZkTest,
>> BasicDistributedZkTest)
>>
>>
>> On Mar 3, 2012, at 10:36 AM, Mark Miller wrote:
>>
>>>
>>> On Mar 3, 2012, at 10:13 AM, Tommaso Teofili wrote:
>>>
>>>> they're only failing during the Maven job,
>>>
>>> Ah, I'll go have a look - haven't noticed these failures then.
>>
>> So I see the BasicZkTest failures here and I've made an attempted fix for really slow/busy machines, but I don't see the BasicDistributedZkTest failures there.
>>
>> - Mark Miller
>> lucidimagination.com
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
>> additional commands, e-mail: dev-help@lucene.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
>> additional commands, e-mail: dev-help@lucene.apache.org
>>
>
> - Mark Miller
> lucidimagination.com
>
>
>
>
>
>
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional commands, e-mail: dev-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
- Mark Miller
lucidimagination.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
RE: The two non-deterministic solr tests (BasicZkTest,
BasicDistributedZkTest)
Posted by Steven A Rowe <sa...@syr.edu>.
BasicDistributedZkTest just succeeded for the first time ever under the trunk Maven build! : https://builds.apache.org/job/Lucene-Solr-Maven-trunk/411/console
(The build as a whole failed because Jenkins couldn't find test results, but that's a different problem.)
Steve
-----Original Message-----
From: Mark Miller [mailto:markrmiller@gmail.com]
Sent: Saturday, March 03, 2012 11:30 AM
To: dev@lucene.apache.org
Subject: Re: The two non-deterministic solr tests (BasicZkTest, BasicDistributedZkTest)
Thanks Steve - went back in the history a bit and found the fail reports - will help to have it enabled so that I can try / receive feedback.
On Mar 3, 2012, at 11:03 AM, Steven A Rowe wrote:
> BasicDistributedZkTest is disabled on the Maven build (via a perl script inserting an @Ignore annotation in the source before compilation), because it has *never* succeeded there.
>
> I'll re-enable it.
>
> Steve
>
> -----Original Message-----
> From: Mark Miller [mailto:markrmiller@gmail.com]
> Sent: Saturday, March 03, 2012 10:52 AM
> To: dev@lucene.apache.org
> Subject: Re: The two non-deterministic solr tests (BasicZkTest,
> BasicDistributedZkTest)
>
>
> On Mar 3, 2012, at 10:36 AM, Mark Miller wrote:
>
>>
>> On Mar 3, 2012, at 10:13 AM, Tommaso Teofili wrote:
>>
>>> they're only failing during the Maven job,
>>
>> Ah, I'll go have a look - haven't noticed these failures then.
>
> So I see the BasicZkTest failures here and I've made an attempted fix for really slow/busy machines, but I don't see the BasicDistributedZkTest failures there.
>
> - Mark Miller
> lucidimagination.com
>
>
>
>
>
>
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
> additional commands, e-mail: dev-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
> additional commands, e-mail: dev-help@lucene.apache.org
>
- Mark Miller
lucidimagination.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional commands, e-mail: dev-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: The two non-deterministic solr tests (BasicZkTest, BasicDistributedZkTest)
Posted by Dawid Weiss <da...@cs.put.poznan.pl>.
Hi Mark.
I think the problem may be with the background load. What I reported
happens very often on the branch where I'm using that custom ant tasks
for spawning parallel tests. This happens on many machines (linux box
with 2 cores, windows with 4 cores, linux with 4 cores) but again -- I
wasn't successful at replicating consistently. Seems like a time-based
condition somewhere and you're probably right in that it depends on
background load.
I will try to reproduce this on trunk and will create an issue with logs.
Dawid
On Sat, Mar 3, 2012 at 5:29 PM, Mark Miller <ma...@gmail.com> wrote:
> Thanks Steve - went back in the history a bit and found the fail reports - will help to have it enabled so that I can try / receive feedback.
>
> On Mar 3, 2012, at 11:03 AM, Steven A Rowe wrote:
>
>> BasicDistributedZkTest is disabled on the Maven build (via a perl script inserting an @Ignore annotation in the source before compilation), because it has *never* succeeded there.
>>
>> I'll re-enable it.
>>
>> Steve
>>
>> -----Original Message-----
>> From: Mark Miller [mailto:markrmiller@gmail.com]
>> Sent: Saturday, March 03, 2012 10:52 AM
>> To: dev@lucene.apache.org
>> Subject: Re: The two non-deterministic solr tests (BasicZkTest, BasicDistributedZkTest)
>>
>>
>> On Mar 3, 2012, at 10:36 AM, Mark Miller wrote:
>>
>>>
>>> On Mar 3, 2012, at 10:13 AM, Tommaso Teofili wrote:
>>>
>>>> they're only failing during the Maven job,
>>>
>>> Ah, I'll go have a look - haven't noticed these failures then.
>>
>> So I see the BasicZkTest failures here and I've made an attempted fix for really slow/busy machines, but I don't see the BasicDistributedZkTest failures there.
>>
>> - Mark Miller
>> lucidimagination.com
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>
> - Mark Miller
> lucidimagination.com
>
>
>
>
>
>
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: The two non-deterministic solr tests (BasicZkTest, BasicDistributedZkTest)
Posted by Mark Miller <ma...@gmail.com>.
Thanks Steve - went back in the history a bit and found the fail reports - will help to have it enabled so that I can try / receive feedback.
On Mar 3, 2012, at 11:03 AM, Steven A Rowe wrote:
> BasicDistributedZkTest is disabled on the Maven build (via a perl script inserting an @Ignore annotation in the source before compilation), because it has *never* succeeded there.
>
> I'll re-enable it.
>
> Steve
>
> -----Original Message-----
> From: Mark Miller [mailto:markrmiller@gmail.com]
> Sent: Saturday, March 03, 2012 10:52 AM
> To: dev@lucene.apache.org
> Subject: Re: The two non-deterministic solr tests (BasicZkTest, BasicDistributedZkTest)
>
>
> On Mar 3, 2012, at 10:36 AM, Mark Miller wrote:
>
>>
>> On Mar 3, 2012, at 10:13 AM, Tommaso Teofili wrote:
>>
>>> they're only failing during the Maven job,
>>
>> Ah, I'll go have a look - haven't noticed these failures then.
>
> So I see the BasicZkTest failures here and I've made an attempted fix for really slow/busy machines, but I don't see the BasicDistributedZkTest failures there.
>
> - Mark Miller
> lucidimagination.com
>
>
>
>
>
>
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional commands, e-mail: dev-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
- Mark Miller
lucidimagination.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
RE: The two non-deterministic solr tests (BasicZkTest,
BasicDistributedZkTest)
Posted by Steven A Rowe <sa...@syr.edu>.
BasicDistributedZkTest is disabled on the Maven build (via a perl script inserting an @Ignore annotation in the source before compilation), because it has *never* succeeded there.
I'll re-enable it.
Steve
-----Original Message-----
From: Mark Miller [mailto:markrmiller@gmail.com]
Sent: Saturday, March 03, 2012 10:52 AM
To: dev@lucene.apache.org
Subject: Re: The two non-deterministic solr tests (BasicZkTest, BasicDistributedZkTest)
On Mar 3, 2012, at 10:36 AM, Mark Miller wrote:
>
> On Mar 3, 2012, at 10:13 AM, Tommaso Teofili wrote:
>
>> they're only failing during the Maven job,
>
> Ah, I'll go have a look - haven't noticed these failures then.
So I see the BasicZkTest failures here and I've made an attempted fix for really slow/busy machines, but I don't see the BasicDistributedZkTest failures there.
- Mark Miller
lucidimagination.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional commands, e-mail: dev-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: The two non-deterministic solr tests (BasicZkTest, BasicDistributedZkTest)
Posted by Mark Miller <ma...@gmail.com>.
On Mar 3, 2012, at 10:36 AM, Mark Miller wrote:
>
> On Mar 3, 2012, at 10:13 AM, Tommaso Teofili wrote:
>
>> they're only failing during the Maven job,
>
> Ah, I'll go have a look - haven't noticed these failures then.
So I see the BasicZkTest failures here and I've made an attempted fix for really slow/busy machines, but I don't see the BasicDistributedZkTest failures there.
- Mark Miller
lucidimagination.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: The two non-deterministic solr tests (BasicZkTest, BasicDistributedZkTest)
Posted by Mark Miller <ma...@gmail.com>.
On Mar 3, 2012, at 10:13 AM, Tommaso Teofili wrote:
> they're only failing during the Maven job,
Ah, I'll go have a look - haven't noticed these failures then.
- Mark Miller
lucidimagination.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: The two non-deterministic solr tests (BasicZkTest, BasicDistributedZkTest)
Posted by Tommaso Teofili <to...@gmail.com>.
As far as I can see from Jenkins build reports it seems they're only
failing during the Maven job, not for ant, is that something that is true
for you as well Dawid?.
I think at leaste the BasicZkTest failure depends (also?) on the amount of
load on the machine during the tests which makes the timeout pass or not
(and also the Thread.sleep(50) at line 406 of ZkStateReader is a candidate
for inspection).
However I agree we'd better track those failures in a dedicate Jira issue.
Tommaso
2012/3/3 Mark Miller <ma...@gmail.com>
>
> On Mar 3, 2012, at 3:21 AM, Dawid Weiss wrote:
>
> > Guys, are these tests broken or is it something with the timing
> > (races)? They do fail for me pretty consistently although they are not
> > reproducible for any particular seed. Is anybody working on fixing
> > this? I realize distributed stuff is among the hardest to debug/ fix,
> > but I feel tempted to make a test group called
> > @KnownIssue(issue="URL") which would be disabled by default (and
> > different than plain @Ignore) because these two are really annoying.
> >
> > Dawid
>
>
> Hey Dawid - can you make an issue with the details? I know you sent me a
> bit of info in the past, but I was not able to replicate.
>
> Currently we are going on what fails on jenkins or local dev machines, and
> I just don't see these consistent fails, so there is not a lot I can do yet.
>
> If it passes on jenkins and it passes on my local machines, the problem is
> invisible to me. So if you are seeing fails on these, lets start tracking
> in an issue.
>
> - Mark Miller
> lucidimagination.com
>
>
>
>
>
>
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
Re: The two non-deterministic solr tests (BasicZkTest, BasicDistributedZkTest)
Posted by Mark Miller <ma...@gmail.com>.
On Mar 3, 2012, at 3:21 AM, Dawid Weiss wrote:
> Guys, are these tests broken or is it something with the timing
> (races)? They do fail for me pretty consistently although they are not
> reproducible for any particular seed. Is anybody working on fixing
> this? I realize distributed stuff is among the hardest to debug/ fix,
> but I feel tempted to make a test group called
> @KnownIssue(issue="URL") which would be disabled by default (and
> different than plain @Ignore) because these two are really annoying.
>
> Dawid
Hey Dawid - can you make an issue with the details? I know you sent me a bit of info in the past, but I was not able to replicate.
Currently we are going on what fails on jenkins or local dev machines, and I just don't see these consistent fails, so there is not a lot I can do yet.
If it passes on jenkins and it passes on my local machines, the problem is invisible to me. So if you are seeing fails on these, lets start tracking in an issue.
- Mark Miller
lucidimagination.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: The two non-deterministic solr tests (BasicZkTest, BasicDistributedZkTest)
Posted by Dawid Weiss <da...@cs.put.poznan.pl>.
Thanks Tommaso!
Dawid
On Sat, Mar 3, 2012 at 10:20 AM, Tommaso Teofili
<to...@gmail.com> wrote:
> Hi Dawid,
> I'll have a look at those tests and see if I can figure out what the problem
> is.
> Tommaso
>
> 2012/3/3 Dawid Weiss <da...@gmail.com>
>>
>> Guys, are these tests broken or is it something with the timing
>> (races)? They do fail for me pretty consistently although they are not
>> reproducible for any particular seed. Is anybody working on fixing
>> this? I realize distributed stuff is among the hardest to debug/ fix,
>> but I feel tempted to make a test group called
>> @KnownIssue(issue="URL") which would be disabled by default (and
>> different than plain @Ignore) because these two are really annoying.
>>
>> Dawid
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: The two non-deterministic solr tests (BasicZkTest, BasicDistributedZkTest)
Posted by Tommaso Teofili <to...@gmail.com>.
Hi Dawid,
I'll have a look at those tests and see if I can figure out what the
problem is.
Tommaso
2012/3/3 Dawid Weiss <da...@gmail.com>
> Guys, are these tests broken or is it something with the timing
> (races)? They do fail for me pretty consistently although they are not
> reproducible for any particular seed. Is anybody working on fixing
> this? I realize distributed stuff is among the hardest to debug/ fix,
> but I feel tempted to make a test group called
> @KnownIssue(issue="URL") which would be disabled by default (and
> different than plain @Ignore) because these two are really annoying.
>
> Dawid
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>