You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Benson Margulies <bi...@gmail.com> on 2014/03/21 18:57:59 UTC
Threads in the LuceneTestCase system
I'm fighting with a test that uses the random analysis chain testing.
It does not repro when I pass in the usual collection of -D's. I think
that the reason is to do with threads; the failure is always on a big
multicore build machine.
Are there any more of those Carrot control -D's that change how many
threads are in the act?
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Threads in the LuceneTestCase system
Posted by Robert Muir <rc...@gmail.com>.
Right, i think its always this way with such bugs? Sometimes you have
to run them a few hundred times in a loop or whatever.
Sometimes it helps to 'make your machine busy' by doing somethign
crazy in another shell window, like run lucene tests.
My point is just that if you hit a failure coming from multiple
threads in basetokenstreamtestcase, it should nearly always mean that
its a thread safety issue, because it previously just ran the the same
"seed" through single-threaded. Nobody wants to debug threads if its a
simple bug that has nothing to do with thread safety. So if you find
its not a thread safety issue, then basetokenstreamtestcase has a bug.
On Fri, Mar 21, 2014 at 2:05 PM, Benson Margulies <bi...@gmail.com> wrote:
> To be clear, it's my own code that is amiss here, nothing in Lucene itself.
>
> I just fixed a thread-safety bug, but I just saw another failure, and
> I'm pulling my hair because it refuses to repro.
>
>
> On Fri, Mar 21, 2014 at 2:03 PM, Robert Muir <rc...@gmail.com> wrote:
>> Do you have a stacktrace of where in BaseTokenStreamTestCase that it hit?
>>
>> We refactored the asserting here, to always test with a single thread
>> *first*, then with multiple threads.
>>
>> So if you are failing in the multithreaded part, it means you have a
>> thread safety issue...
>>
>> (as long as this part of the base test class is working, and i hope it
>> still is, i havent seen anything crazy to indicate otherwise, and i've
>> been in TestRandomChains for a few hours this week)
>>
>> On Fri, Mar 21, 2014 at 1:57 PM, Benson Margulies <bi...@gmail.com> wrote:
>>> I'm fighting with a test that uses the random analysis chain testing.
>>> It does not repro when I pass in the usual collection of -D's. I think
>>> that the reason is to do with threads; the failure is always on a big
>>> multicore build machine.
>>>
>>> Are there any more of those Carrot control -D's that change how many
>>> threads are in the act?
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Threads in the LuceneTestCase system
Posted by Robert Muir <rc...@gmail.com>.
I put it in 4.x too. I tested locally by introducing some thread
hazards into some analyzers and it seemed to fail nice and fast.
but please let me know if for some reason it makes matters worse.
On Fri, Mar 21, 2014 at 2:57 PM, Benson Margulies <bi...@gmail.com> wrote:
> Yes, but in a few hours. THis test currently runs with Lucene 4.1, and
> I'll need to move some furniture to use it with a trunk-y environment.
>
> On Fri, Mar 21, 2014 at 2:54 PM, Robert Muir <rc...@gmail.com> wrote:
>> Can you try http://svn.apache.org/r1580020 and tell me if it is better
>> for catching/reproducing thread safety issues?
>>
>> On Fri, Mar 21, 2014 at 2:48 PM, Benson Margulies <bi...@gmail.com> wrote:
>>> Yea, right now I have this failure that repros every time on a big
>>> computer and never on my not-so-small MacBook Pro.
>>>
>>>
>>> On Fri, Mar 21, 2014 at 2:43 PM, Robert Muir <rc...@gmail.com> wrote:
>>>> I just reviewed the code thinking of how to make it easier to
>>>> reproduce issues, we should give this test class a startingGun.
>>>>
>>>> On Fri, Mar 21, 2014 at 2:38 PM, Benson Margulies <bi...@gmail.com> wrote:
>>>>> I could share it right here, but in any case I just found _another_
>>>>> stupid mistake where I was doing something in which multiple analyzers
>>>>> would end up sharing something unsharable.
>>>>>
>>>>>
>>>>> build 21-Mar-2014 10:36:59
>>>>> testRandomStressWithBasisTokenizer(com.basistech.rosette.lucene.BaseLinguisticsTokenFilterTest)
>>>>> Time elapsed: 9.249 sec <<< FAILURE!
>>>>> build 21-Mar-2014 10:36:59 org.junit.ComparisonFailure: term 5
>>>>> expected:<[kiiiintii?j?rjesstelm?]> but was:<[ssiikojenen]>
>>>>> build 21-Mar-2014 10:36:59 at
>>>>> __randomizedtesting.SeedInfo.seed([A574C0CEE3A9C8A3:47C87E8DE3A08C35]:0)
>>>>> build 21-Mar-2014 10:36:59 at org.junit.Assert.assertEquals(Assert.java:115)
>>>>> build 21-Mar-2014 10:36:59 at
>>>>> org.apache.lucene.analysis.BaseTokenStreamTestCase.assertTokenStreamContents(BaseTokenStreamTestCase.java:169)
>>>>> build 21-Mar-2014 10:36:59 at
>>>>> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkAnalysisConsistency(BaseTokenStreamTestCase.java:747)
>>>>> build 21-Mar-2014 10:36:59 at
>>>>> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:546)
>>>>> build 21-Mar-2014 10:36:59 at
>>>>> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:447)
>>>>> build 21-Mar-2014 10:36:59 at
>>>>> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:375)
>>>>>
>>>>>
>>>>> On Fri, Mar 21, 2014 at 2:33 PM, Dawid Weiss
>>>>> <da...@cs.put.poznan.pl> wrote:
>>>>>>> I just fixed a thread-safety bug, but I just saw another failure, and
>>>>>>> I'm pulling my hair because it refuses to repro.
>>>>>>
>>>>>> You can run with a single JVM by passing -Dtests.jvms=1 (or so I
>>>>>> believe; try ant test-help). This shouldn't affect multi-threaded
>>>>>> tests but if you have a problem with dependency between suites (test
>>>>>> classes), such as some values from preinitialized static fields or the
>>>>>> like then it may be the cause.
>>>>>>
>>>>>> Otherwise the test framework makes it really simple: if a thread was
>>>>>> created within a test class then it should die before the test class
>>>>>> completes. What is the exception (stack?) you're getting? Can you
>>>>>> share it privately?
>>>>>>
>>>>>> Dawid
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Threads in the LuceneTestCase system
Posted by Benson Margulies <bi...@gmail.com>.
Yes, but in a few hours. THis test currently runs with Lucene 4.1, and
I'll need to move some furniture to use it with a trunk-y environment.
On Fri, Mar 21, 2014 at 2:54 PM, Robert Muir <rc...@gmail.com> wrote:
> Can you try http://svn.apache.org/r1580020 and tell me if it is better
> for catching/reproducing thread safety issues?
>
> On Fri, Mar 21, 2014 at 2:48 PM, Benson Margulies <bi...@gmail.com> wrote:
>> Yea, right now I have this failure that repros every time on a big
>> computer and never on my not-so-small MacBook Pro.
>>
>>
>> On Fri, Mar 21, 2014 at 2:43 PM, Robert Muir <rc...@gmail.com> wrote:
>>> I just reviewed the code thinking of how to make it easier to
>>> reproduce issues, we should give this test class a startingGun.
>>>
>>> On Fri, Mar 21, 2014 at 2:38 PM, Benson Margulies <bi...@gmail.com> wrote:
>>>> I could share it right here, but in any case I just found _another_
>>>> stupid mistake where I was doing something in which multiple analyzers
>>>> would end up sharing something unsharable.
>>>>
>>>>
>>>> build 21-Mar-2014 10:36:59
>>>> testRandomStressWithBasisTokenizer(com.basistech.rosette.lucene.BaseLinguisticsTokenFilterTest)
>>>> Time elapsed: 9.249 sec <<< FAILURE!
>>>> build 21-Mar-2014 10:36:59 org.junit.ComparisonFailure: term 5
>>>> expected:<[kiiiintii?j?rjesstelm?]> but was:<[ssiikojenen]>
>>>> build 21-Mar-2014 10:36:59 at
>>>> __randomizedtesting.SeedInfo.seed([A574C0CEE3A9C8A3:47C87E8DE3A08C35]:0)
>>>> build 21-Mar-2014 10:36:59 at org.junit.Assert.assertEquals(Assert.java:115)
>>>> build 21-Mar-2014 10:36:59 at
>>>> org.apache.lucene.analysis.BaseTokenStreamTestCase.assertTokenStreamContents(BaseTokenStreamTestCase.java:169)
>>>> build 21-Mar-2014 10:36:59 at
>>>> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkAnalysisConsistency(BaseTokenStreamTestCase.java:747)
>>>> build 21-Mar-2014 10:36:59 at
>>>> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:546)
>>>> build 21-Mar-2014 10:36:59 at
>>>> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:447)
>>>> build 21-Mar-2014 10:36:59 at
>>>> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:375)
>>>>
>>>>
>>>> On Fri, Mar 21, 2014 at 2:33 PM, Dawid Weiss
>>>> <da...@cs.put.poznan.pl> wrote:
>>>>>> I just fixed a thread-safety bug, but I just saw another failure, and
>>>>>> I'm pulling my hair because it refuses to repro.
>>>>>
>>>>> You can run with a single JVM by passing -Dtests.jvms=1 (or so I
>>>>> believe; try ant test-help). This shouldn't affect multi-threaded
>>>>> tests but if you have a problem with dependency between suites (test
>>>>> classes), such as some values from preinitialized static fields or the
>>>>> like then it may be the cause.
>>>>>
>>>>> Otherwise the test framework makes it really simple: if a thread was
>>>>> created within a test class then it should die before the test class
>>>>> completes. What is the exception (stack?) you're getting? Can you
>>>>> share it privately?
>>>>>
>>>>> Dawid
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Threads in the LuceneTestCase system
Posted by Robert Muir <rc...@gmail.com>.
Can you try http://svn.apache.org/r1580020 and tell me if it is better
for catching/reproducing thread safety issues?
On Fri, Mar 21, 2014 at 2:48 PM, Benson Margulies <bi...@gmail.com> wrote:
> Yea, right now I have this failure that repros every time on a big
> computer and never on my not-so-small MacBook Pro.
>
>
> On Fri, Mar 21, 2014 at 2:43 PM, Robert Muir <rc...@gmail.com> wrote:
>> I just reviewed the code thinking of how to make it easier to
>> reproduce issues, we should give this test class a startingGun.
>>
>> On Fri, Mar 21, 2014 at 2:38 PM, Benson Margulies <bi...@gmail.com> wrote:
>>> I could share it right here, but in any case I just found _another_
>>> stupid mistake where I was doing something in which multiple analyzers
>>> would end up sharing something unsharable.
>>>
>>>
>>> build 21-Mar-2014 10:36:59
>>> testRandomStressWithBasisTokenizer(com.basistech.rosette.lucene.BaseLinguisticsTokenFilterTest)
>>> Time elapsed: 9.249 sec <<< FAILURE!
>>> build 21-Mar-2014 10:36:59 org.junit.ComparisonFailure: term 5
>>> expected:<[kiiiintii?j?rjesstelm?]> but was:<[ssiikojenen]>
>>> build 21-Mar-2014 10:36:59 at
>>> __randomizedtesting.SeedInfo.seed([A574C0CEE3A9C8A3:47C87E8DE3A08C35]:0)
>>> build 21-Mar-2014 10:36:59 at org.junit.Assert.assertEquals(Assert.java:115)
>>> build 21-Mar-2014 10:36:59 at
>>> org.apache.lucene.analysis.BaseTokenStreamTestCase.assertTokenStreamContents(BaseTokenStreamTestCase.java:169)
>>> build 21-Mar-2014 10:36:59 at
>>> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkAnalysisConsistency(BaseTokenStreamTestCase.java:747)
>>> build 21-Mar-2014 10:36:59 at
>>> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:546)
>>> build 21-Mar-2014 10:36:59 at
>>> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:447)
>>> build 21-Mar-2014 10:36:59 at
>>> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:375)
>>>
>>>
>>> On Fri, Mar 21, 2014 at 2:33 PM, Dawid Weiss
>>> <da...@cs.put.poznan.pl> wrote:
>>>>> I just fixed a thread-safety bug, but I just saw another failure, and
>>>>> I'm pulling my hair because it refuses to repro.
>>>>
>>>> You can run with a single JVM by passing -Dtests.jvms=1 (or so I
>>>> believe; try ant test-help). This shouldn't affect multi-threaded
>>>> tests but if you have a problem with dependency between suites (test
>>>> classes), such as some values from preinitialized static fields or the
>>>> like then it may be the cause.
>>>>
>>>> Otherwise the test framework makes it really simple: if a thread was
>>>> created within a test class then it should die before the test class
>>>> completes. What is the exception (stack?) you're getting? Can you
>>>> share it privately?
>>>>
>>>> Dawid
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Threads in the LuceneTestCase system
Posted by Benson Margulies <bi...@gmail.com>.
Yea, right now I have this failure that repros every time on a big
computer and never on my not-so-small MacBook Pro.
On Fri, Mar 21, 2014 at 2:43 PM, Robert Muir <rc...@gmail.com> wrote:
> I just reviewed the code thinking of how to make it easier to
> reproduce issues, we should give this test class a startingGun.
>
> On Fri, Mar 21, 2014 at 2:38 PM, Benson Margulies <bi...@gmail.com> wrote:
>> I could share it right here, but in any case I just found _another_
>> stupid mistake where I was doing something in which multiple analyzers
>> would end up sharing something unsharable.
>>
>>
>> build 21-Mar-2014 10:36:59
>> testRandomStressWithBasisTokenizer(com.basistech.rosette.lucene.BaseLinguisticsTokenFilterTest)
>> Time elapsed: 9.249 sec <<< FAILURE!
>> build 21-Mar-2014 10:36:59 org.junit.ComparisonFailure: term 5
>> expected:<[kiiiintii?j?rjesstelm?]> but was:<[ssiikojenen]>
>> build 21-Mar-2014 10:36:59 at
>> __randomizedtesting.SeedInfo.seed([A574C0CEE3A9C8A3:47C87E8DE3A08C35]:0)
>> build 21-Mar-2014 10:36:59 at org.junit.Assert.assertEquals(Assert.java:115)
>> build 21-Mar-2014 10:36:59 at
>> org.apache.lucene.analysis.BaseTokenStreamTestCase.assertTokenStreamContents(BaseTokenStreamTestCase.java:169)
>> build 21-Mar-2014 10:36:59 at
>> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkAnalysisConsistency(BaseTokenStreamTestCase.java:747)
>> build 21-Mar-2014 10:36:59 at
>> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:546)
>> build 21-Mar-2014 10:36:59 at
>> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:447)
>> build 21-Mar-2014 10:36:59 at
>> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:375)
>>
>>
>> On Fri, Mar 21, 2014 at 2:33 PM, Dawid Weiss
>> <da...@cs.put.poznan.pl> wrote:
>>>> I just fixed a thread-safety bug, but I just saw another failure, and
>>>> I'm pulling my hair because it refuses to repro.
>>>
>>> You can run with a single JVM by passing -Dtests.jvms=1 (or so I
>>> believe; try ant test-help). This shouldn't affect multi-threaded
>>> tests but if you have a problem with dependency between suites (test
>>> classes), such as some values from preinitialized static fields or the
>>> like then it may be the cause.
>>>
>>> Otherwise the test framework makes it really simple: if a thread was
>>> created within a test class then it should die before the test class
>>> completes. What is the exception (stack?) you're getting? Can you
>>> share it privately?
>>>
>>> Dawid
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Threads in the LuceneTestCase system
Posted by Robert Muir <rc...@gmail.com>.
I just reviewed the code thinking of how to make it easier to
reproduce issues, we should give this test class a startingGun.
On Fri, Mar 21, 2014 at 2:38 PM, Benson Margulies <bi...@gmail.com> wrote:
> I could share it right here, but in any case I just found _another_
> stupid mistake where I was doing something in which multiple analyzers
> would end up sharing something unsharable.
>
>
> build 21-Mar-2014 10:36:59
> testRandomStressWithBasisTokenizer(com.basistech.rosette.lucene.BaseLinguisticsTokenFilterTest)
> Time elapsed: 9.249 sec <<< FAILURE!
> build 21-Mar-2014 10:36:59 org.junit.ComparisonFailure: term 5
> expected:<[kiiiintii?j?rjesstelm?]> but was:<[ssiikojenen]>
> build 21-Mar-2014 10:36:59 at
> __randomizedtesting.SeedInfo.seed([A574C0CEE3A9C8A3:47C87E8DE3A08C35]:0)
> build 21-Mar-2014 10:36:59 at org.junit.Assert.assertEquals(Assert.java:115)
> build 21-Mar-2014 10:36:59 at
> org.apache.lucene.analysis.BaseTokenStreamTestCase.assertTokenStreamContents(BaseTokenStreamTestCase.java:169)
> build 21-Mar-2014 10:36:59 at
> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkAnalysisConsistency(BaseTokenStreamTestCase.java:747)
> build 21-Mar-2014 10:36:59 at
> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:546)
> build 21-Mar-2014 10:36:59 at
> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:447)
> build 21-Mar-2014 10:36:59 at
> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:375)
>
>
> On Fri, Mar 21, 2014 at 2:33 PM, Dawid Weiss
> <da...@cs.put.poznan.pl> wrote:
>>> I just fixed a thread-safety bug, but I just saw another failure, and
>>> I'm pulling my hair because it refuses to repro.
>>
>> You can run with a single JVM by passing -Dtests.jvms=1 (or so I
>> believe; try ant test-help). This shouldn't affect multi-threaded
>> tests but if you have a problem with dependency between suites (test
>> classes), such as some values from preinitialized static fields or the
>> like then it may be the cause.
>>
>> Otherwise the test framework makes it really simple: if a thread was
>> created within a test class then it should die before the test class
>> completes. What is the exception (stack?) you're getting? Can you
>> share it privately?
>>
>> Dawid
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Threads in the LuceneTestCase system
Posted by Benson Margulies <bi...@gmail.com>.
I could share it right here, but in any case I just found _another_
stupid mistake where I was doing something in which multiple analyzers
would end up sharing something unsharable.
build 21-Mar-2014 10:36:59
testRandomStressWithBasisTokenizer(com.basistech.rosette.lucene.BaseLinguisticsTokenFilterTest)
Time elapsed: 9.249 sec <<< FAILURE!
build 21-Mar-2014 10:36:59 org.junit.ComparisonFailure: term 5
expected:<[kiiiintii?j?rjesstelm?]> but was:<[ssiikojenen]>
build 21-Mar-2014 10:36:59 at
__randomizedtesting.SeedInfo.seed([A574C0CEE3A9C8A3:47C87E8DE3A08C35]:0)
build 21-Mar-2014 10:36:59 at org.junit.Assert.assertEquals(Assert.java:115)
build 21-Mar-2014 10:36:59 at
org.apache.lucene.analysis.BaseTokenStreamTestCase.assertTokenStreamContents(BaseTokenStreamTestCase.java:169)
build 21-Mar-2014 10:36:59 at
org.apache.lucene.analysis.BaseTokenStreamTestCase.checkAnalysisConsistency(BaseTokenStreamTestCase.java:747)
build 21-Mar-2014 10:36:59 at
org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:546)
build 21-Mar-2014 10:36:59 at
org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:447)
build 21-Mar-2014 10:36:59 at
org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:375)
On Fri, Mar 21, 2014 at 2:33 PM, Dawid Weiss
<da...@cs.put.poznan.pl> wrote:
>> I just fixed a thread-safety bug, but I just saw another failure, and
>> I'm pulling my hair because it refuses to repro.
>
> You can run with a single JVM by passing -Dtests.jvms=1 (or so I
> believe; try ant test-help). This shouldn't affect multi-threaded
> tests but if you have a problem with dependency between suites (test
> classes), such as some values from preinitialized static fields or the
> like then it may be the cause.
>
> Otherwise the test framework makes it really simple: if a thread was
> created within a test class then it should die before the test class
> completes. What is the exception (stack?) you're getting? Can you
> share it privately?
>
> Dawid
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Threads in the LuceneTestCase system
Posted by Dawid Weiss <da...@cs.put.poznan.pl>.
> I just fixed a thread-safety bug, but I just saw another failure, and
> I'm pulling my hair because it refuses to repro.
You can run with a single JVM by passing -Dtests.jvms=1 (or so I
believe; try ant test-help). This shouldn't affect multi-threaded
tests but if you have a problem with dependency between suites (test
classes), such as some values from preinitialized static fields or the
like then it may be the cause.
Otherwise the test framework makes it really simple: if a thread was
created within a test class then it should die before the test class
completes. What is the exception (stack?) you're getting? Can you
share it privately?
Dawid
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Threads in the LuceneTestCase system
Posted by Benson Margulies <bi...@gmail.com>.
To be clear, it's my own code that is amiss here, nothing in Lucene itself.
I just fixed a thread-safety bug, but I just saw another failure, and
I'm pulling my hair because it refuses to repro.
On Fri, Mar 21, 2014 at 2:03 PM, Robert Muir <rc...@gmail.com> wrote:
> Do you have a stacktrace of where in BaseTokenStreamTestCase that it hit?
>
> We refactored the asserting here, to always test with a single thread
> *first*, then with multiple threads.
>
> So if you are failing in the multithreaded part, it means you have a
> thread safety issue...
>
> (as long as this part of the base test class is working, and i hope it
> still is, i havent seen anything crazy to indicate otherwise, and i've
> been in TestRandomChains for a few hours this week)
>
> On Fri, Mar 21, 2014 at 1:57 PM, Benson Margulies <bi...@gmail.com> wrote:
>> I'm fighting with a test that uses the random analysis chain testing.
>> It does not repro when I pass in the usual collection of -D's. I think
>> that the reason is to do with threads; the failure is always on a big
>> multicore build machine.
>>
>> Are there any more of those Carrot control -D's that change how many
>> threads are in the act?
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Threads in the LuceneTestCase system
Posted by Robert Muir <rc...@gmail.com>.
Do you have a stacktrace of where in BaseTokenStreamTestCase that it hit?
We refactored the asserting here, to always test with a single thread
*first*, then with multiple threads.
So if you are failing in the multithreaded part, it means you have a
thread safety issue...
(as long as this part of the base test class is working, and i hope it
still is, i havent seen anything crazy to indicate otherwise, and i've
been in TestRandomChains for a few hours this week)
On Fri, Mar 21, 2014 at 1:57 PM, Benson Margulies <bi...@gmail.com> wrote:
> I'm fighting with a test that uses the random analysis chain testing.
> It does not repro when I pass in the usual collection of -D's. I think
> that the reason is to do with threads; the failure is always on a big
> multicore build machine.
>
> Are there any more of those Carrot control -D's that change how many
> threads are in the act?
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org