You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Hudson Server <hu...@hudson.apache.org> on 2010/11/06 20:28:20 UTC
Lucene-Solr-tests-only-trunk - Build # 1070 - Failure
Build: https://hudson.apache.org/hudson/job/Lucene-Solr-tests-only-trunk/1070/
1 tests failed.
REGRESSION: org.apache.lucene.index.TestIndexWriterMergePolicy.testMaxBufferedDocsChange
Error Message:
maxMergeDocs=2147483647; numSegments=11; upperBound=10; mergeFactor=10; segs=_65:c5950 _5t:c10->_32 _5u:c10->_32 _5v:c10->_32 _5w:c10->_32 _5x:c10->_32 _5y:c10->_32 _5z:c10->_32 _60:c10->_32 _61:c10->_32 _62:c1->_32 _64:c9->_62
Stack Trace:
junit.framework.AssertionFailedError: maxMergeDocs=2147483647; numSegments=11; upperBound=10; mergeFactor=10; segs=_65:c5950 _5t:c10->_32 _5u:c10->_32 _5v:c10->_32 _5w:c10->_32 _5x:c10->_32 _5y:c10->_32 _5z:c10->_32 _60:c10->_32 _61:c10->_32 _62:c1->_32 _64:c9->_62
at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:878)
at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:844)
at org.apache.lucene.index.TestIndexWriterMergePolicy.checkInvariants(TestIndexWriterMergePolicy.java:243)
at org.apache.lucene.index.TestIndexWriterMergePolicy.testMaxBufferedDocsChange(TestIndexWriterMergePolicy.java:169)
Build Log (for compile errors):
[...truncated 3084 lines...]
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Lucene-Solr-tests-only-trunk - Build # 1072 - Still Failing
Posted by Apache Hudson Server <hu...@hudson.apache.org>.
Build: https://hudson.apache.org/hudson/job/Lucene-Solr-tests-only-trunk/1072/
1 tests failed.
FAILED: org.apache.solr.TestDistributedSearch.testDistribSearch
Error Message:
Some threads threw uncaught exceptions!
Stack Trace:
junit.framework.AssertionFailedError: Some threads threw uncaught exceptions!
at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:878)
at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:844)
at org.apache.lucene.util.LuceneTestCase.tearDown(LuceneTestCase.java:437)
at org.apache.solr.SolrTestCaseJ4.tearDown(SolrTestCaseJ4.java:78)
at org.apache.solr.BaseDistributedSearchTestCase.tearDown(BaseDistributedSearchTestCase.java:144)
Build Log (for compile errors):
[...truncated 8756 lines...]
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Lucene-Solr-tests-only-trunk - Build # 1071 - Still Failing
Posted by Michael McCandless <lu...@mikemccandless.com>.
Sadly, yes, this is SimpleText's "speedup", heh.
However, the OOM actually occurs when the RIW used by TestNRQ64 calls
optimize on .close(). But, this optimize is unused since the test has
already pulled its reader. I think a simple workaround would be to
close the RIW but never let it call optimize? Eg we could just reach
in and close the underlying writer (writer.w.close)... or add a
.closeNoOptimize to RIW?
Mike
On Sat, Nov 6, 2010 at 6:07 PM, Uwe Schindler <uw...@thetaphi.de> wrote:
> Hehe, yes!
>
> This test creates lots of terms:
> As far as I remember, it indexes 10,000 documents in 9 different number styles and precisionSteps (2, 4, 6, 8):
>
> But when RANDOM_MULTIPLIER > 1 then much more:
> private static final int noDocs = 10000 * RANDOM_MULTIPLIER;
>
> As the lowerPrec terms are duplicates, the approx. number of terms (I assume that for next shift value, the number of terms reduces by 2 because of overlaps):
> precStep=8: 10 000 + 5 000 + 2 500 + 1 250 + 625 + 312 + 161 + 80 = 19 928 terms
> precStep=6: 10 000 + 5 000 + 2 500 + 1 250 + 625 + 312 + 161 + 80 + 40 + 20 + 10 = 19 998 terms
> precStep=4: some more (16 summands) :-], approx. 20 500
> precStep=2: again more (32 summands), approx. 20800
>
> This makes approx 9*20,000*RANDOM_MULTIPLIER (its 3 on Hudson) terms in the index -- ahm TreeSet incl TermInfo *g*. For TestNumericRangeQuery32 its similar, but fewer precsteps.
>
> We have several options: Run all tests with default Heap size only for SimpleText and check that all tests pass, if not raise -Xmx until it passes (currently tests use 512 M).
>
> Alternatively reduce the index size for some tests if SimpleText codec is used. TestNumeric should be easy possible, as the test uses a some preconfigured constants for building index and run tests.
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>
>> -----Original Message-----
>> From: Robert Muir [mailto:rcmuir@gmail.com]
>> Sent: Saturday, November 06, 2010 9:03 PM
>> To: dev@lucene.apache.org
>> Subject: Re: Lucene-Solr-tests-only-trunk - Build # 1071 - Still Failing
>>
>> On Sat, Nov 6, 2010 at 3:58 PM, Apache Hudson Server
>> <hu...@hudson.apache.org> wrote:
>> > Error Message:
>> > Java heap space
>> >
>> ...
>> > Error Message:
>> > MockDirectoryWrapper: cannot close: there are still open files:
>> > {_2h.pst=1}
>>
>> seems like its likely caused by the simpletext optimization?
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional
>> commands, e-mail: dev-help@lucene.apache.org
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
RE: Lucene-Solr-tests-only-trunk - Build # 1071 - Still Failing
Posted by Uwe Schindler <uw...@thetaphi.de>.
Hehe, yes!
This test creates lots of terms:
As far as I remember, it indexes 10,000 documents in 9 different number styles and precisionSteps (2, 4, 6, 8):
But when RANDOM_MULTIPLIER > 1 then much more:
private static final int noDocs = 10000 * RANDOM_MULTIPLIER;
As the lowerPrec terms are duplicates, the approx. number of terms (I assume that for next shift value, the number of terms reduces by 2 because of overlaps):
precStep=8: 10 000 + 5 000 + 2 500 + 1 250 + 625 + 312 + 161 + 80 = 19 928 terms
precStep=6: 10 000 + 5 000 + 2 500 + 1 250 + 625 + 312 + 161 + 80 + 40 + 20 + 10 = 19 998 terms
precStep=4: some more (16 summands) :-], approx. 20 500
precStep=2: again more (32 summands), approx. 20800
This makes approx 9*20,000*RANDOM_MULTIPLIER (its 3 on Hudson) terms in the index -- ahm TreeSet incl TermInfo *g*. For TestNumericRangeQuery32 its similar, but fewer precsteps.
We have several options: Run all tests with default Heap size only for SimpleText and check that all tests pass, if not raise -Xmx until it passes (currently tests use 512 M).
Alternatively reduce the index size for some tests if SimpleText codec is used. TestNumeric should be easy possible, as the test uses a some preconfigured constants for building index and run tests.
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
> -----Original Message-----
> From: Robert Muir [mailto:rcmuir@gmail.com]
> Sent: Saturday, November 06, 2010 9:03 PM
> To: dev@lucene.apache.org
> Subject: Re: Lucene-Solr-tests-only-trunk - Build # 1071 - Still Failing
>
> On Sat, Nov 6, 2010 at 3:58 PM, Apache Hudson Server
> <hu...@hudson.apache.org> wrote:
> > Error Message:
> > Java heap space
> >
> ...
> > Error Message:
> > MockDirectoryWrapper: cannot close: there are still open files:
> > {_2h.pst=1}
>
> seems like its likely caused by the simpletext optimization?
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional
> commands, e-mail: dev-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Lucene-Solr-tests-only-trunk - Build # 1071 - Still Failing
Posted by Robert Muir <rc...@gmail.com>.
On Sat, Nov 6, 2010 at 3:58 PM, Apache Hudson Server
<hu...@hudson.apache.org> wrote:
> Error Message:
> Java heap space
>
...
> Error Message:
> MockDirectoryWrapper: cannot close: there are still open files: {_2h.pst=1}
seems like its likely caused by the simpletext optimization?
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Lucene-Solr-tests-only-trunk - Build # 1071 - Still Failing
Posted by Apache Hudson Server <hu...@hudson.apache.org>.
Build: https://hudson.apache.org/hudson/job/Lucene-Solr-tests-only-trunk/1071/
3 tests failed.
FAILED: junit.framework.TestSuite.org.apache.lucene.search.TestNumericRangeQuery64
Error Message:
Java heap space
Stack Trace:
java.lang.OutOfMemoryError: Java heap space
at org.apache.lucene.store.RAMFile.newBuffer(RAMFile.java:89)
at org.apache.lucene.store.RAMFile.addBuffer(RAMFile.java:62)
at org.apache.lucene.store.RAMOutputStream.switchCurrentBuffer(RAMOutputStream.java:132)
at org.apache.lucene.store.RAMOutputStream.copyBytes(RAMOutputStream.java:171)
at org.apache.lucene.store.MockIndexOutputWrapper.copyBytes(MockIndexOutputWrapper.java:134)
at org.apache.lucene.index.CompoundFileWriter.copyFile(CompoundFileWriter.java:222)
at org.apache.lucene.index.CompoundFileWriter.close(CompoundFileWriter.java:188)
at org.apache.lucene.index.SegmentMerger.createCompoundFile(SegmentMerger.java:212)
at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4182)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3659)
at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:37)
at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2566)
at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2387)
at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2346)
at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2316)
at org.apache.lucene.index.RandomIndexWriter.close(RandomIndexWriter.java:145)
at org.apache.lucene.search.TestNumericRangeQuery64.beforeClass(TestNumericRangeQuery64.java:92)
FAILED: junit.framework.TestSuite.org.apache.lucene.search.TestNumericRangeQuery64
Error Message:
MockDirectoryWrapper: cannot close: there are still open files: {_2h.pst=1}
Stack Trace:
java.lang.RuntimeException: MockDirectoryWrapper: cannot close: there are still open files: {_2h.pst=1}
at org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:343)
at org.apache.lucene.search.TestNumericRangeQuery64.afterClass(TestNumericRangeQuery64.java:101)
Caused by: java.lang.RuntimeException: unclosed IndexInput
at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:300)
at org.apache.lucene.index.codecs.simpletext.SimpleTextFieldsReader.<init>(SimpleTextFieldsReader.java:57)
at org.apache.lucene.index.codecs.simpletext.SimpleTextCodec.fieldsProducer(SimpleTextCodec.java:53)
at org.apache.lucene.index.SegmentReader$CoreReaders.<init>(SegmentReader.java:136)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:536)
at org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:626)
at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4114)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3659)
at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:37)
at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2566)
at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2387)
at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2346)
at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2316)
at org.apache.lucene.index.RandomIndexWriter.close(RandomIndexWriter.java:145)
at org.apache.lucene.search.TestNumericRangeQuery64.beforeClass(TestNumericRangeQuery64.java:92)
FAILED: junit.framework.TestSuite.org.apache.lucene.search.TestNumericRangeQuery64
Error Message:
directory of test was not closed, opened from: org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:653)
Stack Trace:
junit.framework.AssertionFailedError: directory of test was not closed, opened from: org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:653)
at org.apache.lucene.util.LuceneTestCase.afterClassLuceneTestCaseJ4(LuceneTestCase.java:331)
Build Log (for compile errors):
[...truncated 3110 lines...]
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org