You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Robert Muir (JIRA)" <ji...@apache.org> on 2014/07/23 21:19:40 UTC
[jira] [Updated] (LUCENE-5845) CompressingStoredFieldsWriter on
too-big document
[ https://issues.apache.org/jira/browse/LUCENE-5845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir updated LUCENE-5845:
--------------------------------
Attachment: LUCENE-5845_test.patch
simple test.
it also includes mike's patch. Currently if you get anywhere close you will exceed the VM limit for array size...
> CompressingStoredFieldsWriter on too-big document
> -------------------------------------------------
>
> Key: LUCENE-5845
> URL: https://issues.apache.org/jira/browse/LUCENE-5845
> Project: Lucene - Core
> Issue Type: Bug
> Reporter: Robert Muir
> Attachments: LUCENE-5845_test.patch
>
>
> This guy has a documented limit of 2^31-2^14
> But it becomes possible (with LUCENE-5844) to add a document that exceeds this... we shouldn't give AIOOBE but something more clear than this:
> {noformat}
> [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestDemo -Dtests.method=testMassiveDoc -Dtests.seed=8306F98D2E2B9750 -Dtests.locale=pl -Dtests.timezone=America/Jamaica -Dtests.file.encoding=ISO-8859-1
> [junit4] ERROR 5.76s | TestDemo.testMassiveDoc <<<
> [junit4] > Throwable #1: java.lang.ArrayIndexOutOfBoundsException
> [junit4] > at __randomizedtesting.SeedInfo.seed([8306F98D2E2B9750:20FE488BE80074B9]:0)
> [junit4] > at java.io.BufferedOutputStream.write(BufferedOutputStream.java:128)
> [junit4] > at org.apache.lucene.store.OutputStreamIndexOutput.writeBytes(OutputStreamIndexOutput.java:51)
> [junit4] > at org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:125)
> [junit4] > at org.apache.lucene.codecs.compressing.LZ4.encodeLiterals(LZ4.java:157)
> [junit4] > at org.apache.lucene.codecs.compressing.LZ4.encodeLastLiterals(LZ4.java:162)
> [junit4] > at org.apache.lucene.codecs.compressing.LZ4.compress(LZ4.java:252)
> [junit4] > at org.apache.lucene.codecs.compressing.CompressionMode$LZ4FastCompressor.compress(CompressionMode.java:161)
> [junit4] > at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.flush(CompressingStoredFieldsWriter.java:233)
> [junit4] > at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.finishDocument(CompressingStoredFieldsWriter.java:166)
> [junit4] > at org.apache.lucene.index.DefaultIndexingChain.finishStoredFields(DefaultIndexingChain.java:269)
> [junit4] > at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:363)
> [junit4] > at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:222)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org