You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by jian chen <ch...@gmail.com> on 2005/08/27 20:57:04 UTC
Re: read past EOF
Hi,
It seems this problem only happens when the index files get really large.
Could it be because java has trouble handling very large files on windows
machine (guess there is max file size on windows)?
In Lucene, I think there is a maxDoc kind of parameter that you can use to
specify, when the index gets really large containing more than that many of
documents, it will not try to merge the index files into one. Could this be
used to stop the index files from growing forever?
Cheers,
Jian
On 8/27/05, Ouyang, Hui <hu...@oracle.com> wrote:
>
> Hi,
> I had lots of "docs out of order" issues when the index is optimized. I
> did the changes based on the suggestion in this link
> http://nagoya.apache.org/bugzilla/show_bug.cgi?id=23650
>
> It seems this issue is solved. But some index have "read past EOF" when I
> do optimization. The index is over 2G and there are some documents deleted
> from the index. It is based on Lucene 1.4.3 on Windows.
> Does anyone know how to avoid this issue? Thx.
>
> Regards,
> hui
>
>
>
>
> merging segments _1ny5 (38708 docs) _1ot0 (1000 docs) _1t2m (4810
> docs)java.io.I
>
> OException: read past EOF
>
> at
>
> org.apache.lucene.index.CompoundFileReader$CSInputStream.readInternal
>
> (CompoundFileReader.java:218)
>
> at
>
> org.apache.lucene.store.InputStream.readBytes(InputStream.java:61)
>
> at
>
> org.apache.lucene.index.SegmentReader.norms(SegmentReader.java:356)
>
> at
>
> org.apache.lucene.index.SegmentReader.norms(SegmentReader.java:323)
>
> at
>
> org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java:4
>
> 29)
>
> at
>
> org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:94)
>
> at
>
> org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:51
>
> 0)
>
> at
>
> org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:370)
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>