read past EOF

["Ouyang, Hui" <hu...@oracle.com>]

Hi,
I had lots of "docs out of order" issues when the index is optimized. I did the changes based on the suggestion in this link
http://nagoya.apache.org/bugzilla/show_bug.cgi?id=23650
 
It seems this issue is solved. But some index have "read past EOF" when I do optimization. The index is over 2G and there are some documents deleted from the index. It is based on Lucene 1.4.3 on Windows.
Does anyone know how to avoid this issue? Thx.
 
Regards,
hui
 

 

merging segments _1ny5 (38708 docs) _1ot0 (1000 docs) _1t2m (4810 docs)java.io.I

OException: read past EOF

        at

org.apache.lucene.index.CompoundFileReader$CSInputStream.readInternal

(CompoundFileReader.java:218)

        at

org.apache.lucene.store.InputStream.readBytes(InputStream.java:61)

        at

org.apache.lucene.index.SegmentReader.norms(SegmentReader.java:356)

        at

org.apache.lucene.index.SegmentReader.norms(SegmentReader.java:323)

        at

org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java:4

29)

        at

org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:94)

        at

org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:51

0)

        at

org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:370)


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Re: read past EOF

[jian chen <ch...@gmail.com>]

Hi,

It seems this problem only happens when the index files get really large. 
Could it be because java has trouble handling very large files on windows 
machine (guess there is max file size on windows)?

In Lucene, I think there is a maxDoc kind of parameter that you can use to 
specify, when the index gets really large containing more than that many of 
documents, it will not try to merge the index files into one. Could this be 
used to stop the index files from growing forever?

Cheers,

Jian

On 8/27/05, Ouyang, Hui <hu...@oracle.com> wrote:
> 
> Hi,
> I had lots of "docs out of order" issues when the index is optimized. I 
> did the changes based on the suggestion in this link
> http://nagoya.apache.org/bugzilla/show_bug.cgi?id=23650
> 
> It seems this issue is solved. But some index have "read past EOF" when I 
> do optimization. The index is over 2G and there are some documents deleted 
> from the index. It is based on Lucene 1.4.3 on Windows.
> Does anyone know how to avoid this issue? Thx.
> 
> Regards,
> hui
> 
> 
> 
> 
> merging segments _1ny5 (38708 docs) _1ot0 (1000 docs) _1t2m (4810 
> docs)java.io.I
> 
> OException: read past EOF
> 
> at
> 
> org.apache.lucene.index.CompoundFileReader$CSInputStream.readInternal
> 
> (CompoundFileReader.java:218)
> 
> at
> 
> org.apache.lucene.store.InputStream.readBytes(InputStream.java:61)
> 
> at
> 
> org.apache.lucene.index.SegmentReader.norms(SegmentReader.java:356)
> 
> at
> 
> org.apache.lucene.index.SegmentReader.norms(SegmentReader.java:323)
> 
> at
> 
> org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java:4
> 
> 29)
> 
> at
> 
> org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:94)
> 
> at
> 
> org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:51
> 
> 0)
> 
> at
> 
> org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:370)
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
> 
>