You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Uwe Schindler (JIRA)" <ji...@apache.org> on 2011/03/18 20:21:29 UTC
[jira] Issue Comment Edited: (LUCENE-2975) MMapDirectory on chunk
size boundaries broken
[ https://issues.apache.org/jira/browse/LUCENE-2975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13008587#comment-13008587 ]
Uwe Schindler edited comment on LUCENE-2975 at 3/18/11 7:20 PM:
----------------------------------------------------------------
I just want to add that on Solaris 64 bit I had initially 1.6.0_21 installed. With that version CheckIndex passed, too (verified in my logs). But the bad thing: The application server logs showed random DataFormatErrors like CheckIndex a few times. This is why I started to debug and then upgraded to _24. Since _24 it fails always on Solaris.
Robert does not look into JDK code, I did: They changed some places in NIO's unsafe Bits class (some memory copy methods), so I assume the bug is there.
was (Author: thetaphi):
I just want to add that on Solaris 64 bit I had initially 1.6.0_21 installed. With that version CheckIndex passed, too (verified in my locks). But the bad thing: The application server logs showed random DataFormatErrors like CheckIndex a few times. This is why I started to debug and then upgraded to _24. Since _24 it fails always on Solaris.
Robert does not look into JDK code, I did: They changed some places in NIO's unsafe Bits class (some memory copy methods), so I assume the bug is there.
> MMapDirectory on chunk size boundaries broken
> ---------------------------------------------
>
> Key: LUCENE-2975
> URL: https://issues.apache.org/jira/browse/LUCENE-2975
> Project: Lucene - Java
> Issue Type: Bug
> Affects Versions: 3.1
> Reporter: Uwe Schindler
> Priority: Blocker
> Fix For: 3.1
>
>
> When testing the 3.1-RC1 made by Yonik on the PANGAEA (www.pangaea.de) productive system I figured out that suddenly on a large segment (about 5 GiB) some stored fiels suddenly produce a strange deflate decompression problem (CompressionTools) although the stored fields are no longer pre-3.0 compressed. It seems that the header of the stored field is read incorrectly at the buffer boundary in MultiMMapDir and then FieldsReader just incorrectly detects a deflate-compressed field (CompressionTools).
> The error occurs reproducible on CheckIndex with MMapDirectory, but not with NIODir or SimpleDir. The FDT file of that segment is 2.6 GiB, on Solaris the chunk size is Integer.MAX_VALUE, so we have 2 MultiMMap IndexInputs.
> Robert and me have the index ready as a tar file, we will do tests on our local machines and hopefully solve the bug, maybe introduced by Robert's recent changes to MMap.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org