You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Walt Elder (JIRA)" <ji...@apache.org> on 2012/06/05 23:40:22 UTC
[jira] [Commented] (LUCENE-4114) Large docID / docvalue size
combination produces arithmetic overflow
[ https://issues.apache.org/jira/browse/LUCENE-4114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13289751#comment-13289751 ]
Walt Elder commented on LUCENE-4114:
------------------------------------
Followup: I built my own version of lucene-core with this line corrected (I cast docID to long) and ran my tests successfully.
> Large docID / docvalue size combination produces arithmetic overflow
> --------------------------------------------------------------------
>
> Key: LUCENE-4114
> URL: https://issues.apache.org/jira/browse/LUCENE-4114
> Project: Lucene - Java
> Issue Type: Bug
> Components: core/codecs
> Affects Versions: 4.0, 5.0
> Environment: Ubuntu 10.04
> Sun Java 6 b26
> MultiIndex over six directoryindices with ~80M documents each.
> Reporter: Walt Elder
> Fix For: 4.0, 5.0
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> My test case has run across an arithmetic bug in FixedStraightBytesImpl - specifically in the @Override on position(int). The combination of size=32 and docID = 70M produces a negative integer and causes a stacktrace in MMapDirectory.seek(long) which I will post below. I would imagine that this hasn't been hit before because most index shards have less than 70M documents.
> That is to say, when asking for the docvalue in a FixedStraightBytes field where the size of values are 32 and the docID can be sufficiently high, an arithmetic overflow occurs in line 359 of FixedStraightBytesImpl.java, causing the method to return a negative value when operating on the first (0th) shard with a baseOffset of 0. This produces an IllegalArgumentException:
> Caused by: java.lang.IllegalArgumentException: Seeking to negative position: MMapIndexInput(_7_5_dv.dat in path="/.../_7_dv.cfs" slice=5:2562955684)
> at org.apache.lucene.store.MMapDirectory$MMapIndexInput.seek(MMapDirectory.java:396)
> at org.apache.lucene.codecs.lucene40.values.FixedStraightBytesImpl$DirectFixedStraightSource.position(FixedStraightBytesImpl.java:359)
> at org.apache.lucene.codecs.lucene40.values.DirectSource.getBytes(DirectSource.java:60)
> at org.apache.lucene.codecs.lucene40.values.FixedStraightBytesImpl$DirectFixedStraightSource.getBytes(FixedStraightBytesImpl.java:349)
> ...
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org