You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Dawid Weiss (JIRA)" <ji...@apache.org> on 2018/12/28 11:08:00 UTC
[jira] [Commented] (LUCENE-8625) int overflow in
ByteBuffersDataInput.sliceBufferList
[ https://issues.apache.org/jira/browse/LUCENE-8625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16730187#comment-16730187 ]
Dawid Weiss commented on LUCENE-8625:
-------------------------------------
Thank you for reporting these. I added a test for this and corrected both this issue and LUCENE-8624. Please try it and report any other issues you encounter (not many people have access to heaps this large!).
> int overflow in ByteBuffersDataInput.sliceBufferList
> ----------------------------------------------------
>
> Key: LUCENE-8625
> URL: https://issues.apache.org/jira/browse/LUCENE-8625
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/store
> Affects Versions: 7.5
> Reporter: Mulugeta Mammo
> Assignee: Dawid Weiss
> Priority: Minor
> Fix For: 7.7
>
>
> Hi,
> Once I fixed the bug here, https://issues.apache.org/jira/browse/LUCENE-8624, I encountered another Integer Overflow error in ByteBuffersDataInput:
> Exception in thread "Lucene Merge Thread #1540" Exception in thread "main" org.apache.lucene.index.MergePolicy$MergeException: java.lang.ArithmeticException: integer overflow
> at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:705)
> at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:685)
> Caused by: java.lang.ArithmeticException: integer overflow
> at java.lang.Math.toIntExact(Math.java:1011)
> at org.apache.lucene.store.ByteBuffersDataInput.sliceBufferList(ByteBuffersDataInput.java:299)
> at org.apache.lucene.store.ByteBuffersDataInput.slice(ByteBuffersDataInput.java:223)
> at org.apache.lucene.store.ByteBuffersIndexInput.clone(ByteBuffersIndexInput.java:186)
> at org.apache.lucene.store.ByteBuffersDirectory$FileEntry.openInput(ByteBuffersDirectory.java:254)
> at org.apache.lucene.store.ByteBuffersDirectory.openInput(ByteBuffersDirectory.java:223)
> at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:100)
> at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:100)
> at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:100)
> at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:157)
> at org.apache.lucene.codecs.lucene50.Lucene50CompoundFormat.write(Lucene50CompoundFormat.java:89)
> at org.apache.lucene.index.IndexWriter.createCompoundFile(IndexWriter.java:5004)
> at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4517)
> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4075)
> at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:626)
> at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:663)
> The exception is caused by a Math.toIntExact in sliceBufferList in ByteBuffersDataInput.
> {code:java|title=ByteBuffersDataInput.java|borderStyle=solid}
> private static List<ByteBuffer> sliceBufferList(List<ByteBuffer> buffers, long offset, long length) {
> ensureAssumptions(buffers);
> if (buffers.size() == 1) {
> ByteBuffer cloned = buffers.get(0).asReadOnlyBuffer();
> cloned.position(Math.toIntExact(cloned.position() + offset));
> cloned.limit(Math.toIntExact(length + cloned.position()));
> return Arrays.asList(cloned);
> } else {
> long absStart = buffers.get(0).position() + offset;
> long absEnd = Math.toIntExact(absStart + length); // throws integer overflow
> ...
> {code}
> Removing the Math.toIntExact works but I'm not sure if the logic will still be right since absEnd is used to calculate endOffset after a few lines:
> {code:java}
> int endOffset = (int) absEnd & blockMask;
> {code}
>
> Thanks,
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org