You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Mulugeta Mammo (JIRA)" <ji...@apache.org> on 2018/12/27 20:30:00 UTC
[jira] [Updated] (LUCENE-8625) Integer Overflow Exception in
ByteBuffersDataInpu
[ https://issues.apache.org/jira/browse/LUCENE-8625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mulugeta Mammo updated LUCENE-8625:
-----------------------------------
Description:
Hi,
Once I fixed the bug here, https://issues.apache.org/jira/browse/LUCENE-8624, I encountered another Integer Overflow error in ByteBuffersDataInput:
xception in thread "Lucene Merge Thread #1540" Exception in thread "main" org.apache.lucene.index.MergePolicy$MergeException: java.lang.ArithmeticException: integer overflow
at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:705)
at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:685)
Caused by: java.lang.ArithmeticException: integer overflow
at java.lang.Math.toIntExact(Math.java:1011)
at org.apache.lucene.store.ByteBuffersDataInput.sliceBufferList(ByteBuffersDataInput.java:299)
at org.apache.lucene.store.ByteBuffersDataInput.slice(ByteBuffersDataInput.java:223)
at org.apache.lucene.store.ByteBuffersIndexInput.clone(ByteBuffersIndexInput.java:186)
at org.apache.lucene.store.ByteBuffersDirectory$FileEntry.openInput(ByteBuffersDirectory.java:254)
at org.apache.lucene.store.ByteBuffersDirectory.openInput(ByteBuffersDirectory.java:223)
at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:100)
at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:100)
at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:100)
at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:157)
at org.apache.lucene.codecs.lucene50.Lucene50CompoundFormat.write(Lucene50CompoundFormat.java:89)
at org.apache.lucene.index.IndexWriter.createCompoundFile(IndexWriter.java:5004)
at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4517)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4075)
at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:626)
at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:663)
The exception is caused by a Math.toIntExact made in sliceBufferList in ByteBuffersDataInput.
{code:java|title=ByteBuffersDataInput.java|borderStyle=solid}
private static List<ByteBuffer> sliceBufferList(List<ByteBuffer> buffers, long offset, long length) {
ensureAssumptions(buffers);
if (buffers.size() == 1) {
ByteBuffer cloned = buffers.get(0).asReadOnlyBuffer();
cloned.position(Math.toIntExact(cloned.position() + offset));
cloned.limit(Math.toIntExact(length + cloned.position()));
return Arrays.asList(cloned);
} else {
long absStart = buffers.get(0).position() + offset;
long absEnd = Math.toIntExact(absStart + length); // throws integer overflow
...
{code}
Removing the Math.toIntExact works but I'm not sure if the logic will still be right since absEnd is used to calculate endOffset after a few lines:
{code:java}
int endOffset = (int) absEnd & blockMask;
{code}
Thanks,
was:
Hi,
Once I fixed the bug here, https://issues.apache.org/jira/browse/LUCENE-8624, I encountered another Integer Overflow error in ByteBuffersDataInput:
xception in thread "Lucene Merge Thread #1540" Exception in thread "main" org.apache.lucene.index.MergePolicy$MergeException: java.lang.ArithmeticException: integer overflow
at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:705)
at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:685)
Caused by: java.lang.ArithmeticException: integer overflow
at java.lang.Math.toIntExact(Math.java:1011)
at org.apache.lucene.store.ByteBuffersDataInput.sliceBufferList(ByteBuffersDataInput.java:299)
at org.apache.lucene.store.ByteBuffersDataInput.slice(ByteBuffersDataInput.java:223)
at org.apache.lucene.store.ByteBuffersIndexInput.clone(ByteBuffersIndexInput.java:186)
at org.apache.lucene.store.ByteBuffersDirectory$FileEntry.openInput(ByteBuffersDirectory.java:254)
at org.apache.lucene.store.ByteBuffersDirectory.openInput(ByteBuffersDirectory.java:223)
at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:100)
at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:100)
at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:100)
at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:157)
at org.apache.lucene.codecs.lucene50.Lucene50CompoundFormat.write(Lucene50CompoundFormat.java:89)
at org.apache.lucene.index.IndexWriter.createCompoundFile(IndexWriter.java:5004)
at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4517)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4075)
at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:626)
at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:663)
The exception is caused by a Math.toIntExact made in sliceBufferList in ByteBuffersDataInput.
{{{code:title=ByteBuffersDataInput.java|borderStyle=solid}}}
private static List<ByteBuffer> sliceBufferList(List<ByteBuffer> buffers, long offset, long length) {
ensureAssumptions(buffers);
if (buffers.size() == 1) {
ByteBuffer cloned = buffers.get(0).asReadOnlyBuffer();
cloned.position(Math.toIntExact(cloned.position() + offset));
cloned.limit(Math.toIntExact(length + cloned.position()));
return Arrays.asList(cloned);
} else {
long absStart = buffers.get(0).position() + offset;
long absEnd = Math.toIntExact(absStart + length); // throws integer overflow
...
{{{code}}}
Removing the Math.toIntExact works but I'm not sure if the logic will still be right since absEnd is used to calculate endOffset:
{code:java}
int endOffset = (int) absEnd & blockMask;
{code}
Thanks,
> Integer Overflow Exception in ByteBuffersDataInpu
> -------------------------------------------------
>
> Key: LUCENE-8625
> URL: https://issues.apache.org/jira/browse/LUCENE-8625
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/store
> Affects Versions: 7.5
> Reporter: Mulugeta Mammo
> Priority: Major
> Fix For: 7.5
>
>
> Hi,
> Once I fixed the bug here, https://issues.apache.org/jira/browse/LUCENE-8624, I encountered another Integer Overflow error in ByteBuffersDataInput:
> xception in thread "Lucene Merge Thread #1540" Exception in thread "main" org.apache.lucene.index.MergePolicy$MergeException: java.lang.ArithmeticException: integer overflow
> at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:705)
> at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:685)
> Caused by: java.lang.ArithmeticException: integer overflow
> at java.lang.Math.toIntExact(Math.java:1011)
> at org.apache.lucene.store.ByteBuffersDataInput.sliceBufferList(ByteBuffersDataInput.java:299)
> at org.apache.lucene.store.ByteBuffersDataInput.slice(ByteBuffersDataInput.java:223)
> at org.apache.lucene.store.ByteBuffersIndexInput.clone(ByteBuffersIndexInput.java:186)
> at org.apache.lucene.store.ByteBuffersDirectory$FileEntry.openInput(ByteBuffersDirectory.java:254)
> at org.apache.lucene.store.ByteBuffersDirectory.openInput(ByteBuffersDirectory.java:223)
> at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:100)
> at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:100)
> at org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:100)
> at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:157)
> at org.apache.lucene.codecs.lucene50.Lucene50CompoundFormat.write(Lucene50CompoundFormat.java:89)
> at org.apache.lucene.index.IndexWriter.createCompoundFile(IndexWriter.java:5004)
> at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4517)
> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4075)
> at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:626)
> at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:663)
> The exception is caused by a Math.toIntExact made in sliceBufferList in ByteBuffersDataInput.
> {code:java|title=ByteBuffersDataInput.java|borderStyle=solid}
> private static List<ByteBuffer> sliceBufferList(List<ByteBuffer> buffers, long offset, long length) {
> ensureAssumptions(buffers);
> if (buffers.size() == 1) {
> ByteBuffer cloned = buffers.get(0).asReadOnlyBuffer();
> cloned.position(Math.toIntExact(cloned.position() + offset));
> cloned.limit(Math.toIntExact(length + cloned.position()));
> return Arrays.asList(cloned);
> } else {
> long absStart = buffers.get(0).position() + offset;
> long absEnd = Math.toIntExact(absStart + length); // throws integer overflow
> ...
> {code}
> Removing the Math.toIntExact works but I'm not sure if the logic will still be right since absEnd is used to calculate endOffset after a few lines:
> {code:java}
> int endOffset = (int) absEnd & blockMask;
> {code}
>
> Thanks,
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org