You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@orc.apache.org by "dzcxzl (Jira)" <ji...@apache.org> on 2021/12/28 04:20:00 UTC
[jira] [Created] (ORC-1065) ReaderImpl.extractFileTail IndexOutOfBoundsException
dzcxzl created ORC-1065:
---------------------------
Summary: ReaderImpl.extractFileTail IndexOutOfBoundsException
Key: ORC-1065
URL: https://issues.apache.org/jira/browse/ORC-1065
Project: ORC
Issue Type: Bug
Components: Java
Affects Versions: 1.7.2, 1.6.12
Reporter: dzcxzl
ORC-251 remove `ReaderImpl.extractFileTail`
ORC-685 Add `ReaderImpl.extractFileTail` back
In ORC-685, file length is used as readsize, which causes that if the buffer is read from the cache, the use of length is incorrect, resulting in IndexOutOfBoundsException.
{code:java}
long readSize = fileLen != -1? fileLen: buffer.limit();
int psLen = buffer.get((int) (readSize-1)) & 0xff; {code}
{code:java}
Caused by: java.lang.IndexOutOfBoundsException
at java.nio.Buffer.checkIndex(Buffer.java:540)
at java.nio.HeapByteBuffer.get(HeapByteBuffer.java:139)
at org.apache.orc.impl.ReaderImpl.extractFileTail(ReaderImpl.java:726)
at org.apache.hadoop.hive.ql.io.orc.LocalCache.getAndValidate(LocalCache.java:103)
at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$ETLSplitStrategy.getSplits(OrcInputFormat.java:798)
at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$ETLSplitStrategy.runGetSplitsSync(OrcInputFormat.java:916)
at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$ETLSplitStrategy.generateSplitWork(OrcInputFormat.java:885)
at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.scheduleSplits(OrcInputFormat.java:1759)
at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:1703) {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)