You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@orc.apache.org by "Dongjoon Hyun (Jira)" <ji...@apache.org> on 2021/12/28 19:58:00 UTC
[jira] [Resolved] (ORC-1065) ReaderImpl.extractFileTail IndexOutOfBoundsException
[ https://issues.apache.org/jira/browse/ORC-1065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dongjoon Hyun resolved ORC-1065.
--------------------------------
Fix Version/s: 1.6.13
1.7.3
Resolution: Fixed
This is resolved via https://github.com/apache/orc/pull/979
> ReaderImpl.extractFileTail IndexOutOfBoundsException
> ----------------------------------------------------
>
> Key: ORC-1065
> URL: https://issues.apache.org/jira/browse/ORC-1065
> Project: ORC
> Issue Type: Bug
> Components: Java
> Affects Versions: 1.6.12, 1.7.2
> Reporter: dzcxzl
> Priority: Minor
> Fix For: 1.6.13, 1.7.3
>
>
> ORC-251 remove `ReaderImpl.extractFileTail`
> ORC-685 Add `ReaderImpl.extractFileTail` back
>
> In ORC-685, file length is used as readsize, which causes that if the buffer is read from the cache, the use of length is incorrect, resulting in IndexOutOfBoundsException.
> {code:java}
> long readSize = fileLen != -1? fileLen: buffer.limit();
> int psLen = buffer.get((int) (readSize-1)) & 0xff; {code}
> {code:java}
> Caused by: java.lang.IndexOutOfBoundsException
> at java.nio.Buffer.checkIndex(Buffer.java:540)
> at java.nio.HeapByteBuffer.get(HeapByteBuffer.java:139)
> at org.apache.orc.impl.ReaderImpl.extractFileTail(ReaderImpl.java:726)
> at org.apache.hadoop.hive.ql.io.orc.LocalCache.getAndValidate(LocalCache.java:103)
> at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$ETLSplitStrategy.getSplits(OrcInputFormat.java:798)
> at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$ETLSplitStrategy.runGetSplitsSync(OrcInputFormat.java:916)
> at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$ETLSplitStrategy.generateSplitWork(OrcInputFormat.java:885)
> at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.scheduleSplits(OrcInputFormat.java:1759)
> at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:1703) {code}
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)