You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Sergey Shelukhin (JIRA)" <ji...@apache.org> on 2015/04/07 00:05:13 UTC
[jira] [Commented] (HIVE-10161) LLAP: ORC file contains compression
buffers larger than bufferSize (OR reader has a bug)
[ https://issues.apache.org/jira/browse/HIVE-10161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14482052#comment-14482052 ]
Sergey Shelukhin commented on HIVE-10161:
-----------------------------------------
For reference - repro with 6 instances, ctas3 query, WARN log level
> LLAP: ORC file contains compression buffers larger than bufferSize (OR reader has a bug)
> ----------------------------------------------------------------------------------------
>
> Key: HIVE-10161
> URL: https://issues.apache.org/jira/browse/HIVE-10161
> Project: Hive
> Issue Type: Sub-task
> Affects Versions: llap
> Reporter: Gopal V
> Assignee: Sergey Shelukhin
> Fix For: llap
>
>
> The EncodedReaderImpl will die when reading from the cache, when reading data written by the regular ORC writer
> {code}
> Caused by: java.io.IOException: java.lang.IllegalArgumentException: Buffer size too small. size = 262144 needed = 3919246
> at org.apache.hadoop.hive.llap.io.api.impl.LlapInputFormat$LlapRecordReader.rethrowErrorIfAny(LlapInputFormat.java:249)
> at org.apache.hadoop.hive.llap.io.api.impl.LlapInputFormat$LlapRecordReader.nextCvb(LlapInputFormat.java:201)
> at org.apache.hadoop.hive.llap.io.api.impl.LlapInputFormat$LlapRecordReader.next(LlapInputFormat.java:140)
> at org.apache.hadoop.hive.llap.io.api.impl.LlapInputFormat$LlapRecordReader.next(LlapInputFormat.java:96)
> at org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:350)
> ... 22 more
> Caused by: java.lang.IllegalArgumentException: Buffer size too small. size = 262144 needed = 3919246
> at org.apache.hadoop.hive.ql.io.orc.InStream.addOneCompressionBuffer(InStream.java:780)
> at org.apache.hadoop.hive.ql.io.orc.InStream.uncompressStream(InStream.java:628)
> at org.apache.hadoop.hive.ql.io.orc.EncodedReaderImpl.readEncodedColumns(EncodedReaderImpl.java:309)
> at org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:278)
> at org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:48)
> at org.apache.hadoop.hive.common.CallableWithNdc.call(CallableWithNdc.java:37)
> ... 4 more
> ]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1424502260528_1945_1_00 [Map 1] killed/failed due to:null]
> {code}
> Turning off hive.llap.io.enabled makes the error go away.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)