You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Sergey Shelukhin (JIRA)" <ji...@apache.org> on 2015/11/30 21:09:10 UTC
[jira] [Comment Edited] (HIVE-12532) LLAP Cache: Uncompressed data
cache has NPE
[ https://issues.apache.org/jira/browse/HIVE-12532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15032371#comment-15032371 ]
Sergey Shelukhin edited comment on HIVE-12532 at 11/30/15 8:08 PM:
-------------------------------------------------------------------
[~prasanth_j] can you take a look? The loop is missing the null check.
was (Author: sershe):
[~prasanth_j] can you take a look? Missing a null check
> LLAP Cache: Uncompressed data cache has NPE
> -------------------------------------------
>
> Key: HIVE-12532
> URL: https://issues.apache.org/jira/browse/HIVE-12532
> Project: Hive
> Issue Type: Bug
> Components: llap
> Affects Versions: 2.0.0
> Reporter: Gopal V
> Assignee: Sergey Shelukhin
> Attachments: HIVE-12435.patch
>
>
> {code}
> 2015-11-26 08:28:45,232 [TezTaskRunner_attempt_1448429572030_0255_2_02_000019_2(attempt_1448429572030_0255_2_02_000019_2)] WARN org.apache.tez.runtime.LogicalIOProcessorRuntimeTask: Ignoring exception when closing input a(cleanup). Exception class=java.io.IOException, message=java.lang.NullPointerException
> java.io.IOException: java.lang.NullPointerException
> at org.apache.hadoop.hive.llap.io.api.impl.LlapInputFormat$LlapRecordReader.rethrowErrorIfAny(LlapInputFormat.java:283)
> at org.apache.hadoop.hive.llap.io.api.impl.LlapInputFormat$LlapRecordReader.close(LlapInputFormat.java:275)
> at org.apache.hadoop.hive.ql.io.HiveRecordReader.doClose(HiveRecordReader.java:50)
> at org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.close(HiveContextAwareRecordReader.java:104)
> at org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.close(TezGroupedSplitsInputFormat.java:177)
> at org.apache.tez.mapreduce.lib.MRReaderMapred.close(MRReaderMapred.java:96)
> at org.apache.tez.mapreduce.input.MRInput.close(MRInput.java:559)
> at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.cleanup(LogicalIOProcessorRuntimeTask.java:872)
> at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:104)
> at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:35)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.hive.ql.io.orc.encoded.EncodedReaderImpl.preReadUncompressedStream(EncodedReaderImpl.java:795)
> at org.apache.hadoop.hive.ql.io.orc.encoded.EncodedReaderImpl.readEncodedColumns(EncodedReaderImpl.java:320)
> at org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.performDataRead(OrcEncodedDataReader.java:413)
> at org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader$4.run(OrcEncodedDataReader.java:194)
> at org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader$4.run(OrcEncodedDataReader.java:191)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:191)
> at org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:74)
> ... 5 more
> {code}
> Not clear if current.next can set it to null before the continue;
> {code}
> assert partOffset <= current.getOffset();
> if (partOffset == current.getOffset() && current instanceof CacheChunk) {
> // We assume cache chunks would always match the way we read, so check and skip it.
> assert current.getOffset() == partOffset && current.getEnd() == partEnd;
> lastUncompressed = (CacheChunk)current;
> current = current.next;
> continue;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)