You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Rajesh Balamohan (JIRA)" <ji...@apache.org> on 2017/03/17 10:34:42 UTC
[jira] [Created] (HIVE-16240) LLAP: query fragments should not be
processed if memory related errors are encountered
Rajesh Balamohan created HIVE-16240:
---------------------------------------
Summary: LLAP: query fragments should not be processed if memory related errors are encountered
Key: HIVE-16240
URL: https://issues.apache.org/jira/browse/HIVE-16240
Project: Hive
Issue Type: Bug
Components: llap
Reporter: Rajesh Balamohan
E.g TPC-DS 1 TB scale, Q29 gets stuck in "reducer 8" trying to fail and retry with the following message.
{noformat}
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Vector MapJoin Bytes Hash Table cannot grow any more -- use a smaller container size. Current logical size is 268435456 and the limit is 178956970. Estimated key count was 38384476.
at org.apache.hadoop.hive.ql.exec.tez.LlapObjectCache.retrieve(LlapObjectCache.java:124)
at org.apache.hadoop.hive.ql.exec.tez.LlapObjectCache$1.call(LlapObjectCache.java:149)
... 4 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Vector MapJoin Bytes Hash Table cannot grow any more -- use a smaller container size. Current logical size is 268435456 and the limit is 178956970. Estimated key count was 38384476.
at org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastHashTableLoader.load(VectorMapJoinFastHashTableLoader.java:110)
at org.apache.hadoop.hive.ql.exec.MapJoinOperator.loadHashTable(MapJoinOperator.java:315)
at org.apache.hadoop.hive.ql.exec.MapJoinOperator$1.call(MapJoinOperator.java:187)
at org.apache.hadoop.hive.ql.exec.MapJoinOperator$1.call(MapJoinOperator.java:183)
at org.apache.hadoop.hive.ql.exec.tez.LlapObjectCache.retrieve(LlapObjectCache.java:122)
... 5 more
Caused by: java.lang.RuntimeException: Vector MapJoin Bytes Hash Table cannot grow any more -- use a smaller container size. Current logical size is 268435456 and the limit is 178956970. Estimated key count was 38384476.
at org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastHashTable.throwExpandError(VectorMapJoinFastHashTable.java:53)
at org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastBytesHashTable.expandAndRehash(VectorMapJoinFastBytesHashTable.java:110)
at org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastBytesHashTable.add(VectorMapJoinFastBytesHashTable.java:62)
at org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastBytesHashTable.putRow(VectorMapJoinFastBytesHashTable.java:53)
at org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastTableContainer.putRow(VectorMapJoinFastTableContainer.java:182)
at org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastHashTableLoader.load(VectorMapJoinFastHashTableLoader.java:98)
... 9 more
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)