You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Matt McCline (JIRA)" <ji...@apache.org> on 2015/11/26 03:29:11 UTC
[jira] [Resolved] (HIVE-10047) LLAP: VectorMapJoinOperator gets an
over-flow on batchSize of 1024
[ https://issues.apache.org/jira/browse/HIVE-10047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matt McCline resolved HIVE-10047.
---------------------------------
Resolution: Cannot Reproduce
> LLAP: VectorMapJoinOperator gets an over-flow on batchSize of 1024
> ------------------------------------------------------------------
>
> Key: HIVE-10047
> URL: https://issues.apache.org/jira/browse/HIVE-10047
> Project: Hive
> Issue Type: Sub-task
> Components: llap
> Affects Versions: llap
> Reporter: Gopal V
> Assignee: Matt McCline
> Fix For: llap
>
>
> Simple LLAP queries on constrained resources runs into an exception which suggests that the
> {code}
> Caused by: java.lang.ArrayIndexOutOfBoundsException: 1024
> at org.apache.hadoop.hive.ql.exec.vector.VectorColumnAssignFactory$VectorLongColumnAssign.assignLong(VectorColumnAssignFactory.java:113)
> at org.apache.hadoop.hive.ql.exec.vector.VectorColumnAssignFactory$9.assignObjectValue(VectorColumnAssignFactory.java:293)
> at org.apache.hadoop.hive.ql.exec.vector.VectorMapJoinOperator.internalForward(VectorMapJoinOperator.java:196)
> at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genUniqueJoinObject(CommonJoinOperator.java:653)
> at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genUniqueJoinObject(CommonJoinOperator.java:656)
> at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.checkAndGenObject(CommonJoinOperator.java:752)
> at org.apache.hadoop.hive.ql.exec.MapJoinOperator.processOp(MapJoinOperator.java:316)
> ... 22 more
> {code}
> The relevant line is due to the check for a full output-batch being outside of the loop here - looks like it can be triggered during MxN joins where there are more values than there were input rows in the input batch.
> {code}
> for (int i=0; i<values.length; ++i) {
> vcas[i].assignObjectValue(values[i], outputBatch.size);
> }
> ++outputBatch.size;
> if (outputBatch.size == VectorizedRowBatch.DEFAULT_SIZE) {
> flushOutput();
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)