You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Navis (JIRA)" <ji...@apache.org> on 2011/08/02 07:58:27 UTC
[jira] [Created] (HIVE-2332) If parameter of distinct function is
contained in group by columns, it fails in runtime
If parameter of distinct function is contained in group by columns, it fails in runtime
---------------------------------------------------------------------------------------
Key: HIVE-2332
URL: https://issues.apache.org/jira/browse/HIVE-2332
Project: Hive
Issue Type: Bug
Reporter: Navis
Priority: Critical
Fix For: 0.8.0
select sum(key_int1), sum(distinct key_int1) from t1 group by key_int1;
fails with message..
{code}
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
{code}
hadoop says..
{code}
Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
at java.util.ArrayList.RangeCheck(ArrayList.java:547)
at java.util.ArrayList.get(ArrayList.java:322)
at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:95)
at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.(StandardStructObjectInspector.java:86)
at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:252)
at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initEvaluatorsAndReturnStruct(ReduceSinkOperator.java:188)
at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:197)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:85)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:532)
{code}
I think the deficient number of key expression, compared to number of key column, is the problem, which should be equal or more.
Would it be solved if add some key expression? I'll try.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2332) If all of the parameters of distinct
functions are exists in group by columns, query fails in runtime
Posted by "Navis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Navis updated HIVE-2332:
------------------------
Attachment: HIVE-2332.3.patch
add test case
> If all of the parameters of distinct functions are exists in group by columns, query fails in runtime
> -----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2332
> URL: https://issues.apache.org/jira/browse/HIVE-2332
> Project: Hive
> Issue Type: Bug
> Reporter: Navis
> Priority: Critical
> Fix For: 0.8.0
>
> Attachments: HIVE-2332.1.patch, HIVE-2332.2.patch, HIVE-2332.3.patch
>
>
> select sum(key_int1), sum(distinct key_int1) from t1 group by key_int1;
> fails with message..
> {code}
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
> {code}
> hadoop says..
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
> at java.util.ArrayList.RangeCheck(ArrayList.java:547)
> at java.util.ArrayList.get(ArrayList.java:322)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:95)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.(StandardStructObjectInspector.java:86)
> at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:252)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initEvaluatorsAndReturnStruct(ReduceSinkOperator.java:188)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:197)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:85)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:532)
> {code}
> I think the deficient number of key expression, compared to number of key column, is the problem, which should be equal or more.
> Would it be solved if add some key expression? I'll try.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2332) If all of the parameters of distinct
functions are exists in group by columns, query fails in runtime
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13080794#comment-13080794 ]
jiraposter@reviews.apache.org commented on HIVE-2332:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1314/
-----------------------------------------------------------
Review request for hive.
Summary
-------
If all of the distinct params are in group by keys, union column reserved for distinct params would not be added, which making problems initializing RS operator.
This patch is just a simple bypass adding dummy expression for the union column. Someone would know better way to resolve the problem.
This addresses bug HIVE-2332.
https://issues.apache.org/jira/browse/HIVE-2332
Diffs
-----
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java dc9ab1b
ql/src/test/queries/clientpositive/groupby_distinct_samekey.q PRE-CREATION
ql/src/test/results/clientpositive/groupby_distinct_samekey.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/1314/diff
Testing
-------
Thanks,
Navis
> If all of the parameters of distinct functions are exists in group by columns, query fails in runtime
> -----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2332
> URL: https://issues.apache.org/jira/browse/HIVE-2332
> Project: Hive
> Issue Type: Bug
> Reporter: Navis
> Priority: Critical
> Fix For: 0.8.0
>
> Attachments: HIVE-2332.1.patch, HIVE-2332.2.patch, HIVE-2332.3.patch
>
>
> select sum(key_int1), sum(distinct key_int1) from t1 group by key_int1;
> fails with message..
> {code}
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
> {code}
> hadoop says..
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
> at java.util.ArrayList.RangeCheck(ArrayList.java:547)
> at java.util.ArrayList.get(ArrayList.java:322)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:95)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.(StandardStructObjectInspector.java:86)
> at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:252)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initEvaluatorsAndReturnStruct(ReduceSinkOperator.java:188)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:197)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:85)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:532)
> {code}
> I think the deficient number of key expression, compared to number of key column, is the problem, which should be equal or more.
> Would it be solved if add some key expression? I'll try.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2332) If all of the parameters of distinct
functions are exists in group by columns, query fails in runtime
Posted by "Navis (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Navis updated HIVE-2332:
------------------------
Attachment: HIVE-2332.2.patch.txt
https://reviews.apache.org/r/1314/
> If all of the parameters of distinct functions are exists in group by columns, query fails in runtime
> -----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2332
> URL: https://issues.apache.org/jira/browse/HIVE-2332
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Navis
> Assignee: Navis
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2332.1.patch.txt, HIVE-2332.2.patch.txt
>
>
> select sum(key_int1), sum(distinct key_int1) from t1 group by key_int1;
> fails with message..
> {code}
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
> {code}
> hadoop says..
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
> at java.util.ArrayList.RangeCheck(ArrayList.java:547)
> at java.util.ArrayList.get(ArrayList.java:322)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:95)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.(StandardStructObjectInspector.java:86)
> at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:252)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initEvaluatorsAndReturnStruct(ReduceSinkOperator.java:188)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:197)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:85)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:532)
> {code}
> I think the deficient number of key expression, compared to number of key column, is the problem, which should be equal or more.
> Would it be solved if add some key expression? I'll try.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2332) If all of the parameters of distinct
functions are exists in group by columns, query fails in runtime
Posted by "Navis (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13244945#comment-13244945 ]
Navis commented on HIVE-2332:
-----------------------------
It's already on phabricator by Namit.
> If all of the parameters of distinct functions are exists in group by columns, query fails in runtime
> -----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2332
> URL: https://issues.apache.org/jira/browse/HIVE-2332
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Navis
> Assignee: Navis
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2332.1.patch.txt, HIVE-2332.2.patch.txt, HIVE-2332.D663.1.patch
>
>
> select sum(key_int1), sum(distinct key_int1) from t1 group by key_int1;
> fails with message..
> {code}
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
> {code}
> hadoop says..
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
> at java.util.ArrayList.RangeCheck(ArrayList.java:547)
> at java.util.ArrayList.get(ArrayList.java:322)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:95)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.(StandardStructObjectInspector.java:86)
> at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:252)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initEvaluatorsAndReturnStruct(ReduceSinkOperator.java:188)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:197)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:85)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:532)
> {code}
> I think the deficient number of key expression, compared to number of key column, is the problem, which should be equal or more.
> Would it be solved if add some key expression? I'll try.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2332) If all of the parameters of distinct
functions are exists in group by columns, query fails in runtime
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164193#comment-13164193 ]
jiraposter@reviews.apache.org commented on HIVE-2332:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1314/
-----------------------------------------------------------
(Updated 2011-12-07 07:01:30.547976)
Review request for hive, John Sichi and Carl Steinbach.
Changes
-------
Adding null keys induced confusions especially for optimizers. This patch just modifies key-order minimizing side effects.
Summary
-------
If all of the distinct params are in group by keys, union column reserved for distinct params would not be added, which making problems initializing RS operator.
This patch is just a simple bypass adding dummy expression for the union column. Someone would know better way to resolve the problem.
This addresses bug HIVE-2332.
https://issues.apache.org/jira/browse/HIVE-2332
Diffs (updated)
-----
ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java 732a5aa
ql/src/java/org/apache/hadoop/hive/ql/plan/PlanUtils.java c6ae55d
ql/src/test/queries/clientpositive/groupby_distinct_samekey.q PRE-CREATION
ql/src/test/results/clientpositive/groupby_distinct_samekey.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/1314/diff
Testing
-------
added clientpositive/groupby_distinct_samekey.q
Thanks,
Navis
> If all of the parameters of distinct functions are exists in group by columns, query fails in runtime
> -----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2332
> URL: https://issues.apache.org/jira/browse/HIVE-2332
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Navis
> Assignee: Navis
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2332.1.patch.txt
>
>
> select sum(key_int1), sum(distinct key_int1) from t1 group by key_int1;
> fails with message..
> {code}
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
> {code}
> hadoop says..
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
> at java.util.ArrayList.RangeCheck(ArrayList.java:547)
> at java.util.ArrayList.get(ArrayList.java:322)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:95)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.(StandardStructObjectInspector.java:86)
> at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:252)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initEvaluatorsAndReturnStruct(ReduceSinkOperator.java:188)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:197)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:85)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:532)
> {code}
> I think the deficient number of key expression, compared to number of key column, is the problem, which should be equal or more.
> Would it be solved if add some key expression? I'll try.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2332) If all of the parameters of distinct
functions are exists in group by columns, query fails in runtime
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-2332:
---------------------------------
Fix Version/s: (was: 0.8.0)
> If all of the parameters of distinct functions are exists in group by columns, query fails in runtime
> -----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2332
> URL: https://issues.apache.org/jira/browse/HIVE-2332
> Project: Hive
> Issue Type: Bug
> Reporter: Navis
> Priority: Critical
> Attachments: HIVE-2332.1.patch, HIVE-2332.2.patch, HIVE-2332.3.patch
>
>
> select sum(key_int1), sum(distinct key_int1) from t1 group by key_int1;
> fails with message..
> {code}
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
> {code}
> hadoop says..
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
> at java.util.ArrayList.RangeCheck(ArrayList.java:547)
> at java.util.ArrayList.get(ArrayList.java:322)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:95)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.(StandardStructObjectInspector.java:86)
> at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:252)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initEvaluatorsAndReturnStruct(ReduceSinkOperator.java:188)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:197)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:85)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:532)
> {code}
> I think the deficient number of key expression, compared to number of key column, is the problem, which should be equal or more.
> Would it be solved if add some key expression? I'll try.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2332) If parameter of distinct function is
contained in group by columns, it fails in runtime
Posted by "Navis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Navis updated HIVE-2332:
------------------------
Attachment: HIVE-2332.1.patch
Simple bypass.
Removing not-used output key column made problems in following operations(GBY). So I've added dummy ExprNodeNullDesc for key expression and query has succeeded.
> If parameter of distinct function is contained in group by columns, it fails in runtime
> ---------------------------------------------------------------------------------------
>
> Key: HIVE-2332
> URL: https://issues.apache.org/jira/browse/HIVE-2332
> Project: Hive
> Issue Type: Bug
> Reporter: Navis
> Priority: Critical
> Fix For: 0.8.0
>
> Attachments: HIVE-2332.1.patch
>
>
> select sum(key_int1), sum(distinct key_int1) from t1 group by key_int1;
> fails with message..
> {code}
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
> {code}
> hadoop says..
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
> at java.util.ArrayList.RangeCheck(ArrayList.java:547)
> at java.util.ArrayList.get(ArrayList.java:322)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:95)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.(StandardStructObjectInspector.java:86)
> at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:252)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initEvaluatorsAndReturnStruct(ReduceSinkOperator.java:188)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:197)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:85)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:532)
> {code}
> I think the deficient number of key expression, compared to number of key column, is the problem, which should be equal or more.
> Would it be solved if add some key expression? I'll try.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2332) If all of the parameters of distinct
functions are exists in group by columns, query fails in runtime
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13078306#comment-13078306 ]
John Sichi commented on HIVE-2332:
----------------------------------
Good catch. Can you submit an updated patch which includes the test case (.q and .q.out)?
> If all of the parameters of distinct functions are exists in group by columns, query fails in runtime
> -----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2332
> URL: https://issues.apache.org/jira/browse/HIVE-2332
> Project: Hive
> Issue Type: Bug
> Reporter: Navis
> Priority: Critical
> Fix For: 0.8.0
>
> Attachments: HIVE-2332.1.patch
>
>
> select sum(key_int1), sum(distinct key_int1) from t1 group by key_int1;
> fails with message..
> {code}
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
> {code}
> hadoop says..
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
> at java.util.ArrayList.RangeCheck(ArrayList.java:547)
> at java.util.ArrayList.get(ArrayList.java:322)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:95)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.(StandardStructObjectInspector.java:86)
> at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:252)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initEvaluatorsAndReturnStruct(ReduceSinkOperator.java:188)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:197)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:85)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:532)
> {code}
> I think the deficient number of key expression, compared to number of key column, is the problem, which should be equal or more.
> Would it be solved if add some key expression? I'll try.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2332) If all of the parameters of distinct
functions are exists in group by columns, query fails in runtime
Posted by "Navis (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Navis updated HIVE-2332:
------------------------
Attachment: HIVE-2332.1.patch.txt
> If all of the parameters of distinct functions are exists in group by columns, query fails in runtime
> -----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2332
> URL: https://issues.apache.org/jira/browse/HIVE-2332
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Navis
> Assignee: Navis
> Priority: Critical
> Attachments: HIVE-2332.0.8.0.patch, HIVE-2332.1.patch.txt
>
>
> select sum(key_int1), sum(distinct key_int1) from t1 group by key_int1;
> fails with message..
> {code}
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
> {code}
> hadoop says..
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
> at java.util.ArrayList.RangeCheck(ArrayList.java:547)
> at java.util.ArrayList.get(ArrayList.java:322)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:95)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.(StandardStructObjectInspector.java:86)
> at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:252)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initEvaluatorsAndReturnStruct(ReduceSinkOperator.java:188)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:197)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:85)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:532)
> {code}
> I think the deficient number of key expression, compared to number of key column, is the problem, which should be equal or more.
> Would it be solved if add some key expression? I'll try.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2332) If all of the parameters of distinct
functions are exists in group by columns, query fails in runtime
Posted by "Navis (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Navis updated HIVE-2332:
------------------------
Attachment: (was: HIVE-2332.1.patch)
> If all of the parameters of distinct functions are exists in group by columns, query fails in runtime
> -----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2332
> URL: https://issues.apache.org/jira/browse/HIVE-2332
> Project: Hive
> Issue Type: Bug
> Reporter: Navis
> Assignee: Navis
> Priority: Critical
> Attachments: HIVE-2332.0.8.0.patch
>
>
> select sum(key_int1), sum(distinct key_int1) from t1 group by key_int1;
> fails with message..
> {code}
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
> {code}
> hadoop says..
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
> at java.util.ArrayList.RangeCheck(ArrayList.java:547)
> at java.util.ArrayList.get(ArrayList.java:322)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:95)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.(StandardStructObjectInspector.java:86)
> at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:252)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initEvaluatorsAndReturnStruct(ReduceSinkOperator.java:188)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:197)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:85)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:532)
> {code}
> I think the deficient number of key expression, compared to number of key column, is the problem, which should be equal or more.
> Would it be solved if add some key expression? I'll try.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2332) If all of the parameters of distinct
functions are exists in group by columns, query fails in runtime
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150313#comment-13150313 ]
jiraposter@reviews.apache.org commented on HIVE-2332:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1314/
-----------------------------------------------------------
(Updated 2011-11-15 08:56:07.599798)
Review request for hive and Ashutosh Chauhan.
Changes
-------
Rebased to trunk.
Summary
-------
If all of the distinct params are in group by keys, union column reserved for distinct params would not be added, which making problems initializing RS operator.
This patch is just a simple bypass adding dummy expression for the union column. Someone would know better way to resolve the problem.
This addresses bug HIVE-2332.
https://issues.apache.org/jira/browse/HIVE-2332
Diffs (updated)
-----
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java d60ba09
ql/src/test/queries/clientpositive/groupby_distinct_samekey.q PRE-CREATION
ql/src/test/results/clientpositive/groupby_distinct_samekey.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/1314/diff
Testing (updated)
-------
added clientpositive/groupby_distinct_samekey.q
Thanks,
Navis
> If all of the parameters of distinct functions are exists in group by columns, query fails in runtime
> -----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2332
> URL: https://issues.apache.org/jira/browse/HIVE-2332
> Project: Hive
> Issue Type: Bug
> Reporter: Navis
> Assignee: Navis
> Priority: Critical
> Attachments: HIVE-2332.0.8.0.patch
>
>
> select sum(key_int1), sum(distinct key_int1) from t1 group by key_int1;
> fails with message..
> {code}
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
> {code}
> hadoop says..
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
> at java.util.ArrayList.RangeCheck(ArrayList.java:547)
> at java.util.ArrayList.get(ArrayList.java:322)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:95)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.(StandardStructObjectInspector.java:86)
> at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:252)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initEvaluatorsAndReturnStruct(ReduceSinkOperator.java:188)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:197)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:85)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:532)
> {code}
> I think the deficient number of key expression, compared to number of key column, is the problem, which should be equal or more.
> Would it be solved if add some key expression? I'll try.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2332) If all of the parameters of distinct
functions are exists in group by columns, query fails in runtime
Posted by "Edward Capriolo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Capriolo updated HIVE-2332:
----------------------------------
Priority: Blocker (was: Critical)
> If all of the parameters of distinct functions are exists in group by columns, query fails in runtime
> -----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2332
> URL: https://issues.apache.org/jira/browse/HIVE-2332
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.9.0
> Reporter: Navis
> Assignee: Navis
> Priority: Blocker
> Attachments: HIVE-2332.1.patch.txt, HIVE-2332.2.patch.txt, HIVE-2332.D663.1.patch
>
>
> select sum(key_int1), sum(distinct key_int1) from t1 group by key_int1;
> fails with message..
> {code}
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
> {code}
> hadoop says..
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
> at java.util.ArrayList.RangeCheck(ArrayList.java:547)
> at java.util.ArrayList.get(ArrayList.java:322)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:95)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.(StandardStructObjectInspector.java:86)
> at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:252)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initEvaluatorsAndReturnStruct(ReduceSinkOperator.java:188)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:197)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:85)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:532)
> {code}
> I think the deficient number of key expression, compared to number of key column, is the problem, which should be equal or more.
> Would it be solved if add some key expression? I'll try.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-2332) If all of the parameters of distinct
functions are exists in group by columns, query fails in runtime
Posted by "Ashutosh Chauhan (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan reassigned HIVE-2332:
--------------------------------------
Assignee: Navis
Assigning to Navis. Navis, also mark the issue as Patch Available if its ready for review.
> If all of the parameters of distinct functions are exists in group by columns, query fails in runtime
> -----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2332
> URL: https://issues.apache.org/jira/browse/HIVE-2332
> Project: Hive
> Issue Type: Bug
> Reporter: Navis
> Assignee: Navis
> Priority: Critical
> Attachments: HIVE-2332.1.patch, HIVE-2332.2.patch, HIVE-2332.3.patch
>
>
> select sum(key_int1), sum(distinct key_int1) from t1 group by key_int1;
> fails with message..
> {code}
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
> {code}
> hadoop says..
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
> at java.util.ArrayList.RangeCheck(ArrayList.java:547)
> at java.util.ArrayList.get(ArrayList.java:322)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:95)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.(StandardStructObjectInspector.java:86)
> at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:252)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initEvaluatorsAndReturnStruct(ReduceSinkOperator.java:188)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:197)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:85)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:532)
> {code}
> I think the deficient number of key expression, compared to number of key column, is the problem, which should be equal or more.
> Would it be solved if add some key expression? I'll try.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2332) If all of the parameters of distinct
functions are exists in group by columns, query fails in runtime
Posted by "Ashutosh Chauhan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan updated HIVE-2332:
-----------------------------------
Status: Open (was: Patch Available)
Canceling patch as it first needs to address Namit's comments.
> If all of the parameters of distinct functions are exists in group by columns, query fails in runtime
> -----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2332
> URL: https://issues.apache.org/jira/browse/HIVE-2332
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Navis
> Assignee: Navis
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2332.1.patch.txt, HIVE-2332.2.patch.txt, HIVE-2332.D663.1.patch
>
>
> select sum(key_int1), sum(distinct key_int1) from t1 group by key_int1;
> fails with message..
> {code}
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
> {code}
> hadoop says..
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
> at java.util.ArrayList.RangeCheck(ArrayList.java:547)
> at java.util.ArrayList.get(ArrayList.java:322)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:95)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.(StandardStructObjectInspector.java:86)
> at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:252)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initEvaluatorsAndReturnStruct(ReduceSinkOperator.java:188)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:197)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:85)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:532)
> {code}
> I think the deficient number of key expression, compared to number of key column, is the problem, which should be equal or more.
> Would it be solved if add some key expression? I'll try.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2332) If all of the parameters of distinct
functions are exists in group by columns, query fails in runtime
Posted by "Namit Jain (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165374#comment-13165374 ]
Namit Jain commented on HIVE-2332:
----------------------------------
Navis, please use the following instructions in future for patches
https://cwiki.apache.org/Hive/phabricatorcodereview.html.
We are planning to move to phabricator from review board.
I have already done this for this jira.
Will review it soon.
> If all of the parameters of distinct functions are exists in group by columns, query fails in runtime
> -----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2332
> URL: https://issues.apache.org/jira/browse/HIVE-2332
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Navis
> Assignee: Navis
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2332.1.patch.txt, HIVE-2332.2.patch.txt, HIVE-2332.D663.1.patch
>
>
> select sum(key_int1), sum(distinct key_int1) from t1 group by key_int1;
> fails with message..
> {code}
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
> {code}
> hadoop says..
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
> at java.util.ArrayList.RangeCheck(ArrayList.java:547)
> at java.util.ArrayList.get(ArrayList.java:322)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:95)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.(StandardStructObjectInspector.java:86)
> at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:252)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initEvaluatorsAndReturnStruct(ReduceSinkOperator.java:188)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:197)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:85)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:532)
> {code}
> I think the deficient number of key expression, compared to number of key column, is the problem, which should be equal or more.
> Would it be solved if add some key expression? I'll try.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2332) If all of the parameters of distinct
functions are exists in group by columns, query fails in runtime
Posted by "Navis (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Navis updated HIVE-2332:
------------------------
Attachment: (was: HIVE-2332.3.patch)
> If all of the parameters of distinct functions are exists in group by columns, query fails in runtime
> -----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2332
> URL: https://issues.apache.org/jira/browse/HIVE-2332
> Project: Hive
> Issue Type: Bug
> Reporter: Navis
> Assignee: Navis
> Priority: Critical
> Attachments: HIVE-2332.0.8.0.patch
>
>
> select sum(key_int1), sum(distinct key_int1) from t1 group by key_int1;
> fails with message..
> {code}
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
> {code}
> hadoop says..
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
> at java.util.ArrayList.RangeCheck(ArrayList.java:547)
> at java.util.ArrayList.get(ArrayList.java:322)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:95)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.(StandardStructObjectInspector.java:86)
> at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:252)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initEvaluatorsAndReturnStruct(ReduceSinkOperator.java:188)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:197)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:85)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:532)
> {code}
> I think the deficient number of key expression, compared to number of key column, is the problem, which should be equal or more.
> Would it be solved if add some key expression? I'll try.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2332) If all of the parameters of distinct
functions are exists in group by columns, query fails in runtime
Posted by "Navis (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Navis updated HIVE-2332:
------------------------
Attachment: HIVE-2332.0.8.0.patch
> If all of the parameters of distinct functions are exists in group by columns, query fails in runtime
> -----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2332
> URL: https://issues.apache.org/jira/browse/HIVE-2332
> Project: Hive
> Issue Type: Bug
> Reporter: Navis
> Assignee: Navis
> Priority: Critical
> Attachments: HIVE-2332.0.8.0.patch
>
>
> select sum(key_int1), sum(distinct key_int1) from t1 group by key_int1;
> fails with message..
> {code}
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
> {code}
> hadoop says..
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
> at java.util.ArrayList.RangeCheck(ArrayList.java:547)
> at java.util.ArrayList.get(ArrayList.java:322)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:95)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.(StandardStructObjectInspector.java:86)
> at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:252)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initEvaluatorsAndReturnStruct(ReduceSinkOperator.java:188)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:197)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:85)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:532)
> {code}
> I think the deficient number of key expression, compared to number of key column, is the problem, which should be equal or more.
> Would it be solved if add some key expression? I'll try.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2332) If all of the parameters of distinct
functions are exists in group by columns, query fails in runtime
Posted by "Navis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Navis updated HIVE-2332:
------------------------
Attachment: HIVE-2332.2.patch
> If all of the parameters of distinct functions are exists in group by columns, query fails in runtime
> -----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2332
> URL: https://issues.apache.org/jira/browse/HIVE-2332
> Project: Hive
> Issue Type: Bug
> Reporter: Navis
> Priority: Critical
> Fix For: 0.8.0
>
> Attachments: HIVE-2332.1.patch, HIVE-2332.2.patch, HIVE-2332.2.patch
>
>
> select sum(key_int1), sum(distinct key_int1) from t1 group by key_int1;
> fails with message..
> {code}
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
> {code}
> hadoop says..
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
> at java.util.ArrayList.RangeCheck(ArrayList.java:547)
> at java.util.ArrayList.get(ArrayList.java:322)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:95)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.(StandardStructObjectInspector.java:86)
> at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:252)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initEvaluatorsAndReturnStruct(ReduceSinkOperator.java:188)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:197)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:85)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:532)
> {code}
> I think the deficient number of key expression, compared to number of key column, is the problem, which should be equal or more.
> Would it be solved if add some key expression? I'll try.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2332) If all of the parameters of distinct
functions are exists in group by columns, query fails in runtime
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13078328#comment-13078328 ]
John Sichi commented on HIVE-2332:
----------------------------------
Once you have the patch ready, also please create a Review Board entry:
https://cwiki.apache.org/confluence/display/Hive/HowToContribute#HowToContribute-ReviewProcess
Thanks!
> If all of the parameters of distinct functions are exists in group by columns, query fails in runtime
> -----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2332
> URL: https://issues.apache.org/jira/browse/HIVE-2332
> Project: Hive
> Issue Type: Bug
> Reporter: Navis
> Priority: Critical
> Fix For: 0.8.0
>
> Attachments: HIVE-2332.1.patch
>
>
> select sum(key_int1), sum(distinct key_int1) from t1 group by key_int1;
> fails with message..
> {code}
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
> {code}
> hadoop says..
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
> at java.util.ArrayList.RangeCheck(ArrayList.java:547)
> at java.util.ArrayList.get(ArrayList.java:322)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:95)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.(StandardStructObjectInspector.java:86)
> at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:252)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initEvaluatorsAndReturnStruct(ReduceSinkOperator.java:188)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:197)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:85)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:532)
> {code}
> I think the deficient number of key expression, compared to number of key column, is the problem, which should be equal or more.
> Would it be solved if add some key expression? I'll try.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2332) If all of the parameters of distinct
functions are exists in group by columns, query fails in runtime
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2332:
------------------------------
Attachment: HIVE-2332.D663.1.patch
njain requested code review of "HIVE-2332 [jira] If all of the parameters of distinct functions are exists in group by columns, query fails in runtime".
Reviewers: JIRA
HIVE-2332
select sum(key_int1), sum(distinct key_int1) from t1 group by key_int1;
fails with message.. <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">FAILED: Execution Error, <span class="code-keyword">return</span> code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask</pre> </div></div>
hadoop says.. <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1 at java.util.ArrayList.RangeCheck(ArrayList.java:547) at java.util.ArrayList.get(ArrayList.java:322) at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:95) at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.(StandardStructObjectInspector.java:86) at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:252) at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initEvaluatorsAndReturnStruct(ReduceSinkOperator.java:188) at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:197) at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471) at org.apache.hadoop.hive.ql.e
xec.Operator.forward(Operator.java:744) at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:85) at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471) at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744) at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:532)</pre> </div></div>
I think the deficient number of key expression, compared to number of key column, is the problem, which should be equal or more.
Would it be solved if add some key expression? I'll try.
TEST PLAN
EMPTY
REVISION DETAIL
https://reviews.facebook.net/D663
AFFECTED FILES
ql/src/test/results/clientpositive/groupby_distinct_samekey.q.out
ql/src/test/queries/clientpositive/groupby_distinct_samekey.q
ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java
ql/src/java/org/apache/hadoop/hive/ql/plan/PlanUtils.java
MANAGE HERALD DIFFERENTIAL RULES
https://reviews.facebook.net/herald/view/differential/
WHY DID I GET THIS EMAIL?
https://reviews.facebook.net/herald/transcript/1473/
Tip: use the X-Herald-Rules header to filter Herald messages in your client.
> If all of the parameters of distinct functions are exists in group by columns, query fails in runtime
> -----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2332
> URL: https://issues.apache.org/jira/browse/HIVE-2332
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Navis
> Assignee: Navis
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2332.1.patch.txt, HIVE-2332.2.patch.txt, HIVE-2332.D663.1.patch
>
>
> select sum(key_int1), sum(distinct key_int1) from t1 group by key_int1;
> fails with message..
> {code}
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
> {code}
> hadoop says..
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
> at java.util.ArrayList.RangeCheck(ArrayList.java:547)
> at java.util.ArrayList.get(ArrayList.java:322)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:95)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.(StandardStructObjectInspector.java:86)
> at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:252)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initEvaluatorsAndReturnStruct(ReduceSinkOperator.java:188)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:197)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:85)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:532)
> {code}
> I think the deficient number of key expression, compared to number of key column, is the problem, which should be equal or more.
> Would it be solved if add some key expression? I'll try.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2332) If all of the parameters of distinct
functions are exists in group by columns, query fails in runtime
Posted by "Ashutosh Chauhan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan updated HIVE-2332:
-----------------------------------
Status: Patch Available (was: Open)
Oh, I missed it. Marking Patch Available then.
> If all of the parameters of distinct functions are exists in group by columns, query fails in runtime
> -----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2332
> URL: https://issues.apache.org/jira/browse/HIVE-2332
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Navis
> Assignee: Navis
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2332.1.patch.txt, HIVE-2332.2.patch.txt, HIVE-2332.D663.1.patch
>
>
> select sum(key_int1), sum(distinct key_int1) from t1 group by key_int1;
> fails with message..
> {code}
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
> {code}
> hadoop says..
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
> at java.util.ArrayList.RangeCheck(ArrayList.java:547)
> at java.util.ArrayList.get(ArrayList.java:322)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:95)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.(StandardStructObjectInspector.java:86)
> at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:252)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initEvaluatorsAndReturnStruct(ReduceSinkOperator.java:188)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:197)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:85)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:532)
> {code}
> I think the deficient number of key expression, compared to number of key column, is the problem, which should be equal or more.
> Would it be solved if add some key expression? I'll try.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2332) If all of the parameters of distinct
functions are exists in group by columns, query fails in runtime
Posted by "Ashutosh Chauhan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan updated HIVE-2332:
-----------------------------------
Affects Version/s: 0.9.0
Fix Version/s: (was: 0.9.0)
Unlinking from 0.9
> If all of the parameters of distinct functions are exists in group by columns, query fails in runtime
> -----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2332
> URL: https://issues.apache.org/jira/browse/HIVE-2332
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.9.0
> Reporter: Navis
> Assignee: Navis
> Priority: Critical
> Attachments: HIVE-2332.1.patch.txt, HIVE-2332.2.patch.txt, HIVE-2332.D663.1.patch
>
>
> select sum(key_int1), sum(distinct key_int1) from t1 group by key_int1;
> fails with message..
> {code}
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
> {code}
> hadoop says..
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
> at java.util.ArrayList.RangeCheck(ArrayList.java:547)
> at java.util.ArrayList.get(ArrayList.java:322)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:95)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.(StandardStructObjectInspector.java:86)
> at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:252)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initEvaluatorsAndReturnStruct(ReduceSinkOperator.java:188)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:197)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:85)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:532)
> {code}
> I think the deficient number of key expression, compared to number of key column, is the problem, which should be equal or more.
> Would it be solved if add some key expression? I'll try.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2332) If all of the parameters of distinct
functions are exists in group by columns, query fails in runtime
Posted by "Navis (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Navis updated HIVE-2332:
------------------------
Attachment: (was: HIVE-2332.2.patch)
> If all of the parameters of distinct functions are exists in group by columns, query fails in runtime
> -----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2332
> URL: https://issues.apache.org/jira/browse/HIVE-2332
> Project: Hive
> Issue Type: Bug
> Reporter: Navis
> Assignee: Navis
> Priority: Critical
> Attachments: HIVE-2332.0.8.0.patch
>
>
> select sum(key_int1), sum(distinct key_int1) from t1 group by key_int1;
> fails with message..
> {code}
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
> {code}
> hadoop says..
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
> at java.util.ArrayList.RangeCheck(ArrayList.java:547)
> at java.util.ArrayList.get(ArrayList.java:322)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:95)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.(StandardStructObjectInspector.java:86)
> at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:252)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initEvaluatorsAndReturnStruct(ReduceSinkOperator.java:188)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:197)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:85)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:532)
> {code}
> I think the deficient number of key expression, compared to number of key column, is the problem, which should be equal or more.
> Would it be solved if add some key expression? I'll try.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2332) If all of the parameters of distinct
functions are exists in group by columns, query fails in runtime
Posted by "Navis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Navis updated HIVE-2332:
------------------------
Attachment: HIVE-2332.2.patch
1. Add deficient columns lastly to avoid index mismatch with RR of HASH GBY
2. ExprNodeNullDesc to ExprNodeFieldDesc to avoid NPE
> If all of the parameters of distinct functions are exists in group by columns, query fails in runtime
> -----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2332
> URL: https://issues.apache.org/jira/browse/HIVE-2332
> Project: Hive
> Issue Type: Bug
> Reporter: Navis
> Priority: Critical
> Fix For: 0.8.0
>
> Attachments: HIVE-2332.1.patch, HIVE-2332.2.patch
>
>
> select sum(key_int1), sum(distinct key_int1) from t1 group by key_int1;
> fails with message..
> {code}
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
> {code}
> hadoop says..
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
> at java.util.ArrayList.RangeCheck(ArrayList.java:547)
> at java.util.ArrayList.get(ArrayList.java:322)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:95)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.(StandardStructObjectInspector.java:86)
> at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:252)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initEvaluatorsAndReturnStruct(ReduceSinkOperator.java:188)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:197)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:85)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:532)
> {code}
> I think the deficient number of key expression, compared to number of key column, is the problem, which should be equal or more.
> Would it be solved if add some key expression? I'll try.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2332) If all of the parameters of distinct
functions are exists in group by columns, query fails in runtime
Posted by "Navis (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Navis updated HIVE-2332:
------------------------
Fix Version/s: 0.9.0
Status: Patch Available (was: Open)
> If all of the parameters of distinct functions are exists in group by columns, query fails in runtime
> -----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2332
> URL: https://issues.apache.org/jira/browse/HIVE-2332
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Navis
> Assignee: Navis
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2332.1.patch.txt
>
>
> select sum(key_int1), sum(distinct key_int1) from t1 group by key_int1;
> fails with message..
> {code}
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
> {code}
> hadoop says..
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
> at java.util.ArrayList.RangeCheck(ArrayList.java:547)
> at java.util.ArrayList.get(ArrayList.java:322)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:95)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.(StandardStructObjectInspector.java:86)
> at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:252)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initEvaluatorsAndReturnStruct(ReduceSinkOperator.java:188)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:197)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:85)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:532)
> {code}
> I think the deficient number of key expression, compared to number of key column, is the problem, which should be equal or more.
> Would it be solved if add some key expression? I'll try.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2332) If all of the parameters of distinct
functions are exists in group by columns, query fails in runtime
Posted by "Navis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Navis updated HIVE-2332:
------------------------
Summary: If all of the parameters of distinct functions are exists in group by columns, query fails in runtime (was: If parameter of distinct function is contained in group by columns, it fails in runtime)
> If all of the parameters of distinct functions are exists in group by columns, query fails in runtime
> -----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2332
> URL: https://issues.apache.org/jira/browse/HIVE-2332
> Project: Hive
> Issue Type: Bug
> Reporter: Navis
> Priority: Critical
> Fix For: 0.8.0
>
> Attachments: HIVE-2332.1.patch
>
>
> select sum(key_int1), sum(distinct key_int1) from t1 group by key_int1;
> fails with message..
> {code}
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
> {code}
> hadoop says..
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
> at java.util.ArrayList.RangeCheck(ArrayList.java:547)
> at java.util.ArrayList.get(ArrayList.java:322)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:95)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.(StandardStructObjectInspector.java:86)
> at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:252)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initEvaluatorsAndReturnStruct(ReduceSinkOperator.java:188)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:197)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:85)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:532)
> {code}
> I think the deficient number of key expression, compared to number of key column, is the problem, which should be equal or more.
> Would it be solved if add some key expression? I'll try.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2332) If all of the parameters of distinct
functions are exists in group by columns, query fails in runtime
Posted by "Navis (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Navis updated HIVE-2332:
------------------------
Attachment: (was: HIVE-2332.0.8.0.patch)
> If all of the parameters of distinct functions are exists in group by columns, query fails in runtime
> -----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2332
> URL: https://issues.apache.org/jira/browse/HIVE-2332
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Navis
> Assignee: Navis
> Priority: Critical
> Attachments: HIVE-2332.1.patch.txt
>
>
> select sum(key_int1), sum(distinct key_int1) from t1 group by key_int1;
> fails with message..
> {code}
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
> {code}
> hadoop says..
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
> at java.util.ArrayList.RangeCheck(ArrayList.java:547)
> at java.util.ArrayList.get(ArrayList.java:322)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:95)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.(StandardStructObjectInspector.java:86)
> at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:252)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initEvaluatorsAndReturnStruct(ReduceSinkOperator.java:188)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:197)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:85)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:532)
> {code}
> I think the deficient number of key expression, compared to number of key column, is the problem, which should be equal or more.
> Would it be solved if add some key expression? I'll try.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2332) If all of the parameters of distinct
functions are exists in group by columns, query fails in runtime
Posted by "Navis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Navis updated HIVE-2332:
------------------------
Attachment: (was: HIVE-2332.2.patch)
> If all of the parameters of distinct functions are exists in group by columns, query fails in runtime
> -----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2332
> URL: https://issues.apache.org/jira/browse/HIVE-2332
> Project: Hive
> Issue Type: Bug
> Reporter: Navis
> Priority: Critical
> Fix For: 0.8.0
>
> Attachments: HIVE-2332.1.patch, HIVE-2332.2.patch
>
>
> select sum(key_int1), sum(distinct key_int1) from t1 group by key_int1;
> fails with message..
> {code}
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
> {code}
> hadoop says..
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
> at java.util.ArrayList.RangeCheck(ArrayList.java:547)
> at java.util.ArrayList.get(ArrayList.java:322)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:95)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.(StandardStructObjectInspector.java:86)
> at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:252)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initEvaluatorsAndReturnStruct(ReduceSinkOperator.java:188)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:197)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:85)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:532)
> {code}
> I think the deficient number of key expression, compared to number of key column, is the problem, which should be equal or more.
> Would it be solved if add some key expression? I'll try.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2332) If all of the parameters of distinct
functions are exists in group by columns, query fails in runtime
Posted by "Carl Steinbach (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-2332:
---------------------------------
Component/s: Query Processor
Is this patch ready for review? Is so, please change the status to Patch Available. Thanks.
> If all of the parameters of distinct functions are exists in group by columns, query fails in runtime
> -----------------------------------------------------------------------------------------------------
>
> Key: HIVE-2332
> URL: https://issues.apache.org/jira/browse/HIVE-2332
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Navis
> Assignee: Navis
> Priority: Critical
> Attachments: HIVE-2332.0.8.0.patch
>
>
> select sum(key_int1), sum(distinct key_int1) from t1 group by key_int1;
> fails with message..
> {code}
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
> {code}
> hadoop says..
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
> at java.util.ArrayList.RangeCheck(ArrayList.java:547)
> at java.util.ArrayList.get(ArrayList.java:322)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:95)
> at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.(StandardStructObjectInspector.java:86)
> at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:252)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initEvaluatorsAndReturnStruct(ReduceSinkOperator.java:188)
> at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:197)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:85)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:532)
> {code}
> I think the deficient number of key expression, compared to number of key column, is the problem, which should be equal or more.
> Would it be solved if add some key expression? I'll try.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira