You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Jesús Camacho Rodríguez <jc...@hortonworks.com> on 2017/03/03 10:25:47 UTC
Review Request 57274: HIVE-16102
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57274/
-----------------------------------------------------------
Review request for hive and Ashutosh Chauhan.
Bugs: HIVE-16102
https://issues.apache.org/jira/browse/HIVE-16102
Repository: hive-git
Description
-------
HIVE-16102
Diffs
-----
ql/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java 2ffc1306e727ccde15e507a2bd087d9e964e734e
ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java be561cef8fdec2fc9fdc194e1c4771497582c5a9
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java db8d46e2dca44605a4068c27b4be9f13401f46b0
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 0872e535a9b6a09569c02fc498dab16867ca8783
ql/src/test/results/clientpositive/groupby_cube1.q.out 0486b689f62a98bf2994b2f1d964a36e85b707d2
ql/src/test/results/clientpositive/groupby_grouping_id1.q.out 9ef76155be18f103063fea699d78029acbec1300
ql/src/test/results/clientpositive/groupby_grouping_sets_grouping.q.out 6917dbabb96b5a582416ac7f07968b739823a13f
ql/src/test/results/clientpositive/groupby_grouping_window.q.out 202fad093fee28bdbf347e9d89b61ada5dfaa72f
Diff: https://reviews.apache.org/r/57274/diff/1/
Testing
-------
Thanks,
Jes�s Camacho Rodr�guez
Re: Review Request 57274: HIVE-16102
Posted by Jesús Camacho Rodríguez <jc...@hortonworks.com>.
> On March 7, 2017, 5:50 p.m., Ashutosh Chauhan wrote:
> > ql/src/test/results/clientpositive/llap/multi_count_distinct_null.q.out
> > Lines 292-312 (original), 292-312 (patched)
> > <https://reviews.apache.org/r/57274/diff/3/?file=1657987#file1657987line292>
> >
> > yeah.. that looks correct..
> > GROUPING_ID(DEPARTMENT_ID,GENDER,EDUCATION_LEVEL) DEPARTMENT_ID GENDER EDUCATION_LEVEL
> > 2 - - -
> > 2 - - 1
> > 2 1 - 1
> > 2 2 - 1
> > 2 4 - 1
> > 2 1 - 2
> > 2 3 - 2
> > 2 1 - 3
> > 2 2 - 3
> > 6 - - -
> > 6 - - 1
> > 6 - - 2
> > 6 - - 3
> > 3 - - -
> > 3 1 - -
> > 3 2 - -
> > 3 4 - -
> > 3 3 - -
> > 5 - - -
> > 5 - M -
> > 5 - F -
> >
> > 21 rows selected.
Yes, it was because of the composite/multi grouping set syntax, it is very confusing. I will close it as fixed.
- Jes�s
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57274/#review168160
-----------------------------------------------------------
On March 7, 2017, 9:55 a.m., Jes�s Camacho Rodr�guez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57274/
> -----------------------------------------------------------
>
> (Updated March 7, 2017, 9:55 a.m.)
>
>
> Review request for hive and Ashutosh Chauhan.
>
>
> Bugs: HIVE-16102
> https://issues.apache.org/jira/browse/HIVE-16102
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> HIVE-16102
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java 2ffc1306e727ccde15e507a2bd087d9e964e734e
> ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java be561cef8fdec2fc9fdc194e1c4771497582c5a9
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveExpandDistinctAggregatesRule.java 82624fc33f31f742b25f0a6887efabe7e2834947
> ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java db8d46e2dca44605a4068c27b4be9f13401f46b0
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 0872e535a9b6a09569c02fc498dab16867ca8783
> ql/src/test/queries/clientpositive/multi_count_distinct.q 855cb64c889e4c4309eaf9ee0745f528e611bd54
> ql/src/test/queries/clientpositive/multi_count_distinct_null.q 442d855d86129f0725f4b7c50df59e25c6c6b6cc
> ql/src/test/results/clientnegative/groupby_grouping_sets1.q.out 8290d4c4686455b9b689c8de876e765d5859a111
> ql/src/test/results/clientnegative/groupby_grouping_sets2.q.out 8290d4c4686455b9b689c8de876e765d5859a111
> ql/src/test/results/clientpositive/cte_1.q.out d13ad42a19d866b9a701d02d0bac0d62fb2a3dfa
> ql/src/test/results/clientpositive/groupby_cube1.q.out 0486b689f62a98bf2994b2f1d964a36e85b707d2
> ql/src/test/results/clientpositive/groupby_grouping_id1.q.out 9ef76155be18f103063fea699d78029acbec1300
> ql/src/test/results/clientpositive/groupby_grouping_sets_grouping.q.out 6917dbabb96b5a582416ac7f07968b739823a13f
> ql/src/test/results/clientpositive/groupby_grouping_window.q.out 202fad093fee28bdbf347e9d89b61ada5dfaa72f
> ql/src/test/results/clientpositive/infer_bucket_sort_grouping_operators.q.out d740dea0d6c0b4b284afab3d6f13032407351a3f
> ql/src/test/results/clientpositive/llap/cte_1.q.out 1e6a4385d3f26a5e4ffa502688a95da977d0a4ae
> ql/src/test/results/clientpositive/llap/groupby_grouping_id2.q.out 9c85852863cf4bc858e1b30511764fa33ac57ece
> ql/src/test/results/clientpositive/llap/multi_count_distinct_null.q.out 0cbb9deba0ed6a33faa122b9d6d75fccdcbc6bfb
> ql/src/test/results/clientpositive/llap/vector_grouping_sets.q.out 21769170d948ebf576367fc30861101d0c4f5423
> ql/src/test/results/clientpositive/spark/groupby_cube1.q.out 56b8c2f382840bdc05afffceae1c8b7b1dd6b776
> ql/src/test/results/clientpositive/spark/groupby_grouping_id2.q.out 9c85852863cf4bc858e1b30511764fa33ac57ece
> ql/src/test/results/clientpositive/tez/multi_count_distinct.q.out ef3fcabb4c4184831ba06d9017a68d5fd2a7c687
> ql/src/test/results/clientpositive/vector_grouping_sets.q.out df4631e5f3434be99ebb063e2ff0c45383928f7f
>
>
> Diff: https://reviews.apache.org/r/57274/diff/3/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Jes�s Camacho Rodr�guez
>
>
Re: Review Request 57274: HIVE-16102
Posted by Ashutosh Chauhan <ha...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57274/#review168160
-----------------------------------------------------------
ql/src/test/results/clientpositive/llap/multi_count_distinct_null.q.out
Lines 292-312 (original), 292-312 (patched)
<https://reviews.apache.org/r/57274/#comment240306>
yeah.. that looks correct..
GROUPING_ID(DEPARTMENT_ID,GENDER,EDUCATION_LEVEL) DEPARTMENT_ID GENDER EDUCATION_LEVEL
2 - - -
2 - - 1
2 1 - 1
2 2 - 1
2 4 - 1
2 1 - 2
2 3 - 2
2 1 - 3
2 2 - 3
6 - - -
6 - - 1
6 - - 2
6 - - 3
3 - - -
3 1 - -
3 2 - -
3 4 - -
3 3 - -
5 - - -
5 - M -
5 - F -
21 rows selected.
- Ashutosh Chauhan
On March 7, 2017, 9:55 a.m., Jes�s Camacho Rodr�guez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57274/
> -----------------------------------------------------------
>
> (Updated March 7, 2017, 9:55 a.m.)
>
>
> Review request for hive and Ashutosh Chauhan.
>
>
> Bugs: HIVE-16102
> https://issues.apache.org/jira/browse/HIVE-16102
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> HIVE-16102
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java 2ffc1306e727ccde15e507a2bd087d9e964e734e
> ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java be561cef8fdec2fc9fdc194e1c4771497582c5a9
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveExpandDistinctAggregatesRule.java 82624fc33f31f742b25f0a6887efabe7e2834947
> ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java db8d46e2dca44605a4068c27b4be9f13401f46b0
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 0872e535a9b6a09569c02fc498dab16867ca8783
> ql/src/test/queries/clientpositive/multi_count_distinct.q 855cb64c889e4c4309eaf9ee0745f528e611bd54
> ql/src/test/queries/clientpositive/multi_count_distinct_null.q 442d855d86129f0725f4b7c50df59e25c6c6b6cc
> ql/src/test/results/clientnegative/groupby_grouping_sets1.q.out 8290d4c4686455b9b689c8de876e765d5859a111
> ql/src/test/results/clientnegative/groupby_grouping_sets2.q.out 8290d4c4686455b9b689c8de876e765d5859a111
> ql/src/test/results/clientpositive/cte_1.q.out d13ad42a19d866b9a701d02d0bac0d62fb2a3dfa
> ql/src/test/results/clientpositive/groupby_cube1.q.out 0486b689f62a98bf2994b2f1d964a36e85b707d2
> ql/src/test/results/clientpositive/groupby_grouping_id1.q.out 9ef76155be18f103063fea699d78029acbec1300
> ql/src/test/results/clientpositive/groupby_grouping_sets_grouping.q.out 6917dbabb96b5a582416ac7f07968b739823a13f
> ql/src/test/results/clientpositive/groupby_grouping_window.q.out 202fad093fee28bdbf347e9d89b61ada5dfaa72f
> ql/src/test/results/clientpositive/infer_bucket_sort_grouping_operators.q.out d740dea0d6c0b4b284afab3d6f13032407351a3f
> ql/src/test/results/clientpositive/llap/cte_1.q.out 1e6a4385d3f26a5e4ffa502688a95da977d0a4ae
> ql/src/test/results/clientpositive/llap/groupby_grouping_id2.q.out 9c85852863cf4bc858e1b30511764fa33ac57ece
> ql/src/test/results/clientpositive/llap/multi_count_distinct_null.q.out 0cbb9deba0ed6a33faa122b9d6d75fccdcbc6bfb
> ql/src/test/results/clientpositive/llap/vector_grouping_sets.q.out 21769170d948ebf576367fc30861101d0c4f5423
> ql/src/test/results/clientpositive/spark/groupby_cube1.q.out 56b8c2f382840bdc05afffceae1c8b7b1dd6b776
> ql/src/test/results/clientpositive/spark/groupby_grouping_id2.q.out 9c85852863cf4bc858e1b30511764fa33ac57ece
> ql/src/test/results/clientpositive/tez/multi_count_distinct.q.out ef3fcabb4c4184831ba06d9017a68d5fd2a7c687
> ql/src/test/results/clientpositive/vector_grouping_sets.q.out df4631e5f3434be99ebb063e2ff0c45383928f7f
>
>
> Diff: https://reviews.apache.org/r/57274/diff/3/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Jes�s Camacho Rodr�guez
>
>
Re: Review Request 57274: HIVE-16102
Posted by Jesús Camacho Rodríguez <jc...@hortonworks.com>.
> On March 7, 2017, 5:17 p.m., Ashutosh Chauhan wrote:
> > ql/src/test/results/clientpositive/llap/multi_count_distinct_null.q.out
> > Lines 292-312 (original), 292-312 (patched)
> > <https://reviews.apache.org/r/57274/diff/3/?file=1657987#file1657987line292>
> >
> > Is this correct? Oracle returns grouping_id = 0 for all the rows here (which looks incorrect, too).
> >
> > Ran on oracle:
> > select grouping_id(department_id, gender,education_level), department_id, gender, education_level from employee
> > group by department_id, gender, education_level, grouping sets
> > (department_id, gender, education_level, (education_level, department_id));
I am double-checking one by one, but it looks correct at first sight (grouping ID matching NULL columns).
Does this query work in Oracle?
select grouping_id(department_id, gender,education_level), department_id, gender, education_level from employee
group by grouping sets (department_id, gender, education_level, (education_level, department_id));
- Jes�s
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57274/#review168151
-----------------------------------------------------------
On March 7, 2017, 9:55 a.m., Jes�s Camacho Rodr�guez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57274/
> -----------------------------------------------------------
>
> (Updated March 7, 2017, 9:55 a.m.)
>
>
> Review request for hive and Ashutosh Chauhan.
>
>
> Bugs: HIVE-16102
> https://issues.apache.org/jira/browse/HIVE-16102
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> HIVE-16102
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java 2ffc1306e727ccde15e507a2bd087d9e964e734e
> ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java be561cef8fdec2fc9fdc194e1c4771497582c5a9
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveExpandDistinctAggregatesRule.java 82624fc33f31f742b25f0a6887efabe7e2834947
> ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java db8d46e2dca44605a4068c27b4be9f13401f46b0
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 0872e535a9b6a09569c02fc498dab16867ca8783
> ql/src/test/queries/clientpositive/multi_count_distinct.q 855cb64c889e4c4309eaf9ee0745f528e611bd54
> ql/src/test/queries/clientpositive/multi_count_distinct_null.q 442d855d86129f0725f4b7c50df59e25c6c6b6cc
> ql/src/test/results/clientnegative/groupby_grouping_sets1.q.out 8290d4c4686455b9b689c8de876e765d5859a111
> ql/src/test/results/clientnegative/groupby_grouping_sets2.q.out 8290d4c4686455b9b689c8de876e765d5859a111
> ql/src/test/results/clientpositive/cte_1.q.out d13ad42a19d866b9a701d02d0bac0d62fb2a3dfa
> ql/src/test/results/clientpositive/groupby_cube1.q.out 0486b689f62a98bf2994b2f1d964a36e85b707d2
> ql/src/test/results/clientpositive/groupby_grouping_id1.q.out 9ef76155be18f103063fea699d78029acbec1300
> ql/src/test/results/clientpositive/groupby_grouping_sets_grouping.q.out 6917dbabb96b5a582416ac7f07968b739823a13f
> ql/src/test/results/clientpositive/groupby_grouping_window.q.out 202fad093fee28bdbf347e9d89b61ada5dfaa72f
> ql/src/test/results/clientpositive/infer_bucket_sort_grouping_operators.q.out d740dea0d6c0b4b284afab3d6f13032407351a3f
> ql/src/test/results/clientpositive/llap/cte_1.q.out 1e6a4385d3f26a5e4ffa502688a95da977d0a4ae
> ql/src/test/results/clientpositive/llap/groupby_grouping_id2.q.out 9c85852863cf4bc858e1b30511764fa33ac57ece
> ql/src/test/results/clientpositive/llap/multi_count_distinct_null.q.out 0cbb9deba0ed6a33faa122b9d6d75fccdcbc6bfb
> ql/src/test/results/clientpositive/llap/vector_grouping_sets.q.out 21769170d948ebf576367fc30861101d0c4f5423
> ql/src/test/results/clientpositive/spark/groupby_cube1.q.out 56b8c2f382840bdc05afffceae1c8b7b1dd6b776
> ql/src/test/results/clientpositive/spark/groupby_grouping_id2.q.out 9c85852863cf4bc858e1b30511764fa33ac57ece
> ql/src/test/results/clientpositive/tez/multi_count_distinct.q.out ef3fcabb4c4184831ba06d9017a68d5fd2a7c687
> ql/src/test/results/clientpositive/vector_grouping_sets.q.out df4631e5f3434be99ebb063e2ff0c45383928f7f
>
>
> Diff: https://reviews.apache.org/r/57274/diff/3/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Jes�s Camacho Rodr�guez
>
>
Re: Review Request 57274: HIVE-16102
Posted by Ashutosh Chauhan <ha...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57274/#review168151
-----------------------------------------------------------
ql/src/test/results/clientpositive/llap/multi_count_distinct_null.q.out
Lines 292-312 (original), 292-312 (patched)
<https://reviews.apache.org/r/57274/#comment240300>
Is this correct? Oracle returns grouping_id = 0 for all the rows here (which looks incorrect, too).
Ran on oracle:
select grouping_id(department_id, gender,education_level), department_id, gender, education_level from employee
group by department_id, gender, education_level, grouping sets
(department_id, gender, education_level, (education_level, department_id));
- Ashutosh Chauhan
On March 7, 2017, 9:55 a.m., Jes�s Camacho Rodr�guez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57274/
> -----------------------------------------------------------
>
> (Updated March 7, 2017, 9:55 a.m.)
>
>
> Review request for hive and Ashutosh Chauhan.
>
>
> Bugs: HIVE-16102
> https://issues.apache.org/jira/browse/HIVE-16102
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> HIVE-16102
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java 2ffc1306e727ccde15e507a2bd087d9e964e734e
> ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java be561cef8fdec2fc9fdc194e1c4771497582c5a9
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveExpandDistinctAggregatesRule.java 82624fc33f31f742b25f0a6887efabe7e2834947
> ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java db8d46e2dca44605a4068c27b4be9f13401f46b0
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 0872e535a9b6a09569c02fc498dab16867ca8783
> ql/src/test/queries/clientpositive/multi_count_distinct.q 855cb64c889e4c4309eaf9ee0745f528e611bd54
> ql/src/test/queries/clientpositive/multi_count_distinct_null.q 442d855d86129f0725f4b7c50df59e25c6c6b6cc
> ql/src/test/results/clientnegative/groupby_grouping_sets1.q.out 8290d4c4686455b9b689c8de876e765d5859a111
> ql/src/test/results/clientnegative/groupby_grouping_sets2.q.out 8290d4c4686455b9b689c8de876e765d5859a111
> ql/src/test/results/clientpositive/cte_1.q.out d13ad42a19d866b9a701d02d0bac0d62fb2a3dfa
> ql/src/test/results/clientpositive/groupby_cube1.q.out 0486b689f62a98bf2994b2f1d964a36e85b707d2
> ql/src/test/results/clientpositive/groupby_grouping_id1.q.out 9ef76155be18f103063fea699d78029acbec1300
> ql/src/test/results/clientpositive/groupby_grouping_sets_grouping.q.out 6917dbabb96b5a582416ac7f07968b739823a13f
> ql/src/test/results/clientpositive/groupby_grouping_window.q.out 202fad093fee28bdbf347e9d89b61ada5dfaa72f
> ql/src/test/results/clientpositive/infer_bucket_sort_grouping_operators.q.out d740dea0d6c0b4b284afab3d6f13032407351a3f
> ql/src/test/results/clientpositive/llap/cte_1.q.out 1e6a4385d3f26a5e4ffa502688a95da977d0a4ae
> ql/src/test/results/clientpositive/llap/groupby_grouping_id2.q.out 9c85852863cf4bc858e1b30511764fa33ac57ece
> ql/src/test/results/clientpositive/llap/multi_count_distinct_null.q.out 0cbb9deba0ed6a33faa122b9d6d75fccdcbc6bfb
> ql/src/test/results/clientpositive/llap/vector_grouping_sets.q.out 21769170d948ebf576367fc30861101d0c4f5423
> ql/src/test/results/clientpositive/spark/groupby_cube1.q.out 56b8c2f382840bdc05afffceae1c8b7b1dd6b776
> ql/src/test/results/clientpositive/spark/groupby_grouping_id2.q.out 9c85852863cf4bc858e1b30511764fa33ac57ece
> ql/src/test/results/clientpositive/tez/multi_count_distinct.q.out ef3fcabb4c4184831ba06d9017a68d5fd2a7c687
> ql/src/test/results/clientpositive/vector_grouping_sets.q.out df4631e5f3434be99ebb063e2ff0c45383928f7f
>
>
> Diff: https://reviews.apache.org/r/57274/diff/3/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Jes�s Camacho Rodr�guez
>
>
Re: Review Request 57274: HIVE-16102
Posted by Jesús Camacho Rodríguez <jc...@hortonworks.com>.
> On March 7, 2017, 5:37 p.m., Ashutosh Chauhan wrote:
> > ql/src/test/results/clientpositive/llap/multi_count_distinct_null.q.out
> > Lines 292-312 (original), 292-312 (patched)
> > <https://reviews.apache.org/r/57274/diff/3/?file=1657987#file1657987line292>
> >
> > select grouping_id(department_id, gender,education_level), department_id, gender, education_level from employee
> > group by department_id, gender, education_level, grouping sets
> > (department_id, gender, education_level, (education_level, department_id));
> >
> >
> > Result Set 10
> > GROUPING_ID(DEPARTMENT_ID,GENDER,EDUCATION_LEVEL) DEPARTMENT_ID GENDER EDUCATION_LEVEL
> > 0 - - 1
> > 0 - - -
> > 0 4 M 1
> > 0 3 M 2
> > 0 2 F 3
> > 0 1 M 1
> > 0 1 M 2
> > 0 2 F 1
> > 0 1 F 3
> > 0 - M 1
> > 0 - - 1
> > 0 - - -
> > 0 4 M 1
> > 0 3 M 2
> > 0 2 F 3
> > 0 1 M 1
> > 0 1 M 2
> > 0 2 F 1
> > 0 1 F 3
> > 0 - M 1
> > 0 - - 1
> > 0 - - -
> > 0 4 M 1
> > 0 3 M 2
> > 0 2 F 3
> > 0 1 M 1
> > 0 1 M 2
> > 0 2 F 1
> > 0 1 F 3
> > 0 - M 1
> > 0 - - 1
> > 0 - - -
> > 0 4 M 1
> > 0 3 M 2
> > 0 2 F 3
> > 0 1 M 1
> > 0 1 M 2
> > 0 2 F 1
> > 0 1 F 3
> > 0 - M 1
> >
> > Number of rows are also different. Oracle has 40 while has hive 20.
> >
> > select * from employee;
> >
> >
> >
> > Result Set 11
> > DEPARTMENT_ID GENDER EDUCATION_LEVEL
> > 1 M 1
> > 2 F 1
> > 1 F 3
> > 1 M 2
> > 4 M 1
> > 2 F 1
> > 2 F 3
> > 3 M 2
> > - M 1
> > - - 1
> > - - -
> >
> > 11 rows selected.
> >
> > I will try to verify this on pg, since it looks like oracle is doing something wrong here.
If you execute it as:
select grouping_id(department_id, gender,education_level), department_id, gender, education_level from employee
group by grouping sets (department_id, gender, education_level, (education_level, department_id));
Does it give the same results?
- Jes�s
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57274/#review168156
-----------------------------------------------------------
On March 7, 2017, 9:55 a.m., Jes�s Camacho Rodr�guez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57274/
> -----------------------------------------------------------
>
> (Updated March 7, 2017, 9:55 a.m.)
>
>
> Review request for hive and Ashutosh Chauhan.
>
>
> Bugs: HIVE-16102
> https://issues.apache.org/jira/browse/HIVE-16102
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> HIVE-16102
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java 2ffc1306e727ccde15e507a2bd087d9e964e734e
> ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java be561cef8fdec2fc9fdc194e1c4771497582c5a9
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveExpandDistinctAggregatesRule.java 82624fc33f31f742b25f0a6887efabe7e2834947
> ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java db8d46e2dca44605a4068c27b4be9f13401f46b0
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 0872e535a9b6a09569c02fc498dab16867ca8783
> ql/src/test/queries/clientpositive/multi_count_distinct.q 855cb64c889e4c4309eaf9ee0745f528e611bd54
> ql/src/test/queries/clientpositive/multi_count_distinct_null.q 442d855d86129f0725f4b7c50df59e25c6c6b6cc
> ql/src/test/results/clientnegative/groupby_grouping_sets1.q.out 8290d4c4686455b9b689c8de876e765d5859a111
> ql/src/test/results/clientnegative/groupby_grouping_sets2.q.out 8290d4c4686455b9b689c8de876e765d5859a111
> ql/src/test/results/clientpositive/cte_1.q.out d13ad42a19d866b9a701d02d0bac0d62fb2a3dfa
> ql/src/test/results/clientpositive/groupby_cube1.q.out 0486b689f62a98bf2994b2f1d964a36e85b707d2
> ql/src/test/results/clientpositive/groupby_grouping_id1.q.out 9ef76155be18f103063fea699d78029acbec1300
> ql/src/test/results/clientpositive/groupby_grouping_sets_grouping.q.out 6917dbabb96b5a582416ac7f07968b739823a13f
> ql/src/test/results/clientpositive/groupby_grouping_window.q.out 202fad093fee28bdbf347e9d89b61ada5dfaa72f
> ql/src/test/results/clientpositive/infer_bucket_sort_grouping_operators.q.out d740dea0d6c0b4b284afab3d6f13032407351a3f
> ql/src/test/results/clientpositive/llap/cte_1.q.out 1e6a4385d3f26a5e4ffa502688a95da977d0a4ae
> ql/src/test/results/clientpositive/llap/groupby_grouping_id2.q.out 9c85852863cf4bc858e1b30511764fa33ac57ece
> ql/src/test/results/clientpositive/llap/multi_count_distinct_null.q.out 0cbb9deba0ed6a33faa122b9d6d75fccdcbc6bfb
> ql/src/test/results/clientpositive/llap/vector_grouping_sets.q.out 21769170d948ebf576367fc30861101d0c4f5423
> ql/src/test/results/clientpositive/spark/groupby_cube1.q.out 56b8c2f382840bdc05afffceae1c8b7b1dd6b776
> ql/src/test/results/clientpositive/spark/groupby_grouping_id2.q.out 9c85852863cf4bc858e1b30511764fa33ac57ece
> ql/src/test/results/clientpositive/tez/multi_count_distinct.q.out ef3fcabb4c4184831ba06d9017a68d5fd2a7c687
> ql/src/test/results/clientpositive/vector_grouping_sets.q.out df4631e5f3434be99ebb063e2ff0c45383928f7f
>
>
> Diff: https://reviews.apache.org/r/57274/diff/3/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Jes�s Camacho Rodr�guez
>
>
Re: Review Request 57274: HIVE-16102
Posted by Ashutosh Chauhan <ha...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57274/#review168156
-----------------------------------------------------------
ql/src/test/results/clientpositive/llap/multi_count_distinct_null.q.out
Lines 292-312 (original), 292-312 (patched)
<https://reviews.apache.org/r/57274/#comment240303>
select grouping_id(department_id, gender,education_level), department_id, gender, education_level from employee
group by department_id, gender, education_level, grouping sets
(department_id, gender, education_level, (education_level, department_id));
Result Set 10
GROUPING_ID(DEPARTMENT_ID,GENDER,EDUCATION_LEVEL) DEPARTMENT_ID GENDER EDUCATION_LEVEL
0 - - 1
0 - - -
0 4 M 1
0 3 M 2
0 2 F 3
0 1 M 1
0 1 M 2
0 2 F 1
0 1 F 3
0 - M 1
0 - - 1
0 - - -
0 4 M 1
0 3 M 2
0 2 F 3
0 1 M 1
0 1 M 2
0 2 F 1
0 1 F 3
0 - M 1
0 - - 1
0 - - -
0 4 M 1
0 3 M 2
0 2 F 3
0 1 M 1
0 1 M 2
0 2 F 1
0 1 F 3
0 - M 1
0 - - 1
0 - - -
0 4 M 1
0 3 M 2
0 2 F 3
0 1 M 1
0 1 M 2
0 2 F 1
0 1 F 3
0 - M 1
Number of rows are also different. Oracle has 40 while has hive 20.
select * from employee;
Result Set 11
DEPARTMENT_ID GENDER EDUCATION_LEVEL
1 M 1
2 F 1
1 F 3
1 M 2
4 M 1
2 F 1
2 F 3
3 M 2
- M 1
- - 1
- - -
11 rows selected.
I will try to verify this on pg, since it looks like oracle is doing something wrong here.
- Ashutosh Chauhan
On March 7, 2017, 9:55 a.m., Jes�s Camacho Rodr�guez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57274/
> -----------------------------------------------------------
>
> (Updated March 7, 2017, 9:55 a.m.)
>
>
> Review request for hive and Ashutosh Chauhan.
>
>
> Bugs: HIVE-16102
> https://issues.apache.org/jira/browse/HIVE-16102
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> HIVE-16102
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java 2ffc1306e727ccde15e507a2bd087d9e964e734e
> ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java be561cef8fdec2fc9fdc194e1c4771497582c5a9
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveExpandDistinctAggregatesRule.java 82624fc33f31f742b25f0a6887efabe7e2834947
> ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java db8d46e2dca44605a4068c27b4be9f13401f46b0
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 0872e535a9b6a09569c02fc498dab16867ca8783
> ql/src/test/queries/clientpositive/multi_count_distinct.q 855cb64c889e4c4309eaf9ee0745f528e611bd54
> ql/src/test/queries/clientpositive/multi_count_distinct_null.q 442d855d86129f0725f4b7c50df59e25c6c6b6cc
> ql/src/test/results/clientnegative/groupby_grouping_sets1.q.out 8290d4c4686455b9b689c8de876e765d5859a111
> ql/src/test/results/clientnegative/groupby_grouping_sets2.q.out 8290d4c4686455b9b689c8de876e765d5859a111
> ql/src/test/results/clientpositive/cte_1.q.out d13ad42a19d866b9a701d02d0bac0d62fb2a3dfa
> ql/src/test/results/clientpositive/groupby_cube1.q.out 0486b689f62a98bf2994b2f1d964a36e85b707d2
> ql/src/test/results/clientpositive/groupby_grouping_id1.q.out 9ef76155be18f103063fea699d78029acbec1300
> ql/src/test/results/clientpositive/groupby_grouping_sets_grouping.q.out 6917dbabb96b5a582416ac7f07968b739823a13f
> ql/src/test/results/clientpositive/groupby_grouping_window.q.out 202fad093fee28bdbf347e9d89b61ada5dfaa72f
> ql/src/test/results/clientpositive/infer_bucket_sort_grouping_operators.q.out d740dea0d6c0b4b284afab3d6f13032407351a3f
> ql/src/test/results/clientpositive/llap/cte_1.q.out 1e6a4385d3f26a5e4ffa502688a95da977d0a4ae
> ql/src/test/results/clientpositive/llap/groupby_grouping_id2.q.out 9c85852863cf4bc858e1b30511764fa33ac57ece
> ql/src/test/results/clientpositive/llap/multi_count_distinct_null.q.out 0cbb9deba0ed6a33faa122b9d6d75fccdcbc6bfb
> ql/src/test/results/clientpositive/llap/vector_grouping_sets.q.out 21769170d948ebf576367fc30861101d0c4f5423
> ql/src/test/results/clientpositive/spark/groupby_cube1.q.out 56b8c2f382840bdc05afffceae1c8b7b1dd6b776
> ql/src/test/results/clientpositive/spark/groupby_grouping_id2.q.out 9c85852863cf4bc858e1b30511764fa33ac57ece
> ql/src/test/results/clientpositive/tez/multi_count_distinct.q.out ef3fcabb4c4184831ba06d9017a68d5fd2a7c687
> ql/src/test/results/clientpositive/vector_grouping_sets.q.out df4631e5f3434be99ebb063e2ff0c45383928f7f
>
>
> Diff: https://reviews.apache.org/r/57274/diff/3/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Jes�s Camacho Rodr�guez
>
>
Re: Review Request 57274: HIVE-16102
Posted by Jesús Camacho Rodríguez <jc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57274/
-----------------------------------------------------------
(Updated March 7, 2017, 9:55 a.m.)
Review request for hive and Ashutosh Chauhan.
Bugs: HIVE-16102
https://issues.apache.org/jira/browse/HIVE-16102
Repository: hive-git
Description
-------
HIVE-16102
Diffs (updated)
-----
ql/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java 2ffc1306e727ccde15e507a2bd087d9e964e734e
ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java be561cef8fdec2fc9fdc194e1c4771497582c5a9
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveExpandDistinctAggregatesRule.java 82624fc33f31f742b25f0a6887efabe7e2834947
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java db8d46e2dca44605a4068c27b4be9f13401f46b0
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 0872e535a9b6a09569c02fc498dab16867ca8783
ql/src/test/queries/clientpositive/multi_count_distinct.q 855cb64c889e4c4309eaf9ee0745f528e611bd54
ql/src/test/queries/clientpositive/multi_count_distinct_null.q 442d855d86129f0725f4b7c50df59e25c6c6b6cc
ql/src/test/results/clientnegative/groupby_grouping_sets1.q.out 8290d4c4686455b9b689c8de876e765d5859a111
ql/src/test/results/clientnegative/groupby_grouping_sets2.q.out 8290d4c4686455b9b689c8de876e765d5859a111
ql/src/test/results/clientpositive/cte_1.q.out d13ad42a19d866b9a701d02d0bac0d62fb2a3dfa
ql/src/test/results/clientpositive/groupby_cube1.q.out 0486b689f62a98bf2994b2f1d964a36e85b707d2
ql/src/test/results/clientpositive/groupby_grouping_id1.q.out 9ef76155be18f103063fea699d78029acbec1300
ql/src/test/results/clientpositive/groupby_grouping_sets_grouping.q.out 6917dbabb96b5a582416ac7f07968b739823a13f
ql/src/test/results/clientpositive/groupby_grouping_window.q.out 202fad093fee28bdbf347e9d89b61ada5dfaa72f
ql/src/test/results/clientpositive/infer_bucket_sort_grouping_operators.q.out d740dea0d6c0b4b284afab3d6f13032407351a3f
ql/src/test/results/clientpositive/llap/cte_1.q.out 1e6a4385d3f26a5e4ffa502688a95da977d0a4ae
ql/src/test/results/clientpositive/llap/groupby_grouping_id2.q.out 9c85852863cf4bc858e1b30511764fa33ac57ece
ql/src/test/results/clientpositive/llap/multi_count_distinct_null.q.out 0cbb9deba0ed6a33faa122b9d6d75fccdcbc6bfb
ql/src/test/results/clientpositive/llap/vector_grouping_sets.q.out 21769170d948ebf576367fc30861101d0c4f5423
ql/src/test/results/clientpositive/spark/groupby_cube1.q.out 56b8c2f382840bdc05afffceae1c8b7b1dd6b776
ql/src/test/results/clientpositive/spark/groupby_grouping_id2.q.out 9c85852863cf4bc858e1b30511764fa33ac57ece
ql/src/test/results/clientpositive/tez/multi_count_distinct.q.out ef3fcabb4c4184831ba06d9017a68d5fd2a7c687
ql/src/test/results/clientpositive/vector_grouping_sets.q.out df4631e5f3434be99ebb063e2ff0c45383928f7f
Diff: https://reviews.apache.org/r/57274/diff/3/
Changes: https://reviews.apache.org/r/57274/diff/2-3/
Testing
-------
Thanks,
Jes�s Camacho Rodr�guez
Re: Review Request 57274: HIVE-16102
Posted by Jesús Camacho Rodríguez <jc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57274/
-----------------------------------------------------------
(Updated March 6, 2017, 1:33 p.m.)
Review request for hive and Ashutosh Chauhan.
Bugs: HIVE-16102
https://issues.apache.org/jira/browse/HIVE-16102
Repository: hive-git
Description
-------
HIVE-16102
Diffs (updated)
-----
ql/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java 2ffc1306e727ccde15e507a2bd087d9e964e734e
ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java be561cef8fdec2fc9fdc194e1c4771497582c5a9
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java db8d46e2dca44605a4068c27b4be9f13401f46b0
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 0872e535a9b6a09569c02fc498dab16867ca8783
ql/src/test/results/clientnegative/groupby_grouping_sets1.q.out 8290d4c4686455b9b689c8de876e765d5859a111
ql/src/test/results/clientnegative/groupby_grouping_sets2.q.out 8290d4c4686455b9b689c8de876e765d5859a111
ql/src/test/results/clientpositive/groupby_cube1.q.out 0486b689f62a98bf2994b2f1d964a36e85b707d2
ql/src/test/results/clientpositive/groupby_grouping_id1.q.out 9ef76155be18f103063fea699d78029acbec1300
ql/src/test/results/clientpositive/groupby_grouping_sets_grouping.q.out 6917dbabb96b5a582416ac7f07968b739823a13f
ql/src/test/results/clientpositive/groupby_grouping_window.q.out 202fad093fee28bdbf347e9d89b61ada5dfaa72f
Diff: https://reviews.apache.org/r/57274/diff/2/
Changes: https://reviews.apache.org/r/57274/diff/1-2/
Testing
-------
Thanks,
Jes�s Camacho Rodr�guez