You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Ashutosh Chauhan <ha...@apache.org> on 2015/02/04 03:02:35 UTC
Review Request 30602: HIVE-6099
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30602/
-----------------------------------------------------------
Review request for hive and Navis Ryu.
Bugs: HIVE-6099
https://issues.apache.org/jira/browse/HIVE-6099
Repository: hive-git
Description
-------
remove common.distincts optimization
Diffs
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 6d2748e
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 7b645b8
ql/src/test/queries/clientpositive/groupby10.q 8b1fb52
ql/src/test/queries/clientpositive/groupby_multi_insert_common_distinct.q b009a8b
ql/src/test/results/clientpositive/groupby10.q.out a575089
ql/src/test/results/clientpositive/groupby11.q.out 322140e
ql/src/test/results/clientpositive/groupby8.q.out 68d6e47
ql/src/test/results/clientpositive/groupby8_map.q.out 37301f7
ql/src/test/results/clientpositive/groupby8_map_skew.q.out 37301f7
ql/src/test/results/clientpositive/groupby8_noskew.q.out 37301f7
ql/src/test/results/clientpositive/groupby9.q.out e96a22f
ql/src/test/results/clientpositive/groupby_multi_insert_common_distinct.q.out c6b5edc
ql/src/test/results/clientpositive/spark/groupby10.q.out 2bae6ae
ql/src/test/results/clientpositive/spark/groupby11.q.out 378c166
ql/src/test/results/clientpositive/spark/groupby8.q.out 53ecd96
ql/src/test/results/clientpositive/spark/groupby8_map.q.out 0517b72
ql/src/test/results/clientpositive/spark/groupby8_map_skew.q.out 0517b72
ql/src/test/results/clientpositive/spark/groupby8_noskew.q.out 0517b72
ql/src/test/results/clientpositive/spark/groupby9.q.out 9b5095a
ql/src/test/results/clientpositive/spark/groupby_multi_insert_common_distinct.q.out c2766f8
ql/src/test/results/clientpositive/union17.q.out 75d2253
Diff: https://reviews.apache.org/r/30602/diff/
Testing
-------
Existing tests.
Thanks,
Ashutosh Chauhan
Re: Review Request 30602: HIVE-6099
Posted by Navis Ryu <na...@nexr.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30602/#review70902
-----------------------------------------------------------
We can simplify code further
genGroupByPlanGroupByOperator1.distPartAgg=false ->
GroupByDesc.groupKeyIsNotReduceKey=false ->
GroupByOperator.groupKeyIsNotReduceKey/keysCurrentGroup/firstRowInGroup/startGroup/endGroup is not needed
- Navis Ryu
On Feb. 4, 2015, 2:02 a.m., Ashutosh Chauhan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30602/
> -----------------------------------------------------------
>
> (Updated Feb. 4, 2015, 2:02 a.m.)
>
>
> Review request for hive and Navis Ryu.
>
>
> Bugs: HIVE-6099
> https://issues.apache.org/jira/browse/HIVE-6099
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> remove common.distincts optimization
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 6d2748e
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 7b645b8
> ql/src/test/queries/clientpositive/groupby10.q 8b1fb52
> ql/src/test/queries/clientpositive/groupby_multi_insert_common_distinct.q b009a8b
> ql/src/test/results/clientpositive/groupby10.q.out a575089
> ql/src/test/results/clientpositive/groupby11.q.out 322140e
> ql/src/test/results/clientpositive/groupby8.q.out 68d6e47
> ql/src/test/results/clientpositive/groupby8_map.q.out 37301f7
> ql/src/test/results/clientpositive/groupby8_map_skew.q.out 37301f7
> ql/src/test/results/clientpositive/groupby8_noskew.q.out 37301f7
> ql/src/test/results/clientpositive/groupby9.q.out e96a22f
> ql/src/test/results/clientpositive/groupby_multi_insert_common_distinct.q.out c6b5edc
> ql/src/test/results/clientpositive/spark/groupby10.q.out 2bae6ae
> ql/src/test/results/clientpositive/spark/groupby11.q.out 378c166
> ql/src/test/results/clientpositive/spark/groupby8.q.out 53ecd96
> ql/src/test/results/clientpositive/spark/groupby8_map.q.out 0517b72
> ql/src/test/results/clientpositive/spark/groupby8_map_skew.q.out 0517b72
> ql/src/test/results/clientpositive/spark/groupby8_noskew.q.out 0517b72
> ql/src/test/results/clientpositive/spark/groupby9.q.out 9b5095a
> ql/src/test/results/clientpositive/spark/groupby_multi_insert_common_distinct.q.out c2766f8
> ql/src/test/results/clientpositive/union17.q.out 75d2253
>
> Diff: https://reviews.apache.org/r/30602/diff/
>
>
> Testing
> -------
>
> Existing tests.
>
>
> Thanks,
>
> Ashutosh Chauhan
>
>
Re: Review Request 30602: HIVE-6099
Posted by Alexander Pivovarov <ap...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30602/#review71248
-----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/plan/GroupByDesc.java
<https://reviews.apache.org/r/30602/#comment116916>
trailing spaces
- Alexander Pivovarov
On Feb. 5, 2015, 6:06 a.m., Ashutosh Chauhan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30602/
> -----------------------------------------------------------
>
> (Updated Feb. 5, 2015, 6:06 a.m.)
>
>
> Review request for hive and Navis Ryu.
>
>
> Bugs: HIVE-6099
> https://issues.apache.org/jira/browse/HIVE-6099
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> remove common.distincts optimization
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 31764d9
> ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java 2e40556
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java 428d2ab
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 2d5e6cf
> ql/src/java/org/apache/hadoop/hive/ql/plan/GroupByDesc.java 8804258
> ql/src/test/queries/clientpositive/groupby10.q 8b1fb52
> ql/src/test/queries/clientpositive/groupby_multi_insert_common_distinct.q b009a8b
> ql/src/test/results/clientpositive/groupby10.q.out a575089
> ql/src/test/results/clientpositive/groupby11.q.out 322140e
> ql/src/test/results/clientpositive/groupby8.q.out 68d6e47
> ql/src/test/results/clientpositive/groupby8_map.q.out 37301f7
> ql/src/test/results/clientpositive/groupby8_map_skew.q.out 37301f7
> ql/src/test/results/clientpositive/groupby8_noskew.q.out 37301f7
> ql/src/test/results/clientpositive/groupby9.q.out e96a22f
> ql/src/test/results/clientpositive/groupby_multi_insert_common_distinct.q.out c6b5edc
> ql/src/test/results/clientpositive/spark/groupby10.q.out 2bae6ae
> ql/src/test/results/clientpositive/spark/groupby11.q.out 378c166
> ql/src/test/results/clientpositive/spark/groupby8.q.out 53ecd96
> ql/src/test/results/clientpositive/spark/groupby8_map.q.out 0517b72
> ql/src/test/results/clientpositive/spark/groupby8_map_skew.q.out 0517b72
> ql/src/test/results/clientpositive/spark/groupby8_noskew.q.out 0517b72
> ql/src/test/results/clientpositive/spark/groupby9.q.out 9b5095a
> ql/src/test/results/clientpositive/spark/groupby_multi_insert_common_distinct.q.out c2766f8
> ql/src/test/results/clientpositive/union17.q.out 75d2253
>
> Diff: https://reviews.apache.org/r/30602/diff/
>
>
> Testing
> -------
>
> Existing tests.
>
>
> Thanks,
>
> Ashutosh Chauhan
>
>
Re: Review Request 30602: HIVE-6099
Posted by Ashutosh Chauhan <ha...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30602/
-----------------------------------------------------------
(Updated Feb. 5, 2015, 6:06 a.m.)
Review request for hive and Navis Ryu.
Changes
-------
Incorporated RB comments.
Bugs: HIVE-6099
https://issues.apache.org/jira/browse/HIVE-6099
Repository: hive-git
Description
-------
remove common.distincts optimization
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 31764d9
ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java 2e40556
ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java 428d2ab
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 2d5e6cf
ql/src/java/org/apache/hadoop/hive/ql/plan/GroupByDesc.java 8804258
ql/src/test/queries/clientpositive/groupby10.q 8b1fb52
ql/src/test/queries/clientpositive/groupby_multi_insert_common_distinct.q b009a8b
ql/src/test/results/clientpositive/groupby10.q.out a575089
ql/src/test/results/clientpositive/groupby11.q.out 322140e
ql/src/test/results/clientpositive/groupby8.q.out 68d6e47
ql/src/test/results/clientpositive/groupby8_map.q.out 37301f7
ql/src/test/results/clientpositive/groupby8_map_skew.q.out 37301f7
ql/src/test/results/clientpositive/groupby8_noskew.q.out 37301f7
ql/src/test/results/clientpositive/groupby9.q.out e96a22f
ql/src/test/results/clientpositive/groupby_multi_insert_common_distinct.q.out c6b5edc
ql/src/test/results/clientpositive/spark/groupby10.q.out 2bae6ae
ql/src/test/results/clientpositive/spark/groupby11.q.out 378c166
ql/src/test/results/clientpositive/spark/groupby8.q.out 53ecd96
ql/src/test/results/clientpositive/spark/groupby8_map.q.out 0517b72
ql/src/test/results/clientpositive/spark/groupby8_map_skew.q.out 0517b72
ql/src/test/results/clientpositive/spark/groupby8_noskew.q.out 0517b72
ql/src/test/results/clientpositive/spark/groupby9.q.out 9b5095a
ql/src/test/results/clientpositive/spark/groupby_multi_insert_common_distinct.q.out c2766f8
ql/src/test/results/clientpositive/union17.q.out 75d2253
Diff: https://reviews.apache.org/r/30602/diff/
Testing
-------
Existing tests.
Thanks,
Ashutosh Chauhan
Re: Review Request 30602: HIVE-6099
Posted by Navis Ryu <na...@nexr.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30602/#review71156
-----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java
<https://reviews.apache.org/r/30602/#comment116786>
if groupKeyIsNotReduceKey is always false, firstRowInGroup will not be evaluated. It would be safe to removed it.
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
<https://reviews.apache.org/r/30602/#comment116787>
not-not-isDistinct.. how about replace it with isDistinct
- Navis Ryu
On Feb. 4, 2015, 9:16 p.m., Ashutosh Chauhan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30602/
> -----------------------------------------------------------
>
> (Updated Feb. 4, 2015, 9:16 p.m.)
>
>
> Review request for hive and Navis Ryu.
>
>
> Bugs: HIVE-6099
> https://issues.apache.org/jira/browse/HIVE-6099
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> remove common.distincts optimization
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 6d2748e
> ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java 2e40556
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java 96901c9
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 2d5e6cf
> ql/src/java/org/apache/hadoop/hive/ql/plan/GroupByDesc.java 8804258
> ql/src/test/queries/clientpositive/groupby10.q 8b1fb52
> ql/src/test/queries/clientpositive/groupby_multi_insert_common_distinct.q b009a8b
> ql/src/test/results/clientpositive/groupby10.q.out a575089
> ql/src/test/results/clientpositive/groupby11.q.out 322140e
> ql/src/test/results/clientpositive/groupby8.q.out 68d6e47
> ql/src/test/results/clientpositive/groupby8_map.q.out 37301f7
> ql/src/test/results/clientpositive/groupby8_map_skew.q.out 37301f7
> ql/src/test/results/clientpositive/groupby8_noskew.q.out 37301f7
> ql/src/test/results/clientpositive/groupby9.q.out e96a22f
> ql/src/test/results/clientpositive/groupby_multi_insert_common_distinct.q.out c6b5edc
> ql/src/test/results/clientpositive/spark/groupby10.q.out 2bae6ae
> ql/src/test/results/clientpositive/spark/groupby11.q.out 378c166
> ql/src/test/results/clientpositive/spark/groupby8.q.out 53ecd96
> ql/src/test/results/clientpositive/spark/groupby8_map.q.out 0517b72
> ql/src/test/results/clientpositive/spark/groupby8_map_skew.q.out 0517b72
> ql/src/test/results/clientpositive/spark/groupby8_noskew.q.out 0517b72
> ql/src/test/results/clientpositive/spark/groupby9.q.out 9b5095a
> ql/src/test/results/clientpositive/spark/groupby_multi_insert_common_distinct.q.out c2766f8
> ql/src/test/results/clientpositive/union17.q.out 75d2253
>
> Diff: https://reviews.apache.org/r/30602/diff/
>
>
> Testing
> -------
>
> Existing tests.
>
>
> Thanks,
>
> Ashutosh Chauhan
>
>
Re: Review Request 30602: HIVE-6099
Posted by Ashutosh Chauhan <ha...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30602/
-----------------------------------------------------------
(Updated Feb. 4, 2015, 9:16 p.m.)
Review request for hive and Navis Ryu.
Changes
-------
rebased
Bugs: HIVE-6099
https://issues.apache.org/jira/browse/HIVE-6099
Repository: hive-git
Description
-------
remove common.distincts optimization
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 6d2748e
ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java 2e40556
ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java 96901c9
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 2d5e6cf
ql/src/java/org/apache/hadoop/hive/ql/plan/GroupByDesc.java 8804258
ql/src/test/queries/clientpositive/groupby10.q 8b1fb52
ql/src/test/queries/clientpositive/groupby_multi_insert_common_distinct.q b009a8b
ql/src/test/results/clientpositive/groupby10.q.out a575089
ql/src/test/results/clientpositive/groupby11.q.out 322140e
ql/src/test/results/clientpositive/groupby8.q.out 68d6e47
ql/src/test/results/clientpositive/groupby8_map.q.out 37301f7
ql/src/test/results/clientpositive/groupby8_map_skew.q.out 37301f7
ql/src/test/results/clientpositive/groupby8_noskew.q.out 37301f7
ql/src/test/results/clientpositive/groupby9.q.out e96a22f
ql/src/test/results/clientpositive/groupby_multi_insert_common_distinct.q.out c6b5edc
ql/src/test/results/clientpositive/spark/groupby10.q.out 2bae6ae
ql/src/test/results/clientpositive/spark/groupby11.q.out 378c166
ql/src/test/results/clientpositive/spark/groupby8.q.out 53ecd96
ql/src/test/results/clientpositive/spark/groupby8_map.q.out 0517b72
ql/src/test/results/clientpositive/spark/groupby8_map_skew.q.out 0517b72
ql/src/test/results/clientpositive/spark/groupby8_noskew.q.out 0517b72
ql/src/test/results/clientpositive/spark/groupby9.q.out 9b5095a
ql/src/test/results/clientpositive/spark/groupby_multi_insert_common_distinct.q.out c2766f8
ql/src/test/results/clientpositive/union17.q.out 75d2253
Diff: https://reviews.apache.org/r/30602/diff/
Testing
-------
Existing tests.
Thanks,
Ashutosh Chauhan
Re: Review Request 30602: HIVE-6099
Posted by Ashutosh Chauhan <ha...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30602/
-----------------------------------------------------------
(Updated Feb. 4, 2015, 2:47 a.m.)
Review request for hive and Navis Ryu.
Changes
-------
Incorporated Navis suggestions. I am not sure firstRowInGroup/startGroup can be removed yet.
Bugs: HIVE-6099
https://issues.apache.org/jira/browse/HIVE-6099
Repository: hive-git
Description
-------
remove common.distincts optimization
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 6d2748e
ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java 2e40556
ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java 96901c9
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 7b645b8
ql/src/java/org/apache/hadoop/hive/ql/plan/GroupByDesc.java f031b28
ql/src/test/queries/clientpositive/groupby10.q 8b1fb52
ql/src/test/queries/clientpositive/groupby_multi_insert_common_distinct.q b009a8b
ql/src/test/results/clientpositive/groupby10.q.out a575089
ql/src/test/results/clientpositive/groupby11.q.out 322140e
ql/src/test/results/clientpositive/groupby8.q.out 68d6e47
ql/src/test/results/clientpositive/groupby8_map.q.out 37301f7
ql/src/test/results/clientpositive/groupby8_map_skew.q.out 37301f7
ql/src/test/results/clientpositive/groupby8_noskew.q.out 37301f7
ql/src/test/results/clientpositive/groupby9.q.out e96a22f
ql/src/test/results/clientpositive/groupby_multi_insert_common_distinct.q.out c6b5edc
ql/src/test/results/clientpositive/spark/groupby10.q.out 2bae6ae
ql/src/test/results/clientpositive/spark/groupby11.q.out 378c166
ql/src/test/results/clientpositive/spark/groupby8.q.out 53ecd96
ql/src/test/results/clientpositive/spark/groupby8_map.q.out 0517b72
ql/src/test/results/clientpositive/spark/groupby8_map_skew.q.out 0517b72
ql/src/test/results/clientpositive/spark/groupby8_noskew.q.out 0517b72
ql/src/test/results/clientpositive/spark/groupby9.q.out 9b5095a
ql/src/test/results/clientpositive/spark/groupby_multi_insert_common_distinct.q.out c2766f8
ql/src/test/results/clientpositive/union17.q.out 75d2253
Diff: https://reviews.apache.org/r/30602/diff/
Testing
-------
Existing tests.
Thanks,
Ashutosh Chauhan