You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Ted Xu <fr...@gmail.com> on 2014/12/07 10:30:32 UTC
Review Request 28791: HIVE-9025 join38.q (without map join) produces
incorrect result when testing with multiple reducers
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28791/
-----------------------------------------------------------
Review request for hive, Ashutosh Chauhan and Chao Sun.
Bugs: HIVE-9025
https://issues.apache.org/jira/browse/HIVE-9025
Repository: hive
Description
-------
HIVE-5771 introduced a bug that when all partition columns are constants, the partition is transformed to be a random dispatch, which is not expected.
This patch adds a constant column in the above case to avoid random partitioning.
Diffs
-----
http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConstantPropagateProcFactory.java 1643530
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/cluster.q.out 1643530
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/constprog2.q.out 1643530
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join_nullsafe.q.out 1643530
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd2.q.out 1643530
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_clusterby.q.out 1643530
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join4.q.out 1643530
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join5.q.out 1643530
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/quotedid_basic.q.out 1643530
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_25.q.out 1643530
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/union27.q.out 1643530
Diff: https://reviews.apache.org/r/28791/diff/
Testing
-------
TestCliDriver passed.
Thanks,
Ted Xu
Re: Review Request 28791: HIVE-9025 join38.q (without map join)
produces incorrect result when testing with multiple reducers
Posted by Ashutosh Chauhan <ha...@apache.org>.
> On Dec. 8, 2014, 5:27 p.m., Chao Sun wrote:
> > Hi, [~tedxu], thanks for the quick work! I just have one minor question: do you think it would be good to have a new test case for this? Maybe someone that just like join38.q, but uses common join and set number of reducers to be a number greater than one?
>
> Ashutosh Chauhan wrote:
> yeah.. I think its a good idea to add testcase for this which reproduces the bug. [~tedxu] For having more than 1 reducer in tests, you need to run it through TestMinimrCliDriver. To achieve that, add name of your new test in itests/src/test/resources/testconfiguration.properties
>
> Ted Xu wrote:
> Will do, thank you guys for the advice!
>
> PS: I still got a -1 from Hadoop QA because of some UT failures, however, I can't reproduce any of them on my laptop. Is there anything I'm doing wrong?
Make sure you are using TestMiniTezCliDriver for those reported failures, not TestCliDriver.
If you still cant reproduce them, than don't bother about those. They are just .q.out golden files update. I can take care of those while comitting the patch.
- Ashutosh
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28791/#review64242
-----------------------------------------------------------
On Dec. 7, 2014, 9:30 a.m., Ted Xu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28791/
> -----------------------------------------------------------
>
> (Updated Dec. 7, 2014, 9:30 a.m.)
>
>
> Review request for hive, Ashutosh Chauhan and Chao Sun.
>
>
> Bugs: HIVE-9025
> https://issues.apache.org/jira/browse/HIVE-9025
>
>
> Repository: hive
>
>
> Description
> -------
>
> HIVE-5771 introduced a bug that when all partition columns are constants, the partition is transformed to be a random dispatch, which is not expected.
>
> This patch adds a constant column in the above case to avoid random partitioning.
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConstantPropagateProcFactory.java 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/cluster.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/constprog2.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join_nullsafe.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd2.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_clusterby.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join4.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join5.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/quotedid_basic.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_25.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/union27.q.out 1643530
>
> Diff: https://reviews.apache.org/r/28791/diff/
>
>
> Testing
> -------
>
> TestCliDriver passed.
>
>
> Thanks,
>
> Ted Xu
>
>
Re: Review Request 28791: HIVE-9025 join38.q (without map join)
produces incorrect result when testing with multiple reducers
Posted by Ashutosh Chauhan <ha...@apache.org>.
> On Dec. 8, 2014, 5:27 p.m., Chao Sun wrote:
> > Hi, [~tedxu], thanks for the quick work! I just have one minor question: do you think it would be good to have a new test case for this? Maybe someone that just like join38.q, but uses common join and set number of reducers to be a number greater than one?
yeah.. I think its a good idea to add testcase for this which reproduces the bug. [~tedxu] For having more than 1 reducer in tests, you need to run it through TestMinimrCliDriver. To achieve that, add name of your new test in itests/src/test/resources/testconfiguration.properties
- Ashutosh
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28791/#review64242
-----------------------------------------------------------
On Dec. 7, 2014, 9:30 a.m., Ted Xu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28791/
> -----------------------------------------------------------
>
> (Updated Dec. 7, 2014, 9:30 a.m.)
>
>
> Review request for hive, Ashutosh Chauhan and Chao Sun.
>
>
> Bugs: HIVE-9025
> https://issues.apache.org/jira/browse/HIVE-9025
>
>
> Repository: hive
>
>
> Description
> -------
>
> HIVE-5771 introduced a bug that when all partition columns are constants, the partition is transformed to be a random dispatch, which is not expected.
>
> This patch adds a constant column in the above case to avoid random partitioning.
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConstantPropagateProcFactory.java 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/cluster.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/constprog2.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join_nullsafe.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd2.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_clusterby.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join4.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join5.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/quotedid_basic.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_25.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/union27.q.out 1643530
>
> Diff: https://reviews.apache.org/r/28791/diff/
>
>
> Testing
> -------
>
> TestCliDriver passed.
>
>
> Thanks,
>
> Ted Xu
>
>
Re: Review Request 28791: HIVE-9025 join38.q (without map join)
produces incorrect result when testing with multiple reducers
Posted by Ted Xu <fr...@gmail.com>.
> On 十二月 8, 2014, 5:27 p.m., Chao Sun wrote:
> > Hi, [~tedxu], thanks for the quick work! I just have one minor question: do you think it would be good to have a new test case for this? Maybe someone that just like join38.q, but uses common join and set number of reducers to be a number greater than one?
>
> Ashutosh Chauhan wrote:
> yeah.. I think its a good idea to add testcase for this which reproduces the bug. [~tedxu] For having more than 1 reducer in tests, you need to run it through TestMinimrCliDriver. To achieve that, add name of your new test in itests/src/test/resources/testconfiguration.properties
Will do, thank you guys for the advice!
PS: I still got a -1 from Hadoop QA because of some UT failures, however, I can't reproduce any of them on my laptop. Is there anything I'm doing wrong?
- Ted
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28791/#review64242
-----------------------------------------------------------
On 十二月 7, 2014, 9:30 a.m., Ted Xu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28791/
> -----------------------------------------------------------
>
> (Updated 十二月 7, 2014, 9:30 a.m.)
>
>
> Review request for hive, Ashutosh Chauhan and Chao Sun.
>
>
> Bugs: HIVE-9025
> https://issues.apache.org/jira/browse/HIVE-9025
>
>
> Repository: hive
>
>
> Description
> -------
>
> HIVE-5771 introduced a bug that when all partition columns are constants, the partition is transformed to be a random dispatch, which is not expected.
>
> This patch adds a constant column in the above case to avoid random partitioning.
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConstantPropagateProcFactory.java 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/cluster.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/constprog2.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join_nullsafe.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd2.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_clusterby.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join4.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join5.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/quotedid_basic.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_25.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/union27.q.out 1643530
>
> Diff: https://reviews.apache.org/r/28791/diff/
>
>
> Testing
> -------
>
> TestCliDriver passed.
>
>
> Thanks,
>
> Ted Xu
>
>
Re: Review Request 28791: HIVE-9025 join38.q (without map join)
produces incorrect result when testing with multiple reducers
Posted by Chao Sun <ch...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28791/#review64242
-----------------------------------------------------------
Hi, [~tedxu], thanks for the quick work! I just have one minor question: do you think it would be good to have a new test case for this? Maybe someone that just like join38.q, but uses common join and set number of reducers to be a number greater than one?
- Chao Sun
On Dec. 7, 2014, 9:30 a.m., Ted Xu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28791/
> -----------------------------------------------------------
>
> (Updated Dec. 7, 2014, 9:30 a.m.)
>
>
> Review request for hive, Ashutosh Chauhan and Chao Sun.
>
>
> Bugs: HIVE-9025
> https://issues.apache.org/jira/browse/HIVE-9025
>
>
> Repository: hive
>
>
> Description
> -------
>
> HIVE-5771 introduced a bug that when all partition columns are constants, the partition is transformed to be a random dispatch, which is not expected.
>
> This patch adds a constant column in the above case to avoid random partitioning.
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConstantPropagateProcFactory.java 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/cluster.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/constprog2.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join_nullsafe.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd2.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_clusterby.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join4.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join5.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/quotedid_basic.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_25.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/union27.q.out 1643530
>
> Diff: https://reviews.apache.org/r/28791/diff/
>
>
> Testing
> -------
>
> TestCliDriver passed.
>
>
> Thanks,
>
> Ted Xu
>
>
Re: Review Request 28791: HIVE-9025 join38.q (without map join)
produces incorrect result when testing with multiple reducers
Posted by Chao Sun <ch...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28791/#review64244
-----------------------------------------------------------
Also, some golden files for tez branch need to be updated.
- Chao Sun
On Dec. 7, 2014, 9:30 a.m., Ted Xu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28791/
> -----------------------------------------------------------
>
> (Updated Dec. 7, 2014, 9:30 a.m.)
>
>
> Review request for hive, Ashutosh Chauhan and Chao Sun.
>
>
> Bugs: HIVE-9025
> https://issues.apache.org/jira/browse/HIVE-9025
>
>
> Repository: hive
>
>
> Description
> -------
>
> HIVE-5771 introduced a bug that when all partition columns are constants, the partition is transformed to be a random dispatch, which is not expected.
>
> This patch adds a constant column in the above case to avoid random partitioning.
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConstantPropagateProcFactory.java 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/cluster.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/constprog2.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join_nullsafe.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd2.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_clusterby.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join4.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join5.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/quotedid_basic.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_25.q.out 1643530
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/union27.q.out 1643530
>
> Diff: https://reviews.apache.org/r/28791/diff/
>
>
> Testing
> -------
>
> TestCliDriver passed.
>
>
> Thanks,
>
> Ted Xu
>
>
Re: Review Request 28791: HIVE-9025 join38.q (without map join)
produces incorrect result when testing with multiple reducers
Posted by Ashutosh Chauhan <ha...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28791/#review64632
-----------------------------------------------------------
Ship it!
ya.. those two tests have failed in other runs also. You can ignore those for now.
- Ashutosh Chauhan
On Dec. 10, 2014, 6:09 p.m., Ted Xu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28791/
> -----------------------------------------------------------
>
> (Updated Dec. 10, 2014, 6:09 p.m.)
>
>
> Review request for hive, Ashutosh Chauhan and Chao Sun.
>
>
> Bugs: HIVE-9025
> https://issues.apache.org/jira/browse/HIVE-9025
>
>
> Repository: hive
>
>
> Description
> -------
>
> HIVE-5771 introduced a bug that when all partition columns are constants, the partition is transformed to be a random dispatch, which is not expected.
>
> This patch adds a constant column in the above case to avoid random partitioning.
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/hive/trunk/itests/src/test/resources/testconfiguration.properties 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConstantPropagateProcFactory.java 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/constprog_partitioner.q PRE-CREATION
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/cluster.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/constprog2.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/constprog_partitioner.q.out PRE-CREATION
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join_nullsafe.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd2.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_clusterby.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join4.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join5.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/quotedid_basic.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_25.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/tez/dynamic_partition_pruning.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/tez/dynamic_partition_pruning_2.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/tez/join_nullsafe.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/tez/vector_decimal_mapjoin.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/tez/vectorized_dynamic_partition_pruning.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/union27.q.out 1644497
>
> Diff: https://reviews.apache.org/r/28791/diff/
>
>
> Testing
> -------
>
> TestCliDriver passed.
>
>
> Thanks,
>
> Ted Xu
>
>
Re: Review Request 28791: HIVE-9025 join38.q (without map join)
produces incorrect result when testing with multiple reducers
Posted by Chao Sun <ch...@cloudera.com>.
> On Dec. 10, 2014, 6:16 p.m., Chao Sun wrote:
> >
I don't think optimize_nullscan and vector_decimal_aggregate are related. Ashutosh can correct me if I'm wrong.
- Chao
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28791/#review64582
-----------------------------------------------------------
On Dec. 10, 2014, 6:09 p.m., Ted Xu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28791/
> -----------------------------------------------------------
>
> (Updated Dec. 10, 2014, 6:09 p.m.)
>
>
> Review request for hive, Ashutosh Chauhan and Chao Sun.
>
>
> Bugs: HIVE-9025
> https://issues.apache.org/jira/browse/HIVE-9025
>
>
> Repository: hive
>
>
> Description
> -------
>
> HIVE-5771 introduced a bug that when all partition columns are constants, the partition is transformed to be a random dispatch, which is not expected.
>
> This patch adds a constant column in the above case to avoid random partitioning.
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/hive/trunk/itests/src/test/resources/testconfiguration.properties 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConstantPropagateProcFactory.java 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/constprog_partitioner.q PRE-CREATION
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/cluster.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/constprog2.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/constprog_partitioner.q.out PRE-CREATION
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join_nullsafe.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd2.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_clusterby.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join4.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join5.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/quotedid_basic.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_25.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/tez/dynamic_partition_pruning.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/tez/dynamic_partition_pruning_2.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/tez/join_nullsafe.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/tez/vector_decimal_mapjoin.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/tez/vectorized_dynamic_partition_pruning.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/union27.q.out 1644497
>
> Diff: https://reviews.apache.org/r/28791/diff/
>
>
> Testing
> -------
>
> TestCliDriver passed.
>
>
> Thanks,
>
> Ted Xu
>
>
Re: Review Request 28791: HIVE-9025 join38.q (without map join)
produces incorrect result when testing with multiple reducers
Posted by Chao Sun <ch...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28791/#review64582
-----------------------------------------------------------
http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConstantPropagateProcFactory.java
<https://reviews.apache.org/r/28791/#comment107348>
trailing whitespace.
- Chao Sun
On Dec. 10, 2014, 6:09 p.m., Ted Xu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28791/
> -----------------------------------------------------------
>
> (Updated Dec. 10, 2014, 6:09 p.m.)
>
>
> Review request for hive, Ashutosh Chauhan and Chao Sun.
>
>
> Bugs: HIVE-9025
> https://issues.apache.org/jira/browse/HIVE-9025
>
>
> Repository: hive
>
>
> Description
> -------
>
> HIVE-5771 introduced a bug that when all partition columns are constants, the partition is transformed to be a random dispatch, which is not expected.
>
> This patch adds a constant column in the above case to avoid random partitioning.
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/hive/trunk/itests/src/test/resources/testconfiguration.properties 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConstantPropagateProcFactory.java 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/constprog_partitioner.q PRE-CREATION
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/cluster.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/constprog2.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/constprog_partitioner.q.out PRE-CREATION
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join_nullsafe.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd2.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_clusterby.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join4.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join5.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/quotedid_basic.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_25.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/tez/dynamic_partition_pruning.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/tez/dynamic_partition_pruning_2.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/tez/join_nullsafe.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/tez/vector_decimal_mapjoin.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/tez/vectorized_dynamic_partition_pruning.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/union27.q.out 1644497
>
> Diff: https://reviews.apache.org/r/28791/diff/
>
>
> Testing
> -------
>
> TestCliDriver passed.
>
>
> Thanks,
>
> Ted Xu
>
>
Re: Review Request 28791: HIVE-9025 join38.q (without map join)
produces incorrect result when testing with multiple reducers
Posted by Chao Sun <ch...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28791/#review64666
-----------------------------------------------------------
Ship it!
Ship It!
- Chao Sun
On Dec. 11, 2014, 1:36 a.m., Ted Xu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28791/
> -----------------------------------------------------------
>
> (Updated Dec. 11, 2014, 1:36 a.m.)
>
>
> Review request for hive, Ashutosh Chauhan and Chao Sun.
>
>
> Bugs: HIVE-9025
> https://issues.apache.org/jira/browse/HIVE-9025
>
>
> Repository: hive
>
>
> Description
> -------
>
> HIVE-5771 introduced a bug that when all partition columns are constants, the partition is transformed to be a random dispatch, which is not expected.
>
> This patch adds a constant column in the above case to avoid random partitioning.
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/hive/trunk/itests/src/test/resources/testconfiguration.properties 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConstantPropagateProcFactory.java 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/constprog_partitioner.q PRE-CREATION
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/cluster.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/constprog2.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/constprog_partitioner.q.out PRE-CREATION
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join_nullsafe.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd2.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_clusterby.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join4.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join5.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/quotedid_basic.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_25.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/tez/dynamic_partition_pruning.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/tez/dynamic_partition_pruning_2.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/tez/join_nullsafe.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/tez/vector_decimal_mapjoin.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/tez/vectorized_dynamic_partition_pruning.q.out 1644497
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/union27.q.out 1644497
>
> Diff: https://reviews.apache.org/r/28791/diff/
>
>
> Testing
> -------
>
> TestCliDriver passed.
>
>
> Thanks,
>
> Ted Xu
>
>
Re: Review Request 28791: HIVE-9025 join38.q (without map join)
produces incorrect result when testing with multiple reducers
Posted by Ted Xu <fr...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28791/
-----------------------------------------------------------
(Updated 十二月 11, 2014, 1:36 a.m.)
Review request for hive, Ashutosh Chauhan and Chao Sun.
Changes
-------
Format: removed trailing spaces.
Bugs: HIVE-9025
https://issues.apache.org/jira/browse/HIVE-9025
Repository: hive
Description
-------
HIVE-5771 introduced a bug that when all partition columns are constants, the partition is transformed to be a random dispatch, which is not expected.
This patch adds a constant column in the above case to avoid random partitioning.
Diffs (updated)
-----
http://svn.apache.org/repos/asf/hive/trunk/itests/src/test/resources/testconfiguration.properties 1644497
http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConstantPropagateProcFactory.java 1644497
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/constprog_partitioner.q PRE-CREATION
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/cluster.q.out 1644497
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/constprog2.q.out 1644497
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/constprog_partitioner.q.out PRE-CREATION
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join_nullsafe.q.out 1644497
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd2.q.out 1644497
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_clusterby.q.out 1644497
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join4.q.out 1644497
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join5.q.out 1644497
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/quotedid_basic.q.out 1644497
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_25.q.out 1644497
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/tez/dynamic_partition_pruning.q.out 1644497
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/tez/dynamic_partition_pruning_2.q.out 1644497
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/tez/join_nullsafe.q.out 1644497
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/tez/vector_decimal_mapjoin.q.out 1644497
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/tez/vectorized_dynamic_partition_pruning.q.out 1644497
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/union27.q.out 1644497
Diff: https://reviews.apache.org/r/28791/diff/
Testing
-------
TestCliDriver passed.
Thanks,
Ted Xu
Re: Review Request 28791: HIVE-9025 join38.q (without map join)
produces incorrect result when testing with multiple reducers
Posted by Ted Xu <fr...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28791/
-----------------------------------------------------------
(Updated 十二月 10, 2014, 6:09 p.m.)
Review request for hive, Ashutosh Chauhan and Chao Sun.
Changes
-------
Added a TestMinimrCliDriver case constprog_partitioner.q, updated results of TestMiniTezCliDriver.
The following cases may still fail because I don't have a consistent environment with Jenkins.
optimize_nullscan.q
vector_decimal_aggregate.q.out
Ashutosh could you please help me fix them?
Bugs: HIVE-9025
https://issues.apache.org/jira/browse/HIVE-9025
Repository: hive
Description
-------
HIVE-5771 introduced a bug that when all partition columns are constants, the partition is transformed to be a random dispatch, which is not expected.
This patch adds a constant column in the above case to avoid random partitioning.
Diffs (updated)
-----
http://svn.apache.org/repos/asf/hive/trunk/itests/src/test/resources/testconfiguration.properties 1644497
http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConstantPropagateProcFactory.java 1644497
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/constprog_partitioner.q PRE-CREATION
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/cluster.q.out 1644497
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/constprog2.q.out 1644497
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/constprog_partitioner.q.out PRE-CREATION
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join_nullsafe.q.out 1644497
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd2.q.out 1644497
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_clusterby.q.out 1644497
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_join4.q.out 1644497
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join5.q.out 1644497
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/quotedid_basic.q.out 1644497
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin_25.q.out 1644497
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/tez/dynamic_partition_pruning.q.out 1644497
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/tez/dynamic_partition_pruning_2.q.out 1644497
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/tez/join_nullsafe.q.out 1644497
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/tez/vector_decimal_mapjoin.q.out 1644497
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/tez/vectorized_dynamic_partition_pruning.q.out 1644497
http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/union27.q.out 1644497
Diff: https://reviews.apache.org/r/28791/diff/
Testing
-------
TestCliDriver passed.
Thanks,
Ted Xu