You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Deepak Jaiswal <dj...@hortonworks.com> on 2019/02/21 08:59:54 UTC
Review Request 70031: HIVE-21167
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70031/
-----------------------------------------------------------
Review request for hive, Jason Dere and Vaibhav Gumashta.
Bugs: HIVE-21167
https://issues.apache.org/jira/browse/HIVE-21167
Repository: hive-git
Description
-------
Bucketing: Bucketing version 1 is incorrectly partitioning data
Diffs
-----
ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java 4b10e8974e
ql/src/test/queries/clientpositive/murmur_hash_migration.q 2b8da9f683
ql/src/test/results/clientpositive/llap/dynpart_sort_opt_vectorization.q.out 5a2cd47381
ql/src/test/results/clientpositive/llap/murmur_hash_migration.q.out 5343628252
Diff: https://reviews.apache.org/r/70031/diff/1/
Testing
-------
Thanks,
Deepak Jaiswal
Re: Review Request 70031: HIVE-21167
Posted by Vineet Garg <vg...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70031/#review213097
-----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java
Lines 1850 (patched)
<https://reviews.apache.org/r/70031/#comment298963>
Add NULL check for the parent. If a plan doesn't have reduce sink operator and you hit table scan its parent will be NULL
- Vineet Garg
On Feb. 22, 2019, 7:19 a.m., Deepak Jaiswal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70031/
> -----------------------------------------------------------
>
> (Updated Feb. 22, 2019, 7:19 a.m.)
>
>
> Review request for hive, Jason Dere and Vaibhav Gumashta.
>
>
> Bugs: HIVE-21167
> https://issues.apache.org/jira/browse/HIVE-21167
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Bucketing: Bucketing version 1 is incorrectly partitioning data
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java 4b10e8974e
> ql/src/test/queries/clientpositive/murmur_hash_migration.q 2b8da9f683
> ql/src/test/results/clientpositive/llap/dynpart_sort_opt_vectorization.q.out 5a2cd47381
> ql/src/test/results/clientpositive/llap/murmur_hash_migration.q.out 5343628252
>
>
> Diff: https://reviews.apache.org/r/70031/diff/2/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Deepak Jaiswal
>
>
Re: Review Request 70031: HIVE-21167
Posted by Deepak Jaiswal <dj...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70031/
-----------------------------------------------------------
(Updated Feb. 22, 2019, 7:19 a.m.)
Review request for hive, Jason Dere and Vaibhav Gumashta.
Changes
-------
Added the union test which identified an issue which is fixed.
The followup JIRA to show bucketing version in explain extended is created.
https://issues.apache.org/jira/browse/HIVE-21304
Bugs: HIVE-21167
https://issues.apache.org/jira/browse/HIVE-21167
Repository: hive-git
Description
-------
Bucketing: Bucketing version 1 is incorrectly partitioning data
Diffs (updated)
-----
ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java 4b10e8974e
ql/src/test/queries/clientpositive/murmur_hash_migration.q 2b8da9f683
ql/src/test/results/clientpositive/llap/dynpart_sort_opt_vectorization.q.out 5a2cd47381
ql/src/test/results/clientpositive/llap/murmur_hash_migration.q.out 5343628252
Diff: https://reviews.apache.org/r/70031/diff/2/
Changes: https://reviews.apache.org/r/70031/diff/1-2/
Testing
-------
Thanks,
Deepak Jaiswal
Re: Review Request 70031: HIVE-21167
Posted by Deepak Jaiswal <dj...@hortonworks.com>.
> On Feb. 21, 2019, 6:29 p.m., Vineet Garg wrote:
> > ql/src/test/results/clientpositive/llap/dynpart_sort_opt_vectorization.q.out
> > Line 1332 (original), 1332 (patched)
> > <https://reviews.apache.org/r/70031/diff/1/?file=2126091#file2126091line1332>
> >
> > Do you know the reason this size changed? This seems strange.
The size of one file went down by 2 and another went up by 2. It looks like this bug was hitting the test case.
- Deepak
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70031/#review213034
-----------------------------------------------------------
On Feb. 21, 2019, 8:59 a.m., Deepak Jaiswal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70031/
> -----------------------------------------------------------
>
> (Updated Feb. 21, 2019, 8:59 a.m.)
>
>
> Review request for hive, Jason Dere and Vaibhav Gumashta.
>
>
> Bugs: HIVE-21167
> https://issues.apache.org/jira/browse/HIVE-21167
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Bucketing: Bucketing version 1 is incorrectly partitioning data
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java 4b10e8974e
> ql/src/test/queries/clientpositive/murmur_hash_migration.q 2b8da9f683
> ql/src/test/results/clientpositive/llap/dynpart_sort_opt_vectorization.q.out 5a2cd47381
> ql/src/test/results/clientpositive/llap/murmur_hash_migration.q.out 5343628252
>
>
> Diff: https://reviews.apache.org/r/70031/diff/1/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Deepak Jaiswal
>
>
Re: Review Request 70031: HIVE-21167
Posted by Vineet Garg <vg...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70031/#review213034
-----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java
Lines 230 (patched)
<https://reviews.apache.org/r/70031/#comment298893>
Can you also add comment explaining why this should be the last transformation?
ql/src/test/queries/clientpositive/murmur_hash_migration.q
Lines 71 (patched)
<https://reviews.apache.org/r/70031/#comment298896>
There doesn't seem to be any way currently to see the bucketing version used by reduce sink op. It will be really useful to print this information in explain extended. It will help uncover bugs this like.
ql/src/test/queries/clientpositive/murmur_hash_migration.q
Lines 77 (patched)
<https://reviews.apache.org/r/70031/#comment298894>
Can you also add a test with insert select with union? something like
insert into table acid_ptn_bucket1 select key, count(value), key from (select key, value from src where value > 2 group by key, value union all select key, '45' from src s2 where key > 1 group by key) sub group by key;
ql/src/test/results/clientpositive/llap/dynpart_sort_opt_vectorization.q.out
Line 1332 (original), 1332 (patched)
<https://reviews.apache.org/r/70031/#comment298897>
Do you know the reason this size changed? This seems strange.
- Vineet Garg
On Feb. 21, 2019, 8:59 a.m., Deepak Jaiswal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70031/
> -----------------------------------------------------------
>
> (Updated Feb. 21, 2019, 8:59 a.m.)
>
>
> Review request for hive, Jason Dere and Vaibhav Gumashta.
>
>
> Bugs: HIVE-21167
> https://issues.apache.org/jira/browse/HIVE-21167
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Bucketing: Bucketing version 1 is incorrectly partitioning data
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java 4b10e8974e
> ql/src/test/queries/clientpositive/murmur_hash_migration.q 2b8da9f683
> ql/src/test/results/clientpositive/llap/dynpart_sort_opt_vectorization.q.out 5a2cd47381
> ql/src/test/results/clientpositive/llap/murmur_hash_migration.q.out 5343628252
>
>
> Diff: https://reviews.apache.org/r/70031/diff/1/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Deepak Jaiswal
>
>