You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Suma Shivaprasad <su...@gmail.com> on 2014/08/08 08:21:59 UTC
Review Request 24497: HIVE-7629 - Map joins between two parquet tables
failing
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24497/
-----------------------------------------------------------
Review request for hive.
Bugs: HIVE-7629
https://issues.apache.org/jira/browse/HIVE-7629
Repository: hive-git
Description
-------
Map Joins between 2 parquet tables are failing since the Mapper is trying to access the columns of the first table(bigger table) while trying to load the second table(smaller map join table). Fixed this by adding a guard on the column indexes passed by hive
Diffs
-----
ql/src/java/org/apache/hadoop/hive/ql/io/parquet/ProjectionPusher.java 2f155f6
ql/src/java/org/apache/hadoop/hive/ql/io/parquet/read/DataWritableReadSupport.java d6be4bd
ql/src/test/queries/clientpositive/parquet_join.q PRE-CREATION
ql/src/test/results/clientpositive/parquet_join.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/24497/diff/
Testing
-------
parquet_join.q covers most types of joins between 2 parquet tables - Normal, Map join, SMB join
Thanks,
Suma Shivaprasad
Re: Review Request 24497: HIVE-7629 - Map joins between two parquet tables
failing
Posted by Suma Shivaprasad <su...@gmail.com>.
> On Aug. 11, 2014, 4:05 p.m., Brock Noland wrote:
> > ql/src/test/queries/clientpositive/parquet_join.q, line 1
> > <https://reviews.apache.org/r/24497/diff/1/?file=656508#file656508line1>
> >
> > Can you add comments (start with --) which describe how this reproduces the bug?
there is a parsing issue with the comments added. Will update the diff again
- Suma
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24497/#review50182
-----------------------------------------------------------
On Aug. 12, 2014, 6:28 a.m., Suma Shivaprasad wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24497/
> -----------------------------------------------------------
>
> (Updated Aug. 12, 2014, 6:28 a.m.)
>
>
> Review request for hive.
>
>
> Bugs: HIVE-7629
> https://issues.apache.org/jira/browse/HIVE-7629
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Map Joins between 2 parquet tables are failing since the Mapper is trying to access the columns of the first table(bigger table) while trying to load the second table(smaller map join table). Fixed this by adding a guard on the column indexes passed by hive
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/io/parquet/read/DataWritableReadSupport.java d6be4bd
> ql/src/test/queries/clientpositive/parquet_join.q PRE-CREATION
>
> Diff: https://reviews.apache.org/r/24497/diff/
>
>
> Testing
> -------
>
> parquet_join.q covers most types of joins between 2 parquet tables - Normal, Map join, SMB join
>
>
> Thanks,
>
> Suma Shivaprasad
>
>
Re: Review Request 24497: HIVE-7629 - Map joins between two parquet tables
failing
Posted by Suma Shivaprasad <su...@gmail.com>.
> On Aug. 11, 2014, 4:05 p.m., Brock Noland wrote:
> > Thank you very much! Two comments below.
Thanks for reviewing this quickly
- Suma
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24497/#review50182
-----------------------------------------------------------
On Aug. 12, 2014, 6:28 a.m., Suma Shivaprasad wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24497/
> -----------------------------------------------------------
>
> (Updated Aug. 12, 2014, 6:28 a.m.)
>
>
> Review request for hive.
>
>
> Bugs: HIVE-7629
> https://issues.apache.org/jira/browse/HIVE-7629
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Map Joins between 2 parquet tables are failing since the Mapper is trying to access the columns of the first table(bigger table) while trying to load the second table(smaller map join table). Fixed this by adding a guard on the column indexes passed by hive
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/io/parquet/read/DataWritableReadSupport.java d6be4bd
> ql/src/test/queries/clientpositive/parquet_join.q PRE-CREATION
>
> Diff: https://reviews.apache.org/r/24497/diff/
>
>
> Testing
> -------
>
> parquet_join.q covers most types of joins between 2 parquet tables - Normal, Map join, SMB join
>
>
> Thanks,
>
> Suma Shivaprasad
>
>
Re: Review Request 24497: HIVE-7629 - Map joins between two parquet tables
failing
Posted by Brock Noland <br...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24497/#review50182
-----------------------------------------------------------
Thank you very much! Two comments below.
ql/src/java/org/apache/hadoop/hive/ql/io/parquet/read/DataWritableReadSupport.java
<https://reviews.apache.org/r/24497/#comment87783>
nit: Missing space between if and (
ql/src/test/queries/clientpositive/parquet_join.q
<https://reviews.apache.org/r/24497/#comment87782>
Can you add comments (start with --) which describe how this reproduces the bug?
- Brock Noland
On Aug. 8, 2014, 6:21 a.m., Suma Shivaprasad wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24497/
> -----------------------------------------------------------
>
> (Updated Aug. 8, 2014, 6:21 a.m.)
>
>
> Review request for hive.
>
>
> Bugs: HIVE-7629
> https://issues.apache.org/jira/browse/HIVE-7629
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Map Joins between 2 parquet tables are failing since the Mapper is trying to access the columns of the first table(bigger table) while trying to load the second table(smaller map join table). Fixed this by adding a guard on the column indexes passed by hive
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/io/parquet/ProjectionPusher.java 2f155f6
> ql/src/java/org/apache/hadoop/hive/ql/io/parquet/read/DataWritableReadSupport.java d6be4bd
> ql/src/test/queries/clientpositive/parquet_join.q PRE-CREATION
> ql/src/test/results/clientpositive/parquet_join.q.out PRE-CREATION
>
> Diff: https://reviews.apache.org/r/24497/diff/
>
>
> Testing
> -------
>
> parquet_join.q covers most types of joins between 2 parquet tables - Normal, Map join, SMB join
>
>
> Thanks,
>
> Suma Shivaprasad
>
>
Re: Review Request 24497: HIVE-7629 - Map joins between two parquet tables
failing
Posted by Suma Shivaprasad <su...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24497/
-----------------------------------------------------------
(Updated Aug. 12, 2014, 5:38 p.m.)
Review request for hive.
Changes
-------
Fixed parse error
Bugs: HIVE-7629
https://issues.apache.org/jira/browse/HIVE-7629
Repository: hive-git
Description
-------
Map Joins between 2 parquet tables are failing since the Mapper is trying to access the columns of the first table(bigger table) while trying to load the second table(smaller map join table). Fixed this by adding a guard on the column indexes passed by hive
Diffs (updated)
-----
ql/src/java/org/apache/hadoop/hive/ql/io/parquet/read/DataWritableReadSupport.java d6be4bd
ql/src/test/queries/clientpositive/parquet_join.q PRE-CREATION
ql/src/test/results/clientpositive/parquet_join.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/24497/diff/
Testing
-------
parquet_join.q covers most types of joins between 2 parquet tables - Normal, Map join, SMB join
Thanks,
Suma Shivaprasad
Re: Review Request 24497: HIVE-7629 - Map joins between two parquet tables
failing
Posted by Suma Shivaprasad <su...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24497/
-----------------------------------------------------------
(Updated Aug. 12, 2014, 6:28 a.m.)
Review request for hive.
Changes
-------
Fixed review comments
Bugs: HIVE-7629
https://issues.apache.org/jira/browse/HIVE-7629
Repository: hive-git
Description
-------
Map Joins between 2 parquet tables are failing since the Mapper is trying to access the columns of the first table(bigger table) while trying to load the second table(smaller map join table). Fixed this by adding a guard on the column indexes passed by hive
Diffs (updated)
-----
ql/src/java/org/apache/hadoop/hive/ql/io/parquet/read/DataWritableReadSupport.java d6be4bd
ql/src/test/queries/clientpositive/parquet_join.q PRE-CREATION
Diff: https://reviews.apache.org/r/24497/diff/
Testing
-------
parquet_join.q covers most types of joins between 2 parquet tables - Normal, Map join, SMB join
Thanks,
Suma Shivaprasad