You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by kelly zhang <li...@intel.com> on 2017/08/23 07:29:49 UTC
Review Request 61839: HIVE-16823:ArrayIndexOutOfBoundsException" in
spark_vectorized_dynamic_partition_pruning.q
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61839/
-----------------------------------------------------------
Review request for hive, Rui Li and Sahil Takiar.
Bugs: HIVE-16823
https://issues.apache.org/jira/browse/HIVE-16823
Repository: hive-git
Description
-------
More detailed see jira board
Diffs
-----
ql/src/java/org/apache/hadoop/hive/ql/parse/spark/SparkCompiler.java 73e596e
ql/src/test/results/clientpositive/spark/spark_dynamic_partition_pruning.q.out 63a9548
ql/src/test/results/clientpositive/spark/spark_dynamic_partition_pruning_2.q.out cc7819c
ql/src/test/results/clientpositive/spark/spark_dynamic_partition_pruning_3.q.out 82e746a
ql/src/test/results/clientpositive/spark/spark_dynamic_partition_pruning_mapjoin_only.q.out 41e98a1
ql/src/test/results/clientpositive/spark/spark_vectorized_dynamic_partition_pruning.q.out 699fcc6
Diff: https://reviews.apache.org/r/61839/diff/1/
Testing
-------
Thanks,
kelly zhang
Re: Review Request 61839: HIVE-16823:ArrayIndexOutOfBoundsException"
in spark_vectorized_dynamic_partition_pruning.q
Posted by kelly zhang <li...@intel.com>.
> On Aug. 23, 2017, 8:33 a.m., kelly zhang wrote:
> > ql/src/test/results/clientpositive/spark/spark_vectorized_dynamic_partition_pruning.q.out
> > Line 5331 (original)
> > <https://reviews.apache.org/r/61839/diff/1/?file=1802076#file1802076line5720>
> >
> > will investigate why Map4 is removed in Line 5312 in query
> > " EXPLAIN select count(*) from srcpart full outer join srcpart_date on (srcpart.ds = srcpart_date.ds) where srcpart_date.`date` = '2008-04-08'"
this change is because HIVE-17087:Remove unnecessary HoS DPP trees during map-join conversion
- kelly
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61839/#review183579
-----------------------------------------------------------
On Aug. 23, 2017, 7:29 a.m., kelly zhang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61839/
> -----------------------------------------------------------
>
> (Updated Aug. 23, 2017, 7:29 a.m.)
>
>
> Review request for hive, Rui Li and Sahil Takiar.
>
>
> Bugs: HIVE-16823
> https://issues.apache.org/jira/browse/HIVE-16823
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> More detailed see jira board
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/parse/spark/SparkCompiler.java 73e596e
> ql/src/test/results/clientpositive/spark/spark_dynamic_partition_pruning.q.out 63a9548
> ql/src/test/results/clientpositive/spark/spark_dynamic_partition_pruning_2.q.out cc7819c
> ql/src/test/results/clientpositive/spark/spark_dynamic_partition_pruning_3.q.out 82e746a
> ql/src/test/results/clientpositive/spark/spark_dynamic_partition_pruning_mapjoin_only.q.out 41e98a1
> ql/src/test/results/clientpositive/spark/spark_vectorized_dynamic_partition_pruning.q.out 699fcc6
>
>
> Diff: https://reviews.apache.org/r/61839/diff/1/
>
>
> Testing
> -------
>
>
> Thanks,
>
> kelly zhang
>
>
Re: Review Request 61839: HIVE-16823:ArrayIndexOutOfBoundsException"
in spark_vectorized_dynamic_partition_pruning.q
Posted by kelly zhang <li...@intel.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61839/#review183579
-----------------------------------------------------------
ql/src/test/results/clientpositive/spark/spark_vectorized_dynamic_partition_pruning.q.out
Line 17 (original)
<https://reviews.apache.org/r/61839/#comment259630>
the changes is because we have not add "-- SORT_QUERY_RESULTS"
ql/src/test/results/clientpositive/spark/spark_vectorized_dynamic_partition_pruning.q.out
Line 875 (original)
<https://reviews.apache.org/r/61839/#comment259631>
The reason why Map5 and Map6 is combined is because HIVE-11297
ql/src/test/results/clientpositive/spark/spark_vectorized_dynamic_partition_pruning.q.out
Line 2337 (original)
<https://reviews.apache.org/r/61839/#comment259632>
Why Map5 and Map6 is combined is because HIVE-11297
ql/src/test/results/clientpositive/spark/spark_vectorized_dynamic_partition_pruning.q.out
Lines 2891-2918 (patched)
<https://reviews.apache.org/r/61839/#comment259633>
Map 8 is added. Map 8 is necessary because in query
{code}
select count(*) from srcpart join srcpart_date on (srcpart.ds = srcpart_date.ds) join srcpart_hour on (srcpart.hr = srcpart_hour.hr)
{code}
srcpart_date and srcpart_hour are all small tables, both of them should be converted to dpp operator tree because srcpart has two partitions "srcpart.ds" and "srcpart.hr"
ql/src/test/results/clientpositive/spark/spark_vectorized_dynamic_partition_pruning.q.out
Line 5331 (original)
<https://reviews.apache.org/r/61839/#comment259637>
will investigate why Map4 is removed in Line 5312 in query
" EXPLAIN select count(*) from srcpart full outer join srcpart_date on (srcpart.ds = srcpart_date.ds) where srcpart_date.`date` = '2008-04-08'"
- kelly zhang
On Aug. 23, 2017, 7:29 a.m., kelly zhang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61839/
> -----------------------------------------------------------
>
> (Updated Aug. 23, 2017, 7:29 a.m.)
>
>
> Review request for hive, Rui Li and Sahil Takiar.
>
>
> Bugs: HIVE-16823
> https://issues.apache.org/jira/browse/HIVE-16823
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> More detailed see jira board
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/parse/spark/SparkCompiler.java 73e596e
> ql/src/test/results/clientpositive/spark/spark_dynamic_partition_pruning.q.out 63a9548
> ql/src/test/results/clientpositive/spark/spark_dynamic_partition_pruning_2.q.out cc7819c
> ql/src/test/results/clientpositive/spark/spark_dynamic_partition_pruning_3.q.out 82e746a
> ql/src/test/results/clientpositive/spark/spark_dynamic_partition_pruning_mapjoin_only.q.out 41e98a1
> ql/src/test/results/clientpositive/spark/spark_vectorized_dynamic_partition_pruning.q.out 699fcc6
>
>
> Diff: https://reviews.apache.org/r/61839/diff/1/
>
>
> Testing
> -------
>
>
> Thanks,
>
> kelly zhang
>
>