You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Sergey Shelukhin <se...@hortonworks.com> on 2016/06/10 22:47:29 UTC
Re: Review Request 47735: HIVE-13617 LLAP: support non-vectorized
execution in IO
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47735/
-----------------------------------------------------------
(Updated June 10, 2016, 10:47 p.m.)
Review request for hive and Prasanth_J.
Repository: hive-git
Description
-------
see JIRA
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 285caa3
itests/src/test/resources/testconfiguration.properties 46350a3
llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapInputFormat.java 298f788
ql/src/java/org/apache/hadoop/hive/ql/exec/MapOperator.java afe5ee2
ql/src/java/org/apache/hadoop/hive/ql/io/BatchToRowInputFormat.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/io/BatchToRowReader.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java cfedf35
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java d7a8c2f
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcOiBatchToRowReader.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 7595065
ql/src/java/org/apache/hadoop/hive/ql/plan/MapWork.java f034812
ql/src/test/queries/clientpositive/orc_llap_nonvector.q PRE-CREATION
ql/src/test/results/clientpositive/llap/dynamic_partition_pruning.q.out 912cd7f
ql/src/test/results/clientpositive/llap/hybridgrace_hashjoin_1.q.out ad3a358
ql/src/test/results/clientpositive/llap/llap_udf.q.out e517942
ql/src/test/results/clientpositive/llap/llapdecider.q.out 27ab92a
ql/src/test/results/clientpositive/llap/mapjoin_decimal.q.out 2d1b818
ql/src/test/results/clientpositive/llap/orc_llap_nonvector.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/orc_ppd_basic.q.out 2d0984b
ql/src/test/results/clientpositive/llap/tez_bmj_schema_evolution.q.out f99b829
ql/src/test/results/clientpositive/llap/tez_dynpart_hashjoin_1.q.out c9417f4
ql/src/test/results/clientpositive/llap/tez_dynpart_hashjoin_2.q.out 7f29462
ql/src/test/results/clientpositive/llap/tez_union_group_by.q.out 11882e4
ql/src/test/results/clientpositive/llap/tez_vector_dynpart_hashjoin_1.q.out d02e096
ql/src/test/results/clientpositive/llap/tez_vector_dynpart_hashjoin_2.q.out c0c14e1
ql/src/test/results/clientpositive/llap/vectorized_dynamic_partition_pruning.q.out 487d930
ql/src/test/results/clientpositive/orc_llap_nonvector.q.out PRE-CREATION
ql/src/test/results/clientpositive/tez/llapdecider.q.out 27ab92a
Diff: https://reviews.apache.org/r/47735/diff/
Testing
-------
Thanks,
Sergey Shelukhin
Re: Review Request 47735: HIVE-13617 LLAP: support non-vectorized
execution in IO
Posted by Sergey Shelukhin <se...@hortonworks.com>.
> On June 22, 2016, 2:35 a.m., Prasanth_J wrote:
> > llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapInputFormat.java, line 418
> > <https://reviews.apache.org/r/47735/diff/2/?file=1415331#file1415331line418>
> >
> > Can this not be reused?
no, Vectorizer vectorizes entire MapWork w/o good boundaries
> On June 22, 2016, 2:35 a.m., Prasanth_J wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/io/BatchToRowReader.java, line 78
> > <https://reviews.apache.org/r/47735/diff/2/?file=1415334#file1415334line78>
> >
> > Why UnionType?
hmm?
> On June 22, 2016, 2:35 a.m., Prasanth_J wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/io/BatchToRowReader.java, line 418
> > <https://reviews.apache.org/r/47735/diff/2/?file=1415334#file1415334line418>
> >
> > Should we do enforcing of precision and scale here or is it assumed to be already enforced?
Assuming it's enforced. This code is moved from original ORC code.
- Sergey
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47735/#review139006
-----------------------------------------------------------
On June 10, 2016, 10:47 p.m., Sergey Shelukhin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47735/
> -----------------------------------------------------------
>
> (Updated June 10, 2016, 10:47 p.m.)
>
>
> Review request for hive and Prasanth_J.
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> see JIRA
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 285caa3
> itests/src/test/resources/testconfiguration.properties 46350a3
> llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapInputFormat.java 298f788
> ql/src/java/org/apache/hadoop/hive/ql/exec/MapOperator.java afe5ee2
> ql/src/java/org/apache/hadoop/hive/ql/io/BatchToRowInputFormat.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/io/BatchToRowReader.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java cfedf35
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java d7a8c2f
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcOiBatchToRowReader.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 7595065
> ql/src/java/org/apache/hadoop/hive/ql/plan/MapWork.java f034812
> ql/src/test/queries/clientpositive/orc_llap_nonvector.q PRE-CREATION
> ql/src/test/results/clientpositive/llap/dynamic_partition_pruning.q.out 912cd7f
> ql/src/test/results/clientpositive/llap/hybridgrace_hashjoin_1.q.out ad3a358
> ql/src/test/results/clientpositive/llap/llap_udf.q.out e517942
> ql/src/test/results/clientpositive/llap/llapdecider.q.out 27ab92a
> ql/src/test/results/clientpositive/llap/mapjoin_decimal.q.out 2d1b818
> ql/src/test/results/clientpositive/llap/orc_llap_nonvector.q.out PRE-CREATION
> ql/src/test/results/clientpositive/llap/orc_ppd_basic.q.out 2d0984b
> ql/src/test/results/clientpositive/llap/tez_bmj_schema_evolution.q.out f99b829
> ql/src/test/results/clientpositive/llap/tez_dynpart_hashjoin_1.q.out c9417f4
> ql/src/test/results/clientpositive/llap/tez_dynpart_hashjoin_2.q.out 7f29462
> ql/src/test/results/clientpositive/llap/tez_union_group_by.q.out 11882e4
> ql/src/test/results/clientpositive/llap/tez_vector_dynpart_hashjoin_1.q.out d02e096
> ql/src/test/results/clientpositive/llap/tez_vector_dynpart_hashjoin_2.q.out c0c14e1
> ql/src/test/results/clientpositive/llap/vectorized_dynamic_partition_pruning.q.out 487d930
> ql/src/test/results/clientpositive/orc_llap_nonvector.q.out PRE-CREATION
> ql/src/test/results/clientpositive/tez/llapdecider.q.out 27ab92a
>
> Diff: https://reviews.apache.org/r/47735/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Sergey Shelukhin
>
>
Re: Review Request 47735: HIVE-13617 LLAP: support non-vectorized
execution in IO
Posted by j....@gmail.com.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47735/#review139006
-----------------------------------------------------------
llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapInputFormat.java (line 400)
<https://reviews.apache.org/r/47735/#comment204183>
Can this not be reused?
ql/src/java/org/apache/hadoop/hive/ql/io/BatchToRowReader.java (line 78)
<https://reviews.apache.org/r/47735/#comment204189>
Why UnionType?
ql/src/java/org/apache/hadoop/hive/ql/io/BatchToRowReader.java (line 418)
<https://reviews.apache.org/r/47735/#comment204190>
Should we do enforcing of precision and scale here or is it assumed to be already enforced?
- Prasanth_J
On June 10, 2016, 10:47 p.m., Sergey Shelukhin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47735/
> -----------------------------------------------------------
>
> (Updated June 10, 2016, 10:47 p.m.)
>
>
> Review request for hive and Prasanth_J.
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> see JIRA
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 285caa3
> itests/src/test/resources/testconfiguration.properties 46350a3
> llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapInputFormat.java 298f788
> ql/src/java/org/apache/hadoop/hive/ql/exec/MapOperator.java afe5ee2
> ql/src/java/org/apache/hadoop/hive/ql/io/BatchToRowInputFormat.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/io/BatchToRowReader.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java cfedf35
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java d7a8c2f
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcOiBatchToRowReader.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 7595065
> ql/src/java/org/apache/hadoop/hive/ql/plan/MapWork.java f034812
> ql/src/test/queries/clientpositive/orc_llap_nonvector.q PRE-CREATION
> ql/src/test/results/clientpositive/llap/dynamic_partition_pruning.q.out 912cd7f
> ql/src/test/results/clientpositive/llap/hybridgrace_hashjoin_1.q.out ad3a358
> ql/src/test/results/clientpositive/llap/llap_udf.q.out e517942
> ql/src/test/results/clientpositive/llap/llapdecider.q.out 27ab92a
> ql/src/test/results/clientpositive/llap/mapjoin_decimal.q.out 2d1b818
> ql/src/test/results/clientpositive/llap/orc_llap_nonvector.q.out PRE-CREATION
> ql/src/test/results/clientpositive/llap/orc_ppd_basic.q.out 2d0984b
> ql/src/test/results/clientpositive/llap/tez_bmj_schema_evolution.q.out f99b829
> ql/src/test/results/clientpositive/llap/tez_dynpart_hashjoin_1.q.out c9417f4
> ql/src/test/results/clientpositive/llap/tez_dynpart_hashjoin_2.q.out 7f29462
> ql/src/test/results/clientpositive/llap/tez_union_group_by.q.out 11882e4
> ql/src/test/results/clientpositive/llap/tez_vector_dynpart_hashjoin_1.q.out d02e096
> ql/src/test/results/clientpositive/llap/tez_vector_dynpart_hashjoin_2.q.out c0c14e1
> ql/src/test/results/clientpositive/llap/vectorized_dynamic_partition_pruning.q.out 487d930
> ql/src/test/results/clientpositive/orc_llap_nonvector.q.out PRE-CREATION
> ql/src/test/results/clientpositive/tez/llapdecider.q.out 27ab92a
>
> Diff: https://reviews.apache.org/r/47735/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Sergey Shelukhin
>
>