You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Navis Ryu <na...@nexr.com> on 2013/12/16 03:15:20 UTC
Review Request 16281: Predicates following UDTF operator are removed by PPD
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16281/
-----------------------------------------------------------
Review request for hive.
Bugs: HIVE-4293
https://issues.apache.org/jira/browse/HIVE-4293
Repository: hive-git
Description
-------
For example,
{noformat}
explain SELECT value from (
select explode(array(key, value)) as (value) from (
select * FROM src WHERE key > 200
) A
) B WHERE value > 300
;
{noformat}
Makes plan like this, removing last predicates
{noformat}
TableScan
alias: src
Filter Operator
predicate:
expr: (key > 200.0)
type: boolean
Select Operator
expressions:
expr: array(key,value)
type: array<string>
outputColumnNames: _col0
UDTF Operator
function name: explode
Select Operator
expressions:
expr: col
type: string
outputColumnNames: _col0
File Output Operator
compressed: false
GlobalTableId: 0
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
{noformat}
Diffs
-----
ql/src/java/org/apache/hadoop/hive/ql/exec/LateralViewJoinOperator.java 2fbb81b
ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java c378dc7
ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java 0798470
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java d0a0ec7
ql/src/java/org/apache/hadoop/hive/ql/plan/LateralViewJoinDesc.java ebfcfc8
ql/src/java/org/apache/hadoop/hive/ql/ppd/ExprWalkerInfo.java 6a3dd99
ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java 40298e1
ql/src/java/org/apache/hadoop/hive/ql/ppd/PredicatePushDown.java cd5ae51
ql/src/test/queries/clientpositive/lateral_view_ppd.q 7be86a6
ql/src/test/queries/clientpositive/ppd_udtf.q PRE-CREATION
ql/src/test/results/clientpositive/cluster.q.out 8d14a1d
ql/src/test/results/clientpositive/ctas_colname.q.out a15b698
ql/src/test/results/clientpositive/lateral_view_ppd.q.out f54c809
ql/src/test/results/clientpositive/ppd2.q.out f6af8f8
ql/src/test/results/clientpositive/ppd_gby.q.out 5908450
ql/src/test/results/clientpositive/ppd_gby2.q.out bdd7e89
ql/src/test/results/clientpositive/ppd_udtf.q.out PRE-CREATION
ql/src/test/results/clientpositive/udtf_json_tuple.q.out 1a480b6
ql/src/test/results/clientpositive/udtf_parse_url_tuple.q.out a38b31b
ql/src/test/results/compiler/plan/join1.q.xml 9e4b609
ql/src/test/results/compiler/plan/join2.q.xml efcb865
ql/src/test/results/compiler/plan/join3.q.xml 9bbe64f
ql/src/test/results/compiler/plan/join4.q.xml f8a8f10
ql/src/test/results/compiler/plan/join5.q.xml d43ce00
ql/src/test/results/compiler/plan/join6.q.xml a354e4b
ql/src/test/results/compiler/plan/join7.q.xml ac41995
ql/src/test/results/compiler/plan/join8.q.xml 569cf6e
Diff: https://reviews.apache.org/r/16281/diff/
Testing
-------
Thanks,
Navis Ryu
Re: Review Request 16281: Predicates following UDTF operator are removed by
PPD
Posted by Navis Ryu <na...@nexr.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16281/
-----------------------------------------------------------
(Updated March 5, 2014, 4:07 a.m.)
Review request for hive.
Changes
-------
+ Works of Harish + Partial fix in HIVE-4598
Bugs: HIVE-4293
https://issues.apache.org/jira/browse/HIVE-4293
Repository: hive-git
Description
-------
For example,
{noformat}
explain SELECT value from (
select explode(array(key, value)) as (value) from (
select * FROM src WHERE key > 200
) A
) B WHERE value > 300
;
{noformat}
Makes plan like this, removing last predicates
{noformat}
TableScan
alias: src
Filter Operator
predicate:
expr: (key > 200.0)
type: boolean
Select Operator
expressions:
expr: array(key,value)
type: array<string>
outputColumnNames: _col0
UDTF Operator
function name: explode
Select Operator
expressions:
expr: col
type: string
outputColumnNames: _col0
File Output Operator
compressed: false
GlobalTableId: 0
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
{noformat}
Diffs (updated)
-----
ql/src/java/org/apache/hadoop/hive/ql/exec/LateralViewJoinOperator.java 2fbb81b
ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java c378dc7
ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapRedTask.java 326654f
ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java 0798470
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 89d2a9c
ql/src/java/org/apache/hadoop/hive/ql/plan/LateralViewJoinDesc.java ebfcfc8
ql/src/java/org/apache/hadoop/hive/ql/ppd/ExprWalkerInfo.java 6a3dd99
ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java 40298e1
ql/src/java/org/apache/hadoop/hive/ql/ppd/PredicatePushDown.java cd5ae51
ql/src/test/queries/clientpositive/lateral_view_ppd.q 7be86a6
ql/src/test/queries/clientpositive/ppd_join4.q PRE-CREATION
ql/src/test/queries/clientpositive/ppd_transform.q 65a498d
ql/src/test/queries/clientpositive/ppd_udtf.q PRE-CREATION
ql/src/test/results/clientpositive/cluster.q.out 0cd0886
ql/src/test/results/clientpositive/ctas_colname.q.out 3d568ab
ql/src/test/results/clientpositive/lateral_view_ppd.q.out da77f75
ql/src/test/results/clientpositive/ppd2.q.out 2f2c558
ql/src/test/results/clientpositive/ppd_gby.q.out 68092e0
ql/src/test/results/clientpositive/ppd_gby2.q.out a8ccace
ql/src/test/results/clientpositive/ppd_join4.q.out PRE-CREATION
ql/src/test/results/clientpositive/ppd_transform.q.out e7c07ed
ql/src/test/results/clientpositive/ppd_udtf.q.out PRE-CREATION
ql/src/test/results/clientpositive/udtf_json_tuple.q.out f151740
ql/src/test/results/clientpositive/udtf_parse_url_tuple.q.out 74d9e96
ql/src/test/results/compiler/plan/join1.q.xml 12b01ce
ql/src/test/results/compiler/plan/join2.q.xml ed5bbb8
ql/src/test/results/compiler/plan/join3.q.xml 5437afa
ql/src/test/results/compiler/plan/join4.q.xml aa69ada
ql/src/test/results/compiler/plan/join5.q.xml ef0c69d
ql/src/test/results/compiler/plan/join6.q.xml da528f5
ql/src/test/results/compiler/plan/join7.q.xml fcacc6d
ql/src/test/results/compiler/plan/join8.q.xml c7591a4
Diff: https://reviews.apache.org/r/16281/diff/
Testing
-------
Thanks,
Navis Ryu
Re: Review Request 16281: Predicates following UDTF operator are removed by
PPD
Posted by Navis Ryu <na...@nexr.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16281/
-----------------------------------------------------------
(Updated Jan. 22, 2014, 8:31 a.m.)
Review request for hive.
Changes
-------
Rebased to trunk
Bugs: HIVE-4293
https://issues.apache.org/jira/browse/HIVE-4293
Repository: hive-git
Description
-------
For example,
{noformat}
explain SELECT value from (
select explode(array(key, value)) as (value) from (
select * FROM src WHERE key > 200
) A
) B WHERE value > 300
;
{noformat}
Makes plan like this, removing last predicates
{noformat}
TableScan
alias: src
Filter Operator
predicate:
expr: (key > 200.0)
type: boolean
Select Operator
expressions:
expr: array(key,value)
type: array<string>
outputColumnNames: _col0
UDTF Operator
function name: explode
Select Operator
expressions:
expr: col
type: string
outputColumnNames: _col0
File Output Operator
compressed: false
GlobalTableId: 0
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
{noformat}
Diffs (updated)
-----
ql/src/java/org/apache/hadoop/hive/ql/exec/LateralViewJoinOperator.java 2fbb81b
ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java c378dc7
ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java 0798470
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 1f7aae0
ql/src/java/org/apache/hadoop/hive/ql/plan/LateralViewJoinDesc.java ebfcfc8
ql/src/java/org/apache/hadoop/hive/ql/ppd/ExprWalkerInfo.java 6a3dd99
ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java 40298e1
ql/src/java/org/apache/hadoop/hive/ql/ppd/PredicatePushDown.java cd5ae51
ql/src/test/queries/clientpositive/lateral_view_ppd.q 7be86a6
ql/src/test/queries/clientpositive/ppd_udtf.q PRE-CREATION
ql/src/test/results/clientpositive/cluster.q.out 8d14a1d
ql/src/test/results/clientpositive/ctas_colname.q.out a15b698
ql/src/test/results/clientpositive/lateral_view_ppd.q.out f54c809
ql/src/test/results/clientpositive/ppd2.q.out f6af8f8
ql/src/test/results/clientpositive/ppd_gby.q.out 5908450
ql/src/test/results/clientpositive/ppd_gby2.q.out bdd7e89
ql/src/test/results/clientpositive/ppd_udtf.q.out PRE-CREATION
ql/src/test/results/clientpositive/udtf_json_tuple.q.out 1a480b6
ql/src/test/results/clientpositive/udtf_parse_url_tuple.q.out a38b31b
ql/src/test/results/compiler/plan/join1.q.xml dbb8ca9
ql/src/test/results/compiler/plan/join2.q.xml d13890e
ql/src/test/results/compiler/plan/join3.q.xml 81ce3e2
ql/src/test/results/compiler/plan/join4.q.xml 116f2ad
ql/src/test/results/compiler/plan/join5.q.xml 9dd4af5
ql/src/test/results/compiler/plan/join6.q.xml 7134e08
ql/src/test/results/compiler/plan/join7.q.xml 9b7103e
ql/src/test/results/compiler/plan/join8.q.xml 7e2834f
Diff: https://reviews.apache.org/r/16281/diff/
Testing
-------
Thanks,
Navis Ryu