You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org> on 2011/04/05 17:02:06 UTC
[jira] [Commented] (HIVE-1538) FilterOperator is applied twice with
ppd on.
[ https://issues.apache.org/jira/browse/HIVE-1538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13015943#comment-13015943 ]
jiraposter@reviews.apache.org commented on HIVE-1538:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/550/
-----------------------------------------------------------
Review request for hive, Yongqiang He and namit jain.
Summary
-------
Patch updated to trunk with newly added configuration var hive.ppd.remove.duplicatefilters
This addresses bug HIVE-1538.
https://issues.apache.org/jira/browse/HIVE-1538
Diffs
-----
trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1088944
trunk/contrib/src/test/results/clientpositive/dboutput.q.out 1088944
trunk/contrib/src/test/results/clientpositive/serde_typedbytes4.q.out 1088944
trunk/hbase-handler/src/test/results/hbase_pushdown.q.out 1088944
trunk/hbase-handler/src/test/results/hbase_queries.q.out 1088944
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java 1088944
trunk/ql/src/java/org/apache/hadoop/hive/ql/ppd/ExprWalkerInfo.java 1088944
trunk/ql/src/java/org/apache/hadoop/hive/ql/ppd/ExprWalkerProcFactory.java 1088944
trunk/ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java 1088944
trunk/ql/src/java/org/apache/hadoop/hive/ql/ppd/OpWalkerInfo.java 1088944
trunk/ql/src/java/org/apache/hadoop/hive/ql/ppd/PredicatePushDown.java 1088944
trunk/ql/src/test/queries/clientpositive/ppd1.q 1088944
trunk/ql/src/test/queries/clientpositive/ppd_clusterby.q 1088944
trunk/ql/src/test/queries/clientpositive/ppd_constant_expr.q 1088944
trunk/ql/src/test/queries/clientpositive/ppd_gby.q 1088944
trunk/ql/src/test/queries/clientpositive/ppd_gby2.q 1088944
trunk/ql/src/test/queries/clientpositive/ppd_gby_join.q 1088944
trunk/ql/src/test/queries/clientpositive/ppd_join.q 1088944
trunk/ql/src/test/queries/clientpositive/ppd_join2.q 1088944
trunk/ql/src/test/queries/clientpositive/ppd_join3.q 1088944
trunk/ql/src/test/queries/clientpositive/ppd_multi_insert.q 1088944
trunk/ql/src/test/queries/clientpositive/ppd_outer_join1.q 1088944
trunk/ql/src/test/queries/clientpositive/ppd_outer_join2.q 1088944
trunk/ql/src/test/queries/clientpositive/ppd_outer_join3.q 1088944
trunk/ql/src/test/queries/clientpositive/ppd_outer_join4.q 1088944
trunk/ql/src/test/queries/clientpositive/ppd_random.q 1088944
trunk/ql/src/test/queries/clientpositive/ppd_transform.q 1088944
trunk/ql/src/test/queries/clientpositive/ppd_udf_case.q 1088944
trunk/ql/src/test/queries/clientpositive/ppd_union.q 1088944
trunk/ql/src/test/results/clientpositive/auto_join0.q.out 1088944
trunk/ql/src/test/results/clientpositive/auto_join11.q.out 1088944
trunk/ql/src/test/results/clientpositive/auto_join12.q.out 1088944
trunk/ql/src/test/results/clientpositive/auto_join13.q.out 1088944
trunk/ql/src/test/results/clientpositive/auto_join14.q.out 1088944
trunk/ql/src/test/results/clientpositive/auto_join16.q.out 1088944
trunk/ql/src/test/results/clientpositive/auto_join19.q.out 1088944
trunk/ql/src/test/results/clientpositive/auto_join20.q.out 1088944
trunk/ql/src/test/results/clientpositive/auto_join21.q.out 1088944
trunk/ql/src/test/results/clientpositive/auto_join23.q.out 1088944
trunk/ql/src/test/results/clientpositive/auto_join27.q.out 1088944
trunk/ql/src/test/results/clientpositive/auto_join4.q.out 1088944
trunk/ql/src/test/results/clientpositive/auto_join5.q.out 1088944
trunk/ql/src/test/results/clientpositive/auto_join6.q.out 1088944
trunk/ql/src/test/results/clientpositive/auto_join7.q.out 1088944
trunk/ql/src/test/results/clientpositive/auto_join8.q.out 1088944
trunk/ql/src/test/results/clientpositive/auto_join9.q.out 1088944
trunk/ql/src/test/results/clientpositive/bucket2.q.out 1088944
trunk/ql/src/test/results/clientpositive/bucket3.q.out 1088944
trunk/ql/src/test/results/clientpositive/bucket4.q.out 1088944
trunk/ql/src/test/results/clientpositive/bucket_groupby.q.out 1088944
trunk/ql/src/test/results/clientpositive/bucketmapjoin1.q.out 1088944
trunk/ql/src/test/results/clientpositive/bucketmapjoin2.q.out 1088944
trunk/ql/src/test/results/clientpositive/bucketmapjoin3.q.out 1088944
trunk/ql/src/test/results/clientpositive/bucketmapjoin_negative.q.out 1088944
trunk/ql/src/test/results/clientpositive/case_sensitivity.q.out 1088944
trunk/ql/src/test/results/clientpositive/cast1.q.out 1088944
trunk/ql/src/test/results/clientpositive/cluster.q.out 1088944
trunk/ql/src/test/results/clientpositive/combine2.q.out 1088944
trunk/ql/src/test/results/clientpositive/create_view.q.out 1088944
trunk/ql/src/test/results/clientpositive/disable_merge_for_bucketing.q.out 1088944
trunk/ql/src/test/results/clientpositive/filter_join_breaktask.q.out 1088944
trunk/ql/src/test/results/clientpositive/groupby_map_ppr.q.out 1088944
trunk/ql/src/test/results/clientpositive/groupby_map_ppr_multi_distinct.q.out 1088944
trunk/ql/src/test/results/clientpositive/groupby_ppr.q.out 1088944
trunk/ql/src/test/results/clientpositive/groupby_ppr_multi_distinct.q.out 1088944
trunk/ql/src/test/results/clientpositive/having.q.out 1088944
trunk/ql/src/test/results/clientpositive/implicit_cast1.q.out 1088944
trunk/ql/src/test/results/clientpositive/input11.q.out 1088944
trunk/ql/src/test/results/clientpositive/input11_limit.q.out 1088944
trunk/ql/src/test/results/clientpositive/input14.q.out 1088944
trunk/ql/src/test/results/clientpositive/input18.q.out 1088944
trunk/ql/src/test/results/clientpositive/input23.q.out 1088944
trunk/ql/src/test/results/clientpositive/input24.q.out 1088944
trunk/ql/src/test/results/clientpositive/input25.q.out 1088944
trunk/ql/src/test/results/clientpositive/input26.q.out 1088944
trunk/ql/src/test/results/clientpositive/input2_limit.q.out 1088944
trunk/ql/src/test/results/clientpositive/input31.q.out 1088944
trunk/ql/src/test/results/clientpositive/input39.q.out 1088944
trunk/ql/src/test/results/clientpositive/input42.q.out 1088944
trunk/ql/src/test/results/clientpositive/input6.q.out 1088944
trunk/ql/src/test/results/clientpositive/input9.q.out 1088944
trunk/ql/src/test/results/clientpositive/input_part1.q.out 1088944
trunk/ql/src/test/results/clientpositive/input_part5.q.out 1088944
trunk/ql/src/test/results/clientpositive/input_part6.q.out 1088944
trunk/ql/src/test/results/clientpositive/input_part7.q.out 1088944
trunk/ql/src/test/results/clientpositive/input_part9.q.out 1088944
trunk/ql/src/test/results/clientpositive/input_testxpath2.q.out 1088944
trunk/ql/src/test/results/clientpositive/input_testxpath4.q.out 1088944
trunk/ql/src/test/results/clientpositive/join0.q.out 1088944
trunk/ql/src/test/results/clientpositive/join11.q.out 1088944
trunk/ql/src/test/results/clientpositive/join12.q.out 1088944
trunk/ql/src/test/results/clientpositive/join13.q.out 1088944
trunk/ql/src/test/results/clientpositive/join14.q.out 1088944
trunk/ql/src/test/results/clientpositive/join16.q.out 1088944
trunk/ql/src/test/results/clientpositive/join19.q.out 1088944
trunk/ql/src/test/results/clientpositive/join20.q.out 1088944
trunk/ql/src/test/results/clientpositive/join21.q.out 1088944
trunk/ql/src/test/results/clientpositive/join23.q.out 1088944
trunk/ql/src/test/results/clientpositive/join26.q.out 1088944
trunk/ql/src/test/results/clientpositive/join28.q.out 1088944
trunk/ql/src/test/results/clientpositive/join32.q.out 1088944
trunk/ql/src/test/results/clientpositive/join33.q.out 1088944
trunk/ql/src/test/results/clientpositive/join34.q.out 1088944
trunk/ql/src/test/results/clientpositive/join35.q.out 1088944
trunk/ql/src/test/results/clientpositive/join38.q.out 1088944
trunk/ql/src/test/results/clientpositive/join39.q.out 1088944
trunk/ql/src/test/results/clientpositive/join4.q.out 1088944
trunk/ql/src/test/results/clientpositive/join40.q.out 1088944
trunk/ql/src/test/results/clientpositive/join5.q.out 1088944
trunk/ql/src/test/results/clientpositive/join6.q.out 1088944
trunk/ql/src/test/results/clientpositive/join7.q.out 1088944
trunk/ql/src/test/results/clientpositive/join8.q.out 1088944
trunk/ql/src/test/results/clientpositive/join9.q.out 1088944
trunk/ql/src/test/results/clientpositive/join_map_ppr.q.out 1088944
trunk/ql/src/test/results/clientpositive/lateral_view_ppd.q.out 1088944
trunk/ql/src/test/results/clientpositive/load_dyn_part10.q.out 1088944
trunk/ql/src/test/results/clientpositive/load_dyn_part13.q.out 1088944
trunk/ql/src/test/results/clientpositive/load_dyn_part2.q.out 1088944
trunk/ql/src/test/results/clientpositive/load_dyn_part3.q.out 1088944
trunk/ql/src/test/results/clientpositive/load_dyn_part4.q.out 1088944
trunk/ql/src/test/results/clientpositive/load_dyn_part9.q.out 1088944
trunk/ql/src/test/results/clientpositive/louter_join_ppr.q.out 1088944
trunk/ql/src/test/results/clientpositive/mapjoin_distinct.q.out 1088944
trunk/ql/src/test/results/clientpositive/mapjoin_subquery.q.out 1088944
trunk/ql/src/test/results/clientpositive/merge3.q.out 1088944
trunk/ql/src/test/results/clientpositive/merge4.q.out 1088944
trunk/ql/src/test/results/clientpositive/merge_dynamic_partition.q.out 1088944
trunk/ql/src/test/results/clientpositive/merge_dynamic_partition2.q.out 1088944
trunk/ql/src/test/results/clientpositive/merge_dynamic_partition3.q.out 1088944
trunk/ql/src/test/results/clientpositive/no_hooks.q.out 1088944
trunk/ql/src/test/results/clientpositive/noalias_subq1.q.out 1088944
trunk/ql/src/test/results/clientpositive/notable_alias1.q.out 1088944
trunk/ql/src/test/results/clientpositive/notable_alias2.q.out 1088944
trunk/ql/src/test/results/clientpositive/nullgroup.q.out 1088944
trunk/ql/src/test/results/clientpositive/nullgroup2.q.out 1088944
trunk/ql/src/test/results/clientpositive/nullgroup4.q.out 1088944
trunk/ql/src/test/results/clientpositive/nullgroup4_multi_distinct.q.out 1088944
trunk/ql/src/test/results/clientpositive/nullgroup5.q.out 1088944
trunk/ql/src/test/results/clientpositive/outer_join_ppr.q.out 1088944
trunk/ql/src/test/results/clientpositive/pcr.q.out 1088944
trunk/ql/src/test/results/clientpositive/ppd1.q.out 1088944
trunk/ql/src/test/results/clientpositive/ppd_clusterby.q.out 1088944
trunk/ql/src/test/results/clientpositive/ppd_constant_expr.q.out 1088944
trunk/ql/src/test/results/clientpositive/ppd_gby.q.out 1088944
trunk/ql/src/test/results/clientpositive/ppd_gby2.q.out 1088944
trunk/ql/src/test/results/clientpositive/ppd_gby_join.q.out 1088944
trunk/ql/src/test/results/clientpositive/ppd_join.q.out 1088944
trunk/ql/src/test/results/clientpositive/ppd_join2.q.out 1088944
trunk/ql/src/test/results/clientpositive/ppd_join3.q.out 1088944
trunk/ql/src/test/results/clientpositive/ppd_multi_insert.q.out 1088944
trunk/ql/src/test/results/clientpositive/ppd_outer_join1.q.out 1088944
trunk/ql/src/test/results/clientpositive/ppd_outer_join2.q.out 1088944
trunk/ql/src/test/results/clientpositive/ppd_outer_join3.q.out 1088944
trunk/ql/src/test/results/clientpositive/ppd_outer_join4.q.out 1088944
trunk/ql/src/test/results/clientpositive/ppd_random.q.out 1088944
trunk/ql/src/test/results/clientpositive/ppd_transform.q.out 1088944
trunk/ql/src/test/results/clientpositive/ppd_udf_case.q.out 1088944
trunk/ql/src/test/results/clientpositive/ppd_union.q.out 1088944
trunk/ql/src/test/results/clientpositive/query_result_fileformat.q.out 1088944
trunk/ql/src/test/results/clientpositive/quote1.q.out 1088944
trunk/ql/src/test/results/clientpositive/rand_partitionpruner3.q.out 1088944
trunk/ql/src/test/results/clientpositive/rcfile_null_value.q.out 1088944
trunk/ql/src/test/results/clientpositive/reduce_deduplicate.q.out 1088944
trunk/ql/src/test/results/clientpositive/regex_col.q.out 1088944
trunk/ql/src/test/results/clientpositive/regexp_extract.q.out 1088944
trunk/ql/src/test/results/clientpositive/router_join_ppr.q.out 1088944
trunk/ql/src/test/results/clientpositive/sample1.q.out 1088944
trunk/ql/src/test/results/clientpositive/sample10.q.out 1088944
trunk/ql/src/test/results/clientpositive/sample2.q.out 1088944
trunk/ql/src/test/results/clientpositive/sample3.q.out 1088944
trunk/ql/src/test/results/clientpositive/sample4.q.out 1088944
trunk/ql/src/test/results/clientpositive/sample5.q.out 1088944
trunk/ql/src/test/results/clientpositive/sample6.q.out 1088944
trunk/ql/src/test/results/clientpositive/sample7.q.out 1088944
trunk/ql/src/test/results/clientpositive/sample8.q.out 1088944
trunk/ql/src/test/results/clientpositive/sample9.q.out 1088944
trunk/ql/src/test/results/clientpositive/semijoin.q.out 1088944
trunk/ql/src/test/results/clientpositive/set_processor_namespaces.q.out 1088944
trunk/ql/src/test/results/clientpositive/skewjoin.q.out 1088944
trunk/ql/src/test/results/clientpositive/smb_mapjoin9.q.out 1088944
trunk/ql/src/test/results/clientpositive/smb_mapjoin_6.q.out 1088944
trunk/ql/src/test/results/clientpositive/stats11.q.out 1088944
trunk/ql/src/test/results/clientpositive/stats2.q.out 1088944
trunk/ql/src/test/results/clientpositive/subq.q.out 1088944
trunk/ql/src/test/results/clientpositive/subq2.q.out 1088944
trunk/ql/src/test/results/clientpositive/transform_ppr1.q.out 1088944
trunk/ql/src/test/results/clientpositive/transform_ppr2.q.out 1088944
trunk/ql/src/test/results/clientpositive/udf1.q.out 1088944
trunk/ql/src/test/results/clientpositive/udf9.q.out 1088944
trunk/ql/src/test/results/clientpositive/udf_10_trims.q.out 1088944
trunk/ql/src/test/results/clientpositive/udf_hour.q.out 1088944
trunk/ql/src/test/results/clientpositive/udf_isnull_isnotnull.q.out 1088944
trunk/ql/src/test/results/clientpositive/udf_like.q.out 1088944
trunk/ql/src/test/results/clientpositive/udf_lower.q.out 1088944
trunk/ql/src/test/results/clientpositive/udf_minute.q.out 1088944
trunk/ql/src/test/results/clientpositive/udf_notequal.q.out 1088944
trunk/ql/src/test/results/clientpositive/udf_parse_url.q.out 1088944
trunk/ql/src/test/results/clientpositive/udf_second.q.out 1088944
trunk/ql/src/test/results/clientpositive/udf_size.q.out 1088944
trunk/ql/src/test/results/clientpositive/union.q.out 1088944
trunk/ql/src/test/results/clientpositive/union20.q.out 1088944
trunk/ql/src/test/results/clientpositive/union22.q.out 1088944
trunk/ql/src/test/results/clientpositive/union_ppr.q.out 1088944
trunk/ql/src/test/results/compiler/plan/case_sensitivity.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/cast1.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/groupby1.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/groupby2.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/groupby3.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/groupby4.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/groupby5.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/groupby6.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/input1.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/input2.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/input20.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/input3.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/input4.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/input5.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/input6.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/input7.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/input8.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/input9.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/input_part1.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/input_testsequencefile.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/input_testxpath.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/input_testxpath2.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/join1.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/join2.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/join3.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/join4.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/join5.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/join6.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/join7.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/join8.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/sample1.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/sample2.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/sample3.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/sample4.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/sample5.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/sample6.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/sample7.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/subq.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/udf1.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/udf4.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/udf6.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/udf_case.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/udf_when.q.xml 1088944
trunk/ql/src/test/results/compiler/plan/union.q.xml 1088944
Diff: https://reviews.apache.org/r/550/diff
Testing
-------
All unit tests passed
Thanks,
Amareshwari
> FilterOperator is applied twice with ppd on.
> --------------------------------------------
>
> Key: HIVE-1538
> URL: https://issues.apache.org/jira/browse/HIVE-1538
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Amareshwari Sriramadasu
> Assignee: Amareshwari Sriramadasu
> Attachments: patch-1538-1.txt, patch-1538-2.txt, patch-1538.txt
>
>
> With hive.optimize.ppd set to true, FilterOperator is applied twice. And it seems second operator is always filtering zero rows.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira