You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Zoltan Haindrich <ki...@rxd.hu> on 2018/08/01 21:29:38 UTC
Re: Review Request 68109: HIVE-20260 NDV of a column shouldn't be
scaled when row count is changed by filter on another column
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68109/
-----------------------------------------------------------
(Updated Aug. 1, 2018, 9:29 p.m.)
Review request for hive and Ashutosh Chauhan.
Changes
-------
patch#01
Bugs: HIVE-20260
https://issues.apache.org/jira/browse/HIVE-20260
Repository: hive-git
Description
-------
* keep track of used column; and only rescale affected columns
* much more conservative than old logic - possible too much...
* wip patch
Diffs (updated)
-----
ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/AnnotateStatsProcCtx.java 47ee949fbc
ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java 3c2b085426
ql/src/test/queries/clientpositive/groupby_groupingset_bug.q c6e9a1a590
ql/src/test/queries/clientpositive/reopt_semijoin.q 0eacb8a68a
ql/src/test/queries/clientpositive/stat_estimate_drill.q PRE-CREATION
ql/src/test/queries/clientpositive/stat_estimate_related_col.q 52da2f759a
ql/src/test/results/clientpositive/annotate_stats_deep_filters.q.out 83bb65ede4
ql/src/test/results/clientpositive/cbo_rp_auto_join1.q.out f937834651
ql/src/test/results/clientpositive/llap/auto_join29.q.out 9db7cb3e79
ql/src/test/results/clientpositive/llap/auto_smb_mapjoin_14.q.out 21273c17f3
ql/src/test/results/clientpositive/llap/auto_sortmerge_join_10.q.out 9e07f1ce2b
ql/src/test/results/clientpositive/llap/auto_sortmerge_join_9.q.out 4ffa7e274a
ql/src/test/results/clientpositive/llap/bucket_map_join_tez2.q.out 800bcaa7b0
ql/src/test/results/clientpositive/llap/bucketsortoptimize_insert_7.q.out c7f5b887b6
ql/src/test/results/clientpositive/llap/constprog_semijoin.q.out 2ccde41e29
ql/src/test/results/clientpositive/llap/correlationoptimizer1.q.out b156ce3938
ql/src/test/results/clientpositive/llap/correlationoptimizer2.q.out eb0dcad7a4
ql/src/test/results/clientpositive/llap/correlationoptimizer6.q.out 6c3e65dea3
ql/src/test/results/clientpositive/llap/dynpart_sort_opt_vectorization.q.out 3b72fc4f28
ql/src/test/results/clientpositive/llap/explainuser_1.q.out 708fa17617
ql/src/test/results/clientpositive/llap/explainuser_2.q.out 71e1f29ff6
ql/src/test/results/clientpositive/llap/explainuser_4.q.out fe330727d4
ql/src/test/results/clientpositive/llap/filter_join_breaktask.q.out 79fb6caee3
ql/src/test/results/clientpositive/llap/groupby_groupingset_bug.q.out 5e18f33f67
ql/src/test/results/clientpositive/llap/hybridgrace_hashjoin_1.q.out 333b10323b
ql/src/test/results/clientpositive/llap/limit_pushdown.q.out 252831eed1
ql/src/test/results/clientpositive/llap/load_dyn_part1.q.out 680f82cf57
ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_3.q.out 2538ac1ba9
ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_rebuild_dummy.q.out 193e95909e
ql/src/test/results/clientpositive/llap/mrr.q.out 314031f61d
ql/src/test/results/clientpositive/llap/multiMapJoin2.q.out 1c6bc941ad
ql/src/test/results/clientpositive/llap/orc_llap.q.out 7f8667e637
ql/src/test/results/clientpositive/llap/orc_predicate_pushdown.q.out e29c3d9915
ql/src/test/results/clientpositive/llap/parquet_predicate_pushdown.q.out eaa7e5dec3
ql/src/test/results/clientpositive/llap/reopt_semijoin.q.out e60b207ba9
ql/src/test/results/clientpositive/llap/sample10.q.out 66214ec11e
ql/src/test/results/clientpositive/llap/sample10_mm.q.out 28d0cd052f
ql/src/test/results/clientpositive/llap/semijoin.q.out eb171453fc
ql/src/test/results/clientpositive/llap/semijoin6.q.out 05a191a997
ql/src/test/results/clientpositive/llap/semijoin7.q.out 31df60a1da
ql/src/test/results/clientpositive/llap/skewjoin.q.out c3f70111cb
ql/src/test/results/clientpositive/llap/smb_mapjoin_14.q.out 3c6ba4e6ce
ql/src/test/results/clientpositive/llap/subquery_exists.q.out e5fcd97cfc
ql/src/test/results/clientpositive/llap/subquery_in.q.out 8007025e5b
ql/src/test/results/clientpositive/llap/subquery_in_having.q.out 20428e1444
ql/src/test/results/clientpositive/llap/subquery_multi.q.out 0c0682def3
ql/src/test/results/clientpositive/llap/subquery_notin.q.out 11facd1857
ql/src/test/results/clientpositive/llap/subquery_scalar.q.out 87681d1477
ql/src/test/results/clientpositive/llap/subquery_select.q.out bd569b3285
ql/src/test/results/clientpositive/llap/subquery_views.q.out 370accb94e
ql/src/test/results/clientpositive/llap/tez_dynpart_hashjoin_2.q.out 55c581fc3c
ql/src/test/results/clientpositive/llap/tez_vector_dynpart_hashjoin_2.q.out 55c581fc3c
ql/src/test/results/clientpositive/llap/unionDistinct_1.q.out 47a0be98cd
ql/src/test/results/clientpositive/llap/unionDistinct_3.q.out 998d497a56
ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets2.q.out f8220e1927
ql/src/test/results/clientpositive/llap/vector_mapjoin_reduce.q.out c31b75919c
ql/src/test/results/clientpositive/llap/vector_windowing_gby.q.out 5dffd1561b
ql/src/test/results/clientpositive/llap/vector_windowing_gby2.q.out 75c393e932
ql/src/test/results/clientpositive/llap/vectorization_0.q.out 7abdc3c9a4
ql/src/test/results/clientpositive/llap/vectorization_limit.q.out f7c00f8153
ql/src/test/results/clientpositive/llap/vectorization_short_regress.q.out 2fc659deca
ql/src/test/results/clientpositive/llap/vectorized_context.q.out 6570f745d9
ql/src/test/results/clientpositive/llap/vectorized_mapjoin.q.out 97e8be6973
ql/src/test/results/clientpositive/llap/vectorized_nested_mapjoin.q.out 27bab206f0
ql/src/test/results/clientpositive/llap/vectorized_shufflejoin.q.out 597c432d77
ql/src/test/results/clientpositive/llap/windowing_gby.q.out eef7562b2f
ql/src/test/results/clientpositive/spark/spark_explainuser_1.q.out c5d0d63f8c
ql/src/test/results/clientpositive/stat_estimate_drill.q.out PRE-CREATION
ql/src/test/results/clientpositive/stat_estimate_related_col.q.out 669adafda3
ql/src/test/results/clientpositive/tez/explainanalyze_1.q.out 3ad2ba26f0
ql/src/test/results/clientpositive/tez/explainanalyze_4.q.out d7d11d8a57
ql/src/test/results/clientpositive/tez/hybridgrace_hashjoin_1.q.out 029f79c308
Diff: https://reviews.apache.org/r/68109/diff/3/
Changes: https://reviews.apache.org/r/68109/diff/2-3/
Testing
-------
Thanks,
Zoltan Haindrich