You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by pengcheng xiong <px...@hortonworks.com> on 2016/04/12 06:31:52 UTC

Re: Review Request 43603: Auto-gather column stats

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43603/
-----------------------------------------------------------

(Updated April 12, 2016, 4:31 a.m.)


Review request for hive and Ashutosh Chauhan.


Repository: hive-git


Description
-------

HIVE-11160: Hive will collect table stats when set hive.stats.autogather=true during the INSERT OVERWRITE command. And then the users need to collect the column stats themselves using "Analyze" command. In this patch, the column stats will also be collected automatically. More specifically, INSERT OVERWRITE will automatically create new column stats. INSERT INTO will automatically merge new column stats with existing ones.


Diffs (updated)
-----

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1702eb1 
  data/conf/hive-site.xml cbb5546 
  itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java 83fb15c 
  itests/src/test/resources/testconfiguration.properties 889884c 
  metastore/if/hive_metastore.thrift 6a55962 
  metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h 8a8f8b1 
  metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp 6e5de20 
  metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp 3e7c6e7 
  metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h 97c07a5 
  metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp 2695ffa 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/SetColumnStatsRequest.java PRE-CREATION 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/SetPartitionsStatsRequest.java 6e334f6 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java 13e30db 
  metastore/src/gen/thrift/gen-php/metastore/ThriftHiveMetastore.php 05a0749 
  metastore/src/gen/thrift/gen-php/metastore/Types.php 488a920 
  metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote 516b926 
  metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py ac8d8a4 
  metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py 10eaf4a 
  metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb 1cf40ae 
  metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb e782bb5 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java c9fadad 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 64a26ac 
  metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 39cf927 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 866e1c3 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/StatsCache.java 18f8afc 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/BinaryColumnStatsAggregator.java d81d612 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/BooleanColumnStatsAggregator.java e796df2 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/ColumnStatsAggregator.java 31955b4 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/ColumnStatsAggregatorFactory.java daf8569 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/DecimalColumnStatsAggregator.java 36b2c9c 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/DoubleColumnStatsAggregator.java a88ef84 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/IExtrapolatePartStatus.java 99af060 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/LongColumnStatsAggregator.java 8ac6561 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/StringColumnStatsAggregator.java 2aa4046 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/aggr/BinaryColumnStatsAggregator.java PRE-CREATION 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/aggr/BooleanColumnStatsAggregator.java PRE-CREATION 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/aggr/ColumnStatsAggregatorFactory.java PRE-CREATION 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/aggr/ColumnStatsMerger.java PRE-CREATION 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/aggr/DecimalColumnStatsAggregator.java PRE-CREATION 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/aggr/DoubleColumnStatsAggregator.java PRE-CREATION 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/aggr/IExtrapolatePartStatus.java PRE-CREATION 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/aggr/LongColumnStatsAggregator.java PRE-CREATION 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/aggr/StringColumnStatsAggregator.java PRE-CREATION 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/BinaryColumnStatsMerger.java PRE-CREATION 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/BooleanColumnStatsMerger.java PRE-CREATION 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/ColumnStatsMerger.java PRE-CREATION 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/ColumnStatsMergerFactory.java PRE-CREATION 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/DecimalColumnStatsMerger.java PRE-CREATION 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/DoubleColumnStatsMerger.java PRE-CREATION 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/LongColumnStatsMerger.java PRE-CREATION 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/StringColumnStatsMerger.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/ColumnStatsTask.java 9059928 
  ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java a5489a9 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java 329c1d5 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java c27481f 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/SessionHiveMetaStoreClient.java db2b674 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 55c71dd 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/PartitionPruner.java 02c5a89 
  ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java b59347d 
  ql/src/java/org/apache/hadoop/hive/ql/parse/ColumnStatsAutoGatherContext.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/parse/ColumnStatsSemanticAnalyzer.java bb1bbad 
  ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java 1bccf20 
  ql/src/java/org/apache/hadoop/hive/ql/parse/QBParseInfo.java 3a226e7 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 987f25d 
  ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java f7d7a40 
  ql/src/java/org/apache/hadoop/hive/ql/plan/ColumnStatsDesc.java c000db2 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBridge.java 7433263 
  ql/src/test/queries/clientpositive/autoColumnStats_1.q PRE-CREATION 
  ql/src/test/queries/clientpositive/autoColumnStats_2.q PRE-CREATION 
  ql/src/test/queries/clientpositive/autoColumnStats_3.q PRE-CREATION 
  ql/src/test/queries/clientpositive/autoColumnStats_4.q PRE-CREATION 
  ql/src/test/queries/clientpositive/autoColumnStats_5.q PRE-CREATION 
  ql/src/test/queries/clientpositive/autoColumnStats_6.q PRE-CREATION 
  ql/src/test/queries/clientpositive/autoColumnStats_7.q PRE-CREATION 
  ql/src/test/queries/clientpositive/combine1.q 3bcb8b1 
  ql/src/test/results/clientpositive/alter_merge_stats_orc.q.out 0d5ba01 
  ql/src/test/results/clientpositive/alter_numbuckets_partitioned_table2_h23.q.out 3b71598 
  ql/src/test/results/clientpositive/alter_numbuckets_partitioned_table_h23.q.out cab3de4 
  ql/src/test/results/clientpositive/alter_partition_coltype.q.out a184372 
  ql/src/test/results/clientpositive/alter_table_serde2.q.out dd946e5 
  ql/src/test/results/clientpositive/analyze_table_null_partition.q.out 72a24d5 
  ql/src/test/results/clientpositive/annotate_stats_filter.q.out ba0419e 
  ql/src/test/results/clientpositive/annotate_stats_groupby.q.out 3070407 
  ql/src/test/results/clientpositive/autoColumnStats_1.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/autoColumnStats_2.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/autoColumnStats_3.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/autoColumnStats_4.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/autoColumnStats_5.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/autoColumnStats_6.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/autoColumnStats_7.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/auto_join1.q.out 7c51b50 
  ql/src/test/results/clientpositive/auto_join14.q.out 4a0f2f9 
  ql/src/test/results/clientpositive/auto_join17.q.out f49d81a 
  ql/src/test/results/clientpositive/auto_join19.q.out 3f70055 
  ql/src/test/results/clientpositive/auto_join2.q.out 26d16ee 
  ql/src/test/results/clientpositive/auto_join25.q.out cc9a348 
  ql/src/test/results/clientpositive/auto_join26.q.out c190621 
  ql/src/test/results/clientpositive/auto_join3.q.out 6c6c79f 
  ql/src/test/results/clientpositive/auto_join4.q.out a4afc7b 
  ql/src/test/results/clientpositive/auto_join5.q.out bbc23dc 
  ql/src/test/results/clientpositive/auto_join6.q.out fa0b4ed 
  ql/src/test/results/clientpositive/auto_join7.q.out 58e6298 
  ql/src/test/results/clientpositive/auto_join8.q.out 324f95d 
  ql/src/test/results/clientpositive/auto_join9.q.out d7d7d18 
  ql/src/test/results/clientpositive/auto_join_reordering_values.q.out db79fa5 
  ql/src/test/results/clientpositive/auto_smb_mapjoin_14.q.out 577f5d7 
  ql/src/test/results/clientpositive/auto_sortmerge_join_13.q.out b45411c 
  ql/src/test/results/clientpositive/auto_sortmerge_join_6.q.out 7cf01e5 
  ql/src/test/results/clientpositive/auto_sortmerge_join_9.q.out 70c820f 
  ql/src/test/results/clientpositive/binary_output_format.q.out 32b2feb 
  ql/src/test/results/clientpositive/bucket1.q.out 0680176 
  ql/src/test/results/clientpositive/bucket2.q.out 958e556 
  ql/src/test/results/clientpositive/bucket3.q.out eca9ba5 
  ql/src/test/results/clientpositive/bucket_map_join_spark1.q.out 3b1912a 
  ql/src/test/results/clientpositive/bucket_map_join_spark2.q.out 9a6aef7 
  ql/src/test/results/clientpositive/bucket_map_join_spark3.q.out 1d2166f 
  ql/src/test/results/clientpositive/bucket_map_join_spark4.q.out 7d32376 
  ql/src/test/results/clientpositive/bucketmapjoin13.q.out c78b7a1 
  ql/src/test/results/clientpositive/bucketmapjoin4.q.out 5858160 
  ql/src/test/results/clientpositive/bucketmapjoin_negative.q.out 36b1cfe 
  ql/src/test/results/clientpositive/bucketmapjoin_negative2.q.out 21138d6 
  ql/src/test/results/clientpositive/bucketsortoptimize_insert_1.q.out 9faa0d0 
  ql/src/test/results/clientpositive/bucketsortoptimize_insert_2.q.out 9761293 
  ql/src/test/results/clientpositive/bucketsortoptimize_insert_3.q.out e778e35 
  ql/src/test/results/clientpositive/bucketsortoptimize_insert_4.q.out 2b61b4c 
  ql/src/test/results/clientpositive/bucketsortoptimize_insert_5.q.out 44466b8 
  ql/src/test/results/clientpositive/bucketsortoptimize_insert_6.q.out fe32f45 
  ql/src/test/results/clientpositive/bucketsortoptimize_insert_7.q.out fa73acf 
  ql/src/test/results/clientpositive/bucketsortoptimize_insert_8.q.out 6c4a077 
  ql/src/test/results/clientpositive/case_sensitivity.q.out b3969cc 
  ql/src/test/results/clientpositive/cast1.q.out 48a0c14 
  ql/src/test/results/clientpositive/columnStatsUpdateForStatsOptimizer_1.q.out 8c9664d 
  ql/src/test/results/clientpositive/columnStatsUpdateForStatsOptimizer_2.q.out 179bc66 
  ql/src/test/results/clientpositive/columnstats_part_coltype.q.out 84232b6 
  ql/src/test/results/clientpositive/combine2.q.out 8c59816 
  ql/src/test/results/clientpositive/constprog_dp.q.out 8cf301d 
  ql/src/test/results/clientpositive/constprog_type.q.out d145d37 
  ql/src/test/results/clientpositive/correlationoptimizer5.q.out f8a53a0 
  ql/src/test/results/clientpositive/cp_sel.q.out f42ec84 
  ql/src/test/results/clientpositive/dynamic_partition_skip_default.q.out e0168b3 
  ql/src/test/results/clientpositive/dynamic_rdd_cache.q.out 9a09c4c 
  ql/src/test/results/clientpositive/dynpart_sort_optimization.q.out 13383fb 
  ql/src/test/results/clientpositive/dynpart_sort_optimization2.q.out 3b24a2e 
  ql/src/test/results/clientpositive/explain_ddl.q.out fa73d99 
  ql/src/test/results/clientpositive/extrapolate_part_stats_full.q.out a30c356 
  ql/src/test/results/clientpositive/extrapolate_part_stats_partial.q.out 4e589b8 
  ql/src/test/results/clientpositive/extrapolate_part_stats_partial_ndv.q.out 3185f70 
  ql/src/test/results/clientpositive/filter_join_breaktask.q.out 018fd5d 
  ql/src/test/results/clientpositive/groupby11.q.out 86568e8 
  ql/src/test/results/clientpositive/groupby12.q.out b17da54 
  ql/src/test/results/clientpositive/groupby1_limit.q.out aacd23c 
  ql/src/test/results/clientpositive/groupby1_map_nomap.q.out 7cdf240 
  ql/src/test/results/clientpositive/groupby1_noskew.q.out ef1fd79 
  ql/src/test/results/clientpositive/groupby2_noskew.q.out d3aff42 
  ql/src/test/results/clientpositive/groupby2_noskew_multi_distinct.q.out 2527fcd 
  ql/src/test/results/clientpositive/groupby3.q.out 4247d28 
  ql/src/test/results/clientpositive/groupby3_map.q.out 405daa9 
  ql/src/test/results/clientpositive/groupby3_map_multi_distinct.q.out a1d24e8 
  ql/src/test/results/clientpositive/groupby3_map_skew.q.out 5679770 
  ql/src/test/results/clientpositive/groupby3_noskew.q.out e6c13cf 
  ql/src/test/results/clientpositive/groupby3_noskew_multi_distinct.q.out f6be869 
  ql/src/test/results/clientpositive/groupby4_map.q.out a8a7fbe 
  ql/src/test/results/clientpositive/groupby4_map_skew.q.out 3f38895 
  ql/src/test/results/clientpositive/groupby4_noskew.q.out d1936da 
  ql/src/test/results/clientpositive/groupby5_map.q.out 5fbd3d7 
  ql/src/test/results/clientpositive/groupby5_map_skew.q.out 60b010b 
  ql/src/test/results/clientpositive/groupby6_noskew.q.out 43c7400 
  ql/src/test/results/clientpositive/groupby_cube_multi_gby.q.out 992fd2d 
  ql/src/test/results/clientpositive/groupby_multi_insert_common_distinct.q.out c78cfe5 
  ql/src/test/results/clientpositive/groupby_multi_single_reducer.q.out 1381d91 
  ql/src/test/results/clientpositive/groupby_multi_single_reducer2.q.out 972ed51 
  ql/src/test/results/clientpositive/groupby_multi_single_reducer3.q.out 5362390 
  ql/src/test/results/clientpositive/groupby_ppr.q.out 2e160b3 
  ql/src/test/results/clientpositive/groupby_ppr_multi_distinct.q.out 93fbc0e 
  ql/src/test/results/clientpositive/implicit_cast_during_insert.q.out c4b9dc4 
  ql/src/test/results/clientpositive/index_auto_update.q.out 73bd4c7 
  ql/src/test/results/clientpositive/infer_bucket_sort.q.out 182dd3a 
  ql/src/test/results/clientpositive/infer_bucket_sort_convert_join.q.out 2f7e538 
  ql/src/test/results/clientpositive/infer_bucket_sort_grouping_operators.q.out ebfce60 
  ql/src/test/results/clientpositive/infer_bucket_sort_list_bucket.q.out f45a719 
  ql/src/test/results/clientpositive/infer_bucket_sort_multi_insert.q.out d4c22f4 
  ql/src/test/results/clientpositive/innerjoin.q.out c106004 
  ql/src/test/results/clientpositive/input11.q.out bb22ee8 
  ql/src/test/results/clientpositive/input11_limit.q.out 92db5a9 
  ql/src/test/results/clientpositive/input12.q.out 1557c58 
  ql/src/test/results/clientpositive/input13.q.out 91ff234 
  ql/src/test/results/clientpositive/input14.q.out f5c5b25 
  ql/src/test/results/clientpositive/input14_limit.q.out 9870ad5 
  ql/src/test/results/clientpositive/input17.q.out 1f7e787 
  ql/src/test/results/clientpositive/input18.q.out 7993cb7 
  ql/src/test/results/clientpositive/input1_limit.q.out 0ceb153 
  ql/src/test/results/clientpositive/input20.q.out cf0ee1d 
  ql/src/test/results/clientpositive/input33.q.out b35e2d0 
  ql/src/test/results/clientpositive/input34.q.out 72f66c3 
  ql/src/test/results/clientpositive/input35.q.out 8b86991 
  ql/src/test/results/clientpositive/input36.q.out 76921ad 
  ql/src/test/results/clientpositive/input38.q.out 0c4e81d 
  ql/src/test/results/clientpositive/input5.q.out a399528 
  ql/src/test/results/clientpositive/input6.q.out 5ed2767 
  ql/src/test/results/clientpositive/input8.q.out 03857fc 
  ql/src/test/results/clientpositive/input9.q.out 38216a6 
  ql/src/test/results/clientpositive/input_part1.q.out 501f7a9 
  ql/src/test/results/clientpositive/input_part10.q.out c8fb37e 
  ql/src/test/results/clientpositive/input_part2.q.out 9eba510 
  ql/src/test/results/clientpositive/input_part5.q.out c6ae2fd 
  ql/src/test/results/clientpositive/input_testsequencefile.q.out 60aaf83 
  ql/src/test/results/clientpositive/input_testxpath.q.out e07628a 
  ql/src/test/results/clientpositive/input_testxpath2.q.out a0baccf 
  ql/src/test/results/clientpositive/insert1.q.out 7a2c429 
  ql/src/test/results/clientpositive/insert_into1.q.out 6bbb86c 
  ql/src/test/results/clientpositive/insert_into2.q.out 1910de6 
  ql/src/test/results/clientpositive/insert_into3.q.out ae7523b 
  ql/src/test/results/clientpositive/insert_into4.q.out 07ff99b 
  ql/src/test/results/clientpositive/insert_into5.q.out b9510b9 
  ql/src/test/results/clientpositive/insert_into6.q.out d93a167 
  ql/src/test/results/clientpositive/insertoverwrite_bucket.q.out bfbe87b 
  ql/src/test/results/clientpositive/join14.q.out 24b5a8e 
  ql/src/test/results/clientpositive/join17.q.out 13260ef 
  ql/src/test/results/clientpositive/join2.q.out 1ad7285 
  ql/src/test/results/clientpositive/join25.q.out af707aa 
  ql/src/test/results/clientpositive/join26.q.out 5533ce4 
  ql/src/test/results/clientpositive/join27.q.out 0c07cb2 
  ql/src/test/results/clientpositive/join28.q.out 9aadd28 
  ql/src/test/results/clientpositive/join29.q.out 29dab0e 
  ql/src/test/results/clientpositive/join3.q.out 3f9a1fb 
  ql/src/test/results/clientpositive/join30.q.out ee9c49b 
  ql/src/test/results/clientpositive/join31.q.out 6466ba2 
  ql/src/test/results/clientpositive/join32.q.out 5cb124b 
  ql/src/test/results/clientpositive/join32_lessSize.q.out 544e814 
  ql/src/test/results/clientpositive/join33.q.out 5cb124b 
  ql/src/test/results/clientpositive/join34.q.out e2c2b1a 
  ql/src/test/results/clientpositive/join35.q.out 96f7cc8 
  ql/src/test/results/clientpositive/join36.q.out 275860a 
  ql/src/test/results/clientpositive/join37.q.out 8820c16 
  ql/src/test/results/clientpositive/join39.q.out 02b1837 
  ql/src/test/results/clientpositive/join4.q.out 82b8568 
  ql/src/test/results/clientpositive/join43.q.out 127d5d0 
  ql/src/test/results/clientpositive/join5.q.out fa9c756 
  ql/src/test/results/clientpositive/join6.q.out eab0fd8 
  ql/src/test/results/clientpositive/join7.q.out 558b6f2 
  ql/src/test/results/clientpositive/join8.q.out d7e7cb1 
  ql/src/test/results/clientpositive/join9.q.out e904b31 
  ql/src/test/results/clientpositive/join_map_ppr.q.out 444fd7f 
  ql/src/test/results/clientpositive/lb_fs_stats.q.out 8344125 
  ql/src/test/results/clientpositive/limit_pushdown_negative.q.out 748a6da 
  ql/src/test/results/clientpositive/lineage1.q.out a655c6c 
  ql/src/test/results/clientpositive/lineage2.q.out a08094a 
  ql/src/test/results/clientpositive/lineage3.q.out 61acf52 
  ql/src/test/results/clientpositive/list_bucket_dml_1.q.out 0cd07ef 
  ql/src/test/results/clientpositive/list_bucket_dml_11.q.java1.7.out f304083 
  ql/src/test/results/clientpositive/list_bucket_dml_12.q.java1.7.out 7d8070a 
  ql/src/test/results/clientpositive/list_bucket_dml_13.q.java1.7.out 81b1431 
  ql/src/test/results/clientpositive/list_bucket_dml_14.q.out 790801f 
  ql/src/test/results/clientpositive/list_bucket_dml_2.q.java1.7.out bfc1e43 
  ql/src/test/results/clientpositive/list_bucket_dml_3.q.out ea24f86 
  ql/src/test/results/clientpositive/list_bucket_dml_4.q.java1.7.out 6b8b89f 
  ql/src/test/results/clientpositive/list_bucket_dml_5.q.java1.7.out 1019474 
  ql/src/test/results/clientpositive/list_bucket_dml_6.q.java1.7.out 668e918 
  ql/src/test/results/clientpositive/list_bucket_dml_7.q.out 3aa9003 
  ql/src/test/results/clientpositive/list_bucket_dml_8.q.java1.7.out 672e5ac 
  ql/src/test/results/clientpositive/list_bucket_dml_9.q.java1.7.out 7179f61 
  ql/src/test/results/clientpositive/list_bucket_query_multiskew_1.q.out 1d8ec8b 
  ql/src/test/results/clientpositive/list_bucket_query_multiskew_2.q.out 83576f1 
  ql/src/test/results/clientpositive/list_bucket_query_multiskew_3.q.out 2e77245 
  ql/src/test/results/clientpositive/llap_acid.q.out 429b042 
  ql/src/test/results/clientpositive/load_dyn_part1.q.out e7dace5 
  ql/src/test/results/clientpositive/load_dyn_part10.q.out ca388b9 
  ql/src/test/results/clientpositive/load_dyn_part13.q.out 9e0ac6f 
  ql/src/test/results/clientpositive/load_dyn_part14.q.out 53e9df3 
  ql/src/test/results/clientpositive/load_dyn_part2.q.out 0c5fe6b 
  ql/src/test/results/clientpositive/load_dyn_part3.q.out 3242c3d 
  ql/src/test/results/clientpositive/load_dyn_part4.q.out d24875f 
  ql/src/test/results/clientpositive/load_dyn_part5.q.out e4bc742 
  ql/src/test/results/clientpositive/load_dyn_part8.q.out 84ccb75 
  ql/src/test/results/clientpositive/load_dyn_part9.q.out 300f41e 
  ql/src/test/results/clientpositive/mapjoin_hook.q.out 8a6743c 
  ql/src/test/results/clientpositive/mapreduce1.q.out 3d0a156 
  ql/src/test/results/clientpositive/mapreduce2.q.out 676c387 
  ql/src/test/results/clientpositive/mapreduce3.q.out fc1a402 
  ql/src/test/results/clientpositive/mapreduce4.q.out 17fa029 
  ql/src/test/results/clientpositive/mapreduce5.q.out 21103f8 
  ql/src/test/results/clientpositive/mapreduce6.q.out fe4e631 
  ql/src/test/results/clientpositive/mapreduce7.q.out cc97887 
  ql/src/test/results/clientpositive/mapreduce8.q.out b1763c7 
  ql/src/test/results/clientpositive/merge1.q.out 94089fc 
  ql/src/test/results/clientpositive/merge2.q.out a3a0e8e 
  ql/src/test/results/clientpositive/merge3.q.out 5b581db 
  ql/src/test/results/clientpositive/merge4.q.out 08e4455 
  ql/src/test/results/clientpositive/merge_dynamic_partition.q.out 07102b1 
  ql/src/test/results/clientpositive/merge_dynamic_partition2.q.out 5a2afb0 
  ql/src/test/results/clientpositive/merge_dynamic_partition3.q.out 86978f3 
  ql/src/test/results/clientpositive/merge_dynamic_partition4.q.out 86af660 
  ql/src/test/results/clientpositive/merge_dynamic_partition5.q.out c1468c1 
  ql/src/test/results/clientpositive/mergejoin.q.out a85fd8b 
  ql/src/test/results/clientpositive/metadata_only_queries.q.out 9bbc9b9 
  ql/src/test/results/clientpositive/multiMapJoin2.q.out a582874 
  ql/src/test/results/clientpositive/multi_insert.q.out 3d26b69 
  ql/src/test/results/clientpositive/multi_insert_gby.q.out 7c5e589 
  ql/src/test/results/clientpositive/multi_insert_gby2.q.out f4baf04 
  ql/src/test/results/clientpositive/multi_insert_gby3.q.out 6ee003b 
  ql/src/test/results/clientpositive/multi_insert_lateral_view.q.out 7964405 
  ql/src/test/results/clientpositive/multi_insert_mixed.q.out aaf9346 
  ql/src/test/results/clientpositive/multi_insert_move_tasks_share_dependencies.q.out c302dea 
  ql/src/test/results/clientpositive/multi_insert_union_src.q.out 2036e63 
  ql/src/test/results/clientpositive/nonreserved_keywords_insert_into1.q.out d01461b 
  ql/src/test/results/clientpositive/notable_alias1.q.out 200b31c 
  ql/src/test/results/clientpositive/notable_alias2.q.out 4df8073 
  ql/src/test/results/clientpositive/orc_analyze.q.out 87855fa 
  ql/src/test/results/clientpositive/orc_llap.q.out 6fc73b7 
  ql/src/test/results/clientpositive/orc_merge2.q.out d4c474f 
  ql/src/test/results/clientpositive/orc_merge3.q.out 7bf12c6 
  ql/src/test/results/clientpositive/orc_merge4.q.out 828f204 
  ql/src/test/results/clientpositive/orc_merge5.q.out 77c01c7 
  ql/src/test/results/clientpositive/orc_merge6.q.out 05f548c 
  ql/src/test/results/clientpositive/orc_merge7.q.out beff0ed 
  ql/src/test/results/clientpositive/orc_merge_incompat1.q.out 7a9c772 
  ql/src/test/results/clientpositive/parallel_join1.q.out 3c97671 
  ql/src/test/results/clientpositive/pcr.q.out 684d4d7 
  ql/src/test/results/clientpositive/pcs.q.out a1382f1 
  ql/src/test/results/clientpositive/pointlookup2.q.out 869e4cd 
  ql/src/test/results/clientpositive/pointlookup3.q.out e98ba76 
  ql/src/test/results/clientpositive/pointlookup4.q.out 6236272 
  ql/src/test/results/clientpositive/ppd_constant_expr.q.out 17e2bab 
  ql/src/test/results/clientpositive/ppd_join5.q.out f464c17 
  ql/src/test/results/clientpositive/ppd_multi_insert.q.out 1a7019c 
  ql/src/test/results/clientpositive/ppd_union_view.q.out 36bb214 
  ql/src/test/results/clientpositive/ptf.q.out cdac02a 
  ql/src/test/results/clientpositive/push_or.q.out bc5c930 
  ql/src/test/results/clientpositive/quote1.q.out f8592c4 
  ql/src/test/results/clientpositive/rand_partitionpruner2.q.out 4559b39 
  ql/src/test/results/clientpositive/rcfile_default_format.q.out c961231 
  ql/src/test/results/clientpositive/rcfile_merge3.q.out 1566ec3 
  ql/src/test/results/clientpositive/rcfile_merge4.q.out 6171434 
  ql/src/test/results/clientpositive/rcfile_null_value.q.out 1a361e1 
  ql/src/test/results/clientpositive/sample10.q.out bf0cb59 
  ql/src/test/results/clientpositive/sample2.q.out e70edd7 
  ql/src/test/results/clientpositive/sample4.q.out 90dfc26 
  ql/src/test/results/clientpositive/sample5.q.out 04aa8cc 
  ql/src/test/results/clientpositive/sample6.q.out c2be9d6 
  ql/src/test/results/clientpositive/sample7.q.out e1897ee 
  ql/src/test/results/clientpositive/smb_mapjoin_11.q.out b407402 
  ql/src/test/results/clientpositive/smb_mapjoin_12.q.out e2723a0 
  ql/src/test/results/clientpositive/smb_mapjoin_13.q.out a5a60e4 
  ql/src/test/results/clientpositive/smb_mapjoin_15.q.out cdf2359 
  ql/src/test/results/clientpositive/smb_mapjoin_18.q.out 4b29056 
  ql/src/test/results/clientpositive/smb_mapjoin_19.q.out 95137cd 
  ql/src/test/results/clientpositive/smb_mapjoin_20.q.out 0324aa4 
  ql/src/test/results/clientpositive/smb_mapjoin_21.q.out 319fef3 
  ql/src/test/results/clientpositive/smb_mapjoin_22.q.out 8f1dbd6 
  ql/src/test/results/clientpositive/smb_mapjoin_6.q.out e22ebc8 
  ql/src/test/results/clientpositive/smb_mapjoin_7.q.out 82f5804 
  ql/src/test/results/clientpositive/sort_merge_join_desc_5.q.out 29dee15 
  ql/src/test/results/clientpositive/sort_merge_join_desc_6.q.out b1f99e5 
  ql/src/test/results/clientpositive/sort_merge_join_desc_7.q.out f788fec 
  ql/src/test/results/clientpositive/special_character_in_tabnames_1.q.out 7374714 
  ql/src/test/results/clientpositive/stats0.q.out 99326fc 
  ql/src/test/results/clientpositive/stats1.q.out 72c53e3 
  ql/src/test/results/clientpositive/stats10.q.out d7519af 
  ql/src/test/results/clientpositive/stats14.q.out e8fe776 
  ql/src/test/results/clientpositive/stats15.q.out 59389db 
  ql/src/test/results/clientpositive/stats18.q.out 6971e44 
  ql/src/test/results/clientpositive/stats4.q.out 2ffbba9 
  ql/src/test/results/clientpositive/stats_empty_dyn_part.q.out 982baab 
  ql/src/test/results/clientpositive/stats_invalidation.q.out d24fdc3 
  ql/src/test/results/clientpositive/stats_list_bucket.q.java1.7.out a4908bc 
  ql/src/test/results/clientpositive/stats_only_null.q.out 032f6c8 
  ql/src/test/results/clientpositive/statsfs.q.out 9d63bf7 
  ql/src/test/results/clientpositive/subquery_multiinsert.q.java1.7.out 279843b 
  ql/src/test/results/clientpositive/tez/autoColumnStats_2.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/truncate_column.q.out adbddfa 
  ql/src/test/results/clientpositive/udf1.q.out b3b694b 
  ql/src/test/results/clientpositive/udf3.q.out 8b2ad31 
  ql/src/test/results/clientpositive/udf_10_trims.q.out 3a5303a 
  ql/src/test/results/clientpositive/udf_length.q.out 0dc39ea 
  ql/src/test/results/clientpositive/udf_reverse.q.out 4fee27f 
  ql/src/test/results/clientpositive/union10.q.out 417ae27 
  ql/src/test/results/clientpositive/union12.q.out ee56f15 
  ql/src/test/results/clientpositive/union22.q.out 5309c71 
  ql/src/test/results/clientpositive/union28.q.out c3789d0 
  ql/src/test/results/clientpositive/union29.q.out 87ba275 
  ql/src/test/results/clientpositive/union30.q.out 26a27c8 
  ql/src/test/results/clientpositive/union31.q.out bb35d5c 
  ql/src/test/results/clientpositive/union33.q.out a91e74c 
  ql/src/test/results/clientpositive/union4.q.out 41ca811 
  ql/src/test/results/clientpositive/union6.q.out 0844165 
  ql/src/test/results/clientpositive/unionDistinct_1.q.out 61bfa74 
  ql/src/test/results/clientpositive/union_fast_stats.q.out f0879af 
  ql/src/test/results/clientpositive/union_lateralview.q.out 7b89e50 
  ql/src/test/results/clientpositive/union_remove_26.q.out e4b156a 
  ql/src/test/results/clientpositive/union_top_level.q.out 134c532 
  ql/src/test/results/clientpositive/vector_auto_smb_mapjoin_14.q.out 9837b26 
  ql/src/test/results/clientpositive/vector_bucket.q.out 7a6a4da 
  ql/src/test/results/clientpositive/vector_char_4.q.out 58988bf 
  ql/src/test/results/clientpositive/vector_char_simple.q.out e1dedae 
  ql/src/test/results/clientpositive/vector_multi_insert.q.out e9f106d 
  ql/src/test/results/clientpositive/vector_varchar_4.q.out f7c9cd0 
  ql/src/test/results/clientpositive/vector_varchar_simple.q.out 49d8b34 
  ql/src/test/results/clientpositive/vectorized_context.q.out 913d07c 
  ql/src/test/results/clientpositive/vectorized_ptf.q.out 3b17591 

Diff: https://reviews.apache.org/r/43603/diff/


Testing
-------


Thanks,

pengcheng xiong


Re: Review Request 43603: Auto-gather column stats

Posted by Ashutosh Chauhan <ha...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43603/#review129807
-----------------------------------------------------------




ql/src/test/results/clientpositive/auto_sortmerge_join_6.q.out (line 517)
<https://reviews.apache.org/r/43603/#comment193377>

    SMBJ became MJ, is this expected?



ql/src/test/results/clientpositive/auto_sortmerge_join_6.q.out 
<https://reviews.apache.org/r/43603/#comment193378>

    Not doing multi-way join?



ql/src/test/results/clientpositive/bucketsortoptimize_insert_2.q.out (line 286)
<https://reviews.apache.org/r/43603/#comment193380>

    MJ converted to SMBJ. expected?



ql/src/test/results/clientpositive/bucketsortoptimize_insert_2.q.out (line 1051)
<https://reviews.apache.org/r/43603/#comment193381>

    expected?



ql/src/test/results/clientpositive/bucketsortoptimize_insert_4.q.out (line 89)
<https://reviews.apache.org/r/43603/#comment193382>

    expected?



ql/src/test/results/clientpositive/bucketsortoptimize_insert_6.q.out (line 1110)
<https://reviews.apache.org/r/43603/#comment193383>

    Expected change? Although test case seems to suggest it is suppose to be SMBJ.



ql/src/test/results/clientpositive/columnStatsUpdateForStatsOptimizer_1.q.out (line 120)
<https://reviews.apache.org/r/43603/#comment193384>

    turn off stats optimizer?



ql/src/test/results/clientpositive/correlationoptimizer5.q.out 
<https://reviews.apache.org/r/43603/#comment193385>

    Demux operator removed from plan.



ql/src/test/results/clientpositive/dynpart_sort_opt_vectorization.q.out (lines 239 - 245)
<https://reviews.apache.org/r/43603/#comment193386>

    Only Column Stats Task. No operator tree for collecting stats. Expected?



ql/src/test/results/clientpositive/dynpart_sort_optimization.q.out (lines 173 - 178)
<https://reviews.apache.org/r/43603/#comment193387>

    No operator pipeline to compute stats, but column stats task is there.



ql/src/test/results/clientpositive/dynpart_sort_optimization2.q.out (lines 828 - 829)
<https://reviews.apache.org/r/43603/#comment193388>

    Change in partitioning of previous stage. Expected?



ql/src/test/results/clientpositive/dynpart_sort_optimization2.q.out (lines 1397 - 1399)
<https://reviews.apache.org/r/43603/#comment193389>

    Partitioning key changed.



ql/src/test/results/clientpositive/groupby1.q.out (lines 25 - 26)
<https://reviews.apache.org/r/43603/#comment193390>

    For this query there should have been 1 stage for stats collection and 1 for stats task. Why are there 3 new stages?



ql/src/test/results/clientpositive/lineage3.q.out (line 361)
<https://reviews.apache.org/r/43603/#comment193406>

    Lineage is broken?



ql/src/test/results/clientpositive/llap_acid.q.out (lines 162 - 163)
<https://reviews.apache.org/r/43603/#comment193407>

    Wrong results?



ql/src/test/results/clientpositive/llap_acid.q.out (lines 275 - 279)
<https://reviews.apache.org/r/43603/#comment193408>

    Wrong results?



ql/src/test/results/clientpositive/metadata_only_queries.q.out (line 186)
<https://reviews.apache.org/r/43603/#comment193409>

    stats optimizer off.



ql/src/test/results/clientpositive/stats_only_null.q.out 
<https://reviews.apache.org/r/43603/#comment193369>

    Need to turn off stats optimizer.



ql/src/test/results/clientpositive/union_remove_26.q.out (line 140)
<https://reviews.apache.org/r/43603/#comment193350>

    Seems like stats optimizer kicked in here. Need to turn it off.



ql/src/test/results/clientpositive/union_top_level.q.out 
<https://reviews.apache.org/r/43603/#comment193351>

    Is this expected: hive.limit.pushdown.memory.usage got turned off?



ql/src/test/results/clientpositive/vector_char_4.q.out (line 164)
<https://reviews.apache.org/r/43603/#comment193352>

    We need to vectorize compute_stats udf. Probably in a follow-up.


- Ashutosh Chauhan


On April 20, 2016, 12:25 a.m., pengcheng xiong wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43603/
> -----------------------------------------------------------
> 
> (Updated April 20, 2016, 12:25 a.m.)
> 
> 
> Review request for hive and Ashutosh Chauhan.
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> HIVE-11160: Hive will collect table stats when set hive.stats.autogather=true during the INSERT OVERWRITE command. And then the users need to collect the column stats themselves using "Analyze" command. In this patch, the column stats will also be collected automatically. More specifically, INSERT OVERWRITE will automatically create new column stats. INSERT INTO will automatically merge new column stats with existing ones.
> 
> 
> Diffs
> -----
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 5cf1609 
>   data/conf/hive-site.xml cbb5546 
>   itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java 83fb15c 
>   itests/src/test/resources/testconfiguration.properties e46e6ce 
>   metastore/if/hive_metastore.thrift 6a55962 
>   metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h 8a8f8b1 
>   metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp 6e5de20 
>   metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp 3e7c6e7 
>   metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h 97c07a5 
>   metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp 2695ffa 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/SetColumnStatsRequest.java PRE-CREATION 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/SetPartitionsStatsRequest.java 6e334f6 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java 13e30db 
>   metastore/src/gen/thrift/gen-php/metastore/ThriftHiveMetastore.php 05a0749 
>   metastore/src/gen/thrift/gen-php/metastore/Types.php 488a920 
>   metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote 516b926 
>   metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py ac8d8a4 
>   metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py 10eaf4a 
>   metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb 1cf40ae 
>   metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb e782bb5 
>   metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java c9fadad 
>   metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java cdd12ab 
>   metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 39cf927 
>   metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 76220f4 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/StatsCache.java 18f8afc 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/BinaryColumnStatsAggregator.java d81d612 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/BooleanColumnStatsAggregator.java e796df2 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/ColumnStatsAggregator.java 31955b4 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/ColumnStatsAggregatorFactory.java daf8569 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/DecimalColumnStatsAggregator.java 36b2c9c 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/DoubleColumnStatsAggregator.java a88ef84 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/IExtrapolatePartStatus.java 99af060 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/LongColumnStatsAggregator.java 8ac6561 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/StringColumnStatsAggregator.java 2aa4046 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/aggr/BinaryColumnStatsAggregator.java PRE-CREATION 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/aggr/BooleanColumnStatsAggregator.java PRE-CREATION 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/aggr/ColumnStatsAggregatorFactory.java PRE-CREATION 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/aggr/ColumnStatsMerger.java PRE-CREATION 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/aggr/DecimalColumnStatsAggregator.java PRE-CREATION 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/aggr/DoubleColumnStatsAggregator.java PRE-CREATION 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/aggr/IExtrapolatePartStatus.java PRE-CREATION 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/aggr/LongColumnStatsAggregator.java PRE-CREATION 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/aggr/StringColumnStatsAggregator.java PRE-CREATION 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/BinaryColumnStatsMerger.java PRE-CREATION 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/BooleanColumnStatsMerger.java PRE-CREATION 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/ColumnStatsMerger.java PRE-CREATION 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/ColumnStatsMergerFactory.java PRE-CREATION 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/DecimalColumnStatsMerger.java PRE-CREATION 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/DoubleColumnStatsMerger.java PRE-CREATION 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/LongColumnStatsMerger.java PRE-CREATION 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/StringColumnStatsMerger.java PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/ColumnStatsTask.java 9059928 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java a5489a9 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java 86025ef 
>   ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 4c9acce 
>   ql/src/java/org/apache/hadoop/hive/ql/metadata/SessionHiveMetaStoreClient.java db2b674 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 55c71dd 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/GenMRSkewJoinProcessor.java 9fbbd4c 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/PartitionPruner.java 02c5a89 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java b59347d 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/ColumnStatsAutoGatherContext.java PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/ColumnStatsSemanticAnalyzer.java bb1bbad 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java 1bccf20 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/QBParseInfo.java 3a226e7 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 96df189 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java f7d7a40 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/ColumnStatsDesc.java c000db2 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBridge.java 7433263 
>   ql/src/test/queries/clientpositive/autoColumnStats_1.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/autoColumnStats_2.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/autoColumnStats_3.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/autoColumnStats_4.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/autoColumnStats_5.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/autoColumnStats_6.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/autoColumnStats_7.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/autoColumnStats_8.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/combine1.q 3bcb8b1 
>   ql/src/test/results/clientpositive/alter_merge_stats_orc.q.out 0d5ba01 
>   ql/src/test/results/clientpositive/alter_numbuckets_partitioned_table2_h23.q.out 3b71598 
>   ql/src/test/results/clientpositive/alter_numbuckets_partitioned_table_h23.q.out cab3de4 
>   ql/src/test/results/clientpositive/alter_partition_coltype.q.out a184372 
>   ql/src/test/results/clientpositive/alter_table_serde2.q.out dd946e5 
>   ql/src/test/results/clientpositive/analyze_table_null_partition.q.out 72a24d5 
>   ql/src/test/results/clientpositive/annotate_stats_filter.q.out ba0419e 
>   ql/src/test/results/clientpositive/annotate_stats_groupby.q.out 3070407 
>   ql/src/test/results/clientpositive/autoColumnStats_1.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/autoColumnStats_2.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/autoColumnStats_3.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/autoColumnStats_4.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/autoColumnStats_5.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/autoColumnStats_6.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/autoColumnStats_7.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/autoColumnStats_8.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/auto_join1.q.out 7c51b50 
>   ql/src/test/results/clientpositive/auto_join14.q.out 4a0f2f9 
>   ql/src/test/results/clientpositive/auto_join17.q.out f49d81a 
>   ql/src/test/results/clientpositive/auto_join19.q.out 3f70055 
>   ql/src/test/results/clientpositive/auto_join19_inclause.q.out 3f70055 
>   ql/src/test/results/clientpositive/auto_join2.q.out 26d16ee 
>   ql/src/test/results/clientpositive/auto_join25.q.out cc9a348 
>   ql/src/test/results/clientpositive/auto_join26.q.out c190621 
>   ql/src/test/results/clientpositive/auto_join3.q.out 6c6c79f 
>   ql/src/test/results/clientpositive/auto_join4.q.out a4afc7b 
>   ql/src/test/results/clientpositive/auto_join5.q.out bbc23dc 
>   ql/src/test/results/clientpositive/auto_join6.q.out fa0b4ed 
>   ql/src/test/results/clientpositive/auto_join7.q.out 58e6298 
>   ql/src/test/results/clientpositive/auto_join8.q.out 324f95d 
>   ql/src/test/results/clientpositive/auto_join9.q.out d7d7d18 
>   ql/src/test/results/clientpositive/auto_join_reordering_values.q.out db79fa5 
>   ql/src/test/results/clientpositive/auto_smb_mapjoin_14.q.out 577f5d7 
>   ql/src/test/results/clientpositive/auto_sortmerge_join_13.q.out b45411c 
>   ql/src/test/results/clientpositive/auto_sortmerge_join_6.q.out 7cf01e5 
>   ql/src/test/results/clientpositive/auto_sortmerge_join_9.q.out 70c820f 
>   ql/src/test/results/clientpositive/binary_output_format.q.out 32b2feb 
>   ql/src/test/results/clientpositive/bucket1.q.out 0680176 
>   ql/src/test/results/clientpositive/bucket2.q.out 958e556 
>   ql/src/test/results/clientpositive/bucket3.q.out eca9ba5 
>   ql/src/test/results/clientpositive/bucket_map_join_spark1.q.out 3b1912a 
>   ql/src/test/results/clientpositive/bucket_map_join_spark2.q.out 9a6aef7 
>   ql/src/test/results/clientpositive/bucket_map_join_spark3.q.out 1d2166f 
>   ql/src/test/results/clientpositive/bucket_map_join_spark4.q.out 7d32376 
>   ql/src/test/results/clientpositive/bucketmapjoin1.q.out ac8a76f 
>   ql/src/test/results/clientpositive/bucketmapjoin13.q.out c78b7a1 
>   ql/src/test/results/clientpositive/bucketmapjoin2.q.out d9c4463 
>   ql/src/test/results/clientpositive/bucketmapjoin3.q.out 77b19ba 
>   ql/src/test/results/clientpositive/bucketmapjoin4.q.out 5858160 
>   ql/src/test/results/clientpositive/bucketmapjoin5.q.out 728c0b7 
>   ql/src/test/results/clientpositive/bucketmapjoin_negative.q.out 36b1cfe 
>   ql/src/test/results/clientpositive/bucketmapjoin_negative2.q.out 21138d6 
>   ql/src/test/results/clientpositive/bucketsortoptimize_insert_1.q.out 9faa0d0 
>   ql/src/test/results/clientpositive/bucketsortoptimize_insert_2.q.out 9761293 
>   ql/src/test/results/clientpositive/bucketsortoptimize_insert_3.q.out e778e35 
>   ql/src/test/results/clientpositive/bucketsortoptimize_insert_4.q.out 2b61b4c 
>   ql/src/test/results/clientpositive/bucketsortoptimize_insert_5.q.out 44466b8 
>   ql/src/test/results/clientpositive/bucketsortoptimize_insert_6.q.out fe32f45 
>   ql/src/test/results/clientpositive/bucketsortoptimize_insert_7.q.out fa73acf 
>   ql/src/test/results/clientpositive/bucketsortoptimize_insert_8.q.out 6c4a077 
>   ql/src/test/results/clientpositive/case_sensitivity.q.out b3969cc 
>   ql/src/test/results/clientpositive/cast1.q.out 48a0c14 
>   ql/src/test/results/clientpositive/columnStatsUpdateForStatsOptimizer_1.q.out 8c9664d 
>   ql/src/test/results/clientpositive/columnStatsUpdateForStatsOptimizer_2.q.out 179bc66 
>   ql/src/test/results/clientpositive/columnstats_part_coltype.q.out 84232b6 
>   ql/src/test/results/clientpositive/combine2.q.out 8c59816 
>   ql/src/test/results/clientpositive/constprog_dp.q.out 8cf301d 
>   ql/src/test/results/clientpositive/constprog_type.q.out d145d37 
>   ql/src/test/results/clientpositive/correlationoptimizer5.q.out f8a53a0 
>   ql/src/test/results/clientpositive/cp_sel.q.out f42ec84 
>   ql/src/test/results/clientpositive/dynamic_partition_skip_default.q.out e0168b3 
>   ql/src/test/results/clientpositive/dynamic_rdd_cache.q.out 9a09c4c 
>   ql/src/test/results/clientpositive/dynpart_sort_opt_vectorization.q.out d03bfe4 
>   ql/src/test/results/clientpositive/dynpart_sort_optimization.q.out 13383fb 
>   ql/src/test/results/clientpositive/dynpart_sort_optimization2.q.out 3b24a2e 
>   ql/src/test/results/clientpositive/explain_ddl.q.out fa73d99 
>   ql/src/test/results/clientpositive/extrapolate_part_stats_full.q.out a30c356 
>   ql/src/test/results/clientpositive/extrapolate_part_stats_partial.q.out 4e589b8 
>   ql/src/test/results/clientpositive/extrapolate_part_stats_partial_ndv.q.out 3185f70 
>   ql/src/test/results/clientpositive/filter_join_breaktask.q.out 018fd5d 
>   ql/src/test/results/clientpositive/groupby1.q.out 785c078 
>   ql/src/test/results/clientpositive/groupby10.q.out 5297107 
>   ql/src/test/results/clientpositive/groupby11.q.out 86568e8 
>   ql/src/test/results/clientpositive/groupby12.q.out b17da54 
>   ql/src/test/results/clientpositive/groupby1_limit.q.out aacd23c 
>   ql/src/test/results/clientpositive/groupby1_map.q.out 7cdf240 
>   ql/src/test/results/clientpositive/groupby1_map_nomap.q.out 7cdf240 
>   ql/src/test/results/clientpositive/groupby1_map_skew.q.out be7eeca 
>   ql/src/test/results/clientpositive/groupby1_noskew.q.out ef1fd79 
>   ql/src/test/results/clientpositive/groupby2_map.q.out c157ba7 
>   ql/src/test/results/clientpositive/groupby2_map_multi_distinct.q.out afbcb7f 
>   ql/src/test/results/clientpositive/groupby2_map_skew.q.out 813ae5c 
>   ql/src/test/results/clientpositive/groupby2_noskew.q.out d3aff42 
>   ql/src/test/results/clientpositive/groupby2_noskew_multi_distinct.q.out 2527fcd 
>   ql/src/test/results/clientpositive/groupby3.q.out 4247d28 
>   ql/src/test/results/clientpositive/groupby3_map.q.out 405daa9 
>   ql/src/test/results/clientpositive/groupby3_map_multi_distinct.q.out a1d24e8 
>   ql/src/test/results/clientpositive/groupby3_map_skew.q.out 5679770 
>   ql/src/test/results/clientpositive/groupby3_noskew.q.out e6c13cf 
>   ql/src/test/results/clientpositive/groupby3_noskew_multi_distinct.q.out f6be869 
>   ql/src/test/results/clientpositive/groupby4.q.out 55313fa 
>   ql/src/test/results/clientpositive/groupby4_map.q.out a8a7fbe 
>   ql/src/test/results/clientpositive/groupby4_map_skew.q.out 3f38895 
>   ql/src/test/results/clientpositive/groupby4_noskew.q.out d1936da 
>   ql/src/test/results/clientpositive/groupby5_map.q.out 5fbd3d7 
>   ql/src/test/results/clientpositive/groupby5_map_skew.q.out 60b010b 
>   ql/src/test/results/clientpositive/groupby6.q.out d8cb2ac 
>   ql/src/test/results/clientpositive/groupby6_map.q.out b307eba 
>   ql/src/test/results/clientpositive/groupby6_map_skew.q.out c5af47d 
>   ql/src/test/results/clientpositive/groupby6_noskew.q.out 43c7400 
>   ql/src/test/results/clientpositive/groupby_cube_multi_gby.q.out 992fd2d 
>   ql/src/test/results/clientpositive/groupby_map_ppr.q.out 556995c 
>   ql/src/test/results/clientpositive/groupby_map_ppr_multi_distinct.q.out 393e6fa 
>   ql/src/test/results/clientpositive/groupby_multi_insert_common_distinct.q.out c78cfe5 
>   ql/src/test/results/clientpositive/groupby_multi_single_reducer.q.out 1381d91 
>   ql/src/test/results/clientpositive/groupby_multi_single_reducer2.q.out 972ed51 
>   ql/src/test/results/clientpositive/groupby_multi_single_reducer3.q.out c5488de 
>   ql/src/test/results/clientpositive/groupby_ppr.q.out 2e160b3 
>   ql/src/test/results/clientpositive/groupby_ppr_multi_distinct.q.out 93fbc0e 
>   ql/src/test/results/clientpositive/implicit_cast_during_insert.q.out c4b9dc4 
>   ql/src/test/results/clientpositive/index_auto_update.q.out 73bd4c7 
>   ql/src/test/results/clientpositive/infer_bucket_sort.q.out 182dd3a 
>   ql/src/test/results/clientpositive/infer_bucket_sort_convert_join.q.out 2f7e538 
>   ql/src/test/results/clientpositive/infer_bucket_sort_grouping_operators.q.out ebfce60 
>   ql/src/test/results/clientpositive/infer_bucket_sort_list_bucket.q.out f45a719 
>   ql/src/test/results/clientpositive/infer_bucket_sort_multi_insert.q.out d4c22f4 
>   ql/src/test/results/clientpositive/innerjoin.q.out c106004 
>   ql/src/test/results/clientpositive/input11.q.out bb22ee8 
>   ql/src/test/results/clientpositive/input11_limit.q.out 92db5a9 
>   ql/src/test/results/clientpositive/input12.q.out 1557c58 
>   ql/src/test/results/clientpositive/input13.q.out 91ff234 
>   ql/src/test/results/clientpositive/input14.q.out f5c5b25 
>   ql/src/test/results/clientpositive/input14_limit.q.out 9870ad5 
>   ql/src/test/results/clientpositive/input17.q.out 1f7e787 
>   ql/src/test/results/clientpositive/input18.q.out 7993cb7 
>   ql/src/test/results/clientpositive/input1_limit.q.out 0ceb153 
>   ql/src/test/results/clientpositive/input20.q.out cf0ee1d 
>   ql/src/test/results/clientpositive/input33.q.out b35e2d0 
>   ql/src/test/results/clientpositive/input34.q.out 72f66c3 
>   ql/src/test/results/clientpositive/input35.q.out 8b86991 
>   ql/src/test/results/clientpositive/input36.q.out 76921ad 
>   ql/src/test/results/clientpositive/input38.q.out 0c4e81d 
>   ql/src/test/results/clientpositive/input5.q.out a399528 
>   ql/src/test/results/clientpositive/input6.q.out 5ed2767 
>   ql/src/test/results/clientpositive/input7.q.out 0545b1f 
>   ql/src/test/results/clientpositive/input8.q.out 03857fc 
>   ql/src/test/results/clientpositive/input9.q.out 38216a6 
>   ql/src/test/results/clientpositive/input_part1.q.out 501f7a9 
>   ql/src/test/results/clientpositive/input_part10.q.out c8fb37e 
>   ql/src/test/results/clientpositive/input_part2.q.out 9eba510 
>   ql/src/test/results/clientpositive/input_part5.q.out c6ae2fd 
>   ql/src/test/results/clientpositive/input_testsequencefile.q.out 60aaf83 
>   ql/src/test/results/clientpositive/input_testxpath.q.out e07628a 
>   ql/src/test/results/clientpositive/input_testxpath2.q.out a0baccf 
>   ql/src/test/results/clientpositive/insert1.q.out 7a2c429 
>   ql/src/test/results/clientpositive/insert_into1.q.out 6bbb86c 
>   ql/src/test/results/clientpositive/insert_into2.q.out 1910de6 
>   ql/src/test/results/clientpositive/insert_into3.q.out ae7523b 
>   ql/src/test/results/clientpositive/insert_into4.q.out 07ff99b 
>   ql/src/test/results/clientpositive/insert_into5.q.out b9510b9 
>   ql/src/test/results/clientpositive/insert_into6.q.out d93a167 
>   ql/src/test/results/clientpositive/insertoverwrite_bucket.q.out bfbe87b 
>   ql/src/test/results/clientpositive/join14.q.out 24b5a8e 
>   ql/src/test/results/clientpositive/join17.q.out 13260ef 
>   ql/src/test/results/clientpositive/join2.q.out 1ad7285 
>   ql/src/test/results/clientpositive/join25.q.out af707aa 
>   ql/src/test/results/clientpositive/join26.q.out 5533ce4 
>   ql/src/test/results/clientpositive/join27.q.out 0c07cb2 
>   ql/src/test/results/clientpositive/join28.q.out 9aadd28 
>   ql/src/test/results/clientpositive/join29.q.out 29dab0e 
>   ql/src/test/results/clientpositive/join3.q.out 3f9a1fb 
>   ql/src/test/results/clientpositive/join30.q.out ee9c49b 
>   ql/src/test/results/clientpositive/join31.q.out 6466ba2 
>   ql/src/test/results/clientpositive/join32.q.out 5cb124b 
>   ql/src/test/results/clientpositive/join32_lessSize.q.out 544e814 
>   ql/src/test/results/clientpositive/join33.q.out 5cb124b 
>   ql/src/test/results/clientpositive/join34.q.out e2c2b1a 
>   ql/src/test/results/clientpositive/join35.q.out 96f7cc8 
>   ql/src/test/results/clientpositive/join36.q.out 275860a 
>   ql/src/test/results/clientpositive/join37.q.out 8820c16 
>   ql/src/test/results/clientpositive/join39.q.out 02b1837 
>   ql/src/test/results/clientpositive/join4.q.out 82b8568 
>   ql/src/test/results/clientpositive/join43.q.out 127d5d0 
>   ql/src/test/results/clientpositive/join5.q.out fa9c756 
>   ql/src/test/results/clientpositive/join6.q.out eab0fd8 
>   ql/src/test/results/clientpositive/join7.q.out 558b6f2 
>   ql/src/test/results/clientpositive/join8.q.out d7e7cb1 
>   ql/src/test/results/clientpositive/join9.q.out e904b31 
>   ql/src/test/results/clientpositive/join_map_ppr.q.out 444fd7f 
>   ql/src/test/results/clientpositive/lb_fs_stats.q.out 8344125 
>   ql/src/test/results/clientpositive/limit_pushdown_negative.q.out 748a6da 
>   ql/src/test/results/clientpositive/lineage1.q.out a655c6c 
>   ql/src/test/results/clientpositive/lineage2.q.out a08094a 
>   ql/src/test/results/clientpositive/lineage3.q.out 61acf52 
>   ql/src/test/results/clientpositive/list_bucket_dml_1.q.out 0cd07ef 
>   ql/src/test/results/clientpositive/list_bucket_dml_11.q.java1.7.out f304083 
>   ql/src/test/results/clientpositive/list_bucket_dml_12.q.java1.7.out 7d8070a 
>   ql/src/test/results/clientpositive/list_bucket_dml_13.q.java1.7.out 81b1431 
>   ql/src/test/results/clientpositive/list_bucket_dml_14.q.out 790801f 
>   ql/src/test/results/clientpositive/list_bucket_dml_2.q.java1.7.out bfc1e43 
>   ql/src/test/results/clientpositive/list_bucket_dml_3.q.out ea24f86 
>   ql/src/test/results/clientpositive/list_bucket_dml_4.q.java1.7.out 6b8b89f 
>   ql/src/test/results/clientpositive/list_bucket_dml_5.q.java1.7.out 1019474 
>   ql/src/test/results/clientpositive/list_bucket_dml_6.q.java1.7.out 668e918 
>   ql/src/test/results/clientpositive/list_bucket_dml_7.q.out 3aa9003 
>   ql/src/test/results/clientpositive/list_bucket_dml_8.q.java1.7.out 672e5ac 
>   ql/src/test/results/clientpositive/list_bucket_dml_9.q.java1.7.out 7179f61 
>   ql/src/test/results/clientpositive/list_bucket_query_multiskew_1.q.out 1d8ec8b 
>   ql/src/test/results/clientpositive/list_bucket_query_multiskew_2.q.out 83576f1 
>   ql/src/test/results/clientpositive/list_bucket_query_multiskew_3.q.out 2e77245 
>   ql/src/test/results/clientpositive/llap_acid.q.out 429b042 
>   ql/src/test/results/clientpositive/load_dyn_part1.q.out e7dace5 
>   ql/src/test/results/clientpositive/load_dyn_part10.q.out ca388b9 
>   ql/src/test/results/clientpositive/load_dyn_part13.q.out 9e0ac6f 
>   ql/src/test/results/clientpositive/load_dyn_part14.q.out 53e9df3 
>   ql/src/test/results/clientpositive/load_dyn_part2.q.out 0c5fe6b 
>   ql/src/test/results/clientpositive/load_dyn_part3.q.out 3242c3d 
>   ql/src/test/results/clientpositive/load_dyn_part4.q.out d24875f 
>   ql/src/test/results/clientpositive/load_dyn_part5.q.out e4bc742 
>   ql/src/test/results/clientpositive/load_dyn_part8.q.out 84ccb75 
>   ql/src/test/results/clientpositive/load_dyn_part9.q.out 300f41e 
>   ql/src/test/results/clientpositive/mapjoin_hook.q.out 8a6743c 
>   ql/src/test/results/clientpositive/mapreduce1.q.out 3d0a156 
>   ql/src/test/results/clientpositive/mapreduce2.q.out 676c387 
>   ql/src/test/results/clientpositive/mapreduce3.q.out fc1a402 
>   ql/src/test/results/clientpositive/mapreduce4.q.out 17fa029 
>   ql/src/test/results/clientpositive/mapreduce5.q.out 21103f8 
>   ql/src/test/results/clientpositive/mapreduce6.q.out fe4e631 
>   ql/src/test/results/clientpositive/mapreduce7.q.out cc97887 
>   ql/src/test/results/clientpositive/mapreduce8.q.out b1763c7 
>   ql/src/test/results/clientpositive/merge1.q.out 94089fc 
>   ql/src/test/results/clientpositive/merge2.q.out a3a0e8e 
>   ql/src/test/results/clientpositive/merge3.q.out 5b581db 
>   ql/src/test/results/clientpositive/merge4.q.out 08e4455 
>   ql/src/test/results/clientpositive/merge_dynamic_partition.q.out 07102b1 
>   ql/src/test/results/clientpositive/merge_dynamic_partition2.q.out 5a2afb0 
>   ql/src/test/results/clientpositive/merge_dynamic_partition3.q.out 86978f3 
>   ql/src/test/results/clientpositive/merge_dynamic_partition4.q.out 86af660 
>   ql/src/test/results/clientpositive/merge_dynamic_partition5.q.out c1468c1 
>   ql/src/test/results/clientpositive/mergejoin.q.out a85fd8b 
>   ql/src/test/results/clientpositive/metadata_only_queries.q.out 9bbc9b9 
>   ql/src/test/results/clientpositive/multiMapJoin2.q.out a582874 
>   ql/src/test/results/clientpositive/multi_insert.q.out 3d26b69 
>   ql/src/test/results/clientpositive/multi_insert_gby.q.out 7c5e589 
>   ql/src/test/results/clientpositive/multi_insert_gby2.q.out f4baf04 
>   ql/src/test/results/clientpositive/multi_insert_gby3.q.out 6ee003b 
>   ql/src/test/results/clientpositive/multi_insert_lateral_view.q.out 7964405 
>   ql/src/test/results/clientpositive/multi_insert_mixed.q.out aaf9346 
>   ql/src/test/results/clientpositive/multi_insert_move_tasks_share_dependencies.q.out c302dea 
>   ql/src/test/results/clientpositive/multi_insert_union_src.q.out 2036e63 
>   ql/src/test/results/clientpositive/nonreserved_keywords_insert_into1.q.out d01461b 
>   ql/src/test/results/clientpositive/notable_alias1.q.out 200b31c 
>   ql/src/test/results/clientpositive/notable_alias2.q.out 4df8073 
>   ql/src/test/results/clientpositive/orc_analyze.q.out 87855fa 
>   ql/src/test/results/clientpositive/orc_llap.q.out 6fc73b7 
>   ql/src/test/results/clientpositive/orc_merge1.q.out 3f047da 
>   ql/src/test/results/clientpositive/orc_merge10.q.out cf70dcf 
>   ql/src/test/results/clientpositive/orc_merge2.q.out d4c474f 
>   ql/src/test/results/clientpositive/orc_merge3.q.out 7bf12c6 
>   ql/src/test/results/clientpositive/orc_merge4.q.out 828f204 
>   ql/src/test/results/clientpositive/orc_merge5.q.out 77c01c7 
>   ql/src/test/results/clientpositive/orc_merge6.q.out 05f548c 
>   ql/src/test/results/clientpositive/orc_merge7.q.out beff0ed 
>   ql/src/test/results/clientpositive/orc_merge_incompat1.q.out 7a9c772 
>   ql/src/test/results/clientpositive/parallel_join1.q.out 3c97671 
>   ql/src/test/results/clientpositive/pcr.q.out 684d4d7 
>   ql/src/test/results/clientpositive/pcs.q.out d6d2431 
>   ql/src/test/results/clientpositive/pointlookup2.q.out 6fc6e7f 
>   ql/src/test/results/clientpositive/pointlookup3.q.out 2b25b39 
>   ql/src/test/results/clientpositive/pointlookup4.q.out 6236272 
>   ql/src/test/results/clientpositive/ppd_constant_expr.q.out 17e2bab 
>   ql/src/test/results/clientpositive/ppd_join5.q.out f464c17 
>   ql/src/test/results/clientpositive/ppd_multi_insert.q.out 1a7019c 
>   ql/src/test/results/clientpositive/ppd_union_view.q.out 36bb214 
>   ql/src/test/results/clientpositive/ptf.q.out cdac02a 
>   ql/src/test/results/clientpositive/push_or.q.out bc5c930 
>   ql/src/test/results/clientpositive/quote1.q.out f8592c4 
>   ql/src/test/results/clientpositive/rand_partitionpruner2.q.out 4559b39 
>   ql/src/test/results/clientpositive/rcfile_default_format.q.out c961231 
>   ql/src/test/results/clientpositive/rcfile_merge2.q.out b2f61cf 
>   ql/src/test/results/clientpositive/rcfile_merge3.q.out 1566ec3 
>   ql/src/test/results/clientpositive/rcfile_merge4.q.out 6171434 
>   ql/src/test/results/clientpositive/rcfile_null_value.q.out 1a361e1 
>   ql/src/test/results/clientpositive/sample1.q.out caf62f4 
>   ql/src/test/results/clientpositive/sample10.q.out bf0cb59 
>   ql/src/test/results/clientpositive/sample2.q.out e70edd7 
>   ql/src/test/results/clientpositive/sample4.q.out 90dfc26 
>   ql/src/test/results/clientpositive/sample5.q.out 04aa8cc 
>   ql/src/test/results/clientpositive/sample6.q.out c2be9d6 
>   ql/src/test/results/clientpositive/sample7.q.out e1897ee 
>   ql/src/test/results/clientpositive/schema_evol_stats.q.out 63dab2e 
>   ql/src/test/results/clientpositive/skewjoin.q.out bd954ef 
>   ql/src/test/results/clientpositive/smb_mapjoin_11.q.out b407402 
>   ql/src/test/results/clientpositive/smb_mapjoin_12.q.out e2723a0 
>   ql/src/test/results/clientpositive/smb_mapjoin_13.q.out a5a60e4 
>   ql/src/test/results/clientpositive/smb_mapjoin_15.q.out cdf2359 
>   ql/src/test/results/clientpositive/smb_mapjoin_18.q.out 4b29056 
>   ql/src/test/results/clientpositive/smb_mapjoin_19.q.out 95137cd 
>   ql/src/test/results/clientpositive/smb_mapjoin_20.q.out 0324aa4 
>   ql/src/test/results/clientpositive/smb_mapjoin_21.q.out 319fef3 
>   ql/src/test/results/clientpositive/smb_mapjoin_22.q.out 8f1dbd6 
>   ql/src/test/results/clientpositive/smb_mapjoin_6.q.out e22ebc8 
>   ql/src/test/results/clientpositive/smb_mapjoin_7.q.out 82f5804 
>   ql/src/test/results/clientpositive/sort_merge_join_desc_5.q.out 29dee15 
>   ql/src/test/results/clientpositive/sort_merge_join_desc_6.q.out b1f99e5 
>   ql/src/test/results/clientpositive/sort_merge_join_desc_7.q.out f788fec 
>   ql/src/test/results/clientpositive/special_character_in_tabnames_1.q.out 7374714 
>   ql/src/test/results/clientpositive/stats0.q.out 99326fc 
>   ql/src/test/results/clientpositive/stats1.q.out 72c53e3 
>   ql/src/test/results/clientpositive/stats10.q.out d7519af 
>   ql/src/test/results/clientpositive/stats11.q.out 7598dc0 
>   ql/src/test/results/clientpositive/stats14.q.out e8fe776 
>   ql/src/test/results/clientpositive/stats15.q.out 59389db 
>   ql/src/test/results/clientpositive/stats18.q.out 6971e44 
>   ql/src/test/results/clientpositive/stats4.q.out 2ffbba9 
>   ql/src/test/results/clientpositive/stats_empty_dyn_part.q.out 982baab 
>   ql/src/test/results/clientpositive/stats_invalidation.q.out d24fdc3 
>   ql/src/test/results/clientpositive/stats_list_bucket.q.java1.7.out a4908bc 
>   ql/src/test/results/clientpositive/stats_only_null.q.out 032f6c8 
>   ql/src/test/results/clientpositive/statsfs.q.out 9d63bf7 
>   ql/src/test/results/clientpositive/subquery_multiinsert.q.java1.7.out 279843b 
>   ql/src/test/results/clientpositive/tez/autoColumnStats_2.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/truncate_column.q.out adbddfa 
>   ql/src/test/results/clientpositive/udf1.q.out b3b694b 
>   ql/src/test/results/clientpositive/udf3.q.out 8b2ad31 
>   ql/src/test/results/clientpositive/udf_10_trims.q.out 3a5303a 
>   ql/src/test/results/clientpositive/udf_length.q.out 0dc39ea 
>   ql/src/test/results/clientpositive/udf_reverse.q.out 4fee27f 
>   ql/src/test/results/clientpositive/union10.q.out 417ae27 
>   ql/src/test/results/clientpositive/union12.q.out ee56f15 
>   ql/src/test/results/clientpositive/union22.q.out 5309c71 
>   ql/src/test/results/clientpositive/union28.q.out c3789d0 
>   ql/src/test/results/clientpositive/union29.q.out 87ba275 
>   ql/src/test/results/clientpositive/union30.q.out 26a27c8 
>   ql/src/test/results/clientpositive/union31.q.out bb35d5c 
>   ql/src/test/results/clientpositive/union33.q.out a91e74c 
>   ql/src/test/results/clientpositive/union4.q.out 41ca811 
>   ql/src/test/results/clientpositive/union6.q.out 0844165 
>   ql/src/test/results/clientpositive/unionDistinct_1.q.out 61bfa74 
>   ql/src/test/results/clientpositive/union_fast_stats.q.out f0879af 
>   ql/src/test/results/clientpositive/union_lateralview.q.out 7b89e50 
>   ql/src/test/results/clientpositive/union_remove_26.q.out e4b156a 
>   ql/src/test/results/clientpositive/union_top_level.q.out 134c532 
>   ql/src/test/results/clientpositive/vector_auto_smb_mapjoin_14.q.out 9837b26 
>   ql/src/test/results/clientpositive/vector_bucket.q.out 7a6a4da 
>   ql/src/test/results/clientpositive/vector_char_4.q.out 58988bf 
>   ql/src/test/results/clientpositive/vector_char_simple.q.out e1dedae 
>   ql/src/test/results/clientpositive/vector_multi_insert.q.out e9f106d 
>   ql/src/test/results/clientpositive/vector_tablesample_rows.q.out 25f2996 
>   ql/src/test/results/clientpositive/vector_varchar_4.q.out f7c9cd0 
>   ql/src/test/results/clientpositive/vector_varchar_simple.q.out 49d8b34 
>   ql/src/test/results/clientpositive/vectorized_context.q.out 913d07c 
>   ql/src/test/results/clientpositive/vectorized_ptf.q.out 3b17591 
> 
> Diff: https://reviews.apache.org/r/43603/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> pengcheng xiong
> 
>


Re: Review Request 43603: Auto-gather column stats

Posted by pengcheng xiong <px...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43603/
-----------------------------------------------------------

(Updated April 20, 2016, 12:25 a.m.)


Review request for hive and Ashutosh Chauhan.


Repository: hive-git


Description
-------

HIVE-11160: Hive will collect table stats when set hive.stats.autogather=true during the INSERT OVERWRITE command. And then the users need to collect the column stats themselves using "Analyze" command. In this patch, the column stats will also be collected automatically. More specifically, INSERT OVERWRITE will automatically create new column stats. INSERT INTO will automatically merge new column stats with existing ones.


Diffs (updated)
-----

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 5cf1609 
  data/conf/hive-site.xml cbb5546 
  itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java 83fb15c 
  itests/src/test/resources/testconfiguration.properties e46e6ce 
  metastore/if/hive_metastore.thrift 6a55962 
  metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h 8a8f8b1 
  metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp 6e5de20 
  metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp 3e7c6e7 
  metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h 97c07a5 
  metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp 2695ffa 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/SetColumnStatsRequest.java PRE-CREATION 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/SetPartitionsStatsRequest.java 6e334f6 
  metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java 13e30db 
  metastore/src/gen/thrift/gen-php/metastore/ThriftHiveMetastore.php 05a0749 
  metastore/src/gen/thrift/gen-php/metastore/Types.php 488a920 
  metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote 516b926 
  metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py ac8d8a4 
  metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py 10eaf4a 
  metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb 1cf40ae 
  metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb e782bb5 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java c9fadad 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java cdd12ab 
  metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 39cf927 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 76220f4 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/StatsCache.java 18f8afc 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/BinaryColumnStatsAggregator.java d81d612 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/BooleanColumnStatsAggregator.java e796df2 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/ColumnStatsAggregator.java 31955b4 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/ColumnStatsAggregatorFactory.java daf8569 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/DecimalColumnStatsAggregator.java 36b2c9c 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/DoubleColumnStatsAggregator.java a88ef84 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/IExtrapolatePartStatus.java 99af060 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/LongColumnStatsAggregator.java 8ac6561 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/StringColumnStatsAggregator.java 2aa4046 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/aggr/BinaryColumnStatsAggregator.java PRE-CREATION 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/aggr/BooleanColumnStatsAggregator.java PRE-CREATION 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/aggr/ColumnStatsAggregatorFactory.java PRE-CREATION 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/aggr/ColumnStatsMerger.java PRE-CREATION 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/aggr/DecimalColumnStatsAggregator.java PRE-CREATION 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/aggr/DoubleColumnStatsAggregator.java PRE-CREATION 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/aggr/IExtrapolatePartStatus.java PRE-CREATION 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/aggr/LongColumnStatsAggregator.java PRE-CREATION 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/aggr/StringColumnStatsAggregator.java PRE-CREATION 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/BinaryColumnStatsMerger.java PRE-CREATION 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/BooleanColumnStatsMerger.java PRE-CREATION 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/ColumnStatsMerger.java PRE-CREATION 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/ColumnStatsMergerFactory.java PRE-CREATION 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/DecimalColumnStatsMerger.java PRE-CREATION 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/DoubleColumnStatsMerger.java PRE-CREATION 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/LongColumnStatsMerger.java PRE-CREATION 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/StringColumnStatsMerger.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/ColumnStatsTask.java 9059928 
  ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java a5489a9 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java 86025ef 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 4c9acce 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/SessionHiveMetaStoreClient.java db2b674 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 55c71dd 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/GenMRSkewJoinProcessor.java 9fbbd4c 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/PartitionPruner.java 02c5a89 
  ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java b59347d 
  ql/src/java/org/apache/hadoop/hive/ql/parse/ColumnStatsAutoGatherContext.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/parse/ColumnStatsSemanticAnalyzer.java bb1bbad 
  ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java 1bccf20 
  ql/src/java/org/apache/hadoop/hive/ql/parse/QBParseInfo.java 3a226e7 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 96df189 
  ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java f7d7a40 
  ql/src/java/org/apache/hadoop/hive/ql/plan/ColumnStatsDesc.java c000db2 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBridge.java 7433263 
  ql/src/test/queries/clientpositive/autoColumnStats_1.q PRE-CREATION 
  ql/src/test/queries/clientpositive/autoColumnStats_2.q PRE-CREATION 
  ql/src/test/queries/clientpositive/autoColumnStats_3.q PRE-CREATION 
  ql/src/test/queries/clientpositive/autoColumnStats_4.q PRE-CREATION 
  ql/src/test/queries/clientpositive/autoColumnStats_5.q PRE-CREATION 
  ql/src/test/queries/clientpositive/autoColumnStats_6.q PRE-CREATION 
  ql/src/test/queries/clientpositive/autoColumnStats_7.q PRE-CREATION 
  ql/src/test/queries/clientpositive/autoColumnStats_8.q PRE-CREATION 
  ql/src/test/queries/clientpositive/combine1.q 3bcb8b1 
  ql/src/test/results/clientpositive/alter_merge_stats_orc.q.out 0d5ba01 
  ql/src/test/results/clientpositive/alter_numbuckets_partitioned_table2_h23.q.out 3b71598 
  ql/src/test/results/clientpositive/alter_numbuckets_partitioned_table_h23.q.out cab3de4 
  ql/src/test/results/clientpositive/alter_partition_coltype.q.out a184372 
  ql/src/test/results/clientpositive/alter_table_serde2.q.out dd946e5 
  ql/src/test/results/clientpositive/analyze_table_null_partition.q.out 72a24d5 
  ql/src/test/results/clientpositive/annotate_stats_filter.q.out ba0419e 
  ql/src/test/results/clientpositive/annotate_stats_groupby.q.out 3070407 
  ql/src/test/results/clientpositive/autoColumnStats_1.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/autoColumnStats_2.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/autoColumnStats_3.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/autoColumnStats_4.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/autoColumnStats_5.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/autoColumnStats_6.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/autoColumnStats_7.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/autoColumnStats_8.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/auto_join1.q.out 7c51b50 
  ql/src/test/results/clientpositive/auto_join14.q.out 4a0f2f9 
  ql/src/test/results/clientpositive/auto_join17.q.out f49d81a 
  ql/src/test/results/clientpositive/auto_join19.q.out 3f70055 
  ql/src/test/results/clientpositive/auto_join19_inclause.q.out 3f70055 
  ql/src/test/results/clientpositive/auto_join2.q.out 26d16ee 
  ql/src/test/results/clientpositive/auto_join25.q.out cc9a348 
  ql/src/test/results/clientpositive/auto_join26.q.out c190621 
  ql/src/test/results/clientpositive/auto_join3.q.out 6c6c79f 
  ql/src/test/results/clientpositive/auto_join4.q.out a4afc7b 
  ql/src/test/results/clientpositive/auto_join5.q.out bbc23dc 
  ql/src/test/results/clientpositive/auto_join6.q.out fa0b4ed 
  ql/src/test/results/clientpositive/auto_join7.q.out 58e6298 
  ql/src/test/results/clientpositive/auto_join8.q.out 324f95d 
  ql/src/test/results/clientpositive/auto_join9.q.out d7d7d18 
  ql/src/test/results/clientpositive/auto_join_reordering_values.q.out db79fa5 
  ql/src/test/results/clientpositive/auto_smb_mapjoin_14.q.out 577f5d7 
  ql/src/test/results/clientpositive/auto_sortmerge_join_13.q.out b45411c 
  ql/src/test/results/clientpositive/auto_sortmerge_join_6.q.out 7cf01e5 
  ql/src/test/results/clientpositive/auto_sortmerge_join_9.q.out 70c820f 
  ql/src/test/results/clientpositive/binary_output_format.q.out 32b2feb 
  ql/src/test/results/clientpositive/bucket1.q.out 0680176 
  ql/src/test/results/clientpositive/bucket2.q.out 958e556 
  ql/src/test/results/clientpositive/bucket3.q.out eca9ba5 
  ql/src/test/results/clientpositive/bucket_map_join_spark1.q.out 3b1912a 
  ql/src/test/results/clientpositive/bucket_map_join_spark2.q.out 9a6aef7 
  ql/src/test/results/clientpositive/bucket_map_join_spark3.q.out 1d2166f 
  ql/src/test/results/clientpositive/bucket_map_join_spark4.q.out 7d32376 
  ql/src/test/results/clientpositive/bucketmapjoin1.q.out ac8a76f 
  ql/src/test/results/clientpositive/bucketmapjoin13.q.out c78b7a1 
  ql/src/test/results/clientpositive/bucketmapjoin2.q.out d9c4463 
  ql/src/test/results/clientpositive/bucketmapjoin3.q.out 77b19ba 
  ql/src/test/results/clientpositive/bucketmapjoin4.q.out 5858160 
  ql/src/test/results/clientpositive/bucketmapjoin5.q.out 728c0b7 
  ql/src/test/results/clientpositive/bucketmapjoin_negative.q.out 36b1cfe 
  ql/src/test/results/clientpositive/bucketmapjoin_negative2.q.out 21138d6 
  ql/src/test/results/clientpositive/bucketsortoptimize_insert_1.q.out 9faa0d0 
  ql/src/test/results/clientpositive/bucketsortoptimize_insert_2.q.out 9761293 
  ql/src/test/results/clientpositive/bucketsortoptimize_insert_3.q.out e778e35 
  ql/src/test/results/clientpositive/bucketsortoptimize_insert_4.q.out 2b61b4c 
  ql/src/test/results/clientpositive/bucketsortoptimize_insert_5.q.out 44466b8 
  ql/src/test/results/clientpositive/bucketsortoptimize_insert_6.q.out fe32f45 
  ql/src/test/results/clientpositive/bucketsortoptimize_insert_7.q.out fa73acf 
  ql/src/test/results/clientpositive/bucketsortoptimize_insert_8.q.out 6c4a077 
  ql/src/test/results/clientpositive/case_sensitivity.q.out b3969cc 
  ql/src/test/results/clientpositive/cast1.q.out 48a0c14 
  ql/src/test/results/clientpositive/columnStatsUpdateForStatsOptimizer_1.q.out 8c9664d 
  ql/src/test/results/clientpositive/columnStatsUpdateForStatsOptimizer_2.q.out 179bc66 
  ql/src/test/results/clientpositive/columnstats_part_coltype.q.out 84232b6 
  ql/src/test/results/clientpositive/combine2.q.out 8c59816 
  ql/src/test/results/clientpositive/constprog_dp.q.out 8cf301d 
  ql/src/test/results/clientpositive/constprog_type.q.out d145d37 
  ql/src/test/results/clientpositive/correlationoptimizer5.q.out f8a53a0 
  ql/src/test/results/clientpositive/cp_sel.q.out f42ec84 
  ql/src/test/results/clientpositive/dynamic_partition_skip_default.q.out e0168b3 
  ql/src/test/results/clientpositive/dynamic_rdd_cache.q.out 9a09c4c 
  ql/src/test/results/clientpositive/dynpart_sort_opt_vectorization.q.out d03bfe4 
  ql/src/test/results/clientpositive/dynpart_sort_optimization.q.out 13383fb 
  ql/src/test/results/clientpositive/dynpart_sort_optimization2.q.out 3b24a2e 
  ql/src/test/results/clientpositive/explain_ddl.q.out fa73d99 
  ql/src/test/results/clientpositive/extrapolate_part_stats_full.q.out a30c356 
  ql/src/test/results/clientpositive/extrapolate_part_stats_partial.q.out 4e589b8 
  ql/src/test/results/clientpositive/extrapolate_part_stats_partial_ndv.q.out 3185f70 
  ql/src/test/results/clientpositive/filter_join_breaktask.q.out 018fd5d 
  ql/src/test/results/clientpositive/groupby1.q.out 785c078 
  ql/src/test/results/clientpositive/groupby10.q.out 5297107 
  ql/src/test/results/clientpositive/groupby11.q.out 86568e8 
  ql/src/test/results/clientpositive/groupby12.q.out b17da54 
  ql/src/test/results/clientpositive/groupby1_limit.q.out aacd23c 
  ql/src/test/results/clientpositive/groupby1_map.q.out 7cdf240 
  ql/src/test/results/clientpositive/groupby1_map_nomap.q.out 7cdf240 
  ql/src/test/results/clientpositive/groupby1_map_skew.q.out be7eeca 
  ql/src/test/results/clientpositive/groupby1_noskew.q.out ef1fd79 
  ql/src/test/results/clientpositive/groupby2_map.q.out c157ba7 
  ql/src/test/results/clientpositive/groupby2_map_multi_distinct.q.out afbcb7f 
  ql/src/test/results/clientpositive/groupby2_map_skew.q.out 813ae5c 
  ql/src/test/results/clientpositive/groupby2_noskew.q.out d3aff42 
  ql/src/test/results/clientpositive/groupby2_noskew_multi_distinct.q.out 2527fcd 
  ql/src/test/results/clientpositive/groupby3.q.out 4247d28 
  ql/src/test/results/clientpositive/groupby3_map.q.out 405daa9 
  ql/src/test/results/clientpositive/groupby3_map_multi_distinct.q.out a1d24e8 
  ql/src/test/results/clientpositive/groupby3_map_skew.q.out 5679770 
  ql/src/test/results/clientpositive/groupby3_noskew.q.out e6c13cf 
  ql/src/test/results/clientpositive/groupby3_noskew_multi_distinct.q.out f6be869 
  ql/src/test/results/clientpositive/groupby4.q.out 55313fa 
  ql/src/test/results/clientpositive/groupby4_map.q.out a8a7fbe 
  ql/src/test/results/clientpositive/groupby4_map_skew.q.out 3f38895 
  ql/src/test/results/clientpositive/groupby4_noskew.q.out d1936da 
  ql/src/test/results/clientpositive/groupby5_map.q.out 5fbd3d7 
  ql/src/test/results/clientpositive/groupby5_map_skew.q.out 60b010b 
  ql/src/test/results/clientpositive/groupby6.q.out d8cb2ac 
  ql/src/test/results/clientpositive/groupby6_map.q.out b307eba 
  ql/src/test/results/clientpositive/groupby6_map_skew.q.out c5af47d 
  ql/src/test/results/clientpositive/groupby6_noskew.q.out 43c7400 
  ql/src/test/results/clientpositive/groupby_cube_multi_gby.q.out 992fd2d 
  ql/src/test/results/clientpositive/groupby_map_ppr.q.out 556995c 
  ql/src/test/results/clientpositive/groupby_map_ppr_multi_distinct.q.out 393e6fa 
  ql/src/test/results/clientpositive/groupby_multi_insert_common_distinct.q.out c78cfe5 
  ql/src/test/results/clientpositive/groupby_multi_single_reducer.q.out 1381d91 
  ql/src/test/results/clientpositive/groupby_multi_single_reducer2.q.out 972ed51 
  ql/src/test/results/clientpositive/groupby_multi_single_reducer3.q.out c5488de 
  ql/src/test/results/clientpositive/groupby_ppr.q.out 2e160b3 
  ql/src/test/results/clientpositive/groupby_ppr_multi_distinct.q.out 93fbc0e 
  ql/src/test/results/clientpositive/implicit_cast_during_insert.q.out c4b9dc4 
  ql/src/test/results/clientpositive/index_auto_update.q.out 73bd4c7 
  ql/src/test/results/clientpositive/infer_bucket_sort.q.out 182dd3a 
  ql/src/test/results/clientpositive/infer_bucket_sort_convert_join.q.out 2f7e538 
  ql/src/test/results/clientpositive/infer_bucket_sort_grouping_operators.q.out ebfce60 
  ql/src/test/results/clientpositive/infer_bucket_sort_list_bucket.q.out f45a719 
  ql/src/test/results/clientpositive/infer_bucket_sort_multi_insert.q.out d4c22f4 
  ql/src/test/results/clientpositive/innerjoin.q.out c106004 
  ql/src/test/results/clientpositive/input11.q.out bb22ee8 
  ql/src/test/results/clientpositive/input11_limit.q.out 92db5a9 
  ql/src/test/results/clientpositive/input12.q.out 1557c58 
  ql/src/test/results/clientpositive/input13.q.out 91ff234 
  ql/src/test/results/clientpositive/input14.q.out f5c5b25 
  ql/src/test/results/clientpositive/input14_limit.q.out 9870ad5 
  ql/src/test/results/clientpositive/input17.q.out 1f7e787 
  ql/src/test/results/clientpositive/input18.q.out 7993cb7 
  ql/src/test/results/clientpositive/input1_limit.q.out 0ceb153 
  ql/src/test/results/clientpositive/input20.q.out cf0ee1d 
  ql/src/test/results/clientpositive/input33.q.out b35e2d0 
  ql/src/test/results/clientpositive/input34.q.out 72f66c3 
  ql/src/test/results/clientpositive/input35.q.out 8b86991 
  ql/src/test/results/clientpositive/input36.q.out 76921ad 
  ql/src/test/results/clientpositive/input38.q.out 0c4e81d 
  ql/src/test/results/clientpositive/input5.q.out a399528 
  ql/src/test/results/clientpositive/input6.q.out 5ed2767 
  ql/src/test/results/clientpositive/input7.q.out 0545b1f 
  ql/src/test/results/clientpositive/input8.q.out 03857fc 
  ql/src/test/results/clientpositive/input9.q.out 38216a6 
  ql/src/test/results/clientpositive/input_part1.q.out 501f7a9 
  ql/src/test/results/clientpositive/input_part10.q.out c8fb37e 
  ql/src/test/results/clientpositive/input_part2.q.out 9eba510 
  ql/src/test/results/clientpositive/input_part5.q.out c6ae2fd 
  ql/src/test/results/clientpositive/input_testsequencefile.q.out 60aaf83 
  ql/src/test/results/clientpositive/input_testxpath.q.out e07628a 
  ql/src/test/results/clientpositive/input_testxpath2.q.out a0baccf 
  ql/src/test/results/clientpositive/insert1.q.out 7a2c429 
  ql/src/test/results/clientpositive/insert_into1.q.out 6bbb86c 
  ql/src/test/results/clientpositive/insert_into2.q.out 1910de6 
  ql/src/test/results/clientpositive/insert_into3.q.out ae7523b 
  ql/src/test/results/clientpositive/insert_into4.q.out 07ff99b 
  ql/src/test/results/clientpositive/insert_into5.q.out b9510b9 
  ql/src/test/results/clientpositive/insert_into6.q.out d93a167 
  ql/src/test/results/clientpositive/insertoverwrite_bucket.q.out bfbe87b 
  ql/src/test/results/clientpositive/join14.q.out 24b5a8e 
  ql/src/test/results/clientpositive/join17.q.out 13260ef 
  ql/src/test/results/clientpositive/join2.q.out 1ad7285 
  ql/src/test/results/clientpositive/join25.q.out af707aa 
  ql/src/test/results/clientpositive/join26.q.out 5533ce4 
  ql/src/test/results/clientpositive/join27.q.out 0c07cb2 
  ql/src/test/results/clientpositive/join28.q.out 9aadd28 
  ql/src/test/results/clientpositive/join29.q.out 29dab0e 
  ql/src/test/results/clientpositive/join3.q.out 3f9a1fb 
  ql/src/test/results/clientpositive/join30.q.out ee9c49b 
  ql/src/test/results/clientpositive/join31.q.out 6466ba2 
  ql/src/test/results/clientpositive/join32.q.out 5cb124b 
  ql/src/test/results/clientpositive/join32_lessSize.q.out 544e814 
  ql/src/test/results/clientpositive/join33.q.out 5cb124b 
  ql/src/test/results/clientpositive/join34.q.out e2c2b1a 
  ql/src/test/results/clientpositive/join35.q.out 96f7cc8 
  ql/src/test/results/clientpositive/join36.q.out 275860a 
  ql/src/test/results/clientpositive/join37.q.out 8820c16 
  ql/src/test/results/clientpositive/join39.q.out 02b1837 
  ql/src/test/results/clientpositive/join4.q.out 82b8568 
  ql/src/test/results/clientpositive/join43.q.out 127d5d0 
  ql/src/test/results/clientpositive/join5.q.out fa9c756 
  ql/src/test/results/clientpositive/join6.q.out eab0fd8 
  ql/src/test/results/clientpositive/join7.q.out 558b6f2 
  ql/src/test/results/clientpositive/join8.q.out d7e7cb1 
  ql/src/test/results/clientpositive/join9.q.out e904b31 
  ql/src/test/results/clientpositive/join_map_ppr.q.out 444fd7f 
  ql/src/test/results/clientpositive/lb_fs_stats.q.out 8344125 
  ql/src/test/results/clientpositive/limit_pushdown_negative.q.out 748a6da 
  ql/src/test/results/clientpositive/lineage1.q.out a655c6c 
  ql/src/test/results/clientpositive/lineage2.q.out a08094a 
  ql/src/test/results/clientpositive/lineage3.q.out 61acf52 
  ql/src/test/results/clientpositive/list_bucket_dml_1.q.out 0cd07ef 
  ql/src/test/results/clientpositive/list_bucket_dml_11.q.java1.7.out f304083 
  ql/src/test/results/clientpositive/list_bucket_dml_12.q.java1.7.out 7d8070a 
  ql/src/test/results/clientpositive/list_bucket_dml_13.q.java1.7.out 81b1431 
  ql/src/test/results/clientpositive/list_bucket_dml_14.q.out 790801f 
  ql/src/test/results/clientpositive/list_bucket_dml_2.q.java1.7.out bfc1e43 
  ql/src/test/results/clientpositive/list_bucket_dml_3.q.out ea24f86 
  ql/src/test/results/clientpositive/list_bucket_dml_4.q.java1.7.out 6b8b89f 
  ql/src/test/results/clientpositive/list_bucket_dml_5.q.java1.7.out 1019474 
  ql/src/test/results/clientpositive/list_bucket_dml_6.q.java1.7.out 668e918 
  ql/src/test/results/clientpositive/list_bucket_dml_7.q.out 3aa9003 
  ql/src/test/results/clientpositive/list_bucket_dml_8.q.java1.7.out 672e5ac 
  ql/src/test/results/clientpositive/list_bucket_dml_9.q.java1.7.out 7179f61 
  ql/src/test/results/clientpositive/list_bucket_query_multiskew_1.q.out 1d8ec8b 
  ql/src/test/results/clientpositive/list_bucket_query_multiskew_2.q.out 83576f1 
  ql/src/test/results/clientpositive/list_bucket_query_multiskew_3.q.out 2e77245 
  ql/src/test/results/clientpositive/llap_acid.q.out 429b042 
  ql/src/test/results/clientpositive/load_dyn_part1.q.out e7dace5 
  ql/src/test/results/clientpositive/load_dyn_part10.q.out ca388b9 
  ql/src/test/results/clientpositive/load_dyn_part13.q.out 9e0ac6f 
  ql/src/test/results/clientpositive/load_dyn_part14.q.out 53e9df3 
  ql/src/test/results/clientpositive/load_dyn_part2.q.out 0c5fe6b 
  ql/src/test/results/clientpositive/load_dyn_part3.q.out 3242c3d 
  ql/src/test/results/clientpositive/load_dyn_part4.q.out d24875f 
  ql/src/test/results/clientpositive/load_dyn_part5.q.out e4bc742 
  ql/src/test/results/clientpositive/load_dyn_part8.q.out 84ccb75 
  ql/src/test/results/clientpositive/load_dyn_part9.q.out 300f41e 
  ql/src/test/results/clientpositive/mapjoin_hook.q.out 8a6743c 
  ql/src/test/results/clientpositive/mapreduce1.q.out 3d0a156 
  ql/src/test/results/clientpositive/mapreduce2.q.out 676c387 
  ql/src/test/results/clientpositive/mapreduce3.q.out fc1a402 
  ql/src/test/results/clientpositive/mapreduce4.q.out 17fa029 
  ql/src/test/results/clientpositive/mapreduce5.q.out 21103f8 
  ql/src/test/results/clientpositive/mapreduce6.q.out fe4e631 
  ql/src/test/results/clientpositive/mapreduce7.q.out cc97887 
  ql/src/test/results/clientpositive/mapreduce8.q.out b1763c7 
  ql/src/test/results/clientpositive/merge1.q.out 94089fc 
  ql/src/test/results/clientpositive/merge2.q.out a3a0e8e 
  ql/src/test/results/clientpositive/merge3.q.out 5b581db 
  ql/src/test/results/clientpositive/merge4.q.out 08e4455 
  ql/src/test/results/clientpositive/merge_dynamic_partition.q.out 07102b1 
  ql/src/test/results/clientpositive/merge_dynamic_partition2.q.out 5a2afb0 
  ql/src/test/results/clientpositive/merge_dynamic_partition3.q.out 86978f3 
  ql/src/test/results/clientpositive/merge_dynamic_partition4.q.out 86af660 
  ql/src/test/results/clientpositive/merge_dynamic_partition5.q.out c1468c1 
  ql/src/test/results/clientpositive/mergejoin.q.out a85fd8b 
  ql/src/test/results/clientpositive/metadata_only_queries.q.out 9bbc9b9 
  ql/src/test/results/clientpositive/multiMapJoin2.q.out a582874 
  ql/src/test/results/clientpositive/multi_insert.q.out 3d26b69 
  ql/src/test/results/clientpositive/multi_insert_gby.q.out 7c5e589 
  ql/src/test/results/clientpositive/multi_insert_gby2.q.out f4baf04 
  ql/src/test/results/clientpositive/multi_insert_gby3.q.out 6ee003b 
  ql/src/test/results/clientpositive/multi_insert_lateral_view.q.out 7964405 
  ql/src/test/results/clientpositive/multi_insert_mixed.q.out aaf9346 
  ql/src/test/results/clientpositive/multi_insert_move_tasks_share_dependencies.q.out c302dea 
  ql/src/test/results/clientpositive/multi_insert_union_src.q.out 2036e63 
  ql/src/test/results/clientpositive/nonreserved_keywords_insert_into1.q.out d01461b 
  ql/src/test/results/clientpositive/notable_alias1.q.out 200b31c 
  ql/src/test/results/clientpositive/notable_alias2.q.out 4df8073 
  ql/src/test/results/clientpositive/orc_analyze.q.out 87855fa 
  ql/src/test/results/clientpositive/orc_llap.q.out 6fc73b7 
  ql/src/test/results/clientpositive/orc_merge1.q.out 3f047da 
  ql/src/test/results/clientpositive/orc_merge10.q.out cf70dcf 
  ql/src/test/results/clientpositive/orc_merge2.q.out d4c474f 
  ql/src/test/results/clientpositive/orc_merge3.q.out 7bf12c6 
  ql/src/test/results/clientpositive/orc_merge4.q.out 828f204 
  ql/src/test/results/clientpositive/orc_merge5.q.out 77c01c7 
  ql/src/test/results/clientpositive/orc_merge6.q.out 05f548c 
  ql/src/test/results/clientpositive/orc_merge7.q.out beff0ed 
  ql/src/test/results/clientpositive/orc_merge_incompat1.q.out 7a9c772 
  ql/src/test/results/clientpositive/parallel_join1.q.out 3c97671 
  ql/src/test/results/clientpositive/pcr.q.out 684d4d7 
  ql/src/test/results/clientpositive/pcs.q.out d6d2431 
  ql/src/test/results/clientpositive/pointlookup2.q.out 6fc6e7f 
  ql/src/test/results/clientpositive/pointlookup3.q.out 2b25b39 
  ql/src/test/results/clientpositive/pointlookup4.q.out 6236272 
  ql/src/test/results/clientpositive/ppd_constant_expr.q.out 17e2bab 
  ql/src/test/results/clientpositive/ppd_join5.q.out f464c17 
  ql/src/test/results/clientpositive/ppd_multi_insert.q.out 1a7019c 
  ql/src/test/results/clientpositive/ppd_union_view.q.out 36bb214 
  ql/src/test/results/clientpositive/ptf.q.out cdac02a 
  ql/src/test/results/clientpositive/push_or.q.out bc5c930 
  ql/src/test/results/clientpositive/quote1.q.out f8592c4 
  ql/src/test/results/clientpositive/rand_partitionpruner2.q.out 4559b39 
  ql/src/test/results/clientpositive/rcfile_default_format.q.out c961231 
  ql/src/test/results/clientpositive/rcfile_merge2.q.out b2f61cf 
  ql/src/test/results/clientpositive/rcfile_merge3.q.out 1566ec3 
  ql/src/test/results/clientpositive/rcfile_merge4.q.out 6171434 
  ql/src/test/results/clientpositive/rcfile_null_value.q.out 1a361e1 
  ql/src/test/results/clientpositive/sample1.q.out caf62f4 
  ql/src/test/results/clientpositive/sample10.q.out bf0cb59 
  ql/src/test/results/clientpositive/sample2.q.out e70edd7 
  ql/src/test/results/clientpositive/sample4.q.out 90dfc26 
  ql/src/test/results/clientpositive/sample5.q.out 04aa8cc 
  ql/src/test/results/clientpositive/sample6.q.out c2be9d6 
  ql/src/test/results/clientpositive/sample7.q.out e1897ee 
  ql/src/test/results/clientpositive/schema_evol_stats.q.out 63dab2e 
  ql/src/test/results/clientpositive/skewjoin.q.out bd954ef 
  ql/src/test/results/clientpositive/smb_mapjoin_11.q.out b407402 
  ql/src/test/results/clientpositive/smb_mapjoin_12.q.out e2723a0 
  ql/src/test/results/clientpositive/smb_mapjoin_13.q.out a5a60e4 
  ql/src/test/results/clientpositive/smb_mapjoin_15.q.out cdf2359 
  ql/src/test/results/clientpositive/smb_mapjoin_18.q.out 4b29056 
  ql/src/test/results/clientpositive/smb_mapjoin_19.q.out 95137cd 
  ql/src/test/results/clientpositive/smb_mapjoin_20.q.out 0324aa4 
  ql/src/test/results/clientpositive/smb_mapjoin_21.q.out 319fef3 
  ql/src/test/results/clientpositive/smb_mapjoin_22.q.out 8f1dbd6 
  ql/src/test/results/clientpositive/smb_mapjoin_6.q.out e22ebc8 
  ql/src/test/results/clientpositive/smb_mapjoin_7.q.out 82f5804 
  ql/src/test/results/clientpositive/sort_merge_join_desc_5.q.out 29dee15 
  ql/src/test/results/clientpositive/sort_merge_join_desc_6.q.out b1f99e5 
  ql/src/test/results/clientpositive/sort_merge_join_desc_7.q.out f788fec 
  ql/src/test/results/clientpositive/special_character_in_tabnames_1.q.out 7374714 
  ql/src/test/results/clientpositive/stats0.q.out 99326fc 
  ql/src/test/results/clientpositive/stats1.q.out 72c53e3 
  ql/src/test/results/clientpositive/stats10.q.out d7519af 
  ql/src/test/results/clientpositive/stats11.q.out 7598dc0 
  ql/src/test/results/clientpositive/stats14.q.out e8fe776 
  ql/src/test/results/clientpositive/stats15.q.out 59389db 
  ql/src/test/results/clientpositive/stats18.q.out 6971e44 
  ql/src/test/results/clientpositive/stats4.q.out 2ffbba9 
  ql/src/test/results/clientpositive/stats_empty_dyn_part.q.out 982baab 
  ql/src/test/results/clientpositive/stats_invalidation.q.out d24fdc3 
  ql/src/test/results/clientpositive/stats_list_bucket.q.java1.7.out a4908bc 
  ql/src/test/results/clientpositive/stats_only_null.q.out 032f6c8 
  ql/src/test/results/clientpositive/statsfs.q.out 9d63bf7 
  ql/src/test/results/clientpositive/subquery_multiinsert.q.java1.7.out 279843b 
  ql/src/test/results/clientpositive/tez/autoColumnStats_2.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/truncate_column.q.out adbddfa 
  ql/src/test/results/clientpositive/udf1.q.out b3b694b 
  ql/src/test/results/clientpositive/udf3.q.out 8b2ad31 
  ql/src/test/results/clientpositive/udf_10_trims.q.out 3a5303a 
  ql/src/test/results/clientpositive/udf_length.q.out 0dc39ea 
  ql/src/test/results/clientpositive/udf_reverse.q.out 4fee27f 
  ql/src/test/results/clientpositive/union10.q.out 417ae27 
  ql/src/test/results/clientpositive/union12.q.out ee56f15 
  ql/src/test/results/clientpositive/union22.q.out 5309c71 
  ql/src/test/results/clientpositive/union28.q.out c3789d0 
  ql/src/test/results/clientpositive/union29.q.out 87ba275 
  ql/src/test/results/clientpositive/union30.q.out 26a27c8 
  ql/src/test/results/clientpositive/union31.q.out bb35d5c 
  ql/src/test/results/clientpositive/union33.q.out a91e74c 
  ql/src/test/results/clientpositive/union4.q.out 41ca811 
  ql/src/test/results/clientpositive/union6.q.out 0844165 
  ql/src/test/results/clientpositive/unionDistinct_1.q.out 61bfa74 
  ql/src/test/results/clientpositive/union_fast_stats.q.out f0879af 
  ql/src/test/results/clientpositive/union_lateralview.q.out 7b89e50 
  ql/src/test/results/clientpositive/union_remove_26.q.out e4b156a 
  ql/src/test/results/clientpositive/union_top_level.q.out 134c532 
  ql/src/test/results/clientpositive/vector_auto_smb_mapjoin_14.q.out 9837b26 
  ql/src/test/results/clientpositive/vector_bucket.q.out 7a6a4da 
  ql/src/test/results/clientpositive/vector_char_4.q.out 58988bf 
  ql/src/test/results/clientpositive/vector_char_simple.q.out e1dedae 
  ql/src/test/results/clientpositive/vector_multi_insert.q.out e9f106d 
  ql/src/test/results/clientpositive/vector_tablesample_rows.q.out 25f2996 
  ql/src/test/results/clientpositive/vector_varchar_4.q.out f7c9cd0 
  ql/src/test/results/clientpositive/vector_varchar_simple.q.out 49d8b34 
  ql/src/test/results/clientpositive/vectorized_context.q.out 913d07c 
  ql/src/test/results/clientpositive/vectorized_ptf.q.out 3b17591 

Diff: https://reviews.apache.org/r/43603/diff/


Testing
-------


Thanks,

pengcheng xiong