You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by pengcheng xiong <px...@hortonworks.com> on 2014/08/04 00:47:24 UTC
Review Request 24218: MetadataUpdater: provide a mechanism to edit the
statistics of a column in a table (or a partition of a table)
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24218/
-----------------------------------------------------------
Review request for hive.
Repository: hive-git
Description
-------
This patch provides ability to update certain stats without scanning any data or without "hacking the backend db". It helps (esp for CBO work) to set up unit tests quickly and verify both cbo and the stats subsystem. It also helps when experimenting with the system if you're just trying out hive/hadoop on a small cluster. Finally it gives you a quick and clean way to fix things when something went wrong wrt stats in your environment.
Usage:
ALTER TABLE table_name PARTITION partition_spec UPDATE STATISTICS FOR COLUMN col_name SET col_statistics
For example,
ALTER TABLE src_x_int UPDATE STATISTICS FOR COLUMN key SET ('numDVs'='101','highValue'='10001.0');
ALTER TABLE src_p PARTITION(partitionId=1) UPDATE STATISTICS FOR COLUMN key SET ('numDVs'='100','avgColLen'='1.0001');
Diffs
-----
cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java 3cdedba
conf/hive-default.xml.template ba5b8a9
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java 130fd67
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/BinaryColumnStatsData.java 1516b25
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/BooleanColumnStatsData.java 9ef9c0f
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/DecimalColumnStatsData.java 951d479
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/DoubleColumnStatsData.java 4203fd8
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/LongColumnStatsData.java d817d46
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/StringColumnStatsData.java db3274a
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 32da869
metastore/src/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java c3e2820
metastore/src/model/org/apache/hadoop/hive/metastore/model/MPartitionColumnStatistics.java 89c31dc
metastore/src/model/org/apache/hadoop/hive/metastore/model/MTableColumnStatistics.java 44bbab5
ql/src/java/org/apache/hadoop/hive/ql/exec/ColumnStatsUpdateTask.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/exec/TaskFactory.java ad6e19c
ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapRedTask.java e83bc17
ql/src/java/org/apache/hadoop/hive/ql/metadata/SessionHiveMetaStoreClient.java 37b1669
ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java c0322fb
ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g f5d0602
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java d38270c
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java 268920a
ql/src/java/org/apache/hadoop/hive/ql/plan/ColumnStatsUpdateWork.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/plan/DDLWork.java 79d9d16
Diff: https://reviews.apache.org/r/24218/diff/
Testing
-------
Thanks,
pengcheng xiong