You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Gopal V (JIRA)" <ji...@apache.org> on 2015/03/11 19:21:38 UTC
[jira] [Created] (HIVE-9931) Approximate nDV statistics from ORC
bloom filter population
Gopal V created HIVE-9931:
-----------------------------
Summary: Approximate nDV statistics from ORC bloom filter population
Key: HIVE-9931
URL: https://issues.apache.org/jira/browse/HIVE-9931
Project: Hive
Issue Type: Improvement
Components: Statistics
Affects Versions: 1.2.0
Reporter: Gopal V
The current CBO implementation requires column nDV statistics to produce good estimates of JOIN selectivity and filter selectivity.
The ORC bloom filters provides an opportunity to estimate the net population of a row-group with false-positive rates capped for each row-group.
This is not useful for filter conditions or join conditions with a cardinality which is a large fraction of the row-count, but can collect viable statistics for low-cardinality filter columns (de-normalization scenarios) or for JOIN dimension columns of low cardinality (demographics or store location).
The challenge in this feature is in distinguishing between these two scenarios, not in the derivation of the approximate nDV itself.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)