You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2021/07/11 12:59:00 UTC

[jira] [Commented] (IMPALA-10738) Min/max filters should be enabled for partition columns

    [ https://issues.apache.org/jira/browse/IMPALA-10738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17378653#comment-17378653 ] 

ASF subversion and git services commented on IMPALA-10738:
----------------------------------------------------------

Commit 84d784351c7a3606ec86abf6ea757aef72687b55 in impala's branch refs/heads/master from Qifan Chen
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=84d7843 ]

IMPALA-10738: Min/max filters should be enabled for partition columns

This patch enables min/max filters for partitoned columns to take
advantage of the min/max filter infrastructure already built and to
provide coverage for certain equi-joins in which the stats filters
are not feasible.

The new feature is turned on by default and to turn off the feature,
set the new query option minmax_filter_partition_column to false.

In the patch, the existing query option enabled_runtime_filter_types
is enforced in specifying the types of the filters generated. The
default value ALL generates both the bloom and min/max filters. The
alternative value BLOOM generates only the bloom filters and another
alternative value MIN_MAX generates only the min/max filters.

The normal control knobs minmax_filter_threshold (for threshold) and
minmax_filtering_level (for filtering level) still work. When the
threshold is 0, the patch automatically assigns a reasonable value
for the threshhold.

Testing:
  1). Added new tests in
      overlap_min_max_filters_on_partition_columns.test;
  2). Core tests

Change-Id: I89e135ef48b4bb36d70075287b03d1c12496b042
Reviewed-on: http://gerrit.cloudera.org:8080/17568
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>


> Min/max filters should be enabled for partition columns
> -------------------------------------------------------
>
>                 Key: IMPALA-10738
>                 URL: https://issues.apache.org/jira/browse/IMPALA-10738
>             Project: IMPALA
>          Issue Type: Test
>          Components: Frontend
>            Reporter: Qifan Chen
>            Priority: Major
>
> Currently, the min/max filters are not applied to partition columns and filtering on partition columns are the job of the bloom filter. 
> Given the infrastructure built for the min/max filters, it is quite feasible to apply them on the partition columns. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org