[jira] [Resolved] (IMPALA-6024) Add minimum sample size for COMPUTE STATS TABLESAMPLE

["Alexander Behm (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-6024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Alexander Behm resolved IMPALA-6024.
------------------------------------
       Resolution: Fixed
    Fix Version/s: Impala 2.12.0

commit 22d9ac08937f348b21075b276d487f4b1ba3524c
Author: Alex Behm <al...@cloudera.com>
Date:   Mon Jan 22 23:07:25 2018 -0800

    IMPALA-6024: Min sample bytes for COMPUTE STATS TABLESAMPLE
    
    Adds a new query option COMPUTE_STATS_MIN_SAMPLE_SIZE
    which is the minimum number of bytes that will be scanned
    in COMPUTE STATS TABLESAMPLE, regardless of the user-supplied
    sampling percent.
    
    The motivation is to prevent sampling for very small tables
    where accurate stats can be obtained cheaply without sampling.
    
    This patch changes COMPUTE STATS TABLESAMPLE to run the regular
    COMPUTE STATS if the effective sampling percent is 0% or 100%.
    For a 100% sampling rate, the sampling-based stats queries
    are more expensive and produce less accurate stats than the
    regular COMPUTE STATS.
    
    Default: COMPUTE_STATS_MIN_SAMPLE_SIZE=1GB
    
    Testing:
    - added new unit tests and ran them locally
    
    Change-Id: I2cb91a40bec50b599875109c2f7c5bf6f41c2400
    Reviewed-on: http://gerrit.cloudera.org:8080/9113
    Reviewed-by: Alex Behm <al...@cloudera.com>
    Tested-by: Impala Public Jenkins


> Add minimum sample size for COMPUTE STATS TABLESAMPLE
> -----------------------------------------------------
>
>                 Key: IMPALA-6024
>                 URL: https://issues.apache.org/jira/browse/IMPALA-6024
>             Project: IMPALA
>          Issue Type: Sub-task
>          Components: Frontend
>    Affects Versions: Impala 2.10.0, Impala 2.11.0
>            Reporter: Alexander Behm
>            Assignee: Alexander Behm
>            Priority: Major
>             Fix For: Impala 2.12.0
>
>
> We should introduce a minimum sample size in bytes for COMPUTE STATS TABLESAMPLE. Reasons:
> * For small tables sampling does not make sense. Accurate stats can be obtained cheaply without sampling.
> * Very small sample sizes mostly do not make sense - some minimum of data is required to get meaningful stats. 
> I think a 1GB minimum might be a good choice and ideally this minimum sample size would be configurable.
> Many other DBMS have stats collection with sampling and in many cases a minimum sample size is required to get any meaningful stats.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)