[jira] [Updated] (IMPALA-3561) Planner should request statistics for relevant columns not entire table

["Tim Armstrong (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tim Armstrong updated IMPALA-3561:
----------------------------------
    External issue ID:   (was: IMPALA-3561)
               Parent:     (was: IMPALA-2649)
           Issue Type: Improvement  (was: Sub-task)

> Planner should request statistics for relevant columns not entire table
> -----------------------------------------------------------------------
>
>                 Key: IMPALA-3561
>                 URL: https://issues.apache.org/jira/browse/IMPALA-3561
>             Project: IMPALA
>          Issue Type: Improvement
>          Components: Frontend
>    Affects Versions: Impala 2.6.0
>            Reporter: Mostafa Mokhtar
>            Priority: Minor
>              Labels: catalog-server, performance, planner, scalability
>
> When generating a plan the planner request statistics for all columns referenced by all tables which is not necessary, this creates load on the catalog and bloats the meta data cache memory. 
> Stats are only required for columns involved in :
> * Join
> * Aggregations 
> * Filters
> The query below will fetch statistics for all 22 columns/partitions from store_sales and which is un-necessary as only ss_item_sk and ss_promo_sk are needed. 
> {code}
> select 
>     count(*)
> from
>     store_sales,
>     item
> where
>     ss_item_sk = i_item_sk
> group by ss_promo_sk;
> {code}
> If this issues is addressed it should reduce the meta data cache memory by an order of magnitude.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org