[jira] [Resolved] (IMPALA-4045) Catalog cache update should not tied to statestore update frequency

["Tim Armstrong (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-4045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tim Armstrong resolved IMPALA-4045.
-----------------------------------
    Resolution: Won't Fix

This general problem is solved by the local catalog - IMPALA-7127

> Catalog cache update should not tied to statestore update frequency
> -------------------------------------------------------------------
>
>                 Key: IMPALA-4045
>                 URL: https://issues.apache.org/jira/browse/IMPALA-4045
>             Project: IMPALA
>          Issue Type: Improvement
>          Components: Catalog, Distributed Exec
>    Affects Versions: Impala 2.7.0
>            Reporter: Huaisi Xu
>            Priority: Major
>              Labels: performance, usability
>
> All impalad's catalog cache update depends on statestore's "nature" periodic update.
> As a result, impala's {{INVALIDATE METADATA}} and initial "prioritized loading" are delayed by up to 2x"statestore_update_frequency_ms" in the same connection per coordinator.
> Many use cases have {{statestore_update_frequency_ms}} setting to be larger than 20s, so everyone of them are affected by this unnecessary delay. (20s-40s)
> Consider make "prioritized loading" universal (not just for catalogd).
> {{REFRESH}} is not affected by this statestore_update_frequency_ms.
> Repro:
> start statestore with higher and higher {{statestore_update_frequency_ms}}, and you will find initial query(and invalidate metadata) get slower and slower...



--
This message was sent by Atlassian Jira
(v8.3.4#803005)