You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Csaba Ringhofer (Jira)" <ji...@apache.org> on 2024/09/04 07:28:00 UTC
[jira] [Created] (IMPALA-13356) Avoid unnecessarily updating column stats
Csaba Ringhofer created IMPALA-13356:
----------------------------------------
Summary: Avoid unnecessarily updating column stats
Key: IMPALA-13356
URL: https://issues.apache.org/jira/browse/IMPALA-13356
Project: IMPALA
Issue Type: Improvement
Components: Catalog
Reporter: Csaba Ringhofer
Currently column stats are reloaded every time when the schema is reloaded:
https://github.com/apache/impala/blob/48ee4276be1eb278fb628a4813728134a4910b1f/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java#L1304
This includes the common scenario of processing alter table events.
Since HIVE-22046 introduced the engine field for column stats it is unlikely that Impala's version of column stats are modified by any other component. If there is another Impala catalogd connecting to the same cluster then it should also update table property impala.lastComputeStatsTime, so it is enough to update column stats when a non-self event is seen that modifies this property.
Another case when it can be useful to reload stats is when the schema actually changes, for example when columns are added/removed/renamed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)