[jira] [Created] (IMPALA-10265) Doc about enable_incremental_metadata_updates flag

["Quanlong Huang (Jira)" <ji...@apache.org>]

Quanlong Huang created IMPALA-10265:
---------------------------------------

             Summary: Doc about enable_incremental_metadata_updates flag
                 Key: IMPALA-10265
                 URL: https://issues.apache.org/jira/browse/IMPALA-10265
             Project: IMPALA
          Issue Type: Documentation
            Reporter: Quanlong Huang


IMPALA-10113 adds a feature flag to turn off the incremental metadata update feature which defaults to be turned on. This flag decides how catalogd propagates metadata updates to the catalog topic.

If enable_incremental_metadata_updates is true, catalogd will send metadata updates in partition granularity. So a table that just has one partition changed will only have an update on that partition. This reduces the size of the metadata that needs to be sent from the catalogd.

If enable_incremental_metadata_updates is false, that's the legacy behavior. Catalogd will send metadata updates in table granularity. So a table that just has one partition changed will still have an update for the whole table object. Catalogd still sends the whole table thrift object to the catalog topic.

Note that this is a catalogd-only flag. Don't need to be set on impalads since impalad can process both incremental and non-incremental catalog updates.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org