You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "Pramod Biligiri (Jira)" <ji...@apache.org> on 2022/10/11 09:49:00 UTC

[jira] [Updated] (HUDI-5009) Enabling asynchronous processing of Metastore Sync

     [ https://issues.apache.org/jira/browse/HUDI-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Pramod Biligiri updated HUDI-5009:
----------------------------------
    Summary: Enabling asynchronous processing of Metastore Sync  (was: Enabling invoking runMetaSync() asynchronously)

> Enabling asynchronous processing of Metastore Sync
> --------------------------------------------------
>
>                 Key: HUDI-5009
>                 URL: https://issues.apache.org/jira/browse/HUDI-5009
>             Project: Apache Hudi
>          Issue Type: Task
>          Components: meta-sync
>            Reporter: Pramod Biligiri
>            Priority: Minor
>
> Currently, runMetaSync() invokes each Metastore Sync in a blocking fashion, and iterates over the different Metastores sequentially - ([code link|https://github.com/apache/hudi/blob/release-0.12.0/hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/DeltaSync.java#L695] within 0.12.0 branch)
> And runMetaSync() is invoked during each commit, which can lead to a slow down of commit flow if many metastores are being synced. So enable async invocation of runMetaSync().



--
This message was sent by Atlassian Jira
(v8.20.10#820010)