[jira] [Updated] (HUDI-4329) Add separate control for compaction operation sync/async mode

["Raymond Xu (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HUDI-4329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Raymond Xu updated HUDI-4329:
-----------------------------
    Priority: Critical  (was: Blocker)

> Add separate control for compaction operation sync/async mode
> -------------------------------------------------------------
>
>                 Key: HUDI-4329
>                 URL: https://issues.apache.org/jira/browse/HUDI-4329
>             Project: Apache Hudi
>          Issue Type: Improvement
>          Components: flink
>            Reporter: Shizhi Chen
>            Assignee: Shizhi Chen
>            Priority: Critical
>              Labels: pull-request-available
>             Fix For: 0.13.1, 0.12.3
>
>
> *Problem Review*
> The compact operation sync/async in CompactionFunction is now controlled by FlinkOptions#COMPACTION_ASYNC_ENABLED
> {code:java}
>   public CompactFunction(Configuration conf) {
>     this.conf = conf;
>     this.asyncCompaction = StreamerUtil.needsAsyncCompaction(conf);
>   }
> {code}
> While in fact it cannot be switched to sync mode because the pipeline defined by sync compaction will only include the clean but not compact operators.
> {code:java}
>       // compaction
>       if (StreamerUtil.needsAsyncCompaction(conf)) {
>         return Pipelines.compact(conf, pipeline);
>       } else {
>         return Pipelines.clean(conf, pipeline);
>       }
> {code}
> *Improvement*
> Add another separate control switch for compaction operation sync/async mode.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)