You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "lanyuanxiaoyao (Jira)" <ji...@apache.org> on 2022/05/25 02:45:00 UTC

[jira] [Created] (HUDI-4152) Flink offline compaction allow compact multi compaction plan at once

lanyuanxiaoyao created HUDI-4152:
------------------------------------

             Summary: Flink offline compaction allow compact multi compaction plan at once
                 Key: HUDI-4152
                 URL: https://issues.apache.org/jira/browse/HUDI-4152
             Project: Apache Hudi
          Issue Type: Improvement
          Components: compaction, flink
            Reporter: lanyuanxiaoyao
             Fix For: 0.12.0


Now flink offline compaction tool only handle one compaction plan in one execution.

But some time, compaction plan that created is faster than offline compact.

So it is neccessary to handle multi compaction plan in one offline compact.

Luckly, flink provide a easy way to make the process parallelable. 

I make a new parameter named 'compactionPlanSelectStrategy' to define the offline compact tool how to select compaction plan to compact.

For example, 'SingleCompactionPlanSelectStrategy' will select first or last compaction plan from timeline and 'AllPendingCompactionPlanSelectStrategy' will select all pending compaction plan.





--
This message was sent by Atlassian Jira
(v8.20.7#820007)