You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "yuemeng (Jira)" <ji...@apache.org> on 2022/07/13 13:22:00 UTC
[jira] [Updated] (HUDI-4392) Flink MOR table inline compaction plan execution sequence should be configurable
[ https://issues.apache.org/jira/browse/HUDI-4392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
yuemeng updated HUDI-4392:
--------------------------
Description:
when there leaves too much compaction in some cases. Flink inline compaction always deals the earliest then one by one, it may fail the job because of too much compact operation and we can do nothing
Flink MOR table inline compaction plan execution sequence should be configurable to avoid too much compaction needed to compact to fail the job
When there are a large number of compact plans that need to be executed， inline compact operation handles the latest compaction plan to ensure stability and some external job (offline or compact server) to handle the rest compaction plan.
was:
when there leaves too much compaction in some cases. Flink inline compaction always deals the earliest then one by one, it may fail the job because of too much compact operation and we can do nothing
Flink MOR table inline compaction plan execution sequence should be configurable to avoid too much compaction needed to compacted to fail the job
When there are a large number of compact plans that need to be executed， inline compact operation handles the latest compaction plan to ensure stability and some external job (offline or compact server) to handle the rest compaction plan.
> Flink MOR table inline compaction plan execution sequence should be configurable
> --------------------------------------------------------------------------------
>
> Key: HUDI-4392
> URL: https://issues.apache.org/jira/browse/HUDI-4392
> Project: Apache Hudi
> Issue Type: Improvement
> Components: flink
> Reporter: yuemeng
> Priority: Major
>
> when there leaves too much compaction in some cases. Flink inline compaction always deals the earliest then one by one, it may fail the job because of too much compact operation and we can do nothing
> Flink MOR table inline compaction plan execution sequence should be configurable to avoid too much compaction needed to compact to fail the job
> When there are a large number of compact plans that need to be executed， inline compact operation handles the latest compaction plan to ensure stability and some external job (offline or compact server) to handle the rest compaction plan.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)