You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "Jonathan Vexler (Jira)" <ji...@apache.org> on 2023/04/28 15:14:00 UTC
[jira] [Updated] (HUDI-6147) Retry Pending Compactions before ingestion in Deltastreamer
[ https://issues.apache.org/jira/browse/HUDI-6147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Vexler updated HUDI-6147:
----------------------------------
Status: Patch Available (was: In Progress)
> Retry Pending Compactions before ingestion in Deltastreamer
> -----------------------------------------------------------
>
> Key: HUDI-6147
> URL: https://issues.apache.org/jira/browse/HUDI-6147
> Project: Apache Hudi
> Issue Type: Improvement
> Components: compaction, deltastreamer
> Reporter: Jonathan Vexler
> Assignee: Jonathan Vexler
> Priority: Major
>
> Currently, the timeline looks like this: The compaction is done after the deltacommit
> {code:java}
> drwxr-xr-x 2 jon staff 64 Apr 27 15:15 archived
> -rw-r--r-- 1 jon staff 0 Apr 27 15:15 20230427151550027.deltacommit.requested
> drwxr-xr-x 4 jon staff 128 Apr 27 15:15 metadata
> -rw-r--r-- 1 jon staff 898 Apr 27 15:15 hoodie.properties
> -rw-r--r-- 1 jon staff 2199 Apr 27 15:15 20230427151550027.deltacommit.inflight
> -rw-r--r-- 1 jon staff 5269 Apr 27 15:15 20230427151550027.deltacommit
> -rw-r--r-- 1 jon staff 0 Apr 27 15:15 20230427151558008.deltacommit.requested
> -rw-r--r-- 1 jon staff 4365 Apr 27 15:15 20230427151558008.deltacommit.inflight
> -rw-r--r-- 1 jon staff 6177 Apr 27 15:16 20230427151558008.deltacommit
> -rw-r--r-- 1 jon staff 3211 Apr 27 15:16 20230427151602293.compaction.requested
> -rw-r--r-- 1 jon staff 0 Apr 27 15:16 20230427151604955.deltacommit.requested
> -rw-r--r-- 1 jon staff 4365 Apr 27 15:16 20230427151604955.deltacommit.inflight
> -rw-r--r-- 1 jon staff 5977 Apr 27 15:16 20230427151604955.deltacommit
> -rw-r--r-- 1 jon staff 1229 Apr 27 15:16 20230427151609434.rollback.requested
> -rw-r--r-- 1 jon staff 0 Apr 27 15:16 20230427151609434.rollback.inflight
> -rw-r--r-- 1 jon staff 1413 Apr 27 15:16 20230427151609434.rollback
> -rw-r--r-- 1 jon staff 0 Apr 27 15:16 20230427151602293.compaction.inflight
> -rw-r--r-- 1 jon staff 5271 Apr 27 15:16 20230427151602293.commit {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)