You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "sivabalan narayanan (Jira)" <ji...@apache.org> on 2021/11/19 00:32:00 UTC

[jira] [Updated] (HUDI-2793) Fix copying of deltastreamer checkpoint

     [ https://issues.apache.org/jira/browse/HUDI-2793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

sivabalan narayanan updated HUDI-2793:
--------------------------------------
    Priority: Blocker  (was: Major)

> Fix copying of deltastreamer checkpoint
> ---------------------------------------
>
>                 Key: HUDI-2793
>                 URL: https://issues.apache.org/jira/browse/HUDI-2793
>             Project: Apache Hudi
>          Issue Type: Bug
>            Reporter: sivabalan narayanan
>            Assignee: sivabalan narayanan
>            Priority: Blocker
>             Fix For: 0.10.0
>
>
> Fix copying of deltastreamer checkpoint. 
>  
> If there are concurrent writers from spark-datasource, they are expected to copy over deltastreamer checkpoint key if appropriate config is set. 
> but, compaction may not copy over the delta streamer checkpoint key. So, better to fix it in deltastreamer code to look back to previous commits for deltastreamer checkpoint key. 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)