You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "Neha Aggarwal (Jira)" <ji...@apache.org> on 2023/03/09 12:28:00 UTC

[jira] [Updated] (FLINK-30238) Unified Sink committer does not clean up state on final savepoint

     [ https://issues.apache.org/jira/browse/FLINK-30238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Neha Aggarwal updated FLINK-30238:
----------------------------------
    Attachment: Screenshot 2023-03-09 at 1.47.11 PM.png

> Unified Sink committer does not clean up state on final savepoint
> -----------------------------------------------------------------
>
>                 Key: FLINK-30238
>                 URL: https://issues.apache.org/jira/browse/FLINK-30238
>             Project: Flink
>          Issue Type: Bug
>          Components: Connectors / Common
>    Affects Versions: 1.17.0, 1.15.3, 1.16.1
>            Reporter: Fabian Paul
>            Priority: Critical
>         Attachments: Screenshot 2023-03-09 at 1.47.11 PM.png, image (8).png
>
>
> During stop-with-savepoint the committer only commits the pending committables on notifyCheckpointComplete.
> This has several downsides.
>  * Last committableSummary has checkpoint id LONG.MAX and is never cleared from the state leading to that stop-with-savepoint does not work when the pipeline recovers from a savepoint 
>  * While the committables are committed during stop-with-savepoint they are not forwarded to post-commit topology, potentially losing data and preventing to close open transactions.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)