[jira] [Commented] (FLINK-29459) Sink v2 has bugs in supporting legacy v1 implementations with global committer

["Fabian Paul (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-29459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17612980#comment-17612980 ] 

Fabian Paul commented on FLINK-29459:
-------------------------------------

[~gaoyunhaii] thanks for your analysis. I am currently looking into the issues, and I think it is a good idea to split the different problems into different tickets. 

I already created https://issues.apache.org/jira/browse/FLINK-29509 to fix the subtask id problem during recovery.

Let me know if you have already started with that.

> Sink v2 has bugs in supporting legacy v1 implementations with global committer
> ------------------------------------------------------------------------------
>
>                 Key: FLINK-29459
>                 URL: https://issues.apache.org/jira/browse/FLINK-29459
>             Project: Flink
>          Issue Type: Bug
>          Components: API / DataStream
>    Affects Versions: 1.16.0, 1.17.0, 1.15.2
>            Reporter: Yun Gao
>            Assignee: Yun Gao
>            Priority: Major
>             Fix For: 1.17.0, 1.15.3, 1.16.1
>
>
> Currently when supporting Sink implementation using version 1 interface, there are issues after restoring from a checkpoint after failover:
>  # In global committer operator, when restoring SubtaskCommittableManager, the subtask id is replaced with the one in the current operator. This means that the id originally is the id of the sender task (0 ~ N - 1), but after restoring it has to be 0. This would cause Duplication Key exception during restoring.
>  # For Committer operator, the subtaskId of CheckpointCommittableManagerImpl is always restored to 0 after failover for all the subtasks. This makes the summary sent to the Global Committer is attached with wrong subtask id.
>  # For Committer operator, the checkpoint id of SubtaskCommittableManager is always restored to 1 after failover, this make the following committable sent to the global committer is attached with wrong checkpoint id. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)