[jira] [Updated] (FLINK-26417) Async Sink can violate At Least Once during checkpointing

["Danny Cranmer (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-26417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Danny Cranmer updated FLINK-26417:
----------------------------------
    Summary: Async Sink can violate At Least Once during checkpointing  (was: Async can Sink violate At Least Once during checkpointing)

> Async Sink can violate At Least Once during checkpointing
> ---------------------------------------------------------
>
>                 Key: FLINK-26417
>                 URL: https://issues.apache.org/jira/browse/FLINK-26417
>             Project: Flink
>          Issue Type: Bug
>          Components: Connectors / Common
>            Reporter: Danny Cranmer
>            Assignee: Ahmed Hamdy
>            Priority: Blocker
>             Fix For: 1.15.0
>
>
> Async Sink holds data to send in 2 places:
> - Buffered in queue
> - "In-flight", being submitted by the async writer
> When checkpointing we only capture the buffered requests. It seems like the code to [wait for in-flight requests to complete|https://github.com/apache/flink/blob/aab253a9141f1cdff81658eb47c7bb6ee9d2105c/flink-connectors/flink-connector-base/src/main/java/org/apache/flink/connector/base/sink/writer/AsyncSinkWriter.java#L402-L420] has been removed/
> Reinstate this mechanism to wait for in-flight requests to flush within {{prepareCommit}}. There is even. [comment|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-base/src/main/java/org/apache/flink/connector/base/sink/writer/AsyncSinkWriter.java#L45] to state this is supported.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)