[jira] [Commented] (FLINK-26417) Async Sink can violate At Least Once processing during checkpointing

["Danny Cranmer (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-26417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17499944#comment-17499944 ] 

Danny Cranmer commented on FLINK-26417:
---------------------------------------

Resolving as not a problem. Was a false alarm since sink does indeed wait for in-flight requests via another code path

> Async Sink can violate At Least Once processing during checkpointing
> --------------------------------------------------------------------
>
>                 Key: FLINK-26417
>                 URL: https://issues.apache.org/jira/browse/FLINK-26417
>             Project: Flink
>          Issue Type: Bug
>          Components: Connectors / Common
>            Reporter: Danny Cranmer
>            Assignee: Ahmed Hamdy
>            Priority: Not a Priority
>              Labels: pull-request-available
>
> Async Sink holds data to send in 2 places:
> - Buffered in queue
> - "In-flight", being submitted by the async writer
> When checkpointing we only capture the buffered requests. It seems like the code to [wait for in-flight requests to complete|https://github.com/apache/flink/blob/aab253a9141f1cdff81658eb47c7bb6ee9d2105c/flink-connectors/flink-connector-base/src/main/java/org/apache/flink/connector/base/sink/writer/AsyncSinkWriter.java#L402-L420] has been removed/
> Reinstate this mechanism to wait for in-flight requests to flush within {{prepareCommit}}. There is even. [comment|https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-base/src/main/java/org/apache/flink/connector/base/sink/writer/AsyncSinkWriter.java#L45] to state this is supported.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)