You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flink.apache.org by "Fabian Paul (Jira)" <ji...@apache.org> on 2022/02/18 14:21:00 UTC

[jira] [Created] (FLINK-26254) KafkaSink might violate order of sequence numbers and risk exactly-once processing

Fabian Paul created FLINK-26254:
-----------------------------------

             Summary: KafkaSink might violate order of sequence numbers and risk exactly-once processing
                 Key: FLINK-26254
                 URL: https://issues.apache.org/jira/browse/FLINK-26254
             Project: Flink
          Issue Type: Bug
          Components: Connectors / Kafka
    Affects Versions: 1.14.3, 1.15.0
            Reporter: Fabian Paul


When running the KafkaSink in exactly-once mode with a very low checkpoint interval users are seeing `OutOfOrderSequenceException`.

It could be caused by the fact that the connector has a pool of KafkaProducers and the sequence numbers are not shared/reset if a new KafkaProducer tries to write to a partition while the previous KafkaProducer is still occupied for committing.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)