You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flink.apache.org by "Jiabao Sun (Jira)" <ji...@apache.org> on 2023/02/14 10:49:00 UTC

[jira] [Created] (FLINK-31063) Prevent duplicate reading when restoring from a checkpoint.

Jiabao Sun created FLINK-31063:
----------------------------------

             Summary: Prevent duplicate reading when restoring from a checkpoint.
                 Key: FLINK-31063
                 URL: https://issues.apache.org/jira/browse/FLINK-31063
             Project: Flink
          Issue Type: Bug
          Components: Connectors / MongoDB
    Affects Versions: mongodb-1.0.0
            Reporter: Jiabao Sun
             Fix For: mongodb-1.0.0


Exact-once semantics may not be guaranteed at present on partial reads.
We use a number fetchSize to limit the records count for every fetch loop but we didn't record the offset into the split state. When resuming the split reader from a partially completed split, we may re-read some data.

We should record the current reading offset into split state.
Skip this offset when restoring to prevent duplicate reading.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)