[jira] [Created] (FLINK-28054) Introduce Source API utilities to control records emitted per checkpoint

["Alexander Fedulov (Jira)" <ji...@apache.org>]

Alexander Fedulov created FLINK-28054:
-----------------------------------------

             Summary: Introduce Source API utilities to control records emitted per checkpoint
                 Key: FLINK-28054
                 URL: https://issues.apache.org/jira/browse/FLINK-28054
             Project: Flink
          Issue Type: Sub-task
            Reporter: Alexander Fedulov


Some connectors require verifying the content of a checkpoint after processing specific records. See [1] for more details. An abstraction similar to what is currently done with the SourceFunction in this [2] test is required. 

Idea: SourceReader interface has notifyCheckpointComplete() callback, so a higher level implementation could be provided to emit records in lockstep with checkpoints.

 

[1]https://lists.apache.org/thread/9kfppzbq7r2gk5mhykc7m786sv0fc8j2 [2]https://github.com/apache/iceberg/blob/master/flink/v1.15/flink/src/test/java/org/apache/iceberg/flink/source/BoundedTestSource.java

 



--
This message was sent by Atlassian Jira
(v8.20.7#820007)