You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2021/08/26 10:24:05 UTC

[GitHub] [flink] becketqin commented on a change in pull request #16870: [FLINK-23773][connector/kafka] Mark empty splits as finished to cleanup states in SplitFetcher

becketqin commented on a change in pull request #16870:
URL: https://github.com/apache/flink/pull/16870#discussion_r696486424



##########
File path: flink-connectors/flink-connector-base/src/main/java/org/apache/flink/connector/base/source/reader/fetcher/SplitFetcherManager.java
##########
@@ -78,6 +79,8 @@
     /** Indicating the split fetcher manager has closed or not. */
     private volatile boolean closed;
 
+    private final Consumer<Collection<String>> splitFinishedHook;

Review comment:
       nit: a java doc explaining the purpose would be helpful.

##########
File path: flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSource.java
##########
@@ -120,6 +124,13 @@ public Boundedness getBoundedness() {
     @Override
     public SourceReader<OUT, KafkaPartitionSplit> createReader(SourceReaderContext readerContext)
             throws Exception {
+        return createReader(readerContext, (ignore) -> {});
+    }
+
+    @VisibleForTesting
+    public SourceReader<OUT, KafkaPartitionSplit> createReader(

Review comment:
       It'll be good if we can make this package private so end users won't see it. I am wondering if it would make sense to have a `KafkaSourceTestUtil` class in the same package, which has a public method to create the reader with a split finish hook. The tests can use the test util regardless of the package they are in. This will avoid exposing this method to the end users.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org