[GitHub] [flink] becketqin commented on a change in pull request #14531: [FLINK-20777][Connector][Kafka] Property "partition.discovery.interval.ms" shoule be enabled by default for unbounded mode, and disable for bounded mode

[GitBox <gi...@apache.org>]

becketqin commented on a change in pull request #14531:
URL: https://github.com/apache/flink/pull/14531#discussion_r553402067



##########
File path: flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java
##########
@@ -458,6 +458,29 @@ private boolean maybeOverride(String key, String value, boolean override) {
         return overridden;
     }
 
+    private boolean maybeOverridePartitionDiscovery(String key, String value, boolean override) {

Review comment:
       It looks that we should only disable auto discovery here in the following two cases:
   1. The source is bounded.
   2. The source has stopping offsets and the user did not explicitly set the discovery interval. 
   
   In other cases, we don't need to do anything. So the logic here could just become something like:
   ```
   boolean hasStoppingOffsets = !(stoppingOffsetsInitializer instanceof NoStoppingOffsetsInitializer);
   boolean hasParitionDiscoverySetting = props.getProperty(KafkaSourceOptions.PARTITION_DISCOVERY_INTERVAL_MS.key()) != null;
   if (boundedness == Bounded || 
   (hasStoppingOffsets && !hasParititionDiscoverySetting)) {
       props.set(KafkaSourceOptions.PARTITION_DISCOVERY_INTERVAL_MS.key(-1))
   }
   ```
   It seems easier to understand.
   
   BTW, can we also add a unit test for this?
   




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org