You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2022/01/05 07:18:31 UTC
[GitHub] [flink] ashulin commented on a change in pull request #17601: [FLINK-24697][flink-connectors-kafka] add auto.offset.reset configuration for group-offsets startup mode
ashulin commented on a change in pull request #17601:
URL: https://github.com/apache/flink/pull/17601#discussion_r777856618
##########
File path: flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/streaming/connectors/kafka/table/KafkaDynamicSource.java
##########
@@ -393,7 +396,16 @@ public int hashCode() {
kafkaSourceBuilder.setStartingOffsets(OffsetsInitializer.latest());
break;
case GROUP_OFFSETS:
- kafkaSourceBuilder.setStartingOffsets(OffsetsInitializer.committedOffsets());
+ String offsetResetConfig =
+ properties.getProperty(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG);
+ if (offsetResetConfig == null) {
Review comment:
Maybe you can get a default value `none` and avoid judging null.
##########
File path: flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/streaming/connectors/kafka/table/KafkaDynamicSource.java
##########
@@ -393,7 +396,16 @@ public int hashCode() {
kafkaSourceBuilder.setStartingOffsets(OffsetsInitializer.latest());
break;
case GROUP_OFFSETS:
- kafkaSourceBuilder.setStartingOffsets(OffsetsInitializer.committedOffsets());
+ String offsetResetConfig =
+ properties.getProperty(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG);
+ if (offsetResetConfig == null) {
+ kafkaSourceBuilder.setStartingOffsets(OffsetsInitializer.committedOffsets());
+ } else {
+ kafkaSourceBuilder.setStartingOffsets(
+ OffsetsInitializer.committedOffsets(
+ OffsetResetStrategy.valueOf(
+ offsetResetConfig.toUpperCase(Locale.ROOT))));
Review comment:
`offsetResetConfig` may be a wrong value and needs to be verified.
##########
File path: flink-connectors/flink-connector-kafka/src/test/java/org/apache/flink/streaming/connectors/kafka/table/KafkaDynamicTableFactoryTest.java
##########
@@ -380,6 +381,44 @@ public void testTableSourceCommitOnCheckpointDisabled() {
.noDefaultValue()));
}
+ @Test
+ public void testTableSourceSetOffsetReset() {
+ testSetOffsetResetForStartFromGroupOffsets(null);
+ testSetOffsetResetForStartFromGroupOffsets("none");
+ testSetOffsetResetForStartFromGroupOffsets("earliest");
+ testSetOffsetResetForStartFromGroupOffsets("latest");
+ }
+
+ private void testSetOffsetResetForStartFromGroupOffsets(String value) {
+ final Map<String, String> modifiedOptions =
+ getModifiedOptions(
+ getBasicSourceOptions(),
+ options -> {
+ options.remove("scan.startup.mode");
+ if (value != null) {
Review comment:
Like the code above, null judgment can be avoided.
##########
File path: flink-connectors/flink-connector-kafka/src/test/java/org/apache/flink/streaming/connectors/kafka/table/KafkaDynamicTableFactoryTest.java
##########
@@ -380,6 +381,44 @@ public void testTableSourceCommitOnCheckpointDisabled() {
.noDefaultValue()));
}
+ @Test
+ public void testTableSourceSetOffsetReset() {
Review comment:
> NIT: We can use parameterized test here after migrating to JUnit 5 😄
The current version of junit-jupiter is 5.8.1, which supports parameterized testing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org