You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by "Dominik Wosiński (Jira)" <ji...@apache.org> on 2021/02/01 00:29:02 UTC
[jira] [Created] (FLINK-21219) FlinkKafkaConsumer ignores offset
overrides for new topics when restoring from savepoint.
Dominik Wosiński created FLINK-21219:
----------------------------------------
Summary: FlinkKafkaConsumer ignores offset overrides for new topics when restoring from savepoint.
Key: FLINK-21219
URL: https://issues.apache.org/jira/browse/FLINK-21219
Project: Flink
Issue Type: Bug
Components: Connectors / Kafka
Affects Versions: 1.12.1
Reporter: Dominik Wosiński
Currently, when FlinkKafkaConsumer is restored from savepoint, the following code will handle topics that do not have offsets committed (for example if a new topic was added):
{noformat}
if (restoredState != null) { for (KafkaTopicPartition partition : allPartitions) { if (!restoredState.containsKey(partition)) { restoredState.put(partition, KafkaTopicPartitionStateSentinel.EARLIEST_OFFSET); } }{noformat}
So if we have a KafkaConsumer with topicPattern and the pattern is changed, new topis will always start from earliest offset, even if originally the setting was different.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)