You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Aljoscha Krettek (JIRA)" <ji...@apache.org> on 2018/09/14 15:23:00 UTC
[jira] [Closed] (FLINK-10342) Kafka duplicate topic consumption
when topic name is changed
[ https://issues.apache.org/jira/browse/FLINK-10342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aljoscha Krettek closed FLINK-10342.
------------------------------------
Resolution: Not A Problem
I think this is working as intended. The discoverer is responsible for discovering new topics according to the topic pattern. It's not used to filter out topics that the consumer is already consuming. When restoring, the already discovered topics are in state and the consumer is expected to read from them.
If you want to change the topic name and have the consumer only consume from the new topic you can change the uid of the Kafka Source using {{uid(String)}} on the source. That way it will be considered as a new source that will not restore the state of the previous consumer.
Please reopen this issue if you disagree.
> Kafka duplicate topic consumption when topic name is changed
> ------------------------------------------------------------
>
> Key: FLINK-10342
> URL: https://issues.apache.org/jira/browse/FLINK-10342
> Project: Flink
> Issue Type: Bug
> Reporter: Oleksandr Nitavskyi
> Assignee: Oleksandr Nitavskyi
> Priority: Major
>
> In case of topic name is simply renamed for a KafkaConsumer Flink starts to consume from old and a new topic in the same time which can lead to unexpected behavior.
> Here is the PR with reproduce: https://github.com/apache/flink/pull/6691
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)