You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "John Roesler (Jira)" <ji...@apache.org> on 2020/02/07 16:19:00 UTC
[jira] [Commented] (KAFKA-8810) Add mechanism to detect topology
mismatch between streams instances
[ https://issues.apache.org/jira/browse/KAFKA-8810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17032509#comment-17032509 ]
John Roesler commented on KAFKA-8810:
-------------------------------------
This issue has been lurking for a while, and has been reported a number of different ways. It seems to take two forms:
1. changing the topology at all (in apparently compatible ways) can renumber operators and corrupt the application upon restart
2. changing the topology in combination with a rolling bounce results in members executing a different topology than the leader, which leads to extra problems (such as NPEs)
https://issues.apache.org/jira/browse/KAFKA-7669 is related, and seems to be more about just changing the topology at all
https://issues.apache.org/jira/browse/KAFKA-8307 proposes a fix
https://issues.apache.org/jira/browse/KAFKA-8810 seems to be a duplicate of KAFKA-8307
and https://issues.apache.org/jira/browse/KAFKA-9518 reports an exception that results from a rolling-bounce topology change.
> Add mechanism to detect topology mismatch between streams instances
> -------------------------------------------------------------------
>
> Key: KAFKA-8810
> URL: https://issues.apache.org/jira/browse/KAFKA-8810
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: Vinoth Chandar
> Priority: Major
>
> Noticed this while reading through the StreamsPartitionAssignor related code. If an user accidentally deploys a different topology on one of the instances, there is no mechanism to detect this and refuse assignment/take action. Given Kafka Streams is designed as an embeddable library, I feel this is rather an important scenario to handle. For e.g, kafka streams is embedded into a web front end tier and operators deploy a hot fix for a site issue to a few instances that are leaking memory and that accidentally also deploys some topology changes with it.
> Please feel free to close the issue, if its a duplicate. (Could not find a ticket for this)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)