[jira] [Updated] (KAFKA-15372) MM2 rolling restart can drop configuration changes silently

["Greg Harris (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-15372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Greg Harris updated KAFKA-15372:
--------------------------------
    Issue Type: Bug  (was: Improvement)

> MM2 rolling restart can drop configuration changes silently
> -----------------------------------------------------------
>
>                 Key: KAFKA-15372
>                 URL: https://issues.apache.org/jira/browse/KAFKA-15372
>             Project: Kafka
>          Issue Type: Bug
>          Components: mirrormaker
>            Reporter: Daniel Urban
>            Priority: Major
>
> When MM2 is restarted, it tries to update the Connector configuration in all flows. This is a one-time trial, and fails if the Connect worker is not the leader of the group.
> In a distributed setup and with a rolling restart, it is possible that for a specific flow, the Connect worker of the just restarted MM2 instance is not the leader, meaning that Connector configurations can get dropped.
> For example, assuming 2 MM2 instances, and one flow A->B:
>  # MM2 instance 1 is restarted, the worker inside MM2 instance 2 becomes the leader of A->B Connect group.
>  # MM2 instance 1 tries to update the Connector configurations, but fails (instance 2 has the leader, not instance 1)
>  # MM2 instance 2 is restarted, leadership moves to worker in MM2 instance 1
>  # MM2 instance 2 tries to update the Connector configurations, but fails
> At this point, the configuration changes before the restart are never applied. Many times, this can also happen silently, without any indication.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)