[jira] [Updated] (KAFKA-14418) Add safety checks for modifying partitions of __consumer_offsets

["A. Sophie Blee-Goldman (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-14418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

A. Sophie Blee-Goldman updated KAFKA-14418:
-------------------------------------------
    Fix Version/s:     (was: 3.4.0)

> Add safety checks for modifying partitions of __consumer_offsets
> ----------------------------------------------------------------
>
>                 Key: KAFKA-14418
>                 URL: https://issues.apache.org/jira/browse/KAFKA-14418
>             Project: Kafka
>          Issue Type: Improvement
>            Reporter: Divij Vaidya
>            Priority: Major
>
> Today a user can change the number of partitions of {{{}{}}}{{{}_consumer_{}}}{{{}_{}}}{{{}offsets{}}} topic by changing the configuration value for {{offsets.topic.num.partitions}} or manually by using CreatePartition API. For example, the following command changes the number of partitions to 60 for __consumer_offsets topic.
> {code:java}
> ./bin/kafka-topics.sh --topic __consumer_offsets --alter --partitions 60 --bootstrap-server=localhost:9092{code}
> Changing offsets of this reserved partition leads to problems with consumer groups unless you restart all brokers.  Thus, there is a high probability that is this operations is not done right, users may shoot themselves in the foot. Example scenario: [https://stackoverflow.com/questions/73944561/kafka-consumer-group-coordinator-inconsistent] 
> To remedy this, I propose the following changes:
> 1. `kafka-topic.sh` should explicitly block adding new partitions to internal topics.
> 2. Add an operational guide to the docs for safely modifying partitions of __consumer_offsets at [https://kafka.apache.org/documentation.html#basic_ops] 
> 3. when \{{offsets.topic.num.partitions }}is modified, it should update the value of groupPartitionCount at [https://github.com/apache/kafka/blob/3.3.1/core/src/main/scala/kafka/coordinator/group/GroupMetadataManager.scala#L192] without a broker startup



--
This message was sent by Atlassian Jira
(v8.20.10#820010)