[jira] [Commented] (KAFKA-14418) Add safety checks for modifying partitions of __consumer_offsets

["A. Sophie Blee-Goldman (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-14418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17647822#comment-17647822 ] 

A. Sophie Blee-Goldman commented on KAFKA-14418:
------------------------------------------------

[~divijvaidya] I just went ahead and removed the target version for this one since we are past code freeze for 3.4 and it didn't seem clear whether to target 3.5 based on your PR comment

> Add safety checks for modifying partitions of __consumer_offsets
> ----------------------------------------------------------------
>
>                 Key: KAFKA-14418
>                 URL: https://issues.apache.org/jira/browse/KAFKA-14418
>             Project: Kafka
>          Issue Type: Improvement
>            Reporter: Divij Vaidya
>            Priority: Major
>
> Today a user can change the number of partitions of {{{}{}}}{{{}_consumer_{}}}{{{}_{}}}{{{}offsets{}}} topic by changing the configuration value for {{offsets.topic.num.partitions}} or manually by using CreatePartition API. For example, the following command changes the number of partitions to 60 for __consumer_offsets topic.
> {code:java}
> ./bin/kafka-topics.sh --topic __consumer_offsets --alter --partitions 60 --bootstrap-server=localhost:9092{code}
> Changing offsets of this reserved partition leads to problems with consumer groups unless you restart all brokers.  Thus, there is a high probability that is this operations is not done right, users may shoot themselves in the foot. Example scenario: [https://stackoverflow.com/questions/73944561/kafka-consumer-group-coordinator-inconsistent] 
> To remedy this, I propose the following changes:
> 1. `kafka-topic.sh` should explicitly block adding new partitions to internal topics.
> 2. Add an operational guide to the docs for safely modifying partitions of __consumer_offsets at [https://kafka.apache.org/documentation.html#basic_ops] 
> 3. when \{{offsets.topic.num.partitions }}is modified, it should update the value of groupPartitionCount at [https://github.com/apache/kafka/blob/3.3.1/core/src/main/scala/kafka/coordinator/group/GroupMetadataManager.scala#L192] without a broker startup



--
This message was sent by Atlassian Jira
(v8.20.10#820010)