[jira] [Created] (KAFKA-12280) Consider internal topic configuration when created through `CreateTopic`

["Jason Gustafson (Jira)" <ji...@apache.org>]

Jason Gustafson created KAFKA-12280:
---------------------------------------

             Summary: Consider internal topic configuration when created through `CreateTopic`
                 Key: KAFKA-12280
                 URL: https://issues.apache.org/jira/browse/KAFKA-12280
             Project: Kafka
          Issue Type: Improvement
            Reporter: Jason Gustafson


Currently internal topics such as __consumer_offsets and __transaction_state are created dynamically when we receive a `FindCoordinator` request from a client. It is also possible to create these topics manually using the `CreateTopic` API. We leverage this capability in some integration test cases, but I am not sure if it is generally useful.

The interesting thing when these topics are created manually is which topic configuration will be used. Currently, if the request itself does not indicate overrides, then the broker will apply the standard topic defaults. It seems like it would be more logical to use the explicit configuration overrides such as `offsets.topic.num.partitions`.

I think this raises a few questions:

1. Do we want to support manual creation of the internal topics going forward? I think we have found this convenient in performance testing when we did not want to conflate internal topic creation overhead with consumer performance. 
2. If we do want to support it, the next question is if we should allow the configurations like `offsets.topic.num.partitions` to be overridden? This does create some complexity in `GroupMetadataManager` since we currently don't support re-partitioning of this topic.
3. Finally, assuming we still support this, we probably at a minimum should change the behavior to use the overrides from the configuration.




--
This message was sent by Atlassian Jira
(v8.3.4#803005)