[jira] [Created] (KAFKA-10646) Support dynamic config for "delete.topic.enable"

["Prateek Agarwal (Jira)" <ji...@apache.org>]

Prateek Agarwal created KAFKA-10646:
---------------------------------------

             Summary: Support dynamic config for "delete.topic.enable"
                 Key: KAFKA-10646
                 URL: https://issues.apache.org/jira/browse/KAFKA-10646
             Project: Kafka
          Issue Type: Improvement
          Components: core
    Affects Versions: 1.1.2, 2.5.2
            Reporter: Prateek Agarwal


Topic deletion in Kafka removes data and the operation is not reversible (i.e. there is no "undelete" operation). Hence, we keep the flag "delete.topic.enable" as "False" in our configs, so that someone accidentally doesn't delete a topic, which can lead to incidents.

But sometimes, there are legit client use-cases, where they deprecate a topic and those topics need to be deleted from the cluster. Currently, the process to do this operation is very cumbersome:
{code:java}
1. Change the `server.properties` config on each broker with: `delete.topic.enable=true`
2. Roll the cluster
3. Run Kafka Admin commands to delete the topics
4. Revert the `server.properties` config change for `delete.topic.enable`
5. Roll the cluster again{code}
There are other hacky workarounds as well, like:
{code:java}
1. Set Topic retention of the topic-to-be-deleted to a small value to flush the data
2. Invoke `zkcli.sh rmr /brokers/topic/topic-to-be-deleted`
3. When Topic metadata is no longer available, rm the topic dirs on the brokers{code}
The above process is pretty risky and can lead to unavailability of one or more topics if any mistake happens in the commands.

 

Proposed solution:
{code:java}
Make the `delete.topic.enable` config dynamic so that the flag can be modified without broker restarts {code}
 

Example runs:

1) Enable Topic Deletion Cluster wide
{code:java}
bin/kafka-configs.sh --bootstrap-server localhost:9092 \ 
                     --alter --add-config 'delete.topic.enable=true' \
                     --entity-default --entity-type brokers{code}
2) Enable Topic Deletion on Single Broker with broker ID: 0
{code:java}
bin/kafka-configs.sh --bootstrap-server localhost:9092 \ 
                     --alter --add-config 'delete.topic.enable=true' \
                     --entity-name 0 --entity-type brokers {code}
 

This will be similar to other broker dynamic configs currently available like {{log.cleaner.threads}} which can be defined {{cluster-wide}} OR on a per-broker level.

The precedence will look like so:
 * Dynamic per-broker config stored in ZooKeeper
 * Dynamic cluster-wide default config stored in ZooKeeper
 * Static broker config from {{server.properties}}

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)