[jira] [Created] (KAFKA-3663) Proposal for a kafka broker command - kafka-brokers.sh

["Jayesh Thakrar (JIRA)" <ji...@apache.org>]

Jayesh Thakrar created KAFKA-3663:
-------------------------------------

             Summary: Proposal for a kafka broker command - kafka-brokers.sh
                 Key: KAFKA-3663
                 URL: https://issues.apache.org/jira/browse/KAFKA-3663
             Project: Kafka
          Issue Type: Improvement
          Components: admin
            Reporter: Jayesh Thakrar


This is a proposal for an admin tool - say, kafka-brokers.sh to provide broker related useful information. Note that I could not see an option to create a child page at https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals.

So here are the details for the proposals.

*Motivation*
Some of the key succcess factor for Kafka's success are its performant architecture and operational simplicity. This is further complemented with a set of commandline tools and utilities for managing topics as well as testing/stress-testing. However currently Kafka lacks commands/tools to get a cluster and broker overview. Although it should be mentioned that Kafka does expose cluster information via API and broker metrics via JMX.

*Proposed Change*
This KIP is for a command, say kafka-brokers.sh that provides useful cluster and broker information. 

The command will essentially provide the following pieces of information:

* Cluster Overview Information
** Controller Broker Id (and version/epoch information)
** Broker Count
** Total Topic Count
** Total Partition Count

* Broker Information
** Broker Id
** Rack Id
** Hostname
** Endpoints (protocol, port)
** JMX port
** Topic count
** All partition count
** Leader partition count
** Under-replicated partition count
** Topic partitions (Name, *partition-) - 
An asterisk would indicate that the broker is the leader for that partition.
A hyphen/negative sign would indicates that the partition is not in-sync.


As you can see, the above information provides a view of the cluster and brokers that complements kafka-topics.sh.

This command can be further evolved to do more things like:
- Drain one or more brokers for decommissioning. This feature would allow distributing off all partitions of a list of brokers to other brokers in the cluster (need to be cognizant of rack configuration). 
- Onboard one or more new brokers. This feature would do the reverse of removing brokers and allow controlled distribution of partition to the new brokers.
- Obtain JMX metrics from a broker (see KIP for its usage).
- See KAFKA-3649 for other features




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)