[jira] [Comment Edited] (KAFKA-1300) Added WaitForReplaction admin tool.

["Alexis Midon (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-1300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14059512#comment-14059512 ] 

Alexis Midon edited comment on KAFKA-1300 at 7/11/14 11:46 PM:
---------------------------------------------------------------

Consiering that Kafka is designed to handle some replication lag, if you need to shutdown a broker it does not seem very useful to wait for the replica lag to be zero.
(If the broker is X messages behind, and my maintenance requires Y=F(message throughput) minutes, I can safely shutdown the broker is X+Y < replica.lag.max.messages.

So maybe that command will be more useful if it could take an argument that characterize X, i.e. how far behind can the broker be before a shutdown.


was (Author: alexismidon):
Consiering that Kafka is designed to handle some replication lag, if you need to shutdown a broker it does not seem very useful to wait for the replica lag to be zero.
(If the broker is X messages behind, and my maintenance requires F(message throughput) minutes, I can safely shutdown the broker is X+ throughput*Y < replica.lag.max.messages.

So maybe that command will be more useful if it could take an argument that characterize X, i.e. how far behind can the broker be before a shutdown.

> Added WaitForReplaction admin tool.
> -----------------------------------
>
>                 Key: KAFKA-1300
>                 URL: https://issues.apache.org/jira/browse/KAFKA-1300
>             Project: Kafka
>          Issue Type: New Feature
>          Components: tools
>    Affects Versions: 0.8.0
>         Environment: Ubuntu 12.04
>            Reporter: Brenden Matthews
>              Labels: patch
>             Fix For: 0.8.1
>
>         Attachments: 0001-Added-WaitForReplaction-admin-tool.patch
>
>
> I have created a tool similar to the broker shutdown tool for doing rolling restarts of Kafka clusters.
> The tool watches the max replica lag of the specified broker, and waits until the lag drops to 0 before exiting.
> To do a rolling restart, here's the process we use:
> for (broker <- brokers) {
>   run shutdown tool for broker
>   terminate broker
>   start new broker
>   run wait for replication tool on new broker
> }
> Here's an example command line use:
> ./kafka-run-class.sh kafka.admin.WaitForReplication --zookeeper zk.host.com:2181 --num.retries 100 --retry.interval.ms 60000 --broker 0



--
This message was sent by Atlassian JIRA
(v6.2#6252)