[jira] [Commented] (KAFKA-4415) Reduce time to create and send MetadataUpdateRequest

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15669497#comment-15669497 ] 

ASF GitHub Bot commented on KAFKA-4415:
---------------------------------------

GitHub user lindong28 opened a pull request:

    https://github.com/apache/kafka/pull/2141

    KAFKA-4415; Reduce time to create and send MetadataUpdateRequest

    

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/lindong28/kafka KAFKA-4415

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/kafka/pull/2141.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2141
    
----
commit a317b6de574842298015121b8cd836512728141e
Author: Dong Lin <li...@gmail.com>
Date:   2016-11-16T05:30:03Z

    KAFKA-4415; Reduce time to create and send MetadataUpdateRequest

----


> Reduce time to create and send MetadataUpdateRequest
> ----------------------------------------------------
>
>                 Key: KAFKA-4415
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4415
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Dong Lin
>
> As of current implementation, when controller receives ControlledShutdownRequest, it will 1) for every broker in the cluster, for every partition on the broker which wants to shutdown, create an instance of PartitionStateInfo and add it to ControllerBrokerRequestBatch.,updateMetadataRequestMap; and 2) for every broker, for every follower partitions on the broker which wants to shutdown, send one MetadataUpdateRequst to that broker.
> In order to shutdown a broker, the controller will need to instantiate O(partitionNum * brokerNum) PartitionStateInfo and send O(partitionNum * brokerNum) partitionStateInfo. This is not efficient. The broker should only need to instantiate O(partitionNum) PartitionStateInfo and send O(brokerNum) MetadataUpdateRequest.
> Micro-benchmark results show that this optimization can reduce the time of processing ControlledShutdownRequest by 30%.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)