[jira] [Created] (KAFKA-4415) Reduce time to create and send MetadataUpdateRequest

["Dong Lin (JIRA)" <ji...@apache.org>]

Dong Lin created KAFKA-4415:
-------------------------------

             Summary: Reduce time to create and send MetadataUpdateRequest
                 Key: KAFKA-4415
                 URL: https://issues.apache.org/jira/browse/KAFKA-4415
             Project: Kafka
          Issue Type: Bug
            Reporter: Dong Lin


As of current implementation, when controller receives ControlledShutdownRequest, it will 1) for every broker in the cluster, for every partition on the broker which wants to shutdown, create an instance of PartitionStateInfo and add it to ControllerBrokerRequestBatch.,updateMetadataRequestMap; and 2) for every broker, for every follower partitions on the broker which wants to shutdown, send one MetadataUpdateRequst to that broker.

In order to shutdown a broker, the controller will need to instantiate O(partitionNum * brokerNum) PartitionStateInfo and send O(partitionNum * brokerNum) partitionStateInfo. This is not efficient. The broker should only need to instantiate O(partitionNum) PartitionStateInfo and send O(brokerNum) MetadataUpdateRequest.

Micro-benchmark results show that this optimization can reduce the time of processing ControlledShutdownRequest by 30%.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)