You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "Manikumar Reddy (JIRA)" <ji...@apache.org> on 2015/06/18 17:37:01 UTC
[jira] [Comment Edited] (KAFKA-2265) creating a topic with large
number of partitions takes a long time
[ https://issues.apache.org/jira/browse/KAFKA-2265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14591976#comment-14591976 ]
Manikumar Reddy edited comment on KAFKA-2265 at 6/18/15 3:36 PM:
-----------------------------------------------------------------
For each partition, we are parsing /brokers/topic/ zk node for latest partition replica assignment data. This will take time when we have large number of partitions. In TopicChangeListener , we are adding latest partition replica assignment to controllerContext.partitionReplicaAssignment. So we no need to read this partition info again. Similar code added to AddPartitionsListener. Removed assignReplicasToPartitions method and its usage.
was (Author: omkreddy):
For each partition, we are parsing /brokers/topic/ zk node for latest partition assignment data. This will take time when we have large number of partitions. In TopicChangeListener , we are adding latest partition Assignment to controllerContext.partitionReplicaAssignment. So we no need to read this partition info again. Similar code added to AddPartitionsListener. Removed assignReplicasToPartitions method and its usage.
> creating a topic with large number of partitions takes a long time
> ------------------------------------------------------------------
>
> Key: KAFKA-2265
> URL: https://issues.apache.org/jira/browse/KAFKA-2265
> Project: Kafka
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.8.2.1
> Reporter: Jun Rao
> Assignee: Manikumar Reddy
> Fix For: 0.8.3
>
> Attachments: KAFKA-2265.patch
>
>
> Currently, creating a topic with 3K partitions can take 15 mins. We should be able to do that much faster. There is perhaps some redundant accesses to ZK during topic creation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)