[jira] [Updated] (KAFKA-278) Issues partitioning a new topic

["Matt (Updated) (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Matt updated KAFKA-278:
-----------------------

    Description: 
There are two cases where correct partitioning fails for a new topic.

Case 1: Topic exists on current Kafka cluster. A newly added broker to the cluster will never be notified of the topic and will never host partitions.

To reproduce:
1) Create a cluster of brokers along with a ZooKeeper ensemble.
2) Send messages for a topic to the cluster.
3) Add a new broker to the cluster.
4) New broker will never see the existing topic.

Case 2: Topic does not exist on current Kafka cluster. Session in which producer sends first set of messages for new topic does not last long enough for bootstrapping. Partitions will not be allocated on all brokers.

To reproduce:
1) Create a cluster of brokers along with a ZooKeeper ensemble.
2) Send a "small" number of messages to a new topic, then shut down producer.
3) Partitions will be allocated on a random(?) set of brokers. Not all brokers will host partitions.
4) Brokers that did not allocate partitions for topic will never see the new topic.

For case 2, I'm not sure about the exact length of time, or the number of messages required before all brokers have partitions for the new topic.

  was:
If you add one or more brokers to an existing cluster, existing topics will never be seen by the new brokers.

To reproduce:
1) Create a cluster of brokers along with a ZooKeeper ensemble.
2) Send messages for a topic to the cluster.
3) Add a new broker to the cluster.
4) New broker will never see the existing topic.

        Summary: Issues partitioning a new topic  (was: Topic is never distributed to nodes added to an existing cluster)

I changed the description and title to reflect that there are 2 ways that this condition can arise.
                
> Issues partitioning a new topic
> -------------------------------
>
>                 Key: KAFKA-278
>                 URL: https://issues.apache.org/jira/browse/KAFKA-278
>             Project: Kafka
>          Issue Type: Bug
>          Components: core
>    Affects Versions: 0.7
>            Reporter: Matt
>            Priority: Minor
>
> There are two cases where correct partitioning fails for a new topic.
> Case 1: Topic exists on current Kafka cluster. A newly added broker to the cluster will never be notified of the topic and will never host partitions.
> To reproduce:
> 1) Create a cluster of brokers along with a ZooKeeper ensemble.
> 2) Send messages for a topic to the cluster.
> 3) Add a new broker to the cluster.
> 4) New broker will never see the existing topic.
> Case 2: Topic does not exist on current Kafka cluster. Session in which producer sends first set of messages for new topic does not last long enough for bootstrapping. Partitions will not be allocated on all brokers.
> To reproduce:
> 1) Create a cluster of brokers along with a ZooKeeper ensemble.
> 2) Send a "small" number of messages to a new topic, then shut down producer.
> 3) Partitions will be allocated on a random(?) set of brokers. Not all brokers will host partitions.
> 4) Brokers that did not allocate partitions for topic will never see the new topic.
> For case 2, I'm not sure about the exact length of time, or the number of messages required before all brokers have partitions for the new topic.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira