Random Broker/Partition for Producer

[Jared Wong <ja...@MIT.EDU>]

Hi,

I want to randomly send my messages to a random broker and a random
partition registered in Zookeeper.

If, in my producer configuration, I only specify zk.connect and I
don't specify any broker.list or other information, will this happen?

Best,
Jared
-- 
Jared Wong :: jaredw@mit.edu

Re: Random Broker/Partition for Producer

[Tejas Patil <te...@gmail.com>]

Nope. The property "metadata.broker.list" in producer.properties [0] is
just used to fetch the metadata about which broker owns which
[topic-partition]. Based on this info, the producer data is routed to the
relevant leader broker.

By default Kafka uses DefaultPartitioner [1] which is used to randomly
assign [topic-partitions] amongst the available brokers. Once a broker is
fixed as a leader for a [topic-partition], all messages for that
[topic-partition] will end up to that fixed broker (until that broker dies,
re-balancing occurs or you run the re-assign partitions tool). I believe
that this cannot be changed via any config.

[0] : https://github.com/apache/kafka/blob/0.8/config/producer.properties
[1] :
https://github.com/apache/kafka/blob/0.8/core/src/main/scala/kafka/producer/DefaultPartitioner.scala


On Wed, Jul 31, 2013 at 1:47 PM, Jared Wong <ja...@mit.edu> wrote:

> Hi,
>
> I want to randomly send my messages to a random broker and a random
> partition registered in Zookeeper.
>
> If, in my producer configuration, I only specify zk.connect and I
> don't specify any broker.list or other information, will this happen?
>
> Best,
> Jared
> --
> Jared Wong :: jaredw@mit.edu
>

Re: Random Broker/Partition for Producer

[Jun Rao <ju...@gmail.com>]

I assume that this is for 0.7. It will do what you want, send your message
to a random partition in a random broker.

Thanks,

Jun


On Wed, Jul 31, 2013 at 1:47 PM, Jared Wong <ja...@mit.edu> wrote:

> Hi,
>
> I want to randomly send my messages to a random broker and a random
> partition registered in Zookeeper.
>
> If, in my producer configuration, I only specify zk.connect and I
> don't specify any broker.list or other information, will this happen?
>
> Best,
> Jared
> --
> Jared Wong :: jaredw@mit.edu
>