Writing to Kafka

[Telles Nobrega <te...@gmail.com>]

Hi, 

I started using kafka with Samza. I’m trying to run a test that is supposed to create messages and write to a kafka topic. 
In my test I start writing a small amount of messages and it should grow up to 33200/sec. 

1. With this amount of messages can one broker handle the messages or should I use more than one?
2. Using more than one broker, the producer should have a list with brokers or just point to one and the messages sent to a different broker?
3. I have a java producer that just creates numbers and sends them to kafka, can kafka handle many producers writing to the same topic at once?

Thanks,

Re: Writing to Kafka

[Jay Kreps <ja...@gmail.com>]

Hi Telles,

1. One broker is probably fine with the load, though if you want
replication for fault tolerance you will need more than one.
2. The host/port you configure is just to discover the full cluster
topology. Messages will be partitioned semantically and balanced over
all hosts that have partitions for the topics you interact with.
3. Yes, Kafka is multi-writer.

-Jay

On Thu, Aug 14, 2014 at 12:03 PM, Telles Nobrega
<te...@gmail.com> wrote:
> Hi,
>
> I started using kafka with Samza. I’m trying to run a test that is supposed to create messages and write to a kafka topic.
> In my test I start writing a small amount of messages and it should grow up to 33200/sec.
>
> 1. With this amount of messages can one broker handle the messages or should I use more than one?
> 2. Using more than one broker, the producer should have a list with brokers or just point to one and the messages sent to a different broker?
> 3. I have a java producer that just creates numbers and sends them to kafka, can kafka handle many producers writing to the same topic at once?
>
> Thanks,
>