Producer Performance Test Mesage Count Mismatch

[Andrew Psaltis <An...@Webtrends.com>]

I am using kafaka-0.7.0-incubating and when I run the producer perf test, the total reported messages does not match the number of messages passed in on the command line.

Here is the command:


bin/kafka-producer-perf-test.sh --brokerinfo broker.list=0:localhost:9092 --message-size=1504 --async --batch-size=200 --compression-codec 1 --topic test --messages=50000 --threads=11



And here is the output:

Total Num Messages: 49995 bytes: 75192480 in 2.538 secs (kafka.tools.ProducerPerformance$)


Is there something that I am not understanding about the producer, or the combination of the number of messages and threads I am using that causes the discrepancy?

Thanks in advance,
Andrew

Re: Producer Performance Test Mesage Count Mismatch

[Jay Kreps <ja...@gmail.com>]

The test tries to send the same number of messages from each thread, this
ensures that the throughput numbers are reasonable (if one thread had more
then it would be the only one sending when the other threads had finished).
This means that the number of messages sent is always
  floor(messages/threads) * threads

-Jay

On Tue, Dec 13, 2011 at 12:29 PM, Andrew Psaltis <
Andrew.Psaltis@webtrends.com> wrote:

> I am using kafaka-0.7.0-incubating and when I run the producer perf test,
> the total reported messages does not match the number of messages passed in
> on the command line.
>
> Here is the command:
>
>
> bin/kafka-producer-perf-test.sh --brokerinfo broker.list=0:localhost:9092
> --message-size=1504 --async --batch-size=200 --compression-codec 1 --topic
> test --messages=50000 --threads=11
>
>
>
> And here is the output:
>
> Total Num Messages: 49995 bytes: 75192480 in 2.538 secs
> (kafka.tools.ProducerPerformance$)
>
>
> Is there something that I am not understanding about the producer, or the
> combination of the number of messages and threads I am using that causes
> the discrepancy?
>
> Thanks in advance,
> Andrew
>