You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Markus Jais <ma...@1und1.de> on 2015/09/23 11:14:01 UTC

Which perf-test tool?

Hello,

I have a question about performance testing:

Performance tests for producers can run (using a Java class) with:

bin/kafka-run-class.sh org.apache.kafka.clients.tools.ProducerPerformance

but there is also:
./bin/kafka-producer-perf-test.sh
This is calling a Scala Class called kafka.tools.ProducerPerformance.

Which tool is recommend for 0.8.2.1 and newer?

What are the differences and limitations? I couldn't find anything in 
the Kafka docs.

They seem to be similar according to the source code.
Is one of the tools deprecated?

Best,

Markus

Re: Which perf-test tool?

Posted by Markus Jais <ma...@1und1.de>.
Hi,

thanks. Very helpful.

It might go good to have only one tool incl. a section in the 
documentation about how to use it in a complex distributed environment.

Regards,

Markus

On 23.09.2015 13:51, Ben Stopford wrote:
> Both classes work ok. I prefer the Java one simply because has better output and it does less overriding of default values.
>
> However, in both cases you probably need to tweak settings to suit your use case. Most notably:
> acks
> batch.size
> linger.ms
> based on whether you are interested in latency or throughput. That is usually sufficient for producer performance measurement.
>
> At present nothing is deprecated but there are some changes going in to clean these up a little.
>
> B
>
>
>> On 23 Sep 2015, at 10:14, Markus Jais <ma...@1und1.de> wrote:
>>
>> Hello,
>>
>> I have a question about performance testing:
>>
>> Performance tests for producers can run (using a Java class) with:
>>
>> bin/kafka-run-class.sh org.apache.kafka.clients.tools.ProducerPerformance
>>
>> but there is also:
>> ./bin/kafka-producer-perf-test.sh
>> This is calling a Scala Class called kafka.tools.ProducerPerformance.
>>
>> Which tool is recommend for 0.8.2.1 and newer?
>>
>> What are the differences and limitations? I couldn't find anything in the Kafka docs.
>>
>> They seem to be similar according to the source code.
>> Is one of the tools deprecated?
>>
>> Best,
>>
>> Markus
>

Re: Which perf-test tool?

Posted by Ben Stopford <be...@confluent.io>.
Both classes work ok. I prefer the Java one simply because has better output and it does less overriding of default values.

However, in both cases you probably need to tweak settings to suit your use case. Most notably: 
acks
batch.size
linger.ms
based on whether you are interested in latency or throughput. That is usually sufficient for producer performance measurement. 

At present nothing is deprecated but there are some changes going in to clean these up a little. 

B


> On 23 Sep 2015, at 10:14, Markus Jais <ma...@1und1.de> wrote:
> 
> Hello,
> 
> I have a question about performance testing:
> 
> Performance tests for producers can run (using a Java class) with:
> 
> bin/kafka-run-class.sh org.apache.kafka.clients.tools.ProducerPerformance
> 
> but there is also:
> ./bin/kafka-producer-perf-test.sh
> This is calling a Scala Class called kafka.tools.ProducerPerformance.
> 
> Which tool is recommend for 0.8.2.1 and newer?
> 
> What are the differences and limitations? I couldn't find anything in the Kafka docs.
> 
> They seem to be similar according to the source code.
> Is one of the tools deprecated?
> 
> Best,
> 
> Markus