[jira] [Commented] (KAFKA-7599) Trogdor - Allow configuration for not throttling Benchmark Workers and expose messages per second in task status

["Colin P. McCabe (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-7599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16677128#comment-16677128 ] 

Colin P. McCabe commented on KAFKA-7599:
----------------------------------------

bq. I propose we allow for unbounded `targetMessagesPerSec` if the field is not present.

I guess the reason for having a limit by default is that it tends to give a better "out of the box" experience.  If you produce as fast as you can, you can often make the cluster less responsive for others, which can be annoying.  But I don't feel that strongly about it, I guess.

bq. Further, it would be very useful if some of these workers showed the `messagesPerSecond` they have been producing/consuming at. 

Yeah.  We may want a long-run and short-run average as well.

> Trogdor - Allow configuration for not throttling Benchmark Workers and expose messages per second in task status
> ----------------------------------------------------------------------------------------------------------------
>
>                 Key: KAFKA-7599
>                 URL: https://issues.apache.org/jira/browse/KAFKA-7599
>             Project: Kafka
>          Issue Type: Improvement
>            Reporter: Stanislav Kozlovski
>            Assignee: Stanislav Kozlovski
>            Priority: Major
>
> In Trogdor, the ConsumeBench, ProduceBench and RoundTrip workers all take in an argument called "targetMessagesPerSec". That argument works as an upper bound on the number of messages that can be consumed/produced per second in that worker.
> It is useful to support infinite messages per second. Currently, if the `targetMessagesPerSec` field is not present in the request, the RoundTripWorker will raise an exception, whereas the ConsumeBench and ProduceBench workers will work as if they had `targetMessagesPerSec=10`.
> I propose we allow for unbounded `targetMessagesPerSec` if the field is not present.
> Further, it would be very useful if some of these workers showed the `messagesPerSecond` they have been producing/consuming at. 
> Even now, giving the worker a `targetMessagesPerSec` does not guarantee that the worker will reach the needed `targetMessagesPerSec`. There is no easy way of knowing how the worker performed - you have to subtract the status fields `startedMs` and `doneMs` to get the total duration of the task, convert to seconds and then divide that by the `maxMessages` field.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)