You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Kowshik Prakasam (Jira)" <ji...@apache.org> on 2021/05/31 07:54:00 UTC

[jira] [Created] (KAFKA-12867) Trogdor ConsumeBenchWorker quits prematurely with maxMessages config

Kowshik Prakasam created KAFKA-12867:
----------------------------------------

             Summary: Trogdor ConsumeBenchWorker quits prematurely with maxMessages config
                 Key: KAFKA-12867
                 URL: https://issues.apache.org/jira/browse/KAFKA-12867
             Project: Kafka
          Issue Type: Bug
            Reporter: Kowshik Prakasam


The trogdor [ConsumeBenchWorker|https://github.com/apache/kafka/commits/fc405d792de12a50956195827eaf57bbf64444c9/trogdor/src/main/java/org/apache/kafka/trogdor/workload/ConsumeBenchWorker.java] has a bug. If one of the consumption tasks completes executing successfully due to [maxMessages being consumed|https://github.com/apache/kafka/blob/fc405d792de12a50956195827eaf57bbf64444c9/trogdor/src/main/java/org/apache/kafka/trogdor/workload/ConsumeBenchWorker.java#L245], then, the consumption task [notifies the doneFuture|https://github.com/apache/kafka/blob/fc405d792de12a50956195827eaf57bbf64444c9/trogdor/src/main/java/org/apache/kafka/trogdor/workload/ConsumeBenchWorker.java#L285] causing the ConsumeBenchWorker to halt. This becomes a problem when more than 1 consumption task is running in parallel, because the successful completion of 1 of the tasks shuts down the entire worker while the other tasks are still running. When the worker is shut down, it [kills|https://github.com/apache/kafka/blob/fc405d792de12a50956195827eaf57bbf64444c9/trogdor/src/main/java/org/apache/kafka/trogdor/workload/ConsumeBenchWorker.java#L482] all the active consumption tasks, which is not the desired behavior.

The fix is to not notify the doneFuture when 1 of the consumption tasks complete without error. Instead, we should defer the notification to the [CloseStatusUpdater|https://github.com/apache/kafka/blob/fc405d792de12a50956195827eaf57bbf64444c9/trogdor/src/main/java/org/apache/kafka/trogdor/workload/ConsumeBenchWorker.java#L299] thread.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)