You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Rajini Sivaram (Jira)" <ji...@apache.org> on 2021/06/02 18:34:00 UTC
[jira] [Resolved] (KAFKA-12867) Trogdor ConsumeBenchWorker quits
prematurely with maxMessages config
[ https://issues.apache.org/jira/browse/KAFKA-12867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rajini Sivaram resolved KAFKA-12867.
------------------------------------
Fix Version/s: 3.0.0
Reviewer: Rajini Sivaram
Resolution: Fixed
> Trogdor ConsumeBenchWorker quits prematurely with maxMessages config
> --------------------------------------------------------------------
>
> Key: KAFKA-12867
> URL: https://issues.apache.org/jira/browse/KAFKA-12867
> Project: Kafka
> Issue Type: Bug
> Reporter: Kowshik Prakasam
> Assignee: Kowshik Prakasam
> Priority: Major
> Fix For: 3.0.0
>
>
> The trogdor [ConsumeBenchWorker|https://github.com/apache/kafka/commits/fc405d792de12a50956195827eaf57bbf64444c9/trogdor/src/main/java/org/apache/kafka/trogdor/workload/ConsumeBenchWorker.java] has a bug. If one of the consumption tasks completes executing successfully due to [maxMessages being consumed|https://github.com/apache/kafka/blob/fc405d792de12a50956195827eaf57bbf64444c9/trogdor/src/main/java/org/apache/kafka/trogdor/workload/ConsumeBenchWorker.java#L245], then, the consumption task [notifies the doneFuture|https://github.com/apache/kafka/blob/fc405d792de12a50956195827eaf57bbf64444c9/trogdor/src/main/java/org/apache/kafka/trogdor/workload/ConsumeBenchWorker.java#L285] causing the ConsumeBenchWorker to halt. This becomes a problem when more than 1 consumption task is running in parallel, because the successful completion of 1 of the tasks shuts down the entire worker while the other tasks are still running. When the worker is shut down, it [kills|https://github.com/apache/kafka/blob/fc405d792de12a50956195827eaf57bbf64444c9/trogdor/src/main/java/org/apache/kafka/trogdor/workload/ConsumeBenchWorker.java#L482] all the active consumption tasks, which is not the desired behavior.
> The fix is to not notify the doneFuture when 1 of the consumption tasks complete without error. Instead, we should defer the notification to the [CloseStatusUpdater|https://github.com/apache/kafka/blob/fc405d792de12a50956195827eaf57bbf64444c9/trogdor/src/main/java/org/apache/kafka/trogdor/workload/ConsumeBenchWorker.java#L299] thread.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)