You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "zhaobo (JIRA)" <ji...@apache.org> on 2019/06/16 06:03:00 UTC
[jira] [Created] (KAFKA-8543) Optimize Fetch thread allocation
strategy to ensure full utilization of Fetch thread resources
zhaobo created KAFKA-8543:
-----------------------------
Summary: Optimize Fetch thread allocation strategy to ensure full utilization of Fetch thread resources
Key: KAFKA-8543
URL: https://issues.apache.org/jira/browse/KAFKA-8543
Project: Kafka
Issue Type: Improvement
Components: core
Affects Versions: 2.2.0
Reporter: zhaobo
The Follow TP on the Broker needs to fetch data from the Leader broker. The original fetch thread allocation strategy has two areas that can be optimized:
(1)The configuration file has a parameter "num.consumer.fetchers" which limits the number of threads fetching data from a source broker. In fact, Fetch resources are not fully used, and only a part of the threads will be used with a high probability. When the traffic of the Leader TP increases, the fetch capability may be bottlenecked.
(2)The number of TPs responsible for each Fetch thread may be uneven.
Improve: We propose a polling allocation method to ensure that Fetch thread resources can be fully utilized. And the number of TPs responsible for each fetch thread is balanced.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)