You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Kamal Chandraprakash (Jira)" <ji...@apache.org> on 2023/11/02 18:15:00 UTC
[jira] [Updated] (KAFKA-15776) Configurable delay timeout for DelayedRemoteFetch request
[ https://issues.apache.org/jira/browse/KAFKA-15776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kamal Chandraprakash updated KAFKA-15776:
-----------------------------------------
Description:
We are reusing the {{fetch.max.wait.ms}} config as a delay timeout for DelayedRemoteFetchPurgatory. {{fetch.max.wait.ms}} purpose is to wait for the given amount of time when there is no data available to serve the FETCH request.
{code:java}
The maximum amount of time the server will block before answering the fetch request if there isn't sufficient data to immediately satisfy the requirement given by fetch.min.bytes.
{code}
[https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/server/DelayedRemoteFetch.scala#L41]
Using the same timeout in the DelayedRemoteFetchPurgatory can confuse the user on how to configure optimal value for each purpose. Moreover, the config is of *LOW* importance and most of the users won't configure it and use the default value of 500 ms.
Having the delay timeout of 500 ms in DelayedRemoteFetchPurgatory can lead to higher number of expired delayed remote fetch request when the remote storage have any degradation to serve within the timeout.
We should introduce one config (preferably server config) to define the delay timeout for DelayedRemoteFetch requests.
was:
We are reusing the {{fetch.max.wait.ms}} config as a delay timeout for DelayedRemoteFetchPurgatory. {{fetch.max.wait.ms}} purpose is to wait for the given amount of time when there is no data available to serve the FETCH request.
{code:java}
The maximum amount of time the server will block before answering the fetch request if there isn't sufficient data to immediately satisfy the requirement given by fetch.min.bytes.
{code}
https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/server/DelayedRemoteFetch.scala#L41
Using the same timeout in the DelayedRemoteFetchPurgatory can confuse the user on how to configure optimal value for each purpose. Moreover, the config is of *LOW* importance and most of the users won't configure it and use the default value of 500 ms.
Having the delay timeout of 500 ms in DelayedRemoteFetchPurgatory can lead to larger number of expired delayed remote fetch request when the remote storage have any degradation to serve within the timeout.
We should introduce one config (preferably server config) to define the delay timeout for DelayedRemoteFetch requests.
> Configurable delay timeout for DelayedRemoteFetch request
> ---------------------------------------------------------
>
> Key: KAFKA-15776
> URL: https://issues.apache.org/jira/browse/KAFKA-15776
> Project: Kafka
> Issue Type: Task
> Reporter: Kamal Chandraprakash
> Assignee: Kamal Chandraprakash
> Priority: Major
>
> We are reusing the {{fetch.max.wait.ms}} config as a delay timeout for DelayedRemoteFetchPurgatory. {{fetch.max.wait.ms}} purpose is to wait for the given amount of time when there is no data available to serve the FETCH request.
> {code:java}
> The maximum amount of time the server will block before answering the fetch request if there isn't sufficient data to immediately satisfy the requirement given by fetch.min.bytes.
> {code}
> [https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/server/DelayedRemoteFetch.scala#L41]
> Using the same timeout in the DelayedRemoteFetchPurgatory can confuse the user on how to configure optimal value for each purpose. Moreover, the config is of *LOW* importance and most of the users won't configure it and use the default value of 500 ms.
> Having the delay timeout of 500 ms in DelayedRemoteFetchPurgatory can lead to higher number of expired delayed remote fetch request when the remote storage have any degradation to serve within the timeout.
> We should introduce one config (preferably server config) to define the delay timeout for DelayedRemoteFetch requests.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)