You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "David Mao (Jira)" <ji...@apache.org> on 2022/06/27 01:19:00 UTC
[jira] [Updated] (KAFKA-14025) Request timeout includes time queued per connection
[ https://issues.apache.org/jira/browse/KAFKA-14025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
David Mao updated KAFKA-14025:
------------------------------
Description:
In the NetworkClient we measure request timeouts from when a request is first queued in
{{{}InflightRequests{}}}. A Kafka broker currently only processes at most one in-flight request at a time per connection. This disconnect between client-side request timeout enforcement and broker-side request processing means that an aggressive request timeout can result in spurious timeouts when a connection has multiple requests in-flight.
Example:
broker 0 has avg 25 ms latency, producer 1 has a request timeout of 50 ms.
t=0 p1 sends r1, b0 starts processing r1
t=5 p1 sends r2
t=10 p1 sends r3
t=25 b0 sends response for r1, starts processing r2
t=50 b0 sends response for r2, starts processing r3
t=60 client times out r3, despite the broker only having 10 ms or so to process the request.
Instead, we should be measuring request timeouts only once previous requests have been responded to.
was:
In the NetworkClient we measure request timeouts from when a request is first queued in
{{{}InflightRequests{}}}. A Kafka broker currently only processes at most one in-flight request at a time per connection. This disconnect between client-side request timeout enforcement and broker-side request processing means that an aggressive request timeout can result in spurious timeouts when a connection has multiple requests in-flight.
Example:
broker 0 has avg 25 ms latency, producer 1 has a request timeout of 50 ms.
t=0 p1 sends r1, b0 starts processing r1
t=5 p1 sends r2
t=10 p1 sends r3
t=25 b0 sends response for r1, starts processing r2
t=50 b0 sends response for r2, starts processing r3
t=60 client times out r3
Instead, we should be measuring request timeouts only once previous requests have been responded to.
> Request timeout includes time queued per connection
> ---------------------------------------------------
>
> Key: KAFKA-14025
> URL: https://issues.apache.org/jira/browse/KAFKA-14025
> Project: Kafka
> Issue Type: Bug
> Reporter: David Mao
> Priority: Major
>
> In the NetworkClient we measure request timeouts from when a request is first queued in
> {{{}InflightRequests{}}}. A Kafka broker currently only processes at most one in-flight request at a time per connection. This disconnect between client-side request timeout enforcement and broker-side request processing means that an aggressive request timeout can result in spurious timeouts when a connection has multiple requests in-flight.
> Example:
> broker 0 has avg 25 ms latency, producer 1 has a request timeout of 50 ms.
> t=0 p1 sends r1, b0 starts processing r1
> t=5 p1 sends r2
> t=10 p1 sends r3
> t=25 b0 sends response for r1, starts processing r2
> t=50 b0 sends response for r2, starts processing r3
> t=60 client times out r3, despite the broker only having 10 ms or so to process the request.
>
> Instead, we should be measuring request timeouts only once previous requests have been responded to.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)