[jira] [Updated] (KAFKA-17182) Consumer’s fetch sessions are evicted too quickly

["Kirk True (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-17182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kirk True updated KAFKA-17182:
------------------------------
    Summary: Consumer’s fetch sessions are evicted too quickly  (was: Consumer's fetch sessions are evicted too quickly)

> Consumer’s fetch sessions are evicted too quickly
> -------------------------------------------------
>
>                 Key: KAFKA-17182
>                 URL: https://issues.apache.org/jira/browse/KAFKA-17182
>             Project: Kafka
>          Issue Type: Bug
>          Components: clients, consumer
>    Affects Versions: 3.8.0
>            Reporter: Kirk True
>            Assignee: Kirk True
>            Priority: Critical
>              Labels: consumer-threading-refactor
>             Fix For: 4.0.0
>
>
> In stress testing the new consumer, the new consumer is evicting fetch sessions on the broker much more frequently than expected. There is an ongoing investigation into this behavior, but it appears to stem from a race condition due to the design of the new consumer.
> In the background thread, fetch requests are sent in a near continuous fashion for partitions that are "fetchable." A timing bug appears to cause partitions to be "unfetchable," which then causes them to end up in the "removed" set of partitions. The broker then removes them from the fetch session, which causes the number of remaining partitions for that session to drop below a threshold that allows it to be evicted by another competing session. Within a few milliseconds, though, the partitions become "fetchable" again, and are added to the "added" set of partitions on the next fetch request. This causes thrashing on both the client and broker sides as both are handling a steady stream of evictions, which negatively affects consumption throughput.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)