[jira] [Commented] (KAFKA-10021) When reading to the end of the config log, check if fetch.max.wait.ms is greater than worker.sync.timeout.ms

["Randall Hauch (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-10021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17282158#comment-17282158 ] 

Randall Hauch commented on KAFKA-10021:
---------------------------------------

Including in 2.6.2 due to reports of rebalance storms because of the logs getting stuck. cc [~ableegoldman].

> When reading to the end of the config log, check if fetch.max.wait.ms is greater than worker.sync.timeout.ms
> ------------------------------------------------------------------------------------------------------------
>
>                 Key: KAFKA-10021
>                 URL: https://issues.apache.org/jira/browse/KAFKA-10021
>             Project: Kafka
>          Issue Type: Bug
>          Components: KafkaConnect
>    Affects Versions: 2.3.0, 2.4.0, 2.5.0, 2.6.0, 2.7.0
>            Reporter: Sanjana Kaundinya
>            Assignee: Randall Hauch
>            Priority: Major
>             Fix For: 2.5.2, 2.8.0, 2.7.1, 2.6.2
>
>
> Currently in the Connect code in DistributedHerder.java, we see the following piece of code
>  
> {{            if (!canReadConfigs && !readConfigToEnd(workerSyncTimeoutMs))
>                 return; // Safe to return and tick immediately because readConfigToEnd will do the backoff for us}}
> where the workerSyncTimeoutMs passed in is the timeout given to read to the end of the config log. This is a bug as we should check if fetch.wait.max.ms is greater than worker.sync.timeout.ms and if it is, use worker.sync.timeout.ms as the fetch.wait.max.ms. A better fix would be to use the AdminClient to read to the end of the log, but at a minimum we should check the configs.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)