[jira] [Commented] (KAFKA-7593) Infinite restart loop when failed to store big config for task

["Randall Hauch (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-7593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16676944#comment-16676944 ] 

Randall Hauch commented on KAFKA-7593:
--------------------------------------

[~olkuznsmith], have you tried manually setting the {{max.message.bytes}} config to a larger value on the {{connect-configs}} topic in the broker? If so, then this is a good workaround in case other users run into this.

However, as you suggest we should evaluate how to respond when when attempts to write to the config topic fail. Logging might be easy (e.g., add a callback when the KafkaConfigBackingStore sends messages to the log), but it still might be difficult to prevent the loop.

It might also be good to set {{max.message.bytes}} to a larger value when creating the topic (no KIP required). Or, we could expose a new property for the internal topics on the distributed worker to allow users to set this (requires a KIP). IMO, you can always create this topic ahead of time, so maybe simply increasing the default {{max.message.bytes}} topic config is sufficient.

> Infinite restart loop when failed to store big config for task
> --------------------------------------------------------------
>
>                 Key: KAFKA-7593
>                 URL: https://issues.apache.org/jira/browse/KAFKA-7593
>             Project: Kafka
>          Issue Type: Bug
>          Components: KafkaConnect
>    Affects Versions: 1.1.0
>            Reporter: Oleg Kuznetsov
>            Priority: Major
>
> In case when config message for config topic is greater than kafka broker allows to store, source connector starts infinite restart loop without any error indication.
> There could be an exception thrown in this case or a smarter handling of big config.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)