[jira] [Updated] (KAFKA-13504) Retry connect internal topics' creation in case of InvalidReplicationFactorException

["Viktor Somogyi-Vass (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-13504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Viktor Somogyi-Vass updated KAFKA-13504:
----------------------------------------
    Labels: cloudera  (was: )

> Retry connect internal topics' creation in case of InvalidReplicationFactorException
> ------------------------------------------------------------------------------------
>
>                 Key: KAFKA-13504
>                 URL: https://issues.apache.org/jira/browse/KAFKA-13504
>             Project: Kafka
>          Issue Type: Improvement
>          Components: KafkaConnect
>            Reporter: Andras Katona
>            Assignee: Viktor Somogyi-Vass
>            Priority: Major
>              Labels: cloudera
>
> In case the Kafka Broker cluster and the Kafka Connect cluster is started together and Connect would want to create its topics, there's a high chance to fail the creation with InvalidReplicationFactorException.
> {noformat}
> ERROR org.apache.kafka.connect.runtime.distributed.DistributedHerder [Worker clientId=connect-1, groupId=connect-cluster] Uncaught exception in herder work thread, exiting: 
> org.apache.kafka.connect.errors.ConnectException: Error while attempting to create/find topic(s) 'connect-offsets'
> ...
> Caused by: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.InvalidReplicationFactorException: Replication factor: 3 larger than available brokers: 2.
> {noformat}
> Introducing a retry logic here would make Connect a bit more robust.
> New configurations:
> * offset.storage.topic.create.retries
> * offset.storage.topic.create.retry.backoff.ms
> * config.storage.topic.create.retries
> * config.storage.topic.create.retry.backoff.ms
> * status.storage.topic.create.retries
> * status.storage.topic.create.retry.backoff.ms



--
This message was sent by Atlassian Jira
(v8.20.7#820007)