You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Andras Katona (Jira)" <ji...@apache.org> on 2023/05/19 18:48:00 UTC
[jira] [Assigned] (KAFKA-13504) Retry connect internal topics' creation in case of InvalidReplicationFactorException
[ https://issues.apache.org/jira/browse/KAFKA-13504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andras Katona reassigned KAFKA-13504:
-------------------------------------
Assignee: Andras Katona (was: Viktor Somogyi-Vass)
> Retry connect internal topics' creation in case of InvalidReplicationFactorException
> ------------------------------------------------------------------------------------
>
> Key: KAFKA-13504
> URL: https://issues.apache.org/jira/browse/KAFKA-13504
> Project: Kafka
> Issue Type: Improvement
> Components: KafkaConnect
> Reporter: Andras Katona
> Assignee: Andras Katona
> Priority: Major
> Labels: cloudera
>
> In case the Kafka Broker cluster and the Kafka Connect cluster is started together and Connect would want to create its topics, there's a high chance to fail the creation with InvalidReplicationFactorException.
> {noformat}
> ERROR org.apache.kafka.connect.runtime.distributed.DistributedHerder [Worker clientId=connect-1, groupId=connect-cluster] Uncaught exception in herder work thread, exiting:
> org.apache.kafka.connect.errors.ConnectException: Error while attempting to create/find topic(s) 'connect-offsets'
> ...
> Caused by: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.InvalidReplicationFactorException: Replication factor: 3 larger than available brokers: 2.
> {noformat}
> Introducing a retry logic here would make Connect a bit more robust.
> New configurations:
> * offset.storage.topic.create.retries
> * offset.storage.topic.create.retry.backoff.ms
> * config.storage.topic.create.retries
> * config.storage.topic.create.retry.backoff.ms
> * status.storage.topic.create.retries
> * status.storage.topic.create.retry.backoff.ms
--
This message was sent by Atlassian Jira
(v8.20.10#820010)