You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2022/07/22 11:22:37 UTC
[GitHub] [kafka] akatona84 commented on a diff in pull request #11565: KAFKA-13504: Retry connect internal topics' creation in case of InvalidReplicationFactorException
akatona84 commented on code in PR #11565:
URL: https://github.com/apache/kafka/pull/11565#discussion_r927554756
##########
connect/runtime/src/main/java/org/apache/kafka/connect/util/TopicAdmin.java:
##########
@@ -329,6 +334,48 @@ public Set<String> createTopics(NewTopic... topics) {
return createOrFindTopics(topics).createdTopics();
}
+ /**
+ * Implements a retry logic around creating topic(s) in case it'd fail due to InvalidReplicationFactorException
+ *
+ * @param topicDescription
+ * @param config
+ * @param time
+ * @return the same as {@link TopicAdmin#createTopics(NewTopic...)}
+ */
+ public Set<String> createTopicsWithRetry(NewTopic topicDescription, WorkerConfig config, Time time) {
+ if (config instanceof DistributedConfig) {
+ DistributedConfig distributedConfig = (DistributedConfig) config;
+ long timeoutMs = distributedConfig.getTopicCreateRetryTimeoutMs();
+ long backOffMs = distributedConfig.getTopicCreateBackOffMs();
+
+ Timer timer = time.timer(timeoutMs);
+ do {
+ try {
+ return createTopics(topicDescription);
+ } catch (ConnectException e) {
+ if (retryableTopicCreationException(e)) {
+ log.info("'{}' topic creation failed due to '{}', retrying, {}ms remaining",
+ topicDescription.name(), e.getMessage(), timer.remainingMs());
+ timer.sleep(backOffMs);
+ } else {
+ throw e;
+ }
+ }
+ } while (timer.notExpired());
+ throw new IllegalStateException("This should not happen");
Review Comment:
yeah, thx!
forgot to transform it to timeout exception after the retry based logic rewirte.
##########
connect/runtime/src/main/java/org/apache/kafka/connect/util/TopicAdmin.java:
##########
@@ -329,6 +334,48 @@ public Set<String> createTopics(NewTopic... topics) {
return createOrFindTopics(topics).createdTopics();
}
+ /**
+ * Implements a retry logic around creating topic(s) in case it'd fail due to InvalidReplicationFactorException
+ *
+ * @param topicDescription
+ * @param config
+ * @param time
+ * @return the same as {@link TopicAdmin#createTopics(NewTopic...)}
+ */
+ public Set<String> createTopicsWithRetry(NewTopic topicDescription, WorkerConfig config, Time time) {
Review Comment:
This was exactly what I don't like in my change :D
I was trying to avoid duplicating the code for this instanceof check. I'll find another place.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org