[GitHub] [kafka] vvcephei commented on a change in pull request #8662: HOTFIX: skip listOffsets request for newly created changelog topics

[GitBox <gi...@apache.org>]

vvcephei commented on a change in pull request #8662:
URL: https://github.com/apache/kafka/pull/8662#discussion_r424712060



##########
File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/ClientUtils.java
##########
@@ -105,6 +109,7 @@ public static String getTaskProducerClientId(final String threadClientId, final
                 endOffsets = future.get(timeout.toMillis(), TimeUnit.MILLISECONDS);
             }
         } catch (final TimeoutException | RuntimeException | InterruptedException | ExecutionException e) {
+            LOG.warn("listOffsets request failed due to ", e);

Review comment:
       ```suggestion
               LOG.warn("listOffsets request failed.", e);
   ```
   
   Thanks! (minor suggestion to make the log message more typical)

##########
File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/InternalTopicManager.java
##########
@@ -169,6 +173,9 @@ public void makeReady(final Map<String, InternalTopicConfig> topics) {
             log.error(timeoutAndRetryError);
             throw new StreamsException(timeoutAndRetryError);
         }
+        log.debug("Completed validating internal topics and created {}", newlyCreatedTopics);

Review comment:
       It's not what you signed up for, but I'm wondering if we should at least submit a Jira to give some of these AdminClient methods a "full consistency" mode. In other words, since the command returns a future anyway, it would be nice to be able to tell it not to return until it can guarantee the topic will appear to be fully created on all brokers.
   
   I'm mildly concerned that we're just kicking the can down the road a little ways with this PR. I.e., we let the assignment happen, but then some other metadata (or data) operation for that topic will just fail shortly thereafter.
   
   More generally, we jump through a lot of hoops in our own tests to try and make sure that the topics are really, actually created (or deleted) before proceeding with the test, and I'm sure that our users also suffer from the same problem in their testing and production code.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org