[GitHub] [kafka] C0urante commented on pull request #13284: KAFKA-14718: Make MirrorMaker startup synchronously depend on connector start

["C0urante (via GitHub)" <gi...@apache.org>]

C0urante commented on PR #13284:
URL: https://github.com/apache/kafka/pull/13284#issuecomment-1448579807

   > I think that fixing the test by extending the timeout might hide the problems that users may be facing in production as well.
   
   I don't think that bumping the timeouts in these tests is likely to hide issues that may surface in production; rather, I'm only worried that they'll make local testing and development more cumbersome.
   
   > Hence, I believe, we need a better mechanism to check for successful startup of MirrorMaker. Thoughts?
   
   Agreed 👍 IMO MM2 observability in general can be improved. Right now you're basically limited to JMX, other custom metrics reporters, and logs. There's no public API for viewing cluster-wide health info (the closest thing is reading directly from the status topic, but that's not public API and we shouldn't count on users doing that or recommend it).
   
   If there's an issue starting one of the connectors, instead of failing startup of the MM2 node, we [log a message](https://github.com/apache/kafka/blob/f586fa59d3f938e04bda4e8143ddb1c4310eaf78/connect/mirror/src/main/java/org/apache/kafka/connect/mirror/MirrorMaker.java#L232-L237) and move on. I can't imagine this is very easy for users to work with, though maybe I'm underestimating the utility of JMX and logs here.
   
   > In absence of such a mechanism, today, a user might call MirrorMaker#start() and assume that it has started correctly while in the background, connector start up might have failed (just as it did in this test scenario).
   
   This is less concerning because the `MirrorMaker` class isn't part of public API; users are not expected to directly interact with any instances of this class and we do not support any use cases like that right now.
   
   
   I'm wondering if, instead of a `Future`-based approach, we might add an internal API to the `MirrorMaker` class to expose the status of the connectors running on it?
   
   A brief sketch:
   
   ```java
   public class MirrorMaker {
       public ConnectorStateInfo connectorStatus(String source, String target, String connector) {
           SourceAndTarget sourceAndTarget = new SourceAndTarget(source, target);
           checkHerder(sourceAndTarget);
           return herders.get(new SourceAndTarget(source, target)).connectorStatus(connector);
       }
   }
   ```
   
   We could then leverage this API in the `DedicatedMirrorIntegrationTest` suite to implement an equivalent of the `waitUntilMirrorMakerIsRunning` method from the `MirrorConnectorsIntegrationBaseTest` suite.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org