[GitHub] [kafka] yashmayya commented on pull request #13465: KAFKA-14368: Connect offset write REST API

["yashmayya (via GitHub)" <gi...@apache.org>]

yashmayya commented on PR #13465:
URL: https://github.com/apache/kafka/pull/13465#issuecomment-1562952794

   I just noticed that `testAlterSinkConnectorOffsetsDifferentKafkaClusterTargeted` is failing in the CI run. There's also https://issues.apache.org/jira/browse/KAFKA-14956 where `testGetSinkConnectorOffsetsDifferentKafkaClusterTargeted ` has been failing on CI for a while. Interestingly, neither of them have failed for me locally in over a 100 runs under various loads. The point at which both of them are failing is also interesting:
   
   1. We create an embedded Connect cluster with its own embedded backing Kafka cluster
   2. We create a second embedded Kafka cluster
   3. We configure a sink connector in the embedded Connect cluster which consumes from a topic on the second embedded Kafka cluster
   4. We produce 10 messages each to 5 different partitions of a topic on the second Kafka cluster (which the connector is configured to consume from)
   5. We use the offsets read REST API to get the consumer group offsets for the sink connector and wait until it "catches up" to the expected offsets. This operation is retried up to 15 seconds and if the consumer group offsets (obtained via an admin client in the worker) don't match the expected offsets, the test fails. 
   
   Both the tests are failing at this point. Since they consistently pass locally, it doesn't seem to be a correctness issue with connectors that target different Kafka clusters. I'm wondering if we need to up the timeout although 15 seconds should be enough to consume just 50 messages 😕 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org