You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Chris Egerton (Jira)" <ji...@apache.org> on 2022/07/25 23:40:00 UTC
[jira] [Comment Edited] (KAFKA-14101) Flaky ExactlyOnceSourceIntegrationTest.testConnectorBoundary
[ https://issues.apache.org/jira/browse/KAFKA-14101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17571130#comment-17571130 ]
Chris Egerton edited comment on KAFKA-14101 at 7/25/22 11:39 PM:
-----------------------------------------------------------------
Thanks, [~mimaison]. I believe this is due to a bug in that PR; should be using {{consumeAll}} instead of {{consume}} in the test [here|https://github.com/C0urante/kafka/blob/466a59df2d02de16d285325be4d44c99b24eee4d/connect/runtime/src/test/java/org/apache/kafka/connect/integration/ExactlyOnceSourceIntegrationTest.java#L408-L410] in order to guarantee that we've collected records from every partition of the topic that the connector produces to, instead of just consuming until we get at least 100 from one or more topic partitions.
Please let me know if there are any issues on trunk, or if you see this issue come up again on the branch for [https://github.com/apache/kafka/pull/12429] now that I've pushed a change that _should_ fix this.
Apologies for all the flakiness, and thank you for quickly finding and reporting all of it. Really hoping we can iron out all the kinks before we start producing 3.3.0 RCs 🤞
was (Author: chrisegerton):
Thanks, [~mimaison]. I believe this is due to a bug in that PR; should be using {{consumeAll}} instead of {{consume}} in the test [here|https://github.com/C0urante/kafka/blob/466a59df2d02de16d285325be4d44c99b24eee4d/connect/runtime/src/test/java/org/apache/kafka/connect/integration/ExactlyOnceSourceIntegrationTest.java#L408-L410].
Please let me know if there are any issues on trunk, or if you see this issue come up again on the branch for [https://github.com/apache/kafka/pull/12429] now that I've pushed a change that _should_ fix this.
Apologies for all the flakiness, and thank you for quickly finding and reporting all of it. Really hoping we can iron out all the kinks before we start producing 3.3.0 RCs 🤞
> Flaky ExactlyOnceSourceIntegrationTest.testConnectorBoundary
> ------------------------------------------------------------
>
> Key: KAFKA-14101
> URL: https://issues.apache.org/jira/browse/KAFKA-14101
> Project: Kafka
> Issue Type: Improvement
> Components: KafkaConnect
> Reporter: Mickael Maison
> Assignee: Chris Egerton
> Priority: Major
> Attachments: org.apache.kafka.connect.integration.ExactlyOnceSourceIntegrationTest.testConnectorBoundary.test.stdout
>
>
> I hit this one while running the tests on your branch from https://github.com/apache/kafka/pull/12429
> org.apache.kafka.connect.integration.ExactlyOnceSourceIntegrationTest > testConnectorBoundary FAILED
> java.lang.AssertionError: Committed records should exclude connector-aborted transactions expected:<[1, 3, 4, 5, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89]> but was:<[4, 5, 10, 13, 16, 18, 20, 37, 39, 40, 46, 47, 49, 54, 59, 64, 65, 68, 70, 71, 77, 83, 85, 89, 146, 148, 153, 154, 157, 158, 159, 163, 165, 169, 175, 176, 178, 183, 184, 185, 187, 188, 191, 196, 199, 211, 216, 217, 218, 222, 223, 229, 232, 238, 244, 251, 255, 259, 261, 269, 272, 274, 275, 276, 277, 278, 279, 285, 291, 293, 296, 299]>
> at org.junit.Assert.fail(Assert.java:89)
> at org.junit.Assert.failNotEquals(Assert.java:835)
> at org.junit.Assert.assertEquals(Assert.java:120)
> at org.apache.kafka.connect.integration.ExactlyOnceSourceIntegrationTest.testConnectorBoundary(ExactlyOnceSourceIntegrationTest.java:456)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)