You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Hao Li (Jira)" <ji...@apache.org> on 2022/07/27 17:20:00 UTC

[jira] [Reopened] (KAFKA-12566) Flaky Test MirrorConnectorsIntegrationSSLTest#testReplication

     [ https://issues.apache.org/jira/browse/KAFKA-12566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hao Li reopened KAFKA-12566:
----------------------------

Somehow this is still happening:

```

[2022-07-27T11:32:04.461Z] MirrorConnectorsIntegrationSSLTest > testReplication() FAILED [2022-07-27T11:32:04.461Z] org.opentest4j.AssertionFailedError: Condition not met within timeout 20000. Offsets not translated downstream to primary cluster. ==> expected: <true> but was: <false> [2022-07-27T11:32:04.461Z] at org.junit.jupiter.api.AssertionUtils.fail(AssertionUtils.java:55) [2022-07-27T11:32:04.461Z] at org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:40) [2022-07-27T11:32:04.461Z] at org.junit.jupiter.api.Assertions.assertTrue(Assertions.java:210) [2022-07-27T11:32:04.461Z] at org.apache.kafka.test.TestUtils.lambda$waitForCondition$4(TestUtils.java:334) [2022-07-27T11:32:04.462Z] at org.apache.kafka.test.TestUtils.retryOnExceptionWithTimeout(TestUtils.java:382) [2022-07-27T11:32:04.462Z] at org.apache.kafka.test.TestUtils.waitForCondition(TestUtils.java:331) [2022-07-27T11:32:04.462Z] at org.apache.kafka.test.TestUtils.waitForCondition(TestUtils.java:315) [2022-07-27T11:32:04.462Z] at org.apache.kafka.test.TestUtils.waitForCondition(TestUtils.java:305) [2022-07-27T11:32:04.462Z] at org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationBaseTest.testReplication(MirrorConnectorsIntegrationBaseTest.java:319) [2022-07-27T11:32:04.462Z] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [2022-07-27T11:32:04.462Z] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [2022-07-27T11:32:04.462Z] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [2022-07-27T11:32:04.462Z] at java.lang.reflect.Method.invoke(Method.java:498) [2022-07-27T11:32:04.462Z] at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:725) [2022-07-27T11:32:04.462Z] at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60) [2022-07-27T11:32:04.462Z] at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131) [2022-07-27T11:32:04.463Z] at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149) [2022-07-27T11:32:04.463Z] at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140) [2022-07-27T11:32:04.463Z] at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84) [2022-07-27T11:32:04.463Z] at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115) [2022-07-27T11:32:04.463Z] at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105) [2022-07-27T11:32:04.463Z] at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106) [2022-07-27T11:32:04.463Z] at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64) [2022-07-27T11:32:04.463Z] at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45) [2022-07-27T11:32:04.463Z] at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37) [2022-07-27T11:32:04.463Z] at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104) [2022-07-27T11:32:04.463Z] at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98) [2022-07-27T11:32:04.464Z] at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:214) [2022-07-27T11:32:04.464Z] at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) [2022-07-27T11:32:04.464Z] at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:210) [2022-07-27T11:32:04.465Z] at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:135) [2022-07-27T11:32:04.465Z] at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:66) [2022-07-27T11:32:04.465Z] at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151) [2022-07-27T11:32:04.465Z] at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) [2022-07-27T11:32:04.465Z] at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) [2022-07-27T11:32:04.465Z] at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) [2022-07-27T11:32:04.466Z] at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) [2022-07-27T11:32:04.466Z] at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) [2022-07-27T11:32:04.466Z] at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) [2022-07-27T11:32:04.466Z] at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95) [2022-07-27T11:32:04.467Z] at java.util.ArrayList.forEach(ArrayList.java:1257) [2022-07-27T11:32:04.467Z] at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41) [2022-07-27T11:32:04.467Z] at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155) [2022-07-27T11:32:04.467Z] at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) [2022-07-27T11:32:04.467Z] at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) [2022-07-27T11:32:04.468Z] at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) [2022-07-27T11:32:04.469Z] at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) [2022-07-27T11:32:04.469Z] at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) [2022-07-27T11:32:04.469Z] at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) [2022-07-27T11:32:04.469Z] at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95) [2022-07-27T11:32:04.469Z] at java.util.ArrayList.forEach(ArrayList.java:1257) [2022-07-27T11:32:04.469Z] at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41) [2022-07-27T11:32:04.469Z] at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155) [2022-07-27T11:32:04.469Z] at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) [2022-07-27T11:32:04.469Z] at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) [2022-07-27T11:32:04.469Z] at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) [2022-07-27T11:32:04.469Z] at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) [2022-07-27T11:32:04.470Z] at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) [2022-07-27T11:32:04.470Z] at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) [2022-07-27T11:32:04.470Z] at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95) [2022-07-27T11:32:04.470Z] at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35) [2022-07-27T11:32:04.470Z] at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57) [2022-07-27T11:32:04.470Z] at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54) [2022-07-27T11:32:04.470Z] at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:107) [2022-07-27T11:32:04.470Z] at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88) [2022-07-27T11:32:04.470Z] at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54) [2022-07-27T11:32:04.471Z] at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67) [2022-07-27T11:32:04.471Z] at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52) [2022-07-27T11:32:04.471Z] at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114) [2022-07-27T11:32:04.471Z] at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86) [2022-07-27T11:32:04.471Z] at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86) [2022-07-27T11:32:04.471Z] at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:53) [2022-07-27T11:32:04.471Z] at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:99) [2022-07-27T11:32:04.471Z] at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.access$000(JUnitPlatformTestClassProcessor.java:79) [2022-07-27T11:32:04.471Z] at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor.stop(JUnitPlatformTestClassProcessor.java:75) [2022-07-27T11:32:04.472Z] at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:61) [2022-07-27T11:32:04.472Z] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [2022-07-27T11:32:04.472Z] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [2022-07-27T11:32:04.472Z] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [2022-07-27T11:32:04.472Z] at java.lang.reflect.Method.invoke(Method.java:498) [2022-07-27T11:32:04.472Z] at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36) [2022-07-27T11:32:04.472Z] at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) [2022-07-27T11:32:04.472Z] at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33) [2022-07-27T11:32:04.472Z] at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94) [2022-07-27T11:32:04.472Z] at com.sun.proxy.$Proxy2.stop(Unknown Source) [2022-07-27T11:32:04.472Z] at org.gradle.api.internal.tasks.testing.worker.TestWorker$3.run(TestWorker.java:193) [2022-07-27T11:32:04.472Z] at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129) [2022-07-27T11:32:04.472Z] at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100) [2022-07-27T11:32:04.473Z] at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60) [2022-07-27T11:32:04.473Z] at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56) [2022-07-27T11:32:04.473Z] at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:133) [2022-07-27T11:32:04.473Z] at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:71) [2022-07-27T11:32:04.473Z] at worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69) [2022-07-27T11:32:04.473Z] at worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74) [2022-07-27T11:32:04.473Z] [2022-07-27T11:32:04.473Z] MirrorConnectorsIntegrationSSLTest > testReplicationWithEmptyPartition() STARTED [2022-07-27T11:32:07.665Z]

```

> Flaky Test MirrorConnectorsIntegrationSSLTest#testReplication
> -------------------------------------------------------------
>
>                 Key: KAFKA-12566
>                 URL: https://issues.apache.org/jira/browse/KAFKA-12566
>             Project: Kafka
>          Issue Type: Test
>          Components: mirrormaker, unit tests
>            Reporter: Matthias J. Sax
>            Assignee: Luke Chen
>            Priority: Critical
>              Labels: flaky-test
>
>  
> {code:java}
> org.opentest4j.AssertionFailedError: Condition not met within timeout 20000. Offsets not translated downstream to primary cluster. ==> expected: <true> but was: <false> at org.junit.jupiter.api.AssertionUtils.fail(AssertionUtils.java:55) at org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:40) at org.junit.jupiter.api.Assertions.assertTrue(Assertions.java:193) at org.apache.kafka.test.TestUtils.lambda$waitForCondition$3(TestUtils.java:303) at org.apache.kafka.test.TestUtils.retryOnExceptionWithTimeout(TestUtils.java:351) at org.apache.kafka.test.TestUtils.retryOnExceptionWithTimeout(TestUtils.java:319) at org.apache.kafka.test.TestUtils.waitForCondition(TestUtils.java:300) at org.apache.kafka.test.TestUtils.waitForCondition(TestUtils.java:290) at org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationBaseTest.testReplication(MirrorConnectorsIntegrationBaseTest.java:289)
> {code}
> {{LOGs}}
> {quote}[2021-03-26 03:28:06,157] ERROR Could not check connector state info. (org.apache.kafka.connect.util.clusters.EmbeddedConnectClusterAssertions:420) org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: Could not read connector state. Error response: \{"error_code":404,"message":"No status found for connector MirrorSourceConnector"} at org.apache.kafka.connect.util.clusters.EmbeddedConnectCluster.connectorStatus(EmbeddedConnectCluster.java:479) at org.apache.kafka.connect.util.clusters.EmbeddedConnectClusterAssertions.checkConnectorState(EmbeddedConnectClusterAssertions.java:413) at org.apache.kafka.connect.util.clusters.EmbeddedConnectClusterAssertions.lambda$assertConnectorAndAtLeastNumTasksAreRunning$16(EmbeddedConnectClusterAssertions.java:286) at org.apache.kafka.test.TestUtils.lambda$waitForCondition$3(TestUtils.java:303) at org.apache.kafka.test.TestUtils.retryOnExceptionWithTimeout(TestUtils.java:351) at org.apache.kafka.test.TestUtils.retryOnExceptionWithTimeout(TestUtils.java:319) at org.apache.kafka.test.TestUtils.waitForCondition(TestUtils.java:300) at org.apache.kafka.test.TestUtils.waitForCondition(TestUtils.java:290) at org.apache.kafka.connect.util.clusters.EmbeddedConnectClusterAssertions.assertConnectorAndAtLeastNumTasksAreRunning(EmbeddedConnectClusterAssertions.java:285) at org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationBaseTest.waitUntilMirrorMakerIsRunning(MirrorConnectorsIntegrationBaseTest.java:470) at org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationBaseTest.testReplication(MirrorConnectorsIntegrationBaseTest.java:227){quote}
> and
> {quote}[2021-03-26 03:30:41,524] ERROR [MirrorHeartbeatConnector|task-0] Graceful stop of task MirrorHeartbeatConnector-0 failed. (org.apache.kafka.connect.runtime.Worker:866) [2021-03-26 03:30:41,527] ERROR [MirrorHeartbeatConnector|task-0] WorkerSourceTask\{id=MirrorHeartbeatConnector-0} failed to send record to heartbeats: (org.apache.kafka.connect.runtime.WorkerSourceTask:372) org.apache.kafka.common.KafkaException: Producer is closed forcefully. at org.apache.kafka.clients.producer.internals.RecordAccumulator.abortBatches(RecordAccumulator.java:750) at org.apache.kafka.clients.producer.internals.RecordAccumulator.abortIncompleteBatches(RecordAccumulator.java:737) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:282) at java.lang.Thread.run(Thread.java:748) [2021-03-26 03:30:42,248] ERROR [MirrorHeartbeatConnector|task-0] WorkerSourceTask\{id=MirrorHeartbeatConnector-0} Failed to flush, timed out while waiting for producer to flush outstanding 1 messages (org.apache.kafka.connect.runtime.WorkerSourceTask:512){quote}
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)