You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/04/25 04:01:58 UTC
[GitHub] [pulsar] lhotari opened a new issue #10356: Flaky-test: TransactionEndToEndTest.cleanup / ManagedLedgerFactoryImpl.shutdown doesn't complete (causes "CI - Unit - Brokers - Other" to hang and timeout)
lhotari opened a new issue #10356:
URL: https://github.com/apache/pulsar/issues/10356
<!---
Instructions for reporting a flaky test using this issue template:
1. Replace [test class] in title and body with the test class name
2. Replace [test method] in title and body with the test method that failed. Multiple methods are flaky, remove the content that refers to the test method.
3. Replace "url here" with a url to an example failure. In the Github Actions workflow run logs, you can right click on the line number to copy a link to the line. Example of such url is https://github.com/apache/pulsar/pull/8892/checks?check_run_id=1531075794#step:9:377 . The logs are available for a limited amount of time (usually for a few weeks).
4. Replace "relevant parts of the exception stacktrace here" with the a few lines of the stack trace that shows at leat the exception message and the line of test code where the stacktrace occured.
5. Replace "full exception stacktrace here" with the full exception stacktrace from logs. This section will be hidded by default.
6. Remove all unused fields / content to unclutter the reported issue. Remove this comment too.
-->
TransactionEndToEndTest. The cleanup method sporadically never completes since ManagedLedgerFactoryImpl.shutdown doesn't complete.
[example failure](https://github.com/apache/pulsar/pull/10351/checks?check_run_id=2427671612#step:10:2025) (use CMD/CTRL click to open in new tab)
```
"main" #1 prio=5 os_prio=0 tid=0x00007fd48400b800 nid=0x2a58 waiting on condition [0x00007fd48846d000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000c2082508> (a java.util.concurrent.CountDownLatch$Sync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:837)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:999)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1308)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)
at org.apache.bookkeeper.mledger.impl.ManagedLedgerFactoryImpl.shutdown(ManagedLedgerFactoryImpl.java:526)
at org.apache.pulsar.broker.ManagedLedgerClientFactory.close(ManagedLedgerClientFactory.java:129)
at org.apache.pulsar.broker.PulsarService.closeAsync(PulsarService.java:381)
.
.
.
at org.apache.pulsar.broker.PulsarService$MockitoMock$150664778.close(Unknown Source)
at org.apache.pulsar.broker.transaction.TransactionTestBase.internalCleanup(TransactionTestBase.java:248)
at org.apache.pulsar.client.impl.TransactionEndToEndTest.cleanup(TransactionEndToEndTest.java:124)
```
<details>
<summary>Full exception stacktrace</summary>
<code><pre>
"main" #1 prio=5 os_prio=0 tid=0x00007fd48400b800 nid=0x2a58 waiting on condition [0x00007fd48846d000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000c2082508> (a java.util.concurrent.CountDownLatch$Sync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:837)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:999)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1308)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)
at org.apache.bookkeeper.mledger.impl.ManagedLedgerFactoryImpl.shutdown(ManagedLedgerFactoryImpl.java:526)
at org.apache.pulsar.broker.ManagedLedgerClientFactory.close(ManagedLedgerClientFactory.java:129)
at org.apache.pulsar.broker.PulsarService.closeAsync(PulsarService.java:381)
at org.apache.pulsar.broker.PulsarService$MockitoMock$150664778.closeAsync$accessor$95jR1aDI(Unknown Source)
at org.apache.pulsar.broker.PulsarService$MockitoMock$150664778$auxiliary$QKnPJboY.call(Unknown Source)
at org.mockito.internal.invocation.RealMethod$FromCallable$1.call(RealMethod.java:40)
at org.mockito.internal.invocation.RealMethod$FromBehavior.invoke(RealMethod.java:62)
at org.mockito.internal.invocation.InterceptedInvocation.callRealMethod(InterceptedInvocation.java:141)
at org.mockito.internal.stubbing.answers.CallsRealMethods.answer(CallsRealMethods.java:44)
at org.mockito.Answers.answer(Answers.java:98)
at org.mockito.internal.handler.MockHandlerImpl.handle(MockHandlerImpl.java:106)
at org.mockito.internal.handler.NullResultGuardian.handle(NullResultGuardian.java:29)
at org.mockito.internal.handler.InvocationNotifierHandler.handle(InvocationNotifierHandler.java:33)
at org.mockito.internal.creation.bytebuddy.MockMethodInterceptor.doIntercept(MockMethodInterceptor.java:82)
at org.mockito.internal.creation.bytebuddy.MockMethodInterceptor.doIntercept(MockMethodInterceptor.java:56)
at org.mockito.internal.creation.bytebuddy.MockMethodInterceptor$DispatcherDefaultingToRealMethod.interceptSuperCallable(MockMethodInterceptor.java:141)
at org.apache.pulsar.broker.PulsarService$MockitoMock$150664778.closeAsync(Unknown Source)
at org.apache.pulsar.broker.PulsarService.close(PulsarService.java:314)
at org.apache.pulsar.broker.PulsarService$MockitoMock$150664778.close$accessor$95jR1aDI(Unknown Source)
at org.apache.pulsar.broker.PulsarService$MockitoMock$150664778$auxiliary$aFQDgg5T.call(Unknown Source)
at org.mockito.internal.invocation.RealMethod$FromCallable$1.call(RealMethod.java:40)
at org.mockito.internal.invocation.RealMethod$FromBehavior.invoke(RealMethod.java:62)
at org.mockito.internal.invocation.InterceptedInvocation.callRealMethod(InterceptedInvocation.java:141)
at org.mockito.internal.stubbing.answers.CallsRealMethods.answer(CallsRealMethods.java:44)
at org.mockito.Answers.answer(Answers.java:98)
at org.mockito.internal.handler.MockHandlerImpl.handle(MockHandlerImpl.java:106)
at org.mockito.internal.handler.NullResultGuardian.handle(NullResultGuardian.java:29)
at org.mockito.internal.handler.InvocationNotifierHandler.handle(InvocationNotifierHandler.java:33)
at org.mockito.internal.creation.bytebuddy.MockMethodInterceptor.doIntercept(MockMethodInterceptor.java:82)
at org.mockito.internal.creation.bytebuddy.MockMethodInterceptor.doIntercept(MockMethodInterceptor.java:56)
at org.mockito.internal.creation.bytebuddy.MockMethodInterceptor$DispatcherDefaultingToRealMethod.interceptSuperCallable(MockMethodInterceptor.java:141)
at org.apache.pulsar.broker.PulsarService$MockitoMock$150664778.close(Unknown Source)
at org.apache.pulsar.broker.transaction.TransactionTestBase.internalCleanup(TransactionTestBase.java:248)
at org.apache.pulsar.client.impl.TransactionEndToEndTest.cleanup(TransactionEndToEndTest.java:124)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
at org.testng.internal.MethodInvocationHelper.invokeMethodConsideringTimeout(MethodInvocationHelper.java:61)
at org.testng.internal.ConfigInvoker.invokeConfigurationMethod(ConfigInvoker.java:366)
at org.testng.internal.ConfigInvoker.invokeConfigurations(ConfigInvoker.java:320)
at org.testng.internal.TestInvoker.runConfigMethods(TestInvoker.java:701)
at org.testng.internal.TestInvoker.runAfterGroupsConfigurations(TestInvoker.java:677)
at org.testng.internal.TestInvoker.invokeMethod(TestInvoker.java:661)
at org.testng.internal.TestInvoker.invokeTestMethod(TestInvoker.java:174)
at org.testng.internal.MethodRunner.runInSequence(MethodRunner.java:46)
at org.testng.internal.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:822)
at org.testng.internal.TestInvoker.invokeTestMethods(TestInvoker.java:147)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:146)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:128)
at org.testng.TestRunner$$Lambda$96/70386506.accept(Unknown Source)
at java.util.ArrayList.forEach(ArrayList.java:1259)
at org.testng.TestRunner.privateRun(TestRunner.java:764)
at org.testng.TestRunner.run(TestRunner.java:585)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:384)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:378)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:337)
at org.testng.SuiteRunner.run(SuiteRunner.java:286)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:53)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:96)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1218)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1140)
at org.testng.TestNG.runSuites(TestNG.java:1069)
at org.testng.TestNG.run(TestNG.java:1037)
at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:135)
at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeSingleClass(TestNGDirectoryTestSuite.java:112)
at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeLazy(TestNGDirectoryTestSuite.java:123)
at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:90)
at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:146)
at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
</pre></code>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari commented on issue #10356: Flaky-test: TransactionEndToEndTest.cleanup / ManagedLedgerFactoryImpl.shutdown doesn't complete (causes "CI - Unit - Brokers - Other" to hang and timeout)
Posted by GitBox <gi...@apache.org>.
lhotari commented on issue #10356:
URL: https://github.com/apache/pulsar/issues/10356#issuecomment-826249963
thread dump https://jstack.review/?https://gist.github.com/lhotari/f300646746ea45fa5c60f8478b438cb5#tda_1_dump
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui closed issue #10356: Flaky-test: TransactionEndToEndTest.cleanup / ManagedLedgerFactoryImpl.shutdown doesn't complete (causes "CI - Unit - Brokers - Other" to hang and timeout)
Posted by GitBox <gi...@apache.org>.
codelipenghui closed issue #10356:
URL: https://github.com/apache/pulsar/issues/10356
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui closed issue #10356: Flaky-test: TransactionEndToEndTest.cleanup / ManagedLedgerFactoryImpl.shutdown doesn't complete (causes "CI - Unit - Brokers - Other" to hang and timeout)
Posted by GitBox <gi...@apache.org>.
codelipenghui closed issue #10356:
URL: https://github.com/apache/pulsar/issues/10356
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari commented on issue #10356: Flaky-test: TransactionEndToEndTest.cleanup / ManagedLedgerFactoryImpl.shutdown doesn't complete (causes "CI - Unit - Brokers - Other" to hang and timeout)
Posted by GitBox <gi...@apache.org>.
lhotari commented on issue #10356:
URL: https://github.com/apache/pulsar/issues/10356#issuecomment-826249963
thread dump https://jstack.review/?https://gist.github.com/lhotari/f300646746ea45fa5c60f8478b438cb5#tda_1_dump
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org