You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2018/10/08 07:29:17 UTC
[GitHub] rdhabalia opened a new pull request #2745: Fix: deadlock on
blocking ml operation
rdhabalia opened a new pull request #2745: Fix: deadlock on blocking ml operation
URL: https://github.com/apache/pulsar/pull/2745
### Motivation
Found a deadlock in broker with below jstack.
[Jstack](https://gist.github.com/rdhabalia/9cffa5005716d077a8a767a0cc6afd1b)
expireMessages thread is waiting on topic-map and ml-operation couldn't complete which created a deadlock at broker.
```
Thread 17133: (state = BLOCKED)
- sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise)
- java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled frame)
- java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() @bci=1, line=836 (Compiled frame)
- java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(int) @bci=72, line=997 (Compiled frame)
- java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(int) @bci=24, line=1304 (Compiled frame)
- java.util.concurrent.CountDownLatch.await() @bci=5, line=231 (Compiled frame)
- org.apache.bookkeeper.mledger.impl.ManagedCursorImpl.getNthEntry(int, org.apache.bookkeeper.mledger.ManagedCursor$IndividualDeletedEntries) @bci=38, line=470 (Interpreted frame)
- org.apache.pulsar.broker.service.persistent.PersistentTopic.isOldestMessageExpired(org.apache.bookkeeper.mledger.ManagedCursor, long) @bci=14, line=1646 (Interpreted frame)
- org.apache.pulsar.broker.service.persistent.PersistentReplicator.expireMessages(int) @bci=40, line=571 (Compiled frame)
- org.apache.pulsar.broker.service.persistent.PersistentTopic.lambda$32(org.apache.pulsar.common.policies.data.Policies, java.lang.String, org.apache.pulsar.broker.service.Replicator) @bci=8, line=963 (Compiled frame)
- org.apache.pulsar.broker.service.persistent.PersistentTopic$$Lambda$407.accept(java.lang.Object, java.lang.Object) @bci=12 (Compiled frame)
- org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap$Section.forEach(java.util.function.BiConsumer) @bci=121, line=386 (Compiled frame)
- org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap.forEach(java.util.function.BiConsumer) @bci=26, line=160 (Compiled frame)
- org.apache.pulsar.broker.service.persistent.PersistentTopic.checkMessageExpiry() @bci=86, line=963 (Compiled frame)
- org.apache.pulsar.broker.service.BrokerService$$Lambda$366.accept(java.lang.Object) @bci=4 (Compiled frame)
- org.apache.pulsar.broker.service.BrokerService.lambda$20(java.util.function.Consumer, java.lang.String, java.util.concurrent.CompletableFuture) @bci=20, line=887 (Compiled frame)
- org.apache.pulsar.broker.service.BrokerService$$Lambda$349.accept(java.lang.Object, java.lang.Object) @bci=12 (Compiled frame)
- org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap$Section.forEach(java.util.function.BiConsumer) @bci=121, line=386 (Compiled frame)
- org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap.forEach(java.util.function.BiConsumer) @bci=26, line=160 (Compiled frame)
- org.apache.pulsar.broker.service.BrokerService.forEachTopic(java.util.function.Consumer) @bci=10, line=884 (Compiled frame)
- org.apache.pulsar.broker.service.BrokerService.checkMessageExpiry() @bci=6, line=861 (Interpreted frame)
- org.apache.pulsar.broker.service.BrokerService$$Lambda$135.run() @bci=4 (Interpreted frame)
- org.apache.bookkeeper.mledger.util.SafeRun$1.safeRun() @bci=4, line=32 (Compiled frame)
- org.apache.bookkeeper.common.util.SafeRunnable.run() @bci=1, line=36 (Compiled frame)
- java.util.concurrent.Executors$RunnableAdapter.call() @bci=4, line=511 (Compiled frame)
- java.util.concurrent.FutureTask.runAndReset() @bci=47, line=308 (Compiled frame)
- java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask) @bci=1, line=180 (Compiled frame)
- java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run() @bci=37, line=294 (Compiled frame)
- java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=95, line=1142 (Compiled frame)
- java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame)
- io.netty.util.concurrent.FastThreadLocalRunnable.run() @bci=4, line=30 (Interpreted frame)
- java.lang.Thread.run() @bci=11, line=748 (Interpreted frame)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services