You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/02/04 14:18:36 UTC
[GitHub] [pulsar] eolivelli opened a new pull request #14121: Prevent StackOverFlowException in KEY_SHARED subscription
eolivelli opened a new pull request #14121:
URL: https://github.com/apache/pulsar/pull/14121
Fix a StackOverFlowException that we have seen on Pulsar in some non-OSS user application.
**Problem**
We are calling readMoreEntries() recursively.
**Modifications**
The fix is to schedule the read in another thread, like we did in apache/pulsar#10696
**Details**
The stacktrace is taken from the DataStax fork of Pulsar version 2.8.0_1.1.16, but the problem still applies to master, 2.9 and 2.8 (I didn't check 2.7)
```
> 2022-02-02_17:56:18.396 [pulsar] STDOUT: MultipleConsumers.sendMessagesToConsumers(PersistentStickyKeyDispatcherMultipleConsumers.java:286) ~[com.datastax.oss-pulsar-broker-2.8.0.1.1.16.jar:2.8.0.1.1.16]
2022-02-02_17:56:18.396 [pulsar] STDOUT: at org.apache.pulsar.broker.service.persistent.PersistentDispatcherMultipleConsumers.readEntriesComplete(PersistentDispatcherMultipleConsumers.java:473) ~[com.datastax.oss-pulsar-broker-2.8.0.1.1.16.jar:2.8.0.1.1.16]
2022-02-02_17:56:18.396 [pulsar] STDOUT: at org.apache.bookkeeper.mledger.impl.ManagedCursorImpl$11.readEntryComplete(ManagedCursorImpl.java:1251) ~[com.datastax.oss-managed-ledger-2.8.0.1.1.16.jar:2.8.0.1.1.16]
2022-02-02_17:56:18.396 [pulsar] STDOUT: at org.apache.bookkeeper.mledger.impl.EntryCacheImpl.asyncReadEntry0(EntryCacheImpl.java:211) ~[com.datastax.oss-managed-ledger-2.8.0.1.1.16.jar:2.8.0.1.1.16]
2022-02-02_17:56:18.396 [pulsar] STDOUT: at org.apache.bookkeeper.mledger.impl.EntryCacheImpl.asyncReadEntry(EntryCacheImpl.java:190) ~[com.datastax.oss-managed-ledger-2.8.0.1.1.16.jar:2.8.0.1.1.16]
2022-02-02_17:56:18.396 [pulsar] STDOUT: at org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl.asyncReadEntry(ManagedLedgerImpl.java:1916) ~[com.datastax.oss-managed-ledger-2.8.0.1.1.16.jar:2.8.0.1.1.16]
2022-02-02_17:56:18.396 [pulsar] STDOUT: at org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl.asyncReadEntry(ManagedLedgerImpl.java:1837) ~[com.datastax.oss-managed-ledger-2.8.0.1.1.16.jar:2.8.0.1.1.16]
2022-02-02_17:56:18.396 [pulsar] STDOUT: at org.apache.bookkeeper.mledger.impl.ManagedCursorImpl.lambda$asyncReplayEntries$9(ManagedCursorImpl.java:1276) ~[com.datastax.oss-managed-ledger-2.8.0.1.1.16.jar:2.8.0.1.1.16]
2022-02-02_17:56:18.396 [pulsar] STDOUT: at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) ~[?:?]
2022-02-02_17:56:18.396 [pulsar] STDOUT: at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177) ~[?:?]
2022-02-02_17:56:18.397 [pulsar] STDOUT: at java.util.TreeMap$KeySpliterator.forEachRemaining(TreeMap.java:2739) ~[?:?]
2022-02-02_17:56:18.397 [pulsar] STDOUT: at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]
2022-02-02_17:56:18.397 [pulsar] STDOUT: at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]
2022-02-02_17:56:18.397 [pulsar] STDOUT: at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) ~[?:?]
2022-02-02_17:56:18.397 [pulsar] STDOUT: at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) ~[?:?]
2022-02-02_17:56:18.397 [pulsar] STDOUT: at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
2022-02-02_17:56:18.397 [pulsar] STDOUT: at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497) ~[?:?]
2022-02-02_17:56:18.397 [pulsar] STDOUT: at org.apache.bookkeeper.mledger.impl.ManagedCursorImpl.asyncReplayEntries(ManagedCursorImpl.java:1270) ~[com.datastax.oss-managed-ledger-2.8.0.1.1.16.jar:2.8.0.1.1.16]
2022-02-02_17:56:18.397 [pulsar] STDOUT: at org.apache.pulsar.broker.service.persistent.PersistentDispatcherMultipleConsumers.asyncReplayEntriesInOrder(PersistentDispatcherMultipleConsumers.java:365) ~[com.datastax.oss-pulsar-broker-2.8.0.1.1.16.jar:2.8.0.1.1.16]
2022-02-02_17:56:18.397 [pulsar] STDOUT: at org.apache.pulsar.broker.service.persistent.PersistentDispatcherMultipleConsumers.readMoreEntries(PersistentDispatcherMultipleConsumers.java:247) ~[com.datastax.oss-pulsar-broker-2.8.0.1.1.16.jar:2.8.0.1.1.16]
2022-02-02_17:56:18.397 [pulsar] STDOUT: at org.apache.pulsar.broker.service.persistent.PersistentStickyKeyDispatcherMultipleConsumers.sendMessagesToConsumers(PersistentStickyKeyDispatcherMultipleConsumers.java:286) ~[com.datastax.oss-pulsar-broker-2.8.0.1.1.16.jar:2.8.0.1.1.16]
2022-02-02_17:56:18.397 [pulsar] STDOUT: at org.apache.pulsar.broker.service.persistent.PersistentDispatcherMultipleConsumers.readEntriesComplete(PersistentDispatcherMultipleConsumers.java:473) ~[com.datastax.oss-pulsar-broker-2.8.0.1.1.16.jar:2.8.0.1.1.16]
2022-02-02_17:56:18.397 [pulsar] STDOUT: at org.apache.bookkeeper.mledger.impl.ManagedCursorImpl$11.readEntryComplete(ManagedCursorImpl.java:1251) ~[com.datastax.oss-managed-ledger-2.8.0.1.1.16.jar:2.8.0.1.1.16]
2022-02-02_17:56:18.397 [pulsar] STDOUT: at org.apache.bookkeeper.mledger.impl.EntryCacheImpl.asyncReadEntry0(EntryCacheImpl.java:211) ~[com.datastax.oss-managed-ledger-2.8.0.1.1.16.jar:2.8.0.1.1.16]
2022-02-02_17:56:18.397 [pulsar] STDOUT: at org.apache.bookkeeper.mledger.impl.EntryCacheImpl.asyncReadEntry(EntryCacheImpl.java:190) ~[com.datastax.oss-managed-ledger-2.8.0.1.1.16.jar:2.8.0.1.1.16]
2022-02-02_17:56:18.397 [pulsar] STDOUT: at org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl.asyncReadEntry(ManagedLedgerImpl.java:1916) ~[com.datastax.oss-managed-ledger-2.8.0.1.1.16.jar:2.8.0.1.1.16]
2022-02-02_17:56:18.397 [pulsar] STDOUT: at org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl.asyncReadEntry(ManagedLedgerImpl.java:1837) ~[com.datastax.oss-managed-ledger-2.8.0.1.1.16.jar:2.8.0.1.1.16]
2022-02-02_17:56:18.397 [pulsar] STDOUT: at org.apache.bookkeeper.mledger.impl.ManagedCursorImpl.lambda$asyncReplayEntries$9(ManagedCursorImpl.java:1276) ~[com.datastax.oss-managed-ledger-2.8.0.1.1.16.jar:2.8.0.1.1.16]
2022-02-02_17:56:18.397 [pulsar] STDOUT: at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) ~[?:?]
2022-02-02_17:56:18.397 [pulsar] STDOUT: at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177) ~[?:?]
2022-02-02_17:56:18.397 [pulsar] STDOUT: at java.util.TreeMap$KeySpliterator.forEachRemaining(TreeMap.java:2739) ~[?:?]
2022-02-02_17:56:18.397 [pulsar] STDOUT: at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]
2022-02-02_17:56:18.397 [pulsar] STDOUT: at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]
2022-02-02_17:56:18.397 [pulsar] STDOUT: at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) ~[?:?]
2022-02-02_17:56:18.397 [pulsar] STDOUT: at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) ~[?:?]
2022-02-02_17:56:18.397 [pulsar] STDOUT: at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
2022-02-02_17:56:18.398 [pulsar] STDOUT: at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497) ~[?:?]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] eolivelli commented on pull request #14121: Prevent StackOverFlowException in KEY_SHARED subscription
Posted by GitBox <gi...@apache.org>.
eolivelli commented on pull request #14121:
URL: https://github.com/apache/pulsar/pull/14121#issuecomment-1030827489
@Jason918 I fixed the only case I have seen.
You are right, but I am not sure that we will fall into that case in all of those points.
Probably we should rework readMoreEntries() in general, but this would be outside the scope of this patch.
You know the motto "don't fix things that are not broken"
I would commit this patch as I found a way (in an integration test of a non OSS client application) to trigger the problem systematically, and I am sure that it must be fixed in this point. (But I am not able to contribute easily a reproducer)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] eolivelli commented on pull request #14121: Prevent StackOverFlowException in KEY_SHARED subscription
Posted by GitBox <gi...@apache.org>.
eolivelli commented on pull request #14121:
URL: https://github.com/apache/pulsar/pull/14121#issuecomment-1030031056
@massakam you did a similar fix, please take a look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] eolivelli merged pull request #14121: Prevent StackOverFlowException in KEY_SHARED subscription
Posted by GitBox <gi...@apache.org>.
eolivelli merged pull request #14121:
URL: https://github.com/apache/pulsar/pull/14121
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org