You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "Eywek (via GitHub)" <gi...@apache.org> on 2023/03/29 13:10:49 UTC
[GitHub] [pulsar] Eywek opened a new issue, #19963: [Bug] ConcurrentModificationException when fetching messages
Eywek opened a new issue, #19963:
URL: https://github.com/apache/pulsar/issues/19963
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
### Version
2.7.2 see [Apache Pinot Apache Pulsar stream ingestion plugin pom.xml](https://github.com/apache/pinot/blob/0bbb9455eb7aa374e0c8b1e2c5ec0a4d6cd7a93e/pinot-plugins/pinot-stream-ingestion/pinot-pulsar/pom.xml#L40) on Debian bullseye
### Minimal reproduce step
Hard to reproduce but we're using Apache Pinot [Apache Pulsar stream ingestion plugin](https://docs.pinot.apache.org/basics/data-import/pinot-stream-ingestion/apache-pulsar) to retrieve messages from Pulsar (see implementation code [here](https://github.com/apache/pinot/blob/0bbb9455eb7aa374e0c8b1e2c5ec0a4d6cd7a93e/pinot-plugins/pinot-stream-ingestion/pinot-pulsar/src/main/java/org/apache/pinot/plugin/stream/pulsar/PulsarPartitionLevelConsumer.java#L73-L96)).
We're using the commit https://github.com/apache/pinot/tree/0bbb9455eb7aa374e0c8b1e2c5ec0a4d6cd7a93e for Apache Pinot.
### What did you expect to see?
No error in logs
### What did you see instead?
```
Mar 29 08:50:57 pinot-hosts-9-server bash[2258505]: 2023/03/29 08:50:57.822 ERROR [LLRealtimeSegmentDataManager_worker_datasource_636bd9305a9600010051fb4d_641de187ab174a000713364a_35__0__8__20230329T0846Z] [worker_datasource_636bd9305a9600010051fb4d_641de187ab174a000713364a_35__0__8__20230329T0846Z] Exception while in work
Mar 29 08:50:57 pinot-hosts-9-server bash[2258505]: java.util.ConcurrentModificationException: null
Mar 29 08:50:57 pinot-hosts-9-server bash[2258505]: at java.util.ArrayList$Itr.checkForComodification(Unknown Source) ~[?:?]
Mar 29 08:50:57 pinot-hosts-9-server bash[2258505]: at java.util.ArrayList$Itr.next(Unknown Source) ~[?:?]
Mar 29 08:50:57 pinot-hosts-9-server bash[2258505]: at org.apache.pinot.shaded.com.google.common.collect.Iterators$6.computeNext(Iterators.java:616) ~[pinot-all-0.13.0-SNAPSHOT-jar-with-dependencies.jar:0.13.0-SNAPSHOT-0bbb9455eb7aa374e0c8b1e2c5ec0a4d6cd7a93e]
Mar 29 08:50:57 pinot-hosts-9-server bash[2258505]: at org.apache.pinot.shaded.com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:145) ~[pinot-all-0.13.0-SNAPSHOT-jar-with-dependencies.jar:0.13.0-SNAPSHOT-0bbb9455eb7aa374e0c8b1e2c5ec0a4d6cd7a93e]
Mar 29 08:50:57 pinot-hosts-9-server bash[2258505]: at org.apache.pinot.shaded.com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:140) ~[pinot-all-0.13.0-SNAPSHOT-jar-with-dependencies.jar:0.13.0-SNAPSHOT-0bbb9455eb7aa374e0c8b1e2c5ec0a4d6cd7a93e]
Mar 29 08:50:57 pinot-hosts-9-server bash[2258505]: at java.lang.Iterable.forEach(Unknown Source) ~[?:?]
Mar 29 08:50:57 pinot-hosts-9-server bash[2258505]: at org.apache.pinot.plugin.stream.pulsar.PulsarMessageBatch.<init>(PulsarMessageBatch.java:50) ~[pinot-pulsar-0.13.0-SNAPSHOT-shaded.jar:0.13.0-SNAPSHOT-68b4096745272706af3e09f08ecbbd89eac7e3a3]
Mar 29 08:50:57 pinot-hosts-9-server bash[2258505]: at org.apache.pinot.plugin.stream.pulsar.PulsarPartitionLevelConsumer.fetchMessages(PulsarPartitionLevelConsumer.java:89) ~[pinot-pulsar-0.13.0-SNAPSHOT-shaded.jar:0.13.0-SNAPSHOT-68b4096745272706af3e09f08ecbbd89eac7e3a3]
Mar 29 08:50:57 pinot-hosts-9-server bash[2258505]: at org.apache.pinot.core.data.manager.realtime.LLRealtimeSegmentDataManager.consumeLoop(LLRealtimeSegmentDataManager.java:427) ~[pinot-all-0.13.0-SNAPSHOT-jar-with-dependencies.jar:0.13.0-SNAPSHOT-0bbb9455eb7aa374e0c8b1e2c5ec0a4d6cd7a93e]
Mar 29 08:50:57 pinot-hosts-9-server bash[2258505]: at org.apache.pinot.core.data.manager.realtime.LLRealtimeSegmentDataManager$PartitionConsumer.run(LLRealtimeSegmentDataManager.java:648) [pinot-all-0.13.0-SNAPSHOT-jar-with-dependencies.jar:0.13.0-SNAPSHOT-0bbb9455eb7aa374e0c8b1e2c5ec0a4d6cd7a93e]
Mar 29 08:50:57 pinot-hosts-9-server bash[2258505]: at java.lang.Thread.run(Unknown Source) [?:?]
```
### Anything else?
It seems quite rare to have this issue but some large topic (more than 16 millions entries in few hours) can have them repeatedly sometimes
### Are you willing to submit a PR?
- [X] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Eywek commented on issue #19963: [Bug] ConcurrentModificationException when fetching messages
Posted by "Eywek (via GitHub)" <gi...@apache.org>.
Eywek commented on issue #19963:
URL: https://github.com/apache/pulsar/issues/19963#issuecomment-1488587903
Ignore it, I'm dumb, it's an issue with Apache Pinot implementation
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Eywek closed issue #19963: [Bug] ConcurrentModificationException when fetching messages
Posted by "Eywek (via GitHub)" <gi...@apache.org>.
Eywek closed issue #19963: [Bug] ConcurrentModificationException when fetching messages
URL: https://github.com/apache/pulsar/issues/19963
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org