You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/08/08 04:17:56 UTC
[GitHub] [pulsar] codelipenghui opened a new issue #11593: java.lang.NoSuchMethodError: java.nio.ByteBuffer.position(I)Ljava/nio/ByteBuffer when enabling topic metadata compression
codelipenghui opened a new issue #11593:
URL: https://github.com/apache/pulsar/issues/11593
**Describe the bug**
We have introduced managed ledger metadata compression in https://github.com/apache/pulsar/pull/11490, but we will get `java.lang.NoSuchMethodError: java.nio.ByteBuffer.position(I)Ljava/nio/ByteBuffer;` when run Pulsar on JDK1.8.
```
17:36:54.799 [mock-pulsar-bk-OrderedExecutor-0-0] ERROR org.apache.bookkeeper.common.util.SafeRunnable - Unexpected throwable caught
java.lang.NoSuchMethodError: java.nio.ByteBuffer.position(I)Ljava/nio/ByteBuffer;
at io.airlift.compress.zstd.ZstdCompressor.compress(ZstdCompressor.java:101) ~[aircompressor-0.19.jar:0.19]
at org.apache.pulsar.common.compression.CompressionCodecZstd.encode(CompressionCodecZstd.java:75) ~[pulsar-common-2.8.1-SNAPSHOT.jar:2.8.1-SNAPSHOT]
at org.apache.bookkeeper.mledger.impl.MetaStoreImpl.compressLedgerInfo(MetaStoreImpl.java:321) ~[managed-ledger-2.8.1-SNAPSHOT.jar:2.8.1-SNAPSHOT]
at org.apache.bookkeeper.mledger.impl.MetaStoreImpl.asyncUpdateLedgerIds(MetaStoreImpl.java:134) ~[managed-ledger-2.8.1-SNAPSHOT.jar:2.8.1-SNAPSHOT]
at org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl.lambda$null$1(ManagedLedgerImpl.java:491) ~[managed-ledger-2.8.1-SNAPSHOT.jar:2.8.1-SNAPSHOT]
at org.apache.bookkeeper.mledger.util.SafeRun$1.safeRun(SafeRun.java:32) ~[managed-ledger-2.8.1-SNAPSHOT.jar:2.8.1-SNAPSHOT]
at org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36) [bookkeeper-common-4.14.1.jar:4.14.1]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_281]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_281]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-common-4.1.66.Final.jar:4.1.66.Final]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_281]
17:36:58.356 [pulsar-load-manager-120-1] INFO org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl - Writing local data to metadata store because maximum change 59.76470708847046% exceeded threshold 10%; time since last report written is 5.02 seconds
17:37:11.277 [TestNG-method=testRestartBrokerEnableManagedLedgerInfoCompression-1] INFO org.apache.pulsar.client.impl.ConsumerImpl - [prop/ns-abc/topic-5d9785f202336c4c] [test] Closed Consumer (not connected)
17:37:11.277 [TestNG-method=testRestartBrokerEnableManagedLedgerInfoCompression-1] INFO org.apache.pulsar.client.impl.ProducerImpl - [prop/ns-abc/topic-5d9785f202336c4c] [test-1-0] Closed Producer (not connected)
!!!!!!!!! FAILURE-- [TestClass name=class org.apache.pulsar.broker.service.ManagedLedgerCompressionTest].testRestartBrokerEnableManagedLedgerInfoCompression([])-------
```
For branch-2.8 and master, build on JDK8 or JDK11, run on JDK8 can works.
For branch-2.8 and master, build on JDK8 or JDK11, run on JDK11 can works.
But for branch-2.7, both are works.
After investigating the issue, it's related to the `aircompressor` upgrade, we have upgraded `aircompressor` from 0.16 to 0.19 https://github.com/apache/pulsar/pull/10983. After upgrade the `aircompressor` version for branch-2.7, the error appears.
Looks like related to the `aircompressor`, maybe need to add --release 1.8 for `aircompressor`.
The producer also using the `aircompressor` to compress data, but after the test on both branch-2.7, branch-2.8, and master, it works, no error happens. The difference is the producer run into this branch https://github.com/apache/pulsar/blob/6704f12104219611164aa2bb5bbdfc929613f1bf/pulsar-common/src/main/java/org/apache/pulsar/common/compression/CompressionCodecZstd.java#L65 but the managed ledger metadata compression run into here https://github.com/apache/pulsar/blob/6704f12104219611164aa2bb5bbdfc929613f1bf/pulsar-common/src/main/java/org/apache/pulsar/common/compression/CompressionCodecZstd.java#L72
**To Reproduce**
Steps to reproduce the behavior:
1. Build the master branch
2. Change managedLedgerInfoCompressionType=ZSTD in broker.conf
3. Start the standalone
4. See error
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] merlimat closed issue #11593: java.lang.NoSuchMethodError: java.nio.ByteBuffer.position(I)Ljava/nio/ByteBuffer when enabling topic metadata compression
Posted by GitBox <gi...@apache.org>.
merlimat closed issue #11593:
URL: https://github.com/apache/pulsar/issues/11593
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari commented on issue #11593: java.lang.NoSuchMethodError: java.nio.ByteBuffer.position(I)Ljava/nio/ByteBuffer when enabling topic metadata compression
Posted by GitBox <gi...@apache.org>.
lhotari commented on issue #11593:
URL: https://github.com/apache/pulsar/issues/11593#issuecomment-894832866
This looks like a bug in aircompressor library. it's not specifying the release configuration for maven compiler plugin as described in https://www.morling.dev/blog/bytebuffer-and-the-dreaded-nosuchmethoderror/ .
In Pulsar this was solved with #10343 .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] merlimat commented on issue #11593: java.lang.NoSuchMethodError: java.nio.ByteBuffer.position(I)Ljava/nio/ByteBuffer when enabling topic metadata compression
Posted by GitBox <gi...@apache.org>.
merlimat commented on issue #11593:
URL: https://github.com/apache/pulsar/issues/11593#issuecomment-1019328867
This was fixed in #11790
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org