You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "lhotari (via GitHub)" <gi...@apache.org> on 2024/06/14 16:36:50 UTC
[PR] [fix][fn] Enable optimized Netty direct byte buffer support for Pulsar Function runtimes [pulsar]
lhotari opened a new pull request, #22910:
URL: https://github.com/apache/pulsar/pull/22910
Fixes #22909
### Motivation
Netty's optimized direct byte buffer support is needed for high performance. Currently these debug messages will get logged when Netty library gets initialized:
* "direct buffer constructor: unavailable: Unable to make private java.nio.DirectByteBuffer(long,int) accessible: module java.base does not "opens java.nio" to unnamed module"
* "jdk.internal.misc.Unsafe.allocateUninitializedArray(int): unavailable: class io.netty.util.internal.PlatformDependent0$7 cannot access class jdk.internal.misc.Unsafe (in module java.base) because module java.base does not export jdk.internal.misc to unnamed module"
* "java.nio.DirectByteBuffer.<init>(long, {int,long}): unavailable {}"
Another detail is that Pulsar uses com.scurrilous.circe.checksum.Crc32cIntChecksum class for checksum calculation. For optimized checksum calculation Pulsar attempt to load libcirce-checksum native library. When that isn't available, com.scurrilous.circe.checksum.Java9IntHash is used. That only works when "--add-opens java.base/java.util.zip=ALL-UNNAMED" is passed on the command line.
### Modifications
Pass these parameters in the Java runtime command line:
```
-Dio.netty.tryReflectionSetAccessible=true
-Dorg.apache.pulsar.shade.io.netty.tryReflectionSetAccessible=true
-Dio.grpc.netty.shaded.io.netty.tryReflectionSetAccessible=true
--add-opens java.base/java.nio=ALL-UNNAMED
--add-opens java.base/jdk.internal.misc=ALL-UNNAMED
--add-opens java.base/java.util.zip=ALL-UNNAMED
```
remove `--add-opens java.base/sun.net=ALL-UNNAMED` added in #16423 since it's no longer needed after #22329
### Documentation
<!-- DO NOT REMOVE THIS SECTION. CHECK THE PROPER BOX ONLY. -->
- [ ] `doc` <!-- Your PR contains doc changes. -->
- [ ] `doc-required` <!-- Your PR changes impact docs and you will update later -->
- [x] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-complete` <!-- Docs have been already added -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [fix][fn] Enable optimized Netty direct byte buffer support for Pulsar Function runtimes [pulsar]
Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari merged PR #22910:
URL: https://github.com/apache/pulsar/pull/22910
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org