You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@pekko.apache.org by "He-Pin (via GitHub)" <gi...@apache.org> on 2025/01/13 20:33:20 UTC
[PR] chore: Add support for controlling the NettyTransport's byteBuf allocator type. [pekko]
He-Pin opened a new pull request, #1707:
URL: https://github.com/apache/pekko/pull/1707
Motivation:
As a response to https://github.com/apache/flink/pull/25866
@davidradl want to control the bytebuf allocator type.
Modification:
Add `bytebuf-allocator-type` selector , with `pooled` by default.
Result:
The bytebuf allocator can be configured now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [PR] chore: Add support for controlling the NettyTransport's byteBuf allocator type. [pekko]
Posted by "pjfanning (via GitHub)" <gi...@apache.org>.
pjfanning commented on PR #1707:
URL: https://github.com/apache/pekko/pull/1707#issuecomment-2589563482
> @pjfanning Wdyt about backpacking this to 1.1.x？
Maybe. Let's see what other people say about the PR.
Even if we do backport it, we would still need to do a release and are Flink team that interested in this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [PR] chore: Add support for controlling the NettyTransport's byteBuf allocator type. [pekko]
Posted by "He-Pin (via GitHub)" <gi...@apache.org>.
He-Pin commented on code in PR #1707:
URL: https://github.com/apache/pekko/pull/1707#discussion_r1913804474
##########
remote/src/main/scala/org/apache/pekko/remote/transport/netty/NettyTransport.scala:
##########
@@ -160,6 +167,17 @@ class NettyTransportSettings(config: Config) {
case _ => getBoolean("tcp-reuse-addr")
}
+ val ByteBufAllocator: ByteBufAllocator = getString("bytebuf-allocator-type") match {
+ case "pooled" => PooledByteBufAllocator.DEFAULT
+ case "unpooled" => UnpooledByteBufAllocator.DEFAULT
+ case "unpooled-heap" => new UnpooledByteBufAllocator(false)
+ case "adaptive" => new AdaptiveByteBufAllocator()
+ case "adaptive-heap" => new AdaptiveByteBufAllocator(false)
+ case other => throw new IllegalArgumentException(
+ "Unknown 'bytebuf-allocator-type' [" + other + "]," +
Review Comment:
I will do this when find time, stress testing to 4 am, on bed now, feel free to update this branch ,thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [PR] chore: Add support for controlling the NettyTransport's byteBuf allocator type. [pekko]
Posted by "He-Pin (via GitHub)" <gi...@apache.org>.
He-Pin merged PR #1707:
URL: https://github.com/apache/pekko/pull/1707
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [PR] chore: Add support for controlling the NettyTransport's byteBuf allocator type. [pekko]
Posted by "He-Pin (via GitHub)" <gi...@apache.org>.
He-Pin commented on PR #1707:
URL: https://github.com/apache/pekko/pull/1707#issuecomment-2589545943
@pjfanning Wdyt about backpacking this to 1.1.x？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [PR] chore: Add support for controlling the NettyTransport's byteBuf allocator type. [pekko]
Posted by "He-Pin (via GitHub)" <gi...@apache.org>.
He-Pin commented on PR #1707:
URL: https://github.com/apache/pekko/pull/1707#issuecomment-2589575343
@a1kemist asked for this.
TBH from https://github.com/netty/netty/discussions/14556#discussioncomment-11829840 , migrating to Netty 4 will improve performance with the ByteBufAllocator being shared.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [PR] chore: Add support for controlling the NettyTransport's byteBuf allocator type. [pekko]
Posted by "pjfanning (via GitHub)" <gi...@apache.org>.
pjfanning commented on code in PR #1707:
URL: https://github.com/apache/pekko/pull/1707#discussion_r1913796013
##########
remote/src/main/scala/org/apache/pekko/remote/transport/netty/NettyTransport.scala:
##########
@@ -160,6 +167,17 @@ class NettyTransportSettings(config: Config) {
case _ => getBoolean("tcp-reuse-addr")
}
+ val ByteBufAllocator: ByteBufAllocator = getString("bytebuf-allocator-type") match {
+ case "pooled" => PooledByteBufAllocator.DEFAULT
+ case "unpooled" => UnpooledByteBufAllocator.DEFAULT
+ case "unpooled-heap" => new UnpooledByteBufAllocator(false)
+ case "adaptive" => new AdaptiveByteBufAllocator()
+ case "adaptive-heap" => new AdaptiveByteBufAllocator(false)
+ case other => throw new IllegalArgumentException(
+ "Unknown 'bytebuf-allocator-type' [" + other + "]," +
Review Comment:
Could you add a def (package private) of form `private[netty] def deriveByteBufAllocator(allocatorType: String): ByteBufAllocator` and use this here?
The def can be tested in a unit test to see how it handles the 5 input types (and an invalid input too).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org