You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Yun Gao (JIRA)" <ji...@apache.org> on 2018/11/01 03:06:00 UTC
[jira] [Commented] (FLINK-10742) Let Netty use Flink's buffers
directly in credit-based mode
[ https://issues.apache.org/jira/browse/FLINK-10742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16671052#comment-16671052 ]
Yun Gao commented on FLINK-10742:
---------------------------------
Hi Nico, we have also worked on this problem in recent days and have implemented a version of receiver-side zero-copy handlers. The implementation added a new ZeroCopyNettyMessageDecoder which copies the network stream directly into the Flink buffer applied from the target InputChannel. Experiments shows that the direct memory used by Netty is largely decreased with the receiver-side zero-copy. So would you like me to have a try on this problem? :) I think I can finish the design doc and open a PR with revised implementation in less than one week.
> Let Netty use Flink's buffers directly in credit-based mode
> -----------------------------------------------------------
>
> Key: FLINK-10742
> URL: https://issues.apache.org/jira/browse/FLINK-10742
> Project: Flink
> Issue Type: Sub-task
> Components: Network
> Affects Versions: 1.7.0
> Reporter: Nico Kruber
> Priority: Major
>
> For credit-based flow control, we always have buffers available for data that is sent to use. We could thus use them directly and not copy the network stream into Netty buffers first and then into our buffers.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)