You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "neoremind (JIRA)" <ji...@apache.org> on 2017/08/11 04:29:00 UTC
[jira] [Updated] (SPARK-21703) Why RPC message are transferred with
header and body separately in TCP frame
[ https://issues.apache.org/jira/browse/SPARK-21703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
neoremind updated SPARK-21703:
------------------------------
Description:
After seeing the details of how spark leverage netty, I found one question, typically RPC message wire format would have a header+payload structure, and netty uses a TransportFrameDecoder to deal with how to determine a complete message from remote peer. But after using Wireshark sniffing tool, I found that the message are sent separately with header and then followed by body, although this works fine, but for underlying TCP there would be ACK segments sent back to acknowledge, there might be a little bit redundancy since we can sent them together and the header are usually very small.
The main reason can be found in MessageWithHeader class, since transferTo method write tow times for header and body.
Could some one help me understand the background story on how to implement in such way? Thanks!
was:
After seeing the details of how spark leverage netty, I found one question, typically RPC message wire format would have a header+payload structure, and netty uses a TransportFrameDecoder to deal with how to determine a complete message from remote peer. But after using Wireshark sniffing tool, I found that the message are sent separately with header and then a body, although this works fine, but for underlying TCP there would be ACK segments sent back to acknowledge, there might be a little bit redundancy since we can sent them together and the header are usually very small.
The main reason can be found in MessageWithHeader class, since transferTo method write tow times for header and body.
Could some one help me understand the background story on how to implement in such way? Thanks!
> Why RPC message are transferred with header and body separately in TCP frame
> ----------------------------------------------------------------------------
>
> Key: SPARK-21703
> URL: https://issues.apache.org/jira/browse/SPARK-21703
> Project: Spark
> Issue Type: Question
> Components: Spark Core
> Affects Versions: 1.6.0
> Reporter: neoremind
> Priority: Trivial
> Labels: RPC
>
> After seeing the details of how spark leverage netty, I found one question, typically RPC message wire format would have a header+payload structure, and netty uses a TransportFrameDecoder to deal with how to determine a complete message from remote peer. But after using Wireshark sniffing tool, I found that the message are sent separately with header and then followed by body, although this works fine, but for underlying TCP there would be ACK segments sent back to acknowledge, there might be a little bit redundancy since we can sent them together and the header are usually very small.
> The main reason can be found in MessageWithHeader class, since transferTo method write tow times for header and body.
> Could some one help me understand the background story on how to implement in such way? Thanks!
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org