You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "xinqiu.hu (Jira)" <ji...@apache.org> on 2022/11/22 05:10:00 UTC
[jira] [Created] (HADOOP-18536) RPC Client Improvement
xinqiu.hu created HADOOP-18536:
----------------------------------
Summary: RPC Client Improvement
Key: HADOOP-18536
URL: https://issues.apache.org/jira/browse/HADOOP-18536
Project: Hadoop Common
Issue Type: Improvement
Components: rpc-server
Reporter: xinqiu.hu
In the RPC Client, before a request (including RpcRequestHeaderProto,
RequestHeaderProto, Message Payload) is sent, they will be copied to the three
CodedOutputStream internal byte arrays, and then aggregated to the ResponseBuffer
internal byte array. Then the ResponseBuffer byte array is written to a
BufferedOutputStream and finally to a SocketOutputStream.
To simplify the writing process, Maybe we can copy them directly to a big
CodedOutputStream and send them directly to the IpcStreams#out. To achieve this, I
propose HADOOP-18533. But it brings the following two side effects.
# The generic declaration of rpcRequestQueue inside Client has been changed to Object
# The serialization of protobuf has been moved to rpcRequestThread, because rpcRequestThread is a single thread for each connection, which may have a performance impact.
For the above reasons, I propose this. This pr brings the following benefits
# For each rpc request, avoid creating a ResponseBuffer of 1024 bytes
# For each rpc request, combine the three fragmented CodedOutputStreams into one
# No side effects like HADOOP-18533
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org