You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Íñigo Goiri (JIRA)" <ji...@apache.org> on 2019/04/09 17:31:00 UTC
[jira] [Commented] (HADOOP-13144) Enhancing IPC client throughput
via multiple connections per user
[ https://issues.apache.org/jira/browse/HADOOP-13144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16813650#comment-16813650 ]
Íñigo Goiri commented on HADOOP-13144:
--------------------------------------
We would like to have this by 3.3 as it significantly improves the performance of the Router.
The main change is that now connections don't need to share the connection factory and we can have more parallel connections.
> Enhancing IPC client throughput via multiple connections per user
> -----------------------------------------------------------------
>
> Key: HADOOP-13144
> URL: https://issues.apache.org/jira/browse/HADOOP-13144
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc
> Reporter: Jason Kace
> Assignee: Íñigo Goiri
> Priority: Minor
> Attachments: HADOOP-13144.000.patch, HADOOP-13144.001.patch, HADOOP-13144.002.patch
>
>
> The generic IPC client ({{org.apache.hadoop.ipc.Client}}) utilizes a single connection thread for each {{ConnectionId}}. The {{ConnectionId}} is unique to the connection's remote address, ticket and protocol. Each ConnectionId is 1:1 mapped to a connection thread by the client via a map cache.
> The result is to serialize all IPC read/write activity through a single thread for a each user/ticket + address. If a single user makes repeated calls (1k-100k/sec) to the same destination, the IPC client becomes a bottleneck.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org