[jira] Commented: (HADOOP-2909) Improve IPC idle connection management

["Ankur (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12601047#action_12601047 ] 

Ankur commented on HADOOP-2909:
-------------------------------

> 1. Add data structures to the IPC server ...
I think one very simple way of doing this would be to add a 'pendingRequests'  field to the Connection class that keeps the count of pending requests. However if one needs to see the details of pending requests for a connection then making 'pendingRequests' as a list that contains all the unprocessed calls might be more suitable but slightly redundant as the unprocessed calls are already queued in the server 'callQueue'.

> 2. IPC server does not close a connection ...
> 3. The default client-side ...
> 4. The server-side max ...
This can be taken care of in Connection.timedOut() by checking the connection's responseQueue/pendingRequests count. However we need to be able to differentiate between server slowdown, client slowdown and client crashes. Setting a higher idle time for the client  and even higher idle time for the server seems to be the right choice.

> Improve IPC idle connection management
> --------------------------------------
>
>                 Key: HADOOP-2909
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2909
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: ipc
>    Affects Versions: 0.16.0
>            Reporter: Hairong Kuang
>
> IPC server determines if a connection is idle or not by checking if the connection does not have any IO activity for a predefined max idle time. An idle connection will be closed even if the connection still has outstanding requests or replies. This causes RPC failures when a server becomes slow or if a request takes a long time to be served. In jira, I'd like to propose the following changes to IPC idle management:
> 1. Add data structures to the IPC server that keep track of outstanding requests.
> 2. IPC server does not close a connection that has outstanding requests/replies even when it has no IO activities for a while.
> 3. The default client-side max idle time should be in several minutes not 1 second. 
> 4. The server-side max idle time should be greater than the client-side max idle time, for example, twice of the client-side max idle time. So server mainly deals with clients that are crashed without closing 
> its connections. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.