You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Jason Lowe (JIRA)" <ji...@apache.org> on 2017/03/01 15:13:45 UTC
[jira] [Commented] (MAPREDUCE-6850) Shuffle Handler keep-alive
connections are closed from the server side
[ https://issues.apache.org/jira/browse/MAPREDUCE-6850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890346#comment-15890346 ]
Jason Lowe commented on MAPREDUCE-6850:
---------------------------------------
Thanks for the patch!
I'm wondering about handling the idle timeout. I'm worried about the case where disks are busy or slow. If it takes us longer than the idle timeout to respond to a request then I think the idle handler will close the connection mid-request which doesn't seem appropriate. Unless I'm missing something, the idle handler needs to check if we're in the middle of servicing a request or if we're waiting for the next request and only close for the latter.
> Shuffle Handler keep-alive connections are closed from the server side
> ----------------------------------------------------------------------
>
> Key: MAPREDUCE-6850
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6850
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Jonathan Eagles
> Assignee: Jonathan Eagles
> Attachments: MAPREDUCE-6850.1.patch, MAPREDUCE-6850.2.patch, MAPREDUCE-6850.3.patch, MAPREDUCE-6850.4.patch, With_Issue.png, With_Patch.png, With_Patch_withData.png
>
>
> When performance testing tez shuffle handler (TEZ-3334), it was noticed the keep-alive connections are closed from the server-side. The client silently recovers and logs the connection as keep-alive, despite reestablishing a connection. This jira aims to remove the close from the server side, fixing the bug preventing keep-alive connections.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org