[jira] [Commented] (TEZ-1091) Query_17: Fetcher makes lots of calls to ShuffleHandler (when MapOutput.Type == WAIT)

["Bikas Saha (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-1091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13984465#comment-13984465 ] 

Bikas Saha commented on TEZ-1091:
---------------------------------

Can you please elaborate? Its not clear what is query_17?

> Query_17:  Fetcher makes lots of calls to ShuffleHandler (when MapOutput.Type == WAIT)
> --------------------------------------------------------------------------------------
>
>                 Key: TEZ-1091
>                 URL: https://issues.apache.org/jira/browse/TEZ-1091
>             Project: Apache Tez
>          Issue Type: Bug
>            Reporter: Rajesh Balamohan
>            Assignee: Rajesh Balamohan
>             Fix For: 0.5.0
>
>         Attachments: tasklog.txt.gz
>
>
> Query:  Query_17 (shuffle intensive query)
> 1. Certain fetchers are making too many calls in tight loop fashion to shufflehandler.  This happens only when MapOutput.Type == WAIT.  This would created pressure on ShuffleHandler on large scale cluster.
> Please refer to "fetcher #24" as an example in the attachment.
> 2. If keepalive is enabled, CleanupConnection() should disconnect the connection only in exception path.



--
This message was sent by Atlassian JIRA
(v6.2#6252)