You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@tez.apache.org by "Rajesh Balamohan (JIRA)" <ji...@apache.org> on 2014/04/29 11:25:14 UTC

[jira] [Created] (TEZ-1091) Query_17: Fetcher makes lots of calls to ShuffleHandler (when MapOutput.Type == WAIT)

Rajesh Balamohan created TEZ-1091:
-------------------------------------

             Summary: Query_17:  Fetcher makes lots of calls to ShuffleHandler (when MapOutput.Type == WAIT)
                 Key: TEZ-1091
                 URL: https://issues.apache.org/jira/browse/TEZ-1091
             Project: Apache Tez
          Issue Type: Bug
            Reporter: Rajesh Balamohan
            Assignee: Rajesh Balamohan
             Fix For: 0.5.0


Query:  Query_17 (shuffle intensive query)

1. Certain fetchers are making too many calls in tight loop fashion to shufflehandler.  This happens only when MapOutput.Type == WAIT.  This would created pressure on ShuffleHandler on large scale cluster.

Please refer to "fetcher #24" as an example in the attachment.

2. If keepalive is enabled, CleanupConnection() should disconnect the connection only in exception path.




--
This message was sent by Atlassian JIRA
(v6.2#6252)