You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@tez.apache.org by "Bikas Saha (JIRA)" <ji...@apache.org> on 2015/03/16 00:30:38 UTC

[jira] [Created] (TEZ-2201) Improvements to input split grouping

Bikas Saha created TEZ-2201:
-------------------------------

             Summary: Improvements to input split grouping
                 Key: TEZ-2201
                 URL: https://issues.apache.org/jira/browse/TEZ-2201
             Project: Apache Tez
          Issue Type: Task
            Reporter: Bikas Saha
            Assignee: Bikas Saha


Today Tez has helper grouping libraries that group hadoop input splits into larger groups for efficiency. This grouping logic is based on a min/max size of groups and a wave based heuristic that tries to run mappers in waves to hide the effect of stragglers.
This logic can be brittle based on input size and cluster size and can be improved to be more dynamic and responsive.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)