[jira] [Commented] (TEZ-3772) Allow slowstart for small vertices to be treated differently

["Siddharth Seth (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16081131#comment-16081131 ] 

Siddharth Seth commented on TEZ-3772:
-------------------------------------

The case of 1 downstream reducer should have already been addressed via a ceil method. I'd like to avoid introducing another set of config parameters, especially ones which are very similar to the existing ones. Gets difficult to reason about which ones need to be configured, and which one gets used.

> Allow slowstart for small vertices to be treated differently
> ------------------------------------------------------------
>
>                 Key: TEZ-3772
>                 URL: https://issues.apache.org/jira/browse/TEZ-3772
>             Project: Apache Tez
>          Issue Type: Improvement
>            Reporter: Muhammad Samir Khan
>            Assignee: Muhammad Samir Khan
>         Attachments: tez-3772.001.patch, tez-3772.002.patch
>
>
> If there are a small number of reduces (configurable), then having a different threshold can benefit. Performance of jobs with a small number of reduce tasks can benefit significantly. Yes, the job could specify slowstart as 0.0 instead of the default, but that requires job owners to do something. It would be better if the defaults did something more optimal for both large and small jobs.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)