You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@tez.apache.org by "Bikas Saha (JIRA)" <ji...@apache.org> on 2015/05/22 09:06:17 UTC

[jira] [Updated] (TEZ-2478) Move OneToOne routing to store events in Tasks

     [ https://issues.apache.org/jira/browse/TEZ-2478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Bikas Saha updated TEZ-2478:
----------------------------
    Attachment: 1-1-wip.patch

Not sure why this is so urgently needed as to target 0.7.1.

Attached a wip patch that I was using on top of TEZ-2409 to compare legacy and on-demand code on a 20 node cluster. With a 10Kx10K 1-1 edge empty job. There is no discernible difference in the runtimes ~30s and CPU time ~107-110s in both cases. Given that this is already a fairly large job IMO we can wait to see if there is any real issue in practice.

The pro of keeping on-demand routing is that it moves edge routing off the central dispatcher.

> Move OneToOne routing to store events in Tasks
> ----------------------------------------------
>
>                 Key: TEZ-2478
>                 URL: https://issues.apache.org/jira/browse/TEZ-2478
>             Project: Apache Tez
>          Issue Type: Improvement
>            Reporter: Siddharth Seth
>            Assignee: Siddharth Seth
>         Attachments: 1-1-wip.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)