[jira] [Commented] (TEZ-3805) Analyzer: Add an analyzer to find out scheduling misses in 1:1 edges

["Gopal V (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16273759#comment-16273759 ] 

Gopal V commented on TEZ-3805:
------------------------------

[~kshukla]: I can't get 1:1 to work fast with the local.fetch kicking in every time (this is the big IO savings, the shuffle causes a write to disk, which slows down the shuffle outbound as well).

When the producer has >1 wave (or if there's another vertex at the same level, reducing the width of the cluster).



> Analyzer: Add an analyzer to find out scheduling misses in 1:1 edges
> --------------------------------------------------------------------
>
>                 Key: TEZ-3805
>                 URL: https://issues.apache.org/jira/browse/TEZ-3805
>             Project: Apache Tez
>          Issue Type: Improvement
>            Reporter: Rajesh Balamohan
>            Assignee: Rajesh Balamohan
>             Fix For: 0.9.1
>
>         Attachments: TEZ-3805.1.patch
>
>
> When 1:1 edge is used, it would be helpful to find out whether downstream tasks ran on the same location provided in the hints by the runtime. 
> One of the recent feature in upstream project (hive) used 1:1 edge. Instead of checking the logs, it would be useful to have an analyzer to churn out the details.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)