RE: One to one edges and local fetch

[Bikas Saha <bi...@apache.org>]

Are you seeing it to be blatantly not the case or it happens on some occasions?

The 1-1 destination tasks have an affinity to the container of their source tasks which will get relaxed to node, rack and global based on delay scheduling. So if the container/node that it wants is already taken then it will get scheduled somewhere else based on delay scheduling.

As far as local fetch is concerned. I am not sure if the unordered input is optimized to do local fetch even if its scheduled on the same node.

Based on the above information, if you think there is a bug then please file a jira with the logs.

Thanks!
Bikas

-----Original Message-----
From: Rohini Palaniswamy [mailto:rohini.aditya@gmail.com] 
Sent: Friday, April 29, 2016 2:57 PM
To: dev@tez.apache.org
Subject: One to one edges and local fetch

I was under the assumption that we optimized 1-1 edge scheduling to reuse containers or run as much as possible on same node and do local fetch. But does not seem to be the case. Is there a jira for this already? Could not find any.

Regards,
Rohini