[jira] [Commented] (TEZ-1700) Replace containerId from TaskLocationHint with [TaskIndex+Vertex] based affinity

["Hitesh Shah (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-1700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14189155#comment-14189155 ] 

Hitesh Shah commented on TEZ-1700:
----------------------------------

Comments: 
  
Not sure if this is really safe from a binary compatibility point of view. Might be worth testing a job that uses TaskLocationHints compiled against 0.5.0 and run using a 0.5.2-SNAPSHOT runtime. 

{code}
+    if (affinitizedTask != null) {
+      if (affinitizedTask.getTaskIndex() != other.affinitizedTask.getTaskIndex()) {
+        return false;
+      } else if (!affinitizedTask.getVertexName().equals(other.affinitizedTask.getVertexName())) {
         return false;
       }
-    } else if (other.containerId != null) {
+    } else if (other.affinitizedTask != null) {
       return false;
     }
{code}
   - I believe the "other.affinitizedTask != null" should be done earlier before doing the != comparisons for vertex name and task index

{code}
+          taskScheduler.allocateTask(taskAttempt,
+              event.getCapability(),
+              taskAttempt.getAssignedContainerID(),
+              Priority.newInstance(event.getPriority()),
+              event.getContainerContext(),
+              event);
{code}
  -  should this be using the affinityAttempt's container id? If the unit tests are not catching this, maybe add one more test? 






> Replace containerId from TaskLocationHint with [TaskIndex+Vertex] based affinity
> --------------------------------------------------------------------------------
>
>                 Key: TEZ-1700
>                 URL: https://issues.apache.org/jira/browse/TEZ-1700
>             Project: Apache Tez
>          Issue Type: Bug
>            Reporter: Bikas Saha
>            Assignee: Bikas Saha
>         Attachments: TEZ-1700.1.patch
>
>
> Today 1-1 dependencies are affinitized by creating a task location hint with the producer task container id. It can be created by affinitizing to the producer task-index+vertexname combination instead and internally Tez can map it to the container. This also allows this dependency to be specified before the container is assigned. This allows the dependency to be generic.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)