[jira] [Created] (SAMZA-1334) When host-affinity is turned off, ContainerAllocator should ignore any previous container locality info

["Yi Pan (Data Infrastructure) (JIRA)" <ji...@apache.org>]

Yi Pan (Data Infrastructure) created SAMZA-1334:
---------------------------------------------------

             Summary: When host-affinity is turned off, ContainerAllocator should ignore any previous container locality info
                 Key: SAMZA-1334
                 URL: https://issues.apache.org/jira/browse/SAMZA-1334
             Project: Samza
          Issue Type: Bug
            Reporter: Yi Pan (Data Infrastructure)


Consider a case where host affinity is turned on once for a job, and the locality info is written to the coordinator stream. Then the user may turn off the host affinity feature. 

That triggers a bug in ContainerAllocator:
1) it gets the locality map from JobModel which has the list of preferred hosts from the coordinator stream. Hence, ContainerAllocator is making preferred host resource requests.
2) At the end, ContainerAllocator finishes launching all containers and tries to release all extra containers mapping to ANY_HOST. However, all preferred host resource responses are kept under the specific host's entry. Hence, it failed to release those containers.

The end result is: the job is still successfully launched. However, YARN RM reports a lot of reserved memory/containers not released by the job. In some extreme cases, the reserved memory/container can be huge and affects the availability of the whole cluster.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)