[jira] [Created] (FLINK-3927) TaskManager registration may fail if Yarn versions don't match

["Maximilian Michels (JIRA)" <ji...@apache.org>]

Maximilian Michels created FLINK-3927:
-----------------------------------------

             Summary: TaskManager registration may fail if Yarn versions don't match
                 Key: FLINK-3927
                 URL: https://issues.apache.org/jira/browse/FLINK-3927
             Project: Flink
          Issue Type: Bug
          Components: ResourceManager
    Affects Versions: 1.1.0
            Reporter: Maximilian Michels
            Assignee: Maximilian Michels
             Fix For: 1.1.0


Flink's ResourceManager uses the Yarn container ids to identify connecting task managers. Yarn's stringified container id may not be consistent across different Hadoop versions, e.g. Hadoop 2.3.0 and Hadoop 2.7.1. The ResourceManager gets it from the Yarn reports while the TaskManager infers it from the Yarn environment variables. The ResourceManager may use Hadoop 2.3.0 version while the cluster runs Hadoop 2.7.1. 

The solution is to pass the ID through a custom environment variable which is set by the ResourceManager before launching the TaskManager in the container. That way we will always use the Hadoop client's id generation method.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)