[jira] [Commented] (FLINK-6174) Introduce a leader election service in yarn mode to make JobManager always available

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-6174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001318#comment-16001318 ] 

ASF GitHub Bot commented on FLINK-6174:
---------------------------------------

Github user StephanEwen commented on the issue:

    https://github.com/apache/flink/pull/3599
  
    Thanks for adding this!
    I have a few questions:
    
      - From the formatting, the code looks like might be adapted from some other project. If yes, can you share from where and add a comment above the code that it is adopted from another project. Also, formatting in the common Flink style would be good.
    
      - The HA leader election is very critical, so we usually don't just replace code that has worked well (for many users with new code that we have not extensively tested. Therefore, I suggest to add a config option to activate this smarter latch, thus making it optional for users that want to use it initially.
    
      - Once this proves stable, we can make it the default in the next version.


> Introduce a leader election service in yarn mode to make JobManager always available
> ------------------------------------------------------------------------------------
>
>                 Key: FLINK-6174
>                 URL: https://issues.apache.org/jira/browse/FLINK-6174
>             Project: Flink
>          Issue Type: Sub-task
>          Components: JobManager
>            Reporter: Tao Wang
>            Assignee: Tao Wang
>
> Now in yarn mode, if we use zookeeper as high availability choice, it will create a election service to get a leader depending on zookeeper election.
> When zookeeper leader crashes or the connection between JobManager and zookeeper instance was broken, JobManager's leadership will be revoked and send a Disconnect message to TaskManager, which will cancle all running tasks and make them waiting connection rebuild between JM and ZK.
> In yarn mode, we have one and only JobManager(AM) in same time, and it should be alwasy leader instead of elected through zookeeper. We can introduce a new leader election service in yarn mode to achive that.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)