You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2017/03/23 03:26:41 UTC
[jira] [Commented] (FLINK-6174) Introduce a leader election service
in yarn mode to make JobManager always available
[ https://issues.apache.org/jira/browse/FLINK-6174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15937640#comment-15937640 ]
ASF GitHub Bot commented on FLINK-6174:
---------------------------------------
GitHub user WangTaoTheTonic opened a pull request:
https://github.com/apache/flink/pull/3599
[FLINK-6174][HA]introduce a new election service to make JobManager always available
Now in yarn mode, if we use zookeeper as high availability choice, it will create a election service to get a leader depending on zookeeper election.
When zookeeper leader crashes or the connection between JobManager and zookeeper instance was broken, JobManager's leadership will be revoked and send a Disconnect message to TaskManager, which will cancle all running tasks and make them waiting connection rebuild between JM and ZK.
In yarn mode, we have one and only JobManager(AM) in same time, and it should be alwasy leader instead of elected through zookeeper. We can introduce a new leader election service in yarn mode to achive that.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/WangTaoTheTonic/flink FLINK-6174
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/3599.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3599
----
commit a758ec6320b026fb5d767cdc190c29c8043838da
Author: WangTaoTheTonic <wa...@huawei.com>
Date: 2017-03-23T03:15:40Z
introduce a new election service to make JobManager always available
----
> Introduce a leader election service in yarn mode to make JobManager always available
> ------------------------------------------------------------------------------------
>
> Key: FLINK-6174
> URL: https://issues.apache.org/jira/browse/FLINK-6174
> Project: Flink
> Issue Type: Improvement
> Components: JobManager
> Reporter: Tao Wang
> Assignee: Tao Wang
>
> Now in yarn mode, if we use zookeeper as high availability choice, it will create a election service to get a leader depending on zookeeper election.
> When zookeeper leader crashes or the connection between JobManager and zookeeper instance was broken, JobManager's leadership will be revoked and send a Disconnect message to TaskManager, which will cancle all running tasks and make them waiting connection rebuild between JM and ZK.
> In yarn mode, we have one and only JobManager(AM) in same time, and it should be alwasy leader instead of elected through zookeeper. We can introduce a new leader election service in yarn mode to achive that.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)