You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@flink.apache.org by 杨光 <la...@gmail.com> on 2018/08/06 04:25:20 UTC

HA setting for per-job YARN session

Hi,
i am using  the " per-job YARN session " mode deploy flink job on yarn.
The document https://ci.apache.org/projects/flink/flink-docs-release-1.5/ops/jobmanager_high_availability.html#yarn-cluster-high-availability
 says that "we don’t run multiple JobManager (ApplicationMaster)
instances" but still give an example about how to config an Highly
Available YARN Session cluster.

My question is:  does the " per-job YARN session "  mode should  also
use HA setting ? Because in my test case without using zookeeper HA ,
if the YarnApplicationMasterRunner/Jobmanager is failed or killed
,yarn can only restart the YarnApplicationMasterRunner/Jobmanager
program but no job graph is submitted to the Jobmanager again.

THANKS

Re: HA setting for per-job YARN session

Posted by sanmutongzi <la...@gmail.com>.

Thanks Gary, I have tested  those ha settings and it works well.




--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: HA setting for per-job YARN session

Posted by Gary Yao <ga...@data-artisans.com>.

Hi,

The per-job YARN mode also supports HA. Even if the YARN ResourceManager
only
brings up one JobManager at a time, you require ZooKeeper for storing
metadata, such as which jobs are supposed to be running. In fact, the
distributed components will still take part in a leader election with only
one
contender. The keys that are required to be configured for HA are:

    high-availability: zookeeper
    high-availability.zookeeper.quorum: <your zk quorum>
    high-availability.zookeeper.storageDir: <path on DFS>

Best,
Gary

On Mon, Aug 6, 2018 at 6:25 AM, 杨光 <la...@gmail.com> wrote:

> Hi,
> i am using  the " per-job YARN session " mode deploy flink job on yarn.
> The document https://ci.apache.org/projects/flink/flink-docs-
> release-1.5/ops/jobmanager_high_availability.html#yarn-
> cluster-high-availability
>  says that "we don’t run multiple JobManager (ApplicationMaster)
> instances" but still give an example about how to config an Highly
> Available YARN Session cluster.
>
> My question is:  does the " per-job YARN session "  mode should  also
> use HA setting ? Because in my test case without using zookeeper HA ,
> if the YarnApplicationMasterRunner/Jobmanager is failed or killed
> ,yarn can only restart the YarnApplicationMasterRunner/Jobmanager
> program but no job graph is submitted to the Jobmanager again.
>
> THANKS
>