You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by 程骥 <ji...@qq.com> on 2017/07/28 07:50:22 UTC
How can I submit a flink job to YARN/Cluster from java code?
Hello，
I want to submit a flink job to YARN/Cluster from java code.If this is feasible?
Is there anyone tried to do it before ？
Thanks
Re: re: How can I submit a flink job to YARN/Cluster from java code?
Posted by Till Rohrmann <tr...@apache.org>.
The RemoteEnvironment won’t allow you to spawn a Flink cluster running on
Yarn. But you can start one with yarn-session.sh. The same applies to the
REST endpoint. You first have to start a cluster.
Cheers,
Till
On Fri, Jul 28, 2017 at 12:14 PM, 程骥 <ji...@qq.com> wrote:
> Hi,
> Thanks for yor help.
> This is a good idea.
> Maybe I didn't understand the information of restful API.
> I think the restful API can not submit a job to yarn.
> Is that right?
>
>
>
> ------------------ 原始邮件 ------------------
> *发件人:* "zrc@zjdex.com";<zr...@zjdex.com>;
> *发送时间:* 2017年7月28日(星期五) 下午5:15
> *收件人:* "程骥"<ji...@qq.com>; "user"<us...@flink.apache.org>;
> *主题:* re: How can I submit a flink job to YARN/Cluster from java code?
>
> Hi:
> I think you can use Flink restful API, like "POST:
> /jars/MyProgram.jar/run?savepointPath=/my-savepoints/
> savepoint-1bae02a80464&allowNonRestoredState=true"
> The detail information of restful API , you can see the link
> https://ci.apache.org/projects/flink/flink-docs-
> release-1.3/monitoring/rest_api.html .
>
> ------------------------------
> zrc@zjdex.com
>
>
> *发件人：* 程骥 <ji...@qq.com>
> *发送时间：* 2017-07-28 15:50
> *收件人：* user <us...@flink.apache.org>
> *主题：* How can I submit a flink job to YARN/Cluster from java code?
> Hello，
> I want to submit a flink job to YARN/Cluster from java code.If this is
> feasible?
> Is there anyone tried to do it before ？
>
>
> Thanks
>
>
回复：re: How can I submit a flink job to YARN/Cluster from java code?
Posted by 程骥 <ji...@qq.com>.
ok,thank you.
------------------ 原始邮件 ------------------
发件人: "zrc@zjdex.com";<zr...@zjdex.com>;
发送时间: 2017年7月31日(星期一) 上午9:31
收件人: "程骥"<ji...@qq.com>; "user"<us...@flink.apache.org>;
主题: re: How can I submit a flink job to YARN/Cluster from java code?
You can see the detail information at https://ci.apache.org/projects/flink/flink-docs-release-1.3/monitoring/rest_api.html.
In this link, it is not said the restful API can not used in yarn cluster. I think it is ok, and you can try. :)
zrc@zjdex.com
发件人： 程骥
发送时间： 2017-07-28 18:14
收件人： zrc@zjdex.com; user
主题： 回复：re: How can I submit a flink job to YARN/Cluster from java code?
Hi,
Thanks for yor help.
This is a good idea.
Maybe I didn't understand the information of restful API.
I think the restful API can not submit a job to yarn.
Is that right?
------------------ 原始邮件 ------------------
发件人: "zrc@zjdex.com";<zr...@zjdex.com>;
发送时间: 2017年7月28日(星期五) 下午5:15
收件人: "程骥"<ji...@qq.com>; "user"<us...@flink.apache.org>;
主题: re: How can I submit a flink job to YARN/Cluster from java code?
Hi:
I think you can use Flink restful API, like "POST: /jars/MyProgram.jar/run?savepointPath=/my-savepoints/savepoint-1bae02a80464&allowNonRestoredState=true"
The detail information of restful API , you can see the link https://ci.apache.org/projects/flink/flink-docs-release-1.3/monitoring/rest_api.html .
zrc@zjdex.com
发件人： 程骥
发送时间： 2017-07-28 15:50
收件人： user
主题： How can I submit a flink job to YARN/Cluster from java code?
Hello，
I want to submit a flink job to YARN/Cluster from java code.If this is feasible?
Is there anyone tried to do it before ？
Thanks
re: How can I submit a flink job to YARN/Cluster from java code?
Posted by "zrc@zjdex.com" <zr...@zjdex.com>.
You can see the detail information at https://ci.apache.org/projects/flink/flink-docs-release-1.3/monitoring/rest_api.html.
In this link, it is not said the restful API can not used in yarn cluster. I think it is ok, and you can try. :)
zrc@zjdex.com
发件人： 程骥
发送时间： 2017-07-28 18:14
收件人： zrc@zjdex.com; user
主题： 回复：re: How can I submit a flink job to YARN/Cluster from java code?
Hi,
Thanks for yor help.
This is a good idea.
Maybe I didn't understand the information of restful API.
I think the restful API can not submit a job to yarn.
Is that right?
------------------ 原始邮件 ------------------
发件人: "zrc@zjdex.com";<zr...@zjdex.com>;
发送时间: 2017年7月28日(星期五) 下午5:15
收件人: "程骥"<ji...@qq.com>; "user"<us...@flink.apache.org>;
主题: re: How can I submit a flink job to YARN/Cluster from java code?
Hi:
I think you can use Flink restful API, like "POST: /jars/MyProgram.jar/run?savepointPath=/my-savepoints/savepoint-1bae02a80464&allowNonRestoredState=true"
The detail information of restful API , you can see the link https://ci.apache.org/projects/flink/flink-docs-release-1.3/monitoring/rest_api.html .
zrc@zjdex.com
发件人： 程骥
发送时间： 2017-07-28 15:50
收件人： user
主题： How can I submit a flink job to YARN/Cluster from java code?
Hello，
I want to submit a flink job to YARN/Cluster from java code.If this is feasible?
Is there anyone tried to do it before ？
Thanks
回复：re: How can I submit a flink job to YARN/Cluster from java code?
Posted by 程骥 <ji...@qq.com>.
Hi,
Thanks for yor help.
This is a good idea.
Maybe I didn't understand the information of restful API.
I think the restful API can not submit a job to yarn.
Is that right?
------------------ 原始邮件 ------------------
发件人: "zrc@zjdex.com";<zr...@zjdex.com>;
发送时间: 2017年7月28日(星期五) 下午5:15
收件人: "程骥"<ji...@qq.com>; "user"<us...@flink.apache.org>;
主题: re: How can I submit a flink job to YARN/Cluster from java code?
Hi:
I think you can use Flink restful API, like "POST: /jars/MyProgram.jar/run?savepointPath=/my-savepoints/savepoint-1bae02a80464&allowNonRestoredState=true"
The detail information of restful API , you can see the link https://ci.apache.org/projects/flink/flink-docs-release-1.3/monitoring/rest_api.html .
zrc@zjdex.com
发件人： 程骥
发送时间： 2017-07-28 15:50
收件人： user
主题： How can I submit a flink job to YARN/Cluster from java code?
Hello，
I want to submit a flink job to YARN/Cluster from java code.If this is feasible?
Is there anyone tried to do it before ？
Thanks
re: How can I submit a flink job to YARN/Cluster from java code?
Posted by "zrc@zjdex.com" <zr...@zjdex.com>.
Hi:
I think you can use Flink restful API, like "POST: /jars/MyProgram.jar/run?savepointPath=/my-savepoints/savepoint-1bae02a80464&allowNonRestoredState=true"
The detail information of restful API , you can see the link https://ci.apache.org/projects/flink/flink-docs-release-1.3/monitoring/rest_api.html .
zrc@zjdex.com
发件人： 程骥
发送时间： 2017-07-28 15:50
收件人： user
主题： How can I submit a flink job to YARN/Cluster from java code?
Hello，
I want to submit a flink job to YARN/Cluster from java code.If this is feasible?
Is there anyone tried to do it before ？
Thanks
回复： How can I submit a flink job to YARN/Cluster from java code?
Posted by 程骥 <ji...@qq.com>.
Hi,Till
Actually,to submit a flink job to flink clusetr with ExecutionEnvironment.createRemoteEnvironment,I have implemented.
but how to submit flink job to flink on yarn,I have no idea.
I think this API not supported.
Thanks for your help.
:)
------------------ 原始邮件 ------------------
发件人: "Till Rohrmann";<tr...@apache.org>;
发送时间: 2017年7月28日(星期五) 下午5:13
收件人: "程骥"<ji...@qq.com>;
抄送: "user"<us...@flink.apache.org>;
主题: Re: How can I submit a flink job to YARN/Cluster from java code?
Hi,
it is possible to submit a job to a running cluster (Standalone/Mesos/Yarn) by using the RemoteEnvironment which you can create via ExecutionEnvironment.createRemoteEnvironment and then executing your program (e.g. running it in the IDE). This assumes that the remote cluster is already running.
The RemoteEnvironment requires that you give it the hostname and port of the running cluster or, if using a HA cluster, a properly configured configuration from which the client can retrieve the HA settings.
Cheers,
Till
On Fri, Jul 28, 2017 at 9:50 AM, 程骥 <ji...@qq.com> wrote:
Hello，
I want to submit a flink job to YARN/Cluster from java code.If this is feasible?
Is there anyone tried to do it before ？
Thanks
Re: How can I submit a flink job to YARN/Cluster from java code?
Posted by Till Rohrmann <tr...@apache.org>.
Hi,
it is possible to submit a job to a running cluster (Standalone/Mesos/Yarn)
by using the RemoteEnvironment which you can create via
ExecutionEnvironment.createRemoteEnvironment and then executing your
program (e.g. running it in the IDE). This assumes that the remote cluster
is already running.
The RemoteEnvironment requires that you give it the hostname and port of
the running cluster or, if using a HA cluster, a properly configured
configuration from which the client can retrieve the HA settings.
Cheers,
Till
On Fri, Jul 28, 2017 at 9:50 AM, 程骥 <ji...@qq.com> wrote:
> Hello，
> I want to submit a flink job to YARN/Cluster from java code.I
> f this is feasible?
> Is there anyone tried to do it before ？
>
>
> Thanks
>