You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by gaurav kulkarni <ku...@yahoo.com> on 2021/04/23 01:35:09 UTC
Official flink java client
Hi,
Is there any official flink client in java that's available? I came across RestClusterClient, but I am not sure if its official. I can create my own client, but just wanted to check if there is anything official available already that I can leverage.
Thanks,Gaurav
|
|
|
| | |
|
|
|
| |
Run already deployed job on Flink Cluster using RestClusterClient
I am trying to run already deployed job on Flink Cluster using Rest request.I had success using a simple rest ...
|
|
|
Re: Official flink java client
Posted by gaurav kulkarni <ku...@yahoo.com>.
Thanks for the response, folks! I plan to use the client mostly for monitoring status of jobs, probably to trigger savepoints too. I may extend it in future to submit jobs. Given RestClusterClient is not officially supported, I will probably build something myself. Agree with Flavio, it would be great if there is an official client available or if the client can be generated automatically.
Thanks,Gaurav
On Friday, April 23, 2021, 06:18:35 AM PDT, Flavio Pompermaier <po...@okkam.it> wrote:
Yes, that's a known risk. Indeed it would be awesome if the REST API would be published also using some format that allow automatic client generation (like swagger or openapi). Also release an official client could be an option otherwise...I think that it's very annoying to write a client from scratch.I'll continue to use RestClusterClient until it works..
On Fri, Apr 23, 2021 at 2:48 PM Yun Gao <yu...@aliyun.com> wrote:
Hi Flavio,
Got that, from my view I think RestClusterClient might not be viewed as public API, and might be change between version, thus it might need to be careful when upgrading.
Best,Yun
------------------Original Mail ------------------Sender:Flavio Pompermaier <po...@okkam.it>Send Date:Fri Apr 23 16:10:05 2021Recipients:Yun Gao <yu...@aliyun.com>CC:gaurav kulkarni <ku...@yahoo.com>, User <us...@flink.apache.org>Subject:Re: Re: Official flink java client
Obviously I could rewrite a java client from scratch that interface with the provided REST API but why if I can reuse something already existing?Usually I interface with REST API using auto generated clients (if APIs are exposed via Swagger or OpenApi).If that's not an option, writing a REST client from scratch is something I try to avoid as much as I can..
Best,Flavio
On Fri, Apr 23, 2021 at 9:55 AM Yun Gao <yu...@aliyun.com> wrote:
Hi Falvio,
Very thanks for the explanation, may be another option is to have a look at the http rest API[1] ? Flink provides official http api to submit jar jobs and query job status, and they might be able to help.
Best,Yun
[1] https://ci.apache.org/projects/flink/flink-docs-stable/ops/rest_api.html
------------------Original Mail ------------------Sender:Flavio Pompermaier <po...@okkam.it>Send Date:Fri Apr 23 15:25:55 2021Recipients:Yun Gao <yu...@aliyun.com>CC:gaurav kulkarni <ku...@yahoo.com>, User <us...@flink.apache.org>Subject:Re: Official flink java client
I also interface to Flink clusters using REST in order to avoid many annoying problems (due to dependency conflicts, classpath or env variables).I use an extended version of the RestClusterClient that you can reuse if you want to.It is available at [1] and it add some missing methods to the default Flink version (I also had to copy that class and modify the visibility of some field in order to enable the extension).Officially the Flink RestClusterClient is meant to be used for internal use only but it actually work very well.
Best,Flavio
[1] https://github.com/fpompermaier/flink-job-server/blob/main/flink-rest-client/src/main/java/org/apache/flink/client/program/rest/RestClusterClientExtended.java
On Fri, Apr 23, 2021 at 5:10 AM Yun Gao <yu...@aliyun.com> wrote:
Hi gaurav，
Logicall Flink client is bear inside the StreamExecutionEnvironment, and users could use the StreamExecutionEnvironment to execute their jobs. Could you share more about why you want to directly use the client?
Best,Yun
------------------Original Mail ------------------Sender:gaurav kulkarni <ku...@yahoo.com>Send Date:Fri Apr 23 10:14:08 2021Recipients:User <us...@flink.apache.org>Subject:Official flink java client
Hi,
Is there any official flink client in java that's available? I came across RestClusterClient, but I am not sure if its official. I can create my own client, but just wanted to check if there is anything official available already that I can leverage.
Thanks,Gaurav
|
|
|
| | |
|
|
|
| |
Run already deployed job on Flink Cluster using RestClusterClient
I am trying to run already deployed job on Flink Cluster using Rest request.I had success using a simple rest ...
|
|
|
Re: Re: Re: Official flink java client
Posted by Flavio Pompermaier <po...@okkam.it>.
Yes, that's a known risk. Indeed it would be awesome if the REST API would
be published also using some format that allow automatic client generation
(like swagger or openapi). Also release an official client could be an
option otherwise...I think that it's very annoying to write a client from
scratch.
I'll continue to use RestClusterClient until it works..
On Fri, Apr 23, 2021 at 2:48 PM Yun Gao <yu...@aliyun.com> wrote:
> Hi Flavio,
>
> Got that, from my view I think RestClusterClient might not be viewed as
> public API,
> and might be change between version, thus it might need to be careful when
> upgrading.
>
> Best,
> Yun
>
>
> ------------------Original Mail ------------------
> *Sender:*Flavio Pompermaier <po...@okkam.it>
> *Send Date:*Fri Apr 23 16:10:05 2021
> *Recipients:*Yun Gao <yu...@aliyun.com>
> *CC:*gaurav kulkarni <ku...@yahoo.com>, User <
> user@flink.apache.org>
> *Subject:*Re: Re: Official flink java client
>
>> Obviously I could rewrite a java client from scratch that interface with
>> the provided REST API but why if I can reuse something already existing?
>> Usually I interface with REST API using auto generated clients (if APIs
>> are exposed via Swagger or OpenApi).
>> If that's not an option, writing a REST client from scratch is something
>> I try to avoid as much as I can..
>>
>> Best,
>> Flavio
>>
>> On Fri, Apr 23, 2021 at 9:55 AM Yun Gao <yu...@aliyun.com> wrote:
>>
>>> Hi Falvio,
>>>
>>> Very thanks for the explanation, may be another option is to have a look
>>> at
>>> the http rest API[1] ? Flink provides official http api to submit jar
>>> jobs and query
>>> job status, and they might be able to help.
>>>
>>> Best,
>>> Yun
>>>
>>> [1]
>>> https://ci.apache.org/projects/flink/flink-docs-stable/ops/rest_api.html
>>>
>>> ------------------Original Mail ------------------
>>> *Sender:*Flavio Pompermaier <po...@okkam.it>
>>> *Send Date:*Fri Apr 23 15:25:55 2021
>>> *Recipients:*Yun Gao <yu...@aliyun.com>
>>> *CC:*gaurav kulkarni <ku...@yahoo.com>, User <
>>> user@flink.apache.org>
>>> *Subject:*Re: Official flink java client
>>>
>>>> I also interface to Flink clusters using REST in order to avoid many
>>>> annoying problems (due to dependency conflicts, classpath or env variables).
>>>> I use an extended version of the RestClusterClient that you can reuse
>>>> if you want to.
>>>> It is available at [1] and it add some missing methods to the default
>>>> Flink version (I also had to copy that class and modify the visibility of
>>>> some field in order to enable the extension).
>>>> Officially the Flink RestClusterClient is meant to be used for internal
>>>> use only but it actually work very well.
>>>>
>>>> Best,
>>>> Flavio
>>>>
>>>> [1]
>>>> https://github.com/fpompermaier/flink-job-server/blob/main/flink-rest-client/src/main/java/org/apache/flink/client/program/rest/RestClusterClientExtended.java
>>>>
>>>> On Fri, Apr 23, 2021 at 5:10 AM Yun Gao <yu...@aliyun.com> wrote:
>>>>
>>>>> Hi gaurav，
>>>>>
>>>>> Logicall Flink client is bear inside the StreamExecutionEnvironment,
>>>>> and users could use the
>>>>> StreamExecutionEnvironment to execute their jobs. Could you share more
>>>>> about why you
>>>>> want to directly use the client?
>>>>>
>>>>> Best,
>>>>> Yun
>>>>>
>>>>>
>>>>> ------------------Original Mail ------------------
>>>>> *Sender:*gaurav kulkarni <ku...@yahoo.com>
>>>>> *Send Date:*Fri Apr 23 10:14:08 2021
>>>>> *Recipients:*User <us...@flink.apache.org>
>>>>> *Subject:*Official flink java client
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Is there any official flink client in java that's available? I came
>>>>>> across RestClusterClient
>>>>>> <https://ci.apache.org/projects/flink/flink-docs-master/api/java/org/apache/flink/client/program/rest/RestClusterClient.html>,
>>>>>> but I am not sure if its official
>>>>>> <https://stackoverflow.com/questions/56079788/run-already-deployed-job-on-flink-cluster-using-restclusterclient>.
>>>>>> I can create my own client, but just wanted to check if there is anything
>>>>>> official available already that I can leverage.
>>>>>>
>>>>>> Thanks,
>>>>>> Gaurav
>>>>>>
>>>>>> Run already deployed job on Flink Cluster using RestClusterClient
>>>>>>
>>>>>> I am trying to run already deployed job on Flink Cluster using Rest
>>>>>> request.I had success using a simple rest ...
>>>>>>
>>>>>> <https://stackoverflow.com/questions/56079788/run-already-deployed-job-on-flink-cluster-using-restclusterclient>
>>>>>>
>>>>>>
>>>>>>
Re: Re: Re: Official flink java client
Posted by Yun Gao <yu...@aliyun.com>.
Hi Flavio,
Got that, from my view I think RestClusterClient might not be viewed as public API,
and might be change between version, thus it might need to be careful when upgrading.
Best,
Yun
------------------Original Mail ------------------
Sender:Flavio Pompermaier <po...@okkam.it>
Send Date:Fri Apr 23 16:10:05 2021
Recipients:Yun Gao <yu...@aliyun.com>
CC:gaurav kulkarni <ku...@yahoo.com>, User <us...@flink.apache.org>
Subject:Re: Re: Official flink java client
Obviously I could rewrite a java client from scratch that interface with the provided REST API but why if I can reuse something already existing?
Usually I interface with REST API using auto generated clients (if APIs are exposed via Swagger or OpenApi).
If that's not an option, writing a REST client from scratch is something I try to avoid as much as I can..
Best,
Flavio
On Fri, Apr 23, 2021 at 9:55 AM Yun Gao <yu...@aliyun.com> wrote:
Hi Falvio,
Very thanks for the explanation, may be another option is to have a look at
the http rest API[1] ? Flink provides official http api to submit jar jobs and query
job status, and they might be able to help.
Best,
Yun
[1] https://ci.apache.org/projects/flink/flink-docs-stable/ops/rest_api.html
------------------Original Mail ------------------
Sender:Flavio Pompermaier <po...@okkam.it>
Send Date:Fri Apr 23 15:25:55 2021
Recipients:Yun Gao <yu...@aliyun.com>
CC:gaurav kulkarni <ku...@yahoo.com>, User <us...@flink.apache.org>
Subject:Re: Official flink java client
I also interface to Flink clusters using REST in order to avoid many annoying problems (due to dependency conflicts, classpath or env variables).
I use an extended version of the RestClusterClient that you can reuse if you want to.
It is available at [1] and it add some missing methods to the default Flink version (I also had to copy that class and modify the visibility of some field in order to enable the extension).
Officially the Flink RestClusterClient is meant to be used for internal use only but it actually work very well.
Best,
Flavio
[1] https://github.com/fpompermaier/flink-job-server/blob/main/flink-rest-client/src/main/java/org/apache/flink/client/program/rest/RestClusterClientExtended.java
On Fri, Apr 23, 2021 at 5:10 AM Yun Gao <yu...@aliyun.com> wrote:
Hi gaurav，
Logicall Flink client is bear inside the StreamExecutionEnvironment, and users could use the
StreamExecutionEnvironment to execute their jobs. Could you share more about why you
want to directly use the client?
Best,
Yun
------------------Original Mail ------------------
Sender:gaurav kulkarni <ku...@yahoo.com>
Send Date:Fri Apr 23 10:14:08 2021
Recipients:User <us...@flink.apache.org>
Subject:Official flink java client
Hi,
Is there any official flink client in java that's available? I came across RestClusterClient, but I am not sure if its official. I can create my own client, but just wanted to check if there is anything official available already that I can leverage.
Thanks,
Gaurav
Run already deployed job on Flink Cluster using RestClusterClient
I am trying to run already deployed job on Flink Cluster using Rest request.I had success using a simple rest ...
Re: Re: Official flink java client
Posted by Flavio Pompermaier <po...@okkam.it>.
Obviously I could rewrite a java client from scratch that interface with
the provided REST API but why if I can reuse something already existing?
Usually I interface with REST API using auto generated clients (if APIs are
exposed via Swagger or OpenApi).
If that's not an option, writing a REST client from scratch is something I
try to avoid as much as I can..
Best,
Flavio
On Fri, Apr 23, 2021 at 9:55 AM Yun Gao <yu...@aliyun.com> wrote:
> Hi Falvio,
>
> Very thanks for the explanation, may be another option is to have a look
> at
> the http rest API[1] ? Flink provides official http api to submit jar jobs
> and query
> job status, and they might be able to help.
>
> Best,
> Yun
>
> [1]
> https://ci.apache.org/projects/flink/flink-docs-stable/ops/rest_api.html
>
> ------------------Original Mail ------------------
> *Sender:*Flavio Pompermaier <po...@okkam.it>
> *Send Date:*Fri Apr 23 15:25:55 2021
> *Recipients:*Yun Gao <yu...@aliyun.com>
> *CC:*gaurav kulkarni <ku...@yahoo.com>, User <
> user@flink.apache.org>
> *Subject:*Re: Official flink java client
>
>> I also interface to Flink clusters using REST in order to avoid many
>> annoying problems (due to dependency conflicts, classpath or env variables).
>> I use an extended version of the RestClusterClient that you can reuse if
>> you want to.
>> It is available at [1] and it add some missing methods to the default
>> Flink version (I also had to copy that class and modify the visibility of
>> some field in order to enable the extension).
>> Officially the Flink RestClusterClient is meant to be used for internal
>> use only but it actually work very well.
>>
>> Best,
>> Flavio
>>
>> [1]
>> https://github.com/fpompermaier/flink-job-server/blob/main/flink-rest-client/src/main/java/org/apache/flink/client/program/rest/RestClusterClientExtended.java
>>
>> On Fri, Apr 23, 2021 at 5:10 AM Yun Gao <yu...@aliyun.com> wrote:
>>
>>> Hi gaurav，
>>>
>>> Logicall Flink client is bear inside the StreamExecutionEnvironment, and
>>> users could use the
>>> StreamExecutionEnvironment to execute their jobs. Could you share more
>>> about why you
>>> want to directly use the client?
>>>
>>> Best,
>>> Yun
>>>
>>>
>>> ------------------Original Mail ------------------
>>> *Sender:*gaurav kulkarni <ku...@yahoo.com>
>>> *Send Date:*Fri Apr 23 10:14:08 2021
>>> *Recipients:*User <us...@flink.apache.org>
>>> *Subject:*Official flink java client
>>>
>>>> Hi,
>>>>
>>>> Is there any official flink client in java that's available? I came
>>>> across RestClusterClient
>>>> <https://ci.apache.org/projects/flink/flink-docs-master/api/java/org/apache/flink/client/program/rest/RestClusterClient.html>,
>>>> but I am not sure if its official
>>>> <https://stackoverflow.com/questions/56079788/run-already-deployed-job-on-flink-cluster-using-restclusterclient>.
>>>> I can create my own client, but just wanted to check if there is anything
>>>> official available already that I can leverage.
>>>>
>>>> Thanks,
>>>> Gaurav
>>>>
>>>> Run already deployed job on Flink Cluster using RestClusterClient
>>>>
>>>> I am trying to run already deployed job on Flink Cluster using Rest
>>>> request.I had success using a simple rest ...
>>>>
>>>> <https://stackoverflow.com/questions/56079788/run-already-deployed-job-on-flink-cluster-using-restclusterclient>
>>>>
>>>>
>>>>
Re: Re: Official flink java client
Posted by Yun Gao <yu...@aliyun.com>.
Hi Falvio,
Very thanks for the explanation, may be another option is to have a look at
the http rest API[1] ? Flink provides official http api to submit jar jobs and query
job status, and they might be able to help.
Best,
Yun
[1] https://ci.apache.org/projects/flink/flink-docs-stable/ops/rest_api.html
------------------Original Mail ------------------
Sender:Flavio Pompermaier <po...@okkam.it>
Send Date:Fri Apr 23 15:25:55 2021
Recipients:Yun Gao <yu...@aliyun.com>
CC:gaurav kulkarni <ku...@yahoo.com>, User <us...@flink.apache.org>
Subject:Re: Official flink java client
I also interface to Flink clusters using REST in order to avoid many annoying problems (due to dependency conflicts, classpath or env variables).
I use an extended version of the RestClusterClient that you can reuse if you want to.
It is available at [1] and it add some missing methods to the default Flink version (I also had to copy that class and modify the visibility of some field in order to enable the extension).
Officially the Flink RestClusterClient is meant to be used for internal use only but it actually work very well.
Best,
Flavio
[1] https://github.com/fpompermaier/flink-job-server/blob/main/flink-rest-client/src/main/java/org/apache/flink/client/program/rest/RestClusterClientExtended.java
On Fri, Apr 23, 2021 at 5:10 AM Yun Gao <yu...@aliyun.com> wrote:
Hi gaurav，
Logicall Flink client is bear inside the StreamExecutionEnvironment, and users could use the
StreamExecutionEnvironment to execute their jobs. Could you share more about why you
want to directly use the client?
Best,
Yun
------------------Original Mail ------------------
Sender:gaurav kulkarni <ku...@yahoo.com>
Send Date:Fri Apr 23 10:14:08 2021
Recipients:User <us...@flink.apache.org>
Subject:Official flink java client
Hi,
Is there any official flink client in java that's available? I came across RestClusterClient, but I am not sure if its official. I can create my own client, but just wanted to check if there is anything official available already that I can leverage.
Thanks,
Gaurav
Run already deployed job on Flink Cluster using RestClusterClient
I am trying to run already deployed job on Flink Cluster using Rest request.I had success using a simple rest ...
Re: Official flink java client
Posted by Flavio Pompermaier <po...@okkam.it>.
I also interface to Flink clusters using REST in order to avoid many
annoying problems (due to dependency conflicts, classpath or env variables).
I use an extended version of the RestClusterClient that you can reuse if
you want to.
It is available at [1] and it add some missing methods to the default Flink
version (I also had to copy that class and modify the visibility of some
field in order to enable the extension).
Officially the Flink RestClusterClient is meant to be used for internal use
only but it actually work very well.
Best,
Flavio
[1]
https://github.com/fpompermaier/flink-job-server/blob/main/flink-rest-client/src/main/java/org/apache/flink/client/program/rest/RestClusterClientExtended.java
On Fri, Apr 23, 2021 at 5:10 AM Yun Gao <yu...@aliyun.com> wrote:
> Hi gaurav，
>
> Logicall Flink client is bear inside the StreamExecutionEnvironment, and
> users could use the
> StreamExecutionEnvironment to execute their jobs. Could you share more
> about why you
> want to directly use the client?
>
> Best,
> Yun
>
>
> ------------------Original Mail ------------------
> *Sender:*gaurav kulkarni <ku...@yahoo.com>
> *Send Date:*Fri Apr 23 10:14:08 2021
> *Recipients:*User <us...@flink.apache.org>
> *Subject:*Official flink java client
>
>> Hi,
>>
>> Is there any official flink client in java that's available? I came
>> across RestClusterClient
>> <https://ci.apache.org/projects/flink/flink-docs-master/api/java/org/apache/flink/client/program/rest/RestClusterClient.html>,
>> but I am not sure if its official
>> <https://stackoverflow.com/questions/56079788/run-already-deployed-job-on-flink-cluster-using-restclusterclient>.
>> I can create my own client, but just wanted to check if there is anything
>> official available already that I can leverage.
>>
>> Thanks,
>> Gaurav
>>
>> Run already deployed job on Flink Cluster using RestClusterClient
>>
>> I am trying to run already deployed job on Flink Cluster using Rest
>> request.I had success using a simple rest ...
>>
>> <https://stackoverflow.com/questions/56079788/run-already-deployed-job-on-flink-cluster-using-restclusterclient>
>>
>>
>>
Re: Official flink java client
Posted by Yun Gao <yu...@aliyun.com>.
Hi gaurav，
Logicall Flink client is bear inside the StreamExecutionEnvironment, and users could use the
StreamExecutionEnvironment to execute their jobs. Could you share more about why you
want to directly use the client?
Best,
Yun
------------------Original Mail ------------------
Sender:gaurav kulkarni <ku...@yahoo.com>
Send Date:Fri Apr 23 10:14:08 2021
Recipients:User <us...@flink.apache.org>
Subject:Official flink java client
Hi,
Is there any official flink client in java that's available? I came across RestClusterClient, but I am not sure if its official. I can create my own client, but just wanted to check if there is anything official available already that I can leverage.
Thanks,
Gaurav
Run already deployed job on Flink Cluster using RestClusterClient
I am trying to run already deployed job on Flink Cluster using Rest request.I had success using a simple rest ...