You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Timothy Victor <vi...@gmail.com> on 2019/10/19 14:36:27 UTC
Submitting jobs via REST
I have a flink docker image with my job's JAR already contained within. I
would like to run a job with this jar via the REST api. Is that possible?
I know I can run a job via REST using JarID (ID assigned by flink when a
jar is uploaded). However I don't have such an ID since this jar is
already part of the image.
Via CLI I can start a job using classpath. But can I do the same via the
REST api. Any other ways to achieve this?
Thanks
Tim
Re: Submitting jobs via REST
Posted by Timothy Victor <vi...@gmail.com>.
Thank you for your help Pritam. I got this working now. The missing pool
piece for me was that I had to set the "web.upload.dir" which my default is
some auto generated directory under tmp.
One problem I had was that the jar actually needs to be placed under
$web.upload.dir/flink-web-upload, i.e.a hardcoded subdir under the user
specified dir. It would be nice if that didn't happen and flink just read
directly from the root ($web.upload.dir). I can create a ticket for it to
see if there is any general interest in such a change.
Thanks again for your help
Tim
On Mon, Oct 21, 2019, 10:37 AM Pritam Sadhukhan <sa...@gmail.com>
wrote:
> Can you please share your dockerfile?
> Please upload your jar at /opt/flink/product-libs/flink-web-upload/.
>
> Regards,
> Pritam.
>
> On Mon, 21 Oct 2019 at 19:58, Timothy Victor <vi...@gmail.com> wrote:
>
>> Thanks Pritam.
>>
>> Unfortunately this does not work for me. I get a response that says
>> "jar file /tmp/flink-web-<guid>/flink-web-upload/<jarname> does not exist".
>>
>> It is looking for the jar in the tmp folder. Wonder of there is a way to
>> change that so that it looks in the right folder.
>>
>> Thanks
>>
>> Tim
>>
>> On Sun, Oct 20, 2019, 7:55 AM Pritam Sadhukhan <
>> sadhukhan.pritam@gmail.com> wrote:
>>
>>> Hi Tim,
>>>
>>> I have the similar scenario where I have embedded my jar within the
>>> image.
>>>
>>> I used the following command to submit the job :
>>>
>>> curl -X POST http://localhost:8081/jars/<jar-name>.jar/run
>>> <http://localhost:8081/jars/28f05eb0-9aab-4a18-ae66-f1e10970c11f_soar-ueba-training-service.jar/run>
>>>
>>> with the request parameters if any.
>>>
>>>
>>> Please let me know if this helps.
>>>
>>>
>>> Regards,
>>>
>>> Pritam.
>>>
>>> On Sat, 19 Oct 2019 at 20:06, Timothy Victor <vi...@gmail.com> wrote:
>>>
>>>> I have a flink docker image with my job's JAR already contained
>>>> within. I would like to run a job with this jar via the REST api. Is that
>>>> possible?
>>>>
>>>> I know I can run a job via REST using JarID (ID assigned by flink when
>>>> a jar is uploaded). However I don't have such an ID since this jar is
>>>> already part of the image.
>>>>
>>>> Via CLI I can start a job using classpath. But can I do the same via
>>>> the REST api. Any other ways to achieve this?
>>>>
>>>> Thanks
>>>>
>>>> Tim
>>>>
>>>
Re: Submitting jobs via REST
Posted by Pritam Sadhukhan <sa...@gmail.com>.
Can you please share your dockerfile?
Please upload your jar at /opt/flink/product-libs/flink-web-upload/.
Regards,
Pritam.
On Mon, 21 Oct 2019 at 19:58, Timothy Victor <vi...@gmail.com> wrote:
> Thanks Pritam.
>
> Unfortunately this does not work for me. I get a response that says "jar
> file /tmp/flink-web-<guid>/flink-web-upload/<jarname> does not exist".
>
> It is looking for the jar in the tmp folder. Wonder of there is a way to
> change that so that it looks in the right folder.
>
> Thanks
>
> Tim
>
> On Sun, Oct 20, 2019, 7:55 AM Pritam Sadhukhan <sa...@gmail.com>
> wrote:
>
>> Hi Tim,
>>
>> I have the similar scenario where I have embedded my jar within the image.
>>
>> I used the following command to submit the job :
>>
>> curl -X POST http://localhost:8081/jars/<jar-name>.jar/run
>> <http://localhost:8081/jars/28f05eb0-9aab-4a18-ae66-f1e10970c11f_soar-ueba-training-service.jar/run>
>>
>> with the request parameters if any.
>>
>>
>> Please let me know if this helps.
>>
>>
>> Regards,
>>
>> Pritam.
>>
>> On Sat, 19 Oct 2019 at 20:06, Timothy Victor <vi...@gmail.com> wrote:
>>
>>> I have a flink docker image with my job's JAR already contained within.
>>> I would like to run a job with this jar via the REST api. Is that possible?
>>>
>>> I know I can run a job via REST using JarID (ID assigned by flink when a
>>> jar is uploaded). However I don't have such an ID since this jar is
>>> already part of the image.
>>>
>>> Via CLI I can start a job using classpath. But can I do the same via
>>> the REST api. Any other ways to achieve this?
>>>
>>> Thanks
>>>
>>> Tim
>>>
>>
Re: Submitting jobs via REST
Posted by Timothy Victor <vi...@gmail.com>.
Thanks Pritam.
Unfortunately this does not work for me. I get a response that says "jar
file /tmp/flink-web-<guid>/flink-web-upload/<jarname> does not exist".
It is looking for the jar in the tmp folder. Wonder of there is a way to
change that so that it looks in the right folder.
Thanks
Tim
On Sun, Oct 20, 2019, 7:55 AM Pritam Sadhukhan <sa...@gmail.com>
wrote:
> Hi Tim,
>
> I have the similar scenario where I have embedded my jar within the image.
>
> I used the following command to submit the job :
>
> curl -X POST http://localhost:8081/jars/<jar-name>.jar/run
> <http://localhost:8081/jars/28f05eb0-9aab-4a18-ae66-f1e10970c11f_soar-ueba-training-service.jar/run>
>
> with the request parameters if any.
>
>
> Please let me know if this helps.
>
>
> Regards,
>
> Pritam.
>
> On Sat, 19 Oct 2019 at 20:06, Timothy Victor <vi...@gmail.com> wrote:
>
>> I have a flink docker image with my job's JAR already contained within.
>> I would like to run a job with this jar via the REST api. Is that possible?
>>
>> I know I can run a job via REST using JarID (ID assigned by flink when a
>> jar is uploaded). However I don't have such an ID since this jar is
>> already part of the image.
>>
>> Via CLI I can start a job using classpath. But can I do the same via the
>> REST api. Any other ways to achieve this?
>>
>> Thanks
>>
>> Tim
>>
>
Re: Submitting jobs via REST
Posted by Pritam Sadhukhan <sa...@gmail.com>.
Hi Tim,
I have the similar scenario where I have embedded my jar within the image.
I used the following command to submit the job :
curl -X POST http://localhost:8081/jars/<jar-name>.jar/run
<http://localhost:8081/jars/28f05eb0-9aab-4a18-ae66-f1e10970c11f_soar-ueba-training-service.jar/run>
with the request parameters if any.
Please let me know if this helps.
Regards,
Pritam.
On Sat, 19 Oct 2019 at 20:06, Timothy Victor <vi...@gmail.com> wrote:
> I have a flink docker image with my job's JAR already contained within. I
> would like to run a job with this jar via the REST api. Is that possible?
>
> I know I can run a job via REST using JarID (ID assigned by flink when a
> jar is uploaded). However I don't have such an ID since this jar is
> already part of the image.
>
> Via CLI I can start a job using classpath. But can I do the same via the
> REST api. Any other ways to achieve this?
>
> Thanks
>
> Tim
>