You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Chaim Turkel <ch...@behalf.com> on 2019/05/02 06:48:06 UTC
cancel job
Hi,
I have a batch job that should run for about 40 minutes. There are
times that it can run for hours, and i don't know why.
I need the option to cancel the job if it runs for more than x
minutes. I can do this from the gui or the gcloud cli.
Is there an api code that i can do this preferable in python, java is also ok
chaim
--
Loans are funded by
FinWise Bank, a Utah-chartered bank located in Sandy,
Utah, member FDIC, Equal
Opportunity Lender. Merchant Cash Advances are
made by Behalf. For more
information on ECOA, click here
<https://www.behalf.com/legal/ecoa/>. For important information about
opening a new
account, review Patriot Act procedures here
<https://www.behalf.com/legal/patriot/>.
Visit Legal
<https://www.behalf.com/legal/> to
review our comprehensive program terms,
conditions, and disclosures.
Re: cancel job
Posted by Chaim Turkel <ch...@behalf.com>.
thanks for the reply,
i am using airflow python code to run a java runner, so i do not
have the actual pipleine handler, is there a way to get it?
chaim
On Thu, May 2, 2019 at 7:58 PM Lukasz Cwik <lc...@google.com> wrote:
>
> +user@beam.apache.org
>
> On Thu, May 2, 2019 at 9:51 AM Lukasz Cwik <lc...@google.com> wrote:
>>
>> ... build pipeline ...
>> pipeline_result = p.run()
>> if job_taking_too_long:
>> pipeline_result.cancel()
>>
>> Python: https://github.com/apache/beam/blob/95d0ac5e5cb59fd0c6a8a4861a38a7087a6c46b5/sdks/python/apache_beam/runners/runner.py#L372
>> Java: https://github.com/apache/beam/blob/ce77db10cdd5f021f383721a90f30205aff0fabe/sdks/java/core/src/main/java/org/apache/beam/sdk/PipelineResult.java#L46
>>
>> On Wed, May 1, 2019 at 11:48 PM Chaim Turkel <ch...@behalf.com> wrote:
>>>
>>> Hi,
>>> I have a batch job that should run for about 40 minutes. There are
>>> times that it can run for hours, and i don't know why.
>>> I need the option to cancel the job if it runs for more than x
>>> minutes. I can do this from the gui or the gcloud cli.
>>> Is there an api code that i can do this preferable in python, java is also ok
>>>
>>>
>>> chaim
>>>
>>> --
>>>
>>>
>>> Loans are funded by
>>> FinWise Bank, a Utah-chartered bank located in Sandy,
>>> Utah, member FDIC, Equal
>>> Opportunity Lender. Merchant Cash Advances are
>>> made by Behalf. For more
>>> information on ECOA, click here
>>> <https://www.behalf.com/legal/ecoa/>. For important information about
>>> opening a new
>>> account, review Patriot Act procedures here
>>> <https://www.behalf.com/legal/patriot/>.
>>> Visit Legal
>>> <https://www.behalf.com/legal/> to
>>> review our comprehensive program terms,
>>> conditions, and disclosures.
--
Loans are funded by
FinWise Bank, a Utah-chartered bank located in Sandy,
Utah, member FDIC, Equal
Opportunity Lender. Merchant Cash Advances are
made by Behalf. For more
information on ECOA, click here
<https://www.behalf.com/legal/ecoa/>. For important information about
opening a new
account, review Patriot Act procedures here
<https://www.behalf.com/legal/patriot/>.
Visit Legal
<https://www.behalf.com/legal/> to
review our comprehensive program terms,
conditions, and disclosures.
Re: cancel job
Posted by Lukasz Cwik <lc...@google.com>.
+user@beam.apache.org <us...@beam.apache.org>
On Thu, May 2, 2019 at 9:51 AM Lukasz Cwik <lc...@google.com> wrote:
> ... build pipeline ...
> pipeline_result = p.run()
> if job_taking_too_long:
> pipeline_result.cancel()
>
> Python:
> https://github.com/apache/beam/blob/95d0ac5e5cb59fd0c6a8a4861a38a7087a6c46b5/sdks/python/apache_beam/runners/runner.py#L372
> Java:
> https://github.com/apache/beam/blob/ce77db10cdd5f021f383721a90f30205aff0fabe/sdks/java/core/src/main/java/org/apache/beam/sdk/PipelineResult.java#L46
>
> On Wed, May 1, 2019 at 11:48 PM Chaim Turkel <ch...@behalf.com> wrote:
>
>> Hi,
>> I have a batch job that should run for about 40 minutes. There are
>> times that it can run for hours, and i don't know why.
>> I need the option to cancel the job if it runs for more than x
>> minutes. I can do this from the gui or the gcloud cli.
>> Is there an api code that i can do this preferable in python, java is
>> also ok
>>
>>
>> chaim
>>
>> --
>>
>>
>> Loans are funded by
>> FinWise Bank, a Utah-chartered bank located in Sandy,
>> Utah, member FDIC, Equal
>> Opportunity Lender. Merchant Cash Advances are
>> made by Behalf. For more
>> information on ECOA, click here
>> <https://www.behalf.com/legal/ecoa/>. For important information about
>> opening a new
>> account, review Patriot Act procedures here
>> <https://www.behalf.com/legal/patriot/>.
>> Visit Legal
>> <https://www.behalf.com/legal/> to
>> review our comprehensive program terms,
>> conditions, and disclosures.
>>
>
Re: cancel job
Posted by Lukasz Cwik <lc...@google.com>.
+user@beam.apache.org <us...@beam.apache.org>
On Thu, May 2, 2019 at 9:51 AM Lukasz Cwik <lc...@google.com> wrote:
> ... build pipeline ...
> pipeline_result = p.run()
> if job_taking_too_long:
> pipeline_result.cancel()
>
> Python:
> https://github.com/apache/beam/blob/95d0ac5e5cb59fd0c6a8a4861a38a7087a6c46b5/sdks/python/apache_beam/runners/runner.py#L372
> Java:
> https://github.com/apache/beam/blob/ce77db10cdd5f021f383721a90f30205aff0fabe/sdks/java/core/src/main/java/org/apache/beam/sdk/PipelineResult.java#L46
>
> On Wed, May 1, 2019 at 11:48 PM Chaim Turkel <ch...@behalf.com> wrote:
>
>> Hi,
>> I have a batch job that should run for about 40 minutes. There are
>> times that it can run for hours, and i don't know why.
>> I need the option to cancel the job if it runs for more than x
>> minutes. I can do this from the gui or the gcloud cli.
>> Is there an api code that i can do this preferable in python, java is
>> also ok
>>
>>
>> chaim
>>
>> --
>>
>>
>> Loans are funded by
>> FinWise Bank, a Utah-chartered bank located in Sandy,
>> Utah, member FDIC, Equal
>> Opportunity Lender. Merchant Cash Advances are
>> made by Behalf. For more
>> information on ECOA, click here
>> <https://www.behalf.com/legal/ecoa/>. For important information about
>> opening a new
>> account, review Patriot Act procedures here
>> <https://www.behalf.com/legal/patriot/>.
>> Visit Legal
>> <https://www.behalf.com/legal/> to
>> review our comprehensive program terms,
>> conditions, and disclosures.
>>
>
Re: cancel job
Posted by Lukasz Cwik <lc...@google.com>.
... build pipeline ...
pipeline_result = p.run()
if job_taking_too_long:
pipeline_result.cancel()
Python:
https://github.com/apache/beam/blob/95d0ac5e5cb59fd0c6a8a4861a38a7087a6c46b5/sdks/python/apache_beam/runners/runner.py#L372
Java:
https://github.com/apache/beam/blob/ce77db10cdd5f021f383721a90f30205aff0fabe/sdks/java/core/src/main/java/org/apache/beam/sdk/PipelineResult.java#L46
On Wed, May 1, 2019 at 11:48 PM Chaim Turkel <ch...@behalf.com> wrote:
> Hi,
> I have a batch job that should run for about 40 minutes. There are
> times that it can run for hours, and i don't know why.
> I need the option to cancel the job if it runs for more than x
> minutes. I can do this from the gui or the gcloud cli.
> Is there an api code that i can do this preferable in python, java is
> also ok
>
>
> chaim
>
> --
>
>
> Loans are funded by
> FinWise Bank, a Utah-chartered bank located in Sandy,
> Utah, member FDIC, Equal
> Opportunity Lender. Merchant Cash Advances are
> made by Behalf. For more
> information on ECOA, click here
> <https://www.behalf.com/legal/ecoa/>. For important information about
> opening a new
> account, review Patriot Act procedures here
> <https://www.behalf.com/legal/patriot/>.
> Visit Legal
> <https://www.behalf.com/legal/> to
> review our comprehensive program terms,
> conditions, and disclosures.
>