You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@livy.apache.org by Barrett Strausser <ba...@bossanova.com> on 2017/08/28 22:05:42 UTC
Access Queue and App Name
As documented in https://github.com/cloudera/livy#post-batches
How do I access the queue and app names once submitted in a batch job?
e.g. If I wanted to find an app with a certain name or submitted to a
certain queue how would I do that?
Re: Access Queue and App Name
Posted by Barrett Strausser <ba...@bossanova.com>.
Understood.
In my use case I'll be submitting a streaming job. So I'll need to
periodically stop it as revisions are made.
I imagine I'll end up doing that portion through YARN directly.
On Mon, Aug 28, 2017 at 7:24 PM, Alex Bozarth <aj...@us.ibm.com> wrote:
> That is correct, you will need to know the batchId associated with a
> session if you want to kill it while it is still running. In the case of
> batch session though, they end when finished (unlike interactive sessions
> which run until killed or closed). Also note that you can retrieve the list
> of batches running or recently finished by calling GET /batches but you
> would still need the batch id to know which is which. This process will
> also be easier when the new Web UI is released in Livy 0.4.0 soon.
>
>
> *Alex Bozarth*
> Software Engineer
> Spark Technology Center
> ------------------------------
> *E-mail:* *ajbozart@us.ibm.com* <aj...@us.ibm.com>
> *GitHub: **github.com/ajbozarth* <https://github.com/ajbozarth>
>
>
> 505 Howard Street
> San Francisco, CA 94105
> United States
>
>
>
> [image: Inactive hide details for Barrett Strausser ---08/28/2017 03:39:21
> PM---Thanks for the fast response. My more direct question i]Barrett
> Strausser ---08/28/2017 03:39:21 PM---Thanks for the fast response. My more
> direct question is: If I submit a batch via Livy, my only opti
>
> From: Barrett Strausser <ba...@bossanova.com>
> To: user@livy.incubator.apache.org
> Date: 08/28/2017 03:39 PM
> Subject: Re: Access Queue and App Name
> ------------------------------
>
>
>
> Thanks for the fast response.
>
> My more direct question is: If I submit a batch via Livy, my only option
> is to store the returned batch id, so that I can delete that batch in the
> future?
>
> -b
>
> On Mon, Aug 28, 2017 at 6:21 PM, Alex Bozarth <*ajbozart@us.ibm.com*
> <aj...@us.ibm.com>> wrote:
>
> As far as I know you can't get that info from the Livy API currently,
> but you can get the YARN Application URL from the appInfo returned in a GET
> /batches/{batchId} call and could find that info there.
>
>
>
> *Alex Bozarth*
> Software Engineer
> Spark Technology Center
> ------------------------------
> *E-mail:* *ajbozart@us.ibm.com* <aj...@us.ibm.com>
> *GitHub: **github.com/ajbozarth*
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_ajbozarth&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=S1_S7Dymu4ZL6g7L21O78VQZ53vEnAyZ-cx37DPYDyo&m=tKlVvT4dh80ulW9El17-FBExWVFbcZjkHKNZ07YVCaE&s=xF9ppQkm3ZyLL-aDH3GdP5UJJu6J8yDHJov2t8fvGZE&e=>
>
>
> 505 Howard Street
> San Francisco, CA 94105
> United States
>
>
>
>
> [image: Inactive hide details for Barrett Strausser ---08/28/2017
> 03:05:48 PM---As documented in https://urldefense.proofpoint.com/v2/u]Barrett
> Strausser ---08/28/2017 03:05:48 PM---As documented in
> *https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_cloudera_livy-23pos*
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_cloudera_livy-23pos>
>
> From: Barrett Strausser <*barrett@bossanova.com*
> <ba...@bossanova.com>>
> To: *user@livy.incubator.apache.org* <us...@livy.incubator.apache.org>
> Date: 08/28/2017 03:05 PM
> Subject: Access Queue and App Name
> ------------------------------
>
>
>
> As documented in *https://github.com/cloudera/livy#post-batches*
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_cloudera_livy-23post-2Dbatches&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=S1_S7Dymu4ZL6g7L21O78VQZ53vEnAyZ-cx37DPYDyo&m=k_VY8cqk7fUyyxfbYYGa_rF_YSp3_kzz93AwnCcK4sA&s=yRPCpu5ZWQW7aO8KDBmKQWjgzQaTQRwxC7G510Z6Isg&e=>
>
> How do I access the queue and app names once submitted in a batch job?
>
> e.g. If I wanted to find an app with a certain name or submitted to a
> certain queue how would I do that?
>
>
>
>
>
Re: Access Queue and App Name
Posted by Alex Bozarth <aj...@us.ibm.com>.
That is correct, you will need to know the batchId associated with a
session if you want to kill it while it is still running. In the case of
batch session though, they end when finished (unlike interactive sessions
which run until killed or closed). Also note that you can retrieve the list
of batches running or recently finished by calling GET /batches but you
would still need the batch id to know which is which. This process will
also be easier when the new Web UI is released in Livy 0.4.0 soon.
Alex Bozarth
Software Engineer
Spark Technology Center
E-mail: ajbozart@us.ibm.com
GitHub: github.com/ajbozarth
505 Howard Street
San Francisco, CA 94105
United States
From: Barrett Strausser <ba...@bossanova.com>
To: user@livy.incubator.apache.org
Date: 08/28/2017 03:39 PM
Subject: Re: Access Queue and App Name
Thanks for the fast response.
My more direct question is: If I submit a batch via Livy, my only option is
to store the returned batch id, so that I can delete that batch in the
future?
-b
On Mon, Aug 28, 2017 at 6:21 PM, Alex Bozarth <aj...@us.ibm.com> wrote:
As far as I know you can't get that info from the Livy API currently, but
you can get the YARN Application URL from the appInfo returned in a
GET /batches/{batchId} call and could find that info there.
Alex Bozarth
Software Engineer
Spark Technology Center
E-mail: ajbozart@us.ibm.com
GitHub: github.com/ajbozarth
505 Howard Street
San Francisco, CA 94105
United States
Inactive hide details for Barrett Strausser ---08/28/2017 03:05:48
PM---As documented in https://urldefense.proofpoint.com/v2/uBarrett
Strausser ---08/28/2017 03:05:48 PM---As documented in
https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_cloudera_livy-23pos
From: Barrett Strausser <ba...@bossanova.com>
To: user@livy.incubator.apache.org
Date: 08/28/2017 03:05 PM
Subject: Access Queue and App Name
As documented in https://github.com/cloudera/livy#post-batches
How do I access the queue and app names once submitted in a batch job?
e.g. If I wanted to find an app with a certain name or submitted to a
certain queue how would I do that?
Re: Access Queue and App Name
Posted by Barrett Strausser <ba...@bossanova.com>.
Thanks for the fast response.
My more direct question is: If I submit a batch via Livy, my only option is
to store the returned batch id, so that I can delete that batch in the
future?
-b
On Mon, Aug 28, 2017 at 6:21 PM, Alex Bozarth <aj...@us.ibm.com> wrote:
> As far as I know you can't get that info from the Livy API currently, but
> you can get the YARN Application URL from the appInfo returned in a GET
> /batches/{batchId} call and could find that info there.
>
>
> *Alex Bozarth*
> Software Engineer
> Spark Technology Center
> ------------------------------
> *E-mail:* *ajbozart@us.ibm.com* <aj...@us.ibm.com>
> *GitHub: **github.com/ajbozarth* <https://github.com/ajbozarth>
>
>
> 505 Howard Street
> San Francisco, CA 94105
> United States
>
>
>
> [image: Inactive hide details for Barrett Strausser ---08/28/2017 03:05:48
> PM---As documented in https://urldefense.proofpoint.com/v2/u]Barrett
> Strausser ---08/28/2017 03:05:48 PM---As documented in
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.
> com_cloudera_livy-23pos
>
> From: Barrett Strausser <ba...@bossanova.com>
> To: user@livy.incubator.apache.org
> Date: 08/28/2017 03:05 PM
> Subject: Access Queue and App Name
> ------------------------------
>
>
>
> As documented in *https://github.com/cloudera/livy#post-batches*
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_cloudera_livy-23post-2Dbatches&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=S1_S7Dymu4ZL6g7L21O78VQZ53vEnAyZ-cx37DPYDyo&m=k_VY8cqk7fUyyxfbYYGa_rF_YSp3_kzz93AwnCcK4sA&s=yRPCpu5ZWQW7aO8KDBmKQWjgzQaTQRwxC7G510Z6Isg&e=>
>
> How do I access the queue and app names once submitted in a batch job?
>
> e.g. If I wanted to find an app with a certain name or submitted to a
> certain queue how would I do that?
>
>
>
Re: Access Queue and App Name
Posted by Alex Bozarth <aj...@us.ibm.com>.
As far as I know you can't get that info from the Livy API currently, but
you can get the YARN Application URL from the appInfo returned in a
GET /batches/{batchId} call and could find that info there.
Alex Bozarth
Software Engineer
Spark Technology Center
E-mail: ajbozart@us.ibm.com
GitHub: github.com/ajbozarth
505 Howard Street
San Francisco, CA 94105
United States
From: Barrett Strausser <ba...@bossanova.com>
To: user@livy.incubator.apache.org
Date: 08/28/2017 03:05 PM
Subject: Access Queue and App Name
As documented in https://github.com/cloudera/livy#post-batches
How do I access the queue and app names once submitted in a batch job?
e.g. If I wanted to find an app with a certain name or submitted to a
certain queue how would I do that?