You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@livy.apache.org by kant kodali <ka...@gmail.com> on 2018/06/01 19:01:07 UTC

is there anyway to speed up launching a job?

Hi All,

is there anyway to speed up launching a job? Right now it takes about >=
500ms to launch a job via livy. Trying to see if I can get it under <10ms?

Thanks!

Re: is there anyway to speed up launching a job?

Posted by kant kodali <ka...@gmail.com>.

The nice thing about livy is it offers REST interface so when integrated
with frontend users normally expect fast response times so they can spawn
more streaming / batch jobs.

thanks!

On Fri, Jun 1, 2018 at 1:17 PM, Adrian Petrescu <ap...@gmail.com> wrote:

> Just out of personal curiosity, what is the use-case here? Given the
> massive amount of overhead involved in running Spark jobs of any kind (on
> the order of minutes), I've never heard of anyone angsting over ~500ms of
> latency before.
>
> On Fri, Jun 1, 2018 at 3:01 PM kant kodali <ka...@gmail.com> wrote:
>
>> Hi All,
>>
>> is there anyway to speed up launching a job? Right now it takes about >=
>> 500ms to launch a job via livy. Trying to see if I can get it under <10ms?
>>
>> Thanks!
>>
>

Re: is there anyway to speed up launching a job?

Posted by Adrian Petrescu <ap...@gmail.com>.

Just out of personal curiosity, what is the use-case here? Given the
massive amount of overhead involved in running Spark jobs of any kind (on
the order of minutes), I've never heard of anyone angsting over ~500ms of
latency before.

On Fri, Jun 1, 2018 at 3:01 PM kant kodali <ka...@gmail.com> wrote:

> Hi All,
>
> is there anyway to speed up launching a job? Right now it takes about >=
> 500ms to launch a job via livy. Trying to see if I can get it under <10ms?
>
> Thanks!
>