Query regarding livy session

[Rajeshkumar R <ra...@gmail.com>]

>
> Hello
> I have a livy session which is linked to a spark context. This spark
> context is holding to data from a table in memory. My idea is to use this
> data shareble across multiple users using same livy session and context.
> But livy is not allowing two request at the sametime to the same session.
> The second request is in queue when the first is being processed. Is this
> the way livy is designed or there is some issue in our code?
>
> Thanks
> Rajesh
>
>
>

Re: Query regarding livy session

["Harsch, Tim" <Ti...@Teradata.com>]

If you can guarantee that the statements you are executing are semantically independent, you could create a pool of sessions and execute on any idle session.   Not sure in terms of resources how efficient that would be, depends on the cluster configuration in part.

From: Saisai Shao <sa...@gmail.com>
Reply-To: "user@livy.incubator.apache.org" <us...@livy.incubator.apache.org>
Date: Sunday, June 3, 2018 at 7:07 PM
To: "user@livy.incubator.apache.org" <us...@livy.incubator.apache.org>
Subject: Re: Query regarding livy session

This is by design that Livy doesn't support concurrent execution of different requests. Because Livy doesn't know if the subsequent requests are semantically dependent or not.

Rajeshkumar R <ra...@gmail.com>> 于2018年6月3日周日 下午12:08写道：
Hello
I have a livy session which is linked to a spark context. This spark context is holding to data from a table in memory. My idea is to use this data shareble across multiple users using same livy session and context. But livy is not allowing two request at the sametime to the same session. The second request is in queue when the first is being processed. Is this the way livy is designed or there is some issue in our code?

Thanks
Rajesh

Re: Query regarding livy session

[Saisai Shao <sa...@gmail.com>]

This is by design that Livy doesn't support concurrent execution of
different requests. Because Livy doesn't know if the subsequent requests
are semantically dependent or not.

Rajeshkumar R <ra...@gmail.com> 于2018年6月3日周日 下午12:08写道：

> Hello
>> I have a livy session which is linked to a spark context. This spark
>> context is holding to data from a table in memory. My idea is to use this
>> data shareble across multiple users using same livy session and context.
>> But livy is not allowing two request at the sametime to the same session.
>> The second request is in queue when the first is being processed. Is this
>> the way livy is designed or there is some issue in our code?
>>
>> Thanks
>> Rajesh
>>
>>
>>