Multi tenancy - isolated interpreters block each other?

[Wade Jensen <Wa...@quantium.com.au>]

Hello,

I have been experimenting with multi user modes with 0.7.1 in EMR. I set the Spark interpreter scope to be isolated per user, and then ran two different users side by side in different browser tabs. All the privacy and sharing features worked, but whenever I tried to run a spark sql query in both notes at the same time, one query will hang at 0% until the other is completed. (Note the queries where on different data, so it is not that the data was locked by one query).

Sorry if this is documented explicitly somewhere, but is this expected behaviour? Or should I be able to run multiple queries with separate interpreters in parallel?

I looked in the YARN scheduler GUI and could see two separate Spark processes, so I would have thought this should work.

Kind regards,
Wade Jensen

Re: Multi tenancy - isolated interpreters block each other?

[Fabian Böhnlein <fa...@gmail.com>]

Hi Wade,

how's the experimenting going? We usually run scoped 'Per Note'.

Have you played with

zeppelin.spark.concurrentSQL

wich is false by default?

Best,
Fabian

On Fri, 9 Jun 2017 at 13:40 Jongyoul Lee <jo...@gmail.com> wrote:

> Hi,
>
> Thanks for reporting this bug. Can you make a jira issue and describe how
> to reproduce it?
>
> Thanks,
> Jongyoul
>
> On Tue, Jun 6, 2017 at 9:35 AM, Wade Jensen <Wa...@quantium.com.au>
> wrote:
>
>> Hello,
>>
>>
>>
>> I have been experimenting with multi user modes with 0.7.1 in EMR. I set
>> the Spark interpreter scope to be isolated per user, and then ran two
>> different users side by side in different browser tabs. All the privacy and
>> sharing features worked, but whenever I tried to run a spark sql query in
>> both notes at the same time, one query will hang at 0% until the other is
>> completed. (Note the queries where on different data, so it is not that the
>> data was locked by one query).
>>
>>
>>
>> Sorry if this is documented explicitly somewhere, but is this expected
>> behaviour? Or should I be able to run multiple queries with separate
>> interpreters in parallel?
>>
>>
>>
>> I looked in the YARN scheduler GUI and could see two separate Spark
>> processes, so I would have thought this should work.
>>
>>
>>
>> Kind regards,
>>
>> Wade Jensen
>>
>>
>
>
> --
> 이종열, Jongyoul Lee, 李宗烈
> http://madeng.net
>

Re: Multi tenancy - isolated interpreters block each other?

[Jongyoul Lee <jo...@gmail.com>]

Hi,

Thanks for reporting this bug. Can you make a jira issue and describe how
to reproduce it?

Thanks,
Jongyoul

On Tue, Jun 6, 2017 at 9:35 AM, Wade Jensen <Wa...@quantium.com.au>
wrote:

> Hello,
>
>
>
> I have been experimenting with multi user modes with 0.7.1 in EMR. I set
> the Spark interpreter scope to be isolated per user, and then ran two
> different users side by side in different browser tabs. All the privacy and
> sharing features worked, but whenever I tried to run a spark sql query in
> both notes at the same time, one query will hang at 0% until the other is
> completed. (Note the queries where on different data, so it is not that the
> data was locked by one query).
>
>
>
> Sorry if this is documented explicitly somewhere, but is this expected
> behaviour? Or should I be able to run multiple queries with separate
> interpreters in parallel?
>
>
>
> I looked in the YARN scheduler GUI and could see two separate Spark
> processes, so I would have thought this should work.
>
>
>
> Kind regards,
>
> Wade Jensen
>
>


-- 
이종열, Jongyoul Lee, 李宗烈
http://madeng.net