You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Sadhan Sood <sa...@gmail.com> on 2015/10/20 20:55:55 UTC
hive thriftserver and fair scheduling
Hi All,
Does anyone have fair scheduling working for them in a hive server? I have
one hive thriftserver running and multiple users trying to run queries at
the same time on that server using a beeline client. I see that a big query
is stopping all other queries from making any progress. Is this supposed to
be this way? Is there anything else that I need to be doing for fair
scheduling to be working for the thriftserver?
Re: hive thriftserver and fair scheduling
Posted by Deenar Toraskar <de...@gmail.com>.
Thanks Michael, I'll try it out. Another quick/important question: How do I
make udfs available to all of the hive thriftserver users? Right now, when
I launch a spark-sql client, I notice that it reads the ~/.hiverc file and
all udfs get picked up but this doesn't seem to be working in hive
thriftserver.
Is there a way to make it work in a similar way for all users in hive
thriftserver?
+1 for this request
On 20 October 2015 at 23:49, Sadhan Sood <sa...@gmail.com> wrote:
> Thanks Michael, I'll try it out. Another quick/important question: How do
> I make udfs available to all of the hive thriftserver users? Right now,
> when I launch a spark-sql client, I notice that it reads the ~/.hiverc file
> and all udfs get picked up but this doesn't seem to be working in hive
> thriftserver. Is there a way to make it work in a similar way for all users
> in hive thriftserver?
>
> Thanks again!
>
> On Tue, Oct 20, 2015 at 12:34 PM, Michael Armbrust <michael@databricks.com
> > wrote:
>
>> Not the most obvious place in the docs... but this is probably helpful:
>> https://spark.apache.org/docs/latest/sql-programming-guide.html#scheduling
>>
>> You likely want to put each user in their own pool.
>>
>> On Tue, Oct 20, 2015 at 11:55 AM, Sadhan Sood <sa...@gmail.com>
>> wrote:
>>
>>> Hi All,
>>>
>>> Does anyone have fair scheduling working for them in a hive server? I
>>> have one hive thriftserver running and multiple users trying to run queries
>>> at the same time on that server using a beeline client. I see that a big
>>> query is stopping all other queries from making any progress. Is this
>>> supposed to be this way? Is there anything else that I need to be doing for
>>> fair scheduling to be working for the thriftserver?
>>>
>>
>>
>
Re: hive thriftserver and fair scheduling
Posted by Sadhan Sood <sa...@gmail.com>.
Thanks Michael, I'll try it out. Another quick/important question: How do I
make udfs available to all of the hive thriftserver users? Right now, when
I launch a spark-sql client, I notice that it reads the ~/.hiverc file and
all udfs get picked up but this doesn't seem to be working in hive
thriftserver. Is there a way to make it work in a similar way for all users
in hive thriftserver?
Thanks again!
On Tue, Oct 20, 2015 at 12:34 PM, Michael Armbrust <mi...@databricks.com>
wrote:
> Not the most obvious place in the docs... but this is probably helpful:
> https://spark.apache.org/docs/latest/sql-programming-guide.html#scheduling
>
> You likely want to put each user in their own pool.
>
> On Tue, Oct 20, 2015 at 11:55 AM, Sadhan Sood <sa...@gmail.com>
> wrote:
>
>> Hi All,
>>
>> Does anyone have fair scheduling working for them in a hive server? I
>> have one hive thriftserver running and multiple users trying to run queries
>> at the same time on that server using a beeline client. I see that a big
>> query is stopping all other queries from making any progress. Is this
>> supposed to be this way? Is there anything else that I need to be doing for
>> fair scheduling to be working for the thriftserver?
>>
>
>
Re: hive thriftserver and fair scheduling
Posted by Michael Armbrust <mi...@databricks.com>.
Not the most obvious place in the docs... but this is probably helpful:
https://spark.apache.org/docs/latest/sql-programming-guide.html#scheduling
You likely want to put each user in their own pool.
On Tue, Oct 20, 2015 at 11:55 AM, Sadhan Sood <sa...@gmail.com> wrote:
> Hi All,
>
> Does anyone have fair scheduling working for them in a hive server? I have
> one hive thriftserver running and multiple users trying to run queries at
> the same time on that server using a beeline client. I see that a big query
> is stopping all other queries from making any progress. Is this supposed to
> be this way? Is there anything else that I need to be doing for fair
> scheduling to be working for the thriftserver?
>