You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Raju Bairishetti <ra...@gmail.com> on 2016/05/18 01:03:23 UTC
How to run hive queries in async mode using spark sql
I am using spark sql for running hive queries also. Is there any way to run
hive queries in asyc mode using spark sql.
Does it return any hive handle or if yes how to get the results from hive
handle using spark sql?
--
Thanks,
Raju Bairishetti,
www.lazada.com
Re: How to run hive queries in async mode using spark sql
Posted by Mich Talebzadeh <mi...@gmail.com>.
fine give me an example where you have tried to turn on async for the query
using spark sql. Your actual code.
HTH
Dr Mich Talebzadeh
LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
http://talebzadehmich.wordpress.com
On 24 May 2016 at 12:25, Raju Bairishetti <ra...@gmail.com> wrote:
> Hi Mich,
>
> Thanks for the response.
>
> yes, I do not want to block until the hive query is completed and want to
> know is there any way to poll the status/progress of submitted query.
>
> I can turn on asyc mode for hive queries in spark sql but how to track
> the status of the submitted query?
>
> On Tue, May 24, 2016 at 6:48 PM, Mich Talebzadeh <
> mich.talebzadeh@gmail.com> wrote:
>
>> By Hive queries in async mode, you mean submitting sql queries to Hive
>> and move on to the next operation and wait for return of result set from
>> Hive?
>>
>> Dr Mich Talebzadeh
>>
>>
>>
>> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>>
>>
>>
>> http://talebzadehmich.wordpress.com
>>
>>
>>
>> On 24 May 2016 at 11:26, Raju Bairishetti <ra...@gmail.com> wrote:
>>
>>> Any thoughts on this?
>>>
>>> In hive, it returns operation handle. This handle can be used for
>>> fetching the status of query. Is there any similar mechanism in spark sql?
>>> Looks like almost all the methods in the HiveContext are either protected
>>> or private.
>>>
>>> On Wed, May 18, 2016 at 9:03 AM, Raju Bairishetti <ra...@gmail.com>
>>> wrote:
>>>
>>>> I am using spark sql for running hive queries also. Is there any way to
>>>> run hive queries in asyc mode using spark sql.
>>>>
>>>> Does it return any hive handle or if yes how to get the results from
>>>> hive handle using spark sql?
>>>>
>>>> --
>>>> Thanks,
>>>> Raju Bairishetti,
>>>>
>>>> www.lazada.com
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Thanks,
>>> Raju Bairishetti,
>>>
>>> www.lazada.com
>>>
>>>
>>>
>>
>
>
> --
> Thanks,
> Raju Bairishetti,
>
> www.lazada.com
>
>
>
Re: How to run hive queries in async mode using spark sql
Posted by Raju Bairishetti <ra...@gmail.com>.
Hi Mich,
Thanks for the response.
yes, I do not want to block until the hive query is completed and want to
know is there any way to poll the status/progress of submitted query.
I can turn on asyc mode for hive queries in spark sql but how to track the
status of the submitted query?
On Tue, May 24, 2016 at 6:48 PM, Mich Talebzadeh <mi...@gmail.com>
wrote:
> By Hive queries in async mode, you mean submitting sql queries to Hive and
> move on to the next operation and wait for return of result set from Hive?
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>
>
>
> http://talebzadehmich.wordpress.com
>
>
>
> On 24 May 2016 at 11:26, Raju Bairishetti <ra...@gmail.com> wrote:
>
>> Any thoughts on this?
>>
>> In hive, it returns operation handle. This handle can be used for
>> fetching the status of query. Is there any similar mechanism in spark sql?
>> Looks like almost all the methods in the HiveContext are either protected
>> or private.
>>
>> On Wed, May 18, 2016 at 9:03 AM, Raju Bairishetti <ra...@gmail.com>
>> wrote:
>>
>>> I am using spark sql for running hive queries also. Is there any way to
>>> run hive queries in asyc mode using spark sql.
>>>
>>> Does it return any hive handle or if yes how to get the results from
>>> hive handle using spark sql?
>>>
>>> --
>>> Thanks,
>>> Raju Bairishetti,
>>>
>>> www.lazada.com
>>>
>>>
>>>
>>
>>
>> --
>> Thanks,
>> Raju Bairishetti,
>>
>> www.lazada.com
>>
>>
>>
>
--
Thanks,
Raju Bairishetti,
www.lazada.com
Re: How to run hive queries in async mode using spark sql
Posted by Mich Talebzadeh <mi...@gmail.com>.
By Hive queries in async mode, you mean submitting sql queries to Hive and
move on to the next operation and wait for return of result set from Hive?
Dr Mich Talebzadeh
LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
http://talebzadehmich.wordpress.com
On 24 May 2016 at 11:26, Raju Bairishetti <ra...@gmail.com> wrote:
> Any thoughts on this?
>
> In hive, it returns operation handle. This handle can be used for fetching
> the status of query. Is there any similar mechanism in spark sql? Looks
> like almost all the methods in the HiveContext are either protected or
> private.
>
> On Wed, May 18, 2016 at 9:03 AM, Raju Bairishetti <ra...@gmail.com>
> wrote:
>
>> I am using spark sql for running hive queries also. Is there any way to
>> run hive queries in asyc mode using spark sql.
>>
>> Does it return any hive handle or if yes how to get the results from hive
>> handle using spark sql?
>>
>> --
>> Thanks,
>> Raju Bairishetti,
>>
>> www.lazada.com
>>
>>
>>
>
>
> --
> Thanks,
> Raju Bairishetti,
>
> www.lazada.com
>
>
>
Re: How to run hive queries in async mode using spark sql
Posted by Raju Bairishetti <ra...@gmail.com>.
Any thoughts on this?
In hive, it returns operation handle. This handle can be used for fetching
the status of query. Is there any similar mechanism in spark sql? Looks
like almost all the methods in the HiveContext are either protected or
private.
On Wed, May 18, 2016 at 9:03 AM, Raju Bairishetti <ra...@gmail.com>
wrote:
> I am using spark sql for running hive queries also. Is there any way to
> run hive queries in asyc mode using spark sql.
>
> Does it return any hive handle or if yes how to get the results from hive
> handle using spark sql?
>
> --
> Thanks,
> Raju Bairishetti,
>
> www.lazada.com
>
>
>
--
Thanks,
Raju Bairishetti,
www.lazada.com