Is it possible to execute Hive queries parallelly by writing mapper and reducer

[Bhavesh Shah <bh...@gmail.com>]

Hello all,
I am asking you about the increasing the performance of Hive. I tried with
mappers and reducers but I didn't see difference in execution.
Don't know why, may be I did in some other way which may be not correct or
due to some other reason.

I am thinking that Is it possible to execute Hive queries parallelly?
Means,
Normally the queries get execute in queue manner.
query1
query2
query3
.
.
.
n

I am thinking that if we use mapreduce program  in Hive JDBC program, then
is it possible to execute it parallelly.
Don't know will it work or not? Thats I am asking you about it.
But again my questions are:
1) If it is possible then may be it require multiple Hive Thrift Server?
2) Is it possible to open multiple Hive Thrift Server?
3) I think it is not possible to open multiple Hive Thrift Server on same
port.?
4) Can we open multiple Hive Thrift Server on different different port?

Please suggest me some solution to this. If you have other idea other than
this then pls share with with me
I will also try that.
Thanks


-- 
Regards,
Bhavesh Shah

Re: Is it possible to execute Hive queries parallelly by writing mapper and reducer

[Shiva Kumar <sh...@gmail.com>]

HI Bhavesh Shah,

I didn't know how to answer your query due to many reasons. But, look your
problem in this way, instead of starting the multiple thrift servers and
running a query on multiple thrift servers, it is good approach,  but, if
we do this, then my question is: "you want execute multiple queries
parallel, then, How did you combine the out puts if you want to use the out
put of one in another query (sub - queries)".

Also, just " starting many servers for a problem" due you is it is a good
solution? Shah, my advice is, try to look for alternatives which can solve
your problem and which can execute the queries paralley, for that, i can
give is: ozzie framework. Look into it, it is good. Look at this simple
hive query execution using the ozzie:
http://stackoverflow.com/questions/10499526/simple-oozie-example-of-hive-query

Kumar

On Fri, May 11, 2012 at 5:23 PM, Bhavesh Shah <bh...@gmail.com>wrote:

> Hello all,
> I am asking you about the increasing the performance of Hive. I tried with
> mappers and reducers but I didn't see difference in execution.
> Don't know why, may be I did in some other way which may be not correct or
> due to some other reason.
>
> I am thinking that Is it possible to execute Hive queries parallelly?
> Means,
> Normally the queries get execute in queue manner.
> query1
> query2
> query3
> .
> .
> .
> n
>
> I am thinking that if we use mapreduce program  in Hive JDBC program, then
> is it possible to execute it parallelly.
> Don't know will it work or not? Thats I am asking you about it.
> But again my questions are:
> 1) If it is possible then may be it require multiple Hive Thrift Server?
> 2) Is it possible to open multiple Hive Thrift Server?
> 3) I think it is not possible to open multiple Hive Thrift Server on same
> port.?
> 4) Can we open multiple Hive Thrift Server on different different port?
>
> Please suggest me some solution to this. If you have other idea other than
> this then pls share with with me
> I will also try that.
> Thanks
>
>
> --
> Regards,
> Bhavesh Shah
>