You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Zhuo Liu <zz...@tigermail.auburn.edu> on 2013/05/20 04:24:09 UTC

How to execute parallel jobs inside a query

Hello, everyone

I configured the "hive.exec.parallel" variable to be "true" in the
hive-site.xml and hive-default.xml.
However, each job inside a query still runs sequentially one by one
(though they are supposed to be able to execute concurrently by considering the shape of DAG).
And I also find that the number of "Stages" is always equal to the number
of mapreduce "jobs" (for a query).

Could anyone offer some help?

Thanks,
Zhuo Liu
Ph.D. Student, CSSE department
Auburn University, AL 36849
http://www.auburn.edu/~zzl0014/<http://www.auburn.edu/%7Ezzl0014/>

How to execute parallel jobs inside a query

Posted by Zhuo Liu <zz...@tigermail.auburn.edu>.

Hello, everyone

I configured the "hive.exec.parallel" variable to be "true" in the
hive-site.xml and hive-default.xml.
However, each job inside a query still runs sequentially one by one
(though they are supposed to be able to execute concurrently by considering the shape of DAG).
And I also find that the number of "Stages" is always equal to the number
of mapreduce "jobs" (for a query).

Could anyone offer some help?

Thanks,
Zhuo Liu
Ph.D. Student, CSSE department
Auburn University, AL 36849
http://www.auburn.edu/~zzl0014/<http://www.auburn.edu/%7Ezzl0014/>

Re: How to execute parallel jobs inside a query

Posted by ๏̯͡๏ <ÐΞ€ρ@Ҝ>, de...@gmail.com.
Can you share your code


On Mon, May 20, 2013 at 7:54 AM, Zhuo Liu <zz...@tigermail.auburn.edu>wrote:

>  Hello, everyone
>
> I configured the "*hive.exec.parallel*" variable to be "*true*" in the
> hive-site.xml and hive-default.xml.
> However, each job inside a query *still runs sequentially* one by one
> (though they are supposed to be able to execute concurrently by
> considering the shape of DAG).
> And I also find that the number of "Stages" is always equal to the number
> of mapreduce "jobs" (for a query).
>
> Could anyone offer some help?
>
> Thanks,
> Zhuo Liu
> Ph.D. Student, CSSE department
> Auburn University, AL 36849
> http://www.auburn.edu/~zzl0014/
>



-- 
Deepak