You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-user@hadoop.apache.org by Toby DiPasquale <co...@gmail.com> on 2007/11/15 17:54:09 UTC

run multiple MR jobs concurrently?

Hi all,

Am I missing something here? I fire up 4 MR jobs on a cluster, giving
each job one reducer and 1/4 of the available mappers, and only one of
them runs at once. Is there something special I have to do to get them
to run concurrently? Thanks for any help.

-- 
Toby DiPasquale

Re: run multiple MR jobs concurrently?

Posted by Arun C Murthy <ar...@yahoo-inc.com>.

Toby,

On Thu, Nov 15, 2007 at 11:54:09AM -0500, Toby DiPasquale wrote:
>Hi all,
>
>Am I missing something here? I fire up 4 MR jobs on a cluster, giving
>each job one reducer and 1/4 of the available mappers, and only one of
>them runs at once. Is there something special I have to do to get them
>to run concurrently? Thanks for any help.
>

The JobTracker just picks a task of the highest priority job in it's queue and assigns a task from that job to a TaskTracker. 
Assuming all your jobs are of the same priority I'm not very surprised at that behaviour... 

Arun

>-- 
>Toby DiPasquale