You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-user@hadoop.apache.org by Nitin Khandelwal <ni...@germinait.com> on 2011/04/01 12:03:44 UTC

Problem in Job/TasK scheduling

Hi,


I am right now stuck on the issue of division of tasks among slaves for a
job. Currently, as far as I know, hadoop does not allow us to fix/determine
in advance how many tasks of a job would run on each slave. I am trying to
design a system, where I want a slave to execute only one task of one job
type ,
however at the same time , be able to accept/execute task of jobs of other
types. Is this kind of scheduling possible in hadoop.

Thanks in advance


-- 


Nitin Khandelwal

Re: Problem in Job/TasK scheduling

Posted by Allen Wittenauer <aw...@apache.org>.

On Apr 1, 2011, at 3:03 AM, Nitin Khandelwal wrote:

> Hi,
> 
> 
> I am right now stuck on the issue of division of tasks among slaves for a
> job. Currently, as far as I know, hadoop does not allow us to fix/determine
> in advance how many tasks of a job would run on each slave. I am trying to
> design a system, where I want a slave to execute only one task of one job
> type ,
> however at the same time , be able to accept/execute task of jobs of other
> types. Is this kind of scheduling possible in hadoop.

	This is in the FAQ:

http://wiki.apache.org/hadoop/FAQ#How_do_I_limit_.28or_increase.29_the_number_of_concurrent_tasks_running_on_a_node.3F

Re: Problem in Job/TasK scheduling

Posted by Harsh J <qw...@gmail.com>.

Perhaps you can have a look at the available Job schedulers - Capacity
Scheduler and Fair Scheduler? I think they ought to fit into your
requirements pretty well.

On Fri, Apr 1, 2011 at 3:33 PM, Nitin Khandelwal
<ni...@germinait.com> wrote:
> Hi,
>
>
> I am right now stuck on the issue of division of tasks among slaves for a
> job. Currently, as far as I know, hadoop does not allow us to fix/determine
> in advance how many tasks of a job would run on each slave. I am trying to
> design a system, where I want a slave to execute only one task of one job
> type ,
> however at the same time , be able to accept/execute task of jobs of other
> types. Is this kind of scheduling possible in hadoop.
>
> Thanks in advance
>
>
> --
>
>
> Nitin Khandelwal
>



-- 
Harsh J
http://harshj.com