Re: Maximum # of Tasks

[Enis Soztutar <en...@gmail.com>]

There is some obvious limitation in terms of memory usages in 
Jobtracker. Why would you want such enormous number of tasks?



Derek Gottfrid wrote:
> I tried to create a job that set the # of Tasks to 900k and it hung
> and then ended up killing it's self. The 900k should have been split
> over 300 machines but it never took. When I reduced the task # down to
> 200k it worked with out issue (over the same 300 machines). Are there
> any guidelines about maximum # of tasks ?
>
> thanks,
> derek
>
>   

Re: Maximum # of Tasks

[Derek Gottfrid <de...@codecubed.com>]

Don't think I ran it out of memory. Nothing in the logs and didn't go
about 1G of memory usage.  On the wiki it mentions of up to 300 tasks
per node,  hence I went with more tasks rather than fewer.

d

On 9/3/07, Enis Soztutar <en...@gmail.com> wrote:
> There is some obvious limitation in terms of memory usages in
> Jobtracker. Why would you want such enormous number of tasks?
>
>
>
> Derek Gottfrid wrote:
> > I tried to create a job that set the # of Tasks to 900k and it hung
> > and then ended up killing it's self. The 900k should have been split
> > over 300 machines but it never took. When I reduced the task # down to
> > 200k it worked with out issue (over the same 300 machines). Are there
> > any guidelines about maximum # of tasks ?
> >
> > thanks,
> > derek
> >
> >
>