You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-user@hadoop.apache.org by Adam Shook <as...@clearedgeit.com> on 2011/10/25 19:16:30 UTC

Questions about JVM Reuse

Hello All,

I have a few questions concerning the TaskTracker's JVM re-use that I couldn't unearth some details about:

Is the configured amount of tasks for reuse a suggestion or will it actually use it?  For example, if I've configured it to use a JVM for 4 tasks, will a TaskTracker that has 8 tasks to process use 2 JVMs?  Or does it decide if it actually wants to reuse one up to the maximum configured number?

Will a TaskTracker use the same JVM for tasks from different jobs?  Related to this, will a TaskTracker idle a JVM until it has enough tasks to fit the maximum JVM reuse variable?

Thanks,
Adam

Re: Questions about JVM Reuse

Posted by Joey Echeverria <jo...@cloudera.com>.

> Is the configured amount of tasks for reuse a suggestion or will it actually
> use it?  For example, if I’ve configured it to use a JVM for 4 tasks, will a
> TaskTracker that has 8 tasks to process use 2 JVMs?  Or does it decide if it
> actually wants to reuse one up to the maximum configured number?

It will launch mapred.tasktracker.map.tasks.maximum +
mapred.tasktracker.reduce.tasks.maximum JVMs. If a job has more than
that many tasks for that task tracker, it will then reuse those.

> Will a TaskTracker use the same JVM for tasks from different jobs?  Related
> to this, will a TaskTracker idle a JVM until it has enough tasks to fit the
> maximum JVM reuse variable?

It will not reuse JVMs across jobs. When the job finishes, all task
JVMs will be shutdown.


-Joey

-- 
Joseph Echeverria
Cloudera, Inc.
443.305.9434