You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Saptarshi Guha <sa...@gmail.com> on 2010/07/09 13:38:42 UTC
Task JVM reuse: A Question regarding hadoop 0.20.1
Hello,
I have set mapred.job.reuse.jvm.num.tasks to -1 for re-using the JVM.
My intention is to run a helper program at the beginning of the job and then
feed the key/value pairs
from the tasks to the helper program.
Currently am running it in the call to setup below.
If JVM Task re-use is -1, will the Mapper class containing the run method
below be created once
and Hadoop will call the objects run method as many times as there tasks
assigned to that JVM?
Thank you
Saptarshi
public void run(Context context) throws IOException,
InterruptedException {
setup(context);
while (context.nextKeyValue()) {
map(context.getCurrentKey(), context.getCurrentValue(), context);
}
cleanup(context);
}