You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Paolo Castagna <ca...@googlemail.com> on 2011/06/28 18:42:31 UTC
Sequence of MapReduce jobs... is it possible to upload the jar only
once?
Hi,
I have a sequence of three MapReduce jobs which I run this way:
FirstDriver first = new FirstDriver(configuration);
first.run(new String[] { args[0], args[1] + "_1" });
SecondDriver second = new SecondDriver(configuration);
second.run(new String[] { args[1] + "_1", args[1] + "_2" });
ThirdDriver third = new ThirdDriver(configuration);
third.run(new String[] { args[1] + "_2", args[1] + "_3" });
Each {First|Second|Third}Driver extends Configured and implements Tool.
My jar has quite a few dependencies, is there a way to upload to HDFS
the jar only once?
Thank you,
Paolo