You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Mark Kerzner <ma...@gmail.com> on 2011/02/25 00:23:34 UTC
setJarByClass question
Hi, this call,
job.setJarByClass
tells Hadoop which jar to use. But we also tell Hadoop which jar to use on
the command line,
hadoop jar your-jar parameters
Why do we need this in both places?
Thank you,
Mark
Re: setJarByClass question
Posted by Stanley Xu <we...@gmail.com>.
The jar in the command line might only be the jar to submit the map-reduce
job, rather than the jar contains the Mapper and Reducer which will be
transferred to different node.
What the hadoop jar your-jar really did, is setting the classpath and other
related environment, and run the main method in your-jar. You might have a
different map-reduce-jar in the classpath which contains the real mapper and
reducer used to do the job.
Best wishes,
Stanley Xu
On Fri, Feb 25, 2011 at 7:23 AM, Mark Kerzner <ma...@gmail.com> wrote:
> Hi, this call,
>
> job.setJarByClass
>
> tells Hadoop which jar to use. But we also tell Hadoop which jar to use on
> the command line,
>
> hadoop jar your-jar parameters
>
> Why do we need this in both places?
>
> Thank you,
> Mark
>