Third-party Jars in MapReduce job

[Stu Hood <st...@webmail.us>]

Hello,

I'm working on a MapReduce job that requires some third-party jars. I know that the devs are working on [https://issues.apache.org/jira/browse/HADOOP-1622] HADOOP-1622, which should make this problem easy, but for now (and for backwards compatibility), how is this best accomplished?

I've tried adding the 3rd party jars to lib/ in my job-jar, so that the structure looks like so:
META-INF
us
    webmail
        hadoop
            mapred
                *
lib
    third-party-one.jar
    third-party-two.jar

... but no luck. The job fails saying that it can't find classes that are contained in the third party jars.

How are other users accomplishing this?

Stu Hood
Webmail.us
"You manage your business. We'll manage your email."®

Re: Third-party Jars in MapReduce job

[Dmitry <dm...@hotmail.com>]

Stu,
Are you using for devleopment Maven repository with Hadoop. I have the same 
issue as you are actully with 3-d party jars.
Thanks,
Dt
www.ejinz.com
Search engine Economy
----- Original Message ----- 
From: "Stu Hood" <st...@webmail.us>
To: <ha...@lucene.apache.org>
Sent: Monday, July 23, 2007 5:08 PM
Subject: Third-party Jars in MapReduce job




Hello,

I'm working on a MapReduce job that requires some third-party jars. I know 
that the devs are working on 
[https://issues.apache.org/jira/browse/HADOOP-1622] HADOOP-1622, which 
should make this problem easy, but for now (and for backwards 
compatibility), how is this best accomplished?

I've tried adding the 3rd party jars to lib/ in my job-jar, so that the 
structure looks like so:
META-INF
us
webmail
hadoop
mapred
*
lib
third-party-one.jar
third-party-two.jar

.. but no luck. The job fails saying that it can't find classes that are 
contained in the third party jars.

How are other users accomplishing this?

Stu Hood
Webmail.us
"You manage your business. We'll manage your email."®