You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-user@hadoop.apache.org by Kim Chew <kc...@gmail.com> on 2014/02/05 03:06:25 UTC

Adding jars using DistributedCache API.

Hello there,

I know I can do it with "-libjars" but I want to play around with the
DCache API. First I copy my jar file to hdfs, therefore,

        /user/kim/lib/foo.jar

And my M/R program references a class (Say, Foo) in foo.jar. In my driver,
I use the DCache API,

      DistributedCache.addFileToClassPath(new Path("/user/kim/lib/foo.jar",
conf, fs);

But I get a CNFE when I run the M/R job

 14/02/04 17:52:48 INFO mapred.JobClient: Task Id :
attempt_201401071457_8950_m_000000_0, Status : FAILED
Error: java.lang.ClassNotFoundException: com.myproject.Foo
    at java.net.URLClassLoader.findClass(URLClassLoader.java:434)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:660)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:358)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:626)

I have tried "addArchiveToClassPath()" but still getting the CNFE.
Any pointers? Thanks.

Kim