You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@giraph.apache.org by KAUSHIK SARKAR <co...@gmail.com> on 2012/07/24 03:32:31 UTC
SimpleShortestPathsVertex does not work when compiled separately
Hi,
I created a new test project and copied the code from
SimpleShortestPathsVertex.java file (org.apache.giraph.examples) into it.
But when I run it, it gives me java.lang.ClassNotFoundException. However,
if I run the same program from the giraph-0.1-jar-with-dependencies.jar,
it runs just fine. Following is the relevant information :
Hadoop version: hadoop-1.0.3 (running in pseudo distributed mode)
giraph version: giraph-0.1
jar created with Eclipse 3.3.1
Output:
kaushik@kaushik-laptop:~/hadoop/hadoop-1.0.3$ bin/hadoop jar
../jars/test.jar /shortestPathsInputGraph /shortestPathsOutputGraph 0 3
12/07/23 18:29:10 INFO mapred.JobClient: Running job: job_201207231222_0021
12/07/23 18:29:11 INFO mapred.JobClient: map 0% reduce 0%
12/07/23 18:29:20 INFO mapred.JobClient: Task Id :
attempt_201207231222_0021_m_000005_0, Status : FAILED
java.lang.RuntimeException: java.lang.RuntimeException:
java.lang.ClassNotFoundException:
test.giraph.SimpleShortestPathsVertex$SimpleShortestPathsVertexOutputFormat
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:899)
at
org.apache.giraph.graph.BspUtils.getVertexOutputFormatClass(BspUtils.java:134)
at
org.apache.giraph.bsp.BspOutputFormat.getOutputCommitter(BspOutputFormat.java:56)
at org.apache.hadoop.mapred.Task.initialize(Task.java:515)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:353)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
test.giraph.SimpleShortestPathsVertex$SimpleShortestPathsVertexOutputFormat
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:867)
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:891)
... 9 more
Caused by: java.lang.ClassNotFoundException:
test.giraph.SimpleShortestPathsVertex$SimpleShortestPathsVertexOutputFormat
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:820)
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:865)
... 10 more
I have checked inside the test/giraph folder of the test.jar file. It
contains the class
SimpleShortestPathsVertex$SimpleShortestPathsVertexOutputFormat.
Any help will be appreciated.
Regards,
Kaushik
Re: SimpleShortestPathsVertex does not work when compiled separately
Posted by Vijayakumar Ramdoss <ne...@me.com>.
Thanks Kaushik, it's worked me well.
Sent from my iPad
On Aug 1, 2012, at 11:20 PM, KAUSHIK SARKAR <co...@gmail.com> wrote:
> Hi,
>
> I found an alternative programmatic solution to the problem.
> We need to modify the run() method in the following way -
>
> ...
> @Override
> public int run(String[] argArray) throws Exception {
> Preconditions.checkArgument(argArray.length == 4,
> "run: Must have 4 arguments <input path> <output path> " +
> "<source vertex id> <# of workers>");
>
> GiraphJob job = new GiraphJob(getConf(), getClass().getName());
> // This is the addition - it will make hadoop look for other classes in the same jar that contains this class
> job.getInternalJob().setJarByClass(getClass());
> job.setVertexClass(getClass());
> ...
> }
>
> After this hadoop will look for missing classes in the same jar that contains the class returned by getClass(), and we will not need to add the job jar name separately to the -libjars option (as described by Marcin's stackoverflow post).
>
> I think Giraph may add a constructor like -
>
> GiraphJob(Configuration conf, Class jobClass, String jobName) {
> this(conf, jobName);
> this.getInternalJob().setJarByClass(jobClass);
> }
>
> in tradition of the Hadoop constructor -
>
> public JobConf(Configuration conf, Class exampleClass) {
> this(conf);
> setJarByClass(exampleClass);
> }
>
> It will solve the rather annoying class-not-found problem.
>
> Regards,
> Kaushik
>
> On Tue, Jul 24, 2012 at 11:31 AM, KAUSHIK SARKAR <co...@gmail.com> wrote:
> Hi,
>
> Marcin's solution worked for me. Thanks!
>
> I didn't try out addFileToClassPath solution, but I think it should work. Thanks for the suggestion!
>
> As Marcin has already suggested, it is not very much clear why we have to do it like this and why the normal call to hadoop jar doesn't work. I guess it has more to do with the way Java (or Hadoop) loads class files than Giraph. However, I am a total novice in Java and would like to know the reason.
>
> N.B. I tried with the latest development snapshot of the Giraph (giraph-0.2) , and normal call to hadoop jar didn't work with it as well (of course Marcin's solution worked).
>
> Regards,
> Kaushik
>
> On Tue, Jul 24, 2012 at 6:22 AM, Etienne Dumoulin <et...@idiro.com> wrote:
> Hi,
>
> I cannot explain it.
> But if it any use, for making it working without specifying the library in command line you can use the distributed cache:
> DistributedCache#addFileToClassPath
>
> Regards,
>
> Étienne
>
>
> On 24 July 2012 13:37, Marcin Biczak <ma...@gmail.com> wrote:
> Hi
>
> I had similar problem, here is "my solution", http://stackoverflow.com/questions/10700853/giraph-shortest-paths-example-classnotfoundexception. But I still don't know why I have do it like this, maybe someone here can explain this?
>
> regards
> marcin biczak
>
>
> 2012/7/24 KAUSHIK SARKAR <co...@gmail.com>
> Hi,
>
> I created a new test project and copied the code from SimpleShortestPathsVertex.java file (org.apache.giraph.examples) into it. But when I run it, it gives me java.lang.ClassNotFoundException. However, if I run the same program from the giraph-0.1-jar-with-dependencies.jar, it runs just fine. Following is the relevant information :
>
> Hadoop version: hadoop-1.0.3 (running in pseudo distributed mode)
> giraph version: giraph-0.1
> jar created with Eclipse 3.3.1
> Output:
>
> kaushik@kaushik-laptop:~/hadoop/hadoop-1.0.3$ bin/hadoop jar ../jars/test.jar /shortestPathsInputGraph /shortestPathsOutputGraph 0 3
> 12/07/23 18:29:10 INFO mapred.JobClient: Running job: job_201207231222_0021
> 12/07/23 18:29:11 INFO mapred.JobClient: map 0% reduce 0%
> 12/07/23 18:29:20 INFO mapred.JobClient: Task Id : attempt_201207231222_0021_m_000005_0, Status : FAILED
> java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: test.giraph.SimpleShortestPathsVertex$SimpleShortestPathsVertexOutputFormat
> at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:899)
> at org.apache.giraph.graph.BspUtils.getVertexOutputFormatClass(BspUtils.java:134)
> at org.apache.giraph.bsp.BspOutputFormat.getOutputCommitter(BspOutputFormat.java:56)
> at org.apache.hadoop.mapred.Task.initialize(Task.java:515)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:353)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
> at org.apache.hadoop.mapred.Child.main(Child.java:249)
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: test.giraph.SimpleShortestPathsVertex$SimpleShortestPathsVertexOutputFormat
> at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:867)
> at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:891)
> ... 9 more
> Caused by: java.lang.ClassNotFoundException: test.giraph.SimpleShortestPathsVertex$SimpleShortestPathsVertexOutputFormat
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:247)
> at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:820)
> at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:865)
> ... 10 more
>
>
>
> I have checked inside the test/giraph folder of the test.jar file. It contains the class SimpleShortestPathsVertex$SimpleShortestPathsVertexOutputFormat.
> Any help will be appreciated.
>
> Regards,
> Kaushik
>
>
>
> --
> martin
>
>
>
Re: SimpleShortestPathsVertex does not work when compiled separately
Posted by KAUSHIK SARKAR <co...@gmail.com>.
Hi,
I found an alternative programmatic solution to the problem.
We need to modify the run() method in the following way -
...
@Override
public int run(String[] argArray) throws Exception {
Preconditions.checkArgument(argArray.length == 4,
"run: Must have 4 arguments <input path> <output path> " +
"<source vertex id> <# of workers>");
GiraphJob job = new GiraphJob(getConf(), getClass().getName());
// This is the addition - it will make hadoop look for other
classes in the same jar that contains this class
*job.getInternalJob().setJarByClass(getClass());*
job.setVertexClass(getClass());
...
}
After this hadoop will look for missing classes in the same jar that
contains the class returned by getClass(), and we will not need to add the
job jar name separately to the -libjars option (as described by Marcin's
stackoverflow post).
I think Giraph may add a constructor like -
GiraphJob(Configuration conf, Class jobClass, String jobName) {
this(conf, jobName);
this.getInternalJob().setJarByClass(jobClass);
}
in tradition of the Hadoop constructor -
public JobConf(Configuration conf, Class exampleClass) {
this(conf);
setJarByClass(exampleClass);
}
It will solve the rather annoying class-not-found problem.
Regards,
Kaushik
On Tue, Jul 24, 2012 at 11:31 AM, KAUSHIK SARKAR <co...@gmail.com>wrote:
> Hi,
>
> Marcin's solution worked for me. Thanks!
>
> I didn't try out addFileToClassPath solution, but I think it should work.
> Thanks for the suggestion!
>
> As Marcin has already suggested, it is not very much clear why we have to
> do it like this and why the normal call to hadoop jar doesn't work. I guess
> it has more to do with the way Java (or Hadoop) loads class files than
> Giraph. However, I am a total novice in Java and would like to know the
> reason.
>
> N.B. I tried with the latest development snapshot of the Giraph
> (giraph-0.2) , and normal call to hadoop jar didn't work with it as well
> (of course Marcin's solution worked).
>
> Regards,
> Kaushik
>
> On Tue, Jul 24, 2012 at 6:22 AM, Etienne Dumoulin <
> etienne.dumoulin@idiro.com> wrote:
>
>> Hi,
>>
>> I cannot explain it.
>> But if it any use, for making it working without specifying the library
>> in command line you can use the distributed cache:
>> DistributedCache#addFileToClassPath
>>
>> Regards,
>>
>> Étienne
>>
>>
>> On 24 July 2012 13:37, Marcin Biczak <ma...@gmail.com> wrote:
>>
>>> Hi
>>>
>>> I had similar problem, here is "my solution",
>>> http://stackoverflow.com/questions/10700853/giraph-shortest-paths-example-classnotfoundexception.
>>> But I still don't know why I have do it like this, maybe someone here can
>>> explain this?
>>>
>>> regards
>>> marcin biczak
>>>
>>>
>>> 2012/7/24 KAUSHIK SARKAR <co...@gmail.com>
>>>
>>>> Hi,
>>>>
>>>> I created a new test project and copied the code from
>>>> SimpleShortestPathsVertex.java file (org.apache.giraph.examples) into it.
>>>> But when I run it, it gives me java.lang.ClassNotFoundException. However,
>>>> if I run the same program from the giraph-0.1-jar-with-dependencies.jar,
>>>> it runs just fine. Following is the relevant information :
>>>>
>>>> Hadoop version: hadoop-1.0.3 (running in pseudo distributed mode)
>>>> giraph version: giraph-0.1
>>>> jar created with Eclipse 3.3.1
>>>> Output:
>>>>
>>>> kaushik@kaushik-laptop:~/hadoop/hadoop-1.0.3$ bin/hadoop jar
>>>> ../jars/test.jar /shortestPathsInputGraph /shortestPathsOutputGraph 0 3
>>>> 12/07/23 18:29:10 INFO mapred.JobClient: Running job:
>>>> job_201207231222_0021
>>>> 12/07/23 18:29:11 INFO mapred.JobClient: map 0% reduce 0%
>>>> 12/07/23 18:29:20 INFO mapred.JobClient: Task Id :
>>>> attempt_201207231222_0021_m_000005_0, Status : FAILED
>>>> java.lang.RuntimeException: java.lang.RuntimeException:
>>>> java.lang.ClassNotFoundException:
>>>> test.giraph.SimpleShortestPathsVertex$SimpleShortestPathsVertexOutputFormat
>>>> at
>>>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:899)
>>>> at
>>>> org.apache.giraph.graph.BspUtils.getVertexOutputFormatClass(BspUtils.java:134)
>>>> at
>>>> org.apache.giraph.bsp.BspOutputFormat.getOutputCommitter(BspOutputFormat.java:56)
>>>> at org.apache.hadoop.mapred.Task.initialize(Task.java:515)
>>>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:353)
>>>> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
>>>> at java.security.AccessController.doPrivileged(Native Method)
>>>> at javax.security.auth.Subject.doAs(Subject.java:396)
>>>> at
>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
>>>> at org.apache.hadoop.mapred.Child.main(Child.java:249)
>>>> Caused by: java.lang.RuntimeException:
>>>> java.lang.ClassNotFoundException:
>>>> test.giraph.SimpleShortestPathsVertex$SimpleShortestPathsVertexOutputFormat
>>>> at
>>>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:867)
>>>> at
>>>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:891)
>>>> ... 9 more
>>>> Caused by: java.lang.ClassNotFoundException:
>>>> test.giraph.SimpleShortestPathsVertex$SimpleShortestPathsVertexOutputFormat
>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>>>> at java.security.AccessController.doPrivileged(Native Method)
>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>>>> at java.lang.Class.forName0(Native Method)
>>>> at java.lang.Class.forName(Class.java:247)
>>>> at
>>>> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:820)
>>>> at
>>>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:865)
>>>> ... 10 more
>>>>
>>>>
>>>>
>>>> I have checked inside the test/giraph folder of the test.jar file. It
>>>> contains the class
>>>> SimpleShortestPathsVertex$SimpleShortestPathsVertexOutputFormat.
>>>> Any help will be appreciated.
>>>>
>>>> Regards,
>>>> Kaushik
>>>>
>>>
>>>
>>>
>>> --
>>> martin
>>>
>>
>>
>
Re: SimpleShortestPathsVertex does not work when compiled separately
Posted by KAUSHIK SARKAR <co...@gmail.com>.
Hi,
Marcin's solution worked for me. Thanks!
I didn't try out addFileToClassPath solution, but I think it should work.
Thanks for the suggestion!
As Marcin has already suggested, it is not very much clear why we have to
do it like this and why the normal call to hadoop jar doesn't work. I guess
it has more to do with the way Java (or Hadoop) loads class files than
Giraph. However, I am a total novice in Java and would like to know the
reason.
N.B. I tried with the latest development snapshot of the Giraph
(giraph-0.2) , and normal call to hadoop jar didn't work with it as well
(of course Marcin's solution worked).
Regards,
Kaushik
On Tue, Jul 24, 2012 at 6:22 AM, Etienne Dumoulin <
etienne.dumoulin@idiro.com> wrote:
> Hi,
>
> I cannot explain it.
> But if it any use, for making it working without specifying the library in
> command line you can use the distributed cache:
> DistributedCache#addFileToClassPath
>
> Regards,
>
> Étienne
>
>
> On 24 July 2012 13:37, Marcin Biczak <ma...@gmail.com> wrote:
>
>> Hi
>>
>> I had similar problem, here is "my solution",
>> http://stackoverflow.com/questions/10700853/giraph-shortest-paths-example-classnotfoundexception.
>> But I still don't know why I have do it like this, maybe someone here can
>> explain this?
>>
>> regards
>> marcin biczak
>>
>>
>> 2012/7/24 KAUSHIK SARKAR <co...@gmail.com>
>>
>>> Hi,
>>>
>>> I created a new test project and copied the code from
>>> SimpleShortestPathsVertex.java file (org.apache.giraph.examples) into it.
>>> But when I run it, it gives me java.lang.ClassNotFoundException. However,
>>> if I run the same program from the giraph-0.1-jar-with-dependencies.jar,
>>> it runs just fine. Following is the relevant information :
>>>
>>> Hadoop version: hadoop-1.0.3 (running in pseudo distributed mode)
>>> giraph version: giraph-0.1
>>> jar created with Eclipse 3.3.1
>>> Output:
>>>
>>> kaushik@kaushik-laptop:~/hadoop/hadoop-1.0.3$ bin/hadoop jar
>>> ../jars/test.jar /shortestPathsInputGraph /shortestPathsOutputGraph 0 3
>>> 12/07/23 18:29:10 INFO mapred.JobClient: Running job:
>>> job_201207231222_0021
>>> 12/07/23 18:29:11 INFO mapred.JobClient: map 0% reduce 0%
>>> 12/07/23 18:29:20 INFO mapred.JobClient: Task Id :
>>> attempt_201207231222_0021_m_000005_0, Status : FAILED
>>> java.lang.RuntimeException: java.lang.RuntimeException:
>>> java.lang.ClassNotFoundException:
>>> test.giraph.SimpleShortestPathsVertex$SimpleShortestPathsVertexOutputFormat
>>> at
>>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:899)
>>> at
>>> org.apache.giraph.graph.BspUtils.getVertexOutputFormatClass(BspUtils.java:134)
>>> at
>>> org.apache.giraph.bsp.BspOutputFormat.getOutputCommitter(BspOutputFormat.java:56)
>>> at org.apache.hadoop.mapred.Task.initialize(Task.java:515)
>>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:353)
>>> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
>>> at java.security.AccessController.doPrivileged(Native Method)
>>> at javax.security.auth.Subject.doAs(Subject.java:396)
>>> at
>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
>>> at org.apache.hadoop.mapred.Child.main(Child.java:249)
>>> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
>>> test.giraph.SimpleShortestPathsVertex$SimpleShortestPathsVertexOutputFormat
>>> at
>>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:867)
>>> at
>>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:891)
>>> ... 9 more
>>> Caused by: java.lang.ClassNotFoundException:
>>> test.giraph.SimpleShortestPathsVertex$SimpleShortestPathsVertexOutputFormat
>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>>> at java.security.AccessController.doPrivileged(Native Method)
>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>>> at java.lang.Class.forName0(Native Method)
>>> at java.lang.Class.forName(Class.java:247)
>>> at
>>> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:820)
>>> at
>>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:865)
>>> ... 10 more
>>>
>>>
>>>
>>> I have checked inside the test/giraph folder of the test.jar file. It
>>> contains the class
>>> SimpleShortestPathsVertex$SimpleShortestPathsVertexOutputFormat.
>>> Any help will be appreciated.
>>>
>>> Regards,
>>> Kaushik
>>>
>>
>>
>>
>> --
>> martin
>>
>
>
Re: SimpleShortestPathsVertex does not work when compiled separately
Posted by Etienne Dumoulin <et...@idiro.com>.
Hi,
I cannot explain it.
But if it any use, for making it working without specifying the library in
command line you can use the distributed cache:
DistributedCache#addFileToClassPath
Regards,
Étienne
On 24 July 2012 13:37, Marcin Biczak <ma...@gmail.com> wrote:
> Hi
>
> I had similar problem, here is "my solution",
> http://stackoverflow.com/questions/10700853/giraph-shortest-paths-example-classnotfoundexception.
> But I still don't know why I have do it like this, maybe someone here can
> explain this?
>
> regards
> marcin biczak
>
>
> 2012/7/24 KAUSHIK SARKAR <co...@gmail.com>
>
>> Hi,
>>
>> I created a new test project and copied the code from
>> SimpleShortestPathsVertex.java file (org.apache.giraph.examples) into it.
>> But when I run it, it gives me java.lang.ClassNotFoundException. However,
>> if I run the same program from the giraph-0.1-jar-with-dependencies.jar,
>> it runs just fine. Following is the relevant information :
>>
>> Hadoop version: hadoop-1.0.3 (running in pseudo distributed mode)
>> giraph version: giraph-0.1
>> jar created with Eclipse 3.3.1
>> Output:
>>
>> kaushik@kaushik-laptop:~/hadoop/hadoop-1.0.3$ bin/hadoop jar
>> ../jars/test.jar /shortestPathsInputGraph /shortestPathsOutputGraph 0 3
>> 12/07/23 18:29:10 INFO mapred.JobClient: Running job:
>> job_201207231222_0021
>> 12/07/23 18:29:11 INFO mapred.JobClient: map 0% reduce 0%
>> 12/07/23 18:29:20 INFO mapred.JobClient: Task Id :
>> attempt_201207231222_0021_m_000005_0, Status : FAILED
>> java.lang.RuntimeException: java.lang.RuntimeException:
>> java.lang.ClassNotFoundException:
>> test.giraph.SimpleShortestPathsVertex$SimpleShortestPathsVertexOutputFormat
>> at
>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:899)
>> at
>> org.apache.giraph.graph.BspUtils.getVertexOutputFormatClass(BspUtils.java:134)
>> at
>> org.apache.giraph.bsp.BspOutputFormat.getOutputCommitter(BspOutputFormat.java:56)
>> at org.apache.hadoop.mapred.Task.initialize(Task.java:515)
>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:353)
>> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Subject.java:396)
>> at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
>> at org.apache.hadoop.mapred.Child.main(Child.java:249)
>> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
>> test.giraph.SimpleShortestPathsVertex$SimpleShortestPathsVertexOutputFormat
>> at
>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:867)
>> at
>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:891)
>> ... 9 more
>> Caused by: java.lang.ClassNotFoundException:
>> test.giraph.SimpleShortestPathsVertex$SimpleShortestPathsVertexOutputFormat
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>> at java.lang.Class.forName0(Native Method)
>> at java.lang.Class.forName(Class.java:247)
>> at
>> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:820)
>> at
>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:865)
>> ... 10 more
>>
>>
>>
>> I have checked inside the test/giraph folder of the test.jar file. It
>> contains the class
>> SimpleShortestPathsVertex$SimpleShortestPathsVertexOutputFormat.
>> Any help will be appreciated.
>>
>> Regards,
>> Kaushik
>>
>
>
>
> --
> martin
>
Re: SimpleShortestPathsVertex does not work when compiled separately
Posted by Marcin Biczak <ma...@gmail.com>.
Hi
I had similar problem, here is "my solution",
http://stackoverflow.com/questions/10700853/giraph-shortest-paths-example-classnotfoundexception.
But I still don't know why I have do it like this, maybe someone here can
explain this?
regards
marcin biczak
2012/7/24 KAUSHIK SARKAR <co...@gmail.com>
> Hi,
>
> I created a new test project and copied the code from
> SimpleShortestPathsVertex.java file (org.apache.giraph.examples) into it.
> But when I run it, it gives me java.lang.ClassNotFoundException. However,
> if I run the same program from the giraph-0.1-jar-with-dependencies.jar,
> it runs just fine. Following is the relevant information :
>
> Hadoop version: hadoop-1.0.3 (running in pseudo distributed mode)
> giraph version: giraph-0.1
> jar created with Eclipse 3.3.1
> Output:
>
> kaushik@kaushik-laptop:~/hadoop/hadoop-1.0.3$ bin/hadoop jar
> ../jars/test.jar /shortestPathsInputGraph /shortestPathsOutputGraph 0 3
> 12/07/23 18:29:10 INFO mapred.JobClient: Running job: job_201207231222_0021
> 12/07/23 18:29:11 INFO mapred.JobClient: map 0% reduce 0%
> 12/07/23 18:29:20 INFO mapred.JobClient: Task Id :
> attempt_201207231222_0021_m_000005_0, Status : FAILED
> java.lang.RuntimeException: java.lang.RuntimeException:
> java.lang.ClassNotFoundException:
> test.giraph.SimpleShortestPathsVertex$SimpleShortestPathsVertexOutputFormat
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:899)
> at
> org.apache.giraph.graph.BspUtils.getVertexOutputFormatClass(BspUtils.java:134)
> at
> org.apache.giraph.bsp.BspOutputFormat.getOutputCommitter(BspOutputFormat.java:56)
> at org.apache.hadoop.mapred.Task.initialize(Task.java:515)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:353)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
> at org.apache.hadoop.mapred.Child.main(Child.java:249)
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
> test.giraph.SimpleShortestPathsVertex$SimpleShortestPathsVertexOutputFormat
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:867)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:891)
> ... 9 more
> Caused by: java.lang.ClassNotFoundException:
> test.giraph.SimpleShortestPathsVertex$SimpleShortestPathsVertexOutputFormat
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:247)
> at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:820)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:865)
> ... 10 more
>
>
>
> I have checked inside the test/giraph folder of the test.jar file. It
> contains the class
> SimpleShortestPathsVertex$SimpleShortestPathsVertexOutputFormat.
> Any help will be appreciated.
>
> Regards,
> Kaushik
>
--
martin