You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@giraph.apache.org by 김원배 <zw...@gmail.com> on 2013/06/24 10:33:58 UTC

problem with running shortestpath example.

Hi, I'm beginner on giraph.
I have a problem with running shortestpath example.

I'm using giraph-1.0.0
hadoop 0.20.203.0
maven 3.1.0-alpha-1
java 1.6.0_22


when I command like this,


hadoop jar \
/home/wb/giraph-1.0.0/giraph-core/target/giraph-1.0.0-for-hadoop-0.20.203.0-jar-with-dependencies.jar
\
org.apache.giraph.GiraphRunner \
org.apache.giraph.examples.SimpleShortestPathsVertex \
-vif
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat \
-vip ./in/input \
-of org.apache.giraph.io.formats.IdWithValueTextOutputFormat \
-op ./outShortest \
-w 1


I get this result


Exception in thread "main" java.lang.ClassNotFoundException:
org.apache.giraph.examples.SimpleShortestPathsVertex
        at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:186)
        at
org.apache.giraph.utils.ConfigurationUtils.populateGiraphConfiguration(ConfigurationUtils.java:210)
        at
org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:147)
        at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
        at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:616)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)




and when I command like this,


hadoop jar \
/home/wb/giraph-1.0.0/giraph-examples/target/giraph-examples-1.0.0-for-hadoop-0.20.203.0-jar-with-dependencies.jar
\
org.apache.giraph.GiraphRunner \
org.apache.giraph.examples.SimpleShortestPathsVertex \
-vif
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat \
-vip /in/input \
-of org.apache.giraph.io.formats.IdWithValueTextOutputFormat \
-op /outShortest \
-w 1


I get this result.


13/06/24 16:40:54 INFO utils.ConfigurationUtils: No edge input format
specified. Ensure your InputFormat does not require one.
13/06/24 16:40:54 WARN job.GiraphConfigurationValidator: Output format
vertex index type is not known
13/06/24 16:40:54 WARN job.GiraphConfigurationValidator: Output format
vertex value type is not known
13/06/24 16:40:54 WARN job.GiraphConfigurationValidator: Output format edge
value type is not known
13/06/24 16:40:54 INFO job.GiraphJob: run: Since checkpointing is disabled
(default), do not allow any task retries (setting mapred.map.max.attempts =
0, old value = 4)
13/06/24 16:40:56 INFO mapred.JobClient: Running job: job_201306241432_0005
13/06/24 16:40:57 INFO mapred.JobClient:  map 0% reduce 0%
13/06/24 16:41:18 INFO mapred.JobClient:  map 50% reduce 0%
13/06/24 16:41:21 INFO mapred.JobClient:  map 100% reduce 0%
13/06/24 16:51:29 INFO mapred.JobClient:  map 50% reduce 0%
13/06/24 16:51:37 INFO mapred.JobClient: Job complete: job_201306241432_0005
13/06/24 16:51:37 INFO mapred.JobClient: Counters: 6
13/06/24 16:51:37 INFO mapred.JobClient:   Job Counters
13/06/24 16:51:37 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=1242845
13/06/24 16:51:37 INFO mapred.JobClient:     Total time spent by all
reduces waiting after reserving slots (ms)=0
13/06/24 16:51:37 INFO mapred.JobClient:     Total time spent by all maps
waiting after reserving slots (ms)=0
13/06/24 16:51:37 INFO mapred.JobClient:     Launched map tasks=2
13/06/24 16:51:37 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=0
13/06/24 16:51:37 INFO mapred.JobClient:     Failed map tasks=1


I cannot find out what's the probelm. Thanks.

Re: problem with running shortestpath example.

Posted by Ahmet Emre Aladağ <em...@agmlab.com>.

Hi,

You have to specify EdgeInputFormat.

For another class of mine I use the following command (wrting for just 
reference):

hadoop jar 
giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-1.0.2-jar-with-dependencies.jar 
org.apache.giraph.GiraphRunner 
org.apache.giraph.examples.LinkRank.LinkRankVertex
-vif org.apache.giraph.examples.LinkRank.LinkRankVertexInputFormat
-eif org.apache.giraph.examples.LinkRank.LinkRankEdgeInputFormat
-of org.apache.giraph.examples.LinkRank.LinkRankVertexOutputFormat
-w 1
-vip nodes.txt
-eip edges.txt
-op linkOutput4

You can see the sample EdgeInputFormat on http://bit.ly/174RfHG.



On 06/24/2013 11:33 AM, 김원배 wrote:
> and when I command like this,
>
>
> hadoop jar \
> /home/wb/giraph-1.0.0/giraph-examples/target/giraph-examples-1.0.0-for-hadoop-0.20.203.0-jar-with-dependencies.jar 
> \
> org.apache.giraph.GiraphRunner \
> org.apache.giraph.examples.SimpleShortestPathsVertex \
> -vif 
> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat \
> -vip /in/input \
> -of org.apache.giraph.io.formats.IdWithValueTextOutputFormat \
> -op /outShortest \
> -w 1
>
>
> I get this result.
>
>
> 13/06/24 16:40:54 INFO utils.ConfigurationUtils: No edge input format 
> specified. Ensure your InputFormat does not require one.

Re: problem with running shortestpath example.

Posted by Christian Krause <me...@ckrause.org>.

I am also a Giraph beginner. The default addresses of the web interfaces of
your master nodes are localhost:50030 and localhost:50070 (assuming your
localhost is your master). Just open them in a browser...

Cheers,
Christian

Re: problem with running shortestpath example.

Posted by 김원배 <zw...@gmail.com>.

Thank you all. However, Christian, could you explain more specific for me?
Because I'm beginner :) please.


2013/6/24 Christian Krause <me...@ckrause.org>

> Hi,
> You have a failed map task. Take a look at the web interface of the Job
> Tracker. Open the logs of your job and look for exceptions.
>
> Cheers,
> Christian
>
>
> 2013/6/24 김원배 <zw...@gmail.com>
>
>> Hi, I'm beginner on giraph.
>> I have a problem with running shortestpath example.
>>
>> I'm using giraph-1.0.0
>> hadoop 0.20.203.0
>> maven 3.1.0-alpha-1
>> java 1.6.0_22
>>
>>
>> when I command like this,
>>
>>
>> hadoop jar \
>> /home/wb/giraph-1.0.0/giraph-core/target/giraph-1.0.0-for-hadoop-0.20.203.0-jar-with-dependencies.jar
>> \
>> org.apache.giraph.GiraphRunner \
>> org.apache.giraph.examples.SimpleShortestPathsVertex \
>> -vif
>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat \
>> -vip ./in/input \
>> -of org.apache.giraph.io.formats.IdWithValueTextOutputFormat \
>> -op ./outShortest \
>> -w 1
>>
>>
>> I get this result
>>
>>
>> Exception in thread "main" java.lang.ClassNotFoundException:
>> org.apache.giraph.examples.SimpleShortestPathsVertex
>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
>>         at java.security.AccessController.doPrivileged(Native Method)
>>         at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
>>         at java.lang.Class.forName0(Native Method)
>>         at java.lang.Class.forName(Class.java:186)
>>         at
>> org.apache.giraph.utils.ConfigurationUtils.populateGiraphConfiguration(ConfigurationUtils.java:210)
>>         at
>> org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:147)
>>         at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
>>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>>         at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>         at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>         at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>         at java.lang.reflect.Method.invoke(Method.java:616)
>>         at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>
>>
>>
>>
>> and when I command like this,
>>
>>
>> hadoop jar \
>> /home/wb/giraph-1.0.0/giraph-examples/target/giraph-examples-1.0.0-for-hadoop-0.20.203.0-jar-with-dependencies.jar
>> \
>> org.apache.giraph.GiraphRunner \
>> org.apache.giraph.examples.SimpleShortestPathsVertex \
>> -vif
>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat \
>> -vip /in/input \
>> -of org.apache.giraph.io.formats.IdWithValueTextOutputFormat \
>> -op /outShortest \
>> -w 1
>>
>>
>> I get this result.
>>
>>
>> 13/06/24 16:40:54 INFO utils.ConfigurationUtils: No edge input format
>> specified. Ensure your InputFormat does not require one.
>> 13/06/24 16:40:54 WARN job.GiraphConfigurationValidator: Output format
>> vertex index type is not known
>> 13/06/24 16:40:54 WARN job.GiraphConfigurationValidator: Output format
>> vertex value type is not known
>> 13/06/24 16:40:54 WARN job.GiraphConfigurationValidator: Output format
>> edge value type is not known
>> 13/06/24 16:40:54 INFO job.GiraphJob: run: Since checkpointing is
>> disabled (default), do not allow any task retries (setting
>> mapred.map.max.attempts = 0, old value = 4)
>> 13/06/24 16:40:56 INFO mapred.JobClient: Running job:
>> job_201306241432_0005
>> 13/06/24 16:40:57 INFO mapred.JobClient:  map 0% reduce 0%
>> 13/06/24 16:41:18 INFO mapred.JobClient:  map 50% reduce 0%
>> 13/06/24 16:41:21 INFO mapred.JobClient:  map 100% reduce 0%
>> 13/06/24 16:51:29 INFO mapred.JobClient:  map 50% reduce 0%
>> 13/06/24 16:51:37 INFO mapred.JobClient: Job complete:
>> job_201306241432_0005
>> 13/06/24 16:51:37 INFO mapred.JobClient: Counters: 6
>> 13/06/24 16:51:37 INFO mapred.JobClient:   Job Counters
>> 13/06/24 16:51:37 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=1242845
>> 13/06/24 16:51:37 INFO mapred.JobClient:     Total time spent by all
>> reduces waiting after reserving slots (ms)=0
>> 13/06/24 16:51:37 INFO mapred.JobClient:     Total time spent by all maps
>> waiting after reserving slots (ms)=0
>> 13/06/24 16:51:37 INFO mapred.JobClient:     Launched map tasks=2
>> 13/06/24 16:51:37 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=0
>> 13/06/24 16:51:37 INFO mapred.JobClient:     Failed map tasks=1
>>
>>
>> I cannot find out what's the probelm. Thanks.
>>
>>
>

Re: problem with running shortestpath example.

Posted by Christian Krause <me...@ckrause.org>.

Hi,
You have a failed map task. Take a look at the web interface of the Job
Tracker. Open the logs of your job and look for exceptions.

Cheers,
Christian


2013/6/24 김원배 <zw...@gmail.com>

> Hi, I'm beginner on giraph.
> I have a problem with running shortestpath example.
>
> I'm using giraph-1.0.0
> hadoop 0.20.203.0
> maven 3.1.0-alpha-1
> java 1.6.0_22
>
>
> when I command like this,
>
>
> hadoop jar \
> /home/wb/giraph-1.0.0/giraph-core/target/giraph-1.0.0-for-hadoop-0.20.203.0-jar-with-dependencies.jar
> \
> org.apache.giraph.GiraphRunner \
> org.apache.giraph.examples.SimpleShortestPathsVertex \
> -vif
> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat \
> -vip ./in/input \
> -of org.apache.giraph.io.formats.IdWithValueTextOutputFormat \
> -op ./outShortest \
> -w 1
>
>
> I get this result
>
>
> Exception in thread "main" java.lang.ClassNotFoundException:
> org.apache.giraph.examples.SimpleShortestPathsVertex
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
>         at java.lang.Class.forName0(Native Method)
>         at java.lang.Class.forName(Class.java:186)
>         at
> org.apache.giraph.utils.ConfigurationUtils.populateGiraphConfiguration(ConfigurationUtils.java:210)
>         at
> org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:147)
>         at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>         at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:616)
>         at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>
>
>
>
> and when I command like this,
>
>
> hadoop jar \
> /home/wb/giraph-1.0.0/giraph-examples/target/giraph-examples-1.0.0-for-hadoop-0.20.203.0-jar-with-dependencies.jar
> \
> org.apache.giraph.GiraphRunner \
> org.apache.giraph.examples.SimpleShortestPathsVertex \
> -vif
> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat \
> -vip /in/input \
> -of org.apache.giraph.io.formats.IdWithValueTextOutputFormat \
> -op /outShortest \
> -w 1
>
>
> I get this result.
>
>
> 13/06/24 16:40:54 INFO utils.ConfigurationUtils: No edge input format
> specified. Ensure your InputFormat does not require one.
> 13/06/24 16:40:54 WARN job.GiraphConfigurationValidator: Output format
> vertex index type is not known
> 13/06/24 16:40:54 WARN job.GiraphConfigurationValidator: Output format
> vertex value type is not known
> 13/06/24 16:40:54 WARN job.GiraphConfigurationValidator: Output format
> edge value type is not known
> 13/06/24 16:40:54 INFO job.GiraphJob: run: Since checkpointing is disabled
> (default), do not allow any task retries (setting mapred.map.max.attempts =
> 0, old value = 4)
> 13/06/24 16:40:56 INFO mapred.JobClient: Running job: job_201306241432_0005
> 13/06/24 16:40:57 INFO mapred.JobClient:  map 0% reduce 0%
> 13/06/24 16:41:18 INFO mapred.JobClient:  map 50% reduce 0%
> 13/06/24 16:41:21 INFO mapred.JobClient:  map 100% reduce 0%
> 13/06/24 16:51:29 INFO mapred.JobClient:  map 50% reduce 0%
> 13/06/24 16:51:37 INFO mapred.JobClient: Job complete:
> job_201306241432_0005
> 13/06/24 16:51:37 INFO mapred.JobClient: Counters: 6
> 13/06/24 16:51:37 INFO mapred.JobClient:   Job Counters
> 13/06/24 16:51:37 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=1242845
> 13/06/24 16:51:37 INFO mapred.JobClient:     Total time spent by all
> reduces waiting after reserving slots (ms)=0
> 13/06/24 16:51:37 INFO mapred.JobClient:     Total time spent by all maps
> waiting after reserving slots (ms)=0
> 13/06/24 16:51:37 INFO mapred.JobClient:     Launched map tasks=2
> 13/06/24 16:51:37 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=0
> 13/06/24 16:51:37 INFO mapred.JobClient:     Failed map tasks=1
>
>
> I cannot find out what's the probelm. Thanks.
>
>