You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@giraph.apache.org by Andro Bermúdez Serrano <an...@gmail.com> on 2013/10/09 20:44:15 UTC
Running the example in http://giraph.apache.org/quick_start.html
I am new with Giraph and I am willing to using it in my project. I am
using hadoop 1.2.1.
I builded giraph with mvn package -Phadoop_1.0
-Dhadoop.version=1.2.1 -DskipTests --offline. I ran the
SimpleShortestPathsVertex example and all OK. With the same input I
tried the SimplePageRankVertex example and failed:
This is the command I used:
$HADOOP_HOME/bin/hadoop jar
$GIRAPH_HOME/giraph-examples/target/giraph-examples-1.0.0-for-hadoop-1.2.1-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner
org.apache.giraph.examples.SimplePageRankVertex -vif
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
-vip /user/andro/input/tiny_graph.txt -of
org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
/user/andro/output/pagerank -w 1
This is the output:
13/10/09 14:21:36 INFO utils.ConfigurationUtils: No edge input format
specified. Ensure your InputFormat does not require one.
13/10/09 14:21:36 WARN job.GiraphConfigurationValidator: Output format
vertex index type is not known
13/10/09 14:21:36 WARN job.GiraphConfigurationValidator: Output format
vertex value type is not known
13/10/09 14:21:36 WARN job.GiraphConfigurationValidator: Output format
edge value type is not known
13/10/09 14:21:36 INFO job.GiraphJob: run: Since checkpointing is
disabled (default), do not allow any task retries (setting
mapred.map.max.attempts = 0, old value = 4)
13/10/09 14:21:37 INFO mapred.JobClient: Running job: job_201310091401_0002
13/10/09 14:21:38 INFO mapred.JobClient: map 0% reduce 0%
13/10/09 14:21:52 INFO mapred.JobClient: map 50% reduce 0%
13/10/09 14:21:58 INFO mapred.JobClient: map 100% reduce 0%
13/10/09 14:21:59 INFO mapred.JobClient: map 50% reduce 0%
13/10/09 14:32:01 INFO mapred.JobClient: map 0% reduce 0%
13/10/09 14:32:02 INFO mapred.JobClient: Job complete: job_201310091401_0002
13/10/09 14:32:02 INFO mapred.JobClient: Counters: 6
13/10/09 14:32:02 INFO mapred.JobClient: Job Counters
13/10/09 14:32:02 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=622821
13/10/09 14:32:02 INFO mapred.JobClient: Total time spent by all
reduces waiting after reserving slots (ms)=0
13/10/09 14:32:02 INFO mapred.JobClient: Total time spent by all
maps waiting after reserving slots (ms)=0
13/10/09 14:32:02 INFO mapred.JobClient: Launched map tasks=2
13/10/09 14:32:02 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0
13/10/09 14:32:02 INFO mapred.JobClient: Failed map tasks=1
I appreciate any help. Maybe I did it wrong.
Andro.
Re: Running the example in http://giraph.apache.org/quick_start.html
Posted by Andro Bermúdez Serrano <an...@gmail.com>.
Thanks, it works perfectly. I added:
-mc org.apache.giraph.examples.SimplePageRankVertex\$SimplePageRankVertexMasterCompute
Is there a tutorial that explains GiraphRunner parameters and how to
customize them?
(i.e., -vif, -ve, -mc, -eif, -aw, etc., and all needed classes)
2013/10/9, Wei Zhang <we...@us.ibm.com>:
>
> This is how I used the mc option that I used, not sure if it helps.
>
> -mc org.apache.giraph.examples.SimplePageRankComputation
> \$SimplePageRankMasterCompute
>
> Wei
>
>
>
> From: Claudio Martella <cl...@gmail.com>
> To: "user@giraph.apache.org" <us...@giraph.apache.org>,
> Date: 10/09/2013 05:00 PM
> Subject: Re: Running the example in
> http://giraph.apache.org/quick_start.html
>
>
>
> you need to define the mastercompute class (-mc option) which for your
> example is that SimplePageRankVertexMasterCompute class within that same
> vertex class.
>
>
> On Wed, Oct 9, 2013 at 10:44 PM, Andro Bermúdez Serrano <
> andro.bermudez@gmail.com> wrote:
> Maybe the problem is with agregators. This is one of the exception throw
>
> Caused by: java.lang.IllegalStateException: aggregate: Tried to access
> aggregator which wasn't registered max; Aggregators can be registered
> in MasterCompute.initialize by calling
> registerAggregator(aggregatorName, aggregatorClass). Also be sure that
> you are correctly setting MasterCompute class, currently using
> org.apache.giraph.master.DefaultMasterCompute
> at org.apache.giraph.worker.WorkerAggregatorHandler.aggregate
> (WorkerAggregatorHandler.java:103)
> at org.apache.giraph.graph.Vertex.aggregate(Vertex.java:502)
> at org.apache.giraph.examples.SimplePageRankVertex.compute
> (SimplePageRankVertex.java:74)
> at org.apache.giraph.graph.ComputeCallable.computePartition
> (ComputeCallable.java:234)
> at org.apache.giraph.graph.ComputeCallable.call
> (ComputeCallable.java:162)
> at org.apache.giraph.graph.ComputeCallable.call
> (ComputeCallable.java:70)
> at org.apache.giraph.utils.LogStacktraceCallable.call
> (LogStacktraceCallable.java:51)
> at java.util.concurrent.FutureTask$Sync.innerRun
> (FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker
> (ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run
> (ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:724)
>
> 2013/10/9, Silvio Di gregorio <si...@gmail.com>:
> > There must be another error . See all file. Looking for ERROR no WARN.
> > Il giorno 09/ott/2013 22:06, "Andro Bermúdez Serrano" <
> > andro.bermudez@gmail.com> ha scritto:
> >
> >> there are two failed map:
> >> java.lang.Throwable: Child Error at
> >> org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271) Caused
> >> by: java.io.IOException: Task process exit with nonzero status of 1.
> >> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> >>
> >> and
> >>
> >> java.lang.IllegalStateException: run: Caught an unrecoverable
> >> exception waitFor: ExecutionException occurred while waiting for
> >> org.apache.giraph.utils.ProgressableUtils$FutureWaitable@6ba789fd at
> >> org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102)
> >> (.......)
> >>
> >>
> >>
> >>
> >>
> >> 2013/10/9, Silvio Di gregorio <si...@gmail.com>:
> >> > Try TO the jobtracker UI.
> >> > http://nameofthemaster:50030/.
> >> > You can find the job band a detailed log ...
> >> > Il giorno 09/ott/2013 21:18, "Silvio Di gregorio" <
> >> > silvio.digregorio@gmail.com> ha scritto:
> >> >
> >> >> Can you see the log of the job? Try TO /var/log or find w/ the job
> >> >> name,
> >> >> job_date_002.
> >> >> Il giorno 09/ott/2013 21:02, "Wei Zhang" <we...@us.ibm.com> ha
> scritto:
> >> >>
> >> >>> Hi Andro,
> >> >>>
> >> >>> Not sure if it helps -- maybe you can try to change the "-of " to
> >> >>> "-vof
> >> >>> " on your command line. (i.e, the option flag in front of your
> >> >>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat).
> >> >>>
> >> >>> Wei
> >> >>>
> >> >>> [image: Inactive hide details for Andro Bermúdez Serrano
> >> >>> ---10/09/2013
> >> >>> 02:45:02 PM---I am new with Giraph and I am willing to using
> it]Andro
> >> >>> Bermúdez Serrano ---10/09/2013 02:45:02 PM---I am new with Giraph
> and
> >> >>> I
> >> >>> am
> >> >>> willing to using it in my project. I am using hadoop 1.2.1.
> >> >>>
> >> >>> From: Andro Bermúdez Serrano <an...@gmail.com>
> >> >>> To: user@giraph.apache.org,
> >> >>> Date: 10/09/2013 02:45 PM
> >> >>> Subject: Running the example in
> >> >>> http://giraph.apache.org/quick_start.html
> >> >>> ------------------------------
> >> >>>
> >> >>>
> >> >>>
> >> >>> I am new with Giraph and I am willing to using it in my project. I
> am
> >> >>> using hadoop 1.2.1.
> >> >>> I builded giraph with mvn package -Phadoop_1.0
> >> >>> -Dhadoop.version=1.2.1 -DskipTests --offline. I ran the
> >> >>> SimpleShortestPathsVertex example and all OK. With the same input
> I
> >> >>> tried the SimplePageRankVertex example and failed:
> >> >>>
> >> >>> This is the command I used:
> >> >>> $HADOOP_HOME/bin/hadoop jar
> >> >>>
> >> >>>
> >>
>
> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.0.0-for-hadoop-1.2.1-jar-with-dependencies.jar
>
> >> >>> org.apache.giraph.GiraphRunner
> >> >>> org.apache.giraph.examples.SimplePageRankVertex -vif
> >> >>>
> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
> >> >>> -vip /user/andro/input/tiny_graph.txt -of
> >> >>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
> >> >>> /user/andro/output/pagerank -w 1
> >> >>>
> >> >>> This is the output:
> >> >>> 13/10/09 14:21:36 INFO utils.ConfigurationUtils: No edge input
> format
> >> >>> specified. Ensure your InputFormat does not require one.
> >> >>> 13/10/09 14:21:36 WARN job.GiraphConfigurationValidator: Output
> >> >>> format
> >> >>> vertex index type is not known
> >> >>> 13/10/09 14:21:36 WARN job.GiraphConfigurationValidator: Output
> >> >>> format
> >> >>> vertex value type is not known
> >> >>> 13/10/09 14:21:36 WARN job.GiraphConfigurationValidator: Output
> >> >>> format
> >> >>> edge value type is not known
> >> >>> 13/10/09 14:21:36 INFO job.GiraphJob: run: Since checkpointing is
> >> >>> disabled (default), do not allow any task retries (setting
> >> >>> mapred.map.max.attempts = 0, old value = 4)
> >> >>> 13/10/09 14:21:37 INFO mapred.JobClient: Running job:
> >> >>> job_201310091401_0002
> >> >>> 13/10/09 14:21:38 INFO mapred.JobClient: map 0% reduce 0%
> >> >>> 13/10/09 14:21:52 INFO mapred.JobClient: map 50% reduce 0%
> >> >>> 13/10/09 14:21:58 INFO mapred.JobClient: map 100% reduce 0%
> >> >>> 13/10/09 14:21:59 INFO mapred.JobClient: map 50% reduce 0%
> >> >>> 13/10/09 14:32:01 INFO mapred.JobClient: map 0% reduce 0%
> >> >>> 13/10/09 14:32:02 INFO mapred.JobClient: Job complete:
> >> >>> job_201310091401_0002
> >> >>> 13/10/09 14:32:02 INFO mapred.JobClient: Counters: 6
> >> >>> 13/10/09 14:32:02 INFO mapred.JobClient: Job Counters
> >> >>> 13/10/09 14:32:02 INFO mapred.JobClient:
> SLOTS_MILLIS_MAPS=622821
> >> >>> 13/10/09 14:32:02 INFO mapred.JobClient: Total time spent by
> all
> >> >>> reduces waiting after reserving slots (ms)=0
> >> >>> 13/10/09 14:32:02 INFO mapred.JobClient: Total time spent by
> all
> >> >>> maps waiting after reserving slots (ms)=0
> >> >>> 13/10/09 14:32:02 INFO mapred.JobClient: Launched map tasks=2
> >> >>> 13/10/09 14:32:02 INFO mapred.JobClient:
> SLOTS_MILLIS_REDUCES=0
> >> >>> 13/10/09 14:32:02 INFO mapred.JobClient: Failed map tasks=1
> >> >>>
> >> >>>
> >> >>> I appreciate any help. Maybe I did it wrong.
> >> >>> Andro.
> >> >>>
> >> >>>
> >> >>>
> >> >
> >>
> >
>
>
>
> --
> Claudio Martella
> claudio.martella@gmail.com
Re: Running the example in http://giraph.apache.org/quick_start.html
Posted by Wei Zhang <we...@us.ibm.com>.
This is how I used the mc option that I used, not sure if it helps.
-mc org.apache.giraph.examples.SimplePageRankComputation
\$SimplePageRankMasterCompute
Wei
From: Claudio Martella <cl...@gmail.com>
To: "user@giraph.apache.org" <us...@giraph.apache.org>,
Date: 10/09/2013 05:00 PM
Subject: Re: Running the example in
http://giraph.apache.org/quick_start.html
you need to define the mastercompute class (-mc option) which for your
example is that SimplePageRankVertexMasterCompute class within that same
vertex class.
On Wed, Oct 9, 2013 at 10:44 PM, Andro Bermúdez Serrano <
andro.bermudez@gmail.com> wrote:
Maybe the problem is with agregators. This is one of the exception throw
Caused by: java.lang.IllegalStateException: aggregate: Tried to access
aggregator which wasn't registered max; Aggregators can be registered
in MasterCompute.initialize by calling
registerAggregator(aggregatorName, aggregatorClass). Also be sure that
you are correctly setting MasterCompute class, currently using
org.apache.giraph.master.DefaultMasterCompute
at org.apache.giraph.worker.WorkerAggregatorHandler.aggregate
(WorkerAggregatorHandler.java:103)
at org.apache.giraph.graph.Vertex.aggregate(Vertex.java:502)
at org.apache.giraph.examples.SimplePageRankVertex.compute
(SimplePageRankVertex.java:74)
at org.apache.giraph.graph.ComputeCallable.computePartition
(ComputeCallable.java:234)
at org.apache.giraph.graph.ComputeCallable.call
(ComputeCallable.java:162)
at org.apache.giraph.graph.ComputeCallable.call
(ComputeCallable.java:70)
at org.apache.giraph.utils.LogStacktraceCallable.call
(LogStacktraceCallable.java:51)
at java.util.concurrent.FutureTask$Sync.innerRun
(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker
(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run
(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
2013/10/9, Silvio Di gregorio <si...@gmail.com>:
> There must be another error . See all file. Looking for ERROR no WARN.
> Il giorno 09/ott/2013 22:06, "Andro Bermúdez Serrano" <
> andro.bermudez@gmail.com> ha scritto:
>
>> there are two failed map:
>> java.lang.Throwable: Child Error at
>> org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271) Caused
>> by: java.io.IOException: Task process exit with nonzero status of 1.
>> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
>>
>> and
>>
>> java.lang.IllegalStateException: run: Caught an unrecoverable
>> exception waitFor: ExecutionException occurred while waiting for
>> org.apache.giraph.utils.ProgressableUtils$FutureWaitable@6ba789fd at
>> org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102)
>> (.......)
>>
>>
>>
>>
>>
>> 2013/10/9, Silvio Di gregorio <si...@gmail.com>:
>> > Try TO the jobtracker UI.
>> > http://nameofthemaster:50030/.
>> > You can find the job band a detailed log ...
>> > Il giorno 09/ott/2013 21:18, "Silvio Di gregorio" <
>> > silvio.digregorio@gmail.com> ha scritto:
>> >
>> >> Can you see the log of the job? Try TO /var/log or find w/ the job
>> >> name,
>> >> job_date_002.
>> >> Il giorno 09/ott/2013 21:02, "Wei Zhang" <we...@us.ibm.com> ha
scritto:
>> >>
>> >>> Hi Andro,
>> >>>
>> >>> Not sure if it helps -- maybe you can try to change the "-of " to
>> >>> "-vof
>> >>> " on your command line. (i.e, the option flag in front of your
>> >>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat).
>> >>>
>> >>> Wei
>> >>>
>> >>> [image: Inactive hide details for Andro Bermúdez Serrano
>> >>> ---10/09/2013
>> >>> 02:45:02 PM---I am new with Giraph and I am willing to using
it]Andro
>> >>> Bermúdez Serrano ---10/09/2013 02:45:02 PM---I am new with Giraph
and
>> >>> I
>> >>> am
>> >>> willing to using it in my project. I am using hadoop 1.2.1.
>> >>>
>> >>> From: Andro Bermúdez Serrano <an...@gmail.com>
>> >>> To: user@giraph.apache.org,
>> >>> Date: 10/09/2013 02:45 PM
>> >>> Subject: Running the example in
>> >>> http://giraph.apache.org/quick_start.html
>> >>> ------------------------------
>> >>>
>> >>>
>> >>>
>> >>> I am new with Giraph and I am willing to using it in my project. I
am
>> >>> using hadoop 1.2.1.
>> >>> I builded giraph with mvn package -Phadoop_1.0
>> >>> -Dhadoop.version=1.2.1 -DskipTests --offline. I ran the
>> >>> SimpleShortestPathsVertex example and all OK. With the same input
I
>> >>> tried the SimplePageRankVertex example and failed:
>> >>>
>> >>> This is the command I used:
>> >>> $HADOOP_HOME/bin/hadoop jar
>> >>>
>> >>>
>>
$GIRAPH_HOME/giraph-examples/target/giraph-examples-1.0.0-for-hadoop-1.2.1-jar-with-dependencies.jar
>> >>> org.apache.giraph.GiraphRunner
>> >>> org.apache.giraph.examples.SimplePageRankVertex -vif
>> >>>
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
>> >>> -vip /user/andro/input/tiny_graph.txt -of
>> >>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
>> >>> /user/andro/output/pagerank -w 1
>> >>>
>> >>> This is the output:
>> >>> 13/10/09 14:21:36 INFO utils.ConfigurationUtils: No edge input
format
>> >>> specified. Ensure your InputFormat does not require one.
>> >>> 13/10/09 14:21:36 WARN job.GiraphConfigurationValidator: Output
>> >>> format
>> >>> vertex index type is not known
>> >>> 13/10/09 14:21:36 WARN job.GiraphConfigurationValidator: Output
>> >>> format
>> >>> vertex value type is not known
>> >>> 13/10/09 14:21:36 WARN job.GiraphConfigurationValidator: Output
>> >>> format
>> >>> edge value type is not known
>> >>> 13/10/09 14:21:36 INFO job.GiraphJob: run: Since checkpointing is
>> >>> disabled (default), do not allow any task retries (setting
>> >>> mapred.map.max.attempts = 0, old value = 4)
>> >>> 13/10/09 14:21:37 INFO mapred.JobClient: Running job:
>> >>> job_201310091401_0002
>> >>> 13/10/09 14:21:38 INFO mapred.JobClient: map 0% reduce 0%
>> >>> 13/10/09 14:21:52 INFO mapred.JobClient: map 50% reduce 0%
>> >>> 13/10/09 14:21:58 INFO mapred.JobClient: map 100% reduce 0%
>> >>> 13/10/09 14:21:59 INFO mapred.JobClient: map 50% reduce 0%
>> >>> 13/10/09 14:32:01 INFO mapred.JobClient: map 0% reduce 0%
>> >>> 13/10/09 14:32:02 INFO mapred.JobClient: Job complete:
>> >>> job_201310091401_0002
>> >>> 13/10/09 14:32:02 INFO mapred.JobClient: Counters: 6
>> >>> 13/10/09 14:32:02 INFO mapred.JobClient: Job Counters
>> >>> 13/10/09 14:32:02 INFO mapred.JobClient:
SLOTS_MILLIS_MAPS=622821
>> >>> 13/10/09 14:32:02 INFO mapred.JobClient: Total time spent by
all
>> >>> reduces waiting after reserving slots (ms)=0
>> >>> 13/10/09 14:32:02 INFO mapred.JobClient: Total time spent by
all
>> >>> maps waiting after reserving slots (ms)=0
>> >>> 13/10/09 14:32:02 INFO mapred.JobClient: Launched map tasks=2
>> >>> 13/10/09 14:32:02 INFO mapred.JobClient:
SLOTS_MILLIS_REDUCES=0
>> >>> 13/10/09 14:32:02 INFO mapred.JobClient: Failed map tasks=1
>> >>>
>> >>>
>> >>> I appreciate any help. Maybe I did it wrong.
>> >>> Andro.
>> >>>
>> >>>
>> >>>
>> >
>>
>
--
Claudio Martella
claudio.martella@gmail.com
Re: Running the example in http://giraph.apache.org/quick_start.html
Posted by Claudio Martella <cl...@gmail.com>.
you need to define the mastercompute class (-mc option) which for your
example is that SimplePageRankVertexMasterCompute class within that same
vertex class.
On Wed, Oct 9, 2013 at 10:44 PM, Andro Bermúdez Serrano <
andro.bermudez@gmail.com> wrote:
> Maybe the problem is with agregators. This is one of the exception throw
>
> Caused by: java.lang.IllegalStateException: aggregate: Tried to access
> aggregator which wasn't registered max; Aggregators can be registered
> in MasterCompute.initialize by calling
> registerAggregator(aggregatorName, aggregatorClass). Also be sure that
> you are correctly setting MasterCompute class, currently using
> org.apache.giraph.master.DefaultMasterCompute
> at
> org.apache.giraph.worker.WorkerAggregatorHandler.aggregate(WorkerAggregatorHandler.java:103)
> at org.apache.giraph.graph.Vertex.aggregate(Vertex.java:502)
> at
> org.apache.giraph.examples.SimplePageRankVertex.compute(SimplePageRankVertex.java:74)
> at
> org.apache.giraph.graph.ComputeCallable.computePartition(ComputeCallable.java:234)
> at
> org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:162)
> at
> org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:70)
> at
> org.apache.giraph.utils.LogStacktraceCallable.call(LogStacktraceCallable.java:51)
> at
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:724)
>
> 2013/10/9, Silvio Di gregorio <si...@gmail.com>:
> > There must be another error . See all file. Looking for ERROR no WARN.
> > Il giorno 09/ott/2013 22:06, "Andro Bermúdez Serrano" <
> > andro.bermudez@gmail.com> ha scritto:
> >
> >> there are two failed map:
> >> java.lang.Throwable: Child Error at
> >> org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271) Caused
> >> by: java.io.IOException: Task process exit with nonzero status of 1.
> >> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> >>
> >> and
> >>
> >> java.lang.IllegalStateException: run: Caught an unrecoverable
> >> exception waitFor: ExecutionException occurred while waiting for
> >> org.apache.giraph.utils.ProgressableUtils$FutureWaitable@6ba789fd at
> >> org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102)
> >> (.......)
> >>
> >>
> >>
> >>
> >>
> >> 2013/10/9, Silvio Di gregorio <si...@gmail.com>:
> >> > Try TO the jobtracker UI.
> >> > http://nameofthemaster:50030/.
> >> > You can find the job band a detailed log ...
> >> > Il giorno 09/ott/2013 21:18, "Silvio Di gregorio" <
> >> > silvio.digregorio@gmail.com> ha scritto:
> >> >
> >> >> Can you see the log of the job? Try TO /var/log or find w/ the job
> >> >> name,
> >> >> job_date_002.
> >> >> Il giorno 09/ott/2013 21:02, "Wei Zhang" <we...@us.ibm.com> ha
> scritto:
> >> >>
> >> >>> Hi Andro,
> >> >>>
> >> >>> Not sure if it helps -- maybe you can try to change the "-of " to
> >> >>> "-vof
> >> >>> " on your command line. (i.e, the option flag in front of your
> >> >>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat).
> >> >>>
> >> >>> Wei
> >> >>>
> >> >>> [image: Inactive hide details for Andro Bermúdez Serrano
> >> >>> ---10/09/2013
> >> >>> 02:45:02 PM---I am new with Giraph and I am willing to using
> it]Andro
> >> >>> Bermúdez Serrano ---10/09/2013 02:45:02 PM---I am new with Giraph
> and
> >> >>> I
> >> >>> am
> >> >>> willing to using it in my project. I am using hadoop 1.2.1.
> >> >>>
> >> >>> From: Andro Bermúdez Serrano <an...@gmail.com>
> >> >>> To: user@giraph.apache.org,
> >> >>> Date: 10/09/2013 02:45 PM
> >> >>> Subject: Running the example in
> >> >>> http://giraph.apache.org/quick_start.html
> >> >>> ------------------------------
> >> >>>
> >> >>>
> >> >>>
> >> >>> I am new with Giraph and I am willing to using it in my project. I
> am
> >> >>> using hadoop 1.2.1.
> >> >>> I builded giraph with mvn package -Phadoop_1.0
> >> >>> -Dhadoop.version=1.2.1 -DskipTests --offline. I ran the
> >> >>> SimpleShortestPathsVertex example and all OK. With the same input I
> >> >>> tried the SimplePageRankVertex example and failed:
> >> >>>
> >> >>> This is the command I used:
> >> >>> $HADOOP_HOME/bin/hadoop jar
> >> >>>
> >> >>>
> >>
> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.0.0-for-hadoop-1.2.1-jar-with-dependencies.jar
> >> >>> org.apache.giraph.GiraphRunner
> >> >>> org.apache.giraph.examples.SimplePageRankVertex -vif
> >> >>>
> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
> >> >>> -vip /user/andro/input/tiny_graph.txt -of
> >> >>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
> >> >>> /user/andro/output/pagerank -w 1
> >> >>>
> >> >>> This is the output:
> >> >>> 13/10/09 14:21:36 INFO utils.ConfigurationUtils: No edge input
> format
> >> >>> specified. Ensure your InputFormat does not require one.
> >> >>> 13/10/09 14:21:36 WARN job.GiraphConfigurationValidator: Output
> >> >>> format
> >> >>> vertex index type is not known
> >> >>> 13/10/09 14:21:36 WARN job.GiraphConfigurationValidator: Output
> >> >>> format
> >> >>> vertex value type is not known
> >> >>> 13/10/09 14:21:36 WARN job.GiraphConfigurationValidator: Output
> >> >>> format
> >> >>> edge value type is not known
> >> >>> 13/10/09 14:21:36 INFO job.GiraphJob: run: Since checkpointing is
> >> >>> disabled (default), do not allow any task retries (setting
> >> >>> mapred.map.max.attempts = 0, old value = 4)
> >> >>> 13/10/09 14:21:37 INFO mapred.JobClient: Running job:
> >> >>> job_201310091401_0002
> >> >>> 13/10/09 14:21:38 INFO mapred.JobClient: map 0% reduce 0%
> >> >>> 13/10/09 14:21:52 INFO mapred.JobClient: map 50% reduce 0%
> >> >>> 13/10/09 14:21:58 INFO mapred.JobClient: map 100% reduce 0%
> >> >>> 13/10/09 14:21:59 INFO mapred.JobClient: map 50% reduce 0%
> >> >>> 13/10/09 14:32:01 INFO mapred.JobClient: map 0% reduce 0%
> >> >>> 13/10/09 14:32:02 INFO mapred.JobClient: Job complete:
> >> >>> job_201310091401_0002
> >> >>> 13/10/09 14:32:02 INFO mapred.JobClient: Counters: 6
> >> >>> 13/10/09 14:32:02 INFO mapred.JobClient: Job Counters
> >> >>> 13/10/09 14:32:02 INFO mapred.JobClient:
> SLOTS_MILLIS_MAPS=622821
> >> >>> 13/10/09 14:32:02 INFO mapred.JobClient: Total time spent by all
> >> >>> reduces waiting after reserving slots (ms)=0
> >> >>> 13/10/09 14:32:02 INFO mapred.JobClient: Total time spent by all
> >> >>> maps waiting after reserving slots (ms)=0
> >> >>> 13/10/09 14:32:02 INFO mapred.JobClient: Launched map tasks=2
> >> >>> 13/10/09 14:32:02 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0
> >> >>> 13/10/09 14:32:02 INFO mapred.JobClient: Failed map tasks=1
> >> >>>
> >> >>>
> >> >>> I appreciate any help. Maybe I did it wrong.
> >> >>> Andro.
> >> >>>
> >> >>>
> >> >>>
> >> >
> >>
> >
>
--
Claudio Martella
claudio.martella@gmail.com
Re: Running the example in http://giraph.apache.org/quick_start.html
Posted by Andro Bermúdez Serrano <an...@gmail.com>.
Maybe the problem is with agregators. This is one of the exception throw
Caused by: java.lang.IllegalStateException: aggregate: Tried to access
aggregator which wasn't registered max; Aggregators can be registered
in MasterCompute.initialize by calling
registerAggregator(aggregatorName, aggregatorClass). Also be sure that
you are correctly setting MasterCompute class, currently using
org.apache.giraph.master.DefaultMasterCompute
at org.apache.giraph.worker.WorkerAggregatorHandler.aggregate(WorkerAggregatorHandler.java:103)
at org.apache.giraph.graph.Vertex.aggregate(Vertex.java:502)
at org.apache.giraph.examples.SimplePageRankVertex.compute(SimplePageRankVertex.java:74)
at org.apache.giraph.graph.ComputeCallable.computePartition(ComputeCallable.java:234)
at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:162)
at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:70)
at org.apache.giraph.utils.LogStacktraceCallable.call(LogStacktraceCallable.java:51)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
2013/10/9, Silvio Di gregorio <si...@gmail.com>:
> There must be another error . See all file. Looking for ERROR no WARN.
> Il giorno 09/ott/2013 22:06, "Andro Bermúdez Serrano" <
> andro.bermudez@gmail.com> ha scritto:
>
>> there are two failed map:
>> java.lang.Throwable: Child Error at
>> org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271) Caused
>> by: java.io.IOException: Task process exit with nonzero status of 1.
>> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
>>
>> and
>>
>> java.lang.IllegalStateException: run: Caught an unrecoverable
>> exception waitFor: ExecutionException occurred while waiting for
>> org.apache.giraph.utils.ProgressableUtils$FutureWaitable@6ba789fd at
>> org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102)
>> (.......)
>>
>>
>>
>>
>>
>> 2013/10/9, Silvio Di gregorio <si...@gmail.com>:
>> > Try TO the jobtracker UI.
>> > http://nameofthemaster:50030/.
>> > You can find the job band a detailed log ...
>> > Il giorno 09/ott/2013 21:18, "Silvio Di gregorio" <
>> > silvio.digregorio@gmail.com> ha scritto:
>> >
>> >> Can you see the log of the job? Try TO /var/log or find w/ the job
>> >> name,
>> >> job_date_002.
>> >> Il giorno 09/ott/2013 21:02, "Wei Zhang" <we...@us.ibm.com> ha scritto:
>> >>
>> >>> Hi Andro,
>> >>>
>> >>> Not sure if it helps -- maybe you can try to change the "-of " to
>> >>> "-vof
>> >>> " on your command line. (i.e, the option flag in front of your
>> >>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat).
>> >>>
>> >>> Wei
>> >>>
>> >>> [image: Inactive hide details for Andro Bermúdez Serrano
>> >>> ---10/09/2013
>> >>> 02:45:02 PM---I am new with Giraph and I am willing to using it]Andro
>> >>> Bermúdez Serrano ---10/09/2013 02:45:02 PM---I am new with Giraph and
>> >>> I
>> >>> am
>> >>> willing to using it in my project. I am using hadoop 1.2.1.
>> >>>
>> >>> From: Andro Bermúdez Serrano <an...@gmail.com>
>> >>> To: user@giraph.apache.org,
>> >>> Date: 10/09/2013 02:45 PM
>> >>> Subject: Running the example in
>> >>> http://giraph.apache.org/quick_start.html
>> >>> ------------------------------
>> >>>
>> >>>
>> >>>
>> >>> I am new with Giraph and I am willing to using it in my project. I am
>> >>> using hadoop 1.2.1.
>> >>> I builded giraph with mvn package -Phadoop_1.0
>> >>> -Dhadoop.version=1.2.1 -DskipTests --offline. I ran the
>> >>> SimpleShortestPathsVertex example and all OK. With the same input I
>> >>> tried the SimplePageRankVertex example and failed:
>> >>>
>> >>> This is the command I used:
>> >>> $HADOOP_HOME/bin/hadoop jar
>> >>>
>> >>>
>> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.0.0-for-hadoop-1.2.1-jar-with-dependencies.jar
>> >>> org.apache.giraph.GiraphRunner
>> >>> org.apache.giraph.examples.SimplePageRankVertex -vif
>> >>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
>> >>> -vip /user/andro/input/tiny_graph.txt -of
>> >>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
>> >>> /user/andro/output/pagerank -w 1
>> >>>
>> >>> This is the output:
>> >>> 13/10/09 14:21:36 INFO utils.ConfigurationUtils: No edge input format
>> >>> specified. Ensure your InputFormat does not require one.
>> >>> 13/10/09 14:21:36 WARN job.GiraphConfigurationValidator: Output
>> >>> format
>> >>> vertex index type is not known
>> >>> 13/10/09 14:21:36 WARN job.GiraphConfigurationValidator: Output
>> >>> format
>> >>> vertex value type is not known
>> >>> 13/10/09 14:21:36 WARN job.GiraphConfigurationValidator: Output
>> >>> format
>> >>> edge value type is not known
>> >>> 13/10/09 14:21:36 INFO job.GiraphJob: run: Since checkpointing is
>> >>> disabled (default), do not allow any task retries (setting
>> >>> mapred.map.max.attempts = 0, old value = 4)
>> >>> 13/10/09 14:21:37 INFO mapred.JobClient: Running job:
>> >>> job_201310091401_0002
>> >>> 13/10/09 14:21:38 INFO mapred.JobClient: map 0% reduce 0%
>> >>> 13/10/09 14:21:52 INFO mapred.JobClient: map 50% reduce 0%
>> >>> 13/10/09 14:21:58 INFO mapred.JobClient: map 100% reduce 0%
>> >>> 13/10/09 14:21:59 INFO mapred.JobClient: map 50% reduce 0%
>> >>> 13/10/09 14:32:01 INFO mapred.JobClient: map 0% reduce 0%
>> >>> 13/10/09 14:32:02 INFO mapred.JobClient: Job complete:
>> >>> job_201310091401_0002
>> >>> 13/10/09 14:32:02 INFO mapred.JobClient: Counters: 6
>> >>> 13/10/09 14:32:02 INFO mapred.JobClient: Job Counters
>> >>> 13/10/09 14:32:02 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=622821
>> >>> 13/10/09 14:32:02 INFO mapred.JobClient: Total time spent by all
>> >>> reduces waiting after reserving slots (ms)=0
>> >>> 13/10/09 14:32:02 INFO mapred.JobClient: Total time spent by all
>> >>> maps waiting after reserving slots (ms)=0
>> >>> 13/10/09 14:32:02 INFO mapred.JobClient: Launched map tasks=2
>> >>> 13/10/09 14:32:02 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0
>> >>> 13/10/09 14:32:02 INFO mapred.JobClient: Failed map tasks=1
>> >>>
>> >>>
>> >>> I appreciate any help. Maybe I did it wrong.
>> >>> Andro.
>> >>>
>> >>>
>> >>>
>> >
>>
>
Re: Running the example in http://giraph.apache.org/quick_start.html
Posted by Silvio Di gregorio <si...@gmail.com>.
There must be another error . See all file. Looking for ERROR no WARN.
Il giorno 09/ott/2013 22:06, "Andro Bermúdez Serrano" <
andro.bermudez@gmail.com> ha scritto:
> there are two failed map:
> java.lang.Throwable: Child Error at
> org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271) Caused
> by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
>
> and
>
> java.lang.IllegalStateException: run: Caught an unrecoverable
> exception waitFor: ExecutionException occurred while waiting for
> org.apache.giraph.utils.ProgressableUtils$FutureWaitable@6ba789fd at
> org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102)
> (.......)
>
>
>
>
>
> 2013/10/9, Silvio Di gregorio <si...@gmail.com>:
> > Try TO the jobtracker UI.
> > http://nameofthemaster:50030/.
> > You can find the job band a detailed log ...
> > Il giorno 09/ott/2013 21:18, "Silvio Di gregorio" <
> > silvio.digregorio@gmail.com> ha scritto:
> >
> >> Can you see the log of the job? Try TO /var/log or find w/ the job name,
> >> job_date_002.
> >> Il giorno 09/ott/2013 21:02, "Wei Zhang" <we...@us.ibm.com> ha scritto:
> >>
> >>> Hi Andro,
> >>>
> >>> Not sure if it helps -- maybe you can try to change the "-of " to "-vof
> >>> " on your command line. (i.e, the option flag in front of your
> >>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat).
> >>>
> >>> Wei
> >>>
> >>> [image: Inactive hide details for Andro Bermúdez Serrano ---10/09/2013
> >>> 02:45:02 PM---I am new with Giraph and I am willing to using it]Andro
> >>> Bermúdez Serrano ---10/09/2013 02:45:02 PM---I am new with Giraph and I
> >>> am
> >>> willing to using it in my project. I am using hadoop 1.2.1.
> >>>
> >>> From: Andro Bermúdez Serrano <an...@gmail.com>
> >>> To: user@giraph.apache.org,
> >>> Date: 10/09/2013 02:45 PM
> >>> Subject: Running the example in
> >>> http://giraph.apache.org/quick_start.html
> >>> ------------------------------
> >>>
> >>>
> >>>
> >>> I am new with Giraph and I am willing to using it in my project. I am
> >>> using hadoop 1.2.1.
> >>> I builded giraph with mvn package -Phadoop_1.0
> >>> -Dhadoop.version=1.2.1 -DskipTests --offline. I ran the
> >>> SimpleShortestPathsVertex example and all OK. With the same input I
> >>> tried the SimplePageRankVertex example and failed:
> >>>
> >>> This is the command I used:
> >>> $HADOOP_HOME/bin/hadoop jar
> >>>
> >>>
> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.0.0-for-hadoop-1.2.1-jar-with-dependencies.jar
> >>> org.apache.giraph.GiraphRunner
> >>> org.apache.giraph.examples.SimplePageRankVertex -vif
> >>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
> >>> -vip /user/andro/input/tiny_graph.txt -of
> >>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
> >>> /user/andro/output/pagerank -w 1
> >>>
> >>> This is the output:
> >>> 13/10/09 14:21:36 INFO utils.ConfigurationUtils: No edge input format
> >>> specified. Ensure your InputFormat does not require one.
> >>> 13/10/09 14:21:36 WARN job.GiraphConfigurationValidator: Output format
> >>> vertex index type is not known
> >>> 13/10/09 14:21:36 WARN job.GiraphConfigurationValidator: Output format
> >>> vertex value type is not known
> >>> 13/10/09 14:21:36 WARN job.GiraphConfigurationValidator: Output format
> >>> edge value type is not known
> >>> 13/10/09 14:21:36 INFO job.GiraphJob: run: Since checkpointing is
> >>> disabled (default), do not allow any task retries (setting
> >>> mapred.map.max.attempts = 0, old value = 4)
> >>> 13/10/09 14:21:37 INFO mapred.JobClient: Running job:
> >>> job_201310091401_0002
> >>> 13/10/09 14:21:38 INFO mapred.JobClient: map 0% reduce 0%
> >>> 13/10/09 14:21:52 INFO mapred.JobClient: map 50% reduce 0%
> >>> 13/10/09 14:21:58 INFO mapred.JobClient: map 100% reduce 0%
> >>> 13/10/09 14:21:59 INFO mapred.JobClient: map 50% reduce 0%
> >>> 13/10/09 14:32:01 INFO mapred.JobClient: map 0% reduce 0%
> >>> 13/10/09 14:32:02 INFO mapred.JobClient: Job complete:
> >>> job_201310091401_0002
> >>> 13/10/09 14:32:02 INFO mapred.JobClient: Counters: 6
> >>> 13/10/09 14:32:02 INFO mapred.JobClient: Job Counters
> >>> 13/10/09 14:32:02 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=622821
> >>> 13/10/09 14:32:02 INFO mapred.JobClient: Total time spent by all
> >>> reduces waiting after reserving slots (ms)=0
> >>> 13/10/09 14:32:02 INFO mapred.JobClient: Total time spent by all
> >>> maps waiting after reserving slots (ms)=0
> >>> 13/10/09 14:32:02 INFO mapred.JobClient: Launched map tasks=2
> >>> 13/10/09 14:32:02 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0
> >>> 13/10/09 14:32:02 INFO mapred.JobClient: Failed map tasks=1
> >>>
> >>>
> >>> I appreciate any help. Maybe I did it wrong.
> >>> Andro.
> >>>
> >>>
> >>>
> >
>
Re: Running the example in http://giraph.apache.org/quick_start.html
Posted by Andro Bermúdez Serrano <an...@gmail.com>.
there are two failed map:
java.lang.Throwable: Child Error at
org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271) Caused
by: java.io.IOException: Task process exit with nonzero status of 1.
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
and
java.lang.IllegalStateException: run: Caught an unrecoverable
exception waitFor: ExecutionException occurred while waiting for
org.apache.giraph.utils.ProgressableUtils$FutureWaitable@6ba789fd at
org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102)
(.......)
2013/10/9, Silvio Di gregorio <si...@gmail.com>:
> Try TO the jobtracker UI.
> http://nameofthemaster:50030/.
> You can find the job band a detailed log ...
> Il giorno 09/ott/2013 21:18, "Silvio Di gregorio" <
> silvio.digregorio@gmail.com> ha scritto:
>
>> Can you see the log of the job? Try TO /var/log or find w/ the job name,
>> job_date_002.
>> Il giorno 09/ott/2013 21:02, "Wei Zhang" <we...@us.ibm.com> ha scritto:
>>
>>> Hi Andro,
>>>
>>> Not sure if it helps -- maybe you can try to change the "-of " to "-vof
>>> " on your command line. (i.e, the option flag in front of your
>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat).
>>>
>>> Wei
>>>
>>> [image: Inactive hide details for Andro Bermúdez Serrano ---10/09/2013
>>> 02:45:02 PM---I am new with Giraph and I am willing to using it]Andro
>>> Bermúdez Serrano ---10/09/2013 02:45:02 PM---I am new with Giraph and I
>>> am
>>> willing to using it in my project. I am using hadoop 1.2.1.
>>>
>>> From: Andro Bermúdez Serrano <an...@gmail.com>
>>> To: user@giraph.apache.org,
>>> Date: 10/09/2013 02:45 PM
>>> Subject: Running the example in
>>> http://giraph.apache.org/quick_start.html
>>> ------------------------------
>>>
>>>
>>>
>>> I am new with Giraph and I am willing to using it in my project. I am
>>> using hadoop 1.2.1.
>>> I builded giraph with mvn package -Phadoop_1.0
>>> -Dhadoop.version=1.2.1 -DskipTests --offline. I ran the
>>> SimpleShortestPathsVertex example and all OK. With the same input I
>>> tried the SimplePageRankVertex example and failed:
>>>
>>> This is the command I used:
>>> $HADOOP_HOME/bin/hadoop jar
>>>
>>> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.0.0-for-hadoop-1.2.1-jar-with-dependencies.jar
>>> org.apache.giraph.GiraphRunner
>>> org.apache.giraph.examples.SimplePageRankVertex -vif
>>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
>>> -vip /user/andro/input/tiny_graph.txt -of
>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
>>> /user/andro/output/pagerank -w 1
>>>
>>> This is the output:
>>> 13/10/09 14:21:36 INFO utils.ConfigurationUtils: No edge input format
>>> specified. Ensure your InputFormat does not require one.
>>> 13/10/09 14:21:36 WARN job.GiraphConfigurationValidator: Output format
>>> vertex index type is not known
>>> 13/10/09 14:21:36 WARN job.GiraphConfigurationValidator: Output format
>>> vertex value type is not known
>>> 13/10/09 14:21:36 WARN job.GiraphConfigurationValidator: Output format
>>> edge value type is not known
>>> 13/10/09 14:21:36 INFO job.GiraphJob: run: Since checkpointing is
>>> disabled (default), do not allow any task retries (setting
>>> mapred.map.max.attempts = 0, old value = 4)
>>> 13/10/09 14:21:37 INFO mapred.JobClient: Running job:
>>> job_201310091401_0002
>>> 13/10/09 14:21:38 INFO mapred.JobClient: map 0% reduce 0%
>>> 13/10/09 14:21:52 INFO mapred.JobClient: map 50% reduce 0%
>>> 13/10/09 14:21:58 INFO mapred.JobClient: map 100% reduce 0%
>>> 13/10/09 14:21:59 INFO mapred.JobClient: map 50% reduce 0%
>>> 13/10/09 14:32:01 INFO mapred.JobClient: map 0% reduce 0%
>>> 13/10/09 14:32:02 INFO mapred.JobClient: Job complete:
>>> job_201310091401_0002
>>> 13/10/09 14:32:02 INFO mapred.JobClient: Counters: 6
>>> 13/10/09 14:32:02 INFO mapred.JobClient: Job Counters
>>> 13/10/09 14:32:02 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=622821
>>> 13/10/09 14:32:02 INFO mapred.JobClient: Total time spent by all
>>> reduces waiting after reserving slots (ms)=0
>>> 13/10/09 14:32:02 INFO mapred.JobClient: Total time spent by all
>>> maps waiting after reserving slots (ms)=0
>>> 13/10/09 14:32:02 INFO mapred.JobClient: Launched map tasks=2
>>> 13/10/09 14:32:02 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0
>>> 13/10/09 14:32:02 INFO mapred.JobClient: Failed map tasks=1
>>>
>>>
>>> I appreciate any help. Maybe I did it wrong.
>>> Andro.
>>>
>>>
>>>
>
Re: Running the example in http://giraph.apache.org/quick_start.html
Posted by Silvio Di gregorio <si...@gmail.com>.
Try TO the jobtracker UI.
http://nameofthemaster:50030/.
You can find the job band a detailed log ...
Il giorno 09/ott/2013 21:18, "Silvio Di gregorio" <
silvio.digregorio@gmail.com> ha scritto:
> Can you see the log of the job? Try TO /var/log or find w/ the job name,
> job_date_002.
> Il giorno 09/ott/2013 21:02, "Wei Zhang" <we...@us.ibm.com> ha scritto:
>
>> Hi Andro,
>>
>> Not sure if it helps -- maybe you can try to change the "-of " to "-vof
>> " on your command line. (i.e, the option flag in front of your
>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat).
>>
>> Wei
>>
>> [image: Inactive hide details for Andro Bermúdez Serrano ---10/09/2013
>> 02:45:02 PM---I am new with Giraph and I am willing to using it]Andro
>> Bermúdez Serrano ---10/09/2013 02:45:02 PM---I am new with Giraph and I am
>> willing to using it in my project. I am using hadoop 1.2.1.
>>
>> From: Andro Bermúdez Serrano <an...@gmail.com>
>> To: user@giraph.apache.org,
>> Date: 10/09/2013 02:45 PM
>> Subject: Running the example in http://giraph.apache.org/quick_start.html
>> ------------------------------
>>
>>
>>
>> I am new with Giraph and I am willing to using it in my project. I am
>> using hadoop 1.2.1.
>> I builded giraph with mvn package -Phadoop_1.0
>> -Dhadoop.version=1.2.1 -DskipTests --offline. I ran the
>> SimpleShortestPathsVertex example and all OK. With the same input I
>> tried the SimplePageRankVertex example and failed:
>>
>> This is the command I used:
>> $HADOOP_HOME/bin/hadoop jar
>>
>> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.0.0-for-hadoop-1.2.1-jar-with-dependencies.jar
>> org.apache.giraph.GiraphRunner
>> org.apache.giraph.examples.SimplePageRankVertex -vif
>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
>> -vip /user/andro/input/tiny_graph.txt -of
>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
>> /user/andro/output/pagerank -w 1
>>
>> This is the output:
>> 13/10/09 14:21:36 INFO utils.ConfigurationUtils: No edge input format
>> specified. Ensure your InputFormat does not require one.
>> 13/10/09 14:21:36 WARN job.GiraphConfigurationValidator: Output format
>> vertex index type is not known
>> 13/10/09 14:21:36 WARN job.GiraphConfigurationValidator: Output format
>> vertex value type is not known
>> 13/10/09 14:21:36 WARN job.GiraphConfigurationValidator: Output format
>> edge value type is not known
>> 13/10/09 14:21:36 INFO job.GiraphJob: run: Since checkpointing is
>> disabled (default), do not allow any task retries (setting
>> mapred.map.max.attempts = 0, old value = 4)
>> 13/10/09 14:21:37 INFO mapred.JobClient: Running job:
>> job_201310091401_0002
>> 13/10/09 14:21:38 INFO mapred.JobClient: map 0% reduce 0%
>> 13/10/09 14:21:52 INFO mapred.JobClient: map 50% reduce 0%
>> 13/10/09 14:21:58 INFO mapred.JobClient: map 100% reduce 0%
>> 13/10/09 14:21:59 INFO mapred.JobClient: map 50% reduce 0%
>> 13/10/09 14:32:01 INFO mapred.JobClient: map 0% reduce 0%
>> 13/10/09 14:32:02 INFO mapred.JobClient: Job complete:
>> job_201310091401_0002
>> 13/10/09 14:32:02 INFO mapred.JobClient: Counters: 6
>> 13/10/09 14:32:02 INFO mapred.JobClient: Job Counters
>> 13/10/09 14:32:02 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=622821
>> 13/10/09 14:32:02 INFO mapred.JobClient: Total time spent by all
>> reduces waiting after reserving slots (ms)=0
>> 13/10/09 14:32:02 INFO mapred.JobClient: Total time spent by all
>> maps waiting after reserving slots (ms)=0
>> 13/10/09 14:32:02 INFO mapred.JobClient: Launched map tasks=2
>> 13/10/09 14:32:02 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0
>> 13/10/09 14:32:02 INFO mapred.JobClient: Failed map tasks=1
>>
>>
>> I appreciate any help. Maybe I did it wrong.
>> Andro.
>>
>>
>>
Re: Running the example in http://giraph.apache.org/quick_start.html
Posted by Silvio Di gregorio <si...@gmail.com>.
Can you see the log of the job? Try TO /var/log or find w/ the job name,
job_date_002.
Il giorno 09/ott/2013 21:02, "Wei Zhang" <we...@us.ibm.com> ha scritto:
> Hi Andro,
>
> Not sure if it helps -- maybe you can try to change the "-of " to "-vof "
> on your command line. (i.e, the option flag in front of your
> org.apache.giraph.io.formats.IdWithValueTextOutputFormat).
>
> Wei
>
> [image: Inactive hide details for Andro Bermúdez Serrano ---10/09/2013
> 02:45:02 PM---I am new with Giraph and I am willing to using it]Andro
> Bermúdez Serrano ---10/09/2013 02:45:02 PM---I am new with Giraph and I am
> willing to using it in my project. I am using hadoop 1.2.1.
>
> From: Andro Bermúdez Serrano <an...@gmail.com>
> To: user@giraph.apache.org,
> Date: 10/09/2013 02:45 PM
> Subject: Running the example in http://giraph.apache.org/quick_start.html
> ------------------------------
>
>
>
> I am new with Giraph and I am willing to using it in my project. I am
> using hadoop 1.2.1.
> I builded giraph with mvn package -Phadoop_1.0
> -Dhadoop.version=1.2.1 -DskipTests --offline. I ran the
> SimpleShortestPathsVertex example and all OK. With the same input I
> tried the SimplePageRankVertex example and failed:
>
> This is the command I used:
> $HADOOP_HOME/bin/hadoop jar
>
> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.0.0-for-hadoop-1.2.1-jar-with-dependencies.jar
> org.apache.giraph.GiraphRunner
> org.apache.giraph.examples.SimplePageRankVertex -vif
> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
> -vip /user/andro/input/tiny_graph.txt -of
> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
> /user/andro/output/pagerank -w 1
>
> This is the output:
> 13/10/09 14:21:36 INFO utils.ConfigurationUtils: No edge input format
> specified. Ensure your InputFormat does not require one.
> 13/10/09 14:21:36 WARN job.GiraphConfigurationValidator: Output format
> vertex index type is not known
> 13/10/09 14:21:36 WARN job.GiraphConfigurationValidator: Output format
> vertex value type is not known
> 13/10/09 14:21:36 WARN job.GiraphConfigurationValidator: Output format
> edge value type is not known
> 13/10/09 14:21:36 INFO job.GiraphJob: run: Since checkpointing is
> disabled (default), do not allow any task retries (setting
> mapred.map.max.attempts = 0, old value = 4)
> 13/10/09 14:21:37 INFO mapred.JobClient: Running job: job_201310091401_0002
> 13/10/09 14:21:38 INFO mapred.JobClient: map 0% reduce 0%
> 13/10/09 14:21:52 INFO mapred.JobClient: map 50% reduce 0%
> 13/10/09 14:21:58 INFO mapred.JobClient: map 100% reduce 0%
> 13/10/09 14:21:59 INFO mapred.JobClient: map 50% reduce 0%
> 13/10/09 14:32:01 INFO mapred.JobClient: map 0% reduce 0%
> 13/10/09 14:32:02 INFO mapred.JobClient: Job complete:
> job_201310091401_0002
> 13/10/09 14:32:02 INFO mapred.JobClient: Counters: 6
> 13/10/09 14:32:02 INFO mapred.JobClient: Job Counters
> 13/10/09 14:32:02 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=622821
> 13/10/09 14:32:02 INFO mapred.JobClient: Total time spent by all
> reduces waiting after reserving slots (ms)=0
> 13/10/09 14:32:02 INFO mapred.JobClient: Total time spent by all
> maps waiting after reserving slots (ms)=0
> 13/10/09 14:32:02 INFO mapred.JobClient: Launched map tasks=2
> 13/10/09 14:32:02 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0
> 13/10/09 14:32:02 INFO mapred.JobClient: Failed map tasks=1
>
>
> I appreciate any help. Maybe I did it wrong.
> Andro.
>
>
>
Re: Running the example in http://giraph.apache.org/quick_start.html
Posted by Andro Bermúdez Serrano <an...@gmail.com>.
The "-vof" option is not available in my version of Giraph. (1.0.0) An
error ocurred when I changed it.
2013/10/9, Wei Zhang <we...@us.ibm.com>:
>
> Hi Andro,
>
> Not sure if it helps -- maybe you can try to change the "-of " to "-vof "
> on your command line. (i.e, the option flag in front of your
> org.apache.giraph.io.formats.IdWithValueTextOutputFormat).
>
> Wei
>
>
>
> From: Andro Bermúdez Serrano <an...@gmail.com>
> To: user@giraph.apache.org,
> Date: 10/09/2013 02:45 PM
> Subject: Running the example in
> http://giraph.apache.org/quick_start.html
>
>
>
> I am new with Giraph and I am willing to using it in my project. I am
> using hadoop 1.2.1.
> I builded giraph with mvn package -Phadoop_1.0
> -Dhadoop.version=1.2.1 -DskipTests --offline. I ran the
> SimpleShortestPathsVertex example and all OK. With the same input I
> tried the SimplePageRankVertex example and failed:
>
> This is the command I used:
> $HADOOP_HOME/bin/hadoop jar
> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.0.0-for-hadoop-1.2.1-jar-with-dependencies.jar
>
> org.apache.giraph.GiraphRunner
> org.apache.giraph.examples.SimplePageRankVertex -vif
> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
> -vip /user/andro/input/tiny_graph.txt -of
> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
> /user/andro/output/pagerank -w 1
>
> This is the output:
> 13/10/09 14:21:36 INFO utils.ConfigurationUtils: No edge input format
> specified. Ensure your InputFormat does not require one.
> 13/10/09 14:21:36 WARN job.GiraphConfigurationValidator: Output format
> vertex index type is not known
> 13/10/09 14:21:36 WARN job.GiraphConfigurationValidator: Output format
> vertex value type is not known
> 13/10/09 14:21:36 WARN job.GiraphConfigurationValidator: Output format
> edge value type is not known
> 13/10/09 14:21:36 INFO job.GiraphJob: run: Since checkpointing is
> disabled (default), do not allow any task retries (setting
> mapred.map.max.attempts = 0, old value = 4)
> 13/10/09 14:21:37 INFO mapred.JobClient: Running job: job_201310091401_0002
> 13/10/09 14:21:38 INFO mapred.JobClient: map 0% reduce 0%
> 13/10/09 14:21:52 INFO mapred.JobClient: map 50% reduce 0%
> 13/10/09 14:21:58 INFO mapred.JobClient: map 100% reduce 0%
> 13/10/09 14:21:59 INFO mapred.JobClient: map 50% reduce 0%
> 13/10/09 14:32:01 INFO mapred.JobClient: map 0% reduce 0%
> 13/10/09 14:32:02 INFO mapred.JobClient: Job complete:
> job_201310091401_0002
> 13/10/09 14:32:02 INFO mapred.JobClient: Counters: 6
> 13/10/09 14:32:02 INFO mapred.JobClient: Job Counters
> 13/10/09 14:32:02 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=622821
> 13/10/09 14:32:02 INFO mapred.JobClient: Total time spent by all
> reduces waiting after reserving slots (ms)=0
> 13/10/09 14:32:02 INFO mapred.JobClient: Total time spent by all
> maps waiting after reserving slots (ms)=0
> 13/10/09 14:32:02 INFO mapred.JobClient: Launched map tasks=2
> 13/10/09 14:32:02 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0
> 13/10/09 14:32:02 INFO mapred.JobClient: Failed map tasks=1
>
>
> I appreciate any help. Maybe I did it wrong.
> Andro.
>
>
Re: Running the example in http://giraph.apache.org/quick_start.html
Posted by Wei Zhang <we...@us.ibm.com>.
Hi Andro,
Not sure if it helps -- maybe you can try to change the "-of " to "-vof "
on your command line. (i.e, the option flag in front of your
org.apache.giraph.io.formats.IdWithValueTextOutputFormat).
Wei
From: Andro Bermúdez Serrano <an...@gmail.com>
To: user@giraph.apache.org,
Date: 10/09/2013 02:45 PM
Subject: Running the example in
http://giraph.apache.org/quick_start.html
I am new with Giraph and I am willing to using it in my project. I am
using hadoop 1.2.1.
I builded giraph with mvn package -Phadoop_1.0
-Dhadoop.version=1.2.1 -DskipTests --offline. I ran the
SimpleShortestPathsVertex example and all OK. With the same input I
tried the SimplePageRankVertex example and failed:
This is the command I used:
$HADOOP_HOME/bin/hadoop jar
$GIRAPH_HOME/giraph-examples/target/giraph-examples-1.0.0-for-hadoop-1.2.1-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner
org.apache.giraph.examples.SimplePageRankVertex -vif
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
-vip /user/andro/input/tiny_graph.txt -of
org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
/user/andro/output/pagerank -w 1
This is the output:
13/10/09 14:21:36 INFO utils.ConfigurationUtils: No edge input format
specified. Ensure your InputFormat does not require one.
13/10/09 14:21:36 WARN job.GiraphConfigurationValidator: Output format
vertex index type is not known
13/10/09 14:21:36 WARN job.GiraphConfigurationValidator: Output format
vertex value type is not known
13/10/09 14:21:36 WARN job.GiraphConfigurationValidator: Output format
edge value type is not known
13/10/09 14:21:36 INFO job.GiraphJob: run: Since checkpointing is
disabled (default), do not allow any task retries (setting
mapred.map.max.attempts = 0, old value = 4)
13/10/09 14:21:37 INFO mapred.JobClient: Running job: job_201310091401_0002
13/10/09 14:21:38 INFO mapred.JobClient: map 0% reduce 0%
13/10/09 14:21:52 INFO mapred.JobClient: map 50% reduce 0%
13/10/09 14:21:58 INFO mapred.JobClient: map 100% reduce 0%
13/10/09 14:21:59 INFO mapred.JobClient: map 50% reduce 0%
13/10/09 14:32:01 INFO mapred.JobClient: map 0% reduce 0%
13/10/09 14:32:02 INFO mapred.JobClient: Job complete:
job_201310091401_0002
13/10/09 14:32:02 INFO mapred.JobClient: Counters: 6
13/10/09 14:32:02 INFO mapred.JobClient: Job Counters
13/10/09 14:32:02 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=622821
13/10/09 14:32:02 INFO mapred.JobClient: Total time spent by all
reduces waiting after reserving slots (ms)=0
13/10/09 14:32:02 INFO mapred.JobClient: Total time spent by all
maps waiting after reserving slots (ms)=0
13/10/09 14:32:02 INFO mapred.JobClient: Launched map tasks=2
13/10/09 14:32:02 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0
13/10/09 14:32:02 INFO mapred.JobClient: Failed map tasks=1
I appreciate any help. Maybe I did it wrong.
Andro.
Re: Running the example in http://giraph.apache.org/quick_start.html
Posted by Silvio Di gregorio <si...@gmail.com>.
Try VertexWithDoubleValueNullEdgeTextOutputFormat like output format. The
rank of the vertex is a double .
Il giorno 09/ott/2013 20:44, "Andro Bermúdez Serrano" <
andro.bermudez@gmail.com> ha scritto:
> I am new with Giraph and I am willing to using it in my project. I am
> using hadoop 1.2.1.
> I builded giraph with mvn package -Phadoop_1.0
> -Dhadoop.version=1.2.1 -DskipTests --offline. I ran the
> SimpleShortestPathsVertex example and all OK. With the same input I
> tried the SimplePageRankVertex example and failed:
>
> This is the command I used:
> $HADOOP_HOME/bin/hadoop jar
>
> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.0.0-for-hadoop-1.2.1-jar-with-dependencies.jar
> org.apache.giraph.GiraphRunner
> org.apache.giraph.examples.SimplePageRankVertex -vif
> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
> -vip /user/andro/input/tiny_graph.txt -of
> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
> /user/andro/output/pagerank -w 1
>
> This is the output:
> 13/10/09 14:21:36 INFO utils.ConfigurationUtils: No edge input format
> specified. Ensure your InputFormat does not require one.
> 13/10/09 14:21:36 WARN job.GiraphConfigurationValidator: Output format
> vertex index type is not known
> 13/10/09 14:21:36 WARN job.GiraphConfigurationValidator: Output format
> vertex value type is not known
> 13/10/09 14:21:36 WARN job.GiraphConfigurationValidator: Output format
> edge value type is not known
> 13/10/09 14:21:36 INFO job.GiraphJob: run: Since checkpointing is
> disabled (default), do not allow any task retries (setting
> mapred.map.max.attempts = 0, old value = 4)
> 13/10/09 14:21:37 INFO mapred.JobClient: Running job: job_201310091401_0002
> 13/10/09 14:21:38 INFO mapred.JobClient: map 0% reduce 0%
> 13/10/09 14:21:52 INFO mapred.JobClient: map 50% reduce 0%
> 13/10/09 14:21:58 INFO mapred.JobClient: map 100% reduce 0%
> 13/10/09 14:21:59 INFO mapred.JobClient: map 50% reduce 0%
> 13/10/09 14:32:01 INFO mapred.JobClient: map 0% reduce 0%
> 13/10/09 14:32:02 INFO mapred.JobClient: Job complete:
> job_201310091401_0002
> 13/10/09 14:32:02 INFO mapred.JobClient: Counters: 6
> 13/10/09 14:32:02 INFO mapred.JobClient: Job Counters
> 13/10/09 14:32:02 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=622821
> 13/10/09 14:32:02 INFO mapred.JobClient: Total time spent by all
> reduces waiting after reserving slots (ms)=0
> 13/10/09 14:32:02 INFO mapred.JobClient: Total time spent by all
> maps waiting after reserving slots (ms)=0
> 13/10/09 14:32:02 INFO mapred.JobClient: Launched map tasks=2
> 13/10/09 14:32:02 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0
> 13/10/09 14:32:02 INFO mapred.JobClient: Failed map tasks=1
>
>
> I appreciate any help. Maybe I did it wrong.
> Andro.
>