You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@giraph.apache.org by Maria Stylianou <ma...@gmail.com> on 2013/02/13 12:04:06 UTC
InputFormat for the example SimpleMasterComputeVertex
Hey,
I am trying to run the example SimpleMasterComputeVertex, but no matter
which Input Format and graph I give, it doesn't work. Each worker gives the
error:
Caused by: java.lang.*NullPointerException*
at org.apache.giraph.examples.*SimpleMasterComputeVertex.compute*(SimpleMasterComputeVertex.java:*42*)
This line 42 is the first line of the compute()
public void compute(Iterable<DoubleWritable> messages){
So I guess, the initialization is not done correctly, because the input
file does not have the correct format.
Any help would be appreciated,
Thanks!
Maria
--
Maria Stylianou
Intern at Telefonica, Barcelona, Spain
Master Student of European Master in Distributed
Computing<http://www.kth.se/en/studies/programmes/master/em/emdc>
Universitat Politècnica de Catalunya - BarcelonaTech, Barcelona, Spain
KTH Royal Institute of Technology, Stockholm, Sweden
Re: InputFormat for the example SimpleMasterComputeVertex
Posted by Maria Stylianou <ma...@gmail.com>.
Sorry, I didn't set the parameters -mc for the masterCompute and the -wc
for the WorkerContext in the execution command.
Now, I'm running
hadoop jar target/giraph-0.2-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner
org.apache.giraph.examples.SimpleMasterComputeVertex
-vif org.apache.giraph.io.formats.*
JsonLongDoubleFloatDoubleVertexInputFormat* -vip
/user/hadoop/masterComputeInput
-of org.apache.giraph.io.formats.*
JsonLongDoubleFloatDoubleVertexOutputFormat* -op
/user/hadoop/masterComputeOutput
-w 3 *-mc org.apache.giraph.examples.SimpleMasterComputeVertex\$SimpleMasterCompute
-wc **
org.apache.giraph.examples.SimpleMasterComputeVertex\$SimpleMasterComputeWorkerContext
*
*
*
It works like a charm! :)
Now, I'm looking into the values that the aggregator gets. If I understand
correctly, in each superstep the aggregated value is: #_of_superstep/2+1.
So each vertex adds this aggregated value to its old value (initialized in
the file). And this is repeated in each superstep. Am I missing something?
On Wed, Feb 13, 2013 at 6:39 PM, Maja Kabiljo <ma...@fb.com> wrote:
> Hi Maria,
>
> You also need to set the MasterCompute class, because that's where
> aggregators get registered (just like it's done in
> TstBspBasic.testBspMasterCompute where this vertex is used).
>
> Maja
>
> From: Claudio Martella <cl...@gmail.com>
> Reply-To: "user@giraph.apache.org" <us...@giraph.apache.org>
> Date: Wednesday, February 13, 2013 6:56 AM
> To: "user@giraph.apache.org" <us...@giraph.apache.org>
> Subject: Re: InputFormat for the example SimpleMasterComputeVertex
>
> in the latest trunk, this is line 42:
>
> double newValue= this.<DoubleWritable>getAggregatedValue(SMC_AGG).get();
>
> What kind of input graph are you using?
>
>
> On Wed, Feb 13, 2013 at 1:25 PM, Maria Stylianou <ma...@gmail.com>wrote:
>
>> I didn't change anything in the code. I took the example as it is.
>> Actually, the whole error message is this one:
>>
>> java.lang.IllegalStateException: run: Caught an unrecoverable exception waitFor: ExecutionException occurred while waiting for org.apache.giraph.utils.ProgressableUtils$FutureWaitable@1ea326f9
>> at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102)
>> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:763)
>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:369)
>> at org.apache.hadoop.mapred.Child$4.run(Child.java:259)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Subject.java:415)
>> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
>> at org.apache.hadoop.mapred.Child.main(Child.java:253)
>> Caused by: java.lang.IllegalStateException: waitFor: ExecutionException occurred while waiting for org.apache.giraph.utils.ProgressableUtils$FutureWaitable@1ea326f9
>> at org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:145)
>> at org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:105)
>> at org.apache.giraph.utils.ProgressableUtils.getFutureResult(ProgressableUtils.java:67)
>> at org.apache.giraph.graph.GraphTaskManager.processGraphPartitions(GraphTaskManager.java:723)
>> at org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:270)
>> at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:92)
>> ... 7 more
>> Caused by: java.util.concurrent.ExecutionException: java.lang.NullPointerException
>> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:262)
>> at java.util.concurrent.FutureTask.get(FutureTask.java:119)
>> at org.apache.giraph.utils.ProgressableUtils$FutureWaitable.waitFor(ProgressableUtils.java:232)
>> at org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:137)
>> ... 12 more
>> Caused by: java.lang.NullPointerException
>> at org.apache.giraph.examples.SimpleMasterComputeVertex.compute(SimpleMasterComputeVertex.java:42)
>> at org.apache.giraph.graph.ComputeCallable.computePartition(ComputeCallable.java:204)
>> at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:149)
>> at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:63)
>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>> at java.lang.Thread.run(Thread.java:722)
>>
>>
>>
>>
>> On Wed, Feb 13, 2013 at 1:03 PM, Claudio Martella <
>> claudio.martella@gmail.com> wrote:
>>
>>> Are you sure that is the right line? I do not see how that would cause
>>> an exception, most probably a for loop on messages right after?
>>>
>>>
>>> On Wed, Feb 13, 2013 at 12:04 PM, Maria Stylianou <ma...@gmail.com>wrote:
>>>
>>>> Hey,
>>>>
>>>> I am trying to run the example SimpleMasterComputeVertex, but no
>>>> matter which Input Format and graph I give, it doesn't work. Each worker
>>>> gives the error:
>>>>
>>>> Caused by: java.lang.*NullPointerException*
>>>> at org.apache.giraph.examples.*SimpleMasterComputeVertex.compute*(SimpleMasterComputeVertex.java:*42*)
>>>>
>>>> This line 42 is the first line of the compute()
>>>> public void compute(Iterable<DoubleWritable> messages){
>>>>
>>>> So I guess, the initialization is not done correctly, because the
>>>> input file does not have the correct format.
>>>>
>>>> Any help would be appreciated,
>>>> Thanks!
>>>> Maria
>>>> --
>>>> Maria Stylianou
>>>> Intern at Telefonica, Barcelona, Spain
>>>> Master Student of European Master in Distributed Computing<http://www.kth.se/en/studies/programmes/master/em/emdc>
>>>> Universitat Politècnica de Catalunya - BarcelonaTech, Barcelona, Spain
>>>> KTH Royal Institute of Technology, Stockholm, Sweden
>>>>
>>>>
>>>
>>>
>>> --
>>> Claudio Martella
>>> claudio.martella@gmail.com
>>>
>>
>>
>>
>> --
>> Maria Stylianou
>> Intern at Telefonica, Barcelona, Spain
>> Master Student of European Master in Distributed Computing<http://www.kth.se/en/studies/programmes/master/em/emdc>
>> Universitat Politècnica de Catalunya - BarcelonaTech, Barcelona, Spain
>> KTH Royal Institute of Technology, Stockholm, Sweden
>>
>
>
>
> --
> Claudio Martella
> claudio.martella@gmail.com
>
--
Maria Stylianou
Intern at Telefonica, Barcelona, Spain
Master Student of European Master in Distributed
Computing<http://www.kth.se/en/studies/programmes/master/em/emdc>
Universitat Politècnica de Catalunya - BarcelonaTech, Barcelona, Spain
KTH Royal Institute of Technology, Stockholm, Sweden
Phone: +34 622 211 885
Re: InputFormat for the example SimpleMasterComputeVertex
Posted by Maja Kabiljo <ma...@fb.com>.
Hi Maria,
You also need to set the MasterCompute class, because that's where aggregators get registered (just like it's done in TstBspBasic.testBspMasterCompute where this vertex is used).
Maja
From: Claudio Martella <cl...@gmail.com>>
Reply-To: "user@giraph.apache.org<ma...@giraph.apache.org>" <us...@giraph.apache.org>>
Date: Wednesday, February 13, 2013 6:56 AM
To: "user@giraph.apache.org<ma...@giraph.apache.org>" <us...@giraph.apache.org>>
Subject: Re: InputFormat for the example SimpleMasterComputeVertex
in the latest trunk, this is line 42:
double newValue= this.<DoubleWritable>getAggregatedValue(SMC_AGG).get();
What kind of input graph are you using?
On Wed, Feb 13, 2013 at 1:25 PM, Maria Stylianou <ma...@gmail.com>> wrote:
I didn't change anything in the code. I took the example as it is.
Actually, the whole error message is this one:
java.lang.IllegalStateException: run: Caught an unrecoverable exception waitFor: ExecutionException occurred while waiting for org.apache.giraph.utils.ProgressableUtils$FutureWaitable@1ea326f9
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:763)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:369)
at org.apache.hadoop.mapred.Child$4.run(Child.java:259)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
at org.apache.hadoop.mapred.Child.main(Child.java:253)
Caused by: java.lang.IllegalStateException: waitFor: ExecutionException occurred while waiting for org.apache.giraph.utils.ProgressableUtils$FutureWaitable@1ea326f9
at org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:145)
at org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:105)
at org.apache.giraph.utils.ProgressableUtils.getFutureResult(ProgressableUtils.java:67)
at org.apache.giraph.graph.GraphTaskManager.processGraphPartitions(GraphTaskManager.java:723)
at org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:270)
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:92)
... 7 more
Caused by: java.util.concurrent.ExecutionException: java.lang.NullPointerException
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:262)
at java.util.concurrent.FutureTask.get(FutureTask.java:119)
at org.apache.giraph.utils.ProgressableUtils$FutureWaitable.waitFor(ProgressableUtils.java:232)
at org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:137)
... 12 more
Caused by: java.lang.NullPointerException
at org.apache.giraph.examples.SimpleMasterComputeVertex.compute(SimpleMasterComputeVertex.java:42)
at org.apache.giraph.graph.ComputeCallable.computePartition(ComputeCallable.java:204)
at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:149)
at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:63)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
On Wed, Feb 13, 2013 at 1:03 PM, Claudio Martella <cl...@gmail.com>> wrote:
Are you sure that is the right line? I do not see how that would cause an exception, most probably a for loop on messages right after?
On Wed, Feb 13, 2013 at 12:04 PM, Maria Stylianou <ma...@gmail.com>> wrote:
Hey,
I am trying to run the example SimpleMasterComputeVertex, but no matter which Input Format and graph I give, it doesn't work. Each worker gives the error:
Caused by: java.lang.NullPointerException
at org.apache.giraph.examples.SimpleMasterComputeVertex.compute(SimpleMasterComputeVertex.java:42)
This line 42 is the first line of the compute()
public void compute(Iterable<DoubleWritable> messages){
So I guess, the initialization is not done correctly, because the input file does not have the correct format.
Any help would be appreciated,
Thanks!
Maria
--
Maria Stylianou
Intern at Telefonica, Barcelona, Spain
Master Student of European Master in Distributed Computing<http://www.kth.se/en/studies/programmes/master/em/emdc>
Universitat Politècnica de Catalunya - BarcelonaTech, Barcelona, Spain
KTH Royal Institute of Technology, Stockholm, Sweden
--
Claudio Martella
claudio.martella@gmail.com<ma...@gmail.com>
--
Maria Stylianou
Intern at Telefonica, Barcelona, Spain
Master Student of European Master in Distributed Computing<http://www.kth.se/en/studies/programmes/master/em/emdc>
Universitat Politècnica de Catalunya - BarcelonaTech, Barcelona, Spain
KTH Royal Institute of Technology, Stockholm, Sweden
--
Claudio Martella
claudio.martella@gmail.com<ma...@gmail.com>
Re: InputFormat for the example SimpleMasterComputeVertex
Posted by Claudio Martella <cl...@gmail.com>.
in the latest trunk, this is line 42:
double newValue = this.<DoubleWritable>getAggregatedValue(SMC_AGG).get();
What kind of input graph are you using?
On Wed, Feb 13, 2013 at 1:25 PM, Maria Stylianou <ma...@gmail.com> wrote:
> I didn't change anything in the code. I took the example as it is.
> Actually, the whole error message is this one:
>
> java.lang.IllegalStateException: run: Caught an unrecoverable exception waitFor: ExecutionException occurred while waiting for org.apache.giraph.utils.ProgressableUtils$FutureWaitable@1ea326f9
> at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:763)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:369)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:259)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
> at org.apache.hadoop.mapred.Child.main(Child.java:253)
> Caused by: java.lang.IllegalStateException: waitFor: ExecutionException occurred while waiting for org.apache.giraph.utils.ProgressableUtils$FutureWaitable@1ea326f9
> at org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:145)
> at org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:105)
> at org.apache.giraph.utils.ProgressableUtils.getFutureResult(ProgressableUtils.java:67)
> at org.apache.giraph.graph.GraphTaskManager.processGraphPartitions(GraphTaskManager.java:723)
> at org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:270)
> at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:92)
> ... 7 more
> Caused by: java.util.concurrent.ExecutionException: java.lang.NullPointerException
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:262)
> at java.util.concurrent.FutureTask.get(FutureTask.java:119)
> at org.apache.giraph.utils.ProgressableUtils$FutureWaitable.waitFor(ProgressableUtils.java:232)
> at org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:137)
> ... 12 more
> Caused by: java.lang.NullPointerException
> at org.apache.giraph.examples.SimpleMasterComputeVertex.compute(SimpleMasterComputeVertex.java:42)
> at org.apache.giraph.graph.ComputeCallable.computePartition(ComputeCallable.java:204)
> at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:149)
> at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:63)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
>
>
>
>
> On Wed, Feb 13, 2013 at 1:03 PM, Claudio Martella <
> claudio.martella@gmail.com> wrote:
>
>> Are you sure that is the right line? I do not see how that would cause an
>> exception, most probably a for loop on messages right after?
>>
>>
>> On Wed, Feb 13, 2013 at 12:04 PM, Maria Stylianou <ma...@gmail.com>wrote:
>>
>>> Hey,
>>>
>>> I am trying to run the example SimpleMasterComputeVertex, but no matter
>>> which Input Format and graph I give, it doesn't work. Each worker gives the
>>> error:
>>>
>>> Caused by: java.lang.*NullPointerException*
>>> at org.apache.giraph.examples.*SimpleMasterComputeVertex.compute*(SimpleMasterComputeVertex.java:*42*)
>>>
>>> This line 42 is the first line of the compute()
>>> public void compute(Iterable<DoubleWritable> messages){
>>>
>>> So I guess, the initialization is not done correctly, because the input
>>> file does not have the correct format.
>>>
>>> Any help would be appreciated,
>>> Thanks!
>>> Maria
>>> --
>>> Maria Stylianou
>>> Intern at Telefonica, Barcelona, Spain
>>> Master Student of European Master in Distributed Computing<http://www.kth.se/en/studies/programmes/master/em/emdc>
>>> Universitat Politècnica de Catalunya - BarcelonaTech, Barcelona, Spain
>>> KTH Royal Institute of Technology, Stockholm, Sweden
>>>
>>>
>>
>>
>> --
>> Claudio Martella
>> claudio.martella@gmail.com
>>
>
>
>
> --
> Maria Stylianou
> Intern at Telefonica, Barcelona, Spain
> Master Student of European Master in Distributed Computing<http://www.kth.se/en/studies/programmes/master/em/emdc>
> Universitat Politècnica de Catalunya - BarcelonaTech, Barcelona, Spain
> KTH Royal Institute of Technology, Stockholm, Sweden
>
--
Claudio Martella
claudio.martella@gmail.com
Re: InputFormat for the example SimpleMasterComputeVertex
Posted by Maria Stylianou <ma...@gmail.com>.
I didn't change anything in the code. I took the example as it is.
Actually, the whole error message is this one:
java.lang.IllegalStateException: run: Caught an unrecoverable
exception waitFor: ExecutionException occurred while waiting for
org.apache.giraph.utils.ProgressableUtils$FutureWaitable@1ea326f9
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:763)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:369)
at org.apache.hadoop.mapred.Child$4.run(Child.java:259)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
at org.apache.hadoop.mapred.Child.main(Child.java:253)
Caused by: java.lang.IllegalStateException: waitFor:
ExecutionException occurred while waiting for
org.apache.giraph.utils.ProgressableUtils$FutureWaitable@1ea326f9
at org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:145)
at org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:105)
at org.apache.giraph.utils.ProgressableUtils.getFutureResult(ProgressableUtils.java:67)
at org.apache.giraph.graph.GraphTaskManager.processGraphPartitions(GraphTaskManager.java:723)
at org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:270)
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:92)
... 7 more
Caused by: java.util.concurrent.ExecutionException:
java.lang.NullPointerException
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:262)
at java.util.concurrent.FutureTask.get(FutureTask.java:119)
at org.apache.giraph.utils.ProgressableUtils$FutureWaitable.waitFor(ProgressableUtils.java:232)
at org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:137)
... 12 more
Caused by: java.lang.NullPointerException
at org.apache.giraph.examples.SimpleMasterComputeVertex.compute(SimpleMasterComputeVertex.java:42)
at org.apache.giraph.graph.ComputeCallable.computePartition(ComputeCallable.java:204)
at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:149)
at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:63)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
On Wed, Feb 13, 2013 at 1:03 PM, Claudio Martella <
claudio.martella@gmail.com> wrote:
> Are you sure that is the right line? I do not see how that would cause an
> exception, most probably a for loop on messages right after?
>
>
> On Wed, Feb 13, 2013 at 12:04 PM, Maria Stylianou <ma...@gmail.com>wrote:
>
>> Hey,
>>
>> I am trying to run the example SimpleMasterComputeVertex, but no matter
>> which Input Format and graph I give, it doesn't work. Each worker gives the
>> error:
>>
>> Caused by: java.lang.*NullPointerException*
>> at org.apache.giraph.examples.*SimpleMasterComputeVertex.compute*(SimpleMasterComputeVertex.java:*42*)
>>
>> This line 42 is the first line of the compute()
>> public void compute(Iterable<DoubleWritable> messages){
>>
>> So I guess, the initialization is not done correctly, because the input
>> file does not have the correct format.
>>
>> Any help would be appreciated,
>> Thanks!
>> Maria
>> --
>> Maria Stylianou
>> Intern at Telefonica, Barcelona, Spain
>> Master Student of European Master in Distributed Computing<http://www.kth.se/en/studies/programmes/master/em/emdc>
>> Universitat Politècnica de Catalunya - BarcelonaTech, Barcelona, Spain
>> KTH Royal Institute of Technology, Stockholm, Sweden
>>
>>
>
>
> --
> Claudio Martella
> claudio.martella@gmail.com
>
--
Maria Stylianou
Intern at Telefonica, Barcelona, Spain
Master Student of European Master in Distributed
Computing<http://www.kth.se/en/studies/programmes/master/em/emdc>
Universitat Politècnica de Catalunya - BarcelonaTech, Barcelona, Spain
KTH Royal Institute of Technology, Stockholm, Sweden
Re: InputFormat for the example SimpleMasterComputeVertex
Posted by Claudio Martella <cl...@gmail.com>.
Are you sure that is the right line? I do not see how that would cause an
exception, most probably a for loop on messages right after?
On Wed, Feb 13, 2013 at 12:04 PM, Maria Stylianou <ma...@gmail.com> wrote:
> Hey,
>
> I am trying to run the example SimpleMasterComputeVertex, but no matter
> which Input Format and graph I give, it doesn't work. Each worker gives the
> error:
>
> Caused by: java.lang.*NullPointerException*
> at org.apache.giraph.examples.*SimpleMasterComputeVertex.compute*(SimpleMasterComputeVertex.java:*42*)
>
> This line 42 is the first line of the compute()
> public void compute(Iterable<DoubleWritable> messages){
>
> So I guess, the initialization is not done correctly, because the input
> file does not have the correct format.
>
> Any help would be appreciated,
> Thanks!
> Maria
> --
> Maria Stylianou
> Intern at Telefonica, Barcelona, Spain
> Master Student of European Master in Distributed Computing<http://www.kth.se/en/studies/programmes/master/em/emdc>
> Universitat Politècnica de Catalunya - BarcelonaTech, Barcelona, Spain
> KTH Royal Institute of Technology, Stockholm, Sweden
>
>
--
Claudio Martella
claudio.martella@gmail.com
Re: InputFormat for the example SimpleMasterComputeVertex
Posted by Eli Reisman <ap...@gmail.com>.
Its fantastic thanks again! We need more wiki pages like that!!! This will
save so many emails on the list too! I need to (and will) update the graph
representations page to say we support multigraph, edge and vertex IO, etc.
soon. A lot of our wiki info is old enough to be confusing people now.
On Sat, Mar 2, 2013 at 1:57 PM, Eli Reisman <ap...@gmail.com>wrote:
> Thanks for doing this Maja!
>
>
> On Thu, Feb 21, 2013 at 5:12 PM, Maja Kabiljo <ma...@fb.com> wrote:
>
>> I wrote up a basic info about aggregators:
>> https://cwiki.apache.org/confluence/display/GIRAPH/Aggregators. Please
>> take a look, and let me know if something needs to be changed / improved.
>>
>> From: Eli Reisman <ap...@gmail.com>
>> Reply-To: "user@giraph.apache.org" <us...@giraph.apache.org>
>> Date: Thursday, February 21, 2013 4:21 PM
>>
>> To: "user@giraph.apache.org" <us...@giraph.apache.org>
>> Subject: Re: InputFormat for the example SimpleMasterComputeVertex
>>
>> That sounds great to me, maybe just a mention in the wiki that the two
>> functionalities are tied together will help the idea click for people.
>> Either way this will be a big help I think.
>>
>>
>> On Thu, Feb 21, 2013 at 3:24 PM, Maja Kabiljo <ma...@fb.com> wrote:
>>
>>> Eli, that's an interesting idea, we could have some class which user
>>> extends and which is there only for aggregator registration. Sometimes we
>>> want to register some aggregators later on during the computation, so we
>>> need to keep allowing registration from masterCompute too.
>>>
>>> But I think for users the biggest problem is to realize that they have
>>> to extend and set MasterCompute/this new class in order to use aggregators.
>>> Currently, if user tries to aggregate a value to unregistered aggregator he
>>> will get an exception, but if he tries to get the value of unregistered
>>> aggregator he will just get null. So maybe adding a warning message in that
>>> case, together with a wiki page, might be enough? What do you think?
>>>
>>> From: Eli Reisman <ap...@gmail.com>
>>> Reply-To: "user@giraph.apache.org" <us...@giraph.apache.org>
>>> Date: Thursday, February 21, 2013 10:25 AM
>>>
>>> To: "user@giraph.apache.org" <us...@giraph.apache.org>
>>> Subject: Re: InputFormat for the example SimpleMasterComputeVertex
>>>
>>> Thanks for the explanation, that makes sense. I would love to see a
>>> wiki page at some point, you have so much knowledge of this piece of Giraph
>>> from all your dev work on it and have also the additional bonus of
>>> experience running big cluster jobs using these features so you have a lot
>>> of insight to share.
>>>
>>> Would there be any point to a future JIRA to break out the aggregator
>>> registration from the master compute stuff, at least from the user's view?
>>> Or is it not that confusing once you've used them a bit?
>>>
>>>
>>> On Thu, Feb 14, 2013 at 4:52 PM, Maja Kabiljo <ma...@fb.com>wrote:
>>>
>>>> Progressable exception can be caused by many different reasons (it's
>>>> totally unrelated to aggregators), and when looking at which exception it's
>>>> caused by users should get better sense about what's going on.
>>>> What you are suggesting about providing default master compute is not
>>>> doable, since the part which needs to be done there is aggregator
>>>> registration. We can't know what kind of aggregators (names and types) an
>>>> application needs.
>>>> I remember I was talking about writing a short tutorial for aggregators
>>>> long time ago, sorry for not doing that, will try to get to it soon.
>>>>
>>>> From: Eli Reisman <ap...@gmail.com>
>>>> Reply-To: "user@giraph.apache.org" <us...@giraph.apache.org>
>>>> Date: Thursday, February 14, 2013 2:23 PM
>>>>
>>>> To: "user@giraph.apache.org" <us...@giraph.apache.org>
>>>> Subject: Re: InputFormat for the example SimpleMasterComputeVertex
>>>>
>>>> Other folks on the list are also having this problem with the
>>>> progressable utile exception & job failures. I don't know much about master
>>>> compute usage but if it is needed to make the aggregators work, maybe we
>>>> should have a default dummy class that just handles aggregators if no other
>>>> master compute is specified? Or a wiki page? The progressable error message
>>>> does not lead us to this conclusion directly.
>>>>
>>>> On Wed, Feb 13, 2013 at 3:04 AM, Maria Stylianou <ma...@gmail.com>wrote:
>>>>
>>>>> Hey,
>>>>>
>>>>> I am trying to run the example SimpleMasterComputeVertex, but no
>>>>> matter which Input Format and graph I give, it doesn't work. Each worker
>>>>> gives the error:
>>>>>
>>>>> Caused by: java.lang.*NullPointerException*
>>>>> at org.apache.giraph.examples.*SimpleMasterComputeVertex.compute*(SimpleMasterComputeVertex.java:*42*)
>>>>>
>>>>> This line 42 is the first line of the compute()
>>>>> public void compute(Iterable<DoubleWritable> messages){
>>>>>
>>>>> So I guess, the initialization is not done correctly, because the
>>>>> input file does not have the correct format.
>>>>>
>>>>> Any help would be appreciated,
>>>>> Thanks!
>>>>> Maria
>>>>> --
>>>>> Maria Stylianou
>>>>> Intern at Telefonica, Barcelona, Spain
>>>>> Master Student of European Master in Distributed Computing<http://www.kth.se/en/studies/programmes/master/em/emdc>
>>>>> Universitat Politècnica de Catalunya - BarcelonaTech, Barcelona, Spain
>>>>> KTH Royal Institute of Technology, Stockholm, Sweden
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: InputFormat for the example SimpleMasterComputeVertex
Posted by Eli Reisman <ap...@gmail.com>.
Thanks for doing this Maja!
On Thu, Feb 21, 2013 at 5:12 PM, Maja Kabiljo <ma...@fb.com> wrote:
> I wrote up a basic info about aggregators:
> https://cwiki.apache.org/confluence/display/GIRAPH/Aggregators. Please
> take a look, and let me know if something needs to be changed / improved.
>
> From: Eli Reisman <ap...@gmail.com>
> Reply-To: "user@giraph.apache.org" <us...@giraph.apache.org>
> Date: Thursday, February 21, 2013 4:21 PM
>
> To: "user@giraph.apache.org" <us...@giraph.apache.org>
> Subject: Re: InputFormat for the example SimpleMasterComputeVertex
>
> That sounds great to me, maybe just a mention in the wiki that the two
> functionalities are tied together will help the idea click for people.
> Either way this will be a big help I think.
>
>
> On Thu, Feb 21, 2013 at 3:24 PM, Maja Kabiljo <ma...@fb.com> wrote:
>
>> Eli, that's an interesting idea, we could have some class which user
>> extends and which is there only for aggregator registration. Sometimes we
>> want to register some aggregators later on during the computation, so we
>> need to keep allowing registration from masterCompute too.
>>
>> But I think for users the biggest problem is to realize that they have
>> to extend and set MasterCompute/this new class in order to use aggregators.
>> Currently, if user tries to aggregate a value to unregistered aggregator he
>> will get an exception, but if he tries to get the value of unregistered
>> aggregator he will just get null. So maybe adding a warning message in that
>> case, together with a wiki page, might be enough? What do you think?
>>
>> From: Eli Reisman <ap...@gmail.com>
>> Reply-To: "user@giraph.apache.org" <us...@giraph.apache.org>
>> Date: Thursday, February 21, 2013 10:25 AM
>>
>> To: "user@giraph.apache.org" <us...@giraph.apache.org>
>> Subject: Re: InputFormat for the example SimpleMasterComputeVertex
>>
>> Thanks for the explanation, that makes sense. I would love to see a
>> wiki page at some point, you have so much knowledge of this piece of Giraph
>> from all your dev work on it and have also the additional bonus of
>> experience running big cluster jobs using these features so you have a lot
>> of insight to share.
>>
>> Would there be any point to a future JIRA to break out the aggregator
>> registration from the master compute stuff, at least from the user's view?
>> Or is it not that confusing once you've used them a bit?
>>
>>
>> On Thu, Feb 14, 2013 at 4:52 PM, Maja Kabiljo <ma...@fb.com> wrote:
>>
>>> Progressable exception can be caused by many different reasons (it's
>>> totally unrelated to aggregators), and when looking at which exception it's
>>> caused by users should get better sense about what's going on.
>>> What you are suggesting about providing default master compute is not
>>> doable, since the part which needs to be done there is aggregator
>>> registration. We can't know what kind of aggregators (names and types) an
>>> application needs.
>>> I remember I was talking about writing a short tutorial for aggregators
>>> long time ago, sorry for not doing that, will try to get to it soon.
>>>
>>> From: Eli Reisman <ap...@gmail.com>
>>> Reply-To: "user@giraph.apache.org" <us...@giraph.apache.org>
>>> Date: Thursday, February 14, 2013 2:23 PM
>>>
>>> To: "user@giraph.apache.org" <us...@giraph.apache.org>
>>> Subject: Re: InputFormat for the example SimpleMasterComputeVertex
>>>
>>> Other folks on the list are also having this problem with the
>>> progressable utile exception & job failures. I don't know much about master
>>> compute usage but if it is needed to make the aggregators work, maybe we
>>> should have a default dummy class that just handles aggregators if no other
>>> master compute is specified? Or a wiki page? The progressable error message
>>> does not lead us to this conclusion directly.
>>>
>>> On Wed, Feb 13, 2013 at 3:04 AM, Maria Stylianou <ma...@gmail.com>wrote:
>>>
>>>> Hey,
>>>>
>>>> I am trying to run the example SimpleMasterComputeVertex, but no
>>>> matter which Input Format and graph I give, it doesn't work. Each worker
>>>> gives the error:
>>>>
>>>> Caused by: java.lang.*NullPointerException*
>>>> at org.apache.giraph.examples.*SimpleMasterComputeVertex.compute*(SimpleMasterComputeVertex.java:*42*)
>>>>
>>>> This line 42 is the first line of the compute()
>>>> public void compute(Iterable<DoubleWritable> messages){
>>>>
>>>> So I guess, the initialization is not done correctly, because the
>>>> input file does not have the correct format.
>>>>
>>>> Any help would be appreciated,
>>>> Thanks!
>>>> Maria
>>>> --
>>>> Maria Stylianou
>>>> Intern at Telefonica, Barcelona, Spain
>>>> Master Student of European Master in Distributed Computing<http://www.kth.se/en/studies/programmes/master/em/emdc>
>>>> Universitat Politècnica de Catalunya - BarcelonaTech, Barcelona, Spain
>>>> KTH Royal Institute of Technology, Stockholm, Sweden
>>>>
>>>>
>>>
>>
>
Re: InputFormat for the example SimpleMasterComputeVertex
Posted by Maja Kabiljo <ma...@fb.com>.
I wrote up a basic info about aggregators: https://cwiki.apache.org/confluence/display/GIRAPH/Aggregators. Please take a look, and let me know if something needs to be changed / improved.
From: Eli Reisman <ap...@gmail.com>>
Reply-To: "user@giraph.apache.org<ma...@giraph.apache.org>" <us...@giraph.apache.org>>
Date: Thursday, February 21, 2013 4:21 PM
To: "user@giraph.apache.org<ma...@giraph.apache.org>" <us...@giraph.apache.org>>
Subject: Re: InputFormat for the example SimpleMasterComputeVertex
That sounds great to me, maybe just a mention in the wiki that the two functionalities are tied together will help the idea click for people. Either way this will be a big help I think.
On Thu, Feb 21, 2013 at 3:24 PM, Maja Kabiljo <ma...@fb.com>> wrote:
Eli, that's an interesting idea, we could have some class which user extends and which is there only for aggregator registration. Sometimes we want to register some aggregators later on during the computation, so we need to keep allowing registration from masterCompute too.
But I think for users the biggest problem is to realize that they have to extend and set MasterCompute/this new class in order to use aggregators. Currently, if user tries to aggregate a value to unregistered aggregator he will get an exception, but if he tries to get the value of unregistered aggregator he will just get null. So maybe adding a warning message in that case, together with a wiki page, might be enough? What do you think?
From: Eli Reisman <ap...@gmail.com>>
Reply-To: "user@giraph.apache.org<ma...@giraph.apache.org>" <us...@giraph.apache.org>>
Date: Thursday, February 21, 2013 10:25 AM
To: "user@giraph.apache.org<ma...@giraph.apache.org>" <us...@giraph.apache.org>>
Subject: Re: InputFormat for the example SimpleMasterComputeVertex
Thanks for the explanation, that makes sense. I would love to see a wiki page at some point, you have so much knowledge of this piece of Giraph from all your dev work on it and have also the additional bonus of experience running big cluster jobs using these features so you have a lot of insight to share.
Would there be any point to a future JIRA to break out the aggregator registration from the master compute stuff, at least from the user's view? Or is it not that confusing once you've used them a bit?
On Thu, Feb 14, 2013 at 4:52 PM, Maja Kabiljo <ma...@fb.com>> wrote:
Progressable exception can be caused by many different reasons (it's totally unrelated to aggregators), and when looking at which exception it's caused by users should get better sense about what's going on.
What you are suggesting about providing default master compute is not doable, since the part which needs to be done there is aggregator registration. We can't know what kind of aggregators (names and types) an application needs.
I remember I was talking about writing a short tutorial for aggregators long time ago, sorry for not doing that, will try to get to it soon.
From: Eli Reisman <ap...@gmail.com>>
Reply-To: "user@giraph.apache.org<ma...@giraph.apache.org>" <us...@giraph.apache.org>>
Date: Thursday, February 14, 2013 2:23 PM
To: "user@giraph.apache.org<ma...@giraph.apache.org>" <us...@giraph.apache.org>>
Subject: Re: InputFormat for the example SimpleMasterComputeVertex
Other folks on the list are also having this problem with the progressable utile exception & job failures. I don't know much about master compute usage but if it is needed to make the aggregators work, maybe we should have a default dummy class that just handles aggregators if no other master compute is specified? Or a wiki page? The progressable error message does not lead us to this conclusion directly.
On Wed, Feb 13, 2013 at 3:04 AM, Maria Stylianou <ma...@gmail.com>> wrote:
Hey,
I am trying to run the example SimpleMasterComputeVertex, but no matter which Input Format and graph I give, it doesn't work. Each worker gives the error:
Caused by: java.lang.NullPointerException
at org.apache.giraph.examples.SimpleMasterComputeVertex.compute(SimpleMasterComputeVertex.java:42)
This line 42 is the first line of the compute()
public void compute(Iterable<DoubleWritable> messages){
So I guess, the initialization is not done correctly, because the input file does not have the correct format.
Any help would be appreciated,
Thanks!
Maria
--
Maria Stylianou
Intern at Telefonica, Barcelona, Spain
Master Student of European Master in Distributed Computing<http://www.kth.se/en/studies/programmes/master/em/emdc>
Universitat Politècnica de Catalunya - BarcelonaTech, Barcelona, Spain
KTH Royal Institute of Technology, Stockholm, Sweden
Re: InputFormat for the example SimpleMasterComputeVertex
Posted by Eli Reisman <ap...@gmail.com>.
That sounds great to me, maybe just a mention in the wiki that the two
functionalities are tied together will help the idea click for people.
Either way this will be a big help I think.
On Thu, Feb 21, 2013 at 3:24 PM, Maja Kabiljo <ma...@fb.com> wrote:
> Eli, that's an interesting idea, we could have some class which user
> extends and which is there only for aggregator registration. Sometimes we
> want to register some aggregators later on during the computation, so we
> need to keep allowing registration from masterCompute too.
>
> But I think for users the biggest problem is to realize that they have
> to extend and set MasterCompute/this new class in order to use aggregators.
> Currently, if user tries to aggregate a value to unregistered aggregator he
> will get an exception, but if he tries to get the value of unregistered
> aggregator he will just get null. So maybe adding a warning message in that
> case, together with a wiki page, might be enough? What do you think?
>
> From: Eli Reisman <ap...@gmail.com>
> Reply-To: "user@giraph.apache.org" <us...@giraph.apache.org>
> Date: Thursday, February 21, 2013 10:25 AM
>
> To: "user@giraph.apache.org" <us...@giraph.apache.org>
> Subject: Re: InputFormat for the example SimpleMasterComputeVertex
>
> Thanks for the explanation, that makes sense. I would love to see a wiki
> page at some point, you have so much knowledge of this piece of Giraph from
> all your dev work on it and have also the additional bonus of experience
> running big cluster jobs using these features so you have a lot of insight
> to share.
>
> Would there be any point to a future JIRA to break out the aggregator
> registration from the master compute stuff, at least from the user's view?
> Or is it not that confusing once you've used them a bit?
>
>
> On Thu, Feb 14, 2013 at 4:52 PM, Maja Kabiljo <ma...@fb.com> wrote:
>
>> Progressable exception can be caused by many different reasons (it's
>> totally unrelated to aggregators), and when looking at which exception it's
>> caused by users should get better sense about what's going on.
>> What you are suggesting about providing default master compute is not
>> doable, since the part which needs to be done there is aggregator
>> registration. We can't know what kind of aggregators (names and types) an
>> application needs.
>> I remember I was talking about writing a short tutorial for aggregators
>> long time ago, sorry for not doing that, will try to get to it soon.
>>
>> From: Eli Reisman <ap...@gmail.com>
>> Reply-To: "user@giraph.apache.org" <us...@giraph.apache.org>
>> Date: Thursday, February 14, 2013 2:23 PM
>>
>> To: "user@giraph.apache.org" <us...@giraph.apache.org>
>> Subject: Re: InputFormat for the example SimpleMasterComputeVertex
>>
>> Other folks on the list are also having this problem with the
>> progressable utile exception & job failures. I don't know much about master
>> compute usage but if it is needed to make the aggregators work, maybe we
>> should have a default dummy class that just handles aggregators if no other
>> master compute is specified? Or a wiki page? The progressable error message
>> does not lead us to this conclusion directly.
>>
>> On Wed, Feb 13, 2013 at 3:04 AM, Maria Stylianou <ma...@gmail.com>wrote:
>>
>>> Hey,
>>>
>>> I am trying to run the example SimpleMasterComputeVertex, but no
>>> matter which Input Format and graph I give, it doesn't work. Each worker
>>> gives the error:
>>>
>>> Caused by: java.lang.*NullPointerException*
>>> at org.apache.giraph.examples.*SimpleMasterComputeVertex.compute*(SimpleMasterComputeVertex.java:*42*)
>>>
>>> This line 42 is the first line of the compute()
>>> public void compute(Iterable<DoubleWritable> messages){
>>>
>>> So I guess, the initialization is not done correctly, because the
>>> input file does not have the correct format.
>>>
>>> Any help would be appreciated,
>>> Thanks!
>>> Maria
>>> --
>>> Maria Stylianou
>>> Intern at Telefonica, Barcelona, Spain
>>> Master Student of European Master in Distributed Computing<http://www.kth.se/en/studies/programmes/master/em/emdc>
>>> Universitat Politècnica de Catalunya - BarcelonaTech, Barcelona, Spain
>>> KTH Royal Institute of Technology, Stockholm, Sweden
>>>
>>>
>>
>
Re: InputFormat for the example SimpleMasterComputeVertex
Posted by Maja Kabiljo <ma...@fb.com>.
Eli, that's an interesting idea, we could have some class which user extends and which is there only for aggregator registration. Sometimes we want to register some aggregators later on during the computation, so we need to keep allowing registration from masterCompute too.
But I think for users the biggest problem is to realize that they have to extend and set MasterCompute/this new class in order to use aggregators. Currently, if user tries to aggregate a value to unregistered aggregator he will get an exception, but if he tries to get the value of unregistered aggregator he will just get null. So maybe adding a warning message in that case, together with a wiki page, might be enough? What do you think?
From: Eli Reisman <ap...@gmail.com>>
Reply-To: "user@giraph.apache.org<ma...@giraph.apache.org>" <us...@giraph.apache.org>>
Date: Thursday, February 21, 2013 10:25 AM
To: "user@giraph.apache.org<ma...@giraph.apache.org>" <us...@giraph.apache.org>>
Subject: Re: InputFormat for the example SimpleMasterComputeVertex
Thanks for the explanation, that makes sense. I would love to see a wiki page at some point, you have so much knowledge of this piece of Giraph from all your dev work on it and have also the additional bonus of experience running big cluster jobs using these features so you have a lot of insight to share.
Would there be any point to a future JIRA to break out the aggregator registration from the master compute stuff, at least from the user's view? Or is it not that confusing once you've used them a bit?
On Thu, Feb 14, 2013 at 4:52 PM, Maja Kabiljo <ma...@fb.com>> wrote:
Progressable exception can be caused by many different reasons (it's totally unrelated to aggregators), and when looking at which exception it's caused by users should get better sense about what's going on.
What you are suggesting about providing default master compute is not doable, since the part which needs to be done there is aggregator registration. We can't know what kind of aggregators (names and types) an application needs.
I remember I was talking about writing a short tutorial for aggregators long time ago, sorry for not doing that, will try to get to it soon.
From: Eli Reisman <ap...@gmail.com>>
Reply-To: "user@giraph.apache.org<ma...@giraph.apache.org>" <us...@giraph.apache.org>>
Date: Thursday, February 14, 2013 2:23 PM
To: "user@giraph.apache.org<ma...@giraph.apache.org>" <us...@giraph.apache.org>>
Subject: Re: InputFormat for the example SimpleMasterComputeVertex
Other folks on the list are also having this problem with the progressable utile exception & job failures. I don't know much about master compute usage but if it is needed to make the aggregators work, maybe we should have a default dummy class that just handles aggregators if no other master compute is specified? Or a wiki page? The progressable error message does not lead us to this conclusion directly.
On Wed, Feb 13, 2013 at 3:04 AM, Maria Stylianou <ma...@gmail.com>> wrote:
Hey,
I am trying to run the example SimpleMasterComputeVertex, but no matter which Input Format and graph I give, it doesn't work. Each worker gives the error:
Caused by: java.lang.NullPointerException
at org.apache.giraph.examples.SimpleMasterComputeVertex.compute(SimpleMasterComputeVertex.java:42)
This line 42 is the first line of the compute()
public void compute(Iterable<DoubleWritable> messages){
So I guess, the initialization is not done correctly, because the input file does not have the correct format.
Any help would be appreciated,
Thanks!
Maria
--
Maria Stylianou
Intern at Telefonica, Barcelona, Spain
Master Student of European Master in Distributed Computing<http://www.kth.se/en/studies/programmes/master/em/emdc>
Universitat Politècnica de Catalunya - BarcelonaTech, Barcelona, Spain
KTH Royal Institute of Technology, Stockholm, Sweden
Re: InputFormat for the example SimpleMasterComputeVertex
Posted by Eli Reisman <ap...@gmail.com>.
Thanks for the explanation, that makes sense. I would love to see a wiki
page at some point, you have so much knowledge of this piece of Giraph from
all your dev work on it and have also the additional bonus of experience
running big cluster jobs using these features so you have a lot of insight
to share.
Would there be any point to a future JIRA to break out the aggregator
registration from the master compute stuff, at least from the user's view?
Or is it not that confusing once you've used them a bit?
On Thu, Feb 14, 2013 at 4:52 PM, Maja Kabiljo <ma...@fb.com> wrote:
> Progressable exception can be caused by many different reasons (it's
> totally unrelated to aggregators), and when looking at which exception it's
> caused by users should get better sense about what's going on.
> What you are suggesting about providing default master compute is not
> doable, since the part which needs to be done there is aggregator
> registration. We can't know what kind of aggregators (names and types) an
> application needs.
> I remember I was talking about writing a short tutorial for aggregators
> long time ago, sorry for not doing that, will try to get to it soon.
>
> From: Eli Reisman <ap...@gmail.com>
> Reply-To: "user@giraph.apache.org" <us...@giraph.apache.org>
> Date: Thursday, February 14, 2013 2:23 PM
>
> To: "user@giraph.apache.org" <us...@giraph.apache.org>
> Subject: Re: InputFormat for the example SimpleMasterComputeVertex
>
> Other folks on the list are also having this problem with the
> progressable utile exception & job failures. I don't know much about master
> compute usage but if it is needed to make the aggregators work, maybe we
> should have a default dummy class that just handles aggregators if no other
> master compute is specified? Or a wiki page? The progressable error message
> does not lead us to this conclusion directly.
>
> On Wed, Feb 13, 2013 at 3:04 AM, Maria Stylianou <ma...@gmail.com>wrote:
>
>> Hey,
>>
>> I am trying to run the example SimpleMasterComputeVertex, but no matter
>> which Input Format and graph I give, it doesn't work. Each worker gives the
>> error:
>>
>> Caused by: java.lang.*NullPointerException*
>> at org.apache.giraph.examples.*SimpleMasterComputeVertex.compute*(SimpleMasterComputeVertex.java:*42*)
>>
>> This line 42 is the first line of the compute()
>> public void compute(Iterable<DoubleWritable> messages){
>>
>> So I guess, the initialization is not done correctly, because the input
>> file does not have the correct format.
>>
>> Any help would be appreciated,
>> Thanks!
>> Maria
>> --
>> Maria Stylianou
>> Intern at Telefonica, Barcelona, Spain
>> Master Student of European Master in Distributed Computing<http://www.kth.se/en/studies/programmes/master/em/emdc>
>> Universitat Politècnica de Catalunya - BarcelonaTech, Barcelona, Spain
>> KTH Royal Institute of Technology, Stockholm, Sweden
>>
>>
>
Re: InputFormat for the example SimpleMasterComputeVertex
Posted by Maja Kabiljo <ma...@fb.com>.
Progressable exception can be caused by many different reasons (it's totally unrelated to aggregators), and when looking at which exception it's caused by users should get better sense about what's going on.
What you are suggesting about providing default master compute is not doable, since the part which needs to be done there is aggregator registration. We can't know what kind of aggregators (names and types) an application needs.
I remember I was talking about writing a short tutorial for aggregators long time ago, sorry for not doing that, will try to get to it soon.
From: Eli Reisman <ap...@gmail.com>>
Reply-To: "user@giraph.apache.org<ma...@giraph.apache.org>" <us...@giraph.apache.org>>
Date: Thursday, February 14, 2013 2:23 PM
To: "user@giraph.apache.org<ma...@giraph.apache.org>" <us...@giraph.apache.org>>
Subject: Re: InputFormat for the example SimpleMasterComputeVertex
Other folks on the list are also having this problem with the progressable utile exception & job failures. I don't know much about master compute usage but if it is needed to make the aggregators work, maybe we should have a default dummy class that just handles aggregators if no other master compute is specified? Or a wiki page? The progressable error message does not lead us to this conclusion directly.
On Wed, Feb 13, 2013 at 3:04 AM, Maria Stylianou <ma...@gmail.com>> wrote:
Hey,
I am trying to run the example SimpleMasterComputeVertex, but no matter which Input Format and graph I give, it doesn't work. Each worker gives the error:
Caused by: java.lang.NullPointerException
at org.apache.giraph.examples.SimpleMasterComputeVertex.compute(SimpleMasterComputeVertex.java:42)
This line 42 is the first line of the compute()
public void compute(Iterable<DoubleWritable> messages){
So I guess, the initialization is not done correctly, because the input file does not have the correct format.
Any help would be appreciated,
Thanks!
Maria
--
Maria Stylianou
Intern at Telefonica, Barcelona, Spain
Master Student of European Master in Distributed Computing<http://www.kth.se/en/studies/programmes/master/em/emdc>
Universitat Politècnica de Catalunya - BarcelonaTech, Barcelona, Spain
KTH Royal Institute of Technology, Stockholm, Sweden
Re: InputFormat for the example SimpleMasterComputeVertex
Posted by Eli Reisman <ap...@gmail.com>.
Other folks on the list are also having this problem with the progressable
utile exception & job failures. I don't know much about master compute
usage but if it is needed to make the aggregators work, maybe we should
have a default dummy class that just handles aggregators if no other master
compute is specified? Or a wiki page? The progressable error message does
not lead us to this conclusion directly.
On Wed, Feb 13, 2013 at 3:04 AM, Maria Stylianou <ma...@gmail.com> wrote:
> Hey,
>
> I am trying to run the example SimpleMasterComputeVertex, but no matter
> which Input Format and graph I give, it doesn't work. Each worker gives the
> error:
>
> Caused by: java.lang.*NullPointerException*
> at org.apache.giraph.examples.*SimpleMasterComputeVertex.compute*(SimpleMasterComputeVertex.java:*42*)
>
> This line 42 is the first line of the compute()
> public void compute(Iterable<DoubleWritable> messages){
>
> So I guess, the initialization is not done correctly, because the input
> file does not have the correct format.
>
> Any help would be appreciated,
> Thanks!
> Maria
> --
> Maria Stylianou
> Intern at Telefonica, Barcelona, Spain
> Master Student of European Master in Distributed Computing<http://www.kth.se/en/studies/programmes/master/em/emdc>
> Universitat Politècnica de Catalunya - BarcelonaTech, Barcelona, Spain
> KTH Royal Institute of Technology, Stockholm, Sweden
>
>