You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@giraph.apache.org by Alexandros Daglis <al...@epfl.ch> on 2013/04/18 21:04:33 UTC

Problem running PageRank Example

Hello everybody,

it's been a while since I last ran the PageRank example. Back then, there
was no GiraphRunner, and the example cound be executed directly. Now, I am
trying to test something but I cannot manage to make the example work. I
use the following command:

hadoop jar
giraph-0.2-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner
org.apache.giraph.examples.SimplePageRankVertex
-if org.apache.giraph.io.JsonLongDoubleFloatDoubleVertexInputFormat -w 1
-ip JSON_input -of
org.apache.giraph.io.JsonLongDoubleFloatDoubleVertexOutputFormat -op
outputJSON

The map phase proceeds up to 50% and then it freezes.
If I use 2 workers it freezes at 66%, with 3 workers at 75% etc.

Anyone has any idea about what might be causing the problem?

Thanks a lot in advance,
Alexandros

Re: Problem running PageRank Example

Posted by Maria Stylianou <ma...@gmail.com>.
My first guess:
The directories for inputFormat and outputFormat are missing the folder
'formats'.
-if org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
-of org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexOutputFormat

Maria


On Thu, Apr 18, 2013 at 9:04 PM, Alexandros Daglis <
alexandros.daglis@epfl.ch> wrote:

> Hello everybody,
>
> it's been a while since I last ran the PageRank example. Back then, there
> was no GiraphRunner, and the example cound be executed directly. Now, I am
> trying to test something but I cannot manage to make the example work. I
> use the following command:
>
> hadoop jar
> giraph-0.2-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
> org.apache.giraph.GiraphRunner
> org.apache.giraph.examples.SimplePageRankVertex
> -if org.apache.giraph.io.JsonLongDoubleFloatDoubleVertexInputFormat -w 1
> -ip JSON_input -of
> org.apache.giraph.io.JsonLongDoubleFloatDoubleVertexOutputFormat -op
> outputJSON
>
> The map phase proceeds up to 50% and then it freezes.
> If I use 2 workers it freezes at 66%, with 3 workers at 75% etc.
>
> Anyone has any idea about what might be causing the problem?
>
> Thanks a lot in advance,
> Alexandros
>



-- 
Maria Stylianou
Intern at Telefonica, Barcelona, Spain
Master Student of European Master in Distributed
Computing<http://www.kth.se/en/studies/programmes/master/em/emdc>
Universitat Politècnica de Catalunya - BarcelonaTech, Barcelona, Spain
KTH Royal Institute of Technology, Stockholm, Sweden
Phone: +34 622 211 885

RE: Problem running PageRank Example

Posted by Daglis Alexandros <al...@epfl.ch>.
I was using 0.2, which I had grabbed some time ago from the repo.

Just to make sure I am up-to-date, I just got the last version from

git://git.apache.org/giraph.git

Now, when I try to compile it with maven, i get the following:

[INFO] Reactor Summary:
[INFO]
[INFO] Apache Giraph Parent .............................. SUCCESS [7.361s]
[INFO] Apache Giraph Core ................................ SUCCESS [27.298s]
[INFO] Apache Giraph Hive I/O ............................ SUCCESS [25.584s]
[INFO] Apache Giraph Examples ............................ FAILURE [1.552s]
[INFO] Apache Giraph Accumulo I/O ........................ SKIPPED
[INFO] Apache Giraph HBase I/O ........................... SKIPPED
[INFO] Apache Giraph HCatalog I/O ........................ SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:02.282s
[INFO] Finished at: Fri Apr 19 15:48:46 CEST 2013
[INFO] Final Memory: 106M/1927M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project giraph-examples: Could not resolve dependencies for project org.apache.giraph:giraph-examples:jar:0.2-SNAPSHOT: Could not find artifact org.apache.giraph:giraph-core:jar:tests:0.2-SNAPSHOT in central (http://repo1.maven.org/maven2) -> [Help 1]

Is that a problem of my maven build?

If so, sorry for the trouble.

Regards,
Alexandros


________________________________
From: Claudio Martella [claudio.martella@gmail.com]
Sent: Friday, April 19, 2013 3:20 PM
To: user@giraph.apache.org
Subject: Re: Problem running PageRank Example

What version of giraph are you using?


On Fri, Apr 19, 2013 at 1:05 PM, Alexandros Daglis <al...@epfl.ch>> wrote:
Hello Claudio,

The log says:

2013-04-19 12:50:40,109 FATAL org.apache.giraph.graph.GraphMapper: uncaughtException: OverrideExceptionHandler on thread org.apache.giraph.graph.MasterThread, msg = generateVertexInputSplits: Got IOException, exiting...
java.lang.IllegalStateException: generateVertexInputSplits: Got IOException
    at org.apache.giraph.graph.BspServiceMaster.generateInputSplits(BspServiceMaster.java:268)
    at org.apache.giraph.graph.BspServiceMaster.createInputSplits(BspServiceMaster.java:568)
    at org.apache.giraph.graph.BspServiceMaster.createVertexInputSplits(BspServiceMaster.java:633)
    at org.apache.giraph.graph.MasterThread.run(MasterThread.java:102)
Caused by: java.io.IOException: No input paths specified in job
    at org.apache.giraph.io.GiraphFileInputFormat.listStatus(GiraphFileInputFormat.java:191)
    at org.apache.giraph.io.GiraphFileInputFormat.listVertexStatus(GiraphFileInputFormat.java:251)
    at org.apache.giraph.io.GiraphFileInputFormat.getVertexSplits(GiraphFileInputFormat.java:322)
    at org.apache.giraph.io.TextVertexInputFormat.getSplits(TextVertexInputFormat.java:61)
    at org.apache.giraph.graph.BspServiceMaster.generateInputSplits(BspServiceMaster.java:266)
    ... 3 more
2013-04-19 12:50:40,112 INFO org.apache.giraph.zk.ZooKeeperManager: run: Shutdown hook started.
2013-04-19 12:50:40,113 WARN org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers: Forced a shutdown hook kill of the ZooKeeper process.
2013-04-19 12:50:40,432 INFO org.apache.zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x13e21eac5320000, likely server has closed socket, closing socket connection and attempting reconnect
2013-04-19 12:50:40,432 INFO org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers: ZooKeeper process exited with 143 (note that 143 typically means killed).

My input file is in the hdfs and apparently there is no problem with the paths, as the output directory is created.

Best,
Alexandros


On 19 April 2013 01:56, Claudio Martella <cl...@gmail.com>> wrote:
what do the logs say?


On Thu, Apr 18, 2013 at 9:04 PM, Alexandros Daglis <al...@epfl.ch>> wrote:
Hello everybody,

it's been a while since I last ran the PageRank example. Back then, there was no GiraphRunner, and the example cound be executed directly. Now, I am trying to test something but I cannot manage to make the example work. I use the following command:

hadoop jar giraph-0.2-SNAPSHOT-for-hadoop-0.20.203.0<tel:0.20.203.0>-jar-with-dependencies.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimplePageRankVertex
-if org.apache.giraph.io.JsonLongDoubleFloatDoubleVertexInputFormat -w 1 -ip JSON_input -of org.apache.giraph.io.JsonLongDoubleFloatDoubleVertexOutputFormat -op outputJSON

The map phase proceeds up to 50% and then it freezes.
If I use 2 workers it freezes at 66%, with 3 workers at 75% etc.

Anyone has any idea about what might be causing the problem?

Thanks a lot in advance,
Alexandros



--
   Claudio Martella
   claudio.martella@gmail.com<ma...@gmail.com>




--
   Claudio Martella
   claudio.martella@gmail.com<ma...@gmail.com>

Re: Problem running PageRank Example

Posted by Claudio Martella <cl...@gmail.com>.
What version of giraph are you using?


On Fri, Apr 19, 2013 at 1:05 PM, Alexandros Daglis <
alexandros.daglis@epfl.ch> wrote:

> Hello Claudio,
>
> The log says:
>
> 2013-04-19 12:50:40,109 FATAL org.apache.giraph.graph.GraphMapper:
> uncaughtException: OverrideExceptionHandler on thread
> org.apache.giraph.graph.MasterThread, msg = generateVertexInputSplits: Got
> IOException, exiting...
> java.lang.IllegalStateException: generateVertexInputSplits: Got IOException
>     at
> org.apache.giraph.graph.BspServiceMaster.generateInputSplits(BspServiceMaster.java:268)
>     at
> org.apache.giraph.graph.BspServiceMaster.createInputSplits(BspServiceMaster.java:568)
>     at
> org.apache.giraph.graph.BspServiceMaster.createVertexInputSplits(BspServiceMaster.java:633)
>     at org.apache.giraph.graph.MasterThread.run(MasterThread.java:102)
> Caused by: java.io.IOException: No input paths specified in job
>     at
> org.apache.giraph.io.GiraphFileInputFormat.listStatus(GiraphFileInputFormat.java:191)
>     at
> org.apache.giraph.io.GiraphFileInputFormat.listVertexStatus(GiraphFileInputFormat.java:251)
>     at
> org.apache.giraph.io.GiraphFileInputFormat.getVertexSplits(GiraphFileInputFormat.java:322)
>     at
> org.apache.giraph.io.TextVertexInputFormat.getSplits(TextVertexInputFormat.java:61)
>     at
> org.apache.giraph.graph.BspServiceMaster.generateInputSplits(BspServiceMaster.java:266)
>     ... 3 more
> 2013-04-19 12:50:40,112 INFO org.apache.giraph.zk.ZooKeeperManager: run:
> Shutdown hook started.
> 2013-04-19 12:50:40,113 WARN org.apache.giraph.zk.ZooKeeperManager:
> onlineZooKeeperServers: Forced a shutdown hook kill of the ZooKeeper
> process.
> 2013-04-19 12:50:40,432 INFO org.apache.zookeeper.ClientCnxn: Unable to
> read additional data from server sessionid 0x13e21eac5320000, likely server
> has closed socket, closing socket connection and attempting reconnect
> 2013-04-19 12:50:40,432 INFO org.apache.giraph.zk.ZooKeeperManager:
> onlineZooKeeperServers: ZooKeeper process exited with 143 (note that 143
> typically means killed).
>
> My input file is in the hdfs and apparently there is no problem with the
> paths, as the output directory is created.
>
> Best,
> Alexandros
>
>
> On 19 April 2013 01:56, Claudio Martella <cl...@gmail.com>wrote:
>
>> what do the logs say?
>>
>>
>> On Thu, Apr 18, 2013 at 9:04 PM, Alexandros Daglis <
>> alexandros.daglis@epfl.ch> wrote:
>>
>>> Hello everybody,
>>>
>>> it's been a while since I last ran the PageRank example. Back then,
>>> there was no GiraphRunner, and the example cound be executed directly. Now,
>>> I am trying to test something but I cannot manage to make the example work.
>>> I use the following command:
>>>
>>> hadoop jar giraph-0.2-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
>>> org.apache.giraph.GiraphRunner
>>> org.apache.giraph.examples.SimplePageRankVertex
>>> -if org.apache.giraph.io.JsonLongDoubleFloatDoubleVertexInputFormat -w 1
>>> -ip JSON_input -of
>>> org.apache.giraph.io.JsonLongDoubleFloatDoubleVertexOutputFormat -op
>>> outputJSON
>>>
>>> The map phase proceeds up to 50% and then it freezes.
>>> If I use 2 workers it freezes at 66%, with 3 workers at 75% etc.
>>>
>>> Anyone has any idea about what might be causing the problem?
>>>
>>> Thanks a lot in advance,
>>> Alexandros
>>>
>>
>>
>>
>> --
>>    Claudio Martella
>>    claudio.martella@gmail.com
>>
>
>


-- 
   Claudio Martella
   claudio.martella@gmail.com

Re: Problem running PageRank Example

Posted by Alexandros Daglis <al...@epfl.ch>.
Hello Claudio,

The log says:

2013-04-19 12:50:40,109 FATAL org.apache.giraph.graph.GraphMapper:
uncaughtException: OverrideExceptionHandler on thread
org.apache.giraph.graph.MasterThread, msg = generateVertexInputSplits: Got
IOException, exiting...
java.lang.IllegalStateException: generateVertexInputSplits: Got IOException
    at
org.apache.giraph.graph.BspServiceMaster.generateInputSplits(BspServiceMaster.java:268)
    at
org.apache.giraph.graph.BspServiceMaster.createInputSplits(BspServiceMaster.java:568)
    at
org.apache.giraph.graph.BspServiceMaster.createVertexInputSplits(BspServiceMaster.java:633)
    at org.apache.giraph.graph.MasterThread.run(MasterThread.java:102)
Caused by: java.io.IOException: No input paths specified in job
    at
org.apache.giraph.io.GiraphFileInputFormat.listStatus(GiraphFileInputFormat.java:191)
    at
org.apache.giraph.io.GiraphFileInputFormat.listVertexStatus(GiraphFileInputFormat.java:251)
    at
org.apache.giraph.io.GiraphFileInputFormat.getVertexSplits(GiraphFileInputFormat.java:322)
    at
org.apache.giraph.io.TextVertexInputFormat.getSplits(TextVertexInputFormat.java:61)
    at
org.apache.giraph.graph.BspServiceMaster.generateInputSplits(BspServiceMaster.java:266)
    ... 3 more
2013-04-19 12:50:40,112 INFO org.apache.giraph.zk.ZooKeeperManager: run:
Shutdown hook started.
2013-04-19 12:50:40,113 WARN org.apache.giraph.zk.ZooKeeperManager:
onlineZooKeeperServers: Forced a shutdown hook kill of the ZooKeeper
process.
2013-04-19 12:50:40,432 INFO org.apache.zookeeper.ClientCnxn: Unable to
read additional data from server sessionid 0x13e21eac5320000, likely server
has closed socket, closing socket connection and attempting reconnect
2013-04-19 12:50:40,432 INFO org.apache.giraph.zk.ZooKeeperManager:
onlineZooKeeperServers: ZooKeeper process exited with 143 (note that 143
typically means killed).

My input file is in the hdfs and apparently there is no problem with the
paths, as the output directory is created.

Best,
Alexandros


On 19 April 2013 01:56, Claudio Martella <cl...@gmail.com> wrote:

> what do the logs say?
>
>
> On Thu, Apr 18, 2013 at 9:04 PM, Alexandros Daglis <
> alexandros.daglis@epfl.ch> wrote:
>
>> Hello everybody,
>>
>> it's been a while since I last ran the PageRank example. Back then, there
>> was no GiraphRunner, and the example cound be executed directly. Now, I am
>> trying to test something but I cannot manage to make the example work. I
>> use the following command:
>>
>> hadoop jar giraph-0.2-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
>> org.apache.giraph.GiraphRunner
>> org.apache.giraph.examples.SimplePageRankVertex
>> -if org.apache.giraph.io.JsonLongDoubleFloatDoubleVertexInputFormat -w 1
>> -ip JSON_input -of
>> org.apache.giraph.io.JsonLongDoubleFloatDoubleVertexOutputFormat -op
>> outputJSON
>>
>> The map phase proceeds up to 50% and then it freezes.
>> If I use 2 workers it freezes at 66%, with 3 workers at 75% etc.
>>
>> Anyone has any idea about what might be causing the problem?
>>
>> Thanks a lot in advance,
>> Alexandros
>>
>
>
>
> --
>    Claudio Martella
>    claudio.martella@gmail.com
>

Re: Problem running PageRank Example

Posted by Claudio Martella <cl...@gmail.com>.
what do the logs say?


On Thu, Apr 18, 2013 at 9:04 PM, Alexandros Daglis <
alexandros.daglis@epfl.ch> wrote:

> Hello everybody,
>
> it's been a while since I last ran the PageRank example. Back then, there
> was no GiraphRunner, and the example cound be executed directly. Now, I am
> trying to test something but I cannot manage to make the example work. I
> use the following command:
>
> hadoop jar giraph-0.2-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
> org.apache.giraph.GiraphRunner
> org.apache.giraph.examples.SimplePageRankVertex
> -if org.apache.giraph.io.JsonLongDoubleFloatDoubleVertexInputFormat -w 1
> -ip JSON_input -of
> org.apache.giraph.io.JsonLongDoubleFloatDoubleVertexOutputFormat -op
> outputJSON
>
> The map phase proceeds up to 50% and then it freezes.
> If I use 2 workers it freezes at 66%, with 3 workers at 75% etc.
>
> Anyone has any idea about what might be causing the problem?
>
> Thanks a lot in advance,
> Alexandros
>



-- 
   Claudio Martella
   claudio.martella@gmail.com