You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@giraph.apache.org by Alexander Frolov <al...@gmail.com> on 2014/02/05 14:49:38 UTC

Problem with giraph deployment on the cluster

Hello,

I have deployed Giraph on the 8-node cluster (Giraph:
release-1.0.0-RC2-152-g585511f, Hadoop:0.20.203.0rc1). As I do not have
Internet on the cluster I have copied Giraph from my local computer where
it worked well.

Now I am trying to test it on the cluster and have following issues:

1) When I running Giraph (with -h, just to test) it starts very slowly
comparing to my local machine.
2) When I running a simple test it hangs like this:

frolo@A11:~> $HADOOP_HOME/bin/hadoop jar
$GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner
org.apache.giraph.examples.SimpleShortestPathsComputation -vif
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
-vip /user/frolo/input/tiny_graph.txt -vof
org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
/user/frolo/output/shortestpaths -w 1
14/02/05 17:29:33 INFO utils.ConfigurationUtils: No edge input format
specified. Ensure your InputFormat does not require one.
14/02/05 17:29:33 INFO utils.ConfigurationUtils: No edge output format
specified. Ensure your OutputFormat does not require one.
14/02/05 17:29:33 INFO job.GiraphJob: run: Since checkpointing is disabled
(default), do not allow any task retries (setting mapred.map.max.attempts =
0, old value = 4)
14/02/05 17:29:35 INFO job.GiraphJob: run: Tracking URL:
http://A11:50030/jobdetails.jsp?jobid=job_201402051720_0002

and nothing happens.

frolo@A11:~> $HADOOP_HOME/bin/hadoop job -list
1 jobs currently running
JobId State StartTime UserName Priority SchedulingInfo
job_201402051720_0002 1 1391606975417 frolo NORMAL NA
frolo@A11:~> $HADOOP_HOME/bin/hadoop job -status job_201402051720_0002

Job: job_201402051720_0002
file:
hdfs://A11:54310/home/frolo/hadoop/tmp/mapred/staging/frolo/.staging/job_201402051720_0002/job.xml
tracking URL: http://A11:50030/jobdetails.jsp?jobid=job_201402051720_0002
map() completion: 0.0
reduce() completion: 0.0
Counters: 2
Job Counters
SLOTS_MILLIS_MAPS=4613
Launched map tasks=2

I would appreciate any help in tracking down this issue. Thank you.

Best,
   Alex

Re: Problem with giraph deployment on the cluster

Posted by Sebastian Schelter <ss...@apache.org>.
No need to excuse :)

On 02/05/2014 07:15 PM, Alexander Frolov wrote:
> I think I have solved problem. Configuration of Hadoop was messy. Sorry.
>
>
> On Wed, Feb 5, 2014 at 5:55 PM, Alexander Frolov
> <al...@gmail.com>wrote:
>
>>
>>
>>
>> On Wed, Feb 5, 2014 at 5:49 PM, Alexander Frolov <alexndr.frolov@gmail.com
>>> wrote:
>>
>>> Hello,
>>>
>>> I have deployed Giraph on the 8-node cluster (Giraph:
>>> release-1.0.0-RC2-152-g585511f, Hadoop:0.20.203.0rc1). As I do not have
>>> Internet on the cluster I have copied Giraph from my local computer where
>>> it worked well.
>>>
>>> Now I am trying to test it on the cluster and have following issues:
>>>
>>> 1) When I running Giraph (with -h, just to test) it starts very slowly
>>> comparing to my local machine.
>>> 2) When I running a simple test it hangs like this:
>>>
>>> frolo@A11:~> $HADOOP_HOME/bin/hadoop jar
>>> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
>>> org.apache.giraph.GiraphRunner
>>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif
>>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
>>> -vip /user/frolo/input/tiny_graph.txt -vof
>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
>>> /user/frolo/output/shortestpaths -w 1
>>> 14/02/05 17:29:33 INFO utils.ConfigurationUtils: No edge input format
>>> specified. Ensure your InputFormat does not require one.
>>> 14/02/05 17:29:33 INFO utils.ConfigurationUtils: No edge output format
>>> specified. Ensure your OutputFormat does not require one.
>>> 14/02/05 17:29:33 INFO job.GiraphJob: run: Since checkpointing is
>>> disabled (default), do not allow any task retries (setting
>>> mapred.map.max.attempts = 0, old value = 4)
>>> 14/02/05 17:29:35 INFO job.GiraphJob: run: Tracking URL:
>>> http://A11:50030/jobdetails.jsp?jobid=job_201402051720_0002
>>>
>>> and nothing happens.
>>>
>>> frolo@A11:~> $HADOOP_HOME/bin/hadoop job -list
>>> 1 jobs currently running
>>> JobId State StartTime UserName Priority SchedulingInfo
>>> job_201402051720_0002 1 1391606975417 frolo NORMAL NA
>>> frolo@A11:~> $HADOOP_HOME/bin/hadoop job -status job_201402051720_0002
>>>
>>> Job: job_201402051720_0002
>>> file:
>>> hdfs://A11:54310/home/frolo/hadoop/tmp/mapred/staging/frolo/.staging/job_201402051720_0002/job.xml
>>> tracking URL: http://A11:50030/jobdetails.jsp?jobid=job_201402051720_0002
>>> map() completion: 0.0
>>> reduce() completion: 0.0
>>> Counters: 2
>>> Job Counters
>>> SLOTS_MILLIS_MAPS=4613
>>> Launched map tasks=2
>>>
>>> I would appreciate any help in tracking down this issue. Thank you.
>>>
>>
>> Just forgot to add that  $HADOOP_HOME/bin/hadoop job -kill
>> job_201402051720_0002 does not kill the job actually.
>>
>> Best,
>>>     Alex
>>>
>>
>>
>

Re: Problem with giraph deployment on the cluster

Posted by Alexander Frolov <al...@gmail.com>.
I think I have solved problem. Configuration of Hadoop was messy. Sorry.


On Wed, Feb 5, 2014 at 5:55 PM, Alexander Frolov
<al...@gmail.com>wrote:

>
>
>
> On Wed, Feb 5, 2014 at 5:49 PM, Alexander Frolov <alexndr.frolov@gmail.com
> > wrote:
>
>> Hello,
>>
>> I have deployed Giraph on the 8-node cluster (Giraph:
>> release-1.0.0-RC2-152-g585511f, Hadoop:0.20.203.0rc1). As I do not have
>> Internet on the cluster I have copied Giraph from my local computer where
>> it worked well.
>>
>> Now I am trying to test it on the cluster and have following issues:
>>
>> 1) When I running Giraph (with -h, just to test) it starts very slowly
>> comparing to my local machine.
>> 2) When I running a simple test it hangs like this:
>>
>> frolo@A11:~> $HADOOP_HOME/bin/hadoop jar
>> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
>> org.apache.giraph.GiraphRunner
>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif
>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
>> -vip /user/frolo/input/tiny_graph.txt -vof
>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
>> /user/frolo/output/shortestpaths -w 1
>> 14/02/05 17:29:33 INFO utils.ConfigurationUtils: No edge input format
>> specified. Ensure your InputFormat does not require one.
>> 14/02/05 17:29:33 INFO utils.ConfigurationUtils: No edge output format
>> specified. Ensure your OutputFormat does not require one.
>> 14/02/05 17:29:33 INFO job.GiraphJob: run: Since checkpointing is
>> disabled (default), do not allow any task retries (setting
>> mapred.map.max.attempts = 0, old value = 4)
>> 14/02/05 17:29:35 INFO job.GiraphJob: run: Tracking URL:
>> http://A11:50030/jobdetails.jsp?jobid=job_201402051720_0002
>>
>> and nothing happens.
>>
>> frolo@A11:~> $HADOOP_HOME/bin/hadoop job -list
>> 1 jobs currently running
>> JobId State StartTime UserName Priority SchedulingInfo
>> job_201402051720_0002 1 1391606975417 frolo NORMAL NA
>> frolo@A11:~> $HADOOP_HOME/bin/hadoop job -status job_201402051720_0002
>>
>> Job: job_201402051720_0002
>> file:
>> hdfs://A11:54310/home/frolo/hadoop/tmp/mapred/staging/frolo/.staging/job_201402051720_0002/job.xml
>> tracking URL: http://A11:50030/jobdetails.jsp?jobid=job_201402051720_0002
>> map() completion: 0.0
>> reduce() completion: 0.0
>> Counters: 2
>> Job Counters
>> SLOTS_MILLIS_MAPS=4613
>> Launched map tasks=2
>>
>> I would appreciate any help in tracking down this issue. Thank you.
>>
>
> Just forgot to add that  $HADOOP_HOME/bin/hadoop job -kill
> job_201402051720_0002 does not kill the job actually.
>
> Best,
>>    Alex
>>
>
>

Re: Problem with giraph deployment on the cluster

Posted by Alexander Frolov <al...@gmail.com>.
On Wed, Feb 5, 2014 at 5:49 PM, Alexander Frolov
<al...@gmail.com>wrote:

> Hello,
>
> I have deployed Giraph on the 8-node cluster (Giraph:
> release-1.0.0-RC2-152-g585511f, Hadoop:0.20.203.0rc1). As I do not have
> Internet on the cluster I have copied Giraph from my local computer where
> it worked well.
>
> Now I am trying to test it on the cluster and have following issues:
>
> 1) When I running Giraph (with -h, just to test) it starts very slowly
> comparing to my local machine.
> 2) When I running a simple test it hangs like this:
>
> frolo@A11:~> $HADOOP_HOME/bin/hadoop jar
> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
> org.apache.giraph.GiraphRunner
> org.apache.giraph.examples.SimpleShortestPathsComputation -vif
> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
> -vip /user/frolo/input/tiny_graph.txt -vof
> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
> /user/frolo/output/shortestpaths -w 1
> 14/02/05 17:29:33 INFO utils.ConfigurationUtils: No edge input format
> specified. Ensure your InputFormat does not require one.
> 14/02/05 17:29:33 INFO utils.ConfigurationUtils: No edge output format
> specified. Ensure your OutputFormat does not require one.
> 14/02/05 17:29:33 INFO job.GiraphJob: run: Since checkpointing is disabled
> (default), do not allow any task retries (setting mapred.map.max.attempts =
> 0, old value = 4)
> 14/02/05 17:29:35 INFO job.GiraphJob: run: Tracking URL:
> http://A11:50030/jobdetails.jsp?jobid=job_201402051720_0002
>
> and nothing happens.
>
> frolo@A11:~> $HADOOP_HOME/bin/hadoop job -list
> 1 jobs currently running
> JobId State StartTime UserName Priority SchedulingInfo
> job_201402051720_0002 1 1391606975417 frolo NORMAL NA
> frolo@A11:~> $HADOOP_HOME/bin/hadoop job -status job_201402051720_0002
>
> Job: job_201402051720_0002
> file:
> hdfs://A11:54310/home/frolo/hadoop/tmp/mapred/staging/frolo/.staging/job_201402051720_0002/job.xml
> tracking URL: http://A11:50030/jobdetails.jsp?jobid=job_201402051720_0002
> map() completion: 0.0
> reduce() completion: 0.0
> Counters: 2
> Job Counters
> SLOTS_MILLIS_MAPS=4613
> Launched map tasks=2
>
> I would appreciate any help in tracking down this issue. Thank you.
>

Just forgot to add that  $HADOOP_HOME/bin/hadoop job -kill
job_201402051720_0002 does not kill the job actually.

Best,
>    Alex
>