You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@giraph.apache.org by Wonbae Kim <zw...@gmail.com> on 2013/07/09 12:15:11 UTC
Problem occurs when I increase the number of workers
When running SimpleShortesPath example, it completed well if the number of
worker is 1.
hadoop jar \
/home/wb/giraph-1.0.0/giraph-examples/target/giraph-examples-1.0.0-for-hadoop-0.20.203.0-jar-with-dependencies.jar
\
org.apache.giraph.GiraphRunner \
org.apache.giraph.examples.SimpleShortestPathsVertex \
-vif
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat \
-vip /in/input \
-of org.apache.giraph.io.formats.IdWithValueTextOutputFormat \
-op /outShortest \
-w 1
However, when I increase the number of workers , like 2, map tasks were
killed.
hadoop jar \
/home/wb/giraph-1.0.0/giraph-examples/target/giraph-examples-1.0.0-for-hadoop-0.20.203.0-jar-with-dependencies.jar
\
org.apache.giraph.GiraphRunner \
org.apache.giraph.examples.SimpleShortestPathsVertex \
-vif
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat \
-vip /in/input \
-of org.apache.giraph.io.formats.IdWithValueTextOutputFormat \
-op /outShortest \
-w 2
How can I increase the number of workers?
Re: Problem occurs when I increase the number of workers
Posted by Wonbae Kim <zw...@gmail.com>.
How can I check what you asked ? I'm too newbie sorry...
2013/7/9 Jan van der Lugt <ja...@gmail.com>
> Could be something with memory. Are the workers running on the same
> machine? How much memory has been allocated to each mapper slot? How much
> physical memory is in the machine(s)?
>
> - Jan
>
>
> On Tue, Jul 9, 2013 at 12:15 PM, Wonbae Kim <zw...@gmail.com> wrote:
>
>> When running SimpleShortesPath example, it completed well if the number
>> of worker is 1.
>>
>> hadoop jar \
>> /home/wb/giraph-1.0.0/giraph-examples/target/giraph-examples-1.0.0-for-hadoop-0.20.203.0-jar-with-dependencies.jar
>> \
>> org.apache.giraph.GiraphRunner \
>> org.apache.giraph.examples.SimpleShortestPathsVertex \
>> -vif
>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat \
>> -vip /in/input \
>> -of org.apache.giraph.io.formats.IdWithValueTextOutputFormat \
>> -op /outShortest \
>> -w 1
>>
>>
>> However, when I increase the number of workers , like 2, map tasks were
>> killed.
>>
>> hadoop jar \
>> /home/wb/giraph-1.0.0/giraph-examples/target/giraph-examples-1.0.0-for-hadoop-0.20.203.0-jar-with-dependencies.jar
>> \
>> org.apache.giraph.GiraphRunner \
>> org.apache.giraph.examples.SimpleShortestPathsVertex \
>> -vif
>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat \
>> -vip /in/input \
>> -of org.apache.giraph.io.formats.IdWithValueTextOutputFormat \
>> -op /outShortest \
>> -w 2
>>
>>
>> How can I increase the number of workers?
>>
>
>
Re: Problem occurs when I increase the number of workers
Posted by Jan van der Lugt <ja...@gmail.com>.
Could be something with memory. Are the workers running on the same
machine? How much memory has been allocated to each mapper slot? How much
physical memory is in the machine(s)?
- Jan
On Tue, Jul 9, 2013 at 12:15 PM, Wonbae Kim <zw...@gmail.com> wrote:
> When running SimpleShortesPath example, it completed well if the number of
> worker is 1.
>
> hadoop jar \
> /home/wb/giraph-1.0.0/giraph-examples/target/giraph-examples-1.0.0-for-hadoop-0.20.203.0-jar-with-dependencies.jar
> \
> org.apache.giraph.GiraphRunner \
> org.apache.giraph.examples.SimpleShortestPathsVertex \
> -vif
> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat \
> -vip /in/input \
> -of org.apache.giraph.io.formats.IdWithValueTextOutputFormat \
> -op /outShortest \
> -w 1
>
>
> However, when I increase the number of workers , like 2, map tasks were
> killed.
>
> hadoop jar \
> /home/wb/giraph-1.0.0/giraph-examples/target/giraph-examples-1.0.0-for-hadoop-0.20.203.0-jar-with-dependencies.jar
> \
> org.apache.giraph.GiraphRunner \
> org.apache.giraph.examples.SimpleShortestPathsVertex \
> -vif
> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat \
> -vip /in/input \
> -of org.apache.giraph.io.formats.IdWithValueTextOutputFormat \
> -op /outShortest \
> -w 2
>
>
> How can I increase the number of workers?
>
Re: Problem occurs when I increase the number of workers
Posted by Ahmet Emre Aladağ <em...@agmlab.com>.
It should be related to Hadoop.
In what mode you're running Hadoop? You may need distributed mode to
achieve multiple workers. I don't remember the reason but I had the same
problem and have been testing my codes with 1 worker. But it should run
on a cluster with multiple workers.
On 07/09/2013 01:15 PM, Wonbae Kim wrote:
> However, when I increase the number of workers , like 2, map tasks
> were killed.
>
> How can I increase the number of workers?