You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@giraph.apache.org by Garimella Kiran <ki...@aalto.fi> on 2014/11/04 17:26:25 UTC
Problem running Giraph in local mode (Stuck at
MASTER_ZOOKEEPER_ONLY checkWorkers)
Hi all,
I am having some issues running my code locally on a single machine. I am using Giraph 1.1.0 and Hadoop 2.5.1 ..
The execution gets stuck and the following lines just keeps getting printed:
14/11/04 17:08:09 INFO mapred.LocalJobRunner: MASTER_ZOOKEEPER_ONLY checkWorkers: Only found 0 responses of 1 needed to start superstep -1 > map
14/11/04 17:07:43 INFO master.BspServiceMaster: checkWorkers: Only found 0 responses of 1 needed to start superstep -1. Reporting every 30000 msecs, 569973 more msecs left before giving up.
14/11/04 17:07:43 INFO master.BspServiceMaster: logMissingWorkersOnSuperstep: No response from partition 1 (could be master)
The complete log is here: http://pastebin.com/cyBmHutq
(I was able to run the same code on a different machine, which had Giraph 1.0.0 and Hadoop 0.20 with out any problems) ..
I'm a novice Giraph user, so excuse me if I'm missing something obvious.
Please let me know if I forgot to add something in the email.
Regards,
Kiran
Re: Problem running Giraph in local mode (Stuck at
MASTER_ZOOKEEPER_ONLY checkWorkers)
Posted by Eli Reisman <ap...@gmail.com>.
The hadoop_2 profile should execute Giraph jobs from within MR Mappers as
the profiles for Hadoop 1.x and 0.20.x do. hadoop_yarn build Giraph to spin
up its own Application Master and run direct on YARN without MapReduce.
The error you're experiencing (and your build params) seems as if it would
occur in a run from the former option (running within MR job on Hadoop2)
Hope that helps narrow it down.
On Wed, Nov 5, 2014 at 9:08 AM, Tripti Singh <tr...@yahoo-inc.com> wrote:
> Hey ok,
> I'm not sure but can someone explain why we have a Hadoop 2 profile which
> is not yarn based? R we expected to run giraph 1.1.0 on Hadoop 2 without
> yarn? I'm asking this as I have faced some scalability issues with yarn
> profile in the past so not sure if I should really be trying out the
> non-yarn profile to execute my workflow and eliminate the problems.
>
> Thanks,
> Tripti.
> Sent from my iPhone
>
> On 05-Nov-2014, at 2:46 am, "Garimella Kiran" <ki...@aalto.fi>
> wrote:
>
> Hi,
>
> I built it using the hadoop_2 profile.
>
> More specifically, the command I used is: mvn –Phadoop_2 –Dhadoop=2.5.1
> compile -DskipTests
>
> Regards,
> Kiran
>
> From: Tripti Singh <tr...@yahoo-inc.com>
> Reply-To: "user@giraph.apache.org" <us...@giraph.apache.org>
> Date: Tuesday, November 4, 2014 at 7:27 PM
> To: "<us...@giraph.apache.org>" <us...@giraph.apache.org>
> Subject: Re: Problem running Giraph in local mode (Stuck at
> MASTER_ZOOKEEPER_ONLY checkWorkers)
>
> Which profile did you build?
> I see log statements from GiraphJob class while yarn actually invokes
> GiraphYarnJob class afaik. I have not tried in local job runner mode so I
> might be wrong.(FYI, i built it using hadoop_yarn profile)
>
> Thanks,
> Tripti.
> Sent from my iPhone
>
> On 04-Nov-2014, at 9:57 pm, "Garimella Kiran" <ki...@aalto.fi>
> wrote:
>
> Hi all,
>
> I am having some issues running my code locally on a single machine. I
> am using Giraph 1.1.0 and Hadoop 2.5.1 ..
>
> The execution gets stuck and the following lines just keeps getting
> printed:
>
> 14/11/04 17:08:09 INFO mapred.LocalJobRunner: MASTER_ZOOKEEPER_ONLY
> checkWorkers: Only found 0 responses of 1 needed to start superstep -1 > map
>
> 14/11/04 17:07:43 INFO master.BspServiceMaster: checkWorkers: Only found 0
> responses of 1 needed to start superstep -1. Reporting every 30000 msecs,
> 569973 more msecs left before giving up.
>
> 14/11/04 17:07:43 INFO master.BspServiceMaster:
> logMissingWorkersOnSuperstep: No response from partition 1 (could be master)
>
>
> The complete log is here: http://pastebin.com/cyBmHutq
>
> (I was able to run the same code on a different machine, which had
> Giraph 1.0.0 and Hadoop 0.20 with out any problems) ..
> I'm a novice Giraph user, so excuse me if I'm missing something obvious.
> Please let me know if I forgot to add something in the email.
>
> Regards,
> Kiran
>
>
Re: Problem running Giraph in local mode (Stuck at
MASTER_ZOOKEEPER_ONLY checkWorkers)
Posted by Tripti Singh <tr...@yahoo-inc.com>.
Hey ok,
I'm not sure but can someone explain why we have a Hadoop 2 profile which is not yarn based? R we expected to run giraph 1.1.0 on Hadoop 2 without yarn? I'm asking this as I have faced some scalability issues with yarn profile in the past so not sure if I should really be trying out the non-yarn profile to execute my workflow and eliminate the problems.
Thanks,
Tripti.
Sent from my iPhone
On 05-Nov-2014, at 2:46 am, "Garimella Kiran" <ki...@aalto.fi>> wrote:
Hi,
I built it using the hadoop_2 profile.
More specifically, the command I used is: mvn –Phadoop_2 –Dhadoop=2.5.1 compile -DskipTests
Regards,
Kiran
From: Tripti Singh <tr...@yahoo-inc.com>>
Reply-To: "user@giraph.apache.org<ma...@giraph.apache.org>" <us...@giraph.apache.org>>
Date: Tuesday, November 4, 2014 at 7:27 PM
To: "<us...@giraph.apache.org>>" <us...@giraph.apache.org>>
Subject: Re: Problem running Giraph in local mode (Stuck at MASTER_ZOOKEEPER_ONLY checkWorkers)
Which profile did you build?
I see log statements from GiraphJob class while yarn actually invokes GiraphYarnJob class afaik. I have not tried in local job runner mode so I might be wrong.(FYI, i built it using hadoop_yarn profile)
Thanks,
Tripti.
Sent from my iPhone
On 04-Nov-2014, at 9:57 pm, "Garimella Kiran" <ki...@aalto.fi>> wrote:
Hi all,
I am having some issues running my code locally on a single machine. I am using Giraph 1.1.0 and Hadoop 2.5.1 ..
The execution gets stuck and the following lines just keeps getting printed:
14/11/04 17:08:09 INFO mapred.LocalJobRunner: MASTER_ZOOKEEPER_ONLY checkWorkers: Only found 0 responses of 1 needed to start superstep -1 > map
14/11/04 17:07:43 INFO master.BspServiceMaster: checkWorkers: Only found 0 responses of 1 needed to start superstep -1. Reporting every 30000 msecs, 569973 more msecs left before giving up.
14/11/04 17:07:43 INFO master.BspServiceMaster: logMissingWorkersOnSuperstep: No response from partition 1 (could be master)
The complete log is here: http://pastebin.com/cyBmHutq
(I was able to run the same code on a different machine, which had Giraph 1.0.0 and Hadoop 0.20 with out any problems) ..
I'm a novice Giraph user, so excuse me if I'm missing something obvious.
Please let me know if I forgot to add something in the email.
Regards,
Kiran
Re: Problem running Giraph in local mode (Stuck at
MASTER_ZOOKEEPER_ONLY checkWorkers)
Posted by Garimella Kiran <ki...@aalto.fi>.
Hi,
I built it using the hadoop_2 profile.
More specifically, the command I used is: mvn –Phadoop_2 –Dhadoop=2.5.1 compile -DskipTests
Regards,
Kiran
From: Tripti Singh <tr...@yahoo-inc.com>>
Reply-To: "user@giraph.apache.org<ma...@giraph.apache.org>" <us...@giraph.apache.org>>
Date: Tuesday, November 4, 2014 at 7:27 PM
To: "<us...@giraph.apache.org>>" <us...@giraph.apache.org>>
Subject: Re: Problem running Giraph in local mode (Stuck at MASTER_ZOOKEEPER_ONLY checkWorkers)
Which profile did you build?
I see log statements from GiraphJob class while yarn actually invokes GiraphYarnJob class afaik. I have not tried in local job runner mode so I might be wrong.(FYI, i built it using hadoop_yarn profile)
Thanks,
Tripti.
Sent from my iPhone
On 04-Nov-2014, at 9:57 pm, "Garimella Kiran" <ki...@aalto.fi>> wrote:
Hi all,
I am having some issues running my code locally on a single machine. I am using Giraph 1.1.0 and Hadoop 2.5.1 ..
The execution gets stuck and the following lines just keeps getting printed:
14/11/04 17:08:09 INFO mapred.LocalJobRunner: MASTER_ZOOKEEPER_ONLY checkWorkers: Only found 0 responses of 1 needed to start superstep -1 > map
14/11/04 17:07:43 INFO master.BspServiceMaster: checkWorkers: Only found 0 responses of 1 needed to start superstep -1. Reporting every 30000 msecs, 569973 more msecs left before giving up.
14/11/04 17:07:43 INFO master.BspServiceMaster: logMissingWorkersOnSuperstep: No response from partition 1 (could be master)
The complete log is here: http://pastebin.com/cyBmHutq
(I was able to run the same code on a different machine, which had Giraph 1.0.0 and Hadoop 0.20 with out any problems) ..
I'm a novice Giraph user, so excuse me if I'm missing something obvious.
Please let me know if I forgot to add something in the email.
Regards,
Kiran
Re: Problem running Giraph in local mode (Stuck at
MASTER_ZOOKEEPER_ONLY checkWorkers)
Posted by Tripti Singh <tr...@yahoo-inc.com>.
Which profile did you build?
I see log statements from GiraphJob class while yarn actually invokes GiraphYarnJob class afaik. I have not tried in local job runner mode so I might be wrong.(FYI, i built it using hadoop_yarn profile)
Thanks,
Tripti.
Sent from my iPhone
On 04-Nov-2014, at 9:57 pm, "Garimella Kiran" <ki...@aalto.fi>> wrote:
Hi all,
I am having some issues running my code locally on a single machine. I am using Giraph 1.1.0 and Hadoop 2.5.1 ..
The execution gets stuck and the following lines just keeps getting printed:
14/11/04 17:08:09 INFO mapred.LocalJobRunner: MASTER_ZOOKEEPER_ONLY checkWorkers: Only found 0 responses of 1 needed to start superstep -1 > map
14/11/04 17:07:43 INFO master.BspServiceMaster: checkWorkers: Only found 0 responses of 1 needed to start superstep -1. Reporting every 30000 msecs, 569973 more msecs left before giving up.
14/11/04 17:07:43 INFO master.BspServiceMaster: logMissingWorkersOnSuperstep: No response from partition 1 (could be master)
The complete log is here: http://pastebin.com/cyBmHutq
(I was able to run the same code on a different machine, which had Giraph 1.0.0 and Hadoop 0.20 with out any problems) ..
I'm a novice Giraph user, so excuse me if I'm missing something obvious.
Please let me know if I forgot to add something in the email.
Regards,
Kiran