You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hama.apache.org by "Edward J. Yoon" <ed...@apache.org> on 2011/06/21 14:34:57 UTC
PageRank hang
Hi,
I talked w/ thomas about HAMA-395 on twitter,
Thomas Jungblut:
@eddieyoon the main problem is that I can't figure it out why. The BSP
method finishes and written out the complete results. But no cleanup
13 minutes ago in reply to eddieyoon from web
And, I'm reply to hama-dev@ so that we can share about this.
As mentioned http://incubator.apache.org/hama/docs/r0.2.0/ApacheHama-0.2_UserGuide.pdf,
The BSP job will be finished only when all the processes have no more
local and outgoing queue entries and all processes are done, or killed
by the user.
Hence, you have to clear all queues manually after escape from main while loop.
+ // Clears all queues entries.
+ peer.clear();
+ // finally save the chunk of pageranks
+ PageRankBase.savePageRankMap(peer, conf, lastTentativePagerank);
+ LOG.info("Finished with iteration " + iteration + "!");
HAMA-395-v9.patch fixes that problem, and works well on my test cluster. :)
--
Best Regards, Edward J. Yoon
@eddieyoon
Re: PageRank hang
Posted by Thomas Jungblut <th...@googlemail.com>.
Please see my suggestion in HAMA-395
https://issues.apache.org/jira/browse/HAMA-395?focusedCommentId=13052526#comment-13052526
2011/6/21 Edward J. Yoon <ed...@apache.org>
> Let's commit it.
>
> On Tue, Jun 21, 2011 at 9:38 PM, Thomas Jungblut
> <th...@googlemail.com> wrote:
> > Oh crap :D
> > Thanks Edward, I wasn't aware of this "trick".
> >
> > Thank you very much!
> >
> >
> >
> > 2011/6/21 Edward J. Yoon <ed...@apache.org>
> >
> >> Hi,
> >>
> >> I talked w/ thomas about HAMA-395 on twitter,
> >>
> >> Thomas Jungblut:
> >> @eddieyoon the main problem is that I can't figure it out why. The BSP
> >> method finishes and written out the complete results. But no cleanup
> >> 13 minutes ago in reply to eddieyoon from web
> >>
> >> And, I'm reply to hama-dev@ so that we can share about this.
> >>
> >> As mentioned
> >>
> http://incubator.apache.org/hama/docs/r0.2.0/ApacheHama-0.2_UserGuide.pdf,
> >> The BSP job will be finished only when all the processes have no more
> >> local and outgoing queue entries and all processes are done, or killed
> >> by the user.
> >>
> >> Hence, you have to clear all queues manually after escape from main
> while
> >> loop.
> >>
> >> + // Clears all queues entries.
> >> + peer.clear();
> >> + // finally save the chunk of pageranks
> >> + PageRankBase.savePageRankMap(peer, conf, lastTentativePagerank);
> >> + LOG.info("Finished with iteration " + iteration + "!");
> >>
> >> HAMA-395-v9.patch fixes that problem, and works well on my test cluster.
> :)
> >>
> >> --
> >> Best Regards, Edward J. Yoon
> >> @eddieyoon
> >>
> >
> >
> >
> > --
> > Thomas Jungblut
> > Berlin
> >
> > mobile: 0170-3081070
> >
> > business: thomas.jungblut@testberichte.de
> > private: thomas.jungblut@gmail.com
> >
>
>
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>
--
Thomas Jungblut
Berlin
mobile: 0170-3081070
business: thomas.jungblut@testberichte.de
private: thomas.jungblut@gmail.com
Re: PageRank hang
Posted by "Edward J. Yoon" <ed...@apache.org>.
Let's commit it.
On Tue, Jun 21, 2011 at 9:38 PM, Thomas Jungblut
<th...@googlemail.com> wrote:
> Oh crap :D
> Thanks Edward, I wasn't aware of this "trick".
>
> Thank you very much!
>
>
>
> 2011/6/21 Edward J. Yoon <ed...@apache.org>
>
>> Hi,
>>
>> I talked w/ thomas about HAMA-395 on twitter,
>>
>> Thomas Jungblut:
>> @eddieyoon the main problem is that I can't figure it out why. The BSP
>> method finishes and written out the complete results. But no cleanup
>> 13 minutes ago in reply to eddieyoon from web
>>
>> And, I'm reply to hama-dev@ so that we can share about this.
>>
>> As mentioned
>> http://incubator.apache.org/hama/docs/r0.2.0/ApacheHama-0.2_UserGuide.pdf,
>> The BSP job will be finished only when all the processes have no more
>> local and outgoing queue entries and all processes are done, or killed
>> by the user.
>>
>> Hence, you have to clear all queues manually after escape from main while
>> loop.
>>
>> + // Clears all queues entries.
>> + peer.clear();
>> + // finally save the chunk of pageranks
>> + PageRankBase.savePageRankMap(peer, conf, lastTentativePagerank);
>> + LOG.info("Finished with iteration " + iteration + "!");
>>
>> HAMA-395-v9.patch fixes that problem, and works well on my test cluster. :)
>>
>> --
>> Best Regards, Edward J. Yoon
>> @eddieyoon
>>
>
>
>
> --
> Thomas Jungblut
> Berlin
>
> mobile: 0170-3081070
>
> business: thomas.jungblut@testberichte.de
> private: thomas.jungblut@gmail.com
>
--
Best Regards, Edward J. Yoon
@eddieyoon
Re: PageRank hang
Posted by Thomas Jungblut <th...@googlemail.com>.
Oh crap :D
Thanks Edward, I wasn't aware of this "trick".
Thank you very much!
2011/6/21 Edward J. Yoon <ed...@apache.org>
> Hi,
>
> I talked w/ thomas about HAMA-395 on twitter,
>
> Thomas Jungblut:
> @eddieyoon the main problem is that I can't figure it out why. The BSP
> method finishes and written out the complete results. But no cleanup
> 13 minutes ago in reply to eddieyoon from web
>
> And, I'm reply to hama-dev@ so that we can share about this.
>
> As mentioned
> http://incubator.apache.org/hama/docs/r0.2.0/ApacheHama-0.2_UserGuide.pdf,
> The BSP job will be finished only when all the processes have no more
> local and outgoing queue entries and all processes are done, or killed
> by the user.
>
> Hence, you have to clear all queues manually after escape from main while
> loop.
>
> + // Clears all queues entries.
> + peer.clear();
> + // finally save the chunk of pageranks
> + PageRankBase.savePageRankMap(peer, conf, lastTentativePagerank);
> + LOG.info("Finished with iteration " + iteration + "!");
>
> HAMA-395-v9.patch fixes that problem, and works well on my test cluster. :)
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>
--
Thomas Jungblut
Berlin
mobile: 0170-3081070
business: thomas.jungblut@testberichte.de
private: thomas.jungblut@gmail.com