You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mahout.apache.org by huanchen <ia...@gmail.com> on 2012/05/17 04:33:00 UTC
About Random walk with restart
Hi,
I did not find any documentation about random walk with restart which is implemented in mahout 0.6.
Any one has any experience with RWR ?
Specifically, how to set the weight of edges in the graph?
Any information would be helpful.
Thank you !
Best,
Huanchen
2012-05-17
huanchen
Re: About Random walk with restart
Posted by Nick Pentreath <ni...@gmail.com>.
You could also look at spark and it's implementation of Google's Pregel:
https://github.com/mesos/spark/wiki/Bagel-Programming-Guide
Sent from my iPhone
On 19 May 2012, at 11:40, Ted Dunning <te...@gmail.com> wrote:
> On Sat, May 19, 2012 at 3:40 AM, Huanchen Zhang <ia...@gmail.com>wrote:
>
>> ...
>> Here I am a little confused: isn't Giraph also based on Hadoop? If so, how
>> Giraph solved the 'Hadoop lacks efficient execution of iterative
>> algorithms' problem ?
>>
>
> Hadoop programs are normally map-reduce programs. Giraph misuses the
> map-reduce framework in Hadoop by starting mapper processes that turn
> around and run a BSP program. Giraph is a separate Apache project from
> Hadoop.
>
> So you are correct, Giraph programs run on a Hadoop cluster. But they are
> not Hadoop map-reduce programs and it is the map-reduce programs that are
> very poor at iteration.
Re: Re: About Random walk with restart
Posted by huanchen <ia...@gmail.com>.
Many thanks ! I will do a research on these projects.
2012-05-21
huanchen
发件人： Ted Dunning
发送时间： 2012-05-19 17:41:16
收件人： user
抄送：
主题： Re: About Random walk with restart
On Sat, May 19, 2012 at 3:40 AM, Huanchen Zhang <ia...@gmail.com>wrote:
> ...
> Here I am a little confused: isn't Giraph also based on Hadoop? If so, how
> Giraph solved the 'Hadoop lacks efficient execution of iterative
> algorithms' problem ?
>
Hadoop programs are normally map-reduce programs. Giraph misuses the
map-reduce framework in Hadoop by starting mapper processes that turn
around and run a BSP program. Giraph is a separate Apache project from
Hadoop.
So you are correct, Giraph programs run on a Hadoop cluster. But they are
not Hadoop map-reduce programs and it is the map-reduce programs that are
very poor at iteration.
Re: About Random walk with restart
Posted by Ted Dunning <te...@gmail.com>.
On Sat, May 19, 2012 at 3:40 AM, Huanchen Zhang <ia...@gmail.com>wrote:
> ...
> Here I am a little confused: isn't Giraph also based on Hadoop? If so, how
> Giraph solved the 'Hadoop lacks efficient execution of iterative
> algorithms' problem ?
>
Hadoop programs are normally map-reduce programs. Giraph misuses the
map-reduce framework in Hadoop by starting mapper processes that turn
around and run a BSP program. Giraph is a separate Apache project from
Hadoop.
So you are correct, Giraph programs run on a Hadoop cluster. But they are
not Hadoop map-reduce programs and it is the map-reduce programs that are
very poor at iteration.
Re: About Random walk with restart
Posted by Sebastian Schelter <ss...@apache.org>.
Giraph is an implementation of Google's Pregel system and only applies
some tricks to use Hadoop as infrastructure.
You should have a look at
https://issues.apache.org/jira/browse/GIRAPH-191 coincidentally more
people are interested in RWR!
--sebastian
On 19.05.2012 09:40, Huanchen Zhang wrote:
> Hi,
>
> Thank you. I will look into it.
>
> Here I am a little confused: isn't Giraph also based on Hadoop? If so, how Giraph solved the 'Hadoop lacks efficient execution of iterative algorithms' problem ?
>
> I'm new to Hadoop ..
>
> Thank you !
>
> Best,
> Huanchen
>
>
> On May 17, 2012, at 7:11 AM, Sebastian Schelter wrote:
>
>> It is usable but it is unfortunately very slow because Hadoop lacks
>> efficient execution of iterative algorithms.
>>
>> Implementing RWR in Giraph is really simple, I can help you with that.
>> Am 17.05.2012 15:57 schrieb "huanchen" <ia...@gmail.com>:
>>
>>> Hi,
>>>
>>> Is the RWR implementation in mahout 0.6 usable ?
>>>
>>> Or what problem it has?
>>>
>>> If this implementation is usable, I will avoid reimplement it...
>>>
>>> Thank you !
>>>
>>> Best,
>>> Huanchen
>>>
>>>
>>> 2012-05-17
>>>
>>>
>>>
>>> huanchen
>>>
>>>
>>>
>>> 发件人： Sebastian Schelter
>>> 发送时间： 2012-05-17 18:47:13
>>> 收件人： user
>>> 抄送：
>>> 主题： Re: About Random walk with restart
>>>
>>> Hi,
>>> Unfortunately we chose to remove the graph algorithms from Mahout for
>>> the next version. You might want to have a look at Apache Giraph which
>>> offers an easy way to implement graph algorithms.
>>> RandomWalkWithRestart is essentialy the same as PageRank with the
>>> difference that the teleportation will only hit the source vertex of the
>>> walk not a random vertex in the graph.
>>> http://www.cs.cmu.edu/~ukang/papers/PegasusKAIS.pdf contains a short
>>> description.
>>> Best,
>>> Sebastian
>>> On 17.05.2012 04:33, huanchen wrote:
>>>> Hi,
>>>>
>>>> I did not find any documentation about random walk with restart which is
>>> implemented in mahout 0.6.
>>>>
>>>> Any one has any experience with RWR ?
>>>>
>>>> Specifically, how to set the weight of edges in the graph?
>>>>
>>>> Any information would be helpful.
>>>> no random
>>>> Thank you !
>>>>
>>>> Best,
>>>> Huanchen
>>>>
>>>> 2012-05-17
>>>>
>>>>
>>>>
>>>> huanchen
>>>>
>>>
>
Re: About Random walk with restart
Posted by Huanchen Zhang <ia...@gmail.com>.
Hi,
Thank you. I will look into it.
Here I am a little confused: isn't Giraph also based on Hadoop? If so, how Giraph solved the 'Hadoop lacks efficient execution of iterative algorithms' problem ?
I'm new to Hadoop ..
Thank you !
Best,
Huanchen
On May 17, 2012, at 7:11 AM, Sebastian Schelter wrote:
> It is usable but it is unfortunately very slow because Hadoop lacks
> efficient execution of iterative algorithms.
>
> Implementing RWR in Giraph is really simple, I can help you with that.
> Am 17.05.2012 15:57 schrieb "huanchen" <ia...@gmail.com>:
>
>> Hi,
>>
>> Is the RWR implementation in mahout 0.6 usable ?
>>
>> Or what problem it has?
>>
>> If this implementation is usable, I will avoid reimplement it...
>>
>> Thank you !
>>
>> Best,
>> Huanchen
>>
>>
>> 2012-05-17
>>
>>
>>
>> huanchen
>>
>>
>>
>> 发件人： Sebastian Schelter
>> 发送时间： 2012-05-17 18:47:13
>> 收件人： user
>> 抄送：
>> 主题： Re: About Random walk with restart
>>
>> Hi,
>> Unfortunately we chose to remove the graph algorithms from Mahout for
>> the next version. You might want to have a look at Apache Giraph which
>> offers an easy way to implement graph algorithms.
>> RandomWalkWithRestart is essentialy the same as PageRank with the
>> difference that the teleportation will only hit the source vertex of the
>> walk not a random vertex in the graph.
>> http://www.cs.cmu.edu/~ukang/papers/PegasusKAIS.pdf contains a short
>> description.
>> Best,
>> Sebastian
>> On 17.05.2012 04:33, huanchen wrote:
>>> Hi,
>>>
>>> I did not find any documentation about random walk with restart which is
>> implemented in mahout 0.6.
>>>
>>> Any one has any experience with RWR ?
>>>
>>> Specifically, how to set the weight of edges in the graph?
>>>
>>> Any information would be helpful.
>>> no random
>>> Thank you !
>>>
>>> Best,
>>> Huanchen
>>>
>>> 2012-05-17
>>>
>>>
>>>
>>> huanchen
>>>
>>
Re: Re: About Random walk with restart
Posted by Sebastian Schelter <ss...@apache.org>.
It is usable but it is unfortunately very slow because Hadoop lacks
efficient execution of iterative algorithms.
Implementing RWR in Giraph is really simple, I can help you with that.
Am 17.05.2012 15:57 schrieb "huanchen" <ia...@gmail.com>:
> Hi,
>
> Is the RWR implementation in mahout 0.6 usable ?
>
> Or what problem it has?
>
> If this implementation is usable, I will avoid reimplement it...
>
> Thank you !
>
> Best,
> Huanchen
>
>
> 2012-05-17
>
>
>
> huanchen
>
>
>
> 发件人： Sebastian Schelter
> 发送时间： 2012-05-17 18:47:13
> 收件人： user
> 抄送：
> 主题： Re: About Random walk with restart
>
> Hi,
> Unfortunately we chose to remove the graph algorithms from Mahout for
> the next version. You might want to have a look at Apache Giraph which
> offers an easy way to implement graph algorithms.
> RandomWalkWithRestart is essentialy the same as PageRank with the
> difference that the teleportation will only hit the source vertex of the
> walk not a random vertex in the graph.
> http://www.cs.cmu.edu/~ukang/papers/PegasusKAIS.pdf contains a short
> description.
> Best,
> Sebastian
> On 17.05.2012 04:33, huanchen wrote:
> > Hi,
> >
> > I did not find any documentation about random walk with restart which is
> implemented in mahout 0.6.
> >
> > Any one has any experience with RWR ?
> >
> > Specifically, how to set the weight of edges in the graph?
> >
> > Any information would be helpful.
> > no random
> > Thank you !
> >
> > Best,
> > Huanchen
> >
> > 2012-05-17
> >
> >
> >
> > huanchen
> >
>
Re: Re: About Random walk with restart
Posted by huanchen <ia...@gmail.com>.
Hi,
Is the RWR implementation in mahout 0.6 usable ?
Or what problem it has?
If this implementation is usable, I will avoid reimplement it...
Thank you !
Best,
Huanchen
2012-05-17
huanchen
发件人： Sebastian Schelter
发送时间： 2012-05-17 18:47:13
收件人： user
抄送：
主题： Re: About Random walk with restart
Hi,
Unfortunately we chose to remove the graph algorithms from Mahout for
the next version. You might want to have a look at Apache Giraph which
offers an easy way to implement graph algorithms.
RandomWalkWithRestart is essentialy the same as PageRank with the
difference that the teleportation will only hit the source vertex of the
walk not a random vertex in the graph.
http://www.cs.cmu.edu/~ukang/papers/PegasusKAIS.pdf contains a short
description.
Best,
Sebastian
On 17.05.2012 04:33, huanchen wrote:
> Hi,
>
> I did not find any documentation about random walk with restart which is implemented in mahout 0.6.
>
> Any one has any experience with RWR ?
>
> Specifically, how to set the weight of edges in the graph?
>
> Any information would be helpful.
> no random
> Thank you !
>
> Best,
> Huanchen
>
> 2012-05-17
>
>
>
> huanchen
>
Re: About Random walk with restart
Posted by Sebastian Schelter <ss...@apache.org>.
Hi,
Unfortunately we chose to remove the graph algorithms from Mahout for
the next version. You might want to have a look at Apache Giraph which
offers an easy way to implement graph algorithms.
RandomWalkWithRestart is essentialy the same as PageRank with the
difference that the teleportation will only hit the source vertex of the
walk not a random vertex in the graph.
http://www.cs.cmu.edu/~ukang/papers/PegasusKAIS.pdf contains a short
description.
Best,
Sebastian
On 17.05.2012 04:33, huanchen wrote:
> Hi,
>
> I did not find any documentation about random walk with restart which is implemented in mahout 0.6.
>
> Any one has any experience with RWR ?
>
> Specifically, how to set the weight of edges in the graph?
>
> Any information would be helpful.
> no random
> Thank you !
>
> Best,
> Huanchen
>
> 2012-05-17
>
>
>
> huanchen
>