You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hama.apache.org by Varun Gupta <th...@gmail.com> on 2009/09/30 08:13:52 UTC
Angrapa - The Graph Package
I can take up work on Angrapa and therefore would like know if you guys have
explored existing Graph libraries to have a threshold to build on.
If we are up to creating our own base classes (basic graph class and likes),
I will start by creating JIRA tasks and and then submitting code.
I Remain,
Varun Gupta
Re: Angrapa - The Graph Package
Posted by Varun Gupta <th...@gmail.com>.
It would be helpful for us to know more about BSP model.
>
> http://groups.csail.mit.edu/cag/bayanihan/papers/javapdc99/html/node2.html
>
> Thanks Edward. We should maintain these references on the Wiki page as
well.
> On Wed, Sep 30, 2009 at 4:18 PM, Hyunsik Choi <hy...@gmail.com>
> wrote:
> > Dear Vaijanath N. RAo,
> >
> > Actually, most ideas of the graph package (angrapa) are still rough. Now,
> > this page (http://wiki.apache.org/hadoop/Hamburg) is only document, but
> we
> > will make better documents for angrapa sooner.
> > In this post, I want to try to give you more information. Current state
> of
> > angrapa is as follows. As you know, angrapa will be based on bulk
> > synchronous parallel (BSP). The key concept that we exploit for graph
> > computation is to communicate necessary data while preserving locality of
> > graph data. Therefore, we started with developing the bsp library. We
> > intended that we will design angrapa after we develop the alpha version
> of
> > bsp library. If we can have more contributors, however, we can start to
> > develop the angrapa core package after we only design interfaces of bsp.
> >
> > --
> > Hyunsik Choi
> > Database & Information Systems Group, Korea University
> > http://diveintodata.org
> >
> >
> > On Wed, Sep 30, 2009 at 3:40 PM, Vaijanath N. RAo <vaiju1981@gmail.com
> >wrote:
> >
> >> Hi Varun and Hyunsik,
> >>
> >> Can we atleast try to make a study and see what all Interfaces we need
> to
> >> support for Angrapa. That will be useful in making the interfaces clean
> and
> >> well documented.
> >>
> >> --Thanks and Regards
> >> Vaijanath N. Rao
> >> ----- Original Message -----
> >> From: "Hyunsik Choi" <hy...@gmail.com>
> >> To: hama-dev@incubator.apache.org
> >> Sent: Wednesday, September 30, 2009 12:09:25 PM GMT +05:30 Chennai,
> >> Kolkata, Mumbai, New Delhi
> >> Subject: Re: Angrapa - The Graph Package
> >>
> >> Dear Varun Gupta,,
> >> Actually, I did not try to find out some existing graph library
> >> appropriated
> >> to Angrapa because I didn't think that existing graph library can be
> ported
> >> to distributed systems, especially bsp. However, I really welcome to you
> >> and
> >> your achieves if you can contribute some base classes for graphs.
> >>
> >> Best Regards,
> >> --
> >> Hyunsik Choi
> >> Database & Information Systems Group, Korea University
> >> http://diveintodata.org
> >>
> >>
> >> On Wed, Sep 30, 2009 at 3:13 PM, Varun Gupta <thevarungupta@gmail.com
> >> >wrote:
> >>
> >> > I can take up work on Angrapa and therefore would like know if you
> guys
> >> > have
> >> > explored existing Graph libraries to have a threshold to build on.
> >> >
> >> > If we are up to creating our own base classes (basic graph class and
> >> > likes),
> >> > I will start by creating JIRA tasks and and then submitting code.
> >> >
> >> > I Remain,
> >> > Varun Gupta
> >> >
> >>
> >
>
>
>
> --
> Best Regards, Edward J. Yoon @ NHN, corp.
> edwardyoon@apache.org
> http://blog.udanax.org
>
Re: Angrapa - The Graph Package
Posted by "Edward J. Yoon" <ed...@apache.org>.
It would be helpful for us to know more about BSP model.
http://groups.csail.mit.edu/cag/bayanihan/papers/javapdc99/html/node2.html
On Wed, Sep 30, 2009 at 4:18 PM, Hyunsik Choi <hy...@gmail.com> wrote:
> Dear Vaijanath N. RAo,
>
> Actually, most ideas of the graph package (angrapa) are still rough. Now,
> this page (http://wiki.apache.org/hadoop/Hamburg) is only document, but we
> will make better documents for angrapa sooner.
> In this post, I want to try to give you more information. Current state of
> angrapa is as follows. As you know, angrapa will be based on bulk
> synchronous parallel (BSP). The key concept that we exploit for graph
> computation is to communicate necessary data while preserving locality of
> graph data. Therefore, we started with developing the bsp library. We
> intended that we will design angrapa after we develop the alpha version of
> bsp library. If we can have more contributors, however, we can start to
> develop the angrapa core package after we only design interfaces of bsp.
>
> --
> Hyunsik Choi
> Database & Information Systems Group, Korea University
> http://diveintodata.org
>
>
> On Wed, Sep 30, 2009 at 3:40 PM, Vaijanath N. RAo <va...@gmail.com>wrote:
>
>> Hi Varun and Hyunsik,
>>
>> Can we atleast try to make a study and see what all Interfaces we need to
>> support for Angrapa. That will be useful in making the interfaces clean and
>> well documented.
>>
>> --Thanks and Regards
>> Vaijanath N. Rao
>> ----- Original Message -----
>> From: "Hyunsik Choi" <hy...@gmail.com>
>> To: hama-dev@incubator.apache.org
>> Sent: Wednesday, September 30, 2009 12:09:25 PM GMT +05:30 Chennai,
>> Kolkata, Mumbai, New Delhi
>> Subject: Re: Angrapa - The Graph Package
>>
>> Dear Varun Gupta,,
>> Actually, I did not try to find out some existing graph library
>> appropriated
>> to Angrapa because I didn't think that existing graph library can be ported
>> to distributed systems, especially bsp. However, I really welcome to you
>> and
>> your achieves if you can contribute some base classes for graphs.
>>
>> Best Regards,
>> --
>> Hyunsik Choi
>> Database & Information Systems Group, Korea University
>> http://diveintodata.org
>>
>>
>> On Wed, Sep 30, 2009 at 3:13 PM, Varun Gupta <thevarungupta@gmail.com
>> >wrote:
>>
>> > I can take up work on Angrapa and therefore would like know if you guys
>> > have
>> > explored existing Graph libraries to have a threshold to build on.
>> >
>> > If we are up to creating our own base classes (basic graph class and
>> > likes),
>> > I will start by creating JIRA tasks and and then submitting code.
>> >
>> > I Remain,
>> > Varun Gupta
>> >
>>
>
--
Best Regards, Edward J. Yoon @ NHN, corp.
edwardyoon@apache.org
http://blog.udanax.org
Re: Angrapa - The Graph Package
Posted by Hyunsik Choi <hy...@gmail.com>.
Dear Vaijanath N. RAo,
Actually, most ideas of the graph package (angrapa) are still rough. Now,
this page (http://wiki.apache.org/hadoop/Hamburg) is only document, but we
will make better documents for angrapa sooner.
In this post, I want to try to give you more information. Current state of
angrapa is as follows. As you know, angrapa will be based on bulk
synchronous parallel (BSP). The key concept that we exploit for graph
computation is to communicate necessary data while preserving locality of
graph data. Therefore, we started with developing the bsp library. We
intended that we will design angrapa after we develop the alpha version of
bsp library. If we can have more contributors, however, we can start to
develop the angrapa core package after we only design interfaces of bsp.
--
Hyunsik Choi
Database & Information Systems Group, Korea University
http://diveintodata.org
On Wed, Sep 30, 2009 at 3:40 PM, Vaijanath N. RAo <va...@gmail.com>wrote:
> Hi Varun and Hyunsik,
>
> Can we atleast try to make a study and see what all Interfaces we need to
> support for Angrapa. That will be useful in making the interfaces clean and
> well documented.
>
> --Thanks and Regards
> Vaijanath N. Rao
> ----- Original Message -----
> From: "Hyunsik Choi" <hy...@gmail.com>
> To: hama-dev@incubator.apache.org
> Sent: Wednesday, September 30, 2009 12:09:25 PM GMT +05:30 Chennai,
> Kolkata, Mumbai, New Delhi
> Subject: Re: Angrapa - The Graph Package
>
> Dear Varun Gupta,,
> Actually, I did not try to find out some existing graph library
> appropriated
> to Angrapa because I didn't think that existing graph library can be ported
> to distributed systems, especially bsp. However, I really welcome to you
> and
> your achieves if you can contribute some base classes for graphs.
>
> Best Regards,
> --
> Hyunsik Choi
> Database & Information Systems Group, Korea University
> http://diveintodata.org
>
>
> On Wed, Sep 30, 2009 at 3:13 PM, Varun Gupta <thevarungupta@gmail.com
> >wrote:
>
> > I can take up work on Angrapa and therefore would like know if you guys
> > have
> > explored existing Graph libraries to have a threshold to build on.
> >
> > If we are up to creating our own base classes (basic graph class and
> > likes),
> > I will start by creating JIRA tasks and and then submitting code.
> >
> > I Remain,
> > Varun Gupta
> >
>
Re: Angrapa - The Graph Package
Posted by "Edward J. Yoon" <ed...@apache.org>.
+1
On Wed, Sep 30, 2009 at 3:40 PM, Vaijanath N. RAo <va...@gmail.com> wrote:
> Hi Varun and Hyunsik,
>
> Can we atleast try to make a study and see what all Interfaces we need to support for Angrapa. That will be useful in making the interfaces clean and well documented.
>
> --Thanks and Regards
> Vaijanath N. Rao
> ----- Original Message -----
> From: "Hyunsik Choi" <hy...@gmail.com>
> To: hama-dev@incubator.apache.org
> Sent: Wednesday, September 30, 2009 12:09:25 PM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi
> Subject: Re: Angrapa - The Graph Package
>
> Dear Varun Gupta,,
> Actually, I did not try to find out some existing graph library appropriated
> to Angrapa because I didn't think that existing graph library can be ported
> to distributed systems, especially bsp. However, I really welcome to you and
> your achieves if you can contribute some base classes for graphs.
>
> Best Regards,
> --
> Hyunsik Choi
> Database & Information Systems Group, Korea University
> http://diveintodata.org
>
>
> On Wed, Sep 30, 2009 at 3:13 PM, Varun Gupta <th...@gmail.com>wrote:
>
>> I can take up work on Angrapa and therefore would like know if you guys
>> have
>> explored existing Graph libraries to have a threshold to build on.
>>
>> If we are up to creating our own base classes (basic graph class and
>> likes),
>> I will start by creating JIRA tasks and and then submitting code.
>>
>> I Remain,
>> Varun Gupta
>>
>
--
Best Regards, Edward J. Yoon @ NHN, corp.
edwardyoon@apache.org
http://blog.udanax.org
Re: Angrapa - The Graph Package
Posted by "Vaijanath N. RAo" <va...@gmail.com>.
Hi Varun and Hyunsik,
Can we atleast try to make a study and see what all Interfaces we need to support for Angrapa. That will be useful in making the interfaces clean and well documented.
--Thanks and Regards
Vaijanath N. Rao
----- Original Message -----
From: "Hyunsik Choi" <hy...@gmail.com>
To: hama-dev@incubator.apache.org
Sent: Wednesday, September 30, 2009 12:09:25 PM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi
Subject: Re: Angrapa - The Graph Package
Dear Varun Gupta,,
Actually, I did not try to find out some existing graph library appropriated
to Angrapa because I didn't think that existing graph library can be ported
to distributed systems, especially bsp. However, I really welcome to you and
your achieves if you can contribute some base classes for graphs.
Best Regards,
--
Hyunsik Choi
Database & Information Systems Group, Korea University
http://diveintodata.org
On Wed, Sep 30, 2009 at 3:13 PM, Varun Gupta <th...@gmail.com>wrote:
> I can take up work on Angrapa and therefore would like know if you guys
> have
> explored existing Graph libraries to have a threshold to build on.
>
> If we are up to creating our own base classes (basic graph class and
> likes),
> I will start by creating JIRA tasks and and then submitting code.
>
> I Remain,
> Varun Gupta
>
Re: Angrapa - The Graph Package
Posted by Hyunsik Choi <hy...@gmail.com>.
Dear Varun Gupta,,
Actually, I did not try to find out some existing graph library appropriated
to Angrapa because I didn't think that existing graph library can be ported
to distributed systems, especially bsp. However, I really welcome to you and
your achieves if you can contribute some base classes for graphs.
Best Regards,
--
Hyunsik Choi
Database & Information Systems Group, Korea University
http://diveintodata.org
On Wed, Sep 30, 2009 at 3:13 PM, Varun Gupta <th...@gmail.com>wrote:
> I can take up work on Angrapa and therefore would like know if you guys
> have
> explored existing Graph libraries to have a threshold to build on.
>
> If we are up to creating our own base classes (basic graph class and
> likes),
> I will start by creating JIRA tasks and and then submitting code.
>
> I Remain,
> Varun Gupta
>
Re: Angrapa - The Graph Package
Posted by Hyunsik Choi <hy...@gmail.com>.
Dear Varun Gupta,
Most of parts of Input/OutputFormat of angrapa are similar to
Input/OutputFormat of Hadoop, but those of angrapa must provide random
access to some data record by a given id. If Input/OutputFormat do so, each
cluster node can walk on graph data according to visiting strategies for
graphs. This point will distinguish the angrapa from MapReduce. If Jira
works well, I will post the issue about that.
Best regards,
--
Hyunsik Choi
Database & Information Systems Group, Korea University
http://diveintodata.org
On Wed, Sep 30, 2009 at 7:22 PM, Varun Gupta <th...@gmail.com>wrote:
> I feel we should plan out some concrete things to be implemented under
> Angrapa so that other people/communities start using it.
>
> For example:
> I am in Mahout Dev community and we have had discussions of using Hama but
> we always ended in downplaying the idea because Hama does not have a
> concrete library and road map.
>
> Tasks like http://issues.apache.org/jira/browse/HAMA-163 can be tackled
> first to have a basic library up soon.
>
> Can we also discuss interface for Input/Output formatter?
>
> I Remain,
> Varun Gupta
>
>
> On Wed, Sep 30, 2009 at 12:26 PM, Edward J. Yoon <edwardyoon@apache.org
> >wrote:
>
> > Hello,
> >
> > > I can take up work on Angrapa and therefore would like know if you guys
> > have
> > > explored existing Graph libraries to have a threshold to build on.
> >
> > Not yet. BTW, Since it's the graph computing framework, which is based
> > on graph traversal as a M/R program based on sequential scan, It would
> > be nice to have a basic library for graph data structures such as,
> > Input/Output formatter, Hbase graph storage, .., etc. (At this stage,
> > some library for graph algorithms is not our plan, except some
> > examples)
> >
> > > If we are up to creating our own base classes (basic graph class and
> > likes),
> > > I will start by creating JIRA tasks and and then submitting code.
> >
> > Great. Please feel free to contribute.
> >
> > Plus, In google code project, there is our prototype implementation --
> > http://code.google.com/p/throb/source/checkout
> >
> > On Wed, Sep 30, 2009 at 3:13 PM, Varun Gupta <th...@gmail.com>
> > wrote:
> > > I can take up work on Angrapa and therefore would like know if you guys
> > have
> > > explored existing Graph libraries to have a threshold to build on.
> > >
> > > If we are up to creating our own base classes (basic graph class and
> > likes),
> > > I will start by creating JIRA tasks and and then submitting code.
> > >
> > > I Remain,
> > > Varun Gupta
> > >
> >
> >
> >
> > --
> > Best Regards, Edward J. Yoon @ NHN, corp.
> > edwardyoon@apache.org
> > http://blog.udanax.org
> >
>
Re: Angrapa - The Graph Package
Posted by Varun Gupta <th...@gmail.com>.
I feel we should plan out some concrete things to be implemented under
Angrapa so that other people/communities start using it.
For example:
I am in Mahout Dev community and we have had discussions of using Hama but
we always ended in downplaying the idea because Hama does not have a
concrete library and road map.
Tasks like http://issues.apache.org/jira/browse/HAMA-163 can be tackled
first to have a basic library up soon.
Can we also discuss interface for Input/Output formatter?
I Remain,
Varun Gupta
On Wed, Sep 30, 2009 at 12:26 PM, Edward J. Yoon <ed...@apache.org>wrote:
> Hello,
>
> > I can take up work on Angrapa and therefore would like know if you guys
> have
> > explored existing Graph libraries to have a threshold to build on.
>
> Not yet. BTW, Since it's the graph computing framework, which is based
> on graph traversal as a M/R program based on sequential scan, It would
> be nice to have a basic library for graph data structures such as,
> Input/Output formatter, Hbase graph storage, .., etc. (At this stage,
> some library for graph algorithms is not our plan, except some
> examples)
>
> > If we are up to creating our own base classes (basic graph class and
> likes),
> > I will start by creating JIRA tasks and and then submitting code.
>
> Great. Please feel free to contribute.
>
> Plus, In google code project, there is our prototype implementation --
> http://code.google.com/p/throb/source/checkout
>
> On Wed, Sep 30, 2009 at 3:13 PM, Varun Gupta <th...@gmail.com>
> wrote:
> > I can take up work on Angrapa and therefore would like know if you guys
> have
> > explored existing Graph libraries to have a threshold to build on.
> >
> > If we are up to creating our own base classes (basic graph class and
> likes),
> > I will start by creating JIRA tasks and and then submitting code.
> >
> > I Remain,
> > Varun Gupta
> >
>
>
>
> --
> Best Regards, Edward J. Yoon @ NHN, corp.
> edwardyoon@apache.org
> http://blog.udanax.org
>
Re: Angrapa - The Graph Package
Posted by "Edward J. Yoon" <ed...@apache.org>.
Hello,
> I can take up work on Angrapa and therefore would like know if you guys have
> explored existing Graph libraries to have a threshold to build on.
Not yet. BTW, Since it's the graph computing framework, which is based
on graph traversal as a M/R program based on sequential scan, It would
be nice to have a basic library for graph data structures such as,
Input/Output formatter, Hbase graph storage, .., etc. (At this stage,
some library for graph algorithms is not our plan, except some
examples)
> If we are up to creating our own base classes (basic graph class and likes),
> I will start by creating JIRA tasks and and then submitting code.
Great. Please feel free to contribute.
Plus, In google code project, there is our prototype implementation --
http://code.google.com/p/throb/source/checkout
On Wed, Sep 30, 2009 at 3:13 PM, Varun Gupta <th...@gmail.com> wrote:
> I can take up work on Angrapa and therefore would like know if you guys have
> explored existing Graph libraries to have a threshold to build on.
>
> If we are up to creating our own base classes (basic graph class and likes),
> I will start by creating JIRA tasks and and then submitting code.
>
> I Remain,
> Varun Gupta
>
--
Best Regards, Edward J. Yoon @ NHN, corp.
edwardyoon@apache.org
http://blog.udanax.org