You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hama.apache.org by Claudio Martella <cl...@tis.bz.it> on 2010/07/14 11:46:21 UTC
how hams bsp is going to relate to pregel
Hi,
I'm trying to understand the relationship between hama and pregel, or
massive graph processing in general. At the beginning i checked the
documentation and the code, few months ago, and i relalized that hama
BSP was going towards a Hamburg/Pregel implemetnation, aside of the
matrix package. Now i've been reported that "/Hama's target was changed.
As you know, the pregel is a processing framework for large scale graph.
But, Hama aims at a distributed scientific package on Hadoop for massive
matrix and graph data. It is not specified for graph processing. A
system like Pregel might be an application of Hama."
/What does "a distributed scientific package on hadoop for massive
matrix and graph data" mean? If it's not for processing, are you writing
a graphDB?
How's hama's goal going to be related to something like pregel?
Thanks
Claudio
--
Claudio Martella
Digital Technologies
Unit Research & Development - Analyst
TIS innovation park
Via Siemens 19 | Siemensstr. 19
39100 Bolzano | 39100 Bozen
Tel. +39 0471 068 123
Fax +39 0471 068 129
claudio.martella@tis.bz.it http://www.tis.bz.it
Short information regarding use of personal data. According to Section 13 of Italian Legislative Decree no. 196 of 30 June 2003, we inform you that we process your personal data in order to fulfil contractual and fiscal obligations and also to send you information regarding our services and events. Your personal data are processed with and without electronic means and by respecting data subjects' rights, fundamental freedoms and dignity, particularly with regard to confidentiality, personal identity and the right to personal data protection. At any time and without formalities you can write an e-mail to privacy@tis.bz.it in order to object the processing of your personal data for the purpose of sending advertising materials and also to exercise the right to access personal data and other rights referred to in Section 7 of Decree 196/2003. The data controller is TIS Techno Innovation Alto Adige, Siemens Street n. 19, Bolzano. You can find the complete information on the web site www.tis.bz.it.
Re: how hams bsp is going to relate to pregel
Posted by "Edward J. Yoon" <ed...@apache.org>.
Hello,
Pregel is graph-specific: a large-scale graph computing framework,
based on BSP model.
BTW, as far as I know, the BSP model also has been researched for many
years in the field of sparse matrix computation and QR/QZ
factorisation. In addition to graph analysis, there are many other
algorithms, which have similar problems with graph, in mathematics.
Hence, now we're trying to implement low level BSP computing engine as
a more generalized framework. IMO, it can be a basic part of Pregel.
You can testing the BSP program locally using TRUNK...
Thanks.
On Wed, Jul 14, 2010 at 6:46 PM, Claudio Martella
<cl...@tis.bz.it> wrote:
> Hi,
>
> I'm trying to understand the relationship between hama and pregel, or
> massive graph processing in general. At the beginning i checked the
> documentation and the code, few months ago, and i relalized that hama
> BSP was going towards a Hamburg/Pregel implemetnation, aside of the
> matrix package. Now i've been reported that "/Hama's target was changed.
> As you know, the pregel is a processing framework for large scale graph.
> But, Hama aims at a distributed scientific package on Hadoop for massive
> matrix and graph data. It is not specified for graph processing. A
> system like Pregel might be an application of Hama."
>
> /What does "a distributed scientific package on hadoop for massive
> matrix and graph data" mean? If it's not for processing, are you writing
> a graphDB?
>
> How's hama's goal going to be related to something like pregel?
>
>
> Thanks
>
> Claudio
>
> --
> Claudio Martella
> Digital Technologies
> Unit Research & Development - Analyst
>
> TIS innovation park
> Via Siemens 19 | Siemensstr. 19
> 39100 Bolzano | 39100 Bozen
> Tel. +39 0471 068 123
> Fax +39 0471 068 129
> claudio.martella@tis.bz.it http://www.tis.bz.it
>
> Short information regarding use of personal data. According to Section 13 of Italian Legislative Decree no. 196 of 30 June 2003, we inform you that we process your personal data in order to fulfil contractual and fiscal obligations and also to send you information regarding our services and events. Your personal data are processed with and without electronic means and by respecting data subjects' rights, fundamental freedoms and dignity, particularly with regard to confidentiality, personal identity and the right to personal data protection. At any time and without formalities you can write an e-mail to privacy@tis.bz.it in order to object the processing of your personal data for the purpose of sending advertising materials and also to exercise the right to access personal data and other rights referred to in Section 7 of Decree 196/2003. The data controller is TIS Techno Innovation Alto Adige, Siemens Street n. 19, Bolzano. You can find the complete information on the web site www.tis.bz.it.
>
>
>
--
Best Regards, Edward J. Yoon
edwardyoon@apache.org
http://blog.udanax.org
Re: how hams bsp is going to relate to pregel
Posted by "Edward J. Yoon" <ed...@apache.org>.
Oh, sorry. I just deleted that phrase.
I'd like to say that the main goal of Hama (Hadoop Matrix) project is
a high-performance distributed matrix/graph computing solution. We
won't implement a algorithm library. In the future, the Hama will
consists of the BSP computing engine, and a set of few examples (e.g.,
matrix inversion, pagerank, ..., etc).
Of course, If someone contribute a nice and clean patch such as, a
implementation set of some algorithms or matrix/graph computing
framework based on Hama BSP, we'll accept them.
Is it clear enough for you?
Thanks. :)
On Thu, Jul 15, 2010 at 3:03 PM, Guo Leitao <le...@gmail.com> wrote:
> That's great if 'Hama project goal is still a distributed matrix/graph
> computing framework', but from the former mail communication, seems Hama
> 'aims at a distributed scientific package on Hadoop for massive matrix and
> graph data'. I'm totally confused now.
>
> 2010/7/15 Edward J. Yoon <ed...@apache.org>
>
>> I just wrote how will Hama BSP different from Pregel
>>
>> https://blogs.apache.org/hama/entry/how_will_hama_bsp_different
>>
>> I hope this help you.
>>
>> Thanks.
>>
>> On Wed, Jul 14, 2010 at 6:46 PM, Claudio Martella
>> <cl...@tis.bz.it> wrote:
>> > Hi,
>> >
>> > I'm trying to understand the relationship between hama and pregel, or
>> > massive graph processing in general. At the beginning i checked the
>> > documentation and the code, few months ago, and i relalized that hama
>> > BSP was going towards a Hamburg/Pregel implemetnation, aside of the
>> > matrix package. Now i've been reported that "/Hama's target was changed.
>> > As you know, the pregel is a processing framework for large scale graph.
>> > But, Hama aims at a distributed scientific package on Hadoop for massive
>> > matrix and graph data. It is not specified for graph processing. A
>> > system like Pregel might be an application of Hama."
>> >
>> > /What does "a distributed scientific package on hadoop for massive
>> > matrix and graph data" mean? If it's not for processing, are you writing
>> > a graphDB?
>> >
>> > How's hama's goal going to be related to something like pregel?
>> >
>> >
>> > Thanks
>> >
>> > Claudio
>> >
>> > --
>> > Claudio Martella
>> > Digital Technologies
>> > Unit Research & Development - Analyst
>> >
>> > TIS innovation park
>> > Via Siemens 19 | Siemensstr. 19
>> > 39100 Bolzano | 39100 Bozen
>> > Tel. +39 0471 068 123
>> > Fax +39 0471 068 129
>> > claudio.martella@tis.bz.it http://www.tis.bz.it
>> >
>> > Short information regarding use of personal data. According to Section 13
>> of Italian Legislative Decree no. 196 of 30 June 2003, we inform you that we
>> process your personal data in order to fulfil contractual and fiscal
>> obligations and also to send you information regarding our services and
>> events. Your personal data are processed with and without electronic means
>> and by respecting data subjects' rights, fundamental freedoms and dignity,
>> particularly with regard to confidentiality, personal identity and the right
>> to personal data protection. At any time and without formalities you can
>> write an e-mail to privacy@tis.bz.it in order to object the processing of
>> your personal data for the purpose of sending advertising materials and also
>> to exercise the right to access personal data and other rights referred to
>> in Section 7 of Decree 196/2003. The data controller is TIS Techno
>> Innovation Alto Adige, Siemens Street n. 19, Bolzano. You can find the
>> complete information on the web site www.tis.bz.it.
>> >
>> >
>> >
>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon
>> edwardyoon@apache.org
>> http://blog.udanax.org
>>
>
--
Best Regards, Edward J. Yoon
edwardyoon@apache.org
http://blog.udanax.org
Re: how hams bsp is going to relate to pregel
Posted by Guo Leitao <le...@gmail.com>.
That's great if 'Hama project goal is still a distributed matrix/graph
computing framework', but from the former mail communication, seems Hama
'aims at a distributed scientific package on Hadoop for massive matrix and
graph data'. I'm totally confused now.
2010/7/15 Edward J. Yoon <ed...@apache.org>
> I just wrote how will Hama BSP different from Pregel
>
> https://blogs.apache.org/hama/entry/how_will_hama_bsp_different
>
> I hope this help you.
>
> Thanks.
>
> On Wed, Jul 14, 2010 at 6:46 PM, Claudio Martella
> <cl...@tis.bz.it> wrote:
> > Hi,
> >
> > I'm trying to understand the relationship between hama and pregel, or
> > massive graph processing in general. At the beginning i checked the
> > documentation and the code, few months ago, and i relalized that hama
> > BSP was going towards a Hamburg/Pregel implemetnation, aside of the
> > matrix package. Now i've been reported that "/Hama's target was changed.
> > As you know, the pregel is a processing framework for large scale graph.
> > But, Hama aims at a distributed scientific package on Hadoop for massive
> > matrix and graph data. It is not specified for graph processing. A
> > system like Pregel might be an application of Hama."
> >
> > /What does "a distributed scientific package on hadoop for massive
> > matrix and graph data" mean? If it's not for processing, are you writing
> > a graphDB?
> >
> > How's hama's goal going to be related to something like pregel?
> >
> >
> > Thanks
> >
> > Claudio
> >
> > --
> > Claudio Martella
> > Digital Technologies
> > Unit Research & Development - Analyst
> >
> > TIS innovation park
> > Via Siemens 19 | Siemensstr. 19
> > 39100 Bolzano | 39100 Bozen
> > Tel. +39 0471 068 123
> > Fax +39 0471 068 129
> > claudio.martella@tis.bz.it http://www.tis.bz.it
> >
> > Short information regarding use of personal data. According to Section 13
> of Italian Legislative Decree no. 196 of 30 June 2003, we inform you that we
> process your personal data in order to fulfil contractual and fiscal
> obligations and also to send you information regarding our services and
> events. Your personal data are processed with and without electronic means
> and by respecting data subjects' rights, fundamental freedoms and dignity,
> particularly with regard to confidentiality, personal identity and the right
> to personal data protection. At any time and without formalities you can
> write an e-mail to privacy@tis.bz.it in order to object the processing of
> your personal data for the purpose of sending advertising materials and also
> to exercise the right to access personal data and other rights referred to
> in Section 7 of Decree 196/2003. The data controller is TIS Techno
> Innovation Alto Adige, Siemens Street n. 19, Bolzano. You can find the
> complete information on the web site www.tis.bz.it.
> >
> >
> >
>
>
>
> --
> Best Regards, Edward J. Yoon
> edwardyoon@apache.org
> http://blog.udanax.org
>
Re: how hams bsp is going to relate to pregel
Posted by "Edward J. Yoon" <ed...@apache.org>.
I just wrote how will Hama BSP different from Pregel
https://blogs.apache.org/hama/entry/how_will_hama_bsp_different
I hope this help you.
Thanks.
On Wed, Jul 14, 2010 at 6:46 PM, Claudio Martella
<cl...@tis.bz.it> wrote:
> Hi,
>
> I'm trying to understand the relationship between hama and pregel, or
> massive graph processing in general. At the beginning i checked the
> documentation and the code, few months ago, and i relalized that hama
> BSP was going towards a Hamburg/Pregel implemetnation, aside of the
> matrix package. Now i've been reported that "/Hama's target was changed.
> As you know, the pregel is a processing framework for large scale graph.
> But, Hama aims at a distributed scientific package on Hadoop for massive
> matrix and graph data. It is not specified for graph processing. A
> system like Pregel might be an application of Hama."
>
> /What does "a distributed scientific package on hadoop for massive
> matrix and graph data" mean? If it's not for processing, are you writing
> a graphDB?
>
> How's hama's goal going to be related to something like pregel?
>
>
> Thanks
>
> Claudio
>
> --
> Claudio Martella
> Digital Technologies
> Unit Research & Development - Analyst
>
> TIS innovation park
> Via Siemens 19 | Siemensstr. 19
> 39100 Bolzano | 39100 Bozen
> Tel. +39 0471 068 123
> Fax +39 0471 068 129
> claudio.martella@tis.bz.it http://www.tis.bz.it
>
> Short information regarding use of personal data. According to Section 13 of Italian Legislative Decree no. 196 of 30 June 2003, we inform you that we process your personal data in order to fulfil contractual and fiscal obligations and also to send you information regarding our services and events. Your personal data are processed with and without electronic means and by respecting data subjects' rights, fundamental freedoms and dignity, particularly with regard to confidentiality, personal identity and the right to personal data protection. At any time and without formalities you can write an e-mail to privacy@tis.bz.it in order to object the processing of your personal data for the purpose of sending advertising materials and also to exercise the right to access personal data and other rights referred to in Section 7 of Decree 196/2003. The data controller is TIS Techno Innovation Alto Adige, Siemens Street n. 19, Bolzano. You can find the complete information on the web site www.tis.bz.it.
>
>
>
--
Best Regards, Edward J. Yoon
edwardyoon@apache.org
http://blog.udanax.org
Re: how hams bsp is going to relate to pregel
Posted by "Edward J. Yoon" <ed...@apache.org>.
Hello,
I wrote how will Hama BSP different from Pregel --
https://blogs.apache.org/hama/entry/how_will_hama_bsp_different --
Point me if i'm missing out something. :)
P.S. since i'm on vacation, I won't reply immediately.
Thanks.
On Wed, Jul 14, 2010 at 6:46 PM, Claudio Martella
<cl...@tis.bz.it> wrote:
> Hi,
>
> I'm trying to understand the relationship between hama and pregel, or
> massive graph processing in general. At the beginning i checked the
> documentation and the code, few months ago, and i relalized that hama
> BSP was going towards a Hamburg/Pregel implemetnation, aside of the
> matrix package. Now i've been reported that "/Hama's target was changed.
> As you know, the pregel is a processing framework for large scale graph.
> But, Hama aims at a distributed scientific package on Hadoop for massive
> matrix and graph data. It is not specified for graph processing. A
> system like Pregel might be an application of Hama."
>
> /What does "a distributed scientific package on hadoop for massive
> matrix and graph data" mean? If it's not for processing, are you writing
> a graphDB?
>
> How's hama's goal going to be related to something like pregel?
>
>
> Thanks
>
> Claudio
>
> --
> Claudio Martella
> Digital Technologies
> Unit Research & Development - Analyst
>
> TIS innovation park
> Via Siemens 19 | Siemensstr. 19
> 39100 Bolzano | 39100 Bozen
> Tel. +39 0471 068 123
> Fax +39 0471 068 129
> claudio.martella@tis.bz.it http://www.tis.bz.it
>
> Short information regarding use of personal data. According to Section 13 of Italian Legislative Decree no. 196 of 30 June 2003, we inform you that we process your personal data in order to fulfil contractual and fiscal obligations and also to send you information regarding our services and events. Your personal data are processed with and without electronic means and by respecting data subjects' rights, fundamental freedoms and dignity, particularly with regard to confidentiality, personal identity and the right to personal data protection. At any time and without formalities you can write an e-mail to privacy@tis.bz.it in order to object the processing of your personal data for the purpose of sending advertising materials and also to exercise the right to access personal data and other rights referred to in Section 7 of Decree 196/2003. The data controller is TIS Techno Innovation Alto Adige, Siemens Street n. 19, Bolzano. You can find the complete information on the web site www.tis.bz.it.
>
>
>
--
Best Regards, Edward J. Yoon
edwardyoon@apache.org
http://blog.udanax.org