You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hama.apache.org by Francisco Sanches <sa...@gmail.com> on 2012/10/22 15:37:16 UTC
hadoop X hama
Colleagues,
Would you like tips, information on how to turn a hadoop implementation of
a program in an implementation of the same program in hama bsp. For I have
implemented a program that works centrality in large graphs implemented in
hadoop and would like to pass it to hama.
--
Francisco Sanches
Re: hadoop X hama
Posted by Thomas Jungblut <th...@gmail.com>.
If you're not into getting too deep into BSP itself, you could use the
graph package.
That requires that you can express your logic by the viewpoint of a single
vertex.
Maybe you can teach us a bit more about your algorithm.
2012/10/22 Apurv Verma <da...@gmail.com>
> Hello Francisco,
> Do you mean you want to offer some tips or you are asking for some? For
> the latter case you can see my
>
> http://code.google.com/p/anahad/source/browse/trunk/src/main/java/org/anahata/bsp/WordCount.java
>
> This is crude and at the moment not scalable but you'll get the crux of it.
>
> --
> Regards,
> Apurv Verma
>
>
>
>
>
> On Mon, Oct 22, 2012 at 7:07 PM, Francisco Sanches
> <sa...@gmail.com>wrote:
>
> > Colleagues,
> >
> > Would you like tips, information on how to turn a hadoop implementation
> of
> > a program in an implementation of the same program in hama bsp. For I
> have
> > implemented a program that works centrality in large graphs implemented
> in
> > hadoop and would like to pass it to hama.
> >
> > --
> > Francisco Sanches
> >
>
Re: hadoop X hama
Posted by Francisco Sanches <sa...@gmail.com>.
Hi Tomas
For me understand. Do you believe that finding the highest value returned
by MindistSearch.java and then finding the shortest path using his
SSSP.java I find the diameter of the graph? If so my problem is solved and
with a speed far superior to mapreduce. Just need to support this theory
theorems and prove it. Sorry for my bad english.
2012/10/23 Thomas Jungblut <th...@gmail.com>
> No the number of iterations the algorithm needed to walk through the graph
> is the diameter of it.
> Sorry for the confusion, the number of vertices in a component is obvious
> nonesense.
>
> 2012/10/23 Francisco Sanches <sa...@gmail.com>
>
> > Hi Tomas
> >
> > What does it mean number of interactions? What the program returns
> > Mindist.java? From what he ran here returns 1 file with two columns, I
> > thought it was the first and the second column vertices the distance from
> > the vertex to the nearest connected component of it.
> >
> > 2012/10/23 Thomas Jungblut <th...@gmail.com>
> >
> > > Small correction: the number of iterations not the number of vertices.
> > > Am 22.10.2012 15:58 schrieb "Thomas Jungblut" <
> thomas.jungblut@gmail.com
> > >:
> > >
> > > > As far as I recall you can use the connected components (mindist
> > search)
> > > > example to calculate the largest components.
> > > > The number of connected vertices is then usually the diameter of the
> > > graph.
> > > >
> > > > 2012/10/22 Francisco Sanches <sa...@gmail.com>
> > > >
> > > >> Hi Tomas,
> > > >>
> > > >> The big problem is this. I need to perform calculations with all
> > > vertices.
> > > >> The graph package provides me the perspective of only one node. And
> I
> > > need
> > > >> to all nodes in the graph. I have an implementation using mapreduce
> > > >> developed a colleague, and he uses breadth for all nodes. I need to
> > > >> because
> > > >> we all want to calculate the radius and diameter of the graph using
> > > exact
> > > >> calculations.
> > > >>
> > > >>
> > > >>
> > > >> 2012/10/22 Francisco Sanches <sa...@gmail.com>
> > > >>
> > > >> > Thanks for the reply
> > > >> >
> > > >> > Sorry for my bad english, I'm a Brazilian student. I need
> > > informations,
> > > >> > suggestions, research material.
> > > >> >
> > > >> >
> > > >> >
> > > >> > 2012/10/22 Apurv Verma <da...@gmail.com>
> > > >> >
> > > >> >> Hello Francisco,
> > > >> >> Do you mean you want to offer some tips or you are asking for
> > some?
> > > >> For
> > > >> >> the latter case you can see my
> > > >> >>
> > > >> >>
> > > >>
> > >
> >
> http://code.google.com/p/anahad/source/browse/trunk/src/main/java/org/anahata/bsp/WordCount.java
> > > >> >>
> > > >> >> This is crude and at the moment not scalable but you'll get the
> > crux
> > > of
> > > >> >> it.
> > > >> >>
> > > >> >> --
> > > >> >> Regards,
> > > >> >> Apurv Verma
> > > >> >>
> > > >> >>
> > > >> >>
> > > >> >>
> > > >> >>
> > > >> >> On Mon, Oct 22, 2012 at 7:07 PM, Francisco Sanches
> > > >> >> <sa...@gmail.com>wrote:
> > > >> >>
> > > >> >> > Colleagues,
> > > >> >> >
> > > >> >> > Would you like tips, information on how to turn a hadoop
> > > >> implementation
> > > >> >> of
> > > >> >> > a program in an implementation of the same program in hama bsp.
> > > For I
> > > >> >> have
> > > >> >> > implemented a program that works centrality in large graphs
> > > >> implemented
> > > >> >> in
> > > >> >> > hadoop and would like to pass it to hama.
> > > >> >> >
> > > >> >> > --
> > > >> >> > Francisco Sanches
> > > >> >> >
> > > >> >>
> > > >> >
> > > >> >
> > > >> >
> > > >> > --
> > > >> > Francisco Sanches
> > > >> >
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> Francisco Sanches
> > > >>
> > > >
> > > >
> > >
> >
> >
> >
> > --
> > Francisco Sanches
> >
>
--
Francisco Sanches
Re: hadoop X hama
Posted by Thomas Jungblut <th...@gmail.com>.
No the number of iterations the algorithm needed to walk through the graph
is the diameter of it.
Sorry for the confusion, the number of vertices in a component is obvious
nonesense.
2012/10/23 Francisco Sanches <sa...@gmail.com>
> Hi Tomas
>
> What does it mean number of interactions? What the program returns
> Mindist.java? From what he ran here returns 1 file with two columns, I
> thought it was the first and the second column vertices the distance from
> the vertex to the nearest connected component of it.
>
> 2012/10/23 Thomas Jungblut <th...@gmail.com>
>
> > Small correction: the number of iterations not the number of vertices.
> > Am 22.10.2012 15:58 schrieb "Thomas Jungblut" <thomas.jungblut@gmail.com
> >:
> >
> > > As far as I recall you can use the connected components (mindist
> search)
> > > example to calculate the largest components.
> > > The number of connected vertices is then usually the diameter of the
> > graph.
> > >
> > > 2012/10/22 Francisco Sanches <sa...@gmail.com>
> > >
> > >> Hi Tomas,
> > >>
> > >> The big problem is this. I need to perform calculations with all
> > vertices.
> > >> The graph package provides me the perspective of only one node. And I
> > need
> > >> to all nodes in the graph. I have an implementation using mapreduce
> > >> developed a colleague, and he uses breadth for all nodes. I need to
> > >> because
> > >> we all want to calculate the radius and diameter of the graph using
> > exact
> > >> calculations.
> > >>
> > >>
> > >>
> > >> 2012/10/22 Francisco Sanches <sa...@gmail.com>
> > >>
> > >> > Thanks for the reply
> > >> >
> > >> > Sorry for my bad english, I'm a Brazilian student. I need
> > informations,
> > >> > suggestions, research material.
> > >> >
> > >> >
> > >> >
> > >> > 2012/10/22 Apurv Verma <da...@gmail.com>
> > >> >
> > >> >> Hello Francisco,
> > >> >> Do you mean you want to offer some tips or you are asking for
> some?
> > >> For
> > >> >> the latter case you can see my
> > >> >>
> > >> >>
> > >>
> >
> http://code.google.com/p/anahad/source/browse/trunk/src/main/java/org/anahata/bsp/WordCount.java
> > >> >>
> > >> >> This is crude and at the moment not scalable but you'll get the
> crux
> > of
> > >> >> it.
> > >> >>
> > >> >> --
> > >> >> Regards,
> > >> >> Apurv Verma
> > >> >>
> > >> >>
> > >> >>
> > >> >>
> > >> >>
> > >> >> On Mon, Oct 22, 2012 at 7:07 PM, Francisco Sanches
> > >> >> <sa...@gmail.com>wrote:
> > >> >>
> > >> >> > Colleagues,
> > >> >> >
> > >> >> > Would you like tips, information on how to turn a hadoop
> > >> implementation
> > >> >> of
> > >> >> > a program in an implementation of the same program in hama bsp.
> > For I
> > >> >> have
> > >> >> > implemented a program that works centrality in large graphs
> > >> implemented
> > >> >> in
> > >> >> > hadoop and would like to pass it to hama.
> > >> >> >
> > >> >> > --
> > >> >> > Francisco Sanches
> > >> >> >
> > >> >>
> > >> >
> > >> >
> > >> >
> > >> > --
> > >> > Francisco Sanches
> > >> >
> > >>
> > >>
> > >>
> > >> --
> > >> Francisco Sanches
> > >>
> > >
> > >
> >
>
>
>
> --
> Francisco Sanches
>
Re: hadoop X hama
Posted by Francisco Sanches <sa...@gmail.com>.
Hi Tomas
What does it mean number of interactions? What the program returns
Mindist.java? From what he ran here returns 1 file with two columns, I
thought it was the first and the second column vertices the distance from
the vertex to the nearest connected component of it.
2012/10/23 Thomas Jungblut <th...@gmail.com>
> Small correction: the number of iterations not the number of vertices.
> Am 22.10.2012 15:58 schrieb "Thomas Jungblut" <th...@gmail.com>:
>
> > As far as I recall you can use the connected components (mindist search)
> > example to calculate the largest components.
> > The number of connected vertices is then usually the diameter of the
> graph.
> >
> > 2012/10/22 Francisco Sanches <sa...@gmail.com>
> >
> >> Hi Tomas,
> >>
> >> The big problem is this. I need to perform calculations with all
> vertices.
> >> The graph package provides me the perspective of only one node. And I
> need
> >> to all nodes in the graph. I have an implementation using mapreduce
> >> developed a colleague, and he uses breadth for all nodes. I need to
> >> because
> >> we all want to calculate the radius and diameter of the graph using
> exact
> >> calculations.
> >>
> >>
> >>
> >> 2012/10/22 Francisco Sanches <sa...@gmail.com>
> >>
> >> > Thanks for the reply
> >> >
> >> > Sorry for my bad english, I'm a Brazilian student. I need
> informations,
> >> > suggestions, research material.
> >> >
> >> >
> >> >
> >> > 2012/10/22 Apurv Verma <da...@gmail.com>
> >> >
> >> >> Hello Francisco,
> >> >> Do you mean you want to offer some tips or you are asking for some?
> >> For
> >> >> the latter case you can see my
> >> >>
> >> >>
> >>
> http://code.google.com/p/anahad/source/browse/trunk/src/main/java/org/anahata/bsp/WordCount.java
> >> >>
> >> >> This is crude and at the moment not scalable but you'll get the crux
> of
> >> >> it.
> >> >>
> >> >> --
> >> >> Regards,
> >> >> Apurv Verma
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >> On Mon, Oct 22, 2012 at 7:07 PM, Francisco Sanches
> >> >> <sa...@gmail.com>wrote:
> >> >>
> >> >> > Colleagues,
> >> >> >
> >> >> > Would you like tips, information on how to turn a hadoop
> >> implementation
> >> >> of
> >> >> > a program in an implementation of the same program in hama bsp.
> For I
> >> >> have
> >> >> > implemented a program that works centrality in large graphs
> >> implemented
> >> >> in
> >> >> > hadoop and would like to pass it to hama.
> >> >> >
> >> >> > --
> >> >> > Francisco Sanches
> >> >> >
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > Francisco Sanches
> >> >
> >>
> >>
> >>
> >> --
> >> Francisco Sanches
> >>
> >
> >
>
--
Francisco Sanches
Re: hadoop X hama
Posted by Thomas Jungblut <th...@gmail.com>.
Small correction: the number of iterations not the number of vertices.
Am 22.10.2012 15:58 schrieb "Thomas Jungblut" <th...@gmail.com>:
> As far as I recall you can use the connected components (mindist search)
> example to calculate the largest components.
> The number of connected vertices is then usually the diameter of the graph.
>
> 2012/10/22 Francisco Sanches <sa...@gmail.com>
>
>> Hi Tomas,
>>
>> The big problem is this. I need to perform calculations with all vertices.
>> The graph package provides me the perspective of only one node. And I need
>> to all nodes in the graph. I have an implementation using mapreduce
>> developed a colleague, and he uses breadth for all nodes. I need to
>> because
>> we all want to calculate the radius and diameter of the graph using exact
>> calculations.
>>
>>
>>
>> 2012/10/22 Francisco Sanches <sa...@gmail.com>
>>
>> > Thanks for the reply
>> >
>> > Sorry for my bad english, I'm a Brazilian student. I need informations,
>> > suggestions, research material.
>> >
>> >
>> >
>> > 2012/10/22 Apurv Verma <da...@gmail.com>
>> >
>> >> Hello Francisco,
>> >> Do you mean you want to offer some tips or you are asking for some?
>> For
>> >> the latter case you can see my
>> >>
>> >>
>> http://code.google.com/p/anahad/source/browse/trunk/src/main/java/org/anahata/bsp/WordCount.java
>> >>
>> >> This is crude and at the moment not scalable but you'll get the crux of
>> >> it.
>> >>
>> >> --
>> >> Regards,
>> >> Apurv Verma
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> On Mon, Oct 22, 2012 at 7:07 PM, Francisco Sanches
>> >> <sa...@gmail.com>wrote:
>> >>
>> >> > Colleagues,
>> >> >
>> >> > Would you like tips, information on how to turn a hadoop
>> implementation
>> >> of
>> >> > a program in an implementation of the same program in hama bsp. For I
>> >> have
>> >> > implemented a program that works centrality in large graphs
>> implemented
>> >> in
>> >> > hadoop and would like to pass it to hama.
>> >> >
>> >> > --
>> >> > Francisco Sanches
>> >> >
>> >>
>> >
>> >
>> >
>> > --
>> > Francisco Sanches
>> >
>>
>>
>>
>> --
>> Francisco Sanches
>>
>
>
Re: hadoop X hama
Posted by Thomas Jungblut <th...@gmail.com>.
As far as I recall you can use the connected components (mindist search)
example to calculate the largest components.
The number of connected vertices is then usually the diameter of the graph.
2012/10/22 Francisco Sanches <sa...@gmail.com>
> Hi Tomas,
>
> The big problem is this. I need to perform calculations with all vertices.
> The graph package provides me the perspective of only one node. And I need
> to all nodes in the graph. I have an implementation using mapreduce
> developed a colleague, and he uses breadth for all nodes. I need to because
> we all want to calculate the radius and diameter of the graph using exact
> calculations.
>
>
>
> 2012/10/22 Francisco Sanches <sa...@gmail.com>
>
> > Thanks for the reply
> >
> > Sorry for my bad english, I'm a Brazilian student. I need informations,
> > suggestions, research material.
> >
> >
> >
> > 2012/10/22 Apurv Verma <da...@gmail.com>
> >
> >> Hello Francisco,
> >> Do you mean you want to offer some tips or you are asking for some? For
> >> the latter case you can see my
> >>
> >>
> http://code.google.com/p/anahad/source/browse/trunk/src/main/java/org/anahata/bsp/WordCount.java
> >>
> >> This is crude and at the moment not scalable but you'll get the crux of
> >> it.
> >>
> >> --
> >> Regards,
> >> Apurv Verma
> >>
> >>
> >>
> >>
> >>
> >> On Mon, Oct 22, 2012 at 7:07 PM, Francisco Sanches
> >> <sa...@gmail.com>wrote:
> >>
> >> > Colleagues,
> >> >
> >> > Would you like tips, information on how to turn a hadoop
> implementation
> >> of
> >> > a program in an implementation of the same program in hama bsp. For I
> >> have
> >> > implemented a program that works centrality in large graphs
> implemented
> >> in
> >> > hadoop and would like to pass it to hama.
> >> >
> >> > --
> >> > Francisco Sanches
> >> >
> >>
> >
> >
> >
> > --
> > Francisco Sanches
> >
>
>
>
> --
> Francisco Sanches
>
Re: hadoop X hama
Posted by Francisco Sanches <sa...@gmail.com>.
Hi Tomas,
The big problem is this. I need to perform calculations with all vertices.
The graph package provides me the perspective of only one node. And I need
to all nodes in the graph. I have an implementation using mapreduce
developed a colleague, and he uses breadth for all nodes. I need to because
we all want to calculate the radius and diameter of the graph using exact
calculations.
2012/10/22 Francisco Sanches <sa...@gmail.com>
> Thanks for the reply
>
> Sorry for my bad english, I'm a Brazilian student. I need informations,
> suggestions, research material.
>
>
>
> 2012/10/22 Apurv Verma <da...@gmail.com>
>
>> Hello Francisco,
>> Do you mean you want to offer some tips or you are asking for some? For
>> the latter case you can see my
>>
>> http://code.google.com/p/anahad/source/browse/trunk/src/main/java/org/anahata/bsp/WordCount.java
>>
>> This is crude and at the moment not scalable but you'll get the crux of
>> it.
>>
>> --
>> Regards,
>> Apurv Verma
>>
>>
>>
>>
>>
>> On Mon, Oct 22, 2012 at 7:07 PM, Francisco Sanches
>> <sa...@gmail.com>wrote:
>>
>> > Colleagues,
>> >
>> > Would you like tips, information on how to turn a hadoop implementation
>> of
>> > a program in an implementation of the same program in hama bsp. For I
>> have
>> > implemented a program that works centrality in large graphs implemented
>> in
>> > hadoop and would like to pass it to hama.
>> >
>> > --
>> > Francisco Sanches
>> >
>>
>
>
>
> --
> Francisco Sanches
>
--
Francisco Sanches
Re: hadoop X hama
Posted by Francisco Sanches <sa...@gmail.com>.
Thanks for the reply
Sorry for my bad english, I'm a Brazilian student. I need informations,
suggestions, research material.
2012/10/22 Apurv Verma <da...@gmail.com>
> Hello Francisco,
> Do you mean you want to offer some tips or you are asking for some? For
> the latter case you can see my
>
> http://code.google.com/p/anahad/source/browse/trunk/src/main/java/org/anahata/bsp/WordCount.java
>
> This is crude and at the moment not scalable but you'll get the crux of it.
>
> --
> Regards,
> Apurv Verma
>
>
>
>
>
> On Mon, Oct 22, 2012 at 7:07 PM, Francisco Sanches
> <sa...@gmail.com>wrote:
>
> > Colleagues,
> >
> > Would you like tips, information on how to turn a hadoop implementation
> of
> > a program in an implementation of the same program in hama bsp. For I
> have
> > implemented a program that works centrality in large graphs implemented
> in
> > hadoop and would like to pass it to hama.
> >
> > --
> > Francisco Sanches
> >
>
--
Francisco Sanches
Re: hadoop X hama
Posted by Apurv Verma <da...@gmail.com>.
Hello Francisco,
Do you mean you want to offer some tips or you are asking for some? For
the latter case you can see my
http://code.google.com/p/anahad/source/browse/trunk/src/main/java/org/anahata/bsp/WordCount.java
This is crude and at the moment not scalable but you'll get the crux of it.
--
Regards,
Apurv Verma
On Mon, Oct 22, 2012 at 7:07 PM, Francisco Sanches
<sa...@gmail.com>wrote:
> Colleagues,
>
> Would you like tips, information on how to turn a hadoop implementation of
> a program in an implementation of the same program in hama bsp. For I have
> implemented a program that works centrality in large graphs implemented in
> hadoop and would like to pass it to hama.
>
> --
> Francisco Sanches
>