You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hama.apache.org by Thomas Jungblut <th...@googlemail.com> on 2012/04/22 15:43:56 UTC
BSP Development Tutorial
Hey all,
I took the time to add a documentation about how I write BSP's.
It is written with the new Hama-0.5.0 release which will be available soon.
It introduces a step-by-step guide in eclipse and gives a guiding hand and
also is a introduction into realtime systems with Hama.
I hope it is working for you and helps by implementing BSP systems more
easily.
Hope to hear some feedback ;)
Thomas
Re: BSP Development Tutorial
Posted by Praveen Sripati <pr...@gmail.com>.
Thomas,
I was thinking about this and one of the main drawback I see is that as the
rate of input messages changes, the job can't scaled up/down. There is no
elasticity and I have to pay fixed amount.
Praveen
On Sun, Apr 22, 2012 at 7:13 PM, Thomas Jungblut <
thomas.jungblut@googlemail.com> wrote:
> Hey all,
>
> I took the time to add a documentation about how I write BSP's.
> It is written with the new Hama-0.5.0 release which will be available soon.
> It introduces a step-by-step guide in eclipse and gives a guiding hand and
> also is a introduction into realtime systems with Hama.
> I hope it is working for you and helps by implementing BSP systems more
> easily.
>
> Hope to hear some feedback ;)
>
> Thomas
>
Re: BSP Development Tutorial
Posted by Thomas Jungblut <th...@googlemail.com>.
Thanks for the review.
Shouldn't it be setJar or setJarByClass method.
>
I put that up as well, but normally setJarByClass is more safer and
convenient to use.
2) The master sends messages to itself when Math.abs(newInt %
> peer.getNumPeers()) equals 0, but the master is not processing the incoming
> messages.
>
Obviously, this is a bug. Thanks for catching it.
Please elaborate why when using remote access why there is no need for
> global sync.
>
Wait until it is done, a single RPC call is less costly than a global sync.
4) The Twitter example in your blog (1) was more interesting and clear than
> the random # generation.
>
So I should let the user download the version of twitter4j just because it
is more interesting? Seriously this is an introduction.
5) > It is assuming that you have Hama-0.5.0 running (see GettingStarted),
> either as deamons or in Eclipse (or any other favorable IDE) as a
> localrunner.
>
Except for the generic parameter M, are any other specific features of 0.5
> used? 0.5 is not still in the download area, wiki would be better on the
> 0.4 release.
>
That is why I'm distributing it over the mailing list to sanity check this
before we have 0.5.0 out.
There are other fancy features, but they don't belong into a tutorial like
this.
Am 23. April 2012 15:28 schrieb Praveen Sripati <pr...@gmail.com>:
> The tutorial is very good.
>
> 1) In the section `JarByClass not set`
>
> > So better use the method BSPJob#setBspClass(your_class.class) to let the
> framework determine the jar it needs to distribute properly.
>
> Shouldn't it be setJar or setJarByClass method.
>
> 2) The master sends messages to itself when Math.abs(newInt %
> peer.getNumPeers()) equals 0, but the master is not processing the incoming
> messages.
>
> 3) > We want to provide a better support for real-time systems by adding
> remote (a-)synchronous memory access in the future, see HAMA-546 for
> progress and information on it. This example would then not have overhead
> in using a global sync, because it does not use it.
>
> Please elaborate why when using remote access why there is no need for
> global sync.
>
> 4) The Twitter example in your blog (1) was more interesting and clear than
> the random # generation.
>
> 5) > It is assuming that you have Hama-0.5.0 running (see GettingStarted),
> either as deamons or in Eclipse (or any other favorable IDE) as a
> localrunner.
>
> Except for the generic parameter M, are any other specific features of 0.5
> used? 0.5 is not still in the download area, wiki would be better on the
> 0.4 release.
>
> Praveen
>
> (1)
>
> http://codingwiththomas.blogspot.in/2011/10/apache-hama-realtime-processing.html
>
> On Sun, Apr 22, 2012 at 9:29 PM, Thomas Jungblut <
> thomas.jungblut@googlemail.com> wrote:
>
> > Sure there is also a Link: http://wiki.apache.org/hama/DevelopBSP
> > :)
> >
> > Am 22. April 2012 15:43 schrieb Thomas Jungblut <
> > thomas.jungblut@googlemail.com>:
> >
> > > Hey all,
> > >
> > > I took the time to add a documentation about how I write BSP's.
> > > It is written with the new Hama-0.5.0 release which will be available
> > soon.
> > > It introduces a step-by-step guide in eclipse and gives a guiding hand
> > and
> > > also is a introduction into realtime systems with Hama.
> > > I hope it is working for you and helps by implementing BSP systems more
> > > easily.
> > >
> > > Hope to hear some feedback ;)
> > >
> > > Thomas
> > >
> >
> >
> >
> > --
> > Thomas Jungblut
> > Berlin <th...@gmail.com>
> >
>
--
Thomas Jungblut
Berlin <th...@gmail.com>
Re: BSP Development Tutorial
Posted by Praveen Sripati <pr...@gmail.com>.
The tutorial is very good.
1) In the section `JarByClass not set`
> So better use the method BSPJob#setBspClass(your_class.class) to let the
framework determine the jar it needs to distribute properly.
Shouldn't it be setJar or setJarByClass method.
2) The master sends messages to itself when Math.abs(newInt %
peer.getNumPeers()) equals 0, but the master is not processing the incoming
messages.
3) > We want to provide a better support for real-time systems by adding
remote (a-)synchronous memory access in the future, see HAMA-546 for
progress and information on it. This example would then not have overhead
in using a global sync, because it does not use it.
Please elaborate why when using remote access why there is no need for
global sync.
4) The Twitter example in your blog (1) was more interesting and clear than
the random # generation.
5) > It is assuming that you have Hama-0.5.0 running (see GettingStarted),
either as deamons or in Eclipse (or any other favorable IDE) as a
localrunner.
Except for the generic parameter M, are any other specific features of 0.5
used? 0.5 is not still in the download area, wiki would be better on the
0.4 release.
Praveen
(1)
http://codingwiththomas.blogspot.in/2011/10/apache-hama-realtime-processing.html
On Sun, Apr 22, 2012 at 9:29 PM, Thomas Jungblut <
thomas.jungblut@googlemail.com> wrote:
> Sure there is also a Link: http://wiki.apache.org/hama/DevelopBSP
> :)
>
> Am 22. April 2012 15:43 schrieb Thomas Jungblut <
> thomas.jungblut@googlemail.com>:
>
> > Hey all,
> >
> > I took the time to add a documentation about how I write BSP's.
> > It is written with the new Hama-0.5.0 release which will be available
> soon.
> > It introduces a step-by-step guide in eclipse and gives a guiding hand
> and
> > also is a introduction into realtime systems with Hama.
> > I hope it is working for you and helps by implementing BSP systems more
> > easily.
> >
> > Hope to hear some feedback ;)
> >
> > Thomas
> >
>
>
>
> --
> Thomas Jungblut
> Berlin <th...@gmail.com>
>
Re: BSP Development Tutorial
Posted by Thomas Jungblut <th...@googlemail.com>.
Sure there is also a Link: http://wiki.apache.org/hama/DevelopBSP
:)
Am 22. April 2012 15:43 schrieb Thomas Jungblut <
thomas.jungblut@googlemail.com>:
> Hey all,
>
> I took the time to add a documentation about how I write BSP's.
> It is written with the new Hama-0.5.0 release which will be available soon.
> It introduces a step-by-step guide in eclipse and gives a guiding hand and
> also is a introduction into realtime systems with Hama.
> I hope it is working for you and helps by implementing BSP systems more
> easily.
>
> Hope to hear some feedback ;)
>
> Thomas
>
--
Thomas Jungblut
Berlin <th...@gmail.com>
Re: BSP Development Tutorial
Posted by Thomas Jungblut <th...@googlemail.com>.
Sure there is also a Link: http://wiki.apache.org/hama/DevelopBSP
:)
Am 22. April 2012 15:43 schrieb Thomas Jungblut <
thomas.jungblut@googlemail.com>:
> Hey all,
>
> I took the time to add a documentation about how I write BSP's.
> It is written with the new Hama-0.5.0 release which will be available soon.
> It introduces a step-by-step guide in eclipse and gives a guiding hand and
> also is a introduction into realtime systems with Hama.
> I hope it is working for you and helps by implementing BSP systems more
> easily.
>
> Hope to hear some feedback ;)
>
> Thomas
>
--
Thomas Jungblut
Berlin <th...@gmail.com>
Re: BSP Development Tutorial
Posted by Thomas Jungblut <th...@googlemail.com>.
+1
Am 24. April 2012 00:23 schrieb Edward J. Yoon <ed...@apache.org>:
> Nice documentation. Let's add this Tutorial to our website.
>
> On Sun, Apr 22, 2012 at 10:43 PM, Thomas Jungblut
> <th...@googlemail.com> wrote:
> > Hey all,
> >
> > I took the time to add a documentation about how I write BSP's.
> > It is written with the new Hama-0.5.0 release which will be available
> soon.
> > It introduces a step-by-step guide in eclipse and gives a guiding hand
> and
> > also is a introduction into realtime systems with Hama.
> > I hope it is working for you and helps by implementing BSP systems more
> > easily.
> >
> > Hope to hear some feedback ;)
> >
> > Thomas
>
>
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>
--
Thomas Jungblut
Berlin <th...@gmail.com>
Re: BSP Development Tutorial
Posted by "Edward J. Yoon" <ed...@apache.org>.
Nice documentation. Let's add this Tutorial to our website.
On Sun, Apr 22, 2012 at 10:43 PM, Thomas Jungblut
<th...@googlemail.com> wrote:
> Hey all,
>
> I took the time to add a documentation about how I write BSP's.
> It is written with the new Hama-0.5.0 release which will be available soon.
> It introduces a step-by-step guide in eclipse and gives a guiding hand and
> also is a introduction into realtime systems with Hama.
> I hope it is working for you and helps by implementing BSP systems more
> easily.
>
> Hope to hear some feedback ;)
>
> Thomas
--
Best Regards, Edward J. Yoon
@eddieyoon