You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hama.apache.org by Konrad Hinsen <ko...@fastmail.net> on 2010/02/08 11:27:55 UTC
Getting started with BSP
Hello everyone,
I downloaded Hama because I am interested in trying out its BSP implementation. I have Hama running (the example from the "Quick Start" page works), but I am now facing a lack of documentation. Does anyone have a working example of a BSP application, plus instructions of how to run it?
Thanks in advance,
Konrad.
Fwd: Getting started with BSP
Posted by "Edward J. Yoon" <ed...@apache.org>.
We need to implement more BSP examples.
---------- Forwarded message ----------
From: Edward J. Yoon <ed...@apache.org>
Date: Tue, Feb 9, 2010 at 4:41 PM
Subject: Re: Getting started with BSP
To: hama-user@incubator.apache.org
You can replace localhost to the host's name or IP address.
conf.set(BSPConstants.PEER_HOST, "localhost");
conf.set(BSPConstants.PEER_PORT, String
.valueOf(30000 + randomSequence[i]));
In that example, randomSequence() was just used to avoid the port conflicts.
On Tue, Feb 9, 2010 at 3:28 PM, Konrad Hinsen
<ko...@fastmail.net> wrote:
> On 9 Feb 2010, at 02:46, Edward J. Yoon wrote:
>
>> Nope, it can be run on multi-node cluster. The BSP job handles a list
>> of peer servers, each peer server is run in each node. and threads
>> will be handled by peer server.
>>
>> In other words, a BSP job is a multi-core multi-thread program on
>> distributed system.
>
> So then my question is: how do I start multiple peer servers?
>
> Konrad.
>
--
Best Regards, Edward J. Yoon @ NHN, corp.
edwardyoon@apache.org
http://blog.udanax.org
--
Best Regards, Edward J. Yoon @ NHN, corp.
edwardyoon@apache.org
http://blog.udanax.org
Re: Getting started with BSP
Posted by "Edward J. Yoon" <ed...@apache.org>.
If you need any help or have any suggestion, Pls let us know. Please
feel free to contribute.
Thanks.
On Wed, Feb 10, 2010 at 5:42 PM, Konrad Hinsen
<ko...@fastmail.net> wrote:
> On 9 Feb 2010, at 08:41, Edward J. Yoon wrote:
>
>> You can replace localhost to the host's name or IP address.
>>
>> conf.set(BSPConstants.PEER_HOST, "localhost");
>> conf.set(BSPConstants.PEER_PORT, String
>> .valueOf(30000 + randomSequence[i]));
>>
>> In that example, randomSequence() was just used to avoid the port
>> conflicts.
>
> OK, I will try that!
>
> Thanks,
> Konrad.
>
>
--
Best Regards, Edward J. Yoon @ NHN, corp.
edwardyoon@apache.org
http://blog.udanax.org
Re: Getting started with BSP
Posted by Konrad Hinsen <ko...@fastmail.net>.
On 9 Feb 2010, at 08:41, Edward J. Yoon wrote:
> You can replace localhost to the host's name or IP address.
>
> conf.set(BSPConstants.PEER_HOST, "localhost");
> conf.set(BSPConstants.PEER_PORT, String
> .valueOf(30000 + randomSequence[i]));
>
> In that example, randomSequence() was just used to avoid the port
> conflicts.
OK, I will try that!
Thanks,
Konrad.
Re: Getting started with BSP
Posted by "Edward J. Yoon" <ed...@apache.org>.
You can replace localhost to the host's name or IP address.
conf.set(BSPConstants.PEER_HOST, "localhost");
conf.set(BSPConstants.PEER_PORT, String
.valueOf(30000 + randomSequence[i]));
In that example, randomSequence() was just used to avoid the port conflicts.
On Tue, Feb 9, 2010 at 3:28 PM, Konrad Hinsen
<ko...@fastmail.net> wrote:
> On 9 Feb 2010, at 02:46, Edward J. Yoon wrote:
>
>> Nope, it can be run on multi-node cluster. The BSP job handles a list
>> of peer servers, each peer server is run in each node. and threads
>> will be handled by peer server.
>>
>> In other words, a BSP job is a multi-core multi-thread program on
>> distributed system.
>
> So then my question is: how do I start multiple peer servers?
>
> Konrad.
>
--
Best Regards, Edward J. Yoon @ NHN, corp.
edwardyoon@apache.org
http://blog.udanax.org
Re: Getting started with BSP
Posted by Konrad Hinsen <ko...@fastmail.net>.
On 9 Feb 2010, at 02:46, Edward J. Yoon wrote:
> Nope, it can be run on multi-node cluster. The BSP job handles a list
> of peer servers, each peer server is run in each node. and threads
> will be handled by peer server.
>
> In other words, a BSP job is a multi-core multi-thread program on
> distributed system.
So then my question is: how do I start multiple peer servers?
Konrad.
Re: Getting started with BSP
Posted by "Edward J. Yoon" <ed...@apache.org>.
> I looked at the two examples and conclude that the pattern for running a BSP job is creating a subclass of HamaCluster that initializes various stuff and then starts a number of BSP threads that do the actual work. Is that about right? It would seem to imply that a BSP job can only use one compute node, because each process is just a Java thread. On the other hand, it hardly makes sense to use Hadoop for just one node, so I wonder what I am missing here.
Nope, it can be run on multi-node cluster. The BSP job handles a list
of peer servers, each peer server is run in each node. and threads
will be handled by peer server.
In other words, a BSP job is a multi-core multi-thread program on
distributed system.
On Mon, Feb 8, 2010 at 9:02 PM, Konrad Hinsen
<ko...@fastmail.net> wrote:
> On 08.02.2010, at 12:04, Edward J. Yoon wrote:
>
>> I would recommend you to read below test codes.
>>
>> svn_trunk/src/test/org/apache/hama/bsp/BSPPeerTest and SerializePrinting.
>
> Thanks, that's a starting point!
>
> I looked at the two examples and conclude that the pattern for running a BSP job is creating a subclass of HamaCluster that initializes various stuff and then starts a number of BSP threads that do the actual work. Is that about right? It would seem to imply that a BSP job can only use one compute node, because each process is just a Java thread. On the other hand, it hardly makes sense to use Hadoop for just one node, so I wonder what I am missing here.
>
> Konrad.
>
>
--
Best Regards, Edward J. Yoon @ NHN, corp.
edwardyoon@apache.org
http://blog.udanax.org
Re: Getting started with BSP
Posted by Konrad Hinsen <ko...@fastmail.net>.
On 08.02.2010, at 12:04, Edward J. Yoon wrote:
> I would recommend you to read below test codes.
>
> svn_trunk/src/test/org/apache/hama/bsp/BSPPeerTest and SerializePrinting.
Thanks, that's a starting point!
I looked at the two examples and conclude that the pattern for running a BSP job is creating a subclass of HamaCluster that initializes various stuff and then starts a number of BSP threads that do the actual work. Is that about right? It would seem to imply that a BSP job can only use one compute node, because each process is just a Java thread. On the other hand, it hardly makes sense to use Hadoop for just one node, so I wonder what I am missing here.
Konrad.
Re: Getting started with BSP
Posted by "Edward J. Yoon" <ed...@apache.org>.
Hi,
I would recommend you to read below test codes.
svn_trunk/src/test/org/apache/hama/bsp/BSPPeerTest and SerializePrinting.
Thanks.
On Mon, Feb 8, 2010 at 7:27 PM, Konrad Hinsen
<ko...@fastmail.net> wrote:
> Hello everyone,
>
> I downloaded Hama because I am interested in trying out its BSP implementation. I have Hama running (the example from the "Quick Start" page works), but I am now facing a lack of documentation. Does anyone have a working example of a BSP application, plus instructions of how to run it?
>
> Thanks in advance,
> Konrad.
>
>
--
Best Regards, Edward J. Yoon @ NHN, corp.
edwardyoon@apache.org
http://blog.udanax.org