You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hama.apache.org by "Edward J. Yoon" <ed...@apache.org> on 2012/10/26 03:04:57 UTC
Re: MODERATE for user@hama.apache.org
Hi,
See http://wiki.apache.org/hama/Combiner
And, you can find combineMessages() method in BSPPeerImpl.class. If
combiner is enabled, BSP computing framework executes combining
processing automatically.
On Fri, Oct 26, 2012 at 9:41 AM, Edward J. Yoon <ed...@apache.org> wrote:
> On Fri, Oct 26, 2012 at 8:48 AM,
> <us...@hama.apache.org>
> wrote:
>>
>> To approve:
>> user-accept-1351208920.14155.fcalbkggghohgmokfpdc@hama.apache.org
>> To reject:
>> user-reject-1351208920.14155.fcalbkggghohgmokfpdc@hama.apache.org
>> To give a reason to reject:
>> %%% Start comment
>> %%% End comment
>>
>>
>>
>> ---------- Forwarded message ----------
>> From: "Tu, Min" <mi...@paypal.com>
>> To: "dev@hama.apache.org" <de...@hama.apache.org>
>> Cc: "user@hama.apache.org" <us...@hama.apache.org>
>> Date: Thu, 25 Oct 2012 23:48:07 +0000
>> Subject: Naive questions for Hama
>> Hi,
>>
>> I am new to Hama and just playing with the example.jar bundled in Hama 0.5.
>>
>> I have several questions related to the CombineExample.java
>>
>> BSPJob.setCombinerClass will call "conf.setClass(COMBINER_CLASS_ATTR, cls, Combiner.class)";however, I can not find where the combiner being called/used
>> BSPJob.setNumBspTask will set the number of tasks ( peers ). Here are some data when I try changing this numbers. I can see the TOTAL_MESSAGES_RECEIVED got 3:1 shrinker when I use combiner, however I can not calculate the TOTAL_MESSAGES_SEND or RECEIVED count. For my calculation: 2 peers, one send 3 message to other, the TOTAL_SENT should be 6, and total received should be 2 if use combiner. But the result from program is TOTAL_SENT 24, TOTAL_RECEIVED 4.
>>
>>
>> CombineExample, task 4, NO combiner
>> 12/10/25 16:07:13 INFO bsp.BSPJobClient: TOTAL_MESSAGES_SENT=96
>> 12/10/25 16:07:13 INFO bsp.BSPJobClient: TOTAL_MESSAGES_RECEIVED=48
>>
>> CombineExample, task 4, 3>1 combiner
>> 12/10/25 16:08:43 INFO bsp.BSPJobClient: TOTAL_MESSAGES_SENT=96
>> 12/10/25 16:08:43 INFO bsp.BSPJobClient: TOTAL_MESSAGES_RECEIVED=16
>>
>> CombineExample, task 2, NO combiner
>> 12/10/25 16:09:36 INFO bsp.BSPJobClient: TOTAL_MESSAGES_SENT=24
>> 12/10/25 16:09:36 INFO bsp.BSPJobClient: TOTAL_MESSAGES_RECEIVED=12
>>
>> CombineExample, task 2, 3>1 combiner
>> 12/10/25 16:10:42 INFO bsp.BSPJobClient: TOTAL_MESSAGES_SENT=24
>> 12/10/25 16:10:42 INFO bsp.BSPJobClient: TOTAL_MESSAGES_RECEIVED=4
>>
>> Really appreciated your time and effort.
>>
>> Thanks
>>
>> Min
>>
>
>
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
--
Best Regards, Edward J. Yoon
@eddieyoon
Fwd: MODERATE for user@hama.apache.org
Posted by "Edward J. Yoon" <ed...@apache.org>.
---------- Forwarded message ----------
From: Tu, Min <mi...@paypal.com>
Date: Sat, Oct 27, 2012 at 2:38 AM
Subject: Re: MODERATE for user@hama.apache.org
To: "Edward J. Yoon" <ed...@apache.org>
Hi Edward,
Thank you very much for your tip. I am able to find the implementation
where the combiner is used.
In code BSPJob and BSPJobContext, the combiner class is set with following
entry
protected static final String COMBINER_CLASS_ATTR = "bsp.combiner.class";
-------------------------------------
But in code BSPPeerImpl class, the code use String literal NOT
COMBINER_CLASS_ATTR
conf.get("bsp.combiner.class");
Would it possible to file a simple bug to change it?
Thanks
Min
On 10/25/12 6:04 PM, "Edward J. Yoon" <ed...@apache.org> wrote:
>Hi,
>
>See http://wiki.apache.org/hama/Combiner
>
>And, you can find combineMessages() method in BSPPeerImpl.class. If
>combiner is enabled, BSP computing framework executes combining
>processing automatically.
>
>On Fri, Oct 26, 2012 at 9:41 AM, Edward J. Yoon <ed...@apache.org>
>wrote:
>> On Fri, Oct 26, 2012 at 8:48 AM,
>> <us...@hama.apache.org>
>> wrote:
>>>
>>> To approve:
>>> user-accept-1351208920.14155.fcalbkggghohgmokfpdc@hama.apache.org
>>> To reject:
>>> user-reject-1351208920.14155.fcalbkggghohgmokfpdc@hama.apache.org
>>> To give a reason to reject:
>>> %%% Start comment
>>> %%% End comment
>>>
>>>
>>>
>>> ---------- Forwarded message ----------
>>> From: "Tu, Min" <mi...@paypal.com>
>>> To: "dev@hama.apache.org" <de...@hama.apache.org>
>>> Cc: "user@hama.apache.org" <us...@hama.apache.org>
>>> Date: Thu, 25 Oct 2012 23:48:07 +0000
>>> Subject: Naive questions for Hama
>>> Hi,
>>>
>>> I am new to Hama and just playing with the example.jar bundled in Hama
>>>0.5.
>>>
>>> I have several questions related to the CombineExample.java
>>>
>>> BSPJob.setCombinerClass will call "conf.setClass(COMBINER_CLASS_ATTR,
>>>cls, Combiner.class)";however, I can not find where the combiner being
>>>called/used
>>> BSPJob.setNumBspTask will set the number of tasks ( peers ). Here are
>>>some data when I try changing this numbers. I can see the
>>>TOTAL_MESSAGES_RECEIVED got 3:1 shrinker when I use combiner, however I
>>>can not calculate the TOTAL_MESSAGES_SEND or RECEIVED count. For my
>>>calculation: 2 peers, one send 3 message to other, the TOTAL_SENT
>>>should be 6, and total received should be 2 if use combiner. But the
>>>result from program is TOTAL_SENT 24, TOTAL_RECEIVED 4.
>>>
>>>
>>> CombineExample, task 4, NO combiner
>>> 12/10/25 16:07:13 INFO bsp.BSPJobClient: TOTAL_MESSAGES_SENT=96
>>> 12/10/25 16:07:13 INFO bsp.BSPJobClient: TOTAL_MESSAGES_RECEIVED=48
>>>
>>> CombineExample, task 4, 3>1 combiner
>>> 12/10/25 16:08:43 INFO bsp.BSPJobClient: TOTAL_MESSAGES_SENT=96
>>> 12/10/25 16:08:43 INFO bsp.BSPJobClient: TOTAL_MESSAGES_RECEIVED=16
>>>
>>> CombineExample, task 2, NO combiner
>>> 12/10/25 16:09:36 INFO bsp.BSPJobClient: TOTAL_MESSAGES_SENT=24
>>> 12/10/25 16:09:36 INFO bsp.BSPJobClient: TOTAL_MESSAGES_RECEIVED=12
>>>
>>> CombineExample, task 2, 3>1 combiner
>>> 12/10/25 16:10:42 INFO bsp.BSPJobClient: TOTAL_MESSAGES_SENT=24
>>> 12/10/25 16:10:42 INFO bsp.BSPJobClient: TOTAL_MESSAGES_RECEIVED=4
>>>
>>> Really appreciated your time and effort.
>>>
>>> Thanks
>>>
>>> Min
>>>
>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon
>> @eddieyoon
>
>
>
>--
>Best Regards, Edward J. Yoon
>@eddieyoon
--
Best Regards, Edward J. Yoon
@eddieyoon