You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hama.apache.org by Tommaso Teofili <to...@gmail.com> on 2012/05/28 13:53:42 UTC

Superstep count

Hi all,

I wonder if we have issues with the superstep count in Local mode.
When running the UIMA-BSP AEProcessingJob [1] I expect it to generate a
superstep count of 3 while I get 1.
Here is the scenario:
1. first superstep:
1a. computation: each node instantiates and initializes an AnalysisEngine
object
1b. communication: a "master" node sends the docs to the other nodes
--sync--
2. second superstep
2a. computation: each node process all the docs received with the AE
2b. communication: each node sends the processing results to the "master"
node
--sync--
3. third superstep:
3a. computation: the "master" collects the received results and all the
nodes deallocate resources
3b. communication: nothing

Apart from the very trivial algorithm I'd expect it to give a superstep
count of 3.
And here's what I get on the console when running this job with 4 tasks:

12/05/28 13:48:25 INFO bsp.BSPJobClient: Current supersteps number: 1
12/05/28 13:48:25 INFO bsp.BSPJobClient: The total number of supersteps: 1
12/05/28 13:48:25 INFO bsp.BSPJobClient: Counters: 6
12/05/28 13:48:25 INFO bsp.BSPJobClient:
org.apache.hama.bsp.JobInProgress$JobCounter
12/05/28 13:48:25 INFO bsp.BSPJobClient:     LAUNCHED_TASKS=4
12/05/28 13:48:25 INFO bsp.BSPJobClient:
org.apache.hama.bsp.BSPPeerImpl$PeerCounter
12/05/28 13:48:25 INFO bsp.BSPJobClient:     SUPERSTEPS=1
12/05/28 13:48:25 INFO bsp.BSPJobClient:     SUPERSTEP_SUM=8
12/05/28 13:48:25 INFO bsp.BSPJobClient:     TIME_IN_SYNC_MS=21
12/05/28 13:48:25 INFO bsp.BSPJobClient:     TOTAL_MESSAGES_SENT=32
12/05/28 13:48:25 INFO bsp.BSPJobClient:     TOTAL_MESSAGES_RECEIVED=16

WDYT?
Tommaso

[1] :
http://svn.apache.org/repos/asf/uima/sandbox/trunk/uima-bsp/src/main/java/org/apache/uima/bsp/AEProcessingBSPJob.java

Re: Superstep count

Posted by Thomas Jungblut <th...@googlemail.com>.
BSPPeer starts at -1, because it waits until everything has been written to
ZK / peernames etc., so at initial state of bsp phase it should be zero.
Based on this it should work actually. No other clue if this isn't the
problem.

2012/5/30 Tommaso Teofili <to...@gmail.com>

> 2012/5/28 Thomas Jungblut <th...@googlemail.com>
>
> > Hi Tommaso,
> >
> > SUPERSTEP_SUM tells the number of tasks times supersteps done, beginning
> > from 0.
> > So you have 0,1,2 -> 2*4 = 8.
> > SUPERSTEPS is updated at the end by the BSPJobClient.
> > In peer the superstep starts at -1, thus you will have just 1 in the end.
> >
>
> I think this start from -1 doesn't make much sense, is there any specific
> reason for that?
> Tommaso
>
>
> >
> > We should make this whole thing more consistent...
> >
>



-- 
Thomas Jungblut
Berlin <th...@gmail.com>

Re: Superstep count

Posted by Tommaso Teofili <to...@gmail.com>.
2012/5/28 Thomas Jungblut <th...@googlemail.com>

> Hi Tommaso,
>
> SUPERSTEP_SUM tells the number of tasks times supersteps done, beginning
> from 0.
> So you have 0,1,2 -> 2*4 = 8.
> SUPERSTEPS is updated at the end by the BSPJobClient.
> In peer the superstep starts at -1, thus you will have just 1 in the end.
>

I think this start from -1 doesn't make much sense, is there any specific
reason for that?
Tommaso


>
> We should make this whole thing more consistent...
>

Re: Superstep count

Posted by Thomas Jungblut <th...@googlemail.com>.
Hi Tommaso,

SUPERSTEP_SUM tells the number of tasks times supersteps done, beginning
from 0.
So you have 0,1,2 -> 2*4 = 8.
SUPERSTEPS is updated at the end by the BSPJobClient.
In peer the superstep starts at -1, thus you will have just 1 in the end.

We should make this whole thing more consistent...