[jira] [Commented] (HAMA-643) Introduce BSP_OBLSYNC

["Suraj Menon (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HAMA-643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13455888#comment-13455888 ] 

Suraj Menon commented on HAMA-643:
----------------------------------

Cool. This is something similar to what I wanted to propose for broadcast API. Except that instead of having a round-robin implementation, we should have a tree based communication. Making it round robin would increase the worst case latency by O(N) where N is the number of tasks in the barrier synchronization. For tree based, we can hopefully reduce the latency to O(log(N)). Both implementation would need asynchronous communication with some state maintained in the process. This increases the priority of Asynchronous communication implementation. We should target it for 0.7. It should also improve performance. Pregel paper mentions a bit about their implementation. I shall detail the broadcastAndSync API in JIRA. Also we need to think about handling link faults in this.
                
> Introduce BSP_OBLSYNC
> ---------------------
>
>                 Key: HAMA-643
>                 URL: https://issues.apache.org/jira/browse/HAMA-643
>             Project: Hama
>          Issue Type: New Feature
>          Components: bsp core, messaging
>            Reporter: Thomas Jungblut
>
> The PUB Library [1] a german BSP lib contains a very interesting feature called "oblivious synchronization" or in short "BSP_OBLSYNC". 
> The signature of the method looks like this:
> bq.bsp oblsync (bsp, numMsgs)
> {{{
> The oblivious synchronization should be used if the programmer knows the number of messages
> each processor will receive in a superstep. E.g., if each processor sends a message to its right
> neighbour then every processor knows that it will receive exactly one message. Thus, in the oblivious
> synchronization each processor waits until nmsgs are received. This type of synchronization
> is much faster than the other one since no additional communication is needed. The standard
> synchronization should be used if the number of messages to receive is unknown. Supersteps with
> standard synchronization can alternate with oblivious synchronizations, but within one superstep
> each processor has to use the same type of synchronization
> }}}
> With such an feature, we can greatly speedup kmeans clustering by avoiding barrier synchronization, because the message exchanged are constant throughout all supersteps.
> [1] http://www2.cs.uni-paderborn.de/~pub/

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira