Re: DNN at Tencent

["Edward J. Yoon" <ed...@apache.org>]

Here's the diagram about my idea.

https://docs.google.com/drawings/d/1cjz50sGbpnFp2oab30cZ5MNYsaD3PtaBRVsUWuLiglI/edit?usp=sharing

On Wed, May 27, 2015 at 5:04 PM, Edward J. Yoon <ed...@apache.org> wrote:
> Oh wait. a DNN BSPJob partitions the input data (examples) into the
> number of tasks, then each task executes SGD graph job using
> LocalBSPRunner as a mini-batch, and uses external pm server to avoid
> the heavy cost of synchronization of shared variables such as
> gradients or parameters. Does it make sense?
>
> On Wed, May 27, 2015 at 9:34 AM, Edward J. Yoon <ed...@samsung.com> wrote:
>> According to their architecture, DNN job is consist of multiple (mini-batch)
>> SGD job groups and 'ParameterServer' is used for swapping parameters among
>> them. Since these mini-batch jobs need to be work asynchronously, I guess we
>> can't do this in single job of Hama BSP framework. Hama can be used as a sub
>> component.
>>
>> --
>> Best Regards, Edward J. Yoon
>>
>> -----Original Message-----
>> From: Chia-Hung Lin [mailto:clin4j@googlemail.com]
>> Sent: Tuesday, May 26, 2015 5:40 PM
>> To: dev@hama.apache.org
>> Subject: Re: DNN at Tencent
>>
>> That's why I am working on refactoring core module stuff : )
>>
>> BSP is suitable for iterative applications and is for general purpose
>> parallelism. So as long as the application can be expressed as
>> iterative algorithm, ideally our framework (in terms of higher
>> abstraction level) should have no problem for such applications.
>>
>>
>> On 26 May 2015 at 16:08, Edward J. Yoon <ed...@apache.org> wrote:
>>> Just FYI, http://www.vldb.org/pvldb/vol7/p1772-tencent.pdf
>>>
>>> According to them, they used BSP model (and additional Parameter
>>> server) like google's distbelief. Do you think we can do this?
>>>
>>>
>>> --
>>> Best Regards, Edward J. Yoon
>>
>>
>
>
>
> --
> Best Regards, Edward J. Yoon



-- 
Best Regards, Edward J. Yoon