You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hama.apache.org by "Edward J. Yoon" <ed...@apache.org> on 2015/05/26 10:08:47 UTC
DNN at Tencent
Just FYI, http://www.vldb.org/pvldb/vol7/p1772-tencent.pdf
According to them, they used BSP model (and additional Parameter
server) like google's distbelief. Do you think we can do this?
--
Best Regards, Edward J. Yoon
Re: DNN at Tencent
Posted by "Edward J. Yoon" <ed...@apache.org>.
Here's the diagram about my idea.
https://docs.google.com/drawings/d/1cjz50sGbpnFp2oab30cZ5MNYsaD3PtaBRVsUWuLiglI/edit?usp=sharing
On Wed, May 27, 2015 at 5:04 PM, Edward J. Yoon <ed...@apache.org> wrote:
> Oh wait. a DNN BSPJob partitions the input data (examples) into the
> number of tasks, then each task executes SGD graph job using
> LocalBSPRunner as a mini-batch, and uses external pm server to avoid
> the heavy cost of synchronization of shared variables such as
> gradients or parameters. Does it make sense?
>
> On Wed, May 27, 2015 at 9:34 AM, Edward J. Yoon <ed...@samsung.com> wrote:
>> According to their architecture, DNN job is consist of multiple (mini-batch)
>> SGD job groups and 'ParameterServer' is used for swapping parameters among
>> them. Since these mini-batch jobs need to be work asynchronously, I guess we
>> can't do this in single job of Hama BSP framework. Hama can be used as a sub
>> component.
>>
>> --
>> Best Regards, Edward J. Yoon
>>
>> -----Original Message-----
>> From: Chia-Hung Lin [mailto:clin4j@googlemail.com]
>> Sent: Tuesday, May 26, 2015 5:40 PM
>> To: dev@hama.apache.org
>> Subject: Re: DNN at Tencent
>>
>> That's why I am working on refactoring core module stuff : )
>>
>> BSP is suitable for iterative applications and is for general purpose
>> parallelism. So as long as the application can be expressed as
>> iterative algorithm, ideally our framework (in terms of higher
>> abstraction level) should have no problem for such applications.
>>
>>
>> On 26 May 2015 at 16:08, Edward J. Yoon <ed...@apache.org> wrote:
>>> Just FYI, http://www.vldb.org/pvldb/vol7/p1772-tencent.pdf
>>>
>>> According to them, they used BSP model (and additional Parameter
>>> server) like google's distbelief. Do you think we can do this?
>>>
>>>
>>> --
>>> Best Regards, Edward J. Yoon
>>
>>
>
>
>
> --
> Best Regards, Edward J. Yoon
--
Best Regards, Edward J. Yoon
Re: DNN at Tencent
Posted by "Edward J. Yoon" <ed...@apache.org>.
Oh wait. a DNN BSPJob partitions the input data (examples) into the
number of tasks, then each task executes SGD graph job using
LocalBSPRunner as a mini-batch, and uses external pm server to avoid
the heavy cost of synchronization of shared variables such as
gradients or parameters. Does it make sense?
On Wed, May 27, 2015 at 9:34 AM, Edward J. Yoon <ed...@samsung.com> wrote:
> According to their architecture, DNN job is consist of multiple (mini-batch)
> SGD job groups and 'ParameterServer' is used for swapping parameters among
> them. Since these mini-batch jobs need to be work asynchronously, I guess we
> can't do this in single job of Hama BSP framework. Hama can be used as a sub
> component.
>
> --
> Best Regards, Edward J. Yoon
>
> -----Original Message-----
> From: Chia-Hung Lin [mailto:clin4j@googlemail.com]
> Sent: Tuesday, May 26, 2015 5:40 PM
> To: dev@hama.apache.org
> Subject: Re: DNN at Tencent
>
> That's why I am working on refactoring core module stuff : )
>
> BSP is suitable for iterative applications and is for general purpose
> parallelism. So as long as the application can be expressed as
> iterative algorithm, ideally our framework (in terms of higher
> abstraction level) should have no problem for such applications.
>
>
> On 26 May 2015 at 16:08, Edward J. Yoon <ed...@apache.org> wrote:
>> Just FYI, http://www.vldb.org/pvldb/vol7/p1772-tencent.pdf
>>
>> According to them, they used BSP model (and additional Parameter
>> server) like google's distbelief. Do you think we can do this?
>>
>>
>> --
>> Best Regards, Edward J. Yoon
>
>
--
Best Regards, Edward J. Yoon
RE: DNN at Tencent
Posted by "Edward J. Yoon" <ed...@samsung.com>.
According to their architecture, DNN job is consist of multiple (mini-batch)
SGD job groups and 'ParameterServer' is used for swapping parameters among
them. Since these mini-batch jobs need to be work asynchronously, I guess we
can't do this in single job of Hama BSP framework. Hama can be used as a sub
component.
--
Best Regards, Edward J. Yoon
-----Original Message-----
From: Chia-Hung Lin [mailto:clin4j@googlemail.com]
Sent: Tuesday, May 26, 2015 5:40 PM
To: dev@hama.apache.org
Subject: Re: DNN at Tencent
That's why I am working on refactoring core module stuff : )
BSP is suitable for iterative applications and is for general purpose
parallelism. So as long as the application can be expressed as
iterative algorithm, ideally our framework (in terms of higher
abstraction level) should have no problem for such applications.
On 26 May 2015 at 16:08, Edward J. Yoon <ed...@apache.org> wrote:
> Just FYI, http://www.vldb.org/pvldb/vol7/p1772-tencent.pdf
>
> According to them, they used BSP model (and additional Parameter
> server) like google's distbelief. Do you think we can do this?
>
>
> --
> Best Regards, Edward J. Yoon
Re: DNN at Tencent
Posted by Chia-Hung Lin <cl...@googlemail.com>.
That's why I am working on refactoring core module stuff : )
BSP is suitable for iterative applications and is for general purpose
parallelism. So as long as the application can be expressed as
iterative algorithm, ideally our framework (in terms of higher
abstraction level) should have no problem for such applications.
On 26 May 2015 at 16:08, Edward J. Yoon <ed...@apache.org> wrote:
> Just FYI, http://www.vldb.org/pvldb/vol7/p1772-tencent.pdf
>
> According to them, they used BSP model (and additional Parameter
> server) like google's distbelief. Do you think we can do this?
>
>
> --
> Best Regards, Edward J. Yoon