You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@singa.apache.org by "wangwei (JIRA)" <ji...@apache.org> on 2015/06/19 08:29:00 UTC

[jira] [Created] (SINGA-19) Slice large Param objects for load-balance

wangwei created SINGA-19:
----------------------------

             Summary: Slice large Param objects for load-balance
                 Key: SINGA-19
                 URL: https://issues.apache.org/jira/browse/SINGA-19
             Project: Singa
          Issue Type: New Feature
            Reporter: wangwei
            Assignee: wangwei


Some Param objects in deep learning model are much larger than other Param objects. For example, the weight matrix is usually 100 times larger than the bias vector. The difference in Param size causes two problems,

1. if there are multiple servers in one server group, then the servers may be assigned different number of parameters to update.
2. if there are multiple server groups, e.g., in distributed Hogwild framework, then these server groups may be assigned different number of parameters to maintain.

This ticket its to slice large Param objects to solve the load-balance problem. The slicing operations are done in the stub thread to make them transparent to both workers and servers.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)