[jira] [Commented] (SINGA-58) Fix fan-in dimension of weight matrix

["ASF subversion and git services (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SINGA-58?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14698171#comment-14698171 ] 

ASF subversion and git services commented on SINGA-58:
------------------------------------------------------

Commit fcd377aed543f5a44deeb3145551b107e6cc2324 in incubator-singa's branch refs/heads/master from Wei Wang
[ https://git-wip-us.apache.org/repos/asf?p=incubator-singa.git;h=fcd377a ]

SINGA-58 Fix fan-in dimension of weight matrix

Use the num of cols of a weight matrix as its fan-in.
Layer that have weight matrix should follow this assumption.
Otherwise, if there would be errors when the weight matrix is initialized based on fan-in.


> Fix fan-in dimension of weight matrix
> -------------------------------------
>
>                 Key: SINGA-58
>                 URL: https://issues.apache.org/jira/browse/SINGA-58
>             Project: Singa
>          Issue Type: Improvement
>            Reporter: wangwei
>
> There are many layers using weight matrix, e.g., convolution layer and innerproduct layer. Some initialization methods of the weight matrix are based on the fan-in of the matrix, i.e., the dimension interacts with the other operator. E.g., for v*W, W's fan-in is the number of rows; For v*W^T, W's fan-in is the number of columns.
> However, the Param::InitValues() method does not know the position (and transposition) of the matrix in the multiplication operation. We have to fix the fan-in dimension. Particularly, we fix it to the second dimension (i.e., number of columns) and assume users use the weight matrix correctly.
> The current implementation of convolution layer and innerproduct layer have different fan-in dimension, which should be unified to the second dimension. For the convolution layer, its operation is W*v. W's fan-in is already the number of columns. For the innerproduct layer, the current operation is v*W, which should be updated to v*W^T, then the fan-in is the number of columns.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)