[jira] [Created] (SINGA-58) Fix fan-in dimension of weight matrix

["wangwei (JIRA)" <ji...@apache.org>]

wangwei created SINGA-58:
----------------------------

             Summary: Fix fan-in dimension of weight matrix
                 Key: SINGA-58
                 URL: https://issues.apache.org/jira/browse/SINGA-58
             Project: Singa
          Issue Type: Improvement
            Reporter: wangwei


There are many layers using weight matrix, e.g., convolution layer and innerproduct layer. Some initialization methods of the weight matrix are based on the fan-in of the matrix, i.e., the dimension interacts with the other operator. E.g., for v*W, W's fan-in is the number of rows; For v*W^T, W's fan-in is the number of columns.

However, the Param::InitValues() method does not know the position (and transposition) of the matrix in the multiplication operation. We have to fix the fan-in dimension. Particularly, we fix it to the second dimension (i.e., number of columns) and assume users use the weight matrix correctly.

The current implementation of convolution layer and innerproduct layer have different fan-in dimension, which should be unified to the second dimension. For the convolution layer, its operation is W*v. W's fan-in is already the number of columns. For the innerproduct layer, the current operation is v*W, which should be updated to v*W^T, then the fan-in is the number of columns.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)