[GitHub] [incubator-mxnet] arcadiaphy edited a comment on issue #14490: initializer not work excepted

[GitBox <gi...@apache.org>]

arcadiaphy edited a comment on issue #14490: initializer not work excepted
URL: https://github.com/apache/incubator-mxnet/issues/14490#issuecomment-475287961
 
 
   @EvenIsLee `net.initialize` only set global default init, which has lowest precedence. The correct way to set init:
   ```
   import mxnet as mx
   from mxnet.gluon import nn
   
   class InitWeight(mx.init.Initializer):
       def _init_weight(self, name, arr):
           arr[:] = 0.5
   
   class InitBias(mx.init.Initializer):
       def _init_weight(self, name, arr):
           arr[:] = 1.0
   
   
   class InitGamma(mx.init.Initializer):
       def _init_weight(self, name, arr):
           arr[:] = 2.0
   
   class Mymodel(nn.Block):
       def __init__(self):
           super(Mymodel, self).__init__()
   
           with self.name_scope():
               self.dense = nn.Dense(128, in_units=16,
                                     weight_initializer=InitWeight(),
                                     bias_initializer=InitBias())
   
       def forward(self, x):
           x = self.dense(x)
           return x
   
   
   if __name__=='__main__':
       data = mx.nd.random.uniform(shape=(1, 16))
       net = Mymodel()
       net.initialize()
       print net.dense.weight.data()
       print net.dense.bias.data()
   ```
   
   But the implementation in gluon in really confusion compared to that in Module:
   ```
       >>> # Create and register a custom initializer that
       ... # initializes weights to 0.1 and biases to 1.
       ...
       >>> @mx.init.register
       ... @alias('myinit')
       ... class CustomInit(mx.init.Initializer):
       ...   def __init__(self):
       ...     super(CustomInit, self).__init__()
       ...   def _init_weight(self, _, arr):
       ...     arr[:] = 0.1
       ...   def _init_bias(self, _, arr):
       ...     arr[:] = 1
       ...
       >>> # Module is an instance of 'mxnet.module.Module'
       ...
       >>> module.init_params("custominit")
       >>> # module.init_params("myinit")
       >>> # module.init_params(CustomInit())
   ```
   
   Sometimes, I want to replace all inits in gluon block just like in Module, and expect `Block.initialize` to be just like `Module.init_params`.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services