[GitHub] aidan-plenert-macdonald commented on issue #8260: Trying to use random in hybrid blocks - "TypeError: type not supported"

[gi...@git.apache.org]

aidan-plenert-macdonald commented on issue #8260: Trying to use random in hybrid blocks - "TypeError: type <class 'mxnet.gluon.parameter.Parameter'> not supported"
URL: https://github.com/apache/incubator-mxnet/issues/8260#issuecomment-336593975
 
 
   I fixed the error above,
   
   ```python
   class NoiseDense(gluon.nn.Dense):
       def __init__(self, *args, weight_std_initializer=None, bias_std_initializer='zeros', **kwargs):
           super(NoiseDense, self).__init__(*args, **kwargs)
           with self.name_scope():
               self.weight_std = self.params.get('weight_std', shape=(self._units, self._in_units),
                                                 init=weight_std_initializer,
                                                 allow_deferred_init=True)
               if self.bias is not None:
                   self.bias_std = self.params.get('bias_std', shape=(self._units,),
                                                   init=bias_std_initializer,
                                                   allow_deferred_init=True)
               else:
                   self.bias_std = None
       
       def hybrid_forward(self, F, x, weight, weight_std=None, bias=None, bias_std=None):
           w, b = weight, bias
           print(w, b, weight_std, bias_std)
           if weight_std is not None:
               w = w + sym.random_normal(shape=(self._units, self._in_units)) * weight_std
           if bias_std is not None:
               b = b + sym.random_normal(shape=(self._units,)) * bias_std
           print(w, b)
           return super(NoiseDense, self).hybrid_forward(F, x, weight=w, bias=b)
   ```
   
   which gets the new error,
   ```
   <ipython-input-24-bea0d37548c3> in hybrid_forward(self, F, x, weight, weight_std, bias, bias_std)
        17         print(w, b, weight_std, bias_std)
        18         if weight_std is not None:
   ---> 19             w = w + sym.random_normal(shape=(self._units, self._in_units)) * weight_std
        20         if bias_std is not None:
        21             b = b + sym.random_normal(shape=(self._units,)) * bias_std
   
   /usr/local/lib/python3.6/site-packages/mxnet/symbol/symbol.py in __mul__(self, other)
       148             return _internal._MulScalar(self, scalar=other)
       149         else:
   --> 150             raise TypeError('type %s not supported' % str(type(other)))
       151 
       152     def __imul__(self, other):
   
   TypeError: type <class 'mxnet.ndarray.ndarray.NDArray'> not supported
   ```
   
   This is caused because,
   ```python
   self.weight_std
   ```
   is getting initialized to an NDArray instead of a Symbol. Not sure how to fix this.
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services