[GitHub] szha commented on issue #8494: Autograd bug in mxnet-cu80: 0.12

[GitBox <gi...@apache.org>]

szha commented on issue #8494: Autograd bug in mxnet-cu80: 0.12
URL: https://github.com/apache/incubator-mxnet/issues/8494#issuecomment-340983575
 
 
   Your code just made `b` an alias of `a` which isn't computation.
   
   The following three examples show
   1. the "what have you tried to solve it"
   2. the snippet you posted with the addition of id check for `a` and `b`. it printed `True`.
   3. the snippet where in-place assignment happens. the same id check printed `False`
   
   ```python
   In [1]: import mxnet as mx
      ...:
      ...: a = mx.random.normal(shape = (3, 3, 3))
      ...: a.attach_grad()
      ...:
      ...: b = 0
      ...: with mx.autograd.record():
      ...:     b = a+1
      ...: b.backward()
      ...: a.grad
      ...:
   Out[1]:
   
   [[[ 1.  1.  1.]
     [ 1.  1.  1.]
     [ 1.  1.  1.]]
   
    [[ 1.  1.  1.]
     [ 1.  1.  1.]
     [ 1.  1.  1.]]
   
    [[ 1.  1.  1.]
     [ 1.  1.  1.]
     [ 1.  1.  1.]]]
   <NDArray 3x3x3 @cpu(0)>
   
   In [2]: import mxnet as mx
      ...:
      ...: a = mx.random.normal(shape = (3, 3, 3))
      ...: a.attach_grad()
      ...:
      ...: b = 0
      ...: with mx.autograd.record():
      ...:     b = a
      ...: print(id(a) == id(b))
      ...: b.backward()
      ...: a.grad
      ...:
   True
   Out[2]:
   
   [[[ 0.  0.  0.]
     [ 0.  0.  0.]
     [ 0.  0.  0.]]
   
    [[ 0.  0.  0.]
     [ 0.  0.  0.]
     [ 0.  0.  0.]]
   
    [[ 0.  0.  0.]
     [ 0.  0.  0.]
     [ 0.  0.  0.]]]
   <NDArray 3x3x3 @cpu(0)>
   
   In [3]: import mxnet as mx
      ...:
      ...: a = mx.random.normal(shape = (3, 3, 3))
      ...: a.attach_grad()
      ...:
      ...: b = mx.nd.zeros_like(a)
      ...: with mx.autograd.record():
      ...:     b[:] = a
      ...: print(id(a) == id(b))
      ...: b.backward()
      ...: a.grad
      ...:
   False
   Out[3]:
   
   [[[ 1.  1.  1.]
     [ 1.  1.  1.]
     [ 1.  1.  1.]]
   
    [[ 1.  1.  1.]
     [ 1.  1.  1.]
     [ 1.  1.  1.]]
   
    [[ 1.  1.  1.]
     [ 1.  1.  1.]
     [ 1.  1.  1.]]]
   <NDArray 3x3x3 @cpu(0)>
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services