[GitHub] safrooze opened a new issue #11411: nd.softmax() doesn't support grad_req='add'

[GitBox <gi...@apache.org>]

safrooze opened a new issue #11411: nd.softmax() doesn't support grad_req='add'
URL: https://github.com/apache/incubator-mxnet/issues/11411
 
 
   ## Description
   `nd.softmax()` does not support adding gradients specified by setting `grad_req` to `'add'`. This is obvious in [this line of code](https://github.com/apache/incubator-mxnet/blob/a0a52b38a9046a3ede20c900ff64154642e8e2da/src/operator/nn/softmax-inl.h#L267). 
   However, `nd.SoftmaxActivation` which is deprecated does support `grad_req` equal to `'add'`. This issue came up in [this discuss questions](https://discuss.mxnet.io/t/aggregate-gradients-manually-over-n-batches/504/14?u=safrooze).
   
   Package used: Python
   
   ## Error Message:
   ```
   mxnet.base.MXNetError: [18:24:22] src/operator/nn/./softmax-inl.h:267: Check failed: req[0] != kAddTo (3 vs. 3)
   ```
   
   ## Minimum reproducible example
   ```python
   import mxnet as mx
   from mxnet import gluon, nd, autograd
   
   print(mx.__version__)
   
   grad_req = 'add'
   
   x = (nd.random.uniform(shape=(2, 10)) - 0.5) * 10
   x_g = nd.zeros_like(x)
   
   autograd.mark_variables(x, x_g, grad_req)
   
   with autograd.record():
       # Change nd.softmax to nd.SoftmaxActivation and the code doesn't throw an error
       y = nd.softmax(x)
   y.backward()
   print(x_g.asnumpy())
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services