[GitHub] [incubator-mxnet] anirudhacharya commented on issue #11865: attach_grad of intermediate variables causes the gradient graph to be lost

[GitBox <gi...@apache.org>]

anirudhacharya commented on issue #11865: attach_grad of intermediate variables causes the gradient graph to be lost
URL: https://github.com/apache/incubator-mxnet/issues/11865#issuecomment-488424194
 
 
   To understand this right, in scenarios such as the following 
   ```python
   x = mx.nd.array([0, 7], ctx = mx.cpu())
   x.attach_grad()
   with mx.autograd.record():
       y = ((5 * (x**2)) + (13 * x) + 10)
       y.attach_grad()
       z = 2 * y
   z.backward()
   print(x.grad)
   ```
   what you want is we should be able to get the gradients w.r.t any variable which has been marked with `attach_grad`.
   
   In the above example would you also want the result of `y.grad` to be retained? Because that would be a bit different and more like a feature request than a bug. It will probably involve storing intermediate gradients of non-leaf variables in some sort of a buffer by providing a hook/function call to the user to enable storing the gradients of non-leaf variables because storing it every time by default might be a waste of memory.
   
   Have I understood this right? Which of the two above situations is your issue pointing to?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services