[GitHub] [incubator-mxnet] wkcn commented on a change in pull request #14427: Copy the gradient to the new context when as_in_context() is called

[GitBox <gi...@apache.org>]

wkcn commented on a change in pull request #14427: Copy the gradient to the new context when as_in_context() is called
URL: https://github.com/apache/incubator-mxnet/pull/14427#discussion_r265417769
 
 

 ##########
 File path: python/mxnet/ndarray/ndarray.py
 ##########
 @@ -2086,12 +2088,17 @@ def copyto(self, other):
             if other.handle is self.handle:
                 warnings.warn('You are attempting to copy an array to itself', RuntimeWarning)
                 return False
-            return _internal._copyto(self, out=other)
+            data = _internal._copyto(self, out=other)
         elif isinstance(other, Context):
             hret = NDArray(_new_alloc_handle(self.shape, other, True, self.dtype))
-            return _internal._copyto(self, out=hret)
+            data = _internal._copyto(self, out=hret)
         else:
             raise TypeError('copyto does not support type ' + str(type(other)))
+        if copy_grad:
+            grad = self.grad
+            if grad is not None:
+                setattr(data, '_old_grad', grad.copyto(other, copy_grad=False))
 
 Review comment:
   There is a case:
   ```python
   a = mx.nd.array([1,2,3],ctx=mx.cpu())
   a.attach_grad()
   b = a.as_in_context()
   print(b.grad) # return a gradient which is equal to a.grad
   b.attach_grad()
   print(b.grad) # return a new gradient
   ```
   I need the attribute `_old_grad` to store the gradient.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services