You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@singa.apache.org by "wangwei (JIRA)" <ji...@apache.org> on 2018/07/11 07:19:00 UTC
[jira] [Created] (SINGA-381) Update the autograd API to yeild the
gradients
wangwei created SINGA-381:
-----------------------------
Summary: Update the autograd API to yeild the gradients
Key: SINGA-381
URL: https://issues.apache.org/jira/browse/SINGA-381
Project: Singa
Issue Type: Improvement
Reporter: wangwei
This ticket is going to update one API in autograd.py
{code:java}
def backward(y, dy=None)
# returns the gradient tensor one by one using yield
def gradients(y, dy=None)
# returns a dictionary: param tensor -> gradient tensor{code}
With the backward() API, we can update the param immediately after its gradient is available. Then, the gradient tensor can be deleted and the memory is released.
The gradients() API keeps all gradient tensors in the memory, which incurs memory overhead.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)