You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@systemml.apache.org by "Mike Dusenberry (JIRA)" <ji...@apache.org> on 2017/06/28 19:56:00 UTC
[jira] [Updated] (SYSTEMML-1389) Update API: Pass in all outputs
from `forward` to `backward` for performance
[ https://issues.apache.org/jira/browse/SYSTEMML-1389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mike Dusenberry updated SYSTEMML-1389:
--------------------------------------
Issue Type: Improvement (was: Sub-task)
Parent: (was: SYSTEMML-618)
> Update API: Pass in all outputs from `forward` to `backward` for performance
> ----------------------------------------------------------------------------
>
> Key: SYSTEMML-1389
> URL: https://issues.apache.org/jira/browse/SYSTEMML-1389
> Project: SystemML
> Issue Type: Improvement
> Reporter: Mike Dusenberry
> Assignee: Mike Dusenberry
> Labels: SYSTEMML-618
>
> Currently, we do not pass the outputs of the {{forward}} functions to the {{backward}} functions in the {{nn}} library. This aims to update the {{backward}} API to include (1) all relevant gradients from upstream, (2) *all* outputs from {{forward}}, and (3) *all* inputs given to {{forward}}. Effectively, this would be equivalent to having a class object that maintains all configuration and input + output tensors. If we had an object oriented design, most of the parameters to `forward` and `backward` would just be instance variables and thus would be accessible by `forward` or `backward` as needed. Given that we don't have that design, this API update mimics that by allowing `backward` to have access to any inputs available to `forward`, as well as any outputs produced by `forward`. This provides two benefits: first, many layers can benefit from a performance perspective from having access to the outputs of the {{forward}} function within the {{backward}} function, and second, this makes the API much simpler and less error prone by allowing for simple copy-and-paste of forward inputs and outputs as arguments to {{backward}} and by removing ambiguity related to the parameters. A downside is that often times, not every single parameter is needed by {{backward}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)