[jira] [Commented] (SYSTEMML-1389) Update API: Pass in all outputs from `forward` to `backward` for performance

["Mike Dusenberry (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SYSTEMML-1389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15948061#comment-15948061 ] 

Mike Dusenberry commented on SYSTEMML-1389:
-------------------------------------------

cc [~niketanpansare]  This will require changes to the Barista API, but I believe it will be beneficial for performance.

> Update API: Pass in all outputs from `forward` to `backward` for performance
> ----------------------------------------------------------------------------
>
>                 Key: SYSTEMML-1389
>                 URL: https://issues.apache.org/jira/browse/SYSTEMML-1389
>             Project: SystemML
>          Issue Type: Sub-task
>            Reporter: Mike Dusenberry
>            Assignee: Mike Dusenberry
>
> Currently, we do not pass the outputs of the {{forward}} functions to the {{backward}} functions in the {{nn}} library.  This aims to update the {{backward}} API to include (1) all relevant gradients from upstream, (2) *all* outputs from {{forward}}, and (3) *all* inputs given to {{forward}}.  Effectively, this would be equivalent to having a class object that maintains all configuration and input + output tensors.  This provides two benefits: first, many layers can benefit from a performance perspective from having access to the outputs of the {{forward}} function within the {{backward}} function, and second, this makes the API much simpler and less error prone by allowing for simple copy-and-paste of forward inputs and outputs as arguments to {{backward}} and by removing ambiguity related to the parameters.  A downside is that often times, not every single parameter is needed by {{backward}}.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)