You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@systemds.apache.org by "e-strauss (via GitHub)" <gi...@apache.org> on 2024/07/21 15:51:42 UTC

[PR] BI-LSTM layer: forward pass [systemds]

e-strauss opened a new pull request, #2051:
URL: https://github.com/apache/systemds/pull/2051

   This patch adds a dml script for the bidirectional LSTM layer. The script leverages the existing functionality of the standard LSTM layer for computing one pass in the normal direction and one in the reversed direction by reversing the order input tokens on the temporal dimension.
   
   I tried out two approaches for the reversing and ran small benchmark locally, and chose the better performing one.
   The faster approach 1 uses the rev() operator together with transpose(), since rev() only reverses on column axis. 
   `X_reverse = t(rev(t(X)))                    # reverse the elements inside a row`
   `W_reverse[1:D,] = rev(W_reverse[1:D,])      # have to reverse the input weights as well`
   
   The other approach 2 uses a for loop over the temporal axis and slicing.
   `X_reverse = matrix(0, rows=nrow(X), cols=ncol(X))`
   `for (i in 1:T){X_reverse[,(T - i)*D+1:(T - i + 1)*D] = X[,(i-1)*D+1:i*D]}`
   
   I added test cases which compare the results to csv files with expected results that were computed using pytorch's Bi-LSTM implementation.
   
   Currently, the backward is not included in the commit, since I have to fix an error, that I found while testing. I'll add another commit / do another PR for backward pass today or tomorrow once I got it fixed.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@systemds.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BI-LSTM layer: forward pass [systemds]

Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #2051:
URL: https://github.com/apache/systemds/pull/2051#issuecomment-2244991027

   ## [Codecov](https://app.codecov.io/gh/apache/systemds/pull/2051?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   All modified and coverable lines are covered by tests :white_check_mark:
   > Project coverage is 68.83%. Comparing base [(`e6eebb9`)](https://app.codecov.io/gh/apache/systemds/commit/e6eebb9c900b7de1f6e97b98386a8d88c41c071b?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to head [(`95981fa`)](https://app.codecov.io/gh/apache/systemds/commit/95981fa629b94815f56d92d23f06f3c55b2f17da?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
   > Report is 1 commits behind head on main.
   
   <details><summary>Additional details and impacted files</summary>
   
   
   ```diff
   @@            Coverage Diff            @@
   ##               main    #2051   +/-   ##
   =========================================
     Coverage     68.83%   68.83%           
   - Complexity    40757    40758    +1     
   =========================================
     Files          1440     1440           
     Lines        161688   161688           
     Branches      31448    31448           
   =========================================
   + Hits         111296   111299    +3     
   + Misses        41338    41334    -4     
   - Partials       9054     9055    +1     
   ```
   
   </details>
   
   [:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/systemds/pull/2051?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).   
   :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@systemds.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BI-LSTM layer: forward pass [systemds]

Posted by "e-strauss (via GitHub)" <gi...@apache.org>.
e-strauss commented on PR #2051:
URL: https://github.com/apache/systemds/pull/2051#issuecomment-2243915948

   Forward & backward pass is now working. Approach 1 showed better performance in the backward pass as well.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@systemds.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BI-LSTM layer: forward pass [systemds]

Posted by "mboehm7 (via GitHub)" <gi...@apache.org>.
mboehm7 commented on PR #2051:
URL: https://github.com/apache/systemds/pull/2051#issuecomment-2254211416

   LGTM - thanks for the patch @e-strauss, this is really great.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@systemds.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BI-LSTM layer: forward pass [systemds]

Posted by "mboehm7 (via GitHub)" <gi...@apache.org>.
mboehm7 closed pull request #2051: BI-LSTM layer: forward pass
URL: https://github.com/apache/systemds/pull/2051


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@systemds.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org