You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@systemds.apache.org by "e-strauss (via GitHub)" <gi...@apache.org> on 2024/07/21 15:51:42 UTC
[PR] BI-LSTM layer: forward pass [systemds]
e-strauss opened a new pull request, #2051:
URL: https://github.com/apache/systemds/pull/2051
This patch adds a dml script for the bidirectional LSTM layer. The script leverages the existing functionality of the standard LSTM layer for computing one pass in the normal direction and one in the reversed direction by reversing the order input tokens on the temporal dimension.
I tried out two approaches for the reversing and ran small benchmark locally, and chose the better performing one.
The faster approach 1 uses the rev() operator together with transpose(), since rev() only reverses on column axis.
`X_reverse = t(rev(t(X))) # reverse the elements inside a row`
`W_reverse[1:D,] = rev(W_reverse[1:D,]) # have to reverse the input weights as well`
The other approach 2 uses a for loop over the temporal axis and slicing.
`X_reverse = matrix(0, rows=nrow(X), cols=ncol(X))`
`for (i in 1:T){X_reverse[,(T - i)*D+1:(T - i + 1)*D] = X[,(i-1)*D+1:i*D]}`
I added test cases which compare the results to csv files with expected results that were computed using pytorch's Bi-LSTM implementation.
Currently, the backward is not included in the commit, since I have to fix an error, that I found while testing. I'll add another commit / do another PR for backward pass today or tomorrow once I got it fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@systemds.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BI-LSTM layer: forward pass [systemds]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #2051:
URL: https://github.com/apache/systemds/pull/2051#issuecomment-2244991027
## [Codecov](https://app.codecov.io/gh/apache/systemds/pull/2051?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 68.83%. Comparing base [(`e6eebb9`)](https://app.codecov.io/gh/apache/systemds/commit/e6eebb9c900b7de1f6e97b98386a8d88c41c071b?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to head [(`95981fa`)](https://app.codecov.io/gh/apache/systemds/commit/95981fa629b94815f56d92d23f06f3c55b2f17da?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
> Report is 1 commits behind head on main.
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## main #2051 +/- ##
=========================================
Coverage 68.83% 68.83%
- Complexity 40757 40758 +1
=========================================
Files 1440 1440
Lines 161688 161688
Branches 31448 31448
=========================================
+ Hits 111296 111299 +3
+ Misses 41338 41334 -4
- Partials 9054 9055 +1
```
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/systemds/pull/2051?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@systemds.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BI-LSTM layer: forward pass [systemds]
Posted by "e-strauss (via GitHub)" <gi...@apache.org>.
e-strauss commented on PR #2051:
URL: https://github.com/apache/systemds/pull/2051#issuecomment-2243915948
Forward & backward pass is now working. Approach 1 showed better performance in the backward pass as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@systemds.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BI-LSTM layer: forward pass [systemds]
Posted by "mboehm7 (via GitHub)" <gi...@apache.org>.
mboehm7 commented on PR #2051:
URL: https://github.com/apache/systemds/pull/2051#issuecomment-2254211416
LGTM - thanks for the patch @e-strauss, this is really great.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@systemds.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BI-LSTM layer: forward pass [systemds]
Posted by "mboehm7 (via GitHub)" <gi...@apache.org>.
mboehm7 closed pull request #2051: BI-LSTM layer: forward pass
URL: https://github.com/apache/systemds/pull/2051
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@systemds.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org