You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mxnet.apache.org by GitBox <gi...@apache.org> on 2020/04/30 10:25:00 UTC
[GitHub] [incubator-mxnet] bgawrych commented on a change in pull request #18203: Fix LSTM and GRU layers gradient calculations
bgawrych commented on a change in pull request #18203:
URL: https://github.com/apache/incubator-mxnet/pull/18203#discussion_r417909239
##########
File path: src/operator/rnn-inl.h
##########
@@ -195,7 +195,9 @@ inline size_t GetRNNWorkspaceSize(index_t seq_length,
case rnn_enum::kLstm:
size = seq_length * batch_size * hidden_size * (4 + direction) + // wx*x + inter-y
batch_size * hidden_size * 6 + // wh*h + h + c
- seq_length * hidden_size * 8; // Used in Backward, Δbx, Δbh
+ seq_length * hidden_size * 8 + // Used in Backward, Δbx, Δbh
+ // temporary dy in backward computation for bidirectional layers
Review comment:
what i meant here is temporary space required to derivative in r2l layer - not temporary solution
##########
File path: src/operator/rnn-inl.h
##########
@@ -195,7 +195,9 @@ inline size_t GetRNNWorkspaceSize(index_t seq_length,
case rnn_enum::kLstm:
size = seq_length * batch_size * hidden_size * (4 + direction) + // wx*x + inter-y
batch_size * hidden_size * 6 + // wh*h + h + c
- seq_length * hidden_size * 8; // Used in Backward, Δbx, Δbh
+ seq_length * hidden_size * 8 + // Used in Backward, Δbx, Δbh
+ // temporary dy in backward computation for bidirectional layers
Review comment:
what i meant here is temporary space required for derivative in r2l layer - not temporary solution
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org