[GitHub] [incubator-mxnet] grygielski commented on issue #19323: Variable sequence length not handled correctly for BiDirectional layers

[GitBox <gi...@apache.org>]

grygielski commented on issue #19323:
URL: https://github.com/apache/incubator-mxnet/issues/19323#issuecomment-708325735


   Hi @zjost currently CPU does not support `use_sequence_length=True` in RNN layers. It seems that this information is missing in MKLDNN execution path but when you run this code with `export MXNET_USE_MKLDNN_RNN=0` environment variable you will get the following error: `MXNetError: RNN use_sequence_length option is only available for cuDNN version >= 7.2`.
   
   Your solution to use `F.SequenceMask(x, sequence_length=x_len, use_sequence_length=True)` is equivalent to setting `pad_val = 0` instead of `-1`. However, it's not proper solution and it happened to work by accident. Padding with 0s yields correct result for bidirectional RNN layers only if all biases are equal to 0 which is the case here (default initializer for bias is `zero`). You can check it by changing LSTM layer initialization in your model to:
   ```
   self.rnn = gluon.rnn.LSTM(hidden_size=1, bidirectional=bidirectional, input_size=1, use_sequence_length=True,
                             h2h_bias_initializer='one', i2h_bias_initializer='one')
   ```
   For now, my suggestion would be to either use `batch_size=1` or group sentences into batches of equal length.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For additional commands, e-mail: issues-help@mxnet.apache.org