[GitHub] sbodenstein opened a new issue #9543: Variable Length Support for cuDNN RNN

[GitBox <gi...@apache.org>]

sbodenstein opened a new issue #9543: Variable Length Support for cuDNN RNN
URL: https://github.com/apache/incubator-mxnet/issues/9543
 
 
   We want to add support for variable-length sequences to the cuDNN RNN operator, as we cannot 'fake' support via masking for LSTM (the cuDNN operator does not return a history of cell states) and bidirectional RNNs. Also, there should be some speedups for batches of sequences of very different lengths (can avoid bucketing).
   
   The annoying part is that cuDNN requires that the sequences be sorted by sequence length, and also packed as a contiguous block of memory. PyTorch deals with this by introducing a [pack_padded_sequence](http://pytorch.org/docs/master/nn.html#torch.nn.utils.rnn.pack_padded_sequence) operator, which returns a `PackedSequence` object that the user can feed to cuDNN RNN. MXNet probably (?) doesn't want to take this route, and hide these details from the user (other backends eg upcoming MKL for `symbol.rnn` might anyways not require this) and just pack/unpack inside `symbol.RNN`. So some questions: 
   - is any work being done on this already? (so I don't duplicate)
   - @DickJC123: what is the best way of doing the sorted packing/unpacking efficiently? Are there any plans to add this utility function to cuDNN? 
   
   @sxjscience, @piiswrong, @szha 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services