[GitHub] szha commented on issue #9543: RNN Op Changes (was: Variable Length Support for cuDNN RNN)

[GitBox <gi...@apache.org>]

szha commented on issue #9543: RNN Op Changes (was: Variable Length Support for cuDNN RNN)
URL: https://github.com/apache/incubator-mxnet/issues/9543#issuecomment-415846898
 
 
   Awesome new set of features! They can be found in https://docs.nvidia.com/deeplearning/sdk/cudnn-developer-guide/index.html#features-of-rnn-functions. Since the changes will likely require API changes in RNN op, I took the liberty to repurpose @sbodenstein 's issue for the API discussion.
   
   Desirable new features would be:
   - Allow packed sequence data or padding handling for variable length sequences.
   - Support LSTM-projection for large-scale RNNs.
   - RNN state clipping.
   
   Desirable performance improvements would be:
   - Allow down conversion to enable HMMA when possible.
   - Allow algorithm search for performance tuning through `findIntensity`.
   - Reuse the space for random states when dropout is enabled.
   
   Unclear:
   - What's the recommendation for `_ALGO_PERSIST_*`? @DickJC123 @ptrendx 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services