[GitHub] [incubator-mxnet] szha commented on issue #9543: RNN Op Changes (was: Variable Length Support for cuDNN RNN)

["szha (GitHub)" <gi...@apache.org>]

Awesome new set of features! They can be found in https://docs.nvidia.com/deeplearning/sdk/cudnn-developer-guide/index.html#features-of-rnn-functions. Since the changes will likely require API changes in RNN op, I took the liberty to repurpose @sbodenstein 's issue for the API discussion.

Desirable new features would be:
- Allow packed sequence data or padding handling for variable length sequences.
- Support LSTM-projection for large-scale RNNs.
- RNN state clipping.

Desirable performance improvements would be:
- Allow down conversion to enable HMMA when possible.
- Allow algorithm search for performance tuning through `findIntensity`.
- Reuse the space for random states when dropout is enabled.

Unclear:
- What's the recommendation for `_ALGO_PERSIST_*`? @DickJC123 @ptrendx 

[ Full content available at: https://github.com/apache/incubator-mxnet/issues/9543 ]
This message was relayed via gitbox.apache.org for devnull@infra.apache.org