[GitHub] lihaofd opened a new pull request #10289: [WIP][MXNET-107]Fused GRU implementation for CPU

[GitBox <gi...@apache.org>]

lihaofd opened a new pull request #10289: [WIP][MXNET-107]Fused GRU implementation for CPU
URL: https://github.com/apache/incubator-mxnet/pull/10289
 
 
   ## Description ##
   In this PR, it aligns with #10104's NNVM registration way and creates a fused GRU operator for CPU.
   @pengzhao-intel, @TaoLv , @Sherry-zhang 
   
   ## Feature changes ##
   ### New features ###
   - Single layer and unidirectional GRU, including both forward and backward computation.
   
   ### Removed features ###
   - Since the code is refactored with NNVM interfaces, comment rnn.cu as it is tracked on #10104.
   
   ### Unit-test changes ###
   - Create new test test_gru in tests/python/unittests/test_operator.py.
   - Check consistency with original GRUCell implementation.
   - Check consistency of FP32 and FP64 outputs.
   
   ### Performance ###
   We have tested performance of sym.RNN and rnn.GRUCell on our local Skylake-6148 with 2 Sockets and 40 cores. Use MKL as blas lib in this performance test.
   Test input size is from DS2 default parameters(seq_length = 300, batch_size = 20, input_size = 800, hidden_size = 800).
   
   | API | Inference time(fwd, sec)  |  Training time(fwd + bwd, sec)           |
   | --------           | :-----:                              | :----:                                   |
   | rnn.GRUCell  | 0.12                       |  0.277                            |
   | this PR            | 0.048                      |  0.106                              |
   | speedup         | 2.5x                              |  2.6x                                    | 
   
   ## Opens ##
   - [ ] Add multi-layer and bidirectional support for GRU in the future.
   
   ## Checklist ##
    - [X] Passed code style checking (make lint).
    - [X] All changes have test coverage.
    - [ ] Code is well-documented.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services