[GitHub] mkolod opened a new pull request #11380: Add ability to query cuDNN BatchNorm min. epsilon. Allow ONNX importer to use cuDNN BN if chosen eps >= cuDNN min. aps.

[GitBox <gi...@apache.org>]

mkolod opened a new pull request #11380: Add ability to query cuDNN BatchNorm min. epsilon. Allow ONNX importer to use cuDNN BN if chosen eps >= cuDNN min. aps.
URL: https://github.com/apache/incubator-mxnet/pull/11380
 
 
   ## Description ##
   The ONNX importer [always disables](https://github.com/apache/incubator-mxnet/blob/master/python/mxnet/contrib/onnx/_import/op_translations.py#L212) cuDNN BatchNorm when importing models. The reason for this is that if the epsilon chosen in the model is less than the cuDNN minimum (CUDNN_BN_MIN_EPSILON), the cuDNN call will fail. However, cuDNN BatchNorm need not be always disabled when importing models. This PR adds an ability to query the value of CUDNN_BN_MIN_EPSILON, and only disable cuDNN BatchNorm during ONNX import if the value from the ONNX model is less than the cuDNN minimum. The ability to query the minimum cuDNN value can also be helpful in other scenarios, such as when verifying if the value in a model is too small for numeric stability (the cuDNN value is quite reasonable as a minimum).
   
   ## Checklist ##
   ### Essentials ###
   Please feel free to remove inapplicable items for your PR.
   - [x] Changes are complete (i.e. I finished coding on this PR)
   - [x] All changes have test coverage:
   - Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
   - [x] Code is well-documented: 
   - This is a tiny change so only the call to get the value of CUDNN_BN_MIN_EPSILON (mx.base.get_cudnn_epsilon) needs documentation
   - [x] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change
   
   ### Changes ###
   - [x] `mx.base.get_cudnn_epsilon()` implementation (`python/mxnet/base.py`)
   - [x] test of `get_cudnn_epsilon()` at `tests/python/gpu/test_cudnn_eps.py`
   - [x] adaptation of the ONNX importer to only disable cuDNN BatchNorm if cuDNN is not available, or the epsilon from the ONNX model is less than CUDNN_BN_MIN_EPSILON (`python/mxnet/contrib/onnx/_import/op_translations.py`)
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services