[GitHub] [incubator-mxnet] hankaj commented on issue #20947: Inconsistent behavior of mx.sym.LeakyReLU between CPU and GPU

[GitBox <gi...@apache.org>]

hankaj commented on issue #20947:
URL: https://github.com/apache/incubator-mxnet/issues/20947#issuecomment-1194171973

   Hi! I investigated the issue and the case is that MxNet implementation of Leaky ReLU is different than OneDNN approach. Building the project without OneDNN (flag USE_ONEDNN set to 0) results in the same results for CPU and GPU (nan for -inf and for nans). In OneDNN, for slope=0.0 multiplication by 0 does not occur. They use plain ReLU in this case (max(0, z)) and they would prefer it to stay that way to keep the consistency with older oneDNN versions.
   
   Is it okay to close the issue considering the fact that it is clarified where the difference comes from?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For additional commands, e-mail: issues-help@mxnet.apache.org