[GitHub] [incubator-mxnet] pengzhao-intel commented on issue #18826: [Activation] GELU precision mismatch between MXNet and PyTorch in the CPU version

[GitBox <gi...@apache.org>]

pengzhao-intel commented on issue #18826:
URL: https://github.com/apache/incubator-mxnet/issues/18826#issuecomment-670334816


   > @TaoLv Sorry, missed some imports.
   > 
   > ```python
   > import mxnet as mx
   > import math
   > from numpy.testing import assert_allclose
   > mx.npx.set_np()
   > a = mx.np.random.normal(0, 1, (10000,)) 
   > b = mx.npx.leaky_relu(a, act_type='gelu')
   > c = a * 0.5 * (1.0 + mx.npx.erf(a / math.sqrt(2.0)))
   > 
   > import torch
   > a_torch = torch.from_numpy(a.asnumpy())
   > b_torch = torch.nn.functional.gelu(a_torch)
   > assert_allclose(b_torch.cpu().numpy(), c.asnumpy(), 1E-4, 1E-4)  
   > assert_allclose(b_torch.cpu().numpy(), b.asnumpy(), 1E-4, 1E-4)  
   > ```
   > 
   > (Compiling MXNet takes some time for me so it will be helpful if you can check that...)
   
   Does the issue still exist after Tao's PR?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org