Re: [I] [Bug] ELU produced inconsistent inference results with PyTorch [tvm]

["hxzd5568 (via GitHub)" <gi...@apache.org>]

hxzd5568 commented on issue #15396:
URL: https://github.com/apache/tvm/issues/15396#issuecomment-1934212486

   This is a numerical bug caused by 'FastMath' pass on exp operator decomposed from ELU.
   Disabling 'FastMath' or enhancing the precision can solve it.
   
   ```
   import torch
   from tvm import relay
   import tvm
   import numpy as np
   
   m = torch.nn.ELU(alpha=-1.8574e+38,)
   input_data=torch.tensor(torch.randn([9, 12, 19, 5, 10], dtype=torch.float64))
   torch_outputs = m(input_data)
   trace = torch.jit.trace(m, input_data)
   input_shapes = [('input0', torch.Size([9, 12, 19, 5, 10]))]
   
   mod, params = relay.frontend.from_pytorch(trace, input_shapes)
   print(mod)
   input_data_np = input_data.numpy()
   with tvm.transform.PassContext(opt_level=3,):#disabled_pass=['FastMath']
       exe = relay.create_executor('graph', mod=mod, params=params, device=tvm.device('llvm', 0), target='llvm').evaluate()
   input_tvm = {'input0': tvm.nd.array(input_data_np.astype(np.float64))}
   tvm_outputs = exe(**input_tvm).asnumpy()
   np.testing.assert_allclose(torch_outputs, tvm_outputs, rtol=1e-5, atol=1e-5)
   ```
   
   ## How does the error arise
   TVM decomposes ELU to exp-sub-relu-mul-relu.
   FastMath pass converts exp to fast_exp and causes the errors.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org