[GitHub] [tvm] AndrewZhaoLuo commented on a diff in pull request #11171: [ONNX] Fix cast op to/from bfloat16

[GitBox <gi...@apache.org>]

AndrewZhaoLuo commented on code in PR #11171:
URL: https://github.com/apache/tvm/pull/11171#discussion_r862251360


##########
tests/python/frontend/onnx/test_forward.py:
##########
@@ -5174,6 +5172,10 @@ def test_onnx_nodes(target, dev, onnx_test):
         # roialign results to 4 decimal places
         atol = 1e-4
 
+    if "to_BFLOAT16" in test_dir:
+        # casting to BFLOAT has a larger accuracy loss
+        atol = 1

Review Comment:
   The errors might be large since you are comparing things in uint16 space which might not be indicative. 
   
   E.g. 0 (e.g. 0x0000) and -0 (e.g. 0x1000) would differ by a lot in uint space but not at all in bfloat state
    I would cast back to bfloat16 compatible state
   
   E.g. https://en.wikipedia.org/wiki/Half-precision_floating-point_format
   Which gives the float64 number representing the sameish number as a bfloat16 cast. It does not handle special values though



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org