[GitHub] rahul003 opened a new issue #11403: L2 norm for fp16 needs to be fixed

[GitBox <gi...@apache.org>]

rahul003 opened a new issue #11403: L2 norm for fp16 needs to be fixed
URL: https://github.com/apache/incubator-mxnet/issues/11403
 
 
   When .norm() method is called on a float16 array it tries to compute the norm in fp16, which causes the squared sum to go out of range and causes nan as output. Large normalizations should be kept in float32, similar to how it's done for softmax and batchnorm. Ideally this could be done by accumulating the squared sum into a float32 buffer. 
   This problem might affect the L2 norm operator as well. Haven't verified that.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services