You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2021/12/21 17:18:37 UTC

[GitHub] [tvm] comaniac commented on pull request #9782: [AMP] Disallow converting layer norm to fp16

comaniac commented on pull request #9782:
URL: https://github.com/apache/tvm/pull/9782#issuecomment-998957123


   That's an interesting finding. In general, similar to BatchNorm, LayerNorm itself could be converted to FP16 as long as mean and variance stick to FP32, but I don't have this mechanism for now so this is a fair workaround.
   
   Meanwhile, do you think ToMixedPrecision should be applied later than the expression mutation passes (e.g., after SimplifyExpr, SimplifyInference, etc, and before FuseOps), so that we could make sure all ops (types) in the IR are basically fixed?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org