You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2022/01/06 04:32:17 UTC
[GitHub] [tvm] AndrewZhaoLuo opened a new pull request #9849: [AMP] Register some new ops
AndrewZhaoLuo opened a new pull request #9849:
URL: https://github.com/apache/tvm/pull/9849
Add some unregistered ops to the default AMP lists.
By default the pass places unregistered ops to the follow list so this does not change behavior. It does suppress some errors though.
`nn.depth_to_space` -- this is just a fancy reshape op so should be in follow list
`argmax`, `argmin` -- this one is pretty much the same computation as in `min` and `max` which are in the follow lists
`fast_*` -- these fast operations use approximations involving polynomials in a clipped domain of the normal functions. They seem safe, though `fast_tanh` has coefficients outside of representable range of fp16 (which I think is fine)
`clip` -- usually used to implement relu6, might have problems if clip range are outside of fp16 range but this should be rare
`image.resize2d` -- The amount of accumulation is pretty minimal
Excepting the `fast_*` functions which have no analog I know of, this matches closely with TFs mixed precision list (github.com/tensorflow/tensorflow/blob/v2.5.0/tensorflow/core/grappler/optimizers/auto_mixed_precision_lists.h) so feel confident about this change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi merged pull request #9849: [AMP] Register some new ops
Posted by GitBox <gi...@apache.org>.
masahi merged pull request #9849:
URL: https://github.com/apache/tvm/pull/9849
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] AndrewZhaoLuo commented on pull request #9849: [AMP] Register some new ops
Posted by GitBox <gi...@apache.org>.
AndrewZhaoLuo commented on pull request #9849:
URL: https://github.com/apache/tvm/pull/9849#issuecomment-1006277124
cc @anwang2009
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi merged pull request #9849: [AMP] Register some new ops
Posted by GitBox <gi...@apache.org>.
masahi merged pull request #9849:
URL: https://github.com/apache/tvm/pull/9849
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] comaniac commented on a change in pull request #9849: [AMP] Register some new ops
Posted by GitBox <gi...@apache.org>.
comaniac commented on a change in pull request #9849:
URL: https://github.com/apache/tvm/pull/9849#discussion_r779362293
##########
File path: python/tvm/relay/transform/mixed_precision.py
##########
@@ -88,13 +89,19 @@
"min",
"maximum",
"minimum",
+ "argmax",
+ "argmin",
"nn.relu",
"nn.leaky_relu",
"nn.prelu",
"nn.dropout",
# Complicated activations which saturate in a narrow range
"sigmoid",
"tanh",
+ "fast_tanh", # Some coefficients outside of representable range, but probably ok
+ "fast_exp",
+ "fast_erf",
+ "clip", # Usually safe, my result in oddity if clip greater than fp16 range
Review comment:
```suggestion
"clip", # Usually safe, may result in oddity if clip greater than fp16 range
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org