You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2022/08/03 10:11:03 UTC
[GitHub] [tvm] mikepapadim opened a new pull request, #12287: [QNN] Add qnn op for abs to solve wrong scale on quantize
mikepapadim opened a new pull request, #12287:
URL: https://github.com/apache/tvm/pull/12287
Prior implementation caused the FQI pass to produce the following graph when `abs` was used:
```
%14 = qnn.dequantize(%13, 6.4712f /* ty=float32 */, 119 /* ty=int32 */, axis=1) /* ty=Tensor[(1, 960, 512, 64), float32] */;
%15 = abs(%14) /* ty=Tensor[(1, 960, 512, 64), float32] */;
%16 = qnn.quantize(%15, 3.46196f /* ty=float32 */, 0 /* ty=int32 */, out_dtype="uint8", axis=1) /* ty=Tensor[(1, 960, 512, 64), uint8] */;
```
It seems that the output scale and zero point are different from the input scale and zero point. So, we doubled the precision.
By adding a qnn op for `abs` we manage requantize on the right values.
```
def @main(%x: Tensor[(1, 960, 512, 64), int8] /* ty=Tensor[(1, 960, 512, 64), int8] */) -> Tensor[(1, 960, 512, 64), int8] {
%0 = qnn.abs(%x, 6.4712f /* ty=float32 */, 119 /* ty=int32 */, 6.4712f /* ty=float32 */, 0 /* ty=int32 */) /* ty=Tensor[(1, 960, 512, 64), int8] */;
qnn.requantize(%0, 6.4712f /* ty=float32 */, 119 /* ty=int32 */, 3.46196f /* ty=float32 */, 0 /* ty=int32 */, out_dtype="int8") /* ty=Tensor[(1, 960, 512, 64), int8] */
}
```
@sfvaroglu @mbrookhart @AndrewZhaoLuo @anwang2009
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] mbrookhart merged pull request #12287: [QNN] Add qnn op for abs to fix wrong scale on quantize
Posted by GitBox <gi...@apache.org>.
mbrookhart merged PR #12287:
URL: https://github.com/apache/tvm/pull/12287
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] mbrookhart commented on a diff in pull request #12287: [QNN] Add qnn op for abs to fix wrong scale on quantize
Posted by GitBox <gi...@apache.org>.
mbrookhart commented on code in PR #12287:
URL: https://github.com/apache/tvm/pull/12287#discussion_r937315065
##########
tests/python/relay/test_pass_fake_quantization_to_integer.py:
##########
@@ -27,6 +27,9 @@ def compare_fq_to_int(expr, args, allow_rounding_error=False):
mod = tvm.IRModule.from_expr(expr)
mod = tvm.relay.transform.InferType()(mod)
mod_int = tvm.relay.transform.FakeQuantizationToInteger()(mod)
+ print(mod)
+ print("-----")
+ print(mod_int)
Review Comment:
Remove extra prints
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] mikepapadim commented on pull request #12287: [QNN] Add qnn op for abs to fix wrong scale on quantize
Posted by GitBox <gi...@apache.org>.
mikepapadim commented on PR #12287:
URL: https://github.com/apache/tvm/pull/12287#issuecomment-1205453719
@tvm-bot rerun
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] mikepapadim commented on a diff in pull request #12287: [QNN] Add qnn op for abs to fix wrong scale on quantize
Posted by GitBox <gi...@apache.org>.
mikepapadim commented on code in PR #12287:
URL: https://github.com/apache/tvm/pull/12287#discussion_r937510461
##########
tests/python/relay/test_pass_fake_quantization_to_integer.py:
##########
@@ -27,6 +27,9 @@ def compare_fq_to_int(expr, args, allow_rounding_error=False):
mod = tvm.IRModule.from_expr(expr)
mod = tvm.relay.transform.InferType()(mod)
mod_int = tvm.relay.transform.FakeQuantizationToInteger()(mod)
+ print(mod)
+ print("-----")
+ print(mod_int)
Review Comment:
oups. I am cleaning this now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] mbrookhart commented on pull request #12287: [QNN] Add qnn op for abs to fix wrong scale on quantize
Posted by GitBox <gi...@apache.org>.
mbrookhart commented on PR #12287:
URL: https://github.com/apache/tvm/pull/12287#issuecomment-1205716763
Thanks @mikepapadim
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org