You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2022/11/09 18:34:05 UTC
[GitHub] [tvm] ibsidorenko opened a new pull request, #13332: [FQ2I] Add cast back to input data type after AvgPool2d
ibsidorenko opened a new pull request, #13332:
URL: https://github.com/apache/tvm/pull/13332
This commit fixes the following issue:
For the sequence of `qnn.dequantize -> avg_pool2d -> conv2d -> qnn.quantize` FQ2I pass inserts `qnn.requantize` (or cast) to int32 unconditionally before AvgPool2d. As a result fake quantized `qnn.conv2d` gets input as int32 dtype, but it is forbidden for `qnn.conv2d` (supports only uint8/int8/int16).
This commit adds the following:
1) Add cast back to input data type after AvgPool2d. This preserve output data type as input data type.
2) Support int8/uint8/int16/uint16 as input data type in AvgPool2d compute function (but still use int32 for intermediate computation).
Also this commit fixes issue[#12381](https://github.com/apache/tvm/issues/12381).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi merged pull request #13332: [FQ2I] Add cast back to input data type after AvgPool2d
Posted by GitBox <gi...@apache.org>.
masahi merged PR #13332:
URL: https://github.com/apache/tvm/pull/13332
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] apeskov commented on a diff in pull request #13332: [FQ2I] Add cast back to input data type after AvgPool2d
Posted by GitBox <gi...@apache.org>.
apeskov commented on code in PR #13332:
URL: https://github.com/apache/tvm/pull/13332#discussion_r1019315558
##########
python/tvm/relay/transform/fake_quantization_to_integer.py:
##########
@@ -158,7 +159,10 @@ def avgpool2d(expr, type_map):
else:
arg = relay.op.cast(arg, "int32")
out = relay.op.nn.avg_pool2d(arg, **expr.attrs)
- return [out, TensorAffineType(out_t.scale, out_t.zero_point, "int32", out_t.axis)]
+ # Cast back to input dtype to preserve input dtype == output dtype.
+ out = relay.op.clip(out, a_min=np.iinfo(t.dtype).min, a_max=np.iinfo(t.dtype).max)
+ out = relay.op.cast(out, t.dtype)
Review Comment:
Why do you need to preserve `input dtype == output dtype`? As I understood it should match `out_t.dtype`.
I guess it should be conditional injection of cast op to `out_t.dtype` if it is not a `int32`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tvm-bot commented on pull request #13332: [FQ2I] Add cast back to input data type after AvgPool2d
Posted by GitBox <gi...@apache.org>.
tvm-bot commented on PR #13332:
URL: https://github.com/apache/tvm/pull/13332#issuecomment-1309197366
<!---bot-comment-->
Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from [Reviewers](https://github.com/apache/incubator-tvm/blob/master/CONTRIBUTORS.md#reviewers) by @-ing them in a comment.
<!--bot-comment-ccs-start-->
* No users to tag found in teams: `fq2i` <sub>See [#10317](https://github.com/apache/tvm/issues/10317) for details</sub><!--bot-comment-ccs-end-->
<sub>Generated by [tvm-bot](https://github.com/apache/tvm/blob/main/ci/README.md#github-actions)</sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] ibsidorenko commented on a diff in pull request #13332: [FQ2I] Add cast back to input data type after AvgPool2d
Posted by GitBox <gi...@apache.org>.
ibsidorenko commented on code in PR #13332:
URL: https://github.com/apache/tvm/pull/13332#discussion_r1019895655
##########
python/tvm/relay/transform/fake_quantization_to_integer.py:
##########
@@ -158,7 +159,10 @@ def avgpool2d(expr, type_map):
else:
arg = relay.op.cast(arg, "int32")
out = relay.op.nn.avg_pool2d(arg, **expr.attrs)
- return [out, TensorAffineType(out_t.scale, out_t.zero_point, "int32", out_t.axis)]
+ # Cast back to input dtype to preserve input dtype == output dtype.
+ out = relay.op.clip(out, a_min=np.iinfo(t.dtype).min, a_max=np.iinfo(t.dtype).max)
+ out = relay.op.cast(out, t.dtype)
Review Comment:
Make sense. Changed on `out_t.dtype` + if condition.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org