You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by "MingkangW (via GitHub)" <gi...@apache.org> on 2023/08/11 08:07:55 UTC
[GitHub] [tvm] MingkangW opened a new pull request, #15529: [quantize] fix bug of annotate for output of add op
MingkangW opened a new pull request, #15529:
URL: https://github.com/apache/tvm/pull/15529
The type of add op`s output is activation, it should annotate by QAnnotateKind.ACTIVATION. If not, the graph will cast int32 into int8 directly without quantized, when quantize resnet. It will product overflow error, and it won`t prompt when infer the model. The resnet18_v1 test case is below.
before fixed
%703 = add(%701, %702) /* ty=Tensor[(1, 512, 7, 7), int32] */;
%704 = nn.relu(%703) /* ty=Tensor[(1, 512, 7, 7), int32] */;
%705 = cast(%704, dtype="int8") /* ty=Tensor[(1, 512, 7, 7), int8] */;
%706 = annotation.stop_fusion(%705) /* ty=Tensor[(1, 512, 7, 7), int8] */;
after fixed
%443 = add(%441, %442) /* ty=Tensor[(1, 512, 7, 7), int32] */;
%444 = nn.relu(%443) /* ty=Tensor[(1, 512, 7, 7), int32] */;
%445 = cast(%444, dtype="int64") /* ty=Tensor[(1, 512, 7, 7), int64] */;
%446 = fixed_point_multiply(%445, multiplier=1439683968, shift=-2) /* ty=Tensor[(1, 512, 7, 7), int64] */;
%447 = clip(%446, a_min=-127f, a_max=127f) /* ty=Tensor[(1, 512, 7, 7), int64] */;
%448 = cast(%447, dtype="int32") /* ty=Tensor[(1, 512, 7, 7), int32] */;
%449 = cast(%448, dtype="int8") /* ty=Tensor[(1, 512, 7, 7), int8] */;
%450 = annotation.stop_fusion(%449) /* ty=Tensor[(1, 512, 7, 7), int8] */;
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MingkangW commented on pull request #15529: [quantize] fix bug of annotate for output of add op
Posted by "MingkangW (via GitHub)" <gi...@apache.org>.
MingkangW commented on PR #15529:
URL: https://github.com/apache/tvm/pull/15529#issuecomment-1675602767
@cbalint13
I have added test cases. If the annotation for add is not modified to ACTIVATION, the test case will not pass.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MingkangW commented on pull request #15529: [quantize] fix bug of annotate for output of add op
Posted by "MingkangW (via GitHub)" <gi...@apache.org>.
MingkangW commented on PR #15529:
URL: https://github.com/apache/tvm/pull/15529#issuecomment-1677092750
cc: @masahi
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MingkangW commented on pull request #15529: [quantize] fix bug of annotate for output of add op
Posted by "MingkangW (via GitHub)" <gi...@apache.org>.
MingkangW commented on PR #15529:
URL: https://github.com/apache/tvm/pull/15529#issuecomment-1676511537
PTAL @cbalint13
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MingkangW commented on pull request #15529: [quantize] fix bug of annotate for output of add op
Posted by "MingkangW (via GitHub)" <gi...@apache.org>.
MingkangW commented on PR #15529:
URL: https://github.com/apache/tvm/pull/15529#issuecomment-1675572141
> @MingkangW ,
> Can also add a small test case ?
Sure, I will do it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi merged pull request #15529: [quantize] fix bug of annotate for output of add op
Posted by "masahi (via GitHub)" <gi...@apache.org>.
masahi merged PR #15529:
URL: https://github.com/apache/tvm/pull/15529
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tvm-bot commented on pull request #15529: [quantize] fix bug of annotate for output of add op
Posted by "tvm-bot (via GitHub)" <gi...@apache.org>.
tvm-bot commented on PR #15529:
URL: https://github.com/apache/tvm/pull/15529#issuecomment-1674360904
<!---bot-comment-->
Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from [Reviewers](https://github.com/apache/incubator-tvm/blob/master/CONTRIBUTORS.md#reviewers) by @-ing them in a comment.
<!--bot-comment-ccs-start-->
* No users to tag found in teams: `quantize` <sub>See [#10317](https://github.com/apache/tvm/issues/10317) for details</sub><!--bot-comment-ccs-end-->
<sub>Generated by [tvm-bot](https://github.com/apache/tvm/blob/main/ci/README.md#github-actions)</sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org