You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by "NicolaLancellotti (via GitHub)" <gi...@apache.org> on 2023/04/04 13:28:42 UTC
[GitHub] [tvm] NicolaLancellotti opened a new pull request, #14484: [ACL] Prevent offloading of per-channel quantized operators.
NicolaLancellotti opened a new pull request, #14484:
URL: https://github.com/apache/tvm/pull/14484
Currently, only per-layer quantization is supported in the Arm Compute Library runtime. However, there is no check that prevents the offloading of per-channel quantized operators, as a consequence, tvm fails during inference if such operators are found.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] lhutton1 merged pull request #14484: [ACL] Prevent offloading of per-channel quantized operators
Posted by "lhutton1 (via GitHub)" <gi...@apache.org>.
lhutton1 merged PR #14484:
URL: https://github.com/apache/tvm/pull/14484
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tvm-bot commented on pull request #14484: [ACL] Prevent offloading of per-channel quantized operators.
Posted by "tvm-bot (via GitHub)" <gi...@apache.org>.
tvm-bot commented on PR #14484:
URL: https://github.com/apache/tvm/pull/14484#issuecomment-1495976357
<!---bot-comment-->
Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from [Reviewers](https://github.com/apache/incubator-tvm/blob/master/CONTRIBUTORS.md#reviewers) by @-ing them in a comment.
<!--bot-comment-ccs-start-->
* cc @ashutosh-arm, @leandron, @lhutton1 <sub>See [#10317](https://github.com/apache/tvm/issues/10317) for details</sub><!--bot-comment-ccs-end-->
<sub>Generated by [tvm-bot](https://github.com/apache/tvm/blob/main/ci/README.md#github-actions)</sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] lhutton1 commented on pull request #14484: [ACL] Prevent offloading of per-channel quantized operators
Posted by "lhutton1 (via GitHub)" <gi...@apache.org>.
lhutton1 commented on PR #14484:
URL: https://github.com/apache/tvm/pull/14484#issuecomment-1497594085
Thanks @NicolaLancellotti!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] lhutton1 commented on a diff in pull request #14484: [ACL] Prevent offloading of per-channel quantized operators
Posted by "lhutton1 (via GitHub)" <gi...@apache.org>.
lhutton1 commented on code in PR #14484:
URL: https://github.com/apache/tvm/pull/14484#discussion_r1158456353
##########
tests/python/contrib/test_arm_compute_lib/test_dense.py:
##########
@@ -380,8 +380,51 @@ def test_codegen_qnn_dense(dtype):
verify_codegen(func, exp_codegen)
+@pytest.mark.parametrize(
+ "param",
+ ["kernel_sc", "kernel_zp"],
+)
+def test_codegen_qnn_dense_per_channel_quantization(param):
+ if skip_codegen_test():
+ return
+
+ np.random.seed(0)
+ dtype = "int8"
+ shape = (1, 2)
+ weight_shape = (2, 2)
+ units = 2
+ composite = True
+ inputs = {"a"}
+ args = (shape, weight_shape, units, dtype)
+
+ qnn_params = {
+ "input_zp": 1,
+ "input_sc": 1,
+ "kernel_zp": 1,
+ "kernel_sc": 1,
+ "output_zp": 1,
+ "output_sc": 1,
+ }
+ qnn_params[param] = [1, 1]
+
+ func, _ = _get_qnn_model(
+ *args,
+ var_names=iter(inputs),
+ input_zp=qnn_params["input_zp"],
+ input_sc=qnn_params["input_sc"],
+ kernel_zp=qnn_params["kernel_zp"],
+ kernel_sc=qnn_params["kernel_sc"],
+ output_zp=qnn_params["output_zp"],
+ output_sc=qnn_params["output_sc"],
+ has_bias=composite,
+ )
+ exp_codegen = _get_expected_codegen(*args, has_bias=composite)
+ verify_codegen(func, exp_codegen, num_acl_modules=0, tvm_ops=3)
+
+
if __name__ == "__main__":
test_dense()
test_qnn_dense()
test_codegen_dense()
test_codegen_qnn_dense()
+ test_codegen_qnn_dense_per_channel_quantization()
Review Comment:
No need to add now, we should probably convert to the:
```
if __name__ == "__main__":
tvm.testing.main()
```
format when we get a another chance
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org