You are viewing a plain text version of this content. The canonical link for it is here.
Posted to discuss-archive@tvm.apache.org by JoeyChou via Apache TVM Discuss <no...@discuss.tvm.ai> on 2020/10/01 21:48:01 UTC
[Apache TVM Discuss] [Questions] Support for pre-quantized model
int8/uint8 conversion
Hi,
Does QNN support int8 --> uint8 or uint8 --> int8 pre-quantized model conversion? If no, is there a plan to support it?
Tag @anijain2305 cause you are fantastic! Thank you!
---
[Visit Topic](https://discuss.tvm.apache.org/t/support-for-pre-quantized-model-int8-uint8-conversion/8064/1) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/7e178b8f74528898dc02c766cd2eaaa68e1af576f487eafd69afbec09a5f9ec9).
[Apache TVM Discuss] [Questions] Support for pre-quantized model
int8/uint8 conversion
Posted by JoeyChou via Apache TVM Discuss <no...@discuss.tvm.ai>.
Hi @anijain2305 thanks for the reply. I should've made myself clear. What I meant was if the model(weight and bias) was quantized to uint8, does TVM has a way to convert the uint8 weight and bias to int8 weight and bias?
I will certainly try what you suggested, thank you.
---
[Visit Topic](https://discuss.tvm.apache.org/t/support-for-pre-quantized-model-int8-uint8-conversion/8064/3) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/d256b7082786cfcd651c6b48be03d8c8b16df5b90f570d3358fdcb36144b12de).
[Apache TVM Discuss] [Questions] Support for pre-quantized model
int8/uint8 conversion
Posted by Animesh Jain via Apache TVM Discuss <no...@discuss.tvm.ai>.
Yes, it does. The legalize pass can do this.
---
[Visit Topic](https://discuss.tvm.apache.org/t/support-for-pre-quantized-model-int8-uint8-conversion/8064/4) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/f10c899029f5b17a08f7d092cd6708d2159244f5333bd8fa29171487e9830a3e).
[Apache TVM Discuss] [Questions] Support for pre-quantized model
int8/uint8 conversion
Posted by Animesh Jain via Apache TVM Discuss <no...@discuss.tvm.ai>.
Hi @JoeyChou I am not sure what you mean by int8 -> uint8 conversion.
If you want your conv2d and dense inputs and weights to be of specific data type, yes that is certainly possible with QNN Legalize pass. An example of this is for Intel VNNI instructions which prefer `uint8` datatypes for feature maps and `int8` for the weights. Naturally, the pre-quantized models might not follow this rule. So, QNNLegalize inserts `requantize` node before the conv2d and dense to satisfy the datatype restrictions.
Please look at an example here
https://github.com/apache/incubator-tvm/blob/master/python/tvm/relay/qnn/op/legalizations.py#L296-L300
https://github.com/apache/incubator-tvm/blob/master/python/tvm/relay/qnn/transform.py#L71-L116
https://github.com/apache/incubator-tvm/blob/master/tests/python/relay/test_pass_qnn_legalize.py#L295
---
[Visit Topic](https://discuss.tvm.apache.org/t/support-for-pre-quantized-model-int8-uint8-conversion/8064/2) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/9a23daaf447ef77a359bb5d0cb9db6b330ce1dedfeff86ea901581861068561b).
[Apache TVM Discuss] [Questions] Support for pre-quantized model
int8/uint8 conversion
Posted by JoeyChou via Apache TVM Discuss <no...@discuss.tvm.ai>.
Yes, really appreciate your help!
---
[Visit Topic](https://discuss.tvm.apache.org/t/support-for-pre-quantized-model-int8-uint8-conversion/8064/5) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/d1873ce5010700cb3deb76ab16d5dd7c50fa949030fc619deb4737adee791aaa).