You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2022/08/11 14:57:09 UTC
[GitHub] [tvm] NicolaLancellotti opened a new pull request, #12384: [ETHOSN] Add support for Requantize
NicolaLancellotti opened a new pull request, #12384:
URL: https://github.com/apache/tvm/pull/12384
This commit adds support for the requantize operator for the Arm(R) Ethos(TM)-N NPU.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] lhutton1 merged pull request #12384: [ETHOSN] Add support for Requantize
Posted by GitBox <gi...@apache.org>.
lhutton1 merged PR #12384:
URL: https://github.com/apache/tvm/pull/12384
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] NicolaLancellotti commented on a diff in pull request #12384: [ETHOSN] Add support for Requantize
Posted by GitBox <gi...@apache.org>.
NicolaLancellotti commented on code in PR #12384:
URL: https://github.com/apache/tvm/pull/12384#discussion_r945836877
##########
src/relay/backend/contrib/ethosn/codegen.cc:
##########
@@ -918,6 +943,19 @@ TVM_REGISTER_GLOBAL("relay.ethos-n.support.relu")
err += EthosnError(reason);
});
+TVM_REGISTER_GLOBAL("relay.ethos-n.support.requantize")
+ .set_body([](tvm::TVMArgs args, tvm::TVMRetValue* rv) {
+ Call call = args[0];
+ RequantizeParams params;
+ auto err = EthosnAPI::Requantize(call, ¶ms);
+ err += EthosnCompiler::SupportedSetup();
+ char reason[kReasonMaxLength];
+ reason[0] = '\0';
+ *rv = !err && EthosnCompiler::GetSupported()->IsRequantizeSupported(
+ params.requantize_info, params.input_info, nullptr, reason, sizeof(reason));
Review Comment:
Done.
##########
src/relay/backend/contrib/ethosn/ethosn_api.cc:
##########
@@ -676,6 +677,35 @@ EthosnError EthosnAPI::Relu(const Expr& expr, ReluParams* params) {
return err;
}
+EthosnError EthosnAPI::Requantize(const Expr& expr, RequantizeParams* params) {
+ Call call = Downcast<Call>(expr);
+ const auto* input_dtype = call->args[0]->checked_type().as<TensorTypeNode>();
+ sl::TensorShape input_tensor_shape = {1, 1, 1, 1};
+ sl::DataType input_data_type;
+ EthosnError err = Tvm2Npu(input_dtype->shape, &input_tensor_shape);
+ err += Tvm2Npu(input_dtype->dtype, &input_data_type);
+
+ float input_sc, output_sc;
+ int input_zp, output_zp;
+ err += AsConstant(call->args[1], &input_sc);
+ err += AsConstant(call->args[2], &input_zp);
+ err += AsConstant(call->args[3], &output_sc);
+ err += AsConstant(call->args[4], &output_zp);
+
+ params->requantize_info = sl::RequantizeInfo(sl::QuantizationInfo(output_zp, output_sc));
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] lhutton1 commented on pull request #12384: [ETHOSN] Add support for Requantize
Posted by GitBox <gi...@apache.org>.
lhutton1 commented on PR #12384:
URL: https://github.com/apache/tvm/pull/12384#issuecomment-1216369927
Thanks @NicolaLancellotti!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] lhutton1 commented on a diff in pull request #12384: [ETHOSN] Add support for Requantize
Posted by GitBox <gi...@apache.org>.
lhutton1 commented on code in PR #12384:
URL: https://github.com/apache/tvm/pull/12384#discussion_r944311020
##########
src/relay/backend/contrib/ethosn/ethosn_api.cc:
##########
@@ -676,6 +677,35 @@ EthosnError EthosnAPI::Relu(const Expr& expr, ReluParams* params) {
return err;
}
+EthosnError EthosnAPI::Requantize(const Expr& expr, RequantizeParams* params) {
+ Call call = Downcast<Call>(expr);
+ const auto* input_dtype = call->args[0]->checked_type().as<TensorTypeNode>();
+ sl::TensorShape input_tensor_shape = {1, 1, 1, 1};
+ sl::DataType input_data_type;
+ EthosnError err = Tvm2Npu(input_dtype->shape, &input_tensor_shape);
+ err += Tvm2Npu(input_dtype->dtype, &input_data_type);
+
+ float input_sc, output_sc;
+ int input_zp, output_zp;
+ err += AsConstant(call->args[1], &input_sc);
+ err += AsConstant(call->args[2], &input_zp);
+ err += AsConstant(call->args[3], &output_sc);
+ err += AsConstant(call->args[4], &output_zp);
+
+ params->requantize_info = sl::RequantizeInfo(sl::QuantizationInfo(output_zp, output_sc));
Review Comment:
Nit: Looks like this has been missed in other operators as well, but the same quantization info object is created below as `output_q_info`. Could we just reuse it instead of creating two?
##########
src/relay/backend/contrib/ethosn/codegen.cc:
##########
@@ -918,6 +943,19 @@ TVM_REGISTER_GLOBAL("relay.ethos-n.support.relu")
err += EthosnError(reason);
});
+TVM_REGISTER_GLOBAL("relay.ethos-n.support.requantize")
+ .set_body([](tvm::TVMArgs args, tvm::TVMRetValue* rv) {
+ Call call = args[0];
+ RequantizeParams params;
+ auto err = EthosnAPI::Requantize(call, ¶ms);
+ err += EthosnCompiler::SupportedSetup();
+ char reason[kReasonMaxLength];
+ reason[0] = '\0';
+ *rv = !err && EthosnCompiler::GetSupported()->IsRequantizeSupported(
+ params.requantize_info, params.input_info, nullptr, reason, sizeof(reason));
Review Comment:
Can we also supply the output info to the issupported check?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org