You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2022/02/11 17:23:29 UTC
[GitHub] [tvm] manupa-arm opened a new pull request #10224: [CMSIS-NN] enable USMP with CMSIS-NN
manupa-arm opened a new pull request #10224:
URL: https://github.com/apache/tvm/pull/10224
This commit mainly enables the USMP with CMSIS-NN codegen.
In order to do that, CMSIS-NN functions needed to contain BufferMaps. This commit adds the necessary BufferMaps as well.
All the tests are modified to run with USMP while the networks tests run with and without USMP.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] manupa-arm edited a comment on pull request #10224: [CMSIS-NN] enable USMP with CMSIS-NN
Posted by GitBox <gi...@apache.org>.
manupa-arm edited a comment on pull request #10224:
URL: https://github.com/apache/tvm/pull/10224#issuecomment-1036440951
~~This is blocked on #10022 and changes belonging to this PR is only in the commit : https://github.com/apache/tvm/pull/10224/commits/57851dc95df37aa2529f32dba0e9470436125ada~~
cc : @ashutosh-arm @Mousius
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] manupa-arm commented on a change in pull request #10224: [CMSIS-NN] enable USMP with CMSIS-NN
Posted by GitBox <gi...@apache.org>.
manupa-arm commented on a change in pull request #10224:
URL: https://github.com/apache/tvm/pull/10224#discussion_r816985525
##########
File path: src/relay/backend/contrib/cmsisnn/relay_to_tir.cc
##########
@@ -267,10 +293,13 @@ class RelayToTIRVisitor : public MixedModeMutator {
// %3 = qnn.requantize(%2, %req_input_scale_scalar, %req_input_zero_point_scalar,
// %output_scale_scalar, %output_zero_point_scalar)
// clip(%3, a_min=%min_scalar, a_max=%max_scalar)
- tir::Var input("input", DataType::Handle(8));
- tir::Var filter("filter", DataType::Handle(8));
- tir::Var bias("bias", DataType::Handle(32));
- tir::Var output("output", DataType::Handle(8));
+ BufferCreator buffer_creator_;
Review comment:
You are right! ref : https://google.github.io/styleguide/cppguide.html#Local_Variables
##########
File path: src/relay/backend/contrib/cmsisnn/relay_to_tir.cc
##########
@@ -267,10 +293,13 @@ class RelayToTIRVisitor : public MixedModeMutator {
// %3 = qnn.requantize(%2, %req_input_scale_scalar, %req_input_zero_point_scalar,
// %output_scale_scalar, %output_zero_point_scalar)
// clip(%3, a_min=%min_scalar, a_max=%max_scalar)
- tir::Var input("input", DataType::Handle(8));
- tir::Var filter("filter", DataType::Handle(8));
- tir::Var bias("bias", DataType::Handle(32));
- tir::Var output("output", DataType::Handle(8));
+ BufferCreator buffer_creator_;
Review comment:
Changed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] ashutosh-arm commented on a change in pull request #10224: [CMSIS-NN] enable USMP with CMSIS-NN
Posted by GitBox <gi...@apache.org>.
ashutosh-arm commented on a change in pull request #10224:
URL: https://github.com/apache/tvm/pull/10224#discussion_r816102516
##########
File path: src/relay/backend/contrib/cmsisnn/relay_to_tir.cc
##########
@@ -267,10 +293,13 @@ class RelayToTIRVisitor : public MixedModeMutator {
// %3 = qnn.requantize(%2, %req_input_scale_scalar, %req_input_zero_point_scalar,
// %output_scale_scalar, %output_zero_point_scalar)
// clip(%3, a_min=%min_scalar, a_max=%max_scalar)
- tir::Var input("input", DataType::Handle(8));
- tir::Var filter("filter", DataType::Handle(8));
- tir::Var bias("bias", DataType::Handle(32));
- tir::Var output("output", DataType::Handle(8));
+ BufferCreator buffer_creator_;
Review comment:
I have seen trailing underscores for member variables. Not sure what the guideline is for local vars.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] Mousius merged pull request #10224: [CMSIS-NN] enable USMP with CMSIS-NN
Posted by GitBox <gi...@apache.org>.
Mousius merged pull request #10224:
URL: https://github.com/apache/tvm/pull/10224
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] manupa-arm commented on pull request #10224: [CMSIS-NN] enable USMP with CMSIS-NN
Posted by GitBox <gi...@apache.org>.
manupa-arm commented on pull request #10224:
URL: https://github.com/apache/tvm/pull/10224#issuecomment-1055677310
@ashutosh-arm, I've addressed the comments, PTAL when you have some time!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] Mousius commented on pull request #10224: [CMSIS-NN] enable USMP with CMSIS-NN
Posted by GitBox <gi...@apache.org>.
Mousius commented on pull request #10224:
URL: https://github.com/apache/tvm/pull/10224#issuecomment-1057196107
Thanks @manupa-arm!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] manupa-arm commented on a change in pull request #10224: [CMSIS-NN] enable USMP with CMSIS-NN
Posted by GitBox <gi...@apache.org>.
manupa-arm commented on a change in pull request #10224:
URL: https://github.com/apache/tvm/pull/10224#discussion_r816985292
##########
File path: src/relay/backend/contrib/cmsisnn/relay_to_tir.cc
##########
@@ -196,12 +227,13 @@ class RelayToTIRVisitor : public MixedModeMutator {
tvm::Array<PrimExpr> call_ext_args = {tir::StringImm(cmsisnn_api), input, filter, multiplier};
if (bias_add_call) {
+ tir::Var bias = buffer_creator_.GetBufferVar("bias");
call_ext_args.push_back(bias);
}
call_ext_args.push_back(shift);
call_ext_args.push_back(output);
- std::string context_buffer_name = "NULL";
+ PrimExpr buffer_var = tir::StringImm("NULL");
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] manupa-arm commented on pull request #10224: [CMSIS-NN] enable USMP with CMSIS-NN
Posted by GitBox <gi...@apache.org>.
manupa-arm commented on pull request #10224:
URL: https://github.com/apache/tvm/pull/10224#issuecomment-1048909535
This is ready for review. @ashutosh-arm @Mousius
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] ashutosh-arm commented on a change in pull request #10224: [CMSIS-NN] enable USMP with CMSIS-NN
Posted by GitBox <gi...@apache.org>.
ashutosh-arm commented on a change in pull request #10224:
URL: https://github.com/apache/tvm/pull/10224#discussion_r816103078
##########
File path: src/relay/backend/contrib/cmsisnn/relay_to_tir.cc
##########
@@ -316,14 +345,13 @@ class RelayToTIRVisitor : public MixedModeMutator {
tvm::Array<PrimExpr> call_ext_args = {tir::StringImm("arm_fully_connected_s8"), input, filter};
if (bias_add_call) {
- call_ext_args.push_back(bias);
+ call_ext_args.push_back(buffer_creator_.GetBufferVar("bias"));
}
call_ext_args.push_back(output);
int context_buffer_size = 0;
- std::string context_buffer_name = "NULL";
- tvm::Array<PrimExpr> context_buffer_args = {tir::StringImm(context_buffer_name),
- ToArg(context_buffer_size)};
+ PrimExpr buffer_var = tir::StringImm("NULL");
Review comment:
nit: context_buffer_var would be more meaningful.
##########
File path: src/relay/backend/contrib/cmsisnn/relay_to_tir.cc
##########
@@ -196,12 +227,13 @@ class RelayToTIRVisitor : public MixedModeMutator {
tvm::Array<PrimExpr> call_ext_args = {tir::StringImm(cmsisnn_api), input, filter, multiplier};
if (bias_add_call) {
+ tir::Var bias = buffer_creator_.GetBufferVar("bias");
call_ext_args.push_back(bias);
}
call_ext_args.push_back(shift);
call_ext_args.push_back(output);
- std::string context_buffer_name = "NULL";
+ PrimExpr buffer_var = tir::StringImm("NULL");
Review comment:
nit: Like other places, it can be renamed to context_buffer_var for better clarity.
##########
File path: src/relay/backend/contrib/cmsisnn/relay_to_tir.cc
##########
@@ -267,10 +293,13 @@ class RelayToTIRVisitor : public MixedModeMutator {
// %3 = qnn.requantize(%2, %req_input_scale_scalar, %req_input_zero_point_scalar,
// %output_scale_scalar, %output_zero_point_scalar)
// clip(%3, a_min=%min_scalar, a_max=%max_scalar)
- tir::Var input("input", DataType::Handle(8));
- tir::Var filter("filter", DataType::Handle(8));
- tir::Var bias("bias", DataType::Handle(32));
- tir::Var output("output", DataType::Handle(8));
+ BufferCreator buffer_creator_;
Review comment:
I have seen trailing underscores for member variables. Not sure what the guideline is.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] manupa-arm edited a comment on pull request #10224: [CMSIS-NN] enable USMP with CMSIS-NN
Posted by GitBox <gi...@apache.org>.
manupa-arm edited a comment on pull request #10224:
URL: https://github.com/apache/tvm/pull/10224#issuecomment-1036440951
This is blocked on #10022 and changes belonging to this PR is only in the commit : https://github.com/apache/tvm/pull/10224/commits/57851dc95df37aa2529f32dba0e9470436125ada
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] ashutosh-arm commented on pull request #10224: [CMSIS-NN] enable USMP with CMSIS-NN
Posted by GitBox <gi...@apache.org>.
ashutosh-arm commented on pull request #10224:
URL: https://github.com/apache/tvm/pull/10224#issuecomment-1056718448
Thanks @manupa-arm. I suppose it was already in a good state to be merged :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] lhutton1 commented on pull request #10224: [CMSIS-NN] enable USMP with CMSIS-NN
Posted by GitBox <gi...@apache.org>.
lhutton1 commented on pull request #10224:
URL: https://github.com/apache/tvm/pull/10224#issuecomment-1056854088
No worries, nothing major :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] manupa-arm commented on pull request #10224: [CMSIS-NN] enable USMP with CMSIS-NN
Posted by GitBox <gi...@apache.org>.
manupa-arm commented on pull request #10224:
URL: https://github.com/apache/tvm/pull/10224#issuecomment-1056847408
cc : @Mousius
I think I have missed @lhutton1's comment (Sorry!) which I will address in a follow up.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] lhutton1 commented on a change in pull request #10224: [CMSIS-NN] enable USMP with CMSIS-NN
Posted by GitBox <gi...@apache.org>.
lhutton1 commented on a change in pull request #10224:
URL: https://github.com/apache/tvm/pull/10224#discussion_r815456641
##########
File path: tests/python/contrib/test_cmsisnn/test_fully_connected.py
##########
@@ -27,6 +27,7 @@
from tests.python.relay.aot.aot_test_utils import (
AOTTestModel,
AOT_CORSTONE300_RUNNER,
Review comment:
Nit: unused import? (elsewhere as well)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] manupa-arm commented on pull request #10224: [CMSIS-NN] enable USMP with CMSIS-NN
Posted by GitBox <gi...@apache.org>.
manupa-arm commented on pull request #10224:
URL: https://github.com/apache/tvm/pull/10224#issuecomment-1050777360
Friendly ping for reviews! @ekalda @lhutton1 @ashutosh-arm @leandron
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] manupa-arm edited a comment on pull request #10224: [CMSIS-NN] enable USMP with CMSIS-NN
Posted by GitBox <gi...@apache.org>.
manupa-arm edited a comment on pull request #10224:
URL: https://github.com/apache/tvm/pull/10224#issuecomment-1036440951
This is blocked on #10022 and changes belonging to this PR is only in the commit : https://github.com/apache/tvm/pull/10224/commits/57851dc95df37aa2529f32dba0e9470436125ada
cc : @ashutosh-arm @Mousius
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] manupa-arm commented on pull request #10224: [CMSIS-NN] enable USMP with CMSIS-NN
Posted by GitBox <gi...@apache.org>.
manupa-arm commented on pull request #10224:
URL: https://github.com/apache/tvm/pull/10224#issuecomment-1036440951
This is blocked on #10022
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] manupa-arm commented on a change in pull request #10224: [CMSIS-NN] enable USMP with CMSIS-NN
Posted by GitBox <gi...@apache.org>.
manupa-arm commented on a change in pull request #10224:
URL: https://github.com/apache/tvm/pull/10224#discussion_r816985711
##########
File path: src/relay/backend/contrib/cmsisnn/relay_to_tir.cc
##########
@@ -316,14 +345,13 @@ class RelayToTIRVisitor : public MixedModeMutator {
tvm::Array<PrimExpr> call_ext_args = {tir::StringImm("arm_fully_connected_s8"), input, filter};
if (bias_add_call) {
- call_ext_args.push_back(bias);
+ call_ext_args.push_back(buffer_creator_.GetBufferVar("bias"));
}
call_ext_args.push_back(output);
int context_buffer_size = 0;
- std::string context_buffer_name = "NULL";
- tvm::Array<PrimExpr> context_buffer_args = {tir::StringImm(context_buffer_name),
- ToArg(context_buffer_size)};
+ PrimExpr buffer_var = tir::StringImm("NULL");
Review comment:
Done!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org