You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2022/02/24 18:40:51 UTC

[GitHub] [tvm] lhutton1 opened a new pull request #10375: [CMSIS-NN] Only run ScalarToTensorConstants pass on CMSISNN external functions

lhutton1 opened a new pull request #10375:
URL: https://github.com/apache/tvm/pull/10375


   Ensures the `ScalarToTensorConstants` pass only runs on functions with the "cmsis-nn" compiler attribute. Previously, it was possible for parts of the pass to run on all of the input. For example, `ReplaceScalarWithTensorVariable` was running on all OpNodes without regard to the containing function.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] ashutosh-arm commented on a change in pull request #10375: [CMSIS-NN] Only run ScalarToTensorConstants pass on CMSISNN external functions

Posted by GitBox <gi...@apache.org>.
ashutosh-arm commented on a change in pull request #10375:
URL: https://github.com/apache/tvm/pull/10375#discussion_r816109607



##########
File path: src/relay/backend/contrib/cmsisnn/scalar_to_tensor_constant.cc
##########
@@ -177,14 +173,22 @@ class ScalarToTensorConstantMutator : public MixedModeMutator {
 };
 
 IRModule ScalarToTensorConstant(const IRModule& mod) {
-  auto mutator = ScalarToTensorConstantMutator(mod);
-  Function main_func = Downcast<Function>(mod->Lookup("main"));
-  auto new_main_body = mutator.VisitExpr(main_func->body);
-  if (!new_main_body.same_as(main_func->body)) {
-    auto main_var = mod->GetGlobalVar("main");
-    auto new_main_func = Function(main_func->params, new_main_body, main_func->ret_type,
-                                  main_func->type_params, main_func->attrs);
-    mod->Update(main_var, new_main_func);
+  for (auto gv : mod->GetGlobalVars()) {
+    Function func = Downcast<Function>(mod->Lookup(gv));
+
+    // only mutate CMSIS-NN external functions
+    auto compiler_name = func->GetAttr<String>(attr::kCompiler);
+    if (!compiler_name.defined() || compiler_name != "cmsis-nn") {

Review comment:
       There is no problem with this sort of check from top level visit. But maybe it will have the same meaning if we could  move the checks inside function visits? Something similar to what is being done here: https://github.com/apache/tvm/blob/7127296c2b603e00344800f2d95aff5dc257f194/src/relay/backend/contrib/cmsisnn/extract_constants.cc#L60




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] lhutton1 commented on pull request #10375: [CMSIS-NN] Only run ScalarToTensorConstants pass on CMSISNN external functions

Posted by GitBox <gi...@apache.org>.
lhutton1 commented on pull request #10375:
URL: https://github.com/apache/tvm/pull/10375#issuecomment-1050150616


   After further investigation from #10348 
   
   cc @manupa-arm @ashutosh-arm @Mousius 
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] lhutton1 commented on pull request #10375: [CMSIS-NN] Only run ScalarToTensorConstants pass on CMSISNN external functions

Posted by GitBox <gi...@apache.org>.
lhutton1 commented on pull request #10375:
URL: https://github.com/apache/tvm/pull/10375#issuecomment-1050875740


   Thanks @manupa-arm, I've updated the variable names and improved the commit message and PR description


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] ashutosh-arm commented on a change in pull request #10375: [CMSIS-NN] Only run ScalarToTensorConstants pass on CMSISNN external functions

Posted by GitBox <gi...@apache.org>.
ashutosh-arm commented on a change in pull request #10375:
URL: https://github.com/apache/tvm/pull/10375#discussion_r816587444



##########
File path: src/relay/backend/contrib/cmsisnn/scalar_to_tensor_constant.cc
##########
@@ -177,14 +173,22 @@ class ScalarToTensorConstantMutator : public MixedModeMutator {
 };
 
 IRModule ScalarToTensorConstant(const IRModule& mod) {
-  auto mutator = ScalarToTensorConstantMutator(mod);
-  Function main_func = Downcast<Function>(mod->Lookup("main"));
-  auto new_main_body = mutator.VisitExpr(main_func->body);
-  if (!new_main_body.same_as(main_func->body)) {
-    auto main_var = mod->GetGlobalVar("main");
-    auto new_main_func = Function(main_func->params, new_main_body, main_func->ret_type,
-                                  main_func->type_params, main_func->attrs);
-    mod->Update(main_var, new_main_func);
+  for (auto gv : mod->GetGlobalVars()) {
+    Function func = Downcast<Function>(mod->Lookup(gv));
+
+    // only mutate CMSIS-NN external functions
+    auto compiler_name = func->GetAttr<String>(attr::kCompiler);
+    if (!compiler_name.defined() || compiler_name != "cmsis-nn") {

Review comment:
       Although the pass won't change anything in the main() if we go the extract_constant() way, but your approach completely avoids changes to main(). Its a stricter check. So, let's go with that. Probably, something similar is needed in the extract_constant() then :smile: 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] ashutosh-arm commented on a change in pull request #10375: [CMSIS-NN] Only run ScalarToTensorConstants pass on CMSISNN external functions

Posted by GitBox <gi...@apache.org>.
ashutosh-arm commented on a change in pull request #10375:
URL: https://github.com/apache/tvm/pull/10375#discussion_r816109607



##########
File path: src/relay/backend/contrib/cmsisnn/scalar_to_tensor_constant.cc
##########
@@ -177,14 +173,22 @@ class ScalarToTensorConstantMutator : public MixedModeMutator {
 };
 
 IRModule ScalarToTensorConstant(const IRModule& mod) {
-  auto mutator = ScalarToTensorConstantMutator(mod);
-  Function main_func = Downcast<Function>(mod->Lookup("main"));
-  auto new_main_body = mutator.VisitExpr(main_func->body);
-  if (!new_main_body.same_as(main_func->body)) {
-    auto main_var = mod->GetGlobalVar("main");
-    auto new_main_func = Function(main_func->params, new_main_body, main_func->ret_type,
-                                  main_func->type_params, main_func->attrs);
-    mod->Update(main_var, new_main_func);
+  for (auto gv : mod->GetGlobalVars()) {
+    Function func = Downcast<Function>(mod->Lookup(gv));
+
+    // only mutate CMSIS-NN external functions
+    auto compiler_name = func->GetAttr<String>(attr::kCompiler);
+    if (!compiler_name.defined() || compiler_name != "cmsis-nn") {

Review comment:
       There is no problem with this sort of check from top level visit. But maybe if it will have the same meaning if we could  move the checks inside function visits? Something similar to what is being done here: https://github.com/apache/tvm/blob/7127296c2b603e00344800f2d95aff5dc257f194/src/relay/backend/contrib/cmsisnn/extract_constants.cc#L60




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] masahi merged pull request #10375: [CMSIS-NN] Only run ScalarToTensorConstants pass on CMSISNN external functions

Posted by GitBox <gi...@apache.org>.
masahi merged pull request #10375:
URL: https://github.com/apache/tvm/pull/10375


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] lhutton1 commented on a change in pull request #10375: [CMSIS-NN] Only run ScalarToTensorConstants pass on CMSISNN external functions

Posted by GitBox <gi...@apache.org>.
lhutton1 commented on a change in pull request #10375:
URL: https://github.com/apache/tvm/pull/10375#discussion_r816144228



##########
File path: src/relay/backend/contrib/cmsisnn/scalar_to_tensor_constant.cc
##########
@@ -177,14 +173,22 @@ class ScalarToTensorConstantMutator : public MixedModeMutator {
 };
 
 IRModule ScalarToTensorConstant(const IRModule& mod) {
-  auto mutator = ScalarToTensorConstantMutator(mod);
-  Function main_func = Downcast<Function>(mod->Lookup("main"));
-  auto new_main_body = mutator.VisitExpr(main_func->body);
-  if (!new_main_body.same_as(main_func->body)) {
-    auto main_var = mod->GetGlobalVar("main");
-    auto new_main_func = Function(main_func->params, new_main_body, main_func->ret_type,
-                                  main_func->type_params, main_func->attrs);
-    mod->Update(main_var, new_main_func);
+  for (auto gv : mod->GetGlobalVars()) {
+    Function func = Downcast<Function>(mod->Lookup(gv));
+
+    // only mutate CMSIS-NN external functions
+    auto compiler_name = func->GetAttr<String>(attr::kCompiler);
+    if (!compiler_name.defined() || compiler_name != "cmsis-nn") {

Review comment:
       I believe the two approaches here are slightly different. With the approach mentioned above it seems like we still visit the main function, including the call nodes within it. Its true that we won't visit non CMSIS-NN external functions, but it seems like its possible to still mutate some operators in main - is this okay for this pass? Meanwhile, with the approach in the PR it is guaranteed that only CMSIS-NN functions are mutated and that we won't accidentally mutate things in main. Please let me know if I missed something though as I'm unfamiliar with these passes :)




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] manupa-arm commented on a change in pull request #10375: [CMSIS-NN] Only run ScalarToTensorConstants pass on CMSISNN external functions

Posted by GitBox <gi...@apache.org>.
manupa-arm commented on a change in pull request #10375:
URL: https://github.com/apache/tvm/pull/10375#discussion_r814747174



##########
File path: src/relay/backend/contrib/cmsisnn/scalar_to_tensor_constant.cc
##########
@@ -177,14 +173,22 @@ class ScalarToTensorConstantMutator : public MixedModeMutator {
 };
 
 IRModule ScalarToTensorConstant(const IRModule& mod) {
-  auto mutator = ScalarToTensorConstantMutator(mod);
-  Function main_func = Downcast<Function>(mod->Lookup("main"));
-  auto new_main_body = mutator.VisitExpr(main_func->body);
-  if (!new_main_body.same_as(main_func->body)) {
-    auto main_var = mod->GetGlobalVar("main");
-    auto new_main_func = Function(main_func->params, new_main_body, main_func->ret_type,
-                                  main_func->type_params, main_func->attrs);
-    mod->Update(main_var, new_main_func);
+  for (auto gv : mod->GetGlobalVars()) {
+    Function main_func = Downcast<Function>(mod->Lookup(gv));

Review comment:
       nit : main function seems not right here.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] ashutosh-arm commented on a change in pull request #10375: [CMSIS-NN] Only run ScalarToTensorConstants pass on CMSISNN external functions

Posted by GitBox <gi...@apache.org>.
ashutosh-arm commented on a change in pull request #10375:
URL: https://github.com/apache/tvm/pull/10375#discussion_r816587444



##########
File path: src/relay/backend/contrib/cmsisnn/scalar_to_tensor_constant.cc
##########
@@ -177,14 +173,22 @@ class ScalarToTensorConstantMutator : public MixedModeMutator {
 };
 
 IRModule ScalarToTensorConstant(const IRModule& mod) {
-  auto mutator = ScalarToTensorConstantMutator(mod);
-  Function main_func = Downcast<Function>(mod->Lookup("main"));
-  auto new_main_body = mutator.VisitExpr(main_func->body);
-  if (!new_main_body.same_as(main_func->body)) {
-    auto main_var = mod->GetGlobalVar("main");
-    auto new_main_func = Function(main_func->params, new_main_body, main_func->ret_type,
-                                  main_func->type_params, main_func->attrs);
-    mod->Update(main_var, new_main_func);
+  for (auto gv : mod->GetGlobalVars()) {
+    Function func = Downcast<Function>(mod->Lookup(gv));
+
+    // only mutate CMSIS-NN external functions
+    auto compiler_name = func->GetAttr<String>(attr::kCompiler);
+    if (!compiler_name.defined() || compiler_name != "cmsis-nn") {

Review comment:
       Although the pass won't change if we go the extract_constant() way, but your approach completely avoids changes to main(). Its a stricter check. So, let's go with that. Probably, something similar is needed in the extract_constant() then :smile: 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org