You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2022/02/17 18:55:30 UTC
[GitHub] [tvm] vvchernov opened a new pull request #10293: WIP: [VirtualMachine] new method allowing to set one input tensor by its index
vvchernov opened a new pull request #10293:
URL: https://github.com/apache/tvm/pull/10293
Implementation of `set_input_with_index` method in VirtualMachine class. It provides more convinient way to work with multiple input models on native side.
I can insert functionality of `set_input_with_index` method to base method `set_input`. But I do not think that it is good idea to mix them. Other possible extension is to additionally reload `set_input` with args: func_name, index1, tensor1, index2, tensor2, …, indexN, tensorN, where N is number of all input tensors. But it is still not perfect solution for me.
I think it will be good if we have the same API for both GraphExecutor and VirtualMachine. But I do not see simple way for it. I even do not know how to start motion to this idea. Current solution seems half-hearted for me.
@jwfromm @tmoreau89 please see and discuss
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] vvchernov commented on a change in pull request #10293: [VirtualMachine] new method allowing to set one input tensor by its index or name
Posted by GitBox <gi...@apache.org>.
vvchernov commented on a change in pull request #10293:
URL: https://github.com/apache/tvm/pull/10293#discussion_r813942395
##########
File path: src/runtime/vm/vm.cc
##########
@@ -221,6 +211,9 @@ PackedFunc VirtualMachine::GetFunction(const std::string& name,
} else if (name == "set_input") {
return PackedFunc(
[sptr_to_self, this](TVMArgs args, TVMRetValue* rv) { SetInput(args[0], args, 1); });
+ } else if (name == "set_one_input") {
Review comment:
I've added pytests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] vvchernov commented on a change in pull request #10293: [VirtualMachine] new method allowing to set one input tensor by its index or name
Posted by GitBox <gi...@apache.org>.
vvchernov commented on a change in pull request #10293:
URL: https://github.com/apache/tvm/pull/10293#discussion_r813647053
##########
File path: include/tvm/runtime/vm/vm.h
##########
@@ -93,7 +93,10 @@ struct VMFunction {
params(std::move(params)),
instructions(std::move(instructions)),
register_file_size(register_file_size),
- param_device_indexes(std::move(param_device_indexes)) {}
+ param_device_indexes(std::move(param_device_indexes)) {
+ ICHECK_EQ(params.size(), param_device_indexes.size())
+ << "The number of provided parameters doesn't match the number of assigned devices";
Review comment:
Ok, I've removed it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] vvchernov commented on a change in pull request #10293: [VirtualMachine] new method allowing to set one input tensor by its index or name
Posted by GitBox <gi...@apache.org>.
vvchernov commented on a change in pull request #10293:
URL: https://github.com/apache/tvm/pull/10293#discussion_r813648349
##########
File path: include/tvm/runtime/vm/vm.h
##########
@@ -286,6 +298,48 @@ class VirtualMachine : public runtime::ModuleNode {
*/
virtual void OpStopHook();
+ private:
+ /*!
+ * \brief Get index of input tensor from its name.
+ * \param func_name The function's name.
+ * \param input_name The input tensor name.
+ * \return The input tensor index.
+ */
+ int64_t getInputIndexFromVMFunction(const std::string& func_name,
Review comment:
Thanks! done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] vvchernov commented on a change in pull request #10293: [VirtualMachine] new method allowing to set one input tensor by its index or name
Posted by GitBox <gi...@apache.org>.
vvchernov commented on a change in pull request #10293:
URL: https://github.com/apache/tvm/pull/10293#discussion_r813703708
##########
File path: src/runtime/vm/vm.cc
##########
@@ -221,6 +211,9 @@ PackedFunc VirtualMachine::GetFunction(const std::string& name,
} else if (name == "set_input") {
return PackedFunc(
[sptr_to_self, this](TVMArgs args, TVMRetValue* rv) { SetInput(args[0], args, 1); });
+ } else if (name == "set_one_input") {
Review comment:
Yes, I use runtime module directly on native side. It is a reason why I try to use `set_input` methods instead of direct using of `invoke`. The latter requires to pack all input tensors to TvmArgs in certain sequence on native side. For me it is more controlable and clear to input one by one tensor with its tag (index or name). It is simultaneously the answer to your below question. Nevertheless I agree that my patch was not full and I've extended python API of VirtualMachine by `set_one_input` method. I observed that there is no even `set_input` unit test. Unit tests are in progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] mbs-octoml edited a comment on pull request #10293: [VirtualMachine] new method allowing to set one input tensor by its index or name
Posted by GitBox <gi...@apache.org>.
mbs-octoml edited a comment on pull request #10293:
URL: https://github.com/apache/tvm/pull/10293#issuecomment-1049090663
One more question (sorry to double dip!): I agree a uniform parameter passing convention for all executors would be a Fine Thing. What I don't understand is why we separate binding and invocation in the first place? Surely the uniform API would be invoke(func, *args)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] mbs-octoml commented on a change in pull request #10293: [VirtualMachine] new method allowing to set one input tensor by its index or name
Posted by GitBox <gi...@apache.org>.
mbs-octoml commented on a change in pull request #10293:
URL: https://github.com/apache/tvm/pull/10293#discussion_r812314673
##########
File path: include/tvm/runtime/vm/vm.h
##########
@@ -93,7 +93,10 @@ struct VMFunction {
params(std::move(params)),
instructions(std::move(instructions)),
register_file_size(register_file_size),
- param_device_indexes(std::move(param_device_indexes)) {}
+ param_device_indexes(std::move(param_device_indexes)) {
+ ICHECK_EQ(params.size(), param_device_indexes.size())
+ << "The number of provided parameters doesn't match the number of assigned devices";
Review comment:
nit: message does not make sense
##########
File path: src/runtime/vm/vm.cc
##########
@@ -221,6 +211,9 @@ PackedFunc VirtualMachine::GetFunction(const std::string& name,
} else if (name == "set_input") {
return PackedFunc(
[sptr_to_self, this](TVMArgs args, TVMRetValue* rv) { SetInput(args[0], args, 1); });
+ } else if (name == "set_one_input") {
Review comment:
nit: suggest a unit test exercising this new method from vm.py (though I'm assuming you need to change that anyway, perhaps in a follow up PR?) Maybe you are just using the runtime module directly?
##########
File path: include/tvm/runtime/vm/vm.h
##########
@@ -270,6 +273,15 @@ class VirtualMachine : public runtime::ModuleNode {
*/
void SetInput(std::string name, TVMArgs args, int offset);
+ /*!
+ * \brief Set one input tensor with index or name to a function.
+ * \param name The function name
+ * \param args args[1:] are two arguments (index or name, tensor) to the
Review comment:
nit: might as well just pass two args since fixed size
nit: Would call it just SetOneInput to match above.
##########
File path: include/tvm/runtime/vm/vm.h
##########
@@ -286,6 +298,48 @@ class VirtualMachine : public runtime::ModuleNode {
*/
virtual void OpStopHook();
+ private:
+ /*!
+ * \brief Get index of input tensor from its name.
+ * \param func_name The function's name.
+ * \param input_name The input tensor name.
+ * \return The input tensor index.
+ */
+ int64_t getInputIndexFromVMFunction(const std::string& func_name,
Review comment:
nit: capitalize (and below)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] mbs-octoml commented on pull request #10293: [VirtualMachine] new method allowing to set one input tensor by its index or name
Posted by GitBox <gi...@apache.org>.
mbs-octoml commented on pull request #10293:
URL: https://github.com/apache/tvm/pull/10293#issuecomment-1049090663
One more question (sorry to double dip!): I agree a uniform parameter passing convention for all executions would be a Fine Thing. What I don't understand is why we separate binding and invocation in the first place? Surely the uniform API would be invoke(func, *args)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi merged pull request #10293: [VirtualMachine] new method allowing to set one input tensor by its index or name
Posted by GitBox <gi...@apache.org>.
masahi merged pull request #10293:
URL: https://github.com/apache/tvm/pull/10293
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tmoreau89 commented on pull request #10293: [VirtualMachine] new method allowing to set one input tensor by its index or name
Posted by GitBox <gi...@apache.org>.
tmoreau89 commented on pull request #10293:
URL: https://github.com/apache/tvm/pull/10293#issuecomment-1050219007
It sounds like there are more deeply rooted issues with the VM design choice to do explicit parameter binding etc. This seems indeed like a bigger discussion that warrants its own RFC, tracking issue etc; not sure a TODO here would be the best way to track the need to redesign. Due to time difference and latency in getting the PR changed, then approved, we could merge this without the TODO change. WDYT @mbs-octoml ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] mbs-octoml commented on pull request #10293: [VirtualMachine] new method allowing to set one input tensor by its index or name
Posted by GitBox <gi...@apache.org>.
mbs-octoml commented on pull request #10293:
URL: https://github.com/apache/tvm/pull/10293#issuecomment-1050240423
I filed CORE-142, though obviously that's not visible to the wider community. @jroesch expressed concern with bringing GE APIs over to the VM since the one-param-at-a-time API is fragile.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] vvchernov commented on a change in pull request #10293: [VirtualMachine] new method allowing to set one input tensor by its index or name
Posted by GitBox <gi...@apache.org>.
vvchernov commented on a change in pull request #10293:
URL: https://github.com/apache/tvm/pull/10293#discussion_r813648151
##########
File path: include/tvm/runtime/vm/vm.h
##########
@@ -270,6 +273,15 @@ class VirtualMachine : public runtime::ModuleNode {
*/
void SetInput(std::string name, TVMArgs args, int offset);
+ /*!
+ * \brief Set one input tensor with index or name to a function.
+ * \param name The function name
+ * \param args args[1:] are two arguments (index or name, tensor) to the
Review comment:
if I correctly understood your first tip it was done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] vvchernov edited a comment on pull request #10293: [VirtualMachine] new method allowing to set one input tensor by its index or name
Posted by GitBox <gi...@apache.org>.
vvchernov edited a comment on pull request #10293:
URL: https://github.com/apache/tvm/pull/10293#issuecomment-1050159908
Hello @mbs-octoml, thanks for your comment. I'm not sure that I fully caught your idea, I think it is related to design improvement of an executor, but for this the more qualified ingeneer is needed who knows functionality and design of GE, VM and possible future executor to evaluate profits and risk related to back compatability of the code. My aim was to solve my local task only and try to do it so that somebody also can use my tool. @tmoreau89 what do you think about it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] vvchernov commented on pull request #10293: [VirtualMachine] new method allowing to set one input tensor by its index or name
Posted by GitBox <gi...@apache.org>.
vvchernov commented on pull request #10293:
URL: https://github.com/apache/tvm/pull/10293#issuecomment-1050159908
Hello@mbs-octoml, thanks for your comment. I'm not sure that I fully caught your idea, I think it is related to design improvement of an executor, but for this the more qualified ingeneer is needed who knows functionality and design of GE, VM and possible future executor to evaluate profits and risk related to back compatability of the code. My aim was to solve my local task only and try to do it so that somebody also can use my tool. @tmoreau89 what do you think about it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org