You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2021/12/16 00:54:18 UTC
[GitHub] [tvm] huajsj opened a new pull request #9751: [Runtime][Pipeline Executor] Add the map logic of global input and subgraph input.
huajsj opened a new pull request #9751:
URL: https://github.com/apache/tvm/pull/9751
User can use "global input name" to feed input data for pipeline runtime. The name like
"data_a" will be mapped into a input interface of subgraph. In this PR, we
create the related logic to do the following things.
1. building the input map configuration
2. in runtime c++ module, parseing the input connection configuration then
creating related data structure to record the said connection map.
3. providing the function to return the map information for verification.
Thanks for contributing to TVM! Please refer to guideline https://tvm.apache.org/docs/contribute/ for useful information and tips. After the pull request is submitted, please request code reviews from [Reviewers](https://github.com/apache/incubator-tvm/blob/master/CONTRIBUTORS.md#reviewers) by @ them in the pull request thread.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] huajsj commented on a change in pull request #9751: [Runtime][Pipeline Executor] Add the map logic of global input and subgraph input.
Posted by GitBox <gi...@apache.org>.
huajsj commented on a change in pull request #9751:
URL: https://github.com/apache/tvm/pull/9751#discussion_r770185861
##########
File path: src/runtime/pipeline/pipeline_struct.h
##########
@@ -25,20 +25,26 @@
#include <limits>
#include <string>
#include <unordered_map>
+#include <utility>
#include <vector>
+#define GLOBAL_MODULE_INDEX -1
/*!
* \brief All binding information of a output interface.
*/
-struct OutputBindings {
+class ConfigBindings {
+ private:
Review comment:
fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] liangfu commented on pull request #9751: [Runtime][Pipeline Executor] Add the map logic of global input and subgraph input.
Posted by GitBox <gi...@apache.org>.
liangfu commented on pull request #9751:
URL: https://github.com/apache/tvm/pull/9751#issuecomment-995441405
Thanks for the update @huajsj , I have left additional change requests, otherwise good to me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] huajsj commented on a change in pull request #9751: [Runtime][Pipeline Executor] Add the map logic of global input and subgraph input.
Posted by GitBox <gi...@apache.org>.
huajsj commented on a change in pull request #9751:
URL: https://github.com/apache/tvm/pull/9751#discussion_r770232171
##########
File path: src/runtime/pipeline/pipeline_struct.h
##########
@@ -25,20 +25,17 @@
#include <limits>
#include <string>
#include <unordered_map>
+#include <utility>
#include <vector>
+#define GLOBAL_MODULE_INDEX -1
Review comment:
fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] HungYangChang edited a comment on pull request #9751: [Runtime][Pipeline Executor] Add the map logic of global input and subgraph input.
Posted by GitBox <gi...@apache.org>.
HungYangChang edited a comment on pull request #9751:
URL: https://github.com/apache/tvm/pull/9751#issuecomment-1042547954
This is a follow-up question from the last question.
After re-reading your [code](https://github.com/huajsj/tvm/blob/284c08e0e9ca3c10516e07adc279b03657ad1c3a/tests/python/relay/test_pipeline_executor.py)
I notice you created **data** as batch data for pipeline computation and feed data into the pipeline module.
However, from my understanding, It looks like 5 multiple stream inputs are fed to the pipeline module one by one, and each of them runs in sequential mode.
I think this implementation is different from what you claim in your example (running in pipeline format )
I am wondering do I misunderstand anything?
Or when pipeline_module_test.run("False"). Does it mean the pipeline executor executes these 5 multiple stream inputs one by one **in pipeline format** to utilize the advantage of the pipeline executor?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] HungYangChang edited a comment on pull request #9751: [Runtime][Pipeline Executor] Add the map logic of global input and subgraph input.
Posted by GitBox <gi...@apache.org>.
HungYangChang edited a comment on pull request #9751:
URL: https://github.com/apache/tvm/pull/9751#issuecomment-1042542646
Hello @huajsj
I have questions regarding how to set continuous stream inputs into the pipeline module.
In your example, there are continuous stream inputs (input1 ~ input4, ...) that are fed into the pipeline module.
In the[ test_pipeline_executor.py](https://github.com/huajsj/tvm/blob/3ab377d6be2d84fba5f773fb6c7b9e6bf79be184/tests/python/relay/test_pipeline_executor.py#L385), I noticed there is set_input function to set input data.
```
pipeline_module_test.set_input("data_a", data)
pipeline_module_test.set_input("data_b", data)
input_data = pipeline_module_test.get_input("data_a")
tvm.testing.assert_allclose(data, input_data.numpy())
# Running the pipeline executor in sequential mode.
pipeline_module_test.run(True)
```
As far as I understand, multiple inputs are essential to utilize the advantage of pipeline executor.
However, it seems like it only feeds single input instead of multiple stream inputs.
I am wondering in your pipeline implementation, do you have any codes to support such multiple stream inputs feeding?
Thanks :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] huajsj commented on a change in pull request #9751: [Runtime][Pipeline Executor] Add the map logic of global input and subgraph input.
Posted by GitBox <gi...@apache.org>.
huajsj commented on a change in pull request #9751:
URL: https://github.com/apache/tvm/pull/9751#discussion_r770185465
##########
File path: src/runtime/pipeline/pipeline_struct.h
##########
@@ -130,42 +145,113 @@ struct OutputMap {
}
}
ICHECK(output_idx >= 0);
- output_binding_map[output_idx] = binding;
+ output_binding_map_[output_idx] = binding;
}
}
};
+
/*!
* \brief The binding or dependency information of each module output interface.
*/
-struct PipelineConfig {
- /*!\brief The key is the module index, this variable records all module pipeline configuration
+class ConfigPipelineExecution {
+ private:
+ /*
+ *!\brief The key is the module index, this variable records all module pipeline configuration
* information.
*/
- std::unordered_map<int, OutputMap> config;
- OutputMap& operator[](int key) {
- ICHECK(config.find(key) != config.end());
- return config[key];
- }
+ std::unordered_map<int, ConfigOutputBindings> config_;
Review comment:
fxied.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] liangfu commented on a change in pull request #9751: [Runtime][Pipeline Executor] Add the map logic of global input and subgraph input.
Posted by GitBox <gi...@apache.org>.
liangfu commented on a change in pull request #9751:
URL: https://github.com/apache/tvm/pull/9751#discussion_r770820049
##########
File path: src/runtime/pipeline/pipeline_executor.h
##########
@@ -115,35 +123,25 @@ class TVM_DLL PipelineExecutor : public ModuleNode {
/*!\brief The class used to execute and schedule the pipeline logic.*/
PipelineScheduler pipeline_scheduler_;
/*!\brief The dependency information of each graph runtime module of the pipeline.*/
- PipelineConfig pipeline_config_;
+ ConfigPipelineExecution pipeline_config_;
+ /*!\brief The map of global input and subgraph input.*/
+ InputConnectionConfig input_connection_config;
/*!\brief The module information used to create the graph runtimes.*/
ModuleConfig mod_config_;
/*!\brief How many outputs are in this pipeline executor.*/
size_t num_outputs_ = 0;
/*!\brief Json loader.*/
- PipelineConfig& LoadPipelineConfig(dmlc::JSONReader* reader) {
- reader->BeginArray();
- while (reader->NextArrayItem()) {
- std::string key;
- reader->BeginObject();
- int mod_idx = -1;
- OutputMap output;
- std::string dev;
- while (reader->NextObjectItem(&key)) {
- if (key == "mod_idx") {
- reader->Read(&mod_idx);
- } else if (key == "dev") {
- reader->Read(&dev);
- } else if (key == "output") {
- reader->Read(&output);
- } else {
- LOG(FATAL) << "do not support key " << key;
- }
+ ConfigPipelineExecution& LoadConfig(dmlc::JSONReader* reader) {
Review comment:
A minor nitpick here is that this function `Read` everything, and returns ConfigPipelineExecution only. Since pipeline_config_ is available in public, why not Read from json and Get when it's available?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] HungYangChang commented on pull request #9751: [Runtime][Pipeline Executor] Add the map logic of global input and subgraph input.
Posted by GitBox <gi...@apache.org>.
HungYangChang commented on pull request #9751:
URL: https://github.com/apache/tvm/pull/9751#issuecomment-1042542646
Hello @huajsj
I have questions regarding how to set continuous stream inputs into the pipeline module.
In your example, there are continuous stream inputs (input1 ~ input4, ...) to feed into the pipeline module.
In the[ test_pipeline_executor.py](https://github.com/huajsj/tvm/blob/3ab377d6be2d84fba5f773fb6c7b9e6bf79be184/tests/python/relay/test_pipeline_executor.py#L385), I noticed there is set_input function to set input data.
```
pipeline_module_test.set_input("data_a", data)
pipeline_module_test.set_input("data_b", data)
input_data = pipeline_module_test.get_input("data_a")
tvm.testing.assert_allclose(data, input_data.numpy())
# Running the pipeline executor in sequential mode.
pipeline_module_test.run(True)
```
As far as I understand, multiple inputs are essential to utilize the advantage of pipeline executor.
However, it seems like it only feeds single input instead of multiple stream inputs.
I am wondering in your pipeline implementation, do you have any codes to support such multiple stream inputs feeding?
Could you confirm it?
Thanks :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] HungYangChang edited a comment on pull request #9751: [Runtime][Pipeline Executor] Add the map logic of global input and subgraph input.
Posted by GitBox <gi...@apache.org>.
HungYangChang edited a comment on pull request #9751:
URL: https://github.com/apache/tvm/pull/9751#issuecomment-1042547954
This is a follow-up question from the last question.
After re-reading your [code](https://github.com/huajsj/tvm/blob/284c08e0e9ca3c10516e07adc279b03657ad1c3a/tests/python/relay/test_pipeline_executor.py),
I notice you created **data** as batch data for pipeline computation and feed data into the pipeline module.
ges.githubusercontent.com/70759685/154417373-481c89c6-bee0-4ad3-959b-b2f8fe27a429.png)
However, from my understanding, It looks like 5 multiple stream inputs are fed to the pipeline module one by one, and each of them runs in sequential mode.
I think this implementation is different from what you claim in your example (running in pipeline format )
I am wondering do I misunderstand anything?
Or when pipeline_module_test.run("False"). Does it mean the pipeline executor executes these 5 multiple stream inputs one by one **in pipeline format** to utilize the advantage of the pipeline executor?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] HungYangChang edited a comment on pull request #9751: [Runtime][Pipeline Executor] Add the map logic of global input and subgraph input.
Posted by GitBox <gi...@apache.org>.
HungYangChang edited a comment on pull request #9751:
URL: https://github.com/apache/tvm/pull/9751#issuecomment-1042542646
Hello @huajsj
I have questions regarding how to set continuous stream inputs into the pipeline module.
In your example, there are continuous stream inputs (input1 ~ input4, ...) that are fed into the pipeline module.
In the[ test_pipeline_executor.py](https://github.com/huajsj/tvm/blob/284c08e0e9ca3c10516e07adc279b03657ad1c3a/tests/python/relay/test_pipeline_executor.py#L384), I noticed there is set_input function to set input data.
```
pipeline_module_test.set_input("data_a", data)
pipeline_module_test.set_input("data_b", data)
input_data = pipeline_module_test.get_input("data_a")
tvm.testing.assert_allclose(data, input_data.numpy())
# Running the pipeline executor in sequential mode.
pipeline_module_test.run(True)
```
As far as I understand, multiple inputs are essential to utilize the advantage of pipeline executor.
However, it seems like it only feeds single input instead of multiple stream inputs.
I am wondering in your pipeline implementation, do you have any codes to support such multiple stream inputs feeding?
Thanks :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] liangfu commented on a change in pull request #9751: [Runtime][Pipeline Executor] Add the map logic of global input and subgraph input.
Posted by GitBox <gi...@apache.org>.
liangfu commented on a change in pull request #9751:
URL: https://github.com/apache/tvm/pull/9751#discussion_r770222356
##########
File path: src/runtime/pipeline/pipeline_executor.h
##########
@@ -115,35 +123,25 @@ class TVM_DLL PipelineExecutor : public ModuleNode {
/*!\brief The class used to execute and schedule the pipeline logic.*/
PipelineScheduler pipeline_scheduler_;
/*!\brief The dependency information of each graph runtime module of the pipeline.*/
- PipelineConfig pipeline_config_;
+ ConfigPipelineExecution pipeline_config_;
+ /*!\brief The map of global input and subgraph input.*/
+ InputConnectionConfig input_connection_config;
/*!\brief The module information used to create the graph runtimes.*/
ModuleConfig mod_config_;
/*!\brief How many outputs are in this pipeline executor.*/
size_t num_outputs_ = 0;
/*!\brief Json loader.*/
- PipelineConfig& LoadPipelineConfig(dmlc::JSONReader* reader) {
- reader->BeginArray();
- while (reader->NextArrayItem()) {
- std::string key;
- reader->BeginObject();
- int mod_idx = -1;
- OutputMap output;
- std::string dev;
- while (reader->NextObjectItem(&key)) {
- if (key == "mod_idx") {
- reader->Read(&mod_idx);
- } else if (key == "dev") {
- reader->Read(&dev);
- } else if (key == "output") {
- reader->Read(&output);
- } else {
- LOG(FATAL) << "do not support key " << key;
- }
+ ConfigPipelineExecution& LoadConfigPipelineExecution(dmlc::JSONReader* reader) {
Review comment:
This logic in this function looks a bit weird to me. The function name indicate it would load ConfigPipelineExecution, but it loads InputConnectionConfig as well. I think the name of the function should be consistent with its behavior.
##########
File path: src/runtime/pipeline/pipeline_struct.h
##########
@@ -25,20 +25,17 @@
#include <limits>
#include <string>
#include <unordered_map>
+#include <utility>
#include <vector>
+#define GLOBAL_MODULE_INDEX -1
Review comment:
Please remove the macro here, since it has been used only once.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] liangfu commented on a change in pull request #9751: [Runtime][Pipeline Executor] Add the map logic of global input and subgraph input.
Posted by GitBox <gi...@apache.org>.
liangfu commented on a change in pull request #9751:
URL: https://github.com/apache/tvm/pull/9751#discussion_r770168691
##########
File path: src/runtime/pipeline/pipeline_struct.h
##########
@@ -71,44 +77,53 @@ struct OutputBindings {
// When this output is bound to a global interface, check if the global interface index
// start from 0.
if (global_binding) {
- ICHECK(global_output_index >= 0);
+ ICHECK(global_output_index_ >= 0);
} else {
// When this output is bound to a graph executor module interface, check if the module
// index start from 0.
ICHECK(mod_idx >= 0);
- bindings[mod_idx] = input_name;
+ bindings_[mod_idx] = input_name;
}
}
}
};
-
/*!
* \brief The binding information of all outputs of a module.
*/
-struct OutputMap {
- /*! \brief Output binding map, 'int' is output interface index.*/
- std::unordered_map<int, OutputBindings> output_binding_map;
- OutputMap& operator=(const OutputMap& output) {
- output_binding_map = output.output_binding_map;
+class ConfigOutputBindings {
+ private:
+ /*!\brief The map of output binding, 'int' is the output interface index.*/
+ std::unordered_map<int, ConfigBindings> output_binding_map_;
Review comment:
put `private:` members on the bottom.
##########
File path: src/runtime/pipeline/pipeline_struct.h
##########
@@ -130,42 +145,113 @@ struct OutputMap {
}
}
ICHECK(output_idx >= 0);
- output_binding_map[output_idx] = binding;
+ output_binding_map_[output_idx] = binding;
}
}
};
+
/*!
* \brief The binding or dependency information of each module output interface.
*/
-struct PipelineConfig {
- /*!\brief The key is the module index, this variable records all module pipeline configuration
+class ConfigPipelineExecution {
+ private:
+ /*
+ *!\brief The key is the module index, this variable records all module pipeline configuration
* information.
*/
- std::unordered_map<int, OutputMap> config;
- OutputMap& operator[](int key) {
- ICHECK(config.find(key) != config.end());
- return config[key];
- }
+ std::unordered_map<int, ConfigOutputBindings> config_;
Review comment:
put `private:` members on the bottom.
##########
File path: src/runtime/pipeline/pipeline_struct.h
##########
@@ -25,20 +25,26 @@
#include <limits>
#include <string>
#include <unordered_map>
+#include <utility>
#include <vector>
+#define GLOBAL_MODULE_INDEX -1
/*!
* \brief All binding information of a output interface.
*/
-struct OutputBindings {
+class ConfigBindings {
+ private:
Review comment:
Please follow Google C++ code style guide to put `private:` members on the bottom.
##########
File path: python/tvm/contrib/pipeline_executor.py
##########
@@ -199,12 +229,48 @@ def is_pipeline_executor_interface(self):
return not isinstance(self.io_owner, PipelineConfig.ModuleWrapper)
def __repr__(self):
- # Get all binding information.
- ret = " |{}: ".format(self.name)
+ # Geting the binding information in the form of text.
+ str_format = " |{}: ".format(self.name)
for binding in self.bindings:
mname, dname = binding.get_name()
- ret += "{0}:{1} ".format(mname, dname)
- return ret
+ str_format += "{0}:{1} ".format(mname, dname)
+
+ return str_format
+
+ def check_binding_dict(self, connection_dict):
+ """Checking the binding dictionary.
+ Parameter
+ ---------
+ connection_dict : Dict[str, Any]
+ It is a dictionary of module connections.
+ """
+ if "interface_name" not in connection_dict:
+ raise RuntimeError(f'"inteface_name" is missing in global config!"')
Review comment:
There is no variable in the string, please remove the f prefix.
```suggestion
raise RuntimeError('"inteface_name" is missing in global config!"')
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] huajsj commented on pull request #9751: [Runtime][Pipeline Executor] Add the map logic of global input and subgraph input.
Posted by GitBox <gi...@apache.org>.
huajsj commented on pull request #9751:
URL: https://github.com/apache/tvm/pull/9751#issuecomment-996122916
all review comments addressed, thanks @liangfu.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] huajsj commented on a change in pull request #9751: [Runtime][Pipeline Executor] Add the map logic of global input and subgraph input.
Posted by GitBox <gi...@apache.org>.
huajsj commented on a change in pull request #9751:
URL: https://github.com/apache/tvm/pull/9751#discussion_r770185406
##########
File path: src/runtime/pipeline/pipeline_struct.h
##########
@@ -71,44 +77,53 @@ struct OutputBindings {
// When this output is bound to a global interface, check if the global interface index
// start from 0.
if (global_binding) {
- ICHECK(global_output_index >= 0);
+ ICHECK(global_output_index_ >= 0);
} else {
// When this output is bound to a graph executor module interface, check if the module
// index start from 0.
ICHECK(mod_idx >= 0);
- bindings[mod_idx] = input_name;
+ bindings_[mod_idx] = input_name;
}
}
}
};
-
/*!
* \brief The binding information of all outputs of a module.
*/
-struct OutputMap {
- /*! \brief Output binding map, 'int' is output interface index.*/
- std::unordered_map<int, OutputBindings> output_binding_map;
- OutputMap& operator=(const OutputMap& output) {
- output_binding_map = output.output_binding_map;
+class ConfigOutputBindings {
+ private:
+ /*!\brief The map of output binding, 'int' is the output interface index.*/
+ std::unordered_map<int, ConfigBindings> output_binding_map_;
Review comment:
fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] huajsj commented on a change in pull request #9751: [Runtime][Pipeline Executor] Add the map logic of global input and subgraph input.
Posted by GitBox <gi...@apache.org>.
huajsj commented on a change in pull request #9751:
URL: https://github.com/apache/tvm/pull/9751#discussion_r770232372
##########
File path: src/runtime/pipeline/pipeline_executor.h
##########
@@ -115,35 +123,25 @@ class TVM_DLL PipelineExecutor : public ModuleNode {
/*!\brief The class used to execute and schedule the pipeline logic.*/
PipelineScheduler pipeline_scheduler_;
/*!\brief The dependency information of each graph runtime module of the pipeline.*/
- PipelineConfig pipeline_config_;
+ ConfigPipelineExecution pipeline_config_;
+ /*!\brief The map of global input and subgraph input.*/
+ InputConnectionConfig input_connection_config;
/*!\brief The module information used to create the graph runtimes.*/
ModuleConfig mod_config_;
/*!\brief How many outputs are in this pipeline executor.*/
size_t num_outputs_ = 0;
/*!\brief Json loader.*/
- PipelineConfig& LoadPipelineConfig(dmlc::JSONReader* reader) {
- reader->BeginArray();
- while (reader->NextArrayItem()) {
- std::string key;
- reader->BeginObject();
- int mod_idx = -1;
- OutputMap output;
- std::string dev;
- while (reader->NextObjectItem(&key)) {
- if (key == "mod_idx") {
- reader->Read(&mod_idx);
- } else if (key == "dev") {
- reader->Read(&dev);
- } else if (key == "output") {
- reader->Read(&output);
- } else {
- LOG(FATAL) << "do not support key " << key;
- }
+ ConfigPipelineExecution& LoadConfigPipelineExecution(dmlc::JSONReader* reader) {
Review comment:
changed the name into "LoadConfig" to avoid confuse.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] HungYangChang edited a comment on pull request #9751: [Runtime][Pipeline Executor] Add the map logic of global input and subgraph input.
Posted by GitBox <gi...@apache.org>.
HungYangChang edited a comment on pull request #9751:
URL: https://github.com/apache/tvm/pull/9751#issuecomment-1042547954
This is a follow-up question from the last question.
After re-reading your [code](https://github.com/huajsj/tvm/blob/284c08e0e9ca3c10516e07adc279b03657ad1c3a/tests/python/relay/test_pipeline_executor.py),
I notice you created **data** as batch data for pipeline computation and feed data into the pipeline module.
However, from my understanding, It looks like 5 multiple stream inputs are fed to the pipeline module one by one, and each of them runs in sequential mode.
I think this implementation is different from what you claim in your example (running in pipeline format )
I am wondering do I misunderstand anything?
Or when pipeline_module_test.run("False"). Does it mean the pipeline executor executes these 5 multiple stream inputs one by one **in pipeline format** to utilize the advantage of the pipeline executor?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] HungYangChang commented on pull request #9751: [Runtime][Pipeline Executor] Add the map logic of global input and subgraph input.
Posted by GitBox <gi...@apache.org>.
HungYangChang commented on pull request #9751:
URL: https://github.com/apache/tvm/pull/9751#issuecomment-1042547954
This is a follow-up question from the last question.
After re-reading your code,
I notice you created **data** as batch data for pipeline computation and feed data into the pipeline module.
data:
[array([[3., 3., 3.],
[3., 3., 3.],
[3., 3., 3.]], dtype=float32), array([[4., 4., 4.],
[4., 4., 4.],
[4., 4., 4.]], dtype=float32), array([[5., 5., 5.],
[5., 5., 5.],
[5., 5., 5.]], dtype=float32), array([[6., 6., 6.],
[6., 6., 6.],
[6., 6., 6.]], dtype=float32), array([[7., 7., 7.],
[7., 7., 7.],
[7., 7., 7.]], dtype=float32)]
I am wondering does it means the pipeline executor executes these 5 multiple stream inputs one by one in pipeline format to utilize the advantage of the pipeline executor?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] huajsj commented on a change in pull request #9751: [Runtime][Pipeline Executor] Add the map logic of global input and subgraph input.
Posted by GitBox <gi...@apache.org>.
huajsj commented on a change in pull request #9751:
URL: https://github.com/apache/tvm/pull/9751#discussion_r770869156
##########
File path: src/runtime/pipeline/pipeline_executor.h
##########
@@ -115,35 +123,25 @@ class TVM_DLL PipelineExecutor : public ModuleNode {
/*!\brief The class used to execute and schedule the pipeline logic.*/
PipelineScheduler pipeline_scheduler_;
/*!\brief The dependency information of each graph runtime module of the pipeline.*/
- PipelineConfig pipeline_config_;
+ ConfigPipelineExecution pipeline_config_;
+ /*!\brief The map of global input and subgraph input.*/
+ InputConnectionConfig input_connection_config;
/*!\brief The module information used to create the graph runtimes.*/
ModuleConfig mod_config_;
/*!\brief How many outputs are in this pipeline executor.*/
size_t num_outputs_ = 0;
/*!\brief Json loader.*/
- PipelineConfig& LoadPipelineConfig(dmlc::JSONReader* reader) {
- reader->BeginArray();
- while (reader->NextArrayItem()) {
- std::string key;
- reader->BeginObject();
- int mod_idx = -1;
- OutputMap output;
- std::string dev;
- while (reader->NextObjectItem(&key)) {
- if (key == "mod_idx") {
- reader->Read(&mod_idx);
- } else if (key == "dev") {
- reader->Read(&dev);
- } else if (key == "output") {
- reader->Read(&output);
- } else {
- LOG(FATAL) << "do not support key " << key;
- }
+ ConfigPipelineExecution& LoadConfig(dmlc::JSONReader* reader) {
Review comment:
fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] liangfu commented on pull request #9751: [Runtime][Pipeline Executor] Add the map logic of global input and subgraph input.
Posted by GitBox <gi...@apache.org>.
liangfu commented on pull request #9751:
URL: https://github.com/apache/tvm/pull/9751#issuecomment-996969333
Thanks for contributing and the updates @huajsj. This is now merged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] huajsj commented on a change in pull request #9751: [Runtime][Pipeline Executor] Add the map logic of global input and subgraph input.
Posted by GitBox <gi...@apache.org>.
huajsj commented on a change in pull request #9751:
URL: https://github.com/apache/tvm/pull/9751#discussion_r770185502
##########
File path: python/tvm/contrib/pipeline_executor.py
##########
@@ -199,12 +229,48 @@ def is_pipeline_executor_interface(self):
return not isinstance(self.io_owner, PipelineConfig.ModuleWrapper)
def __repr__(self):
- # Get all binding information.
- ret = " |{}: ".format(self.name)
+ # Geting the binding information in the form of text.
+ str_format = " |{}: ".format(self.name)
for binding in self.bindings:
mname, dname = binding.get_name()
- ret += "{0}:{1} ".format(mname, dname)
- return ret
+ str_format += "{0}:{1} ".format(mname, dname)
+
+ return str_format
+
+ def check_binding_dict(self, connection_dict):
+ """Checking the binding dictionary.
+ Parameter
+ ---------
+ connection_dict : Dict[str, Any]
+ It is a dictionary of module connections.
+ """
+ if "interface_name" not in connection_dict:
+ raise RuntimeError(f'"inteface_name" is missing in global config!"')
Review comment:
fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] liangfu merged pull request #9751: [Runtime][Pipeline Executor] Add the map logic of global input and subgraph input.
Posted by GitBox <gi...@apache.org>.
liangfu merged pull request #9751:
URL: https://github.com/apache/tvm/pull/9751
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] huajsj commented on pull request #9751: [Runtime][Pipeline Executor] Add the map logic of global input and subgraph input.
Posted by GitBox <gi...@apache.org>.
huajsj commented on pull request #9751:
URL: https://github.com/apache/tvm/pull/9751#issuecomment-995449517
Thanks @liangfu, all review comments addressed, please take a look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] HungYangChang edited a comment on pull request #9751: [Runtime][Pipeline Executor] Add the map logic of global input and subgraph input.
Posted by GitBox <gi...@apache.org>.
HungYangChang edited a comment on pull request #9751:
URL: https://github.com/apache/tvm/pull/9751#issuecomment-1042547954
This is a follow-up question from the last question.
After re-reading your [code](https://github.com/huajsj/tvm/blob/284c08e0e9ca3c10516e07adc279b03657ad1c3a/tests/python/relay/test_pipeline_executor.py)
I notice you created **data** as batch data for pipeline computation and feed data into the pipeline module.
However, from my understanding, It looks like 5 multiple stream inputs are fed to the pipeline module one by one, and each of them runs in sequential mode as the below figure shwon.
I think this implementation is different from what you claim in your example (running in pipeline format).
I am wondering do I misunderstand anything?
Or when pipeline_module_test.run("False"). Does it mean the pipeline executor executes these 5 multiple stream inputs one by one **in pipeline format** to utilize the advantage of the pipeline executor?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] HungYangChang edited a comment on pull request #9751: [Runtime][Pipeline Executor] Add the map logic of global input and subgraph input.
Posted by GitBox <gi...@apache.org>.
HungYangChang edited a comment on pull request #9751:
URL: https://github.com/apache/tvm/pull/9751#issuecomment-1042547954
This is a follow-up question from the last question.
After re-reading your [code](https://github.com/huajsj/tvm/blob/284c08e0e9ca3c10516e07adc279b03657ad1c3a/tests/python/relay/test_pipeline_executor.py)
I notice you created **data** as batch data for pipeline computation and feed data into the pipeline module.
However, from my understanding, It looks like 5 multiple stream inputs are fed to the pipeline module one by one, and each of them runs in sequential mode as the below figure shown.
I think this implementation is different from what you claim in your example (running in pipeline format).
I am wondering do I misunderstand anything?
Or when pipeline_module_test.run("False"). Does it mean the pipeline executor executes these 5 multiple stream inputs one by one **in pipeline format** to utilize the advantage of the pipeline executor?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] HungYangChang edited a comment on pull request #9751: [Runtime][Pipeline Executor] Add the map logic of global input and subgraph input.
Posted by GitBox <gi...@apache.org>.
HungYangChang edited a comment on pull request #9751:
URL: https://github.com/apache/tvm/pull/9751#issuecomment-1042542646
Hello @huajsj
I have questions regarding how to set continuous stream inputs into the pipeline module.
In your example, there are continuous stream inputs (input1 ~ input4, ...) to feed into the pipeline module.
In the[ test_pipeline_executor.py](https://github.com/huajsj/tvm/blob/3ab377d6be2d84fba5f773fb6c7b9e6bf79be184/tests/python/relay/test_pipeline_executor.py#L385), I noticed there is set_input function to set input data.
```
pipeline_module_test.set_input("data_a", data)
pipeline_module_test.set_input("data_b", data)
input_data = pipeline_module_test.get_input("data_a")
tvm.testing.assert_allclose(data, input_data.numpy())
# Running the pipeline executor in sequential mode.
pipeline_module_test.run(True)
```
As far as I understand, multiple inputs are essential to utilize the advantage of pipeline executor.
However, it seems like it only feeds single input instead of multiple stream inputs.
I am wondering in your pipeline implementation, do you have any codes to support such multiple stream inputs feeding?
Thanks :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org