You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2022/06/03 16:46:00 UTC
[GitHub] [tvm-rfcs] mehrdadh commented on a diff in pull request #76: [RFC][MLF] Model Library Format with Multiple Modules
mehrdadh commented on code in PR #76:
URL: https://github.com/apache/tvm-rfcs/pull/76#discussion_r889130105
##########
rfcs/0076_mlf_with_multiple_modules.md:
##########
@@ -0,0 +1,165 @@
+- Feature Name: mlf_with_multiple_module_support
+- Start Date: 2022-05-31
+- RFC PR: [apache/tvm-rfcs#0075](https://github.com/apache/tvm-rfcs/pull/0075)
+- GitHub Issue: [apache/tvm#0000](https://github.com/apache/tvm/issues/0000)
+
+# Summary
+[summary]: #summary
+
+This RFC describes a new version of model library format where we can support adding multiple Relay module builds into a single MLF file.
+
+# Motivation
+[motivation]: #motivation
+
+MLF artifact is an important piece of building a microTVM project. In the first version of the MLF artifact,
+MLF only supports a single "Relay module"/"PackedFunc" build. However, there are cases where adding
+multiple Relay modules is required. For example, corstone300 tests using multiple Relay modules was introduced by
+contributors from ARM. The goal of this RFC is to create a new version of MLF which has standard support for these cases.
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+Users call `export_model_library_format` function with multiple Relay builds (output from `tvm.relay.build`) or
+single a Relay build module. We change the `export_model_library_format` API to support multiple Relay Build, however
Review Comment:
done.
##########
rfcs/0076_mlf_with_multiple_modules.md:
##########
@@ -0,0 +1,165 @@
+- Feature Name: mlf_with_multiple_module_support
+- Start Date: 2022-05-31
+- RFC PR: [apache/tvm-rfcs#0075](https://github.com/apache/tvm-rfcs/pull/0075)
+- GitHub Issue: [apache/tvm#0000](https://github.com/apache/tvm/issues/0000)
+
+# Summary
+[summary]: #summary
+
+This RFC describes a new version of model library format where we can support adding multiple Relay module builds into a single MLF file.
+
+# Motivation
+[motivation]: #motivation
+
+MLF artifact is an important piece of building a microTVM project. In the first version of the MLF artifact,
+MLF only supports a single "Relay module"/"PackedFunc" build. However, there are cases where adding
+multiple Relay modules is required. For example, corstone300 tests using multiple Relay modules was introduced by
+contributors from ARM. The goal of this RFC is to create a new version of MLF which has standard support for these cases.
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+Users call `export_model_library_format` function with multiple Relay builds (output from `tvm.relay.build`) or
+single a Relay build module. We change the `export_model_library_format` API to support multiple Relay Build, however
+it is backward compatible and, we handle required changes internally.
+
+Since we are including multiple modules in a single MLF file, we need to make some changes. Let's assume we
+are want to export MLF file for two Relay build modules called `mod1` and `mod2`.
+
+1. We change format of `metadata.json` file. We will add `modules` as a key to JSON file which represents
+ multiple Relay build module. `modules` is a dictionary and each Relay build module is differentiated by
+ its module name. For example:
+ ```javascript
+ {
+ 'modules': {
+ 'mod1': {
Review Comment:
added more details to the RFC. Everything would be the same as before except version which will remain at top level.
##########
rfcs/0076_mlf_with_multiple_modules.md:
##########
@@ -0,0 +1,165 @@
+- Feature Name: mlf_with_multiple_module_support
+- Start Date: 2022-05-31
+- RFC PR: [apache/tvm-rfcs#0075](https://github.com/apache/tvm-rfcs/pull/0075)
+- GitHub Issue: [apache/tvm#0000](https://github.com/apache/tvm/issues/0000)
+
+# Summary
+[summary]: #summary
+
+This RFC describes a new version of model library format where we can support adding multiple Relay module builds into a single MLF file.
+
+# Motivation
+[motivation]: #motivation
+
+MLF artifact is an important piece of building a microTVM project. In the first version of the MLF artifact,
+MLF only supports a single "Relay module"/"PackedFunc" build. However, there are cases where adding
+multiple Relay modules is required. For example, corstone300 tests using multiple Relay modules was introduced by
+contributors from ARM. The goal of this RFC is to create a new version of MLF which has standard support for these cases.
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+Users call `export_model_library_format` function with multiple Relay builds (output from `tvm.relay.build`) or
+single a Relay build module. We change the `export_model_library_format` API to support multiple Relay Build, however
+it is backward compatible and, we handle required changes internally.
+
+Since we are including multiple modules in a single MLF file, we need to make some changes. Let's assume we
+are want to export MLF file for two Relay build modules called `mod1` and `mod2`.
Review Comment:
done.
##########
rfcs/0076_mlf_with_multiple_modules.md:
##########
@@ -0,0 +1,165 @@
+- Feature Name: mlf_with_multiple_module_support
+- Start Date: 2022-05-31
+- RFC PR: [apache/tvm-rfcs#0075](https://github.com/apache/tvm-rfcs/pull/0075)
+- GitHub Issue: [apache/tvm#0000](https://github.com/apache/tvm/issues/0000)
+
+# Summary
+[summary]: #summary
+
+This RFC describes a new version of model library format where we can support adding multiple Relay module builds into a single MLF file.
+
+# Motivation
+[motivation]: #motivation
+
+MLF artifact is an important piece of building a microTVM project. In the first version of the MLF artifact,
+MLF only supports a single "Relay module"/"PackedFunc" build. However, there are cases where adding
+multiple Relay modules is required. For example, corstone300 tests using multiple Relay modules was introduced by
+contributors from ARM. The goal of this RFC is to create a new version of MLF which has standard support for these cases.
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+Users call `export_model_library_format` function with multiple Relay builds (output from `tvm.relay.build`) or
+single a Relay build module. We change the `export_model_library_format` API to support multiple Relay Build, however
+it is backward compatible and, we handle required changes internally.
+
+Since we are including multiple modules in a single MLF file, we need to make some changes. Let's assume we
+are want to export MLF file for two Relay build modules called `mod1` and `mod2`.
+
+1. We change format of `metadata.json` file. We will add `modules` as a key to JSON file which represents
+ multiple Relay build module. `modules` is a dictionary and each Relay build module is differentiated by
+ its module name. For example:
+ ```javascript
+ {
+ 'modules': {
+ 'mod1': {
+
+ },
+ 'mod2': {
+
+ },
+ ...
+ },
+ 'version': XXX,
+ }
+ ```
+ 2. There are multiple Relay text file generated per each Relay build. We propose to structure these files by using
+ module name and `.relay` extension to differentiate its format(E.g. {`mod1.relay`, `mod2.relay`, ...}). Similarly, for
+ `graph.json` file we propose to restructure to graph JSON file to {`mod1.graph`, `mod2.graph`, ...}. This is
+ compatible with existing structure for parameters where we use {`mod1.params`, `mod2.params`, ...}.
+ 3. Finally, we keep higher level information which are not specific to a module at higher level in the metadata file.
+ Currently, we only have `version` which shows the MLF version, but this could grow in the future.
+
+# Reference-level explanation
+[reference-level-explanation]: #reference-level-explanation
+
+Building on the same example in previous section, here we explain what are the API calls and expected output after this change.
+
+First, we build two modules:
+```python
+mod1 = ...
+mod2 = ...
+
+executor = Executor("graph")
+runtime = Runtime("crt")
+target = tvm.target.target.micro("host")
+
+with tvm.transform.PassContext(opt_level=3):
+ factory1 = tvm.relay.build(mod1, target, runtime=runtime, executor=executor, mod_name="mod1")
+ factory2 = tvm.relay.build(mod2, target, runtime=runtime, executor=executor, mod_name="mod2")
+```
+
+Then, we pass both results of `tvm.relay.build` to `export_model_library_format` function and path to generated model
+library format:
+```python
+micro.export_model_library_format([factory1, factory2], mlf_tar_path)
+```
+
+Now, if we extract MLF file here is the file structure:
+```bash
+# codegen source files
+codegen/
+ host/
+ src/
+ mod1_lib0.c
+ mod1_lib1.c
+ mod1_lib2.c
+ mod2_lib0.c
+ mod2_lib1.c
+ mod2_lib2.c
+
+# graph
+executor-config/
+ graph/
+ mod1.graph
+ mod2.graph
+
+# metadata file
+metadata.json
+
+# parameters
+parameters/
+ mod1.params
+ mod2.params
+
+# relay text output
+src/
+ mod1.relay
+ mod2.relay
+```
+
+# Drawbacks
+[drawbacks]: #drawbacks
+
+The drawback here is that we are changing generated metadata and file structure in the MLF file which means external
+tools which are dependent to this need to update their tool. Hopefully this RFC makes it clear on what steps they need
+to take.
+
+# **Rationale and alternatives**
+An alternative way to implement this feature is to break down each Relay build module to a subdirectory and keep the
+previous format inside each sub Relay build module. Using the example of `mod1` and `mod2`, in this approach we have
+an MLF file format with structure bellow if we extract:
+```bash
+# mod1
+mod1/
+ codegen/
+ host/
+ src/
+ mod_lib0.c
+ mod_lib1.c
+ mod_lib2.c
+ executor-config/
+ graph/
+ mod.graph
+ parameters/
+ mod.params
+ src/
+ mod.relay
+
+#mod2
+mod2/
+ codegen/
+ host/
+ src/
+ mod_lib0.c
+ mod_lib1.c
+ mod_lib2.c
+ executor-config/
+ graph/
+ mod.graph
+ parameters/
+ mod.params
+ src/
+ mod.relay
+```
+
+One of the benefits of this approach is that create a more readable file structure which is modularized for
+each Relay build module. However, the downside is that this approach will result in more modifications in project
Review Comment:
I agree, I think it will add more complexity to the build process.
##########
rfcs/0076_mlf_with_multiple_modules.md:
##########
@@ -0,0 +1,165 @@
+- Feature Name: mlf_with_multiple_module_support
+- Start Date: 2022-05-31
+- RFC PR: [apache/tvm-rfcs#0075](https://github.com/apache/tvm-rfcs/pull/0075)
+- GitHub Issue: [apache/tvm#0000](https://github.com/apache/tvm/issues/0000)
+
+# Summary
+[summary]: #summary
+
+This RFC describes a new version of model library format where we can support adding multiple Relay module builds into a single MLF file.
+
+# Motivation
+[motivation]: #motivation
+
+MLF artifact is an important piece of building a microTVM project. In the first version of the MLF artifact,
+MLF only supports a single "Relay module"/"PackedFunc" build. However, there are cases where adding
+multiple Relay modules is required. For example, corstone300 tests using multiple Relay modules was introduced by
+contributors from ARM. The goal of this RFC is to create a new version of MLF which has standard support for these cases.
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+Users call `export_model_library_format` function with multiple Relay builds (output from `tvm.relay.build`) or
+single a Relay build module. We change the `export_model_library_format` API to support multiple Relay Build, however
+it is backward compatible and, we handle required changes internally.
+
+Since we are including multiple modules in a single MLF file, we need to make some changes. Let's assume we
+are want to export MLF file for two Relay build modules called `mod1` and `mod2`.
+
+1. We change format of `metadata.json` file. We will add `modules` as a key to JSON file which represents
+ multiple Relay build module. `modules` is a dictionary and each Relay build module is differentiated by
+ its module name. For example:
+ ```javascript
+ {
+ 'modules': {
+ 'mod1': {
+
+ },
+ 'mod2': {
+
+ },
+ ...
+ },
+ 'version': XXX,
+ }
+ ```
+ 2. There are multiple Relay text file generated per each Relay build. We propose to structure these files by using
+ module name and `.relay` extension to differentiate its format(E.g. {`mod1.relay`, `mod2.relay`, ...}). Similarly, for
+ `graph.json` file we propose to restructure to graph JSON file to {`mod1.graph`, `mod2.graph`, ...}. This is
+ compatible with existing structure for parameters where we use {`mod1.params`, `mod2.params`, ...}.
+ 3. Finally, we keep higher level information which are not specific to a module at higher level in the metadata file.
+ Currently, we only have `version` which shows the MLF version, but this could grow in the future.
+
+# Reference-level explanation
+[reference-level-explanation]: #reference-level-explanation
+
+Building on the same example in previous section, here we explain what are the API calls and expected output after this change.
+
+First, we build two modules:
+```python
+mod1 = ...
+mod2 = ...
+
+executor = Executor("graph")
+runtime = Runtime("crt")
+target = tvm.target.target.micro("host")
+
+with tvm.transform.PassContext(opt_level=3):
+ factory1 = tvm.relay.build(mod1, target, runtime=runtime, executor=executor, mod_name="mod1")
+ factory2 = tvm.relay.build(mod2, target, runtime=runtime, executor=executor, mod_name="mod2")
+```
+
+Then, we pass both results of `tvm.relay.build` to `export_model_library_format` function and path to generated model
+library format:
+```python
+micro.export_model_library_format([factory1, factory2], mlf_tar_path)
+```
+
+Now, if we extract MLF file here is the file structure:
+```bash
+# codegen source files
+codegen/
+ host/
+ src/
+ mod1_lib0.c
+ mod1_lib1.c
+ mod1_lib2.c
+ mod2_lib0.c
+ mod2_lib1.c
+ mod2_lib2.c
+
+# graph
+executor-config/
+ graph/
+ mod1.graph
+ mod2.graph
+
+# metadata file
+metadata.json
+
+# parameters
+parameters/
+ mod1.params
+ mod2.params
+
+# relay text output
+src/
+ mod1.relay
+ mod2.relay
+```
+
+# Drawbacks
+[drawbacks]: #drawbacks
+
+The drawback here is that we are changing generated metadata and file structure in the MLF file which means external
+tools which are dependent to this need to update their tool. Hopefully this RFC makes it clear on what steps they need
+to take.
+
+# **Rationale and alternatives**
+An alternative way to implement this feature is to break down each Relay build module to a subdirectory and keep the
+previous format inside each sub Relay build module. Using the example of `mod1` and `mod2`, in this approach we have
+an MLF file format with structure bellow if we extract:
Review Comment:
done.
##########
rfcs/0076_mlf_with_multiple_modules.md:
##########
@@ -0,0 +1,165 @@
+- Feature Name: mlf_with_multiple_module_support
+- Start Date: 2022-05-31
+- RFC PR: [apache/tvm-rfcs#0075](https://github.com/apache/tvm-rfcs/pull/0075)
+- GitHub Issue: [apache/tvm#0000](https://github.com/apache/tvm/issues/0000)
+
+# Summary
+[summary]: #summary
+
+This RFC describes a new version of model library format where we can support adding multiple Relay module builds into a single MLF file.
+
+# Motivation
+[motivation]: #motivation
+
+MLF artifact is an important piece of building a microTVM project. In the first version of the MLF artifact,
+MLF only supports a single "Relay module"/"PackedFunc" build. However, there are cases where adding
+multiple Relay modules is required. For example, corstone300 tests using multiple Relay modules was introduced by
+contributors from ARM. The goal of this RFC is to create a new version of MLF which has standard support for these cases.
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+Users call `export_model_library_format` function with multiple Relay builds (output from `tvm.relay.build`) or
+single a Relay build module. We change the `export_model_library_format` API to support multiple Relay Build, however
+it is backward compatible and, we handle required changes internally.
+
+Since we are including multiple modules in a single MLF file, we need to make some changes. Let's assume we
+are want to export MLF file for two Relay build modules called `mod1` and `mod2`.
+
+1. We change format of `metadata.json` file. We will add `modules` as a key to JSON file which represents
+ multiple Relay build module. `modules` is a dictionary and each Relay build module is differentiated by
+ its module name. For example:
+ ```javascript
+ {
+ 'modules': {
+ 'mod1': {
+
+ },
+ 'mod2': {
+
+ },
+ ...
+ },
+ 'version': XXX,
+ }
+ ```
+ 2. There are multiple Relay text file generated per each Relay build. We propose to structure these files by using
+ module name and `.relay` extension to differentiate its format(E.g. {`mod1.relay`, `mod2.relay`, ...}). Similarly, for
+ `graph.json` file we propose to restructure to graph JSON file to {`mod1.graph`, `mod2.graph`, ...}. This is
+ compatible with existing structure for parameters where we use {`mod1.params`, `mod2.params`, ...}.
+ 3. Finally, we keep higher level information which are not specific to a module at higher level in the metadata file.
+ Currently, we only have `version` which shows the MLF version, but this could grow in the future.
+
+# Reference-level explanation
+[reference-level-explanation]: #reference-level-explanation
+
+Building on the same example in previous section, here we explain what are the API calls and expected output after this change.
+
+First, we build two modules:
+```python
+mod1 = ...
+mod2 = ...
+
+executor = Executor("graph")
+runtime = Runtime("crt")
+target = tvm.target.target.micro("host")
+
+with tvm.transform.PassContext(opt_level=3):
+ factory1 = tvm.relay.build(mod1, target, runtime=runtime, executor=executor, mod_name="mod1")
+ factory2 = tvm.relay.build(mod2, target, runtime=runtime, executor=executor, mod_name="mod2")
+```
+
+Then, we pass both results of `tvm.relay.build` to `export_model_library_format` function and path to generated model
+library format:
+```python
+micro.export_model_library_format([factory1, factory2], mlf_tar_path)
+```
+
+Now, if we extract MLF file here is the file structure:
+```bash
+# codegen source files
+codegen/
+ host/
+ src/
+ mod1_lib0.c
+ mod1_lib1.c
+ mod1_lib2.c
+ mod2_lib0.c
+ mod2_lib1.c
+ mod2_lib2.c
+
+# graph
+executor-config/
+ graph/
+ mod1.graph
+ mod2.graph
+
+# metadata file
+metadata.json
+
+# parameters
+parameters/
+ mod1.params
+ mod2.params
+
+# relay text output
+src/
+ mod1.relay
+ mod2.relay
+```
+
+# Drawbacks
+[drawbacks]: #drawbacks
+
+The drawback here is that we are changing generated metadata and file structure in the MLF file which means external
+tools which are dependent to this need to update their tool. Hopefully this RFC makes it clear on what steps they need
+to take.
+
+# **Rationale and alternatives**
+An alternative way to implement this feature is to break down each Relay build module to a subdirectory and keep the
+previous format inside each sub Relay build module. Using the example of `mod1` and `mod2`, in this approach we have
+an MLF file format with structure bellow if we extract:
+```bash
+# mod1
+mod1/
+ codegen/
+ host/
+ src/
+ mod_lib0.c
+ mod_lib1.c
+ mod_lib2.c
+ executor-config/
+ graph/
+ mod.graph
+ parameters/
+ mod.params
+ src/
+ mod.relay
+
+#mod2
+mod2/
+ codegen/
+ host/
+ src/
+ mod_lib0.c
+ mod_lib1.c
+ mod_lib2.c
+ executor-config/
+ graph/
+ mod.graph
+ parameters/
+ mod.params
+ src/
+ mod.relay
+```
+
+One of the benefits of this approach is that create a more readable file structure which is modularized for
Review Comment:
should be `is that is creates` I think. I changed it to that.
##########
rfcs/0076_mlf_with_multiple_modules.md:
##########
@@ -0,0 +1,165 @@
+- Feature Name: mlf_with_multiple_module_support
+- Start Date: 2022-05-31
+- RFC PR: [apache/tvm-rfcs#0075](https://github.com/apache/tvm-rfcs/pull/0075)
+- GitHub Issue: [apache/tvm#0000](https://github.com/apache/tvm/issues/0000)
+
+# Summary
+[summary]: #summary
+
+This RFC describes a new version of model library format where we can support adding multiple Relay module builds into a single MLF file.
+
+# Motivation
+[motivation]: #motivation
+
+MLF artifact is an important piece of building a microTVM project. In the first version of the MLF artifact,
+MLF only supports a single "Relay module"/"PackedFunc" build. However, there are cases where adding
+multiple Relay modules is required. For example, corstone300 tests using multiple Relay modules was introduced by
+contributors from ARM. The goal of this RFC is to create a new version of MLF which has standard support for these cases.
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+Users call `export_model_library_format` function with multiple Relay builds (output from `tvm.relay.build`) or
+single a Relay build module. We change the `export_model_library_format` API to support multiple Relay Build, however
+it is backward compatible and, we handle required changes internally.
+
+Since we are including multiple modules in a single MLF file, we need to make some changes. Let's assume we
+are want to export MLF file for two Relay build modules called `mod1` and `mod2`.
+
+1. We change format of `metadata.json` file. We will add `modules` as a key to JSON file which represents
+ multiple Relay build module. `modules` is a dictionary and each Relay build module is differentiated by
+ its module name. For example:
+ ```javascript
+ {
+ 'modules': {
+ 'mod1': {
+
+ },
+ 'mod2': {
+
+ },
+ ...
+ },
+ 'version': XXX,
+ }
+ ```
+ 2. There are multiple Relay text file generated per each Relay build. We propose to structure these files by using
+ module name and `.relay` extension to differentiate its format(E.g. {`mod1.relay`, `mod2.relay`, ...}). Similarly, for
Review Comment:
By old version if you mean MLF version 6, yes we are not changing that. However, for the new version we are changing `relay.txt` to `MOD_NAME.relay`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org