You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2022/09/15 10:03:50 UTC
[GitHub] [tvm] fPecc opened a new pull request, #12789: Added macro generation in MLF export
fPecc opened a new pull request, #12789:
URL: https://github.com/apache/tvm/pull/12789
For description see this [forum topic](https://discuss.tvm.apache.org/t/exporting-generated-functions-input-and-output-sizes-in-the-model-library-format/13532).
Tagged reviewers: @areusch (don't know who else can review microTVM-related code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] leandron commented on pull request #12789: Added macro generation in MLF export
Posted by GitBox <gi...@apache.org>.
leandron commented on PR #12789:
URL: https://github.com/apache/tvm/pull/12789#issuecomment-1248338192
@fPecc from what I understood from reading your Discuss post, this is a non-breaking change, right?
Apart from that, I think the problem is quite restrict and it could've been summarised/discussed in the PR, rather than in a separate location, but that's just a minor suggestion.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] areusch commented on a diff in pull request #12789: Added macro generation in MLF export
Posted by GitBox <gi...@apache.org>.
areusch commented on code in PR #12789:
URL: https://github.com/apache/tvm/pull/12789#discussion_r972463776
##########
src/target/source/interface_c.cc:
##########
@@ -46,20 +46,46 @@ class InterfaceCNode : public runtime::ModuleNode {
InterfaceCNode(std::string module_name, Array<String> inputs, Array<String> outputs,
Array<tir::usmp::AllocatedPoolInfo> pools,
Map<String, tir::usmp::PoolAllocation> io_pool_allocations, Array<String> devices,
- int workspace_size)
+ int workspace_size, Map<String, IntImm> input_sizes,
+ Map<String, IntImm> output_sizes)
: module_name_(module_name),
inputs_(inputs),
outputs_(outputs),
devices_(devices),
pools_(FilterExternalPools(pools)),
io_pool_allocations_(io_pool_allocations),
- workspace_size_(workspace_size) {}
+ workspace_size_(workspace_size),
+ input_sizes_(input_sizes),
+ output_sizes_(output_sizes) {}
const char* type_key() const final { return "h"; }
std::string GetSource(const std::string& format) final {
std::stringstream code;
EmitUpperHeaderGuard(code);
+
+ // Emit macros for input sizes
+ for (auto const& it : input_sizes_) {
+ auto input_name = std::string(it.first);
+ std::replace(input_name.begin(), input_name.end(), ':', '_');
Review Comment:
can you consolidate this logic so we can reuse it?
##########
src/target/source/interface_c.cc:
##########
@@ -46,20 +46,46 @@ class InterfaceCNode : public runtime::ModuleNode {
InterfaceCNode(std::string module_name, Array<String> inputs, Array<String> outputs,
Array<tir::usmp::AllocatedPoolInfo> pools,
Map<String, tir::usmp::PoolAllocation> io_pool_allocations, Array<String> devices,
- int workspace_size)
+ int workspace_size, Map<String, IntImm> input_sizes,
+ Map<String, IntImm> output_sizes)
: module_name_(module_name),
inputs_(inputs),
outputs_(outputs),
devices_(devices),
pools_(FilterExternalPools(pools)),
io_pool_allocations_(io_pool_allocations),
- workspace_size_(workspace_size) {}
+ workspace_size_(workspace_size),
+ input_sizes_(input_sizes),
+ output_sizes_(output_sizes) {}
const char* type_key() const final { return "h"; }
std::string GetSource(const std::string& format) final {
std::stringstream code;
EmitUpperHeaderGuard(code);
+
+ // Emit macros for input sizes
+ for (auto const& it : input_sizes_) {
+ auto input_name = std::string(it.first);
+ std::replace(input_name.begin(), input_name.end(), ':', '_');
+ std::replace(input_name.begin(), input_name.end(), '+', '_');
+ std::replace(input_name.begin(), input_name.end(), '-', '_');
+ auto input_size = it.second->value;
+ EmitIntegerValueMacro(code, "Input tensor " + input_name + " size (in bytes)", input_name,
Review Comment:
can we include `_SIZE` in the macro name like the other postfixes? also, we might want to consider this as a prefix after "tvmgen_modname_" to avoid potential collision (if for some reason a layer function has the same name).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] fPecc commented on a diff in pull request #12789: Added macro generation in MLF export
Posted by GitBox <gi...@apache.org>.
fPecc commented on code in PR #12789:
URL: https://github.com/apache/tvm/pull/12789#discussion_r972978251
##########
python/tvm/micro/model_library_format.py:
##########
@@ -519,7 +570,7 @@ def _export_operator_model_library_format(mod: build_module.OperatorModule, temp
"""
targets = []
for target in mod.ir_module_by_target.keys():
- if str(target.kind) not in ("llvm", "c"):
+ if str(target.kind) not in ("llvm", "c", "gemmini"):
Review Comment:
added a new commit fixing this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] Mousius commented on pull request #12789: Added macro generation in MLF export
Posted by GitBox <gi...@apache.org>.
Mousius commented on PR #12789:
URL: https://github.com/apache/tvm/pull/12789#issuecomment-1340813362
Thanks for working through this one @fPecc! This is an awesome improvement and it has now landed 👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] Mousius commented on a diff in pull request #12789: Added macro generation in MLF export
Posted by GitBox <gi...@apache.org>.
Mousius commented on code in PR #12789:
URL: https://github.com/apache/tvm/pull/12789#discussion_r1029766343
##########
python/tvm/micro/model_library_format.py:
##########
@@ -277,6 +301,32 @@ def _create_empty_entry(target_device_type):
main_func_metadata.io_sizes[target]
)
+ # Now, we also add the information about the size of each input and output of the main
+ # function (in bytes)
+ input_dict = {}
+ for input_param in main_func_metadata.relay_primfuncs[target].params:
+ input_dict[input_param.name_hint] = _create_type_metadata(input_param.checked_type)
+ target_main_entries[int(target.get_target_device_type())]["inputs"] = input_dict
+
+ output_dict = {}
+ # For output, we dont have the name of the output, so we enumerate them
+ if isinstance(main_func_metadata.relay_primfuncs[target].ret_type, tvm.ir.type.TupleType):
+ output_list = _convert_tuple_to_outputs(
+ main_func_metadata.relay_primfuncs[target].ret_type
+ )
+ for i, output_type in enumerate(output_list):
+ if hasattr(output_type, "shape"):
+ output_dict[f"output{i}"] = _create_type_metadata(output_type)
+ else:
+ output_dict[f"output{i}"] = {"size": 0, "dtype": ""}
+ else:
+ output_type = main_func_metadata.relay_primfuncs[target].ret_type
+ if hasattr(output_type, "shape"):
+ output_dict["output"] = _create_type_metadata(output_type)
+ else:
+ output_dict["output"] = {"size": 0, "dtype": ""}
Review Comment:
Is there a case where this happens? I can't see it in the test cases
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] fPecc commented on pull request #12789: Added macro generation in MLF export
Posted by GitBox <gi...@apache.org>.
fPecc commented on PR #12789:
URL: https://github.com/apache/tvm/pull/12789#issuecomment-1248965397
Thanks for the feedback @areusch . Indeed, it shouldn't break, but there are still some tests that are not passing, so I am trying to fix that locally before continuing and commiting again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] fPecc commented on a diff in pull request #12789: Added macro generation in MLF export
Posted by GitBox <gi...@apache.org>.
fPecc commented on code in PR #12789:
URL: https://github.com/apache/tvm/pull/12789#discussion_r972943502
##########
python/tvm/micro/model_library_format.py:
##########
@@ -519,7 +570,7 @@ def _export_operator_model_library_format(mod: build_module.OperatorModule, temp
"""
targets = []
for target in mod.ir_module_by_target.keys():
- if str(target.kind) not in ("llvm", "c"):
+ if str(target.kind) not in ("llvm", "c", "gemmini"):
Review Comment:
Yeah, sorry for that, will change it back.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] Mousius merged pull request #12789: Added macro generation in MLF export
Posted by GitBox <gi...@apache.org>.
Mousius merged PR #12789:
URL: https://github.com/apache/tvm/pull/12789
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] fPecc commented on a diff in pull request #12789: Added macro generation in MLF export
Posted by GitBox <gi...@apache.org>.
fPecc commented on code in PR #12789:
URL: https://github.com/apache/tvm/pull/12789#discussion_r972979558
##########
src/target/source/interface_c.cc:
##########
@@ -46,20 +46,46 @@ class InterfaceCNode : public runtime::ModuleNode {
InterfaceCNode(std::string module_name, Array<String> inputs, Array<String> outputs,
Array<tir::usmp::AllocatedPoolInfo> pools,
Map<String, tir::usmp::PoolAllocation> io_pool_allocations, Array<String> devices,
- int workspace_size)
+ int workspace_size, Map<String, IntImm> input_sizes,
+ Map<String, IntImm> output_sizes)
: module_name_(module_name),
inputs_(inputs),
outputs_(outputs),
devices_(devices),
pools_(FilterExternalPools(pools)),
io_pool_allocations_(io_pool_allocations),
- workspace_size_(workspace_size) {}
+ workspace_size_(workspace_size),
+ input_sizes_(input_sizes),
+ output_sizes_(output_sizes) {}
const char* type_key() const final { return "h"; }
std::string GetSource(const std::string& format) final {
std::stringstream code;
EmitUpperHeaderGuard(code);
+
+ // Emit macros for input sizes
+ for (auto const& it : input_sizes_) {
+ auto input_name = std::string(it.first);
+ std::replace(input_name.begin(), input_name.end(), ':', '_');
Review Comment:
Added the usage of SanitizeName in order to keep a uniform "sanitization"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] Mousius commented on pull request #12789: Added macro generation in MLF export
Posted by GitBox <gi...@apache.org>.
Mousius commented on PR #12789:
URL: https://github.com/apache/tvm/pull/12789#issuecomment-1326431202
@fPecc there's some new conflicts, can you rebase and push again? might also fix CI 😸
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] mehrdadh commented on pull request #12789: Added macro generation in MLF export
Posted by GitBox <gi...@apache.org>.
mehrdadh commented on PR #12789:
URL: https://github.com/apache/tvm/pull/12789#issuecomment-1307578807
@fPecc Friendly reminder about this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] fPecc commented on a diff in pull request #12789: Added macro generation in MLF export
Posted by GitBox <gi...@apache.org>.
fPecc commented on code in PR #12789:
URL: https://github.com/apache/tvm/pull/12789#discussion_r1029073426
##########
python/tvm/micro/model_library_format.py:
##########
@@ -277,6 +294,37 @@ def _create_empty_entry(target_device_type):
main_func_metadata.io_sizes[target]
)
+ # Now, we also add the information about the size of each input and output of the main
+ # function (in bytes)
+ input_dict = {}
+ for input_param in main_func_metadata.relay_primfuncs[target].params:
+ if hasattr(input_param, "checked_type"):
+ input_dict[input_param.name_hint] = int(
+ _shape_to_size(input_param.checked_type.shape, input_param.checked_type.dtype)
+ )
+ else:
+ # TODO: maybe fill checked_type here?
+ input_dict[input_param.name_hint] = 0
+ target_main_entries[int(target.kind.device_type)]["inputs"] = input_dict
+
+ output_dict = {}
+ # For output, we dont have the name of the output, so we enumerate them
+ if isinstance(main_func_metadata.relay_primfuncs[target].ret_type, tvm.ir.type.TupleType):
Review Comment:
Applied the proposed change
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] areusch commented on pull request #12789: Added macro generation in MLF export
Posted by GitBox <gi...@apache.org>.
areusch commented on PR #12789:
URL: https://github.com/apache/tvm/pull/12789#issuecomment-1281022346
@Mousius could you take another look here?
@fPecc do you mind looking at the test failures? i think there are a few that need to get updated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] Mousius commented on a diff in pull request #12789: Added macro generation in MLF export
Posted by GitBox <gi...@apache.org>.
Mousius commented on code in PR #12789:
URL: https://github.com/apache/tvm/pull/12789#discussion_r999171602
##########
python/tvm/micro/model_library_format.py:
##########
@@ -277,6 +294,37 @@ def _create_empty_entry(target_device_type):
main_func_metadata.io_sizes[target]
)
+ # Now, we also add the information about the size of each input and output of the main
+ # function (in bytes)
+ input_dict = {}
+ for input_param in main_func_metadata.relay_primfuncs[target].params:
+ if hasattr(input_param, "checked_type"):
+ input_dict[input_param.name_hint] = int(
+ _shape_to_size(input_param.checked_type.shape, input_param.checked_type.dtype)
+ )
+ else:
+ # TODO: maybe fill checked_type here?
+ input_dict[input_param.name_hint] = 0
+ target_main_entries[int(target.kind.device_type)]["inputs"] = input_dict
+
+ output_dict = {}
+ # For output, we dont have the name of the output, so we enumerate them
+ if isinstance(main_func_metadata.relay_primfuncs[target].ret_type, tvm.ir.type.TupleType):
+ for i, output_type in enumerate(
+ main_func_metadata.relay_primfuncs[target].ret_type.fields
+ ):
+ if hasattr(output_type, "shape"):
+ output_dict[i] = int(_shape_to_size(output_type.shape, output_type.dtype))
+ else:
+ output_dict[i] = 0
+ else:
+ output_type = main_func_metadata.relay_primfuncs[target].ret_type
+ if hasattr(output_type, "shape"):
+ output_dict[0] = int(_shape_to_size(output_type.shape, output_type.dtype))
+ else:
+ output_dict[0] = 0
Review Comment:
This is duplicated between input and output, we can replace it with a function:
```python
def _create_type_metadata(input_type):
return {
"size": int(
_shape_to_size(input_type.shape, input_type.dtype)
),
"dtype": str(input_type.dtype)
}
```
##########
python/tvm/micro/model_library_format.py:
##########
@@ -277,6 +294,37 @@ def _create_empty_entry(target_device_type):
main_func_metadata.io_sizes[target]
)
+ # Now, we also add the information about the size of each input and output of the main
+ # function (in bytes)
+ input_dict = {}
+ for input_param in main_func_metadata.relay_primfuncs[target].params:
+ if hasattr(input_param, "checked_type"):
+ input_dict[input_param.name_hint] = int(
+ _shape_to_size(input_param.checked_type.shape, input_param.checked_type.dtype)
+ )
+ else:
+ # TODO: maybe fill checked_type here?
+ input_dict[input_param.name_hint] = 0
+ target_main_entries[int(target.kind.device_type)]["inputs"] = input_dict
+
+ output_dict = {}
+ # For output, we dont have the name of the output, so we enumerate them
+ if isinstance(main_func_metadata.relay_primfuncs[target].ret_type, tvm.ir.type.TupleType):
Review Comment:
Given tuples could be more heavily nested than this, we should use some recursive function to generate the outputs? We can re-purpose this function (which is no longer used): https://github.com/apache/tvm/blob/main/python/tvm/micro/model_library_format.py#L293
##########
tests/python/unittest/test_micro_model_library_format.py:
##########
@@ -208,7 +208,12 @@ def @main(%a : Tensor[(1, 2), uint8], %b : Tensor[(1, 2), float32], %c : Tensor[
{
"constants_size_bytes": json_constants_size_bytes,
"device": 1,
+ "inputs": {
+ "a": 2,
+ "b": 8,
+ },
Review Comment:
For future proofing, it'd be good to have more than just the size? But also dtype here?
```suggestion
"inputs": {
"a": {"dtype": "uint8", "size_in_bytes": 2},
"b": {"dtype": "float32", "size_in_bytes": 8},
},
```
##########
python/tvm/micro/model_library_format.py:
##########
@@ -277,6 +294,37 @@ def _create_empty_entry(target_device_type):
main_func_metadata.io_sizes[target]
)
+ # Now, we also add the information about the size of each input and output of the main
+ # function (in bytes)
+ input_dict = {}
+ for input_param in main_func_metadata.relay_primfuncs[target].params:
+ if hasattr(input_param, "checked_type"):
Review Comment:
I don't see a test case for when this is not set? How do we reproduce it?
##########
tests/python/unittest/test_micro_model_library_format.py:
##########
@@ -208,7 +208,12 @@ def @main(%a : Tensor[(1, 2), uint8], %b : Tensor[(1, 2), float32], %c : Tensor[
{
"constants_size_bytes": json_constants_size_bytes,
"device": 1,
+ "inputs": {
+ "a": 2,
+ "b": 8,
+ },
"io_size_bytes": 18,
+ "outputs": {"0": 8},
Review Comment:
This should match with the name of the output in the generated C (`output` for single, `outputX` for multiple, or the actual name if we can get to it)
```suggestion
"outputs": {"output": 8},
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] fPecc commented on pull request #12789: Added macro generation in MLF export
Posted by GitBox <gi...@apache.org>.
fPecc commented on PR #12789:
URL: https://github.com/apache/tvm/pull/12789#issuecomment-1339112728
Hi @Mousius ,
I did the rebase, but I noticed there are still tests failing in CPU MINIMAL. They seem to be related to the CRT, but I cant find a direct relation with this change
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] fPecc commented on a diff in pull request #12789: Added macro generation in MLF export
Posted by GitBox <gi...@apache.org>.
fPecc commented on code in PR #12789:
URL: https://github.com/apache/tvm/pull/12789#discussion_r1029649977
##########
python/tvm/micro/model_library_format.py:
##########
@@ -277,6 +294,37 @@ def _create_empty_entry(target_device_type):
main_func_metadata.io_sizes[target]
)
+ # Now, we also add the information about the size of each input and output of the main
+ # function (in bytes)
+ input_dict = {}
+ for input_param in main_func_metadata.relay_primfuncs[target].params:
+ if hasattr(input_param, "checked_type"):
Review Comment:
Hi @Mousius . I reviewed the code and noticed that the parameters of the primfuncs in the main_func_metadata always seem to have the checked_type set, so I removed the check here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] fPecc commented on pull request #12789: Added macro generation in MLF export
Posted by GitBox <gi...@apache.org>.
fPecc commented on PR #12789:
URL: https://github.com/apache/tvm/pull/12789#issuecomment-1324034366
@mehrdadh sorry for the delay, there it is, all tests fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] fPecc commented on a diff in pull request #12789: Added macro generation in MLF export
Posted by GitBox <gi...@apache.org>.
fPecc commented on code in PR #12789:
URL: https://github.com/apache/tvm/pull/12789#discussion_r972978640
##########
src/target/source/interface_c.cc:
##########
@@ -46,20 +46,46 @@ class InterfaceCNode : public runtime::ModuleNode {
InterfaceCNode(std::string module_name, Array<String> inputs, Array<String> outputs,
Array<tir::usmp::AllocatedPoolInfo> pools,
Map<String, tir::usmp::PoolAllocation> io_pool_allocations, Array<String> devices,
- int workspace_size)
+ int workspace_size, Map<String, IntImm> input_sizes,
+ Map<String, IntImm> output_sizes)
: module_name_(module_name),
inputs_(inputs),
outputs_(outputs),
devices_(devices),
pools_(FilterExternalPools(pools)),
io_pool_allocations_(io_pool_allocations),
- workspace_size_(workspace_size) {}
+ workspace_size_(workspace_size),
+ input_sizes_(input_sizes),
+ output_sizes_(output_sizes) {}
const char* type_key() const final { return "h"; }
std::string GetSource(const std::string& format) final {
std::stringstream code;
EmitUpperHeaderGuard(code);
+
+ // Emit macros for input sizes
+ for (auto const& it : input_sizes_) {
+ auto input_name = std::string(it.first);
+ std::replace(input_name.begin(), input_name.end(), ':', '_');
+ std::replace(input_name.begin(), input_name.end(), '+', '_');
+ std::replace(input_name.begin(), input_name.end(), '-', '_');
+ auto input_size = it.second->value;
+ EmitIntegerValueMacro(code, "Input tensor " + input_name + " size (in bytes)", input_name,
Review Comment:
just added it in a new commit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] leandron commented on pull request #12789: Added macro generation in MLF export
Posted by GitBox <gi...@apache.org>.
leandron commented on PR #12789:
URL: https://github.com/apache/tvm/pull/12789#issuecomment-1248042532
cc @Mousius @ashutosh-arm @grant-arm
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] fPecc commented on a diff in pull request #12789: Added macro generation in MLF export
Posted by GitBox <gi...@apache.org>.
fPecc commented on code in PR #12789:
URL: https://github.com/apache/tvm/pull/12789#discussion_r1029073124
##########
python/tvm/micro/model_library_format.py:
##########
@@ -277,6 +294,37 @@ def _create_empty_entry(target_device_type):
main_func_metadata.io_sizes[target]
)
+ # Now, we also add the information about the size of each input and output of the main
+ # function (in bytes)
+ input_dict = {}
+ for input_param in main_func_metadata.relay_primfuncs[target].params:
+ if hasattr(input_param, "checked_type"):
+ input_dict[input_param.name_hint] = int(
+ _shape_to_size(input_param.checked_type.shape, input_param.checked_type.dtype)
+ )
+ else:
+ # TODO: maybe fill checked_type here?
+ input_dict[input_param.name_hint] = 0
+ target_main_entries[int(target.kind.device_type)]["inputs"] = input_dict
+
+ output_dict = {}
+ # For output, we dont have the name of the output, so we enumerate them
+ if isinstance(main_func_metadata.relay_primfuncs[target].ret_type, tvm.ir.type.TupleType):
+ for i, output_type in enumerate(
+ main_func_metadata.relay_primfuncs[target].ret_type.fields
+ ):
+ if hasattr(output_type, "shape"):
+ output_dict[i] = int(_shape_to_size(output_type.shape, output_type.dtype))
+ else:
+ output_dict[i] = 0
+ else:
+ output_type = main_func_metadata.relay_primfuncs[target].ret_type
+ if hasattr(output_type, "shape"):
+ output_dict[0] = int(_shape_to_size(output_type.shape, output_type.dtype))
+ else:
+ output_dict[0] = 0
Review Comment:
Applied the proposed change
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] guberti commented on pull request #12789: Added macro generation in MLF export
Posted by GitBox <gi...@apache.org>.
guberti commented on PR #12789:
URL: https://github.com/apache/tvm/pull/12789#issuecomment-1339193064
@fPecc Are you sure you did a rebase, not a merge? I suspect `git rebase -i main` will fix the CRT tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] Mousius commented on a diff in pull request #12789: Added macro generation in MLF export
Posted by GitBox <gi...@apache.org>.
Mousius commented on code in PR #12789:
URL: https://github.com/apache/tvm/pull/12789#discussion_r972909757
##########
src/target/source/interface_c.cc:
##########
@@ -46,20 +46,46 @@ class InterfaceCNode : public runtime::ModuleNode {
InterfaceCNode(std::string module_name, Array<String> inputs, Array<String> outputs,
Array<tir::usmp::AllocatedPoolInfo> pools,
Map<String, tir::usmp::PoolAllocation> io_pool_allocations, Array<String> devices,
- int workspace_size)
+ int workspace_size, Map<String, IntImm> input_sizes,
+ Map<String, IntImm> output_sizes)
: module_name_(module_name),
inputs_(inputs),
outputs_(outputs),
devices_(devices),
pools_(FilterExternalPools(pools)),
io_pool_allocations_(io_pool_allocations),
- workspace_size_(workspace_size) {}
+ workspace_size_(workspace_size),
+ input_sizes_(input_sizes),
+ output_sizes_(output_sizes) {}
const char* type_key() const final { return "h"; }
std::string GetSource(const std::string& format) final {
std::stringstream code;
EmitUpperHeaderGuard(code);
+
+ // Emit macros for input sizes
+ for (auto const& it : input_sizes_) {
+ auto input_name = std::string(it.first);
+ std::replace(input_name.begin(), input_name.end(), ':', '_');
+ std::replace(input_name.begin(), input_name.end(), '+', '_');
+ std::replace(input_name.begin(), input_name.end(), '-', '_');
Review Comment:
We can use the existing name sanitisation logic here:
```suggestion
std::string input_name = SanitizeName(it.first);
```
##########
python/tvm/micro/model_library_format.py:
##########
@@ -519,7 +570,7 @@ def _export_operator_model_library_format(mod: build_module.OperatorModule, temp
"""
targets = []
for target in mod.ir_module_by_target.keys():
- if str(target.kind) not in ("llvm", "c"):
+ if str(target.kind) not in ("llvm", "c", "gemmini"):
Review Comment:
This seems like an unrelated change?
##########
src/target/source/interface_c.cc:
##########
@@ -46,20 +46,46 @@ class InterfaceCNode : public runtime::ModuleNode {
InterfaceCNode(std::string module_name, Array<String> inputs, Array<String> outputs,
Array<tir::usmp::AllocatedPoolInfo> pools,
Map<String, tir::usmp::PoolAllocation> io_pool_allocations, Array<String> devices,
- int workspace_size)
+ int workspace_size, Map<String, IntImm> input_sizes,
+ Map<String, IntImm> output_sizes)
: module_name_(module_name),
inputs_(inputs),
outputs_(outputs),
devices_(devices),
pools_(FilterExternalPools(pools)),
io_pool_allocations_(io_pool_allocations),
- workspace_size_(workspace_size) {}
+ workspace_size_(workspace_size),
+ input_sizes_(input_sizes),
+ output_sizes_(output_sizes) {}
const char* type_key() const final { return "h"; }
std::string GetSource(const std::string& format) final {
std::stringstream code;
EmitUpperHeaderGuard(code);
+
+ // Emit macros for input sizes
+ for (auto const& it : input_sizes_) {
+ auto input_name = std::string(it.first);
+ std::replace(input_name.begin(), input_name.end(), ':', '_');
+ std::replace(input_name.begin(), input_name.end(), '+', '_');
+ std::replace(input_name.begin(), input_name.end(), '-', '_');
+ auto input_size = it.second->value;
Review Comment:
Where possible, we should avoid `auto` types:
```suggestion
int input_size = it.second->value;
```
##########
src/target/source/interface_c.cc:
##########
@@ -46,20 +46,46 @@ class InterfaceCNode : public runtime::ModuleNode {
InterfaceCNode(std::string module_name, Array<String> inputs, Array<String> outputs,
Array<tir::usmp::AllocatedPoolInfo> pools,
Map<String, tir::usmp::PoolAllocation> io_pool_allocations, Array<String> devices,
- int workspace_size)
+ int workspace_size, Map<String, IntImm> input_sizes,
+ Map<String, IntImm> output_sizes)
: module_name_(module_name),
inputs_(inputs),
outputs_(outputs),
devices_(devices),
pools_(FilterExternalPools(pools)),
io_pool_allocations_(io_pool_allocations),
- workspace_size_(workspace_size) {}
+ workspace_size_(workspace_size),
+ input_sizes_(input_sizes),
+ output_sizes_(output_sizes) {}
const char* type_key() const final { return "h"; }
std::string GetSource(const std::string& format) final {
std::stringstream code;
EmitUpperHeaderGuard(code);
+
+ // Emit macros for input sizes
+ for (auto const& it : input_sizes_) {
+ auto input_name = std::string(it.first);
+ std::replace(input_name.begin(), input_name.end(), ':', '_');
+ std::replace(input_name.begin(), input_name.end(), '+', '_');
+ std::replace(input_name.begin(), input_name.end(), '-', '_');
+ auto input_size = it.second->value;
+ EmitIntegerValueMacro(code, "Input tensor " + input_name + " size (in bytes)", input_name,
+ input_size);
+ }
+
+ // Emit macros for output sizes
+ for (auto const& it : output_sizes_) {
+ auto output_name = std::string(it.first);
+ std::replace(output_name.begin(), output_name.end(), ':', '_');
+ std::replace(output_name.begin(), output_name.end(), '+', '_');
+ std::replace(output_name.begin(), output_name.end(), '-', '_');
+ auto output_size = it.second->value;
Review Comment:
```suggestion
std::string output_name = SanitizeName(it.first);
int output_size = it.second->value;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] fPecc commented on a diff in pull request #12789: Added macro generation in MLF export
Posted by GitBox <gi...@apache.org>.
fPecc commented on code in PR #12789:
URL: https://github.com/apache/tvm/pull/12789#discussion_r1029074332
##########
tests/python/unittest/test_micro_model_library_format.py:
##########
@@ -208,7 +208,12 @@ def @main(%a : Tensor[(1, 2), uint8], %b : Tensor[(1, 2), float32], %c : Tensor[
{
"constants_size_bytes": json_constants_size_bytes,
"device": 1,
+ "inputs": {
+ "a": 2,
+ "b": 8,
+ },
Review Comment:
Added the check for the dtype
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tvm-bot commented on pull request #12789: Added macro generation in MLF export
Posted by GitBox <gi...@apache.org>.
tvm-bot commented on PR #12789:
URL: https://github.com/apache/tvm/pull/12789#issuecomment-1324030637
<!---bot-comment-->
Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from [Reviewers](https://github.com/apache/incubator-tvm/blob/master/CONTRIBUTORS.md#reviewers) by @-ing them in a comment.
<!--bot-comment-docs-start-->
* Built docs for commit 97f0b5ddd15fdb857a6f699d4ffe606accd82795 can be found [here](https://pr-docs.tlcpack.ai/PR-12789/16/docs/index.html).<!--bot-comment-docs-end-->
<sub>Generated by [tvm-bot](https://github.com/apache/tvm/blob/main/ci/README.md#github-actions)</sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] fPecc commented on a diff in pull request #12789: Added macro generation in MLF export
Posted by GitBox <gi...@apache.org>.
fPecc commented on code in PR #12789:
URL: https://github.com/apache/tvm/pull/12789#discussion_r1030138323
##########
python/tvm/micro/model_library_format.py:
##########
@@ -277,6 +301,32 @@ def _create_empty_entry(target_device_type):
main_func_metadata.io_sizes[target]
)
+ # Now, we also add the information about the size of each input and output of the main
+ # function (in bytes)
+ input_dict = {}
+ for input_param in main_func_metadata.relay_primfuncs[target].params:
+ input_dict[input_param.name_hint] = _create_type_metadata(input_param.checked_type)
+ target_main_entries[int(target.get_target_device_type())]["inputs"] = input_dict
+
+ output_dict = {}
+ # For output, we dont have the name of the output, so we enumerate them
+ if isinstance(main_func_metadata.relay_primfuncs[target].ret_type, tvm.ir.type.TupleType):
+ output_list = _convert_tuple_to_outputs(
+ main_func_metadata.relay_primfuncs[target].ret_type
+ )
+ for i, output_type in enumerate(output_list):
+ if hasattr(output_type, "shape"):
+ output_dict[f"output{i}"] = _create_type_metadata(output_type)
+ else:
+ output_dict[f"output{i}"] = {"size": 0, "dtype": ""}
+ else:
+ output_type = main_func_metadata.relay_primfuncs[target].ret_type
+ if hasattr(output_type, "shape"):
+ output_dict["output"] = _create_type_metadata(output_type)
+ else:
+ output_dict["output"] = {"size": 0, "dtype": ""}
Review Comment:
You are right, there is no case this happens. I removed the check because it was redundant
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org