You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2020/03/11 03:40:46 UTC
[GitHub] [incubator-tvm] FrozenGene opened a new issue #5038: [RFC] Module
based Model Runtime Interface
FrozenGene opened a new issue #5038: [RFC] Module based Model Runtime Interface
URL: https://github.com/apache/incubator-tvm/issues/5038
As https://discuss.tvm.ai/t/discuss-module-based-model-runtime-interface/5025 discussed, we want to support Module based Model Runtime Interface and solve the following challenges:
- R1: The creation of ML model can be context dependent. For example, the user needs to be able to specify which GPU to run the graph runtime on.
- R2: We start to have multiple variations of model runtime, such as RelayVM. While it does not makes sense to force all model runtime to have the same set of APIs, it would be helpful to have a same mechanism for packaging and loading.
- R3: In advanced use cases, we want to be able to bundle multiple models into a single shared library.
After discussion, we have sorted out the API and reach an agreement. Here, I want to summary the API and give it an example.
```python
# lib is a GraphRuntimeFactoryModule
# that contains json and parameters
lib = relay.build(...)
# we could export it to shared library and load it back
# Here, we provide one option to let user control whether we
# want to package_params or not. The default value is true.
lib.export_library("resnet18.so", package_params=true)
# load it back
lib = tvm.module.load("resnet18.so")
# Call into the factory module to create a graph runtime
# Having this additional factory create step solves R1
# Note that parameters are already set
# The first argument is a key that helps to solve R3, allow list of context in the future
# gmod = lib["resnet18"]([tvm.cpu(0), tvm.gpu(0)])
gmod = lib["resnet18"](tvm.cpu(0))
set_input = gmod["set_input"]
run = gmod["run"]
get_output = gmod["get_output"]
# We do not need to set the parameters here
# as the models
set_input(data=my_data)
run()
get_output()
# we could use wrapper
gmod = tvm.graph_runtime.create(lib["resnet18"], tvm.cpu(0))
gmod.set_input(data=my_data)
gmod.run()
gmod.get_output()
```
More details and the decision procedure could be seen: https://discuss.tvm.ai/t/discuss-module-based-model-runtime-interface/5025
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] tqchen commented on issue #5038: [RFC] Module based
Model Runtime Interface
Posted by GitBox <gi...@apache.org>.
tqchen commented on issue #5038: [RFC] Module based Model Runtime Interface
URL: https://github.com/apache/incubator-tvm/issues/5038#issuecomment-599629252
Good summary, would be great if we can land it
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] FrozenGene commented on issue #5038: [RFC] Module based Model Runtime Interface
Posted by GitBox <gi...@apache.org>.
FrozenGene commented on issue #5038:
URL: https://github.com/apache/incubator-tvm/issues/5038#issuecomment-641354644
Draft pr: https://github.com/apache/incubator-tvm/pull/5753 There is still many things to do but will update it next in this pr.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] tqchen commented on issue #5038: [RFC] Module based Model Runtime Interface
Posted by GitBox <gi...@apache.org>.
tqchen commented on issue #5038:
URL: https://github.com/apache/incubator-tvm/issues/5038#issuecomment-636198783
@FrozenGene can we follow up on this?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] tqchen closed issue #5038: [RFC] Module based Model Runtime Interface
Posted by GitBox <gi...@apache.org>.
tqchen closed issue #5038:
URL: https://github.com/apache/incubator-tvm/issues/5038
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] FrozenGene commented on issue #5038: [RFC] Module based Model Runtime Interface
Posted by GitBox <gi...@apache.org>.
FrozenGene commented on issue #5038:
URL: https://github.com/apache/incubator-tvm/issues/5038#issuecomment-636294461
> @FrozenGene can we follow up on this?
Hi, @tqchen I will start to work on it from next Monday! Sorry for working on it lately because of other things.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] tqchen commented on issue #5038: [RFC] Module based Model Runtime Interface
Posted by GitBox <gi...@apache.org>.
tqchen commented on issue #5038:
URL: https://github.com/apache/incubator-tvm/issues/5038#issuecomment-661495470
Closing for now, #5753 lands the interface, let us open a new thread for the multi model support. Thanks to @FrozenGene !
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org