You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2021/10/27 11:02:11 UTC
[GitHub] [tvm] colinyl opened a new pull request #9380: [ONNX][Converter] Add dynamic nodes support
colinyl opened a new pull request #9380:
URL: https://github.com/apache/tvm/pull/9380
* Added dynamic nodes support for ONNX converter if a Relay model contains such.
* Fix the inconsistent behavior of `relay.create_executor(kind, device, target).evaluate(func)(args)`, kind is either **graph** or **vm**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi merged pull request #9380: [ONNX][Converter] Add dynamic nodes support
Posted by GitBox <gi...@apache.org>.
masahi merged pull request #9380:
URL: https://github.com/apache/tvm/pull/9380
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] leeexyz commented on a change in pull request #9380: [ONNX][Converter] Add dynamic nodes support
Posted by GitBox <gi...@apache.org>.
leeexyz commented on a change in pull request #9380:
URL: https://github.com/apache/tvm/pull/9380#discussion_r741584173
##########
File path: python/tvm/relay/backend/vm.py
##########
@@ -275,14 +275,18 @@ def __init__(self, mod, device, target):
self.mod = mod
self.device = device
self.target = target
- self.executable = compile(mod, target)
- self.vm = vm_rt.VirtualMachine(self.executable, device)
+ self.executable = None
+ self.vm = None
def _make_executor(self, expr=None):
- main = self.mod["main"]
+ if expr:
+ self.mod["main"] = expr
+
+ self.executable = compile(self.mod, self.target)
+ self.vm = vm_rt.VirtualMachine(self.executable, self.device)
def _vm_wrapper(*args, **kwargs):
- args = self._convert_args(main, args, kwargs)
+ args = self._convert_args(self.mod["main"], args, kwargs)
Review comment:
Thanks for your reply. It is because of this usage `relay.create_executor(kind, device, target).evaluate(func)(args)` which is inconsistent with graph. If kind is `vm`, an error will be prompted
```
Traceback (most recent call last):
File "tests/python/contrib/test_onnx.py", line 752, in <module>
test_dyn()
File "tests/python/contrib/test_onnx.py", line 718, in test_dyn
verify_dyn_bcast((1, 3, 32, 1), (1, 3, 1, 3), "float32")
File "tests/python/contrib/test_onnx.py", line 715, in verify_dyn_bcast
func, [lhs_data, rhs_data], "test_dyn_bcast", rtol=1e-5, atol=1e-5, is_dyn=True
File "tests/python/contrib/test_onnx.py", line 65, in verify_results
relay_results = run_relay(relay_func, indata, is_dyn)
File "tests/python/contrib/test_onnx.py", line 54, in run_relay
relay_res = relay.create_executor(kind, device=dev, target=target).evaluate(func)(*data_tuple)
File "/home/tuisku/project/apache/tvm/python/tvm/relay/build_module.py", line 575, in create_executor
return VMExecutor(mod, device, target)
File "/home/tuisku/project/apache/tvm/python/tvm/relay/backend/vm.py", line 278, in __init__
self.executable = compile(mod, target)
File "/home/tuisku/project/apache/tvm/python/tvm/relay/backend/vm.py", line 72, in compile
compiler.lower(mod, target)
File "/home/tuisku/project/apache/tvm/python/tvm/relay/backend/vm.py", line 143, in lower
self._lower(mod, target, target_host)
......
File "../src/ir/module.cc", line 143
ValueError: Cannot find global var "main" in the Module
candidates are: []
```
Because there is no `main` in this Module, it cannot be evaluated. For graph executor, it has https://github.com/apache/tvm/blob/main/python/tvm/relay/build_module.py#L476-L477.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi merged pull request #9380: [ONNX][Converter] Add dynamic nodes support
Posted by GitBox <gi...@apache.org>.
masahi merged pull request #9380:
URL: https://github.com/apache/tvm/pull/9380
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] leeexyz commented on a change in pull request #9380: [ONNX][Converter] Add dynamic nodes support
Posted by GitBox <gi...@apache.org>.
leeexyz commented on a change in pull request #9380:
URL: https://github.com/apache/tvm/pull/9380#discussion_r741584173
##########
File path: python/tvm/relay/backend/vm.py
##########
@@ -275,14 +275,18 @@ def __init__(self, mod, device, target):
self.mod = mod
self.device = device
self.target = target
- self.executable = compile(mod, target)
- self.vm = vm_rt.VirtualMachine(self.executable, device)
+ self.executable = None
+ self.vm = None
def _make_executor(self, expr=None):
- main = self.mod["main"]
+ if expr:
+ self.mod["main"] = expr
+
+ self.executable = compile(self.mod, self.target)
+ self.vm = vm_rt.VirtualMachine(self.executable, self.device)
def _vm_wrapper(*args, **kwargs):
- args = self._convert_args(main, args, kwargs)
+ args = self._convert_args(self.mod["main"], args, kwargs)
Review comment:
Thanks for your reply. It is because of this usage `relay.create_executor(kind, device, target).evaluate(func)(args)` which is inconsistent with graph. If kind is `vm`, an error will be prompted
```
Traceback (most recent call last):
File "tests/python/contrib/test_onnx.py", line 752, in <module>
test_dyn()
File "tests/python/contrib/test_onnx.py", line 718, in test_dyn
verify_dyn_bcast((1, 3, 32, 1), (1, 3, 1, 3), "float32")
File "tests/python/contrib/test_onnx.py", line 715, in verify_dyn_bcast
func, [lhs_data, rhs_data], "test_dyn_bcast", rtol=1e-5, atol=1e-5, is_dyn=True
File "tests/python/contrib/test_onnx.py", line 65, in verify_results
relay_results = run_relay(relay_func, indata, is_dyn)
File "tests/python/contrib/test_onnx.py", line 54, in run_relay
relay_res = relay.create_executor(kind, device=dev, target=target).evaluate(func)(*data_tuple)
File "/home/tuisku/project/apache/tvm/python/tvm/relay/build_module.py", line 575, in create_executor
return VMExecutor(mod, device, target)
File "/home/tuisku/project/apache/tvm/python/tvm/relay/backend/vm.py", line 278, in __init__
self.executable = compile(mod, target)
File "/home/tuisku/project/apache/tvm/python/tvm/relay/backend/vm.py", line 72, in compile
compiler.lower(mod, target)
File "/home/tuisku/project/apache/tvm/python/tvm/relay/backend/vm.py", line 143, in lower
self._lower(mod, target, target_host)
......
File "../src/ir/module.cc", line 143
ValueError: Cannot find global var "main" in the Module
candidates are: []
```
Because there is no `main` in this Module, it cannot be evaluated. For graph executor, it has https://github.com/apache/tvm/blob/main/python/tvm/relay/build_module.py#L476-L477.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on a change in pull request #9380: [ONNX][Converter] Add dynamic nodes support
Posted by GitBox <gi...@apache.org>.
masahi commented on a change in pull request #9380:
URL: https://github.com/apache/tvm/pull/9380#discussion_r741475626
##########
File path: python/tvm/relay/backend/vm.py
##########
@@ -275,14 +275,18 @@ def __init__(self, mod, device, target):
self.mod = mod
self.device = device
self.target = target
- self.executable = compile(mod, target)
- self.vm = vm_rt.VirtualMachine(self.executable, device)
+ self.executable = None
+ self.vm = None
def _make_executor(self, expr=None):
- main = self.mod["main"]
+ if expr:
+ self.mod["main"] = expr
+
+ self.executable = compile(self.mod, self.target)
+ self.vm = vm_rt.VirtualMachine(self.executable, self.device)
def _vm_wrapper(*args, **kwargs):
- args = self._convert_args(main, args, kwargs)
+ args = self._convert_args(self.mod["main"], args, kwargs)
Review comment:
I do not understand why you need to change this file?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] leeexyz commented on a change in pull request #9380: [ONNX][Converter] Add dynamic nodes support
Posted by GitBox <gi...@apache.org>.
leeexyz commented on a change in pull request #9380:
URL: https://github.com/apache/tvm/pull/9380#discussion_r741584173
##########
File path: python/tvm/relay/backend/vm.py
##########
@@ -275,14 +275,18 @@ def __init__(self, mod, device, target):
self.mod = mod
self.device = device
self.target = target
- self.executable = compile(mod, target)
- self.vm = vm_rt.VirtualMachine(self.executable, device)
+ self.executable = None
+ self.vm = None
def _make_executor(self, expr=None):
- main = self.mod["main"]
+ if expr:
+ self.mod["main"] = expr
+
+ self.executable = compile(self.mod, self.target)
+ self.vm = vm_rt.VirtualMachine(self.executable, self.device)
def _vm_wrapper(*args, **kwargs):
- args = self._convert_args(main, args, kwargs)
+ args = self._convert_args(self.mod["main"], args, kwargs)
Review comment:
Thanks for your reply. It is because of this usage `relay.create_executor(kind, device, target).evaluate(func)(args)` which is inconsistent with graph. If kind is `vm`, an error will be prompted
```
Traceback (most recent call last):
File "tests/python/contrib/test_onnx.py", line 752, in <module>
test_dyn()
File "tests/python/contrib/test_onnx.py", line 718, in test_dyn
verify_dyn_bcast((1, 3, 32, 1), (1, 3, 1, 3), "float32")
File "tests/python/contrib/test_onnx.py", line 715, in verify_dyn_bcast
func, [lhs_data, rhs_data], "test_dyn_bcast", rtol=1e-5, atol=1e-5, is_dyn=True
File "tests/python/contrib/test_onnx.py", line 65, in verify_results
relay_results = run_relay(relay_func, indata, is_dyn)
File "tests/python/contrib/test_onnx.py", line 54, in run_relay
relay_res = relay.create_executor(kind, device=dev, target=target).evaluate(func)(*data_tuple)
File "/home/tuisku/project/apache/tvm/python/tvm/relay/build_module.py", line 575, in create_executor
return VMExecutor(mod, device, target)
File "/home/tuisku/project/apache/tvm/python/tvm/relay/backend/vm.py", line 278, in __init__
self.executable = compile(mod, target)
File "/home/tuisku/project/apache/tvm/python/tvm/relay/backend/vm.py", line 72, in compile
compiler.lower(mod, target)
File "/home/tuisku/project/apache/tvm/python/tvm/relay/backend/vm.py", line 143, in lower
self._lower(mod, target, target_host)
......
File "../src/ir/module.cc", line 143
ValueError: Cannot find global var "main" in the Module
candidates are: []
```
Because there is no `main` in this Module, it cannot be evaluated. For graph executor, it has https://github.com/apache/tvm/blob/main/python/tvm/relay/build_module.py#L476-L477.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi merged pull request #9380: [ONNX][Converter] Add dynamic nodes support
Posted by GitBox <gi...@apache.org>.
masahi merged pull request #9380:
URL: https://github.com/apache/tvm/pull/9380
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] leeexyz commented on a change in pull request #9380: [ONNX][Converter] Add dynamic nodes support
Posted by GitBox <gi...@apache.org>.
leeexyz commented on a change in pull request #9380:
URL: https://github.com/apache/tvm/pull/9380#discussion_r741584173
##########
File path: python/tvm/relay/backend/vm.py
##########
@@ -275,14 +275,18 @@ def __init__(self, mod, device, target):
self.mod = mod
self.device = device
self.target = target
- self.executable = compile(mod, target)
- self.vm = vm_rt.VirtualMachine(self.executable, device)
+ self.executable = None
+ self.vm = None
def _make_executor(self, expr=None):
- main = self.mod["main"]
+ if expr:
+ self.mod["main"] = expr
+
+ self.executable = compile(self.mod, self.target)
+ self.vm = vm_rt.VirtualMachine(self.executable, self.device)
def _vm_wrapper(*args, **kwargs):
- args = self._convert_args(main, args, kwargs)
+ args = self._convert_args(self.mod["main"], args, kwargs)
Review comment:
Thanks for your reply. It is because of this usage `relay.create_executor(kind, device, target).evaluate(func)(args)` which is inconsistent with graph. If kind is `vm`, an error will be prompted
```
Traceback (most recent call last):
File "tests/python/contrib/test_onnx.py", line 752, in <module>
test_dyn()
File "tests/python/contrib/test_onnx.py", line 718, in test_dyn
verify_dyn_bcast((1, 3, 32, 1), (1, 3, 1, 3), "float32")
File "tests/python/contrib/test_onnx.py", line 715, in verify_dyn_bcast
func, [lhs_data, rhs_data], "test_dyn_bcast", rtol=1e-5, atol=1e-5, is_dyn=True
File "tests/python/contrib/test_onnx.py", line 65, in verify_results
relay_results = run_relay(relay_func, indata, is_dyn)
File "tests/python/contrib/test_onnx.py", line 54, in run_relay
relay_res = relay.create_executor(kind, device=dev, target=target).evaluate(func)(*data_tuple)
File "/home/tuisku/project/apache/tvm/python/tvm/relay/build_module.py", line 575, in create_executor
return VMExecutor(mod, device, target)
File "/home/tuisku/project/apache/tvm/python/tvm/relay/backend/vm.py", line 278, in __init__
self.executable = compile(mod, target)
File "/home/tuisku/project/apache/tvm/python/tvm/relay/backend/vm.py", line 72, in compile
compiler.lower(mod, target)
File "/home/tuisku/project/apache/tvm/python/tvm/relay/backend/vm.py", line 143, in lower
self._lower(mod, target, target_host)
......
File "../src/ir/module.cc", line 143
ValueError: Cannot find global var "main" in the Module
candidates are: []
```
Because there is no `main` in this Module, it cannot be evaluated. For graph executor, it has https://github.com/apache/tvm/blob/main/python/tvm/relay/build_module.py#L476-L477.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi merged pull request #9380: [ONNX][Converter] Add dynamic nodes support
Posted by GitBox <gi...@apache.org>.
masahi merged pull request #9380:
URL: https://github.com/apache/tvm/pull/9380
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org