You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2020/12/28 23:56:14 UTC
[GitHub] [tvm] comaniac opened a new pull request #7173: [AutoScheduler] Use VM to extract tasks for dynamic models
comaniac opened a new pull request #7173:
URL: https://github.com/apache/tvm/pull/7173
Discussion: https://discuss.tvm.apache.org/t/autoscheduler-and-vm/8750
Remaining issues to auto-schedule PyTorch Mask R-CNN but not handled in this PR:
1. There are still lots of extracted tasks only contain a shape_of op. This is because shape_of has TOpPattern kOpaque which is one target of auto_scheduler task extraction. The problem is other ops with this pattern may be tunable (e.g., softmax).
2. Although the source tensor of shape_of in Mask R-CNN has dynamic shape (e.g., (Any, 28, 28)), it cannot be detected by checking I/O tensors, because it has no input tensors, and the shape of its output tensor is (3,) which is static.
3. When extracting tasks for a NCHW model on x86, Conv2D NCHWc compute will be selected. However, we should provide the general NCHW compute to auto_scheduler for tuning.
cc @merrymercy @masahi @jcf94
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on pull request #7173: [AutoScheduler] Use VM to extract tasks for dynamic models
Posted by GitBox <gi...@apache.org>.
masahi commented on pull request #7173:
URL: https://github.com/apache/tvm/pull/7173#issuecomment-752417084
thanks @comaniac @jcf94 @merrymercy
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] jcf94 commented on a change in pull request #7173: [AutoScheduler] Use VM to extract tasks for dynamic models
Posted by GitBox <gi...@apache.org>.
jcf94 commented on a change in pull request #7173:
URL: https://github.com/apache/tvm/pull/7173#discussion_r549539994
##########
File path: python/tvm/auto_scheduler/relay_integration.py
##########
@@ -251,6 +273,9 @@ def auto_schedule_topi(outs, has_complex_op):
from tvm import relay
io_tensors, has_layout_free = traverse_to_get_io_tensors(outs)
+ if not io_tensors: # The compute includes dynamic shapes which are not supported yet.
+ return None
Review comment:
Suggest to add comment to L214-215:
```python
io_tensors: List[Tensor]
The input and output tensors with static shape.
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] jcf94 commented on a change in pull request #7173: [AutoScheduler] Use VM to extract tasks for dynamic models
Posted by GitBox <gi...@apache.org>.
jcf94 commented on a change in pull request #7173:
URL: https://github.com/apache/tvm/pull/7173#discussion_r549539386
##########
File path: python/tvm/auto_scheduler/relay_integration.py
##########
@@ -56,9 +58,20 @@ def call_all_topi_funcs(mod, params, target):
config={"relay.backend.use_auto_scheduler": True},
disabled_pass={"AutoSchedulerLayoutRewrite"},
):
- opt_mod, _ = relay.optimize(mod, target, params)
- grc = graph_runtime_codegen.GraphRuntimeCodegen(None, target)
- grc.codegen(opt_mod["main"])
+ try:
+ opt_mod, _ = relay.optimize(mod, target, params)
+ grc = graph_runtime_codegen.GraphRuntimeCodegen(None, target)
+ grc.codegen(opt_mod["main"])
+ except tvm.TVMError:
+ print(
+ "Get errors with GraphRuntimeCodegen for task extraction. "
+ "Fallback to VMCompiler."
+ )
+ compiler = relay.vm.VMCompiler()
Review comment:
So this error will only occur in VM environment?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] comaniac commented on a change in pull request #7173: [AutoScheduler] Use VM to extract tasks for dynamic models
Posted by GitBox <gi...@apache.org>.
comaniac commented on a change in pull request #7173:
URL: https://github.com/apache/tvm/pull/7173#discussion_r549563409
##########
File path: python/tvm/auto_scheduler/relay_integration.py
##########
@@ -56,9 +58,20 @@ def call_all_topi_funcs(mod, params, target):
config={"relay.backend.use_auto_scheduler": True},
disabled_pass={"AutoSchedulerLayoutRewrite"},
):
- opt_mod, _ = relay.optimize(mod, target, params)
- grc = graph_runtime_codegen.GraphRuntimeCodegen(None, target)
- grc.codegen(opt_mod["main"])
+ try:
+ opt_mod, _ = relay.optimize(mod, target, params)
+ grc = graph_runtime_codegen.GraphRuntimeCodegen(None, target)
+ grc.codegen(opt_mod["main"])
+ except tvm.TVMError:
+ print(
+ "Get errors with GraphRuntimeCodegen for task extraction. "
+ "Fallback to VMCompiler."
+ )
+ compiler = relay.vm.VMCompiler()
Review comment:
You meant this exception? This will occur when graph runtime fails to compile a model (e.g., the model with IF).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on pull request #7173: [AutoScheduler] Use VM to extract tasks for dynamic models
Posted by GitBox <gi...@apache.org>.
masahi commented on pull request #7173:
URL: https://github.com/apache/tvm/pull/7173#issuecomment-751937055
Is the order of extracted tasks supposed to be deterministic? I think this also applies to graph codegen, but I'm seeing different tasks appear in different orders on each run.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] comaniac commented on pull request #7173: [AutoScheduler] Use VM to extract tasks for dynamic models
Posted by GitBox <gi...@apache.org>.
comaniac commented on pull request #7173:
URL: https://github.com/apache/tvm/pull/7173#issuecomment-751942169
> Is the order of extracted tasks supposed to be deterministic? I think this also applies to graph codegen, but I'm seeing different tasks appear in different orders on each run.
The task order is the same as the op execution order so it should be deterministic, because the graph visiting order is deterministic. However, I'm not familiar with the VM execution, so I'm not sure if VM execution order is deterministic or not.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi merged pull request #7173: [AutoScheduler] Use VM to extract tasks for dynamic models
Posted by GitBox <gi...@apache.org>.
masahi merged pull request #7173:
URL: https://github.com/apache/tvm/pull/7173
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] jcf94 commented on a change in pull request #7173: [AutoScheduler] Use VM to extract tasks for dynamic models
Posted by GitBox <gi...@apache.org>.
jcf94 commented on a change in pull request #7173:
URL: https://github.com/apache/tvm/pull/7173#discussion_r549539994
##########
File path: python/tvm/auto_scheduler/relay_integration.py
##########
@@ -251,6 +273,9 @@ def auto_schedule_topi(outs, has_complex_op):
from tvm import relay
io_tensors, has_layout_free = traverse_to_get_io_tensors(outs)
+ if not io_tensors: # The compute includes dynamic shapes which are not supported yet.
+ return None
Review comment:
Suggest to add comment to L214-215:
```python
"""
...
io_tensors: List[Tensor]
The input and output tensors with static shape.
"""
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] jcf94 commented on pull request #7173: [AutoScheduler] Use VM to extract tasks for dynamic models
Posted by GitBox <gi...@apache.org>.
jcf94 commented on pull request #7173:
URL: https://github.com/apache/tvm/pull/7173#issuecomment-751915446
Thanks! I have encountered the case 3, too. Which should be easy to fix, just add some auto scheduler related check in op Strategy.
For 1 and 2, we may need more discussion...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org