You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2020/11/12 01:51:42 UTC
[GitHub] [incubator-tvm] comaniac opened a new pull request #6903: [AutoSchedule] Extract tasks via compile engine
comaniac opened a new pull request #6903:
URL: https://github.com/apache/incubator-tvm/pull/6903
See RFC: for details.
## TODO
- [ ] Collect feedback from RFC.
- [ ] Add unit tests.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] merrymercy commented on a change in pull request #6903: [AutoSchedule] Extract tasks via compile engine
Posted by GitBox <gi...@apache.org>.
merrymercy commented on a change in pull request #6903:
URL: https://github.com/apache/incubator-tvm/pull/6903#discussion_r524224815
##########
File path: tutorials/auto_scheduler/tune_network_cuda.py
##########
@@ -105,7 +105,6 @@ def get_network(name, batch_size, layout="NHWC", dtype="float32"):
mod, params = relay.testing.squeezenet.get_workload(
version="1.1",
batch_size=batch_size,
- layout=layout,
Review comment:
We can add `assert layout == "NCHW"` for this squeezenet
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] merrymercy commented on a change in pull request #6903: [AutoSchedule] Extract tasks via compile engine
Posted by GitBox <gi...@apache.org>.
merrymercy commented on a change in pull request #6903:
URL: https://github.com/apache/incubator-tvm/pull/6903#discussion_r523802883
##########
File path: python/tvm/relay/op/strategy/cuda.py
##########
@@ -222,12 +218,11 @@ def conv2d_strategy_cuda(attrs, inputs, out_type, target):
# register auto-scheduler implementations
if judge_winograd_auto_scheduler:
- strategy.add_auto_scheduler(
- wrap_compute_conv2d(topi.nn.conv2d_winograd_nhwc), name="conv2d_nhwc.winograd"
- )
- else:
- strategy.add_auto_scheduler(
- wrap_compute_conv2d(topi.nn.conv2d_nhwc), name="conv2d_nhwc"
+ strategy.add_implementation(
+ wrap_compute_conv2d(topi.nn.conv2d_winograd_nhwc),
+ wrap_topi_schedule(tvm.te.create_schedule),
+ name="conv2d_nhwc.winograd",
+ plevel=10,
Review comment:
Both the plevel for `conv2d_nhwc.cuda` and `conv2d_nhwc.winograd` are 10.
Which one will auto-scheduler pick? Which one will autotvm pick?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] comaniac commented on a change in pull request #6903: [AutoSchedule] Extract tasks via compile engine
Posted by GitBox <gi...@apache.org>.
comaniac commented on a change in pull request #6903:
URL: https://github.com/apache/incubator-tvm/pull/6903#discussion_r523809528
##########
File path: python/tvm/relay/op/strategy/cuda.py
##########
@@ -222,12 +218,11 @@ def conv2d_strategy_cuda(attrs, inputs, out_type, target):
# register auto-scheduler implementations
if judge_winograd_auto_scheduler:
- strategy.add_auto_scheduler(
- wrap_compute_conv2d(topi.nn.conv2d_winograd_nhwc), name="conv2d_nhwc.winograd"
- )
- else:
- strategy.add_auto_scheduler(
- wrap_compute_conv2d(topi.nn.conv2d_nhwc), name="conv2d_nhwc"
+ strategy.add_implementation(
+ wrap_compute_conv2d(topi.nn.conv2d_winograd_nhwc),
+ wrap_topi_schedule(tvm.te.create_schedule),
+ name="conv2d_nhwc.winograd",
+ plevel=10,
Review comment:
I was thinking about this too. The current semantic is when extracting AutoTVM tasks, only the direct one will be extracted because Winograd one isn't registered as an AutoTVM task, so does the case of applying AutoTVM tuning logs. So we can probably just make this plevel higher (say, 12).
The only issue is when no tuning logs are given, the Winograd one will be selected if judge_Winograd is True. In this case we need to make sure Winograd fallback is better than the direct.
What do you think?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] merrymercy commented on a change in pull request #6903: [AutoSchedule] Extract tasks via compile engine
Posted by GitBox <gi...@apache.org>.
merrymercy commented on a change in pull request #6903:
URL: https://github.com/apache/incubator-tvm/pull/6903#discussion_r523811310
##########
File path: python/tvm/relay/op/strategy/cuda.py
##########
@@ -222,12 +218,11 @@ def conv2d_strategy_cuda(attrs, inputs, out_type, target):
# register auto-scheduler implementations
if judge_winograd_auto_scheduler:
- strategy.add_auto_scheduler(
- wrap_compute_conv2d(topi.nn.conv2d_winograd_nhwc), name="conv2d_nhwc.winograd"
- )
- else:
- strategy.add_auto_scheduler(
- wrap_compute_conv2d(topi.nn.conv2d_nhwc), name="conv2d_nhwc"
+ strategy.add_implementation(
+ wrap_compute_conv2d(topi.nn.conv2d_winograd_nhwc),
+ wrap_topi_schedule(tvm.te.create_schedule),
+ name="conv2d_nhwc.winograd",
+ plevel=10,
Review comment:
Autotvm should never select this winograd implementation because there is no schedule templates for it.
We should have a way to specify this. But as you said, a single plevel cannot work for both autotvm and auto-scheduler.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] merrymercy commented on a change in pull request #6903: [AutoSchedule] Extract tasks via compile engine
Posted by GitBox <gi...@apache.org>.
merrymercy commented on a change in pull request #6903:
URL: https://github.com/apache/incubator-tvm/pull/6903#discussion_r523802883
##########
File path: python/tvm/relay/op/strategy/cuda.py
##########
@@ -222,12 +218,11 @@ def conv2d_strategy_cuda(attrs, inputs, out_type, target):
# register auto-scheduler implementations
if judge_winograd_auto_scheduler:
- strategy.add_auto_scheduler(
- wrap_compute_conv2d(topi.nn.conv2d_winograd_nhwc), name="conv2d_nhwc.winograd"
- )
- else:
- strategy.add_auto_scheduler(
- wrap_compute_conv2d(topi.nn.conv2d_nhwc), name="conv2d_nhwc"
+ strategy.add_implementation(
+ wrap_compute_conv2d(topi.nn.conv2d_winograd_nhwc),
+ wrap_topi_schedule(tvm.te.create_schedule),
+ name="conv2d_nhwc.winograd",
+ plevel=10,
Review comment:
Both the plevel for `conv2d_nhwc.cuda` and `conv2d_nhwc.winograd` are 10.
Which one will auto-scheduler pick? Which one will autotvm pick?
We want the auto-scheduler to pick this Winograd compute if Winograd is applicable.
We want autotvm always to never pick this.
##########
File path: python/tvm/relay/op/strategy/cuda.py
##########
@@ -222,12 +218,11 @@ def conv2d_strategy_cuda(attrs, inputs, out_type, target):
# register auto-scheduler implementations
if judge_winograd_auto_scheduler:
- strategy.add_auto_scheduler(
- wrap_compute_conv2d(topi.nn.conv2d_winograd_nhwc), name="conv2d_nhwc.winograd"
- )
- else:
- strategy.add_auto_scheduler(
- wrap_compute_conv2d(topi.nn.conv2d_nhwc), name="conv2d_nhwc"
+ strategy.add_implementation(
+ wrap_compute_conv2d(topi.nn.conv2d_winograd_nhwc),
+ wrap_topi_schedule(tvm.te.create_schedule),
+ name="conv2d_nhwc.winograd",
+ plevel=10,
Review comment:
Both the plevel for `conv2d_nhwc.cuda` and `conv2d_nhwc.winograd` are 10.
Which one will auto-scheduler pick? Which one will autotvm pick?
We want the auto-scheduler to pick this Winograd compute if Winograd is applicable.
We want autotvm to never pick this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] comaniac commented on a change in pull request #6903: [AutoSchedule] Extract tasks via compile engine
Posted by GitBox <gi...@apache.org>.
comaniac commented on a change in pull request #6903:
URL: https://github.com/apache/incubator-tvm/pull/6903#discussion_r524450562
##########
File path: python/tvm/auto_scheduler/relay_integration.py
##########
@@ -34,18 +34,26 @@
def call_all_topi_funcs(mod, params, target):
- """Call all TOPI compute + schedule to extract tasks in a relay program"""
+ """Call all TOPI compute to extract auto_scheduler tasks in a Relay program"""
# pylint: disable=import-outside-toplevel
from tvm import relay
from tvm.relay.backend import graph_runtime_codegen
- with transform.PassContext(opt_level=3):
+ # Turn off AutoTVM config not found warnings
+ old_autotvm_silent = autotvm.GLOBAL_SCOPE.silent
+ autotvm.GLOBAL_SCOPE.silent = True
+
+ with transform.PassContext(opt_level=3, config={"relay.backend.use_auto_scheduler": True}):
Review comment:
Considering auto_scheduler would be a widely used feature, would this config be too long for most users to remember? Should we provide syntactic sugar, or maybe we can help users set this config in the `ApplyHistoryBest` like the following, although the order of setting build environments has to be fixed in this case.
```python
with transform.PassContext(opt_level3):
with auto_scheduler.ApplyHistoryBest(log_file): # Set "relay.backend": {"use_auto_scheduler": True} to the current pass context?
# ...
```
cc @merrymercy
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] zhiics commented on a change in pull request #6903: [AutoSchedule] Extract tasks via compile engine
Posted by GitBox <gi...@apache.org>.
zhiics commented on a change in pull request #6903:
URL: https://github.com/apache/incubator-tvm/pull/6903#discussion_r523704272
##########
File path: src/relay/backend/compile_engine.cc
##########
@@ -98,8 +98,10 @@ Array<IndexExpr> GetShape(const Array<IndexExpr>& shape) {
// Get schedule from functor.
class ScheduleGetter : public backend::MemoizedExprTranslator<Array<te::Tensor>> {
public:
- explicit ScheduleGetter(Target target)
- : target_(target), device_copy_op_(Op::Get("device_copy")) {}
+ explicit ScheduleGetter(Target target, bool use_auto_schedule)
Review comment:
Can we just get the flag from PassContext::Current() instead of passing it around?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] mbaret commented on pull request #6903: [AutoSchedule] Extract tasks via compile engine
Posted by GitBox <gi...@apache.org>.
mbaret commented on pull request #6903:
URL: https://github.com/apache/incubator-tvm/pull/6903#issuecomment-725988146
At a high level, it seems to me this may benefit from the refactor I proposed in https://github.com/apache/incubator-tvm/pull/6888. In particular, rather than switching the behaviour of ScheduleGetter with a flag it may make more sense to simply provide an alternative ScheduleGetter for the auto-scheduler. With a 'TETranslator' de-coupled, we should be able to avoid duplicating lots of code. What are your thoughts?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] tqchen commented on a change in pull request #6903: [AutoSchedule] Extract tasks via compile engine
Posted by GitBox <gi...@apache.org>.
tqchen commented on a change in pull request #6903:
URL: https://github.com/apache/incubator-tvm/pull/6903#discussion_r524453803
##########
File path: python/tvm/auto_scheduler/relay_integration.py
##########
@@ -34,18 +34,26 @@
def call_all_topi_funcs(mod, params, target):
- """Call all TOPI compute + schedule to extract tasks in a relay program"""
+ """Call all TOPI compute to extract auto_scheduler tasks in a Relay program"""
# pylint: disable=import-outside-toplevel
from tvm import relay
from tvm.relay.backend import graph_runtime_codegen
- with transform.PassContext(opt_level=3):
+ # Turn off AutoTVM config not found warnings
+ old_autotvm_silent = autotvm.GLOBAL_SCOPE.silent
+ autotvm.GLOBAL_SCOPE.silent = True
+
+ with transform.PassContext(opt_level=3, config={"relay.backend.use_auto_scheduler": True}):
Review comment:
I see, I think we can go with the current proposal if that is the concern
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] comaniac commented on a change in pull request #6903: [AutoSchedule] Extract tasks via compile engine
Posted by GitBox <gi...@apache.org>.
comaniac commented on a change in pull request #6903:
URL: https://github.com/apache/incubator-tvm/pull/6903#discussion_r523711839
##########
File path: src/relay/backend/compile_engine.cc
##########
@@ -98,8 +98,10 @@ Array<IndexExpr> GetShape(const Array<IndexExpr>& shape) {
// Get schedule from functor.
class ScheduleGetter : public backend::MemoizedExprTranslator<Array<te::Tensor>> {
public:
- explicit ScheduleGetter(Target target)
- : target_(target), device_copy_op_(Op::Get("device_copy")) {}
+ explicit ScheduleGetter(Target target, bool use_auto_schedule)
Review comment:
Good point. All added arguments are removed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] merrymercy commented on a change in pull request #6903: [AutoSchedule] Extract tasks via compile engine
Posted by GitBox <gi...@apache.org>.
merrymercy commented on a change in pull request #6903:
URL: https://github.com/apache/incubator-tvm/pull/6903#discussion_r523802588
##########
File path: python/tvm/relay/op/strategy/cuda.py
##########
@@ -222,12 +218,11 @@ def conv2d_strategy_cuda(attrs, inputs, out_type, target):
# register auto-scheduler implementations
if judge_winograd_auto_scheduler:
- strategy.add_auto_scheduler(
- wrap_compute_conv2d(topi.nn.conv2d_winograd_nhwc), name="conv2d_nhwc.winograd"
- )
- else:
- strategy.add_auto_scheduler(
- wrap_compute_conv2d(topi.nn.conv2d_nhwc), name="conv2d_nhwc"
+ strategy.add_implementation(
Review comment:
The plevel for`conv2d_nhwc.cuda` and
Which one will auto-scheduler pick? Which one will autotvm pick?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] comaniac commented on pull request #6903: [AutoSchedule] Extract tasks via compile engine
Posted by GitBox <gi...@apache.org>.
comaniac commented on pull request #6903:
URL: https://github.com/apache/incubator-tvm/pull/6903#issuecomment-726264707
> At a high level, it seems to me this may benefit from the refactor I proposed in #6888. In particular, rather than switching the behaviour of ScheduleGetter with a flag it may make more sense to simply provide an alternative ScheduleGetter for the auto-scheduler. With a 'TETranslator' de-coupled, we should be able to avoid duplicating lots of code. What are your thoughts?
This PR can definitely work with #6888 to have a more concise implementation. However, this PR needs more than a TE translator, and that's why I exposed it first to have more discussions instead of waiting for #6888. Although we might be able to simplify the changes of `compile_engine.cc` in this PR with the TE translator, other changes in this PR (e.g., add a flag to graph runtime codegen and many other places) are still required. Thus, we can still work in parallel, and modify the later merged one accordingly to achieve the same goal.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] merrymercy commented on a change in pull request #6903: [AutoSchedule] Extract tasks via compile engine
Posted by GitBox <gi...@apache.org>.
merrymercy commented on a change in pull request #6903:
URL: https://github.com/apache/incubator-tvm/pull/6903#discussion_r523802883
##########
File path: python/tvm/relay/op/strategy/cuda.py
##########
@@ -222,12 +218,11 @@ def conv2d_strategy_cuda(attrs, inputs, out_type, target):
# register auto-scheduler implementations
if judge_winograd_auto_scheduler:
- strategy.add_auto_scheduler(
- wrap_compute_conv2d(topi.nn.conv2d_winograd_nhwc), name="conv2d_nhwc.winograd"
- )
- else:
- strategy.add_auto_scheduler(
- wrap_compute_conv2d(topi.nn.conv2d_nhwc), name="conv2d_nhwc"
+ strategy.add_implementation(
+ wrap_compute_conv2d(topi.nn.conv2d_winograd_nhwc),
+ wrap_topi_schedule(tvm.te.create_schedule),
+ name="conv2d_nhwc.winograd",
+ plevel=10,
Review comment:
Both the plevel for `conv2d_nhwc.cuda` and `conv2d_nhwc.winograd` are 10.
Which one will auto-scheduler pick? Which one will autotvm pick?
We want the auto-scheduler to pick this Winograd compute if Winograd is applicable.
We want autotvm to never pick this.
Autotvm and auto-scheduler have different preferences. It seems that a single plevel is not enough.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] comaniac commented on a change in pull request #6903: [AutoSchedule] Extract tasks via compile engine
Posted by GitBox <gi...@apache.org>.
comaniac commented on a change in pull request #6903:
URL: https://github.com/apache/incubator-tvm/pull/6903#discussion_r523948808
##########
File path: tutorials/auto_scheduler/tune_network_cuda.py
##########
@@ -105,7 +105,6 @@ def get_network(name, batch_size, layout="NHWC", dtype="float32"):
mod, params = relay.testing.squeezenet.get_workload(
version="1.1",
batch_size=batch_size,
- layout=layout,
Review comment:
Because the squeezenet get_network API doesn't have this argument.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] merrymercy commented on a change in pull request #6903: [AutoSchedule] Extract tasks via compile engine
Posted by GitBox <gi...@apache.org>.
merrymercy commented on a change in pull request #6903:
URL: https://github.com/apache/incubator-tvm/pull/6903#discussion_r523802883
##########
File path: python/tvm/relay/op/strategy/cuda.py
##########
@@ -222,12 +218,11 @@ def conv2d_strategy_cuda(attrs, inputs, out_type, target):
# register auto-scheduler implementations
if judge_winograd_auto_scheduler:
- strategy.add_auto_scheduler(
- wrap_compute_conv2d(topi.nn.conv2d_winograd_nhwc), name="conv2d_nhwc.winograd"
- )
- else:
- strategy.add_auto_scheduler(
- wrap_compute_conv2d(topi.nn.conv2d_nhwc), name="conv2d_nhwc"
+ strategy.add_implementation(
+ wrap_compute_conv2d(topi.nn.conv2d_winograd_nhwc),
+ wrap_topi_schedule(tvm.te.create_schedule),
+ name="conv2d_nhwc.winograd",
+ plevel=10,
Review comment:
Both the plevel for `conv2d_nhwc.cuda` and `conv2d_nhwc.winograd` are 10.
Which one will auto-scheduler pick? Which one will autotvm pick?
We want the auto-scheduler to pick this Winograd compute if Winograd is applicable.
We want autotvm always not to pick this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] comaniac commented on a change in pull request #6903: [AutoSchedule] Extract tasks via compile engine
Posted by GitBox <gi...@apache.org>.
comaniac commented on a change in pull request #6903:
URL: https://github.com/apache/incubator-tvm/pull/6903#discussion_r523812386
##########
File path: python/tvm/relay/op/strategy/cuda.py
##########
@@ -222,12 +218,11 @@ def conv2d_strategy_cuda(attrs, inputs, out_type, target):
# register auto-scheduler implementations
if judge_winograd_auto_scheduler:
- strategy.add_auto_scheduler(
- wrap_compute_conv2d(topi.nn.conv2d_winograd_nhwc), name="conv2d_nhwc.winograd"
- )
- else:
- strategy.add_auto_scheduler(
- wrap_compute_conv2d(topi.nn.conv2d_nhwc), name="conv2d_nhwc"
+ strategy.add_implementation(
+ wrap_compute_conv2d(topi.nn.conv2d_winograd_nhwc),
+ wrap_topi_schedule(tvm.te.create_schedule),
+ name="conv2d_nhwc.winograd",
+ plevel=10,
Review comment:
Hmm you're right. We can never pick Winograd when auto_scheduler is not enabled. Then how about we check PassContext and only add this Winograd when use_auto_scheduler is True? In long term we will need a better mechanism to select the implementation anyways.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] merrymercy commented on a change in pull request #6903: [AutoSchedule] Extract tasks via compile engine
Posted by GitBox <gi...@apache.org>.
merrymercy commented on a change in pull request #6903:
URL: https://github.com/apache/incubator-tvm/pull/6903#discussion_r523703285
##########
File path: python/tvm/auto_scheduler/relay_integration.py
##########
@@ -34,18 +34,26 @@
def call_all_topi_funcs(mod, params, target):
- """Call all TOPI compute + schedule to extract tasks in a relay program"""
+ """Call all TOPI compute to extract auto_scheduler tasks in a Relay program"""
# pylint: disable=import-outside-toplevel
from tvm import relay
from tvm.relay.backend import graph_runtime_codegen
+ # Turn off AutoTVM config not found warnings
+ old_autotvm_silent = autotvm.GLOBAL_SCOPE.silent
+ autotvm.GLOBAL_SCOPE.silent = True
+
with transform.PassContext(opt_level=3):
opt_mod, _ = relay.optimize(mod, target, params)
- grc = graph_runtime_codegen.GraphRuntimeCodegen(None, target)
+ grc = graph_runtime_codegen.GraphRuntimeCodegen(None, target, use_auto_schedule=True)
Review comment:
rename all `use_auto_schedule` to `use_auto_scheduler`
##########
File path: python/tvm/auto_scheduler/relay_integration.py
##########
@@ -34,18 +34,26 @@
def call_all_topi_funcs(mod, params, target):
- """Call all TOPI compute + schedule to extract tasks in a relay program"""
+ """Call all TOPI compute to extract auto_scheduler tasks in a Relay program"""
# pylint: disable=import-outside-toplevel
from tvm import relay
from tvm.relay.backend import graph_runtime_codegen
+ # Turn off AutoTVM config not found warnings
+ old_autotvm_silent = autotvm.GLOBAL_SCOPE.silent
+ autotvm.GLOBAL_SCOPE.silent = True
+
with transform.PassContext(opt_level=3):
opt_mod, _ = relay.optimize(mod, target, params)
- grc = graph_runtime_codegen.GraphRuntimeCodegen(None, target)
+ grc = graph_runtime_codegen.GraphRuntimeCodegen(None, target, use_auto_schedule=True)
grc.codegen(opt_mod["main"])
+ autotvm.GLOBAL_SCOPE.silent = old_autotvm_silent
+
-def extract_tasks(mod, params, target, target_host=None, hardware_params=None):
+def extract_tasks(
+ mod, params, target, include_simple_tasks=False, target_host=None, hardware_params=None
Review comment:
nit:
move `include_simple_tasks` after `hardware_params`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] comaniac commented on a change in pull request #6903: [AutoSchedule] Extract tasks via compile engine
Posted by GitBox <gi...@apache.org>.
comaniac commented on a change in pull request #6903:
URL: https://github.com/apache/incubator-tvm/pull/6903#discussion_r523948808
##########
File path: tutorials/auto_scheduler/tune_network_cuda.py
##########
@@ -105,7 +105,6 @@ def get_network(name, batch_size, layout="NHWC", dtype="float32"):
mod, params = relay.testing.squeezenet.get_workload(
version="1.1",
batch_size=batch_size,
- layout=layout,
Review comment:
Because the squeezenet get_workload API doesn't have this argument.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] merrymercy commented on a change in pull request #6903: [AutoSchedule] Extract tasks via compile engine
Posted by GitBox <gi...@apache.org>.
merrymercy commented on a change in pull request #6903:
URL: https://github.com/apache/incubator-tvm/pull/6903#discussion_r523820661
##########
File path: python/tvm/relay/op/strategy/cuda.py
##########
@@ -222,12 +218,11 @@ def conv2d_strategy_cuda(attrs, inputs, out_type, target):
# register auto-scheduler implementations
if judge_winograd_auto_scheduler:
- strategy.add_auto_scheduler(
- wrap_compute_conv2d(topi.nn.conv2d_winograd_nhwc), name="conv2d_nhwc.winograd"
- )
- else:
- strategy.add_auto_scheduler(
- wrap_compute_conv2d(topi.nn.conv2d_nhwc), name="conv2d_nhwc"
+ strategy.add_implementation(
+ wrap_compute_conv2d(topi.nn.conv2d_winograd_nhwc),
+ wrap_topi_schedule(tvm.te.create_schedule),
+ name="conv2d_nhwc.winograd",
+ plevel=10,
Review comment:
Sounds good
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] comaniac commented on pull request #6903: [AutoSchedule] Extract tasks via compile engine
Posted by GitBox <gi...@apache.org>.
comaniac commented on pull request #6903:
URL: https://github.com/apache/incubator-tvm/pull/6903#issuecomment-727684651
Comment addressed. Now we could also use the flag in PassContext to add the TOPI compute for auto_scheduler with higher plevel (e.g., 15). Since this changes one task from direct to Winograd in the tutorial, I'm re-generating CI logs for the tutorial and will update it tomorrow.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] tqchen commented on a change in pull request #6903: [AutoSchedule] Extract tasks via compile engine
Posted by GitBox <gi...@apache.org>.
tqchen commented on a change in pull request #6903:
URL: https://github.com/apache/incubator-tvm/pull/6903#discussion_r524291161
##########
File path: python/tvm/auto_scheduler/relay_integration.py
##########
@@ -34,18 +34,26 @@
def call_all_topi_funcs(mod, params, target):
- """Call all TOPI compute + schedule to extract tasks in a relay program"""
+ """Call all TOPI compute to extract auto_scheduler tasks in a Relay program"""
# pylint: disable=import-outside-toplevel
from tvm import relay
from tvm.relay.backend import graph_runtime_codegen
- with transform.PassContext(opt_level=3):
+ # Turn off AutoTVM config not found warnings
+ old_autotvm_silent = autotvm.GLOBAL_SCOPE.silent
+ autotvm.GLOBAL_SCOPE.silent = True
+
+ with transform.PassContext(opt_level=3, config={"relay.backend.use_auto_scheduler": True}):
Review comment:
Given that there might be multiple backend arguments in the future, consider change to
```
config= {
"relay.backend": {"use_auto_scheduler": True},
}
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] merrymercy commented on a change in pull request #6903: [AutoSchedule] Extract tasks via compile engine
Posted by GitBox <gi...@apache.org>.
merrymercy commented on a change in pull request #6903:
URL: https://github.com/apache/incubator-tvm/pull/6903#discussion_r523802588
##########
File path: python/tvm/relay/op/strategy/cuda.py
##########
@@ -222,12 +218,11 @@ def conv2d_strategy_cuda(attrs, inputs, out_type, target):
# register auto-scheduler implementations
if judge_winograd_auto_scheduler:
- strategy.add_auto_scheduler(
- wrap_compute_conv2d(topi.nn.conv2d_winograd_nhwc), name="conv2d_nhwc.winograd"
- )
- else:
- strategy.add_auto_scheduler(
- wrap_compute_conv2d(topi.nn.conv2d_nhwc), name="conv2d_nhwc"
+ strategy.add_implementation(
Review comment:
This plevel is the same as `conv2d_nhwc.cuda`, which one will auto-scheduler pick?
which one will autotvm pick?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] zhiics commented on pull request #6903: [AutoSchedule] Extract tasks via compile engine
Posted by GitBox <gi...@apache.org>.
zhiics commented on pull request #6903:
URL: https://github.com/apache/incubator-tvm/pull/6903#issuecomment-728674105
Thanks everyone.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] zhiics commented on pull request #6903: [AutoSchedule] Extract tasks via compile engine
Posted by GitBox <gi...@apache.org>.
zhiics commented on pull request #6903:
URL: https://github.com/apache/incubator-tvm/pull/6903#issuecomment-727609861
please rebase
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] merrymercy commented on a change in pull request #6903: [AutoSchedule] Extract tasks via compile engine
Posted by GitBox <gi...@apache.org>.
merrymercy commented on a change in pull request #6903:
URL: https://github.com/apache/incubator-tvm/pull/6903#discussion_r523811310
##########
File path: python/tvm/relay/op/strategy/cuda.py
##########
@@ -222,12 +218,11 @@ def conv2d_strategy_cuda(attrs, inputs, out_type, target):
# register auto-scheduler implementations
if judge_winograd_auto_scheduler:
- strategy.add_auto_scheduler(
- wrap_compute_conv2d(topi.nn.conv2d_winograd_nhwc), name="conv2d_nhwc.winograd"
- )
- else:
- strategy.add_auto_scheduler(
- wrap_compute_conv2d(topi.nn.conv2d_nhwc), name="conv2d_nhwc"
+ strategy.add_implementation(
+ wrap_compute_conv2d(topi.nn.conv2d_winograd_nhwc),
+ wrap_topi_schedule(tvm.te.create_schedule),
+ name="conv2d_nhwc.winograd",
+ plevel=10,
Review comment:
Autotvm should never select this winograd implementation because there is no schedule template for it.
We should have a way to specify this. But a single plevel cannot work for both autotvm and auto-scheduler.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] zhiics merged pull request #6903: [AutoSchedule] Extract tasks via compile engine
Posted by GitBox <gi...@apache.org>.
zhiics merged pull request #6903:
URL: https://github.com/apache/incubator-tvm/pull/6903
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] merrymercy commented on a change in pull request #6903: [AutoSchedule] Extract tasks via compile engine
Posted by GitBox <gi...@apache.org>.
merrymercy commented on a change in pull request #6903:
URL: https://github.com/apache/incubator-tvm/pull/6903#discussion_r523919649
##########
File path: tutorials/auto_scheduler/tune_network_cuda.py
##########
@@ -105,7 +105,6 @@ def get_network(name, batch_size, layout="NHWC", dtype="float32"):
mod, params = relay.testing.squeezenet.get_workload(
version="1.1",
batch_size=batch_size,
- layout=layout,
Review comment:
why do we delete this?
We can add an assertion here if it does not support NHWC layout.
##########
File path: tutorials/auto_scheduler/tune_network_cuda.py
##########
@@ -105,7 +105,6 @@ def get_network(name, batch_size, layout="NHWC", dtype="float32"):
mod, params = relay.testing.squeezenet.get_workload(
version="1.1",
batch_size=batch_size,
- layout=layout,
Review comment:
Why do we delete this?
We can add an assertion here if it does not support NHWC layout.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] comaniac commented on pull request #6903: [AutoSchedule] Extract tasks via compile engine
Posted by GitBox <gi...@apache.org>.
comaniac commented on pull request #6903:
URL: https://github.com/apache/incubator-tvm/pull/6903#issuecomment-727520104
@merrymercy @zhiics thanks for the review and the comments are addressed. PTAL.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] merrymercy commented on a change in pull request #6903: [AutoSchedule] Extract tasks via compile engine
Posted by GitBox <gi...@apache.org>.
merrymercy commented on a change in pull request #6903:
URL: https://github.com/apache/incubator-tvm/pull/6903#discussion_r523811310
##########
File path: python/tvm/relay/op/strategy/cuda.py
##########
@@ -222,12 +218,11 @@ def conv2d_strategy_cuda(attrs, inputs, out_type, target):
# register auto-scheduler implementations
if judge_winograd_auto_scheduler:
- strategy.add_auto_scheduler(
- wrap_compute_conv2d(topi.nn.conv2d_winograd_nhwc), name="conv2d_nhwc.winograd"
- )
- else:
- strategy.add_auto_scheduler(
- wrap_compute_conv2d(topi.nn.conv2d_nhwc), name="conv2d_nhwc"
+ strategy.add_implementation(
+ wrap_compute_conv2d(topi.nn.conv2d_winograd_nhwc),
+ wrap_topi_schedule(tvm.te.create_schedule),
+ name="conv2d_nhwc.winograd",
+ plevel=10,
Review comment:
Autotvm should never select this winograd implementation because there is no schedule template for it. It cannot be compiled correctly.
We should have a way to specify this. But a single plevel cannot work for both autotvm and auto-scheduler.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] zhiics commented on a change in pull request #6903: [AutoSchedule] Extract tasks via compile engine
Posted by GitBox <gi...@apache.org>.
zhiics commented on a change in pull request #6903:
URL: https://github.com/apache/incubator-tvm/pull/6903#discussion_r523704414
##########
File path: src/relay/backend/build_module.cc
##########
@@ -433,9 +435,14 @@ class RelayBuildModule : public runtime::ModuleNode {
// Get the updated function.
auto func = Downcast<Function>(relay_module->Lookup("main"));
+ // Check whether to use TOPI schedule.
Review comment:
Can we do this in compile_engine instead of here? It seems that we just pass the extracted flag to graph_runtime_codegen and then to compile_egine.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] merrymercy commented on a change in pull request #6903: [AutoSchedule] Extract tasks via compile engine
Posted by GitBox <gi...@apache.org>.
merrymercy commented on a change in pull request #6903:
URL: https://github.com/apache/incubator-tvm/pull/6903#discussion_r523802883
##########
File path: python/tvm/relay/op/strategy/cuda.py
##########
@@ -222,12 +218,11 @@ def conv2d_strategy_cuda(attrs, inputs, out_type, target):
# register auto-scheduler implementations
if judge_winograd_auto_scheduler:
- strategy.add_auto_scheduler(
- wrap_compute_conv2d(topi.nn.conv2d_winograd_nhwc), name="conv2d_nhwc.winograd"
- )
- else:
- strategy.add_auto_scheduler(
- wrap_compute_conv2d(topi.nn.conv2d_nhwc), name="conv2d_nhwc"
+ strategy.add_implementation(
+ wrap_compute_conv2d(topi.nn.conv2d_winograd_nhwc),
+ wrap_topi_schedule(tvm.te.create_schedule),
+ name="conv2d_nhwc.winograd",
+ plevel=10,
Review comment:
Both the plevel for `conv2d_nhwc.cuda` and `conv2d_nhwc.winograd` are 10.
Which one will auto-scheduler pick? Which one will autotvm pick?
We want the auto-scheduler to pick this Winograd compute if Winograd is applicable.
We want autotvm to never pick this.
autotvm and auto-scheduler have different preferences. It seems that a single plevel is not enough.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org