You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2022/11/09 08:42:51 UTC
[GitHub] [tvm] masahi opened a new pull request, #13329: [MetaSchedule] Fix the order of applying `AutoInline` in `ScheduleUsingAnchorTrace`
masahi opened a new pull request, #13329:
URL: https://github.com/apache/tvm/pull/13329
In anchor-block tuning, we need to manually apply `AutoInline` to some blocks (those that are not part of the anchor subgraph). Currently the order of blocks to apply `AutoInline` is undefined, but I've hit a case where this is problematic.
For example, given these four blocks,
```
for i0_7, i1_7, i2_7, i3_7 in T.grid(16, 56, 56, 256):
with T.block("compute_2"):
i0_8, i1_8, i2_8, i3_8 = T.axis.remap("SSSS", [i0_7, i1_7, i2_7, i3_7])
T.reads(T_subtract_1[i0_8, i1_8, i2_8, i3_8])
T.writes(compute_3[i0_8, i1_8, i2_8, i3_8])
compute_3[i0_8, i1_8, i2_8, i3_8] = T.q_multiply_shift(T_subtract_1[i0_8, i1_8, i2_8, i3_8], 1457846997, 31, 0, dtype="int32")
for i0_9, i1_9, i2_9, i3_9 in T.grid(16, 56, 56, 256):
with T.block("compute_3"):
i0_10, i1_10, i2_10, i3_10 = T.axis.remap("SSSS", [i0_9, i1_9, i2_9, i3_9])
T.reads(p9[i0_10, i1_10, i2_10, i3_10])
T.writes(compute_4[i0_10, i1_10, i2_10, i3_10])
compute_4[i0_10, i1_10, i2_10, i3_10] = T.q_multiply_shift(p9[i0_10, i1_10, i2_10, i3_10], 2101000910, 31, 0, dtype="int32")
for i0_11, i1_11, i2_11, i3_11 in T.grid(16, 56, 56, 256):
with T.block("T_add_2"):
ax0, ax1, ax2, ax3 = T.axis.remap("SSSS", [i0_11, i1_11, i2_11, i3_11])
T.reads(compute_3[ax0, ax1, ax2, ax3], compute_4[ax0, ax1, ax2, ax3])
T.writes(T_add_2[ax0, ax1, ax2, ax3])
T_add_2[ax0, ax1, ax2, ax3] = compute_3[ax0, ax1, ax2, ax3] + compute_4[ax0, ax1, ax2, ax3]
for i0_12, i1_12, i2_12, i3_12 in T.grid(16, 56, 56, 256):
with T.block("compute_4"):
i0_13, i1_13, i2_13, i3_13 = T.axis.remap("SSSS", [i0_12, i1_12, i2_12, i3_12])
T.reads(T_add_2[i0_13, i1_13, i2_13, i3_13])
T.writes(compute[i0_13, i1_13, i2_13, i3_13])
compute[i0_13, i1_13, i2_13, i3_13] = T.max(T.min(T_add_2[i0_13, i1_13, i2_13, i3_13], 255), 0)
```
, we want to `AutoInline` "compute_3", "T_add_2" and "compute_4". If the order is "T_add_2" -> "compute_3" -> "compute_4", all three blocks can be inlined / reverse inlined to "compute_2". However, if the order is "T_add_2" -> "compute_4" -> "compute_3" , "compute_4" can neither be inlined or reverse inlined. This in turn can result in a buggy schedule to be generated (see the description in the test case).
We can avoid this problem by always `AutoInlin`ing the last block after all other blocks have been processed. This ensures that the last block can be reverse inlined.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on a diff in pull request #13329: [MetaSchedule] Fix the order of applying `AutoInline` in `ScheduleUsingAnchorTrace`
Posted by GitBox <gi...@apache.org>.
masahi commented on code in PR #13329:
URL: https://github.com/apache/tvm/pull/13329#discussion_r1017638468
##########
tests/python/unittest/test_meta_schedule_trace_apply.py:
##########
@@ -2742,5 +3026,351 @@ def apply_trace(sch):
)
+def test_inline_order():
+ # In this test, the order of applying AutoInline is tested.
+ # We need to make sure that the last block in Conv2dInt8_with_predicate_target,
+ # "compute_4", is AutoInline-ed after all other blocks have been processed.
+ #
+ # Otherwise, if the order is "T_add_2" -> "compute_4" -> "compute_3", "compute_4" is neither
+ # inlined (because this is the last block) nor reverse-inlined
+ # (because it has multiple producers). This results in the "compute_4" block being
+ # reverse-inlined at the very end of ScheduleUsingAnchorTrace, where its producer block
+ # "conv2d_nhwc_reindex_shared" has the predicate
+ # T.where(((ax1_0 * 4 + ax1_1) * 32 + ax1_2) * 2 + ax1_3 < 64) due to anchor-block scheduling
+ # (see Conv2dInt8_with_predicate_scheduled). Currently, if we try to reverse-inline a block to
+ # its producer that has a predicate, the predicate disappears after reverse inlining.
Review Comment:
cc @vinx13 to confirm if applying `reverse_compute_inline` when the producer has a predicate should be disallowed. Currently it is allowed, and the predicate disappears. A minimum repro in https://gist.github.com/masahi/01a80b86062122ad57b9b1fd785fb960
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] junrushao commented on pull request #13329: [MetaSchedule] Fix the order of applying `AutoInline` in `ScheduleUsingAnchorTrace`
Posted by GitBox <gi...@apache.org>.
junrushao commented on PR #13329:
URL: https://github.com/apache/tvm/pull/13329#issuecomment-1309374111
Will leave this PR to @vinx13 :-)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] vinx13 closed pull request #13329: [MetaSchedule] Fix the order of applying `AutoInline` in `ScheduleUsingAnchorTrace`
Posted by GitBox <gi...@apache.org>.
vinx13 closed pull request #13329: [MetaSchedule] Fix the order of applying `AutoInline` in `ScheduleUsingAnchorTrace`
URL: https://github.com/apache/tvm/pull/13329
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on a diff in pull request #13329: [MetaSchedule] Fix the order of applying `AutoInline` in `ScheduleUsingAnchorTrace`
Posted by GitBox <gi...@apache.org>.
masahi commented on code in PR #13329:
URL: https://github.com/apache/tvm/pull/13329#discussion_r1017631570
##########
src/meta_schedule/trace_apply.cc:
##########
@@ -140,9 +159,13 @@ std::vector<BlockRV> ApplyAnchorTrace(Schedule sch, Trace anchor_trace) {
// Similar to the reverse_compute_inline case above.
auto block = Downcast<BlockRV>(inputs[0]);
auto block_sref = sch->GetSRef(block);
- if (!CanComputeInline(sch->state(), block_sref)) {
- ICHECK(CanReverseComputeInline(sch->state(), block_sref));
- sch->ReverseComputeInline(block);
+ auto state = sch->state();
+ if (!CanComputeInline(state, block_sref)) {
+ ICHECK(IsOutputBlock(state, block_sref, GetScopeRoot(state, block_sref, false)))
+ << "If a spatial block cannot be inlined, it should be the output block";
+ if (CanReverseComputeInline(sch->state(), block_sref)) {
+ sch->ReverseComputeInline(block);
+ }
Review Comment:
This is another fix, just relaxing the wrong assumption at L144.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tvm-bot commented on pull request #13329: [MetaSchedule] Fix the order of applying `AutoInline` in `ScheduleUsingAnchorTrace`
Posted by GitBox <gi...@apache.org>.
tvm-bot commented on PR #13329:
URL: https://github.com/apache/tvm/pull/13329#issuecomment-1308407298
<!---bot-comment-->
Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from [Reviewers](https://github.com/apache/incubator-tvm/blob/master/CONTRIBUTORS.md#reviewers) by @-ing them in a comment.
<!--bot-comment-ccs-start-->
* cc @Hzfengsy, @elvin-n, @junrushao <sub>See [#10317](https://github.com/apache/tvm/issues/10317) for details</sub><!--bot-comment-ccs-end-->
<sub>Generated by [tvm-bot](https://github.com/apache/tvm/blob/main/ci/README.md#github-actions)</sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org