You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2022/12/02 01:55:39 UTC
[GitHub] [tvm] Hzfengsy opened a new pull request, #13534: [TIR] Introduce Pass UnifyKernelLaunch and speedup split op
Hzfengsy opened a new pull request, #13534:
URL: https://github.com/apache/tvm/pull/13534
This PR introduces a new pass UnifyKernelLaunch to fuse the nearby kernels with same thread config, which will reduce the kernel launch overhead in `split` operator.
Test on the following settings:
```python
A = te.placeholder((75, 182), "int64")
B = topi.split(A, 182, axis=1)
func = te.create_prim_func([A, *B])
```
The pass will reduce the latency from 347.03 us to 5.22 us on my machine.
cc @MasterJH5574 @vinx13 @junrushao
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] junrushao commented on a diff in pull request #13534: [TIR] Introduce Pass UnifyKernelLaunch and speedup split op
Posted by GitBox <gi...@apache.org>.
junrushao commented on code in PR #13534:
URL: https://github.com/apache/tvm/pull/13534#discussion_r1039152953
##########
src/meta_schedule/schedule/cuda/split.cc:
##########
@@ -0,0 +1,34 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+#include "../../utils.h"
+
+namespace tvm {
+namespace meta_schedule {
+
+using namespace tvm::tir;
+
+TVM_REGISTER_GLOBAL("meta_schedule.cuda.split")
+ .set_body_typed([](Schedule sch, BlockRV data_pack) -> Array<Schedule> {
+ // Do nothing, which ensure the same thread config for each block.
+ // We can fuse these kernels into only one.
+ return {sch};
+ });
Review Comment:
if we want to skip a block, use {"schedule_rule": "None"} instead
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] Hzfengsy commented on pull request #13534: [TIR] Introduce Pass UnifyKernelLaunch and speedup split op
Posted by GitBox <gi...@apache.org>.
Hzfengsy commented on PR #13534:
URL: https://github.com/apache/tvm/pull/13534#issuecomment-1334903060
@tvm-bot rerun
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] Hzfengsy closed pull request #13534: [TIR] Introduce Pass UnifyKernelLaunch and speedup split op
Posted by "Hzfengsy (via GitHub)" <gi...@apache.org>.
Hzfengsy closed pull request #13534: [TIR] Introduce Pass UnifyKernelLaunch and speedup split op
URL: https://github.com/apache/tvm/pull/13534
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] junrushao commented on pull request #13534: [TIR] Introduce Pass UnifyKernelLaunch and speedup split op
Posted by GitBox <gi...@apache.org>.
junrushao commented on PR #13534:
URL: https://github.com/apache/tvm/pull/13534#issuecomment-1371598225
@vinx13 for context, @Hzfengsy and I are having some issues where synchronization might potentially be missing, so we are not proceeding with the PR yet
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tvm-bot commented on pull request #13534: [TIR] Introduce Pass UnifyKernelLaunch and speedup split op
Posted by GitBox <gi...@apache.org>.
tvm-bot commented on PR #13534:
URL: https://github.com/apache/tvm/pull/13534#issuecomment-1334665409
<!---bot-comment-->
Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from [Reviewers](https://github.com/apache/incubator-tvm/blob/master/CONTRIBUTORS.md#reviewers) by @-ing them in a comment.
<!--bot-comment-ccs-start-->
* cc @junrushao <sub>See [#10317](https://github.com/apache/tvm/issues/10317) for details</sub><!--bot-comment-ccs-end-->
<sub>Generated by [tvm-bot](https://github.com/apache/tvm/blob/main/ci/README.md#github-actions)</sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] Hzfengsy commented on a diff in pull request #13534: [TIR] Introduce Pass UnifyKernelLaunch and speedup split op
Posted by GitBox <gi...@apache.org>.
Hzfengsy commented on code in PR #13534:
URL: https://github.com/apache/tvm/pull/13534#discussion_r1039155884
##########
src/meta_schedule/schedule/cuda/split.cc:
##########
@@ -0,0 +1,34 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+#include "../../utils.h"
+
+namespace tvm {
+namespace meta_schedule {
+
+using namespace tvm::tir;
+
+TVM_REGISTER_GLOBAL("meta_schedule.cuda.split")
+ .set_body_typed([](Schedule sch, BlockRV data_pack) -> Array<Schedule> {
+ // Do nothing, which ensure the same thread config for each block.
+ // We can fuse these kernels into only one.
+ return {sch};
+ });
Review Comment:
We only want to skip the block for GPU backend, but still will apply schedule on other targets.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] junrushao commented on a diff in pull request #13534: [TIR] Introduce Pass UnifyKernelLaunch and speedup split op
Posted by GitBox <gi...@apache.org>.
junrushao commented on code in PR #13534:
URL: https://github.com/apache/tvm/pull/13534#discussion_r1039156944
##########
src/meta_schedule/schedule/cuda/split.cc:
##########
@@ -0,0 +1,34 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+#include "../../utils.h"
+
+namespace tvm {
+namespace meta_schedule {
+
+using namespace tvm::tir;
+
+TVM_REGISTER_GLOBAL("meta_schedule.cuda.split")
+ .set_body_typed([](Schedule sch, BlockRV data_pack) -> Array<Schedule> {
+ // Do nothing, which ensure the same thread config for each block.
+ // We can fuse these kernels into only one.
+ return {sch};
+ });
Review Comment:
perhaps we should support per-device None in meta schedule :-)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org