You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tvm.apache.org by Zhuwenxi via Apache TVM Discuss <no...@discuss.tvm.ai> on 2021/06/29 11:37:04 UTC
[Apache TVM Discuss] [Development] [AutoScheduler] Do we have plan to
support auto schedule ExternOp?
Hi All.
I just noticed that AutoScheduler lacks support for ExternOp. Currently AutoScheduler supports ComputeOp only.
I understand that it is non-trial to auto schedule a op with external function calls, however there are a bunch of topi ops whose algorithm are purely written with tensor expression and NO extern function call involved, are still using "te.Extern()" as their wrapper. It's really frustrating because those ops can't be auto-tuned by with auto_scheduler. I can give a few examples, such as scatter_nd: https://github.com/apache/tvm/blob/8fce89500c520c4dc6ce8733172fa87ead107709/python/tvm/topi/cuda/scatter.py#L830
Welcome for comments, @merrymercy @tqchen
Thank you!
---
[Visit Topic](https://discuss.tvm.apache.org/t/autoscheduler-do-we-have-plan-to-support-auto-schedule-externop/10346/1) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/aa2b837a9b747f84dc547f22b6fb93e7d0ed75b465a9f73fca5c082eced92525).
[Apache TVM Discuss] [Development] [AutoScheduler] Do we have plan to support auto schedule ExternOp?
Posted by "Cody H. Yu via Apache TVM Discuss" <no...@discuss.tvm.ai>.
It seems not impossible to me, since `ExternOp` doesn't even have a schedule but simply calls an "extern" implementation such as CuBLAS.
---
[Visit Topic](https://discuss.tvm.apache.org/t/autoscheduler-do-we-have-plan-to-support-auto-schedule-externop/10346/11) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/36c210e0e2e0ed3599f9763da75143ffc21d91333322a603b8a09e5127147ba3).
[Apache TVM Discuss] [Development] [AutoScheduler] Do we have plan to support auto schedule ExternOp?
Posted by Hao Cheng via Apache TVM Discuss <no...@discuss.tvm.ai>.
[quote="comaniac, post:2, topic:10346"]
toScheduler can only tune TE compute, it cannot tune such ops anyway
[/quote]
I am working with atuo-scheduler with `te.extern`， but failed with compile. is there any chance using auto-schedule with `te.extern` op?
---
[Visit Topic](https://discuss.tvm.apache.org/t/autoscheduler-do-we-have-plan-to-support-auto-schedule-externop/10346/10) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/019b949c524921dc53e21706815d697fc715a4d602309162ceea006e5f205360).
[Apache TVM Discuss] [Development] [AutoScheduler] Do we have plan to
support auto schedule ExternOp?
Posted by Zhuwenxi via Apache TVM Discuss <no...@discuss.tvm.ai>.
@comaniac By the way, do you know the specific reason why ops like "scatter" choose to implement use TIR instead of TE?
According to my quick statistic, there are at least 13 ops in relay uses TIR as their implementation:
1. argwhere
2. non_max_suppression
3. scanop
4. scatter
5. scatter_nd
6. scatter_add
7. sort
8. topk
9. sparse_reshape
10. sparse
11. unique
12. proposal
13. multibox
---
[Visit Topic](https://discuss.tvm.apache.org/t/autoscheduler-do-we-have-plan-to-support-auto-schedule-externop/10346/4) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/3bf55526423c406dbd72da0a2cf802895084647868229b6729525c9d1656c52a).
[Apache TVM Discuss] [Development] [AutoScheduler] Do we have plan to
support auto schedule ExternOp?
Posted by Zhuwenxi via Apache TVM Discuss <no...@discuss.tvm.ai>.
Thank you @comaniac , really appreciate!
The reason "...because they are written in TIR instead of TE" does make sense to me. And I agree for the case "scatter", the improvement would be small. I guess Relay's default schedule is probably good enough for my case.
---
[Visit Topic](https://discuss.tvm.apache.org/t/autoscheduler-do-we-have-plan-to-support-auto-schedule-externop/10346/3) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/3852322f044c83568c11717a2777dc9e5de144b1e87daa399252bb99f2ea6871).
[Apache TVM Discuss] [Development] [AutoScheduler] Do we have plan to
support auto schedule ExternOp?
Posted by "Cody H. Yu via Apache TVM Discuss" <no...@discuss.tvm.ai>.
I guess the reason it needs a wrapper is because they are written in TIR instead of TE? Since AutoScheduler can only tune TE compute, it cannot tune such ops anyways. On the other hand, the AutoTIR that @junrushao1994 is working on supports tuning for all levels so you may look forward it.
Meanwhile, I'm not sure how many improvements you will get by tuning such ops (e.g., scatter). There computes are not that complicate, so the tuning space is expected to be small.
---
[Visit Topic](https://discuss.tvm.apache.org/t/autoscheduler-do-we-have-plan-to-support-auto-schedule-externop/10346/2) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/be01c62e7529c8597365dd269ae81edd7fffa510a06d693e98f72ca1d25e0c04).
[Apache TVM Discuss] [Development] [AutoScheduler] Do we have plan to
support auto schedule ExternOp?
Posted by Zhuwenxi via Apache TVM Discuss <no...@discuss.tvm.ai>.
@mbrookhart Make sense, thank you!
---
[Visit Topic](https://discuss.tvm.apache.org/t/autoscheduler-do-we-have-plan-to-support-auto-schedule-externop/10346/9) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/280bd912e5e6324194364b5fb48f359ff2798bc10e434fe07023b2259a7cbe4d).
[Apache TVM Discuss] [Development] [AutoScheduler] Do we have plan to
support auto schedule ExternOp?
Posted by Matthew Brookhart via Apache TVM Discuss <no...@discuss.tvm.ai>.
I spent a lot of time optimizing the sort/argsort kernel for GPUs, we get pretty good performance on GPUs from multiple vendors that competes with those vendor's hand tuned libraries.
If these TIR kernels are well optimized, they shouldn't end up being the bottleneck in models.
---
[Visit Topic](https://discuss.tvm.apache.org/t/autoscheduler-do-we-have-plan-to-support-auto-schedule-externop/10346/8) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/14c295c34570a97d001db55f9298884a22c6dc456ae89c68ae205b388f91ad5e).
[Apache TVM Discuss] [Development] [AutoScheduler] Do we have plan to
support auto schedule ExternOp?
Posted by Zhuwenxi via Apache TVM Discuss <no...@discuss.tvm.ai>.
Thank you, @comaniac .
@jroesch @mbrookhart @ritwikdas54 I noticed you've participated in implementing those ops above (by git blame :stuck_out_tongue_winking_eye:), could you explain a little bit about why use TIR instead of TE?
---
[Visit Topic](https://discuss.tvm.apache.org/t/autoscheduler-do-we-have-plan-to-support-auto-schedule-externop/10346/6) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/75ccf2ac345908e168fe73eb096f246a5a04e5fd59ea26338a39483b7895859d).
[Apache TVM Discuss] [Development] [AutoScheduler] Do we have plan to
support auto schedule ExternOp?
Posted by "Cody H. Yu via Apache TVM Discuss" <no...@discuss.tvm.ai>.
It's possible that those ops need a finer control to buffers and it is hard to achieve in TE, but I didn't write those ops so I don't know the exact reason.
---
[Visit Topic](https://discuss.tvm.apache.org/t/autoscheduler-do-we-have-plan-to-support-auto-schedule-externop/10346/5) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/0be47656dcdec985e272b5a3bb13026a00c30f793cf7c3d7bc333fb4ab97a120).
[Apache TVM Discuss] [Development] [AutoScheduler] Do we have plan to
support auto schedule ExternOp?
Posted by Matthew Brookhart via Apache TVM Discuss <no...@discuss.tvm.ai>.
TE is a limited declarative programming model, it's not possible to write operations that do data-dependent indexing in TE.
Anything that's sort/scatter related needs to be written directly in the more imperative TIR.
---
[Visit Topic](https://discuss.tvm.apache.org/t/autoscheduler-do-we-have-plan-to-support-auto-schedule-externop/10346/7) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/3803bbcecfed0708515fb8317e0f4d156f55f70ad15b02a322deae3c13cd1f73).