You are viewing a plain text version of this content. The canonical link for it is here.
Posted to discuss-archive@tvm.apache.org by Xiaoran Weng via Apache TVM Discuss <no...@discuss.tvm.ai> on 2021/12/22 05:32:16 UTC
[Apache TVM Discuss] [Questions] [BYOC] How to partition specific region of a Relay graph to CPU
I'd like the post-processing part of the model to be executed on the CPU instead of the accelerator, is there a method to tell TVM to 'stop' partitioning for external BYOC compiler at some specific Relay operators (e.g., the `ArgMax` and `Gather` op in this case)? Should I write a custom pass or use the existing functionality to achieve this (like using `AnnotateTarget` and defining a custom pattern to isolate this subgraph)?
---
[Visit Topic](https://discuss.tvm.apache.org/t/byoc-how-to-partition-specific-region-of-a-relay-graph-to-cpu/11758/1) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/22b667ec3edb9b37edab15522403fda0cfbc4c3d79fc97378c089949443d16b6).
[Apache TVM Discuss] [Questions] [BYOC] How to partition specific region of a Relay graph to CPU
Posted by Xiaoran Weng via Apache TVM Discuss <no...@discuss.tvm.ai>.
Thanks, I'll check that out.
---
[Visit Topic](https://discuss.tvm.apache.org/t/byoc-how-to-partition-specific-region-of-a-relay-graph-to-cpu/11758/5) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/5a29c93408c6520b87f89f4e5cdcce7eef593eefd292b6b4cca0e9570d3e2df1).
[Apache TVM Discuss] [Questions] [BYOC] How to partition specific region of a Relay graph to CPU
Posted by Johnson9009 via Apache TVM Discuss <no...@discuss.tvm.ai>.
Yes, this is the common issue that hardware vendor meet when integrating there tool chain through BYOC.
You can refer the method used by tensorRT,
1. parttion any op your acc supported, then maybe you get multiple your acc subgraph.
2. reinline the subgraphs that haven't big computation into relay main function.
The code of tensorRT already in the TVM repo.
---
[Visit Topic](https://discuss.tvm.apache.org/t/byoc-how-to-partition-specific-region-of-a-relay-graph-to-cpu/11758/4) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/5b77fdb21494960b36c6ad5414287762d2f903b3ba261d8844482f682bcc45e1).
[Apache TVM Discuss] [Questions] [BYOC] How to partition specific region of a Relay graph to CPU
Posted by Xiaoran Weng via Apache TVM Discuss <no...@discuss.tvm.ai>.
A "region" is a subgraph that can be offloaded to some target as in the `MergeCompilerRegions` pass. Essentially I want all the operators after the specific two operators to be unoffloadable, how can I do this?
---
[Visit Topic](https://discuss.tvm.apache.org/t/byoc-how-to-partition-specific-region-of-a-relay-graph-to-cpu/11758/3) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/dd5020edc05b885d8c5f0e9dd338c7a5434ee1cfdc69968a2b722a837234d34b).
[Apache TVM Discuss] [Questions] [BYOC] How to partition specific region of a Relay graph to CPU
Posted by "Cody H. Yu via Apache TVM Discuss" <no...@discuss.tvm.ai>.
I don't quite understand your question. In particular how do you define a "region"? If a "region" is just an operator, then BYOC won't partition them for the accelerator if you didn't mark those ops as offloadable.
---
[Visit Topic](https://discuss.tvm.apache.org/t/byoc-how-to-partition-specific-region-of-a-relay-graph-to-cpu/11758/2) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/c73b180715c536cd53d2c1a06b8ffd235c3356772af0c2d0ae60b2957d6ec709).