[apache/tvm-rfcs] [RFC] Improved multi-target handling (#38)

[Mark Shields <no...@github.com.INVALID>]

While working on unifying some device planning passes and cleaning up the TECompiler we noticed we have a lot of issues around how we transition from devices to Targets in the hetogeneous compilation flow. This pre-RFC is a stab at fixing those. There's a somewhat larger issue around bringing BYOC target name annotation and device planning closer together since they are very very similar but currently incompatible. However I don't understand the BYOC flow enough to launch into that.

I realize we normally start with a pre-RFC discussion on discord, but would like to experiment with having that conversation directly on GitHub. That way I can update the RFC text to account for discussion as we go, capturing where we have consensus and capturing alternatives directly in the text. Humor me?
You can view, comment on, or merge this pull request online at:

  https://github.com/apache/tvm-rfcs/pull/38

-- Commit Summary --

  * <a href="https://github.com/apache/tvm-rfcs/pull/38/commits/d4e54007a0fee68aa906f8bd578e01593a905fe0">[checkpoint] first stab at summary</a>
  * <a href="https://github.com/apache/tvm-rfcs/pull/38/commits/861366cf3e9e8636124c52fd495b0563871dc996">[checkpoint] wrong dir</a>
  * <a href="https://github.com/apache/tvm-rfcs/pull/38/commits/dc0f50e8857d2ba530daad840616d4dd55b621e1">rewrite with focus on CompileOptions instead of TargetDevice</a>
  * <a href="https://github.com/apache/tvm-rfcs/pull/38/commits/b3809e9b6615b788812d8f04975e42e4db0ca0b7">Another rewrite.</a>

-- File Changes --

    A .gitignore (1)
    A rfcs/00xx-improved-multi-target-handling.md (175)

-- Patch Links --

https://github.com/apache/tvm-rfcs/pull/38.patch
https://github.com/apache/tvm-rfcs/pull/38.diff

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/apache/tvm-rfcs/pull/38

Re: [apache/tvm-rfcs] [RFC] Unified device/target/memory scope planning (#38)

[Mark Shields <no...@github.com.INVALID>]

Closed #38.

-- 
Reply to this email directly or view it on GitHub:
https://github.com/apache/tvm-rfcs/pull/38#event-6221275454
You are receiving this because you are subscribed to this thread.

Message ID: <ap...@github.com>

Re: [apache/tvm-rfcs] [RFC] Improved multi-target handling (#38)

[Mark Shields <no...@github.com.INVALID>]

Note to self: The With<Target> convention should probably also be removed by this work also, but I've not audited the code to see how pervasive it is.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/apache/tvm-rfcs/pull/38#issuecomment-932464732

Re: [apache/tvm-rfcs] [RFC] Improved multi-target handling (#38)

[Manupa Karunaratne <no...@github.com.INVALID>]

Hi @mbs-octoml ,

I may have put a related comment here : https://github.com/apache/tvm/pull/8892#issuecomment-932020564
However, partitioning for devices of same kind is a step forward from unifying the BYOC and Device annotations.

Is this the RFC intended to cover these all ?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/apache/tvm-rfcs/pull/38#issuecomment-932097824

Re: [apache/tvm-rfcs] [RFC] Unified device/target/memory scope planning (#38)

[Cody Yu <no...@github.com.INVALID>]

For the proposed BYOC flow (i.e., `MergeAndAnnotate`/`AnnotateSEScopes`/`PlanDevices`/`PartitionBySEScope`), it doesn't clear to me that whether the developer programming model will change or not. Specifically, could we still use the current approaches (i.e., op-based annotation and pattern-based annotation)? Also how the approach that has a custom Relay pass to annotate `compiler_begin`/`compiler_end` would change? Thanks.


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/apache/tvm-rfcs/pull/38#issuecomment-955085747

Re: [apache/tvm-rfcs] [RFC] Improved multi-target handling (#38)

[Tianqi Chen <no...@github.com.INVALID>]

cc @ zxybazh since you authored the Target system, cc @zhiics @comaniac as it is related to BYOC

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/apache/tvm-rfcs/pull/38#issuecomment-939150864

Re: [apache/tvm-rfcs] [RFC] Unified device/target/memory scope planning (#38)

[Mark Shields <no...@github.com.INVALID>]

Thanks for the comments @areusch . Now that I've started working on this (with an emphasis on handling memory scopes) I've decided to shift focus a bit. PTAL.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/apache/tvm-rfcs/pull/38#issuecomment-948990025

Re: [apache/tvm-rfcs] [RFC] Unified device/target/memory scope planning (#38)

[Mark Shields <no...@github.com.INVALID>]

Closing as obsolete, since most of this is either already done or has been subsumed by the Collage proposal.

-- 
Reply to this email directly or view it on GitHub:
https://github.com/apache/tvm-rfcs/pull/38#issuecomment-1064590913
You are receiving this because you are subscribed to this thread.

Message ID: <ap...@github.com>

Re: [apache/tvm-rfcs] [RFC] Improved multi-target handling (#38)

[Mark Shields <no...@github.com.INVALID>]

More notes to self:
 - You can get the device_type for a Target from target_kind->device_type. Somehow I missed that very obvious fact.
 - Eric has a very nice write-up explaining devices vs targets vs device api at docs/dev/device_target_interactions.rst 

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/apache/tvm-rfcs/pull/38#issuecomment-939103388

Re: [apache/tvm-rfcs] [RFC] Improved multi-target handling (#38)

[Mark Shields <no...@github.com.INVALID>]

Thanks @manupa-arm for the reminder there were some good comments on #8892. I see a work stream:
1. get the multi-target handling under control, and stop relying on the limiting device type -> target mapping
2. allow device planning to be re-run with additional memory scope constraints to further partition, those constraints may originate in already lowered PrimFuncs.
3. allow labeling of sets of target/devices a la BYOC target labeling (but probably continue to just pick the 'first')
4. bring BYOC target labeling / partitioning and device planning together
5. replace naive partitioning with something subject to optimization

Here I want to just focus on 1 -- everything beyond that really needs face-to-face discussion.
 
From 2 onward obviously overlaps your USMP. My vague thought was we can work from opposite ends and reconcile at 5. Ie 2-4 sets us up to work in a combined Relay+TIR world, then 5 is where everything we've learned from USMP could perhaps be replayed. Anyway, that's just a vague thought so I'd love to talk more about it.



-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/apache/tvm-rfcs/pull/38#issuecomment-932274086