Re: [apache/tvm-rfcs] [RFC] Establish TVM Unity Connection -- A Technical Strategy (PR #91)

[Siyuan Feng <no...@github.com.INVALID>]

Thanks @tqchen!!! I'm excited to see the pre-RFC become this formal RFC.  

The Unity Connection is a great step from multi-level lowering compilation to a flexible, unified abstraction for the end-to-end model compilation. I'd like to summarize the [discuss thread](https://discuss.tvm.apache.org/t/establish-tvm-unity-connection-a-technical-strategy/13344) here for readers who did not participate.

## Modularized Compilation Flow
The TVM unity uses cross-layer abstraction to represent:
1. Graph IR: how to organize ops/kernels
2. Tensor IR/Libraries/FFI: how to execute the ops/kernels
Based on such abstraction, we can build the module at any stage as long as the module is legal. However, the current multi-stage lowering pipeline requires we must have a `GraphIR->TensorIR->RuntimeModule` pipeline.

## Easy to customize 
Fast customization is critical during researching and prototyping. The modularized workflow natively enables it. Here I'd like to share two cases:

### Ex1: Adding new operator supports
Instead of [7-step tutorial](https://tvm.apache.org/docs/dev/how_to/relay_add_op.html), here are only two steps with the unity connection:
- implement how the op is computed (both tir or libraries are good),
- Directly call the implementation in the unified abstraction.

### Ex2: Customizable operator fusion
Each pass is decoupled, which means we are able to fuse operators (to be general, optimize the module) in multiple passes. i.e. We can have a customized pass to fuse two convs while using the internal fusor to fuse the following element-wise ops.

## Cross-layer optimization opportunities
Layout optimization is a typical cross-layer optimization, that we are able to do with TVM Unity. Also, we do have some prototype results to prove it works. Additionally, I'm happy to see the community members working on different backends are all looking forward to this feature.

Note that this RFC is a technical strategy, which is a bit different with the Relax Upstream RFC https://github.com/apache/tvm-rfcs/pull/89. Please turn to that thread if you have specific comments on relax itself.

Love to hear ideas from the community.


-- 
Reply to this email directly or view it on GitHub:
https://github.com/apache/tvm-rfcs/pull/91#issuecomment-1225899168
You are receiving this because you are subscribed to this thread.

Message ID: <ap...@github.com>