You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tvm.apache.org by Zackcquic via Apache TVM Discuss <no...@discuss.tvm.ai> on 2021/05/01 00:24:55 UTC
[Apache TVM Discuss] [Development/RFC] Pass Instrument Framework
Proposal
# [IR][Pass][Instrument] Pass Instrument Framework
https://github.com/apache/tvm/pull/7952
Proposal
=======
Currently in TVM, there are trace mechanisms and passes time profiling.
* ### Trace
```cpp
/*!
* \brief PassContextNode contains the information that a pass can rely on,
* such as analysis results.
* \sa PassContext
*/
class PassContextNode : public Object {
public:
// Skipped
/*! \brief Trace function to be invoked before and after each pass. */
TraceFunc trace_func;
// Skipped
```
* ### Pass Time Profiling
https://github.com/apache/tvm/pull/7500
## They have similar semantics that want to profile/trace passes.
## This PR tries to generalize and integrate the concepts:
### 1. Trace is rename to PassInstrumentors with more functionalities:
* Explicitly split RunBeforePass and RunAfterPass into different functions instead of checking in trace call back funcion.
```
def trace(ir_module, pass_info, is_before):
if is_before:
# Before Pass Run
else:
# After Pass Run
# ==>
pi = tvm.instrument.PassInstrument()
@pi.register_run_before_pass
def run_before_pass(ir_module, pass_info):
# Before Pass Run
@pi.regiser_run_after_pass
def run_after_pass(ir_module, pass_info):
# After Pass Run
@pi.register_set_up
def set_up():
# Instrumentation environment set up
@pi.register_tear_down
def tear_down():
# Instrumentation environment clean up
```
* PassInstrumentor collects a set of PassInstrument instead of single call back
```
with tvm.transform.PassContext(Trace=_trace):
# Call _trace in build flow
# ==>
pi1 = tvm.instrument.PassInstrument()
pi2 = tmv.instrument.PassInstrument()
with tvm.transofm.PassContext(
pass_instrumentor=tvm.instrument.PassInstrumentor([pi1, pi2]):
# Call pi1 and pi2's callbacks in build flow
```
### 2. Provide a PassesTimeInstrument that leverages previous passes time profiling c++ code.
```cpp
TVM_REGISTER_GLOBAL("instrument.MakePassesTimeInstrument").set_body_typed([]() {
auto pi = PassInstrument("PassesTimeInstrument");
// No set up function for this time instrumentation.
pi->RegisterTearDownCallback([]() { PassProfileThreadLocalStore::Get()->root.children.clear(); });
pi->RegisterRunBeforePassCallback([](const IRModule&, const transform::PassInfo& pass_info) {
PassProfile::EnterPass(pass_info->name);
return true;
});
pi->RegisterRunAfterPassCallback(
[](const IRModule&, const transform::PassInfo&) { PassProfile::ExitPass(); });
return pi;
});
```
### 3. Inspired by LLVM and MLIR, it might be good to let run_before_pass() determines whether to run a pass with some instrumentation logics. (Return true to run pass; return false to skip pass)
```cpp
/*!
* \file tvm/ir/instrument.h
*
* This file implements a pass instrument infrastructure, inspired from LLVM and MLIR.
* It inserts instrumentation points between passes run.
*
* Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
*
* Instrument SetUp
*
* if (Instrument Before Pass)
* Pass Run
* Instrument After Pass
*
* if (Instrument Before Pass)
* Pass Run
* Instrument After Pass
*
* Instrument TearDown
*
*
* Instrument point before pass can determine particular pass is disable or not depends on the
* callback registered.
*/
```
## Some Question Received
Thanks to **[tkonolige]** and **[tqchen]**
### Q.
**[tkonolige]**
> In order to avoid duplicating code it might be worth unifying this with the runtime profiling framework. I have a branch (which I haven't submitted yet) that allows users to extend which kinds of information are collected.
**[tqchen]**
> I agree it is important to have pass instrumentations, would be good to know how can it interact with Trace, since some of the callbacks might be similar and we need to unify the interface. On the design side, I think the runtime profiling and pass instrumentation might be different enough that might worth two separate solutions(perhaps a bit of reuse of timer if needed) As the former have more complexity wrt to GPU timer etc, while the later allows more statistics to be collected
### A.
I would like to separate runtime profiling and pass instrumentation, too. This PR only focuses on pass profiling mechanisms like the passes time profiling, wants to make it easier to add more passes profiling implementations.
You might notice that this PR introduces a new namespace tvm.intrument. It intends to cover all instrument related (not limited to pass, but this PR only shows pass instrument), instead of mixing instrumentation/profiling code with transformation codes in tvm.transform.
RuntimeProfiling could be add to this namespace, eg: tvm.instrument.Runtime.Profiling.
# Please let me know how you think about this proposal :slight_smile:
---
[Visit Topic](https://discuss.tvm.apache.org/t/pass-instrument-framework-proposal/9874/1) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/8549949a8b1c80259ef1fa968dc48814a7fd6fb8b827c6e206a143aac3856a3a).
[Apache TVM Discuss] [Development/RFC] Pass Instrument Framework
Proposal
Posted by Junru Shao via Apache TVM Discuss <no...@discuss.tvm.ai>.
Also CC @zhiics @zxybazh @vinx13
---
[Visit Topic](https://discuss.tvm.apache.org/t/pass-instrument-framework-proposal/9874/3) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/95e26a601e55293acec6364cfabc8201852d8411dc22f04a9a4a590bd956217a).
[Apache TVM Discuss] [Development/RFC] Pass Instrument Framework
Proposal
Posted by tqchen via Apache TVM Discuss <no...@discuss.tvm.ai>.
Thanks @zackcquic . To follow up on naming,
My understanding is that instrument goes beyond profiling(e.g. can use to collect the IRs being passed around, dump the IR across pass runs for generate debugging purposes). The proposal is essentially a structured form of tracing.
Because of that reason, instrument/tracing would be a better namespace than profiling. We can discuss the specific naming convention by looking into the convention used by existing frameworks. If the class overload and instrumentation is more common than functional trace callback, we can go with that route, as long as the naming is consistent.
---
[Visit Topic](https://discuss.tvm.apache.org/t/pass-instrument-framework-proposal/9874/5) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/fb8c64ff48ae9c34c2475c967f4ae26b2c860b65a9768bd2f1d0ca428e2c2928).
[Apache TVM Discuss] [Development/RFC] Pass Instrument Framework
Proposal
Posted by Tristan Konolige via Apache TVM Discuss <no...@discuss.tvm.ai>.
Thanks for the PR @zackcquic.
1. Why would you like to keep runtime profiling and pass profiling separate? The benefit I see is that a lot of the code is similar. We could avoid a lot of code duplication. On the other hand runtime profiling has does have a lot of code around handling timing on different devices, which would not be used for pass profiling.
2. I think we need consistent naming around profiling/instrumentation. I'd prefer names like `tvm.pass.profiling` and `tvm.runtime.profiling`.
3. Reading through your PR, there seems to be two interfaces for adding a new `PassInstrument`. Either you can subclass `PassInstrument` or you can register callbacks. I'd like to see a single API. Subclassing `PassInstrument` is more in line with how we normally handle extensibility in the codebase. It also a better interface if the instrument needs to maintain some internal state.
4. What instruments do you see being useful in the future? Time and memory usage are the only ones that come to mind for me.
5. What are parts of the code do you imagine we will instrument/profile in the future? (It might be useful to talk about adding a more general to do instrumentation in the codebase, but maybe that is not appropriate for this discussion).
@areusch May have some feedback too.
---
[Visit Topic](https://discuss.tvm.apache.org/t/pass-instrument-framework-proposal/9874/4) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/0dc8d08527479f2a6e7f42a6ad35a2fe840720b9336937268ca42b33d794b2b7).
[Apache TVM Discuss] [Development/RFC] Pass Instrument Framework
Proposal
Posted by Zackcquic via Apache TVM Discuss <no...@discuss.tvm.ai>.
@tqchen @tkonolige Here is the RFC created for discussion.
---
[Visit Topic](https://discuss.tvm.apache.org/t/pass-instrument-framework-proposal/9874/2) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/f7e3abdd98fb0bde64e9d80743564b845e400ab699fa00d734e1d3f4467183c9).
[Apache TVM Discuss] [Development/RFC] Pass Instrument Framework
Proposal
Posted by tqchen via Apache TVM Discuss <no...@discuss.tvm.ai>.
Thanks @zackcquic . I also did a pass over the draft PR and I think the overall proposal looks great. It would be good to think a bit about API naming and naming convention in related previous designs.
e.g. how can we choose the argument name to the PassContext constructor
---
[Visit Topic](https://discuss.tvm.apache.org/t/pass-instrument-framework-proposal/9874/8) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/1bc61ec57d979a7b895d0b224abbf30e7711b87e66370864e10ce9ff04be1d52).