[TVM Discuss] [Questions] What to do with passes

[Thomas V via TVM Discuss <no...@discuss.tvm.ai>]

I have a small collection of relay passes that I found useful and that might be interesting for more general use:
- Merge consecutive transpose ops (useful for converted models when the other source framework had different conventions, I'm using PyTorch),
- merge equal (shape) constants to enable CSE (also for converted models, but probably also our own),
- specialize `_like` ops for static shapes to `_to` (and similar) ops (broadcast, collapse_sum, zeros, ones, reshape) - useful after taking gradients because can reduce the complexity of the graph immensely. Removing broadcast/collapse_sum when not needed,
- remove *1 and +0 (with tensor ones and zeros, these appear in gradients),
- spell out training LayerNorm (batch norm etc. could be done, too).

I implemented these (mostly) using the PatternMatching in Python. Moving them to TVM likely involves going to C++.

Are these useful and if so,  would we want to automatically include them in optimization (and how)?

Best regards

Thomas





---
[Visit Topic](https://discuss.tvm.ai/t/what-to-do-with-passes/7147/1) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/fd1b806a2e1f6c841207676c29e87e90489307fc0422e5e03fa07a582448565f).