You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@tvm.apache.org by "Cody H. Yu via TVM Discuss" <no...@discuss.tvm.ai> on 2020/05/19 17:39:17 UTC

[TVM Discuss] [Development/RFC] [BYOC] Use pattern language to create composite functions


Since the pattern language has been merged, we are planning to rewrite `MergeComposite` pass for BYOC with it.

## Brief Introduction to Merge Composite Pass

BYOC is designed to help 3rd party codegen and accelerators integrate into TVM. Since 3rd party accelerator may have a specialized and powerful instruction to deal with multiple operators (e.g., conv2d+add+relu), the first pass in BYOC is to fuse those operators to a separate function and annotate the function with the instruction name. As a result, the external codegen can easily replace the entire function with a single instruction.

## Reasons to Use Pattern Language

The current implementation of `MergeComposite` pass accepts a user-specified map such as `{'my-inst': graph}`, where `graph` is a small Relay program used as a pattern. More detail use cases can be found in the unit test:

https://github.com/apache/incubator-tvm/blob/master/tests/python/relay/test_pass_merge_composite.py

On the other hand, using a Relay program as a pattern causes lots of limitations. Basically all motivations specified in the pattern language (https://discuss.tvm.ai/t/rfc-relay-program-matching-for-relay-pt-1-a-pattern-language/5833) are applicable.

## Proposal and POC

Accordingly, we propose to use pattern language for `MergeComposite`. It's simple, robust, and more general. For example, I've written a POC of `MergeComposite` in pattern language. As can be seen, we only need less than 100 lines to achieve the same functionality.

https://gist.github.com/comaniac/1f399dfdfee05a2f7a087c65c21f550c

The above POC includes two solutions.

#### Pattern.Partition

Thiis is a builtin functionality of pattern language that partitions the matched subgraph to a separate function. However, this builtin function doesn't allow users to specify function attributes, so we cannot add `Composite = inst`.

#### PatternCallback

Another solution uses callback functions to manually create composite functions. The problem with this solution is that we need an extra visit to mutate the Relay graph in order to create the function.

## Discussion

While we prefer the first solution that uses `pattern.partition()`, we need to figure out how to add the composite attribute.

#### S.1: Enhance Partition

A straightforward approach is enhancing `pattern.partition` to accept more configurations.

```python
for pattern, inst in patterns:
  out = pattern.partition(out, {'Composite': inst})
```

#### S.2: Post-Processing

If we do not want to change the partition, we could use a set know the new generated functions after each partition, and add the attribute to them:

```python
for pattern, inst in patterns:
  curr_funcs = get_func_as_set(out)
  out = pattern.partition(out)
  for func in out.functions:
     if func not in curr_funcs:
         func.with_attr('Composite', inst)
```

Any other thoughts and comments are welcome.

cc @mbrookhart @zhiics @masahi @matt-arm





---
[Visit Topic](https://discuss.tvm.ai/t/byoc-use-pattern-language-to-create-composite-functions/6727/1) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/51fffc57d5629553881b1628ccadbc29295abbb72d5e716c70aa6c71a0977b76).

[TVM Discuss] [Development/RFC] [BYOC] Use pattern language to create composite functions

Posted by Matthew Brookhart via TVM Discuss <no...@discuss.tvm.ai>.


While I wasn't paying attention to the discussion, I implemented Cody's original ask here:

https://github.com/apache/incubator-tvm/pull/5627

@masahi, what do you think of stringing together the types of all of the nodes/names of all of the called ops in the partitioned function, in topological order, and putting that as the default "Paritioned" attribute? I think that would do what you're asking for, but it would still allow the user to add their own tag if they want.





---
[Visit Topic](https://discuss.tvm.ai/t/byoc-use-pattern-language-to-create-composite-functions/6727/9) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/b2a20bee86555f1d679485d4b8b8dddba8595639a5f8d84736e094b307343f4d).

[TVM Discuss] [Development/RFC] [BYOC] Use pattern language to create composite functions

Posted by "Cody H. Yu via TVM Discuss" <no...@discuss.tvm.ai>.


@mbrookhart the PR is exactly I'm expecting!

I am also fine with the tags in a pattern. In this case, we can keep the infra simple and transparent tag analysis to external codegens.





---
[Visit Topic](https://discuss.tvm.ai/t/byoc-use-pattern-language-to-create-composite-functions/6727/11) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/a82c971a077f385f269b8977c85faf0781be4ea3b826c00212267033044cbe59).

[TVM Discuss] [Development/RFC] [BYOC] Use pattern language to create composite functions

Posted by tqchen via TVM Discuss <no...@discuss.tvm.ai>.


Given that the old composite pattern is  not yet part of a release, it might easier to directly migrate to the new one so that we don't have to maintain two variants of pattern languages in the spirit of reducing techinical debts.





---
[Visit Topic](https://discuss.tvm.ai/t/byoc-use-pattern-language-to-create-composite-functions/6727/15) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/1896c35f2cc164043eec0c212a1d0fd3f0c814948a8b104225fec889ec5b44d2).

[TVM Discuss] [Development/RFC] [BYOC] Use pattern language to create composite functions

Posted by masahi via TVM Discuss <no...@discuss.tvm.ai>.


We should have a utility that converts existing relay op based patterns to the new pattern language. Otherwise it would break existing users code and we need to rewrite merge composite test cases manually.





---
[Visit Topic](https://discuss.tvm.ai/t/byoc-use-pattern-language-to-create-composite-functions/6727/14) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/ffda088032c8c93a6919ce61144fbd33b21d87da0670b37a93e2dd7f1045b09a).

[TVM Discuss] [Development/RFC] [BYOC] Use pattern language to create composite functions

Posted by Matthew Brookhart via TVM Discuss <no...@discuss.tvm.ai>.


We'd still need something like this:

`use pattern language to match fused_conv, and let codegen decide whether it is conv+bias or conv+bias+relu when the codegen is parsing a composite function with fused_conv attribute.`

But it would be analyzing strings instead of graphs.





---
[Visit Topic](https://discuss.tvm.ai/t/byoc-use-pattern-language-to-create-composite-functions/6727/10) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/c9ea4f9c07a06b2b78fa19e706d91e3c403ccbcd95639026328aa5343c7b5f3f).

[TVM Discuss] [Development/RFC] [BYOC] Use pattern language to create composite functions

Posted by "Cody H. Yu via TVM Discuss" <no...@discuss.tvm.ai>.


New `MergeComposite` pass PR filed.

https://github.com/apache/incubator-tvm/pull/5656





---
[Visit Topic](https://discuss.tvm.ai/t/byoc-use-pattern-language-to-create-composite-functions/6727/29) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/bc65afbd2c2adcb95089c73ff6d13d710c4f506f2770049f33fbd32bf11a5335).

[TVM Discuss] [Development/RFC] [BYOC] Use pattern language to create composite functions

Posted by "Cody H. Yu via TVM Discuss" <no...@discuss.tvm.ai>.


> Do you envisage this having a similar API to the existing pass? In particular it’s important for us to retain the added ‘check’ function to confirm whether a match is valid.

Good point. We need to evaluate if the check function can be fully integrated into the pattern language.

@mbrookhart FYI, one use case for the check function we are talking about is:

https://github.com/apache/incubator-tvm/blob/master/tests/python/relay/test_pass_merge_composite.py#L754 

In short, it provides a packed function to match ops with specific attributes. I imagine this can be done by something like `is_op('nn.conv2d').has_attr(...)`, but I'm not sure how powerful `has_attr` can achieve.

> Looking forward slightly, I do wonder whether it will start making sense to instead of creating composite functions just directly insert the annotations around the match. This would give us an opportunity to link up ‘composite’ matches with single ops as it’s currently quite unintuitive that these two things are treated so differently.

Actually the plan Zhi and I originally have was similar to the one you are talking about. The concept of composite function should be merged to annotation target and the external codegen is in charge of replacing patterns with accelerator specific instructions.

> I think it would be good to know the pattern matcher has the flexibility to annotate rather than partition. However, that will be a more significant change to the overall BYOC flow so the priority first should be just replicating existing functionality as described in this RFC.

I agree. We can first make the composite pass with pattern language anyhow. The composite function is being processed by every passes in BYOC infra already and we need to more time to carefully figure out how to use annotations to replace composite functions.





---
[Visit Topic](https://discuss.tvm.ai/t/byoc-use-pattern-language-to-create-composite-functions/6727/19) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/bf40d8aa11ce69fdbf152b3bc97d43781fd1f387a991804cfac9c72d70c16671).

[TVM Discuss] [Development/RFC] [BYOC] Use pattern language to create composite functions

Posted by masahi via TVM Discuss <no...@discuss.tvm.ai>.


[quote="comaniac, post:8, topic:6727"]
I see your point, although I personally think in this case we should either 1) specify 4 patterns, or 2) use pattern language to match `fused_conv` , and let codegen decide whether it is conv+bias or conv+bias+relu when the codegen is parsing a composite function with `fused_conv` attribute.
[/quote]

Note that it is not 4 patterns, but currently we need 2^4 patterns with unique names. We also need to order them carefully so that a bigger patterns come first. Optional matching is introduced to mitigate this combinatorial mess.

I also realized that "a list of optional names" solution works only for the simplest case: if the pattern is tree-ish shaped or contains the same op twice and only one of them is optional, it doesn't work.





---
[Visit Topic](https://discuss.tvm.ai/t/byoc-use-pattern-language-to-create-composite-functions/6727/12) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/2a02d43ba32b26961a2c56a37f9c75d9200bcf140e01061d10c7277218cd9833).

[TVM Discuss] [Development/RFC] [BYOC] Use pattern language to create composite functions

Posted by "Cody H. Yu via TVM Discuss" <no...@discuss.tvm.ai>.


I see your point, although I personally think in this case we should either 1) specify 4 patterns, or 2) use pattern language to match `fused_conv`, and let codegen decide whether it is conv+bias or conv+bias+relu when the codegen is parsing a composite function with `fused_conv` attribute.

On the other hand, the scenario of associating a list of optional op names for each pattern is unclear to me. I imagine you need to specify "sub-patterns" in a pattern and assign a name to each of the sub-pattern. To me this is similar to specify 4 patterns directly.

Feel free to correct me if I lost the track, since pattern language is a brand new feature and I haven't fully hands on.





---
[Visit Topic](https://discuss.tvm.ai/t/byoc-use-pattern-language-to-create-composite-functions/6727/8) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/ef97f599f490645e8c9d1b87aa245f81a260d793cdf8eb5d41c3728df5beb273).

[TVM Discuss] [Development/RFC] [BYOC] Use pattern language to create composite functions

Posted by masahi via TVM Discuss <no...@discuss.tvm.ai>.


No, it is more complicated than that.

Currently our DNNL backends has two patterns, one for conv + relu and another for conv + bias + relu:

https://github.com/apache/incubator-tvm/blob/master/python/tvm/relay/op/contrib/dnnl.py#L84-L86

Suppose we use optional to represent the two patterns. We would have one pattern named "fused_conv", let's say, that can match both conv + bias and conv + bias + relu.

In the current infra, the DNNL backend would receive a composite function with attribute "fused_conv", and the subgraph is either conv + bias or conv + bias + relu, depending on whether bias is matched. To tell the difference, we need `IsOp(relu->args[0], "nn.bias_add")`. 

In my use case, I have four optionals to match "qconv + hswish", so I'd need four `IsOp(...)` calls. It would be really nice if the composite attribute tells me which optionals are matched ("conv_relu" or "conv_bias_relu", rather than "fused_conv").

A user can associate a list of optional op names for each pattern, and `MergeComposite` pass can compare the matched fragement with optional names to come up with a good attribute.





---
[Visit Topic](https://discuss.tvm.ai/t/byoc-use-pattern-language-to-create-composite-functions/6727/7) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/ad4478b2e009fb3cafc65d6a30b61a6b6d72ca6427229705d1c3c3a1dc0f37af).

[TVM Discuss] [Development/RFC] [BYOC] Use pattern language to create composite functions

Posted by Matt Barrett via TVM Discuss <no...@discuss.tvm.ai>.


I think it would be good to know the pattern matcher has the flexibility to annotate rather than partition. However, that will be a more significant change to the overall BYOC flow so the priority first should be just replicating existing functionality as described in this RFC.





---
[Visit Topic](https://discuss.tvm.ai/t/byoc-use-pattern-language-to-create-composite-functions/6727/18) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/f9289763b90c07d7352b07dbb415fa0d2d6226d1174a55a4bc75bb65389b63f6).

[TVM Discuss] [Development/RFC] [BYOC] Use pattern language to create composite functions

Posted by Matthew Brookhart via TVM Discuss <no...@discuss.tvm.ai>.


@comaniac @matt-arm @masahi

Sorry for the delay! The Meetup this morning ate more of my time than I expected.
   
https://github.com/apache/incubator-tvm/pull/5646





---
[Visit Topic](https://discuss.tvm.ai/t/byoc-use-pattern-language-to-create-composite-functions/6727/28) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/a0e73b5ca12dc98b00eefe2bc67e8bad6f205b80f486e5bfa567dd3e8eff1758).

[TVM Discuss] [Development/RFC] [BYOC] Use pattern language to create composite functions

Posted by Matthew Brookhart via TVM Discuss <no...@discuss.tvm.ai>.


[quote="matt-arm, post:13, topic:6727"]
Looking forward slightly, I do wonder whether it will start making sense to instead of creating composite functions just directly insert the annotations around the match. This would give us an opportunity to link up ‘composite’ matches with single ops as it’s currently quite unintuitive that these two things are treated so differently.
[/quote]

In the PR, we talked about adding a pass that can annotate pattern matches instead of partitioning them. I think it's easy-ish to do. I can start working on it if other people agree this is a good direction to go.





---
[Visit Topic](https://discuss.tvm.ai/t/byoc-use-pattern-language-to-create-composite-functions/6727/17) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/d9b933524f2baf3a8aa69c8e47c1a122022b1bfba141aac6bf535c0e36732b14).

[TVM Discuss] [Development/RFC] [BYOC] Use pattern language to create composite functions

Posted by Matthew Brookhart via TVM Discuss <no...@discuss.tvm.ai>.


[quote="matt-arm, post:13, topic:6727"]
Do you envisage this having a similar API to the existing pass? In particular it’s important for us to retain the added ‘check’ function to confirm whether a match is valid.
[/quote]


@matt-arm Can you point me at the API? I have something like this internal to the Parition Pass to validated Function creation, but I haven't exposed it.





---
[Visit Topic](https://discuss.tvm.ai/t/byoc-use-pattern-language-to-create-composite-functions/6727/16) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/f83ca7aba4d6ae8bcbda84a022b595721ea6fcde58d21c5faba88dd7944b5dbc).

[TVM Discuss] [Development/RFC] [BYOC] Use pattern language to create composite functions

Posted by Matt Barrett via TVM Discuss <no...@discuss.tvm.ai>.


Thanks for this RFC Cody! I fully agree this pass should be re-implemented using the pattern language, the logic was very much a stop-gap solution until more powerful pattern matching appeared.

Do you envisage this having a similar API to the existing pass? In particular it's important for us to retain the added 'check' function to confirm whether a match is valid. 

Looking forward slightly, I do wonder whether it will start making sense to instead of creating composite functions just directly insert the annotations around the match. This would give us an opportunity to link up 'composite' matches with single ops as it's currently quite unintuitive that these two things are treated so differently.





---
[Visit Topic](https://discuss.tvm.ai/t/byoc-use-pattern-language-to-create-composite-functions/6727/13) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/93f40307ef07673b09bc508439cb9d2a22b46ae4d715b0afff3f1ee29a21adc2).

[TVM Discuss] [Development/RFC] [BYOC] Use pattern language to create composite functions

Posted by "Cody H. Yu via TVM Discuss" <no...@discuss.tvm.ai>.


Thanks for the comments. Here are a short summary of supporting `check` using pattern language based on my POC (https://gist.github.com/comaniac/1f399dfdfee05a2f7a087c65c21f550c):

#### A. Callback
The `check` logic is used in callbacks to determine if this subgraph really matches the pattern. If matching, the callback needs to create a composite function as the POC I provided.
* User API would be similar as now. Specifically, user provides the following inputs to `PatternCallback`:
    1. A pattern.
    2. A map of check functions for this pattern (e.g., `{'nn.conv2d': lambda ...: true}`).
* A redundant visit to the matched subgraph in order to create the composite function seems inevitable.

#### B. A More Powerful but Complex Pattern Logic
The `check` logic is implemented along with the pattern using `AttrPattern`.
* User API would be just a pattern.
* We can use `pattern.partition()` to easily create composite functions.
* The logic of implementing a pattern might be more complex, implying a more steep learning curve.

Here are my two cents. Since we already have `pattern.partition()`, it seems not practical to improve the rewriter API to make the partition efficient. As a result, I prefer solution **B**. On the other hand, I am not sure how complex this logic would be and if it's reasonable for 3rd-party backend developers to learn and implement. It would be great if @mbrookhart you could provide an illustration or even an RFC to give us a more concrete idea :)





---
[Visit Topic](https://discuss.tvm.ai/t/byoc-use-pattern-language-to-create-composite-functions/6727/21) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/c089294d535f0fc5dab6b1040c5836324306ba713bef19805fd8fd8b034ab986).

[TVM Discuss] [Development/RFC] [BYOC] Use pattern language to create composite functions

Posted by Matthew Brookhart via TVM Discuss <no...@discuss.tvm.ai>.


I see. This is easily doable with the Rewriter API, since you can do an arbitrary callback, but it doesn't have the niceties of the Parition pass.

The current AttrPattern is only matching Op attributes, but that's fairly easy to extend.

I think I prefer doing this with the AttrPattern, it's a simpler API. That being said, an arbitrary callback will always be more flexible. Do you guys have a strong preference toward a callback mechanism or a usecase with more complex logic?





---
[Visit Topic](https://discuss.tvm.ai/t/byoc-use-pattern-language-to-create-composite-functions/6727/20) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/380792f9583bf5a17f32c94ebf2a93bbff80598016215f4810415b0efd6466ad).

[TVM Discuss] [Development/RFC] [BYOC] Use pattern language to create composite functions

Posted by "Cody H. Yu via TVM Discuss" <no...@discuss.tvm.ai>.


I was mainly referring to this case https://github.com/apache/incubator-tvm/pull/5637/files#diff-f9920485e5e341787129348ce1985db9R247-R251 and it does achieve the same functionality as the current `check` functions in `MergeComposite` pass in my opinion.

In terms of ease-of-use, to me this programming model is even more straightforward than the `check` functions.

@matt-arm @masahi what do you guys think?





---
[Visit Topic](https://discuss.tvm.ai/t/byoc-use-pattern-language-to-create-composite-functions/6727/23) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/bdb861428bfc3636f06f77807b73bde02e2fec070dadfbb4cf7b543fc384c564).

[TVM Discuss] [Development/RFC] [BYOC] Use pattern language to create composite functions

Posted by "Cody H. Yu via TVM Discuss" <no...@discuss.tvm.ai>.


Pattern language infra works based on a pattern, so we will know which subgraph is matched by which pattern.

For example, when we use `graph = pattern.partition(graph)`, the new partitioned functions are based on the pattern. As long as we add the composite attribute with this pattern's name to those functions, we are at the same point as the current `MergeComposite` pass. In short, the backend will not see any `IsOp` kinds of stuffs.





---
[Visit Topic](https://discuss.tvm.ai/t/byoc-use-pattern-language-to-create-composite-functions/6727/6) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/703f6501cbd93ba0c2e6a2104e5e4d6603e11ab6ca4f8554f1d79fce6a06f0c1).

[TVM Discuss] [Development/RFC] [BYOC] Use pattern language to create composite functions

Posted by masahi via TVM Discuss <no...@discuss.tvm.ai>.


One thing that's not obvious to me is how we should interprete the result of optional matching. Ideally there should be an easy way to tell which of optionals are actually matched, for example by coming up with an appropriate value for composite attribute. 

Otherwise each backend needs to manually examine a fragment it receives by multiple calls to `IsOp(call->args[0], ...)`.





---
[Visit Topic](https://discuss.tvm.ai/t/byoc-use-pattern-language-to-create-composite-functions/6727/5) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/8e50fd99b2f9c2e8c2ba8253061952aeda1e8bc34a662058a9054281f2152491).

[TVM Discuss] [Development/RFC] [BYOC] Use pattern language to create composite functions

Posted by "Cody H. Yu via TVM Discuss" <no...@discuss.tvm.ai>.


Updated the gist with merged features (configurable partition function attributes; pattern with more powerful attributes).

Once the last feature (partition callback) is in, we should be fine to rewrite `MergeComposite` pass without any loss.

https://gist.github.com/comaniac/1f399dfdfee05a2f7a087c65c21f550c





---
[Visit Topic](https://discuss.tvm.ai/t/byoc-use-pattern-language-to-create-composite-functions/6727/27) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/2b6341d58d258870825dfc1bfa1d322e3c00185544719786d2bd98b0094ff98e).

[TVM Discuss] [Development/RFC] [BYOC] Use pattern language to create composite functions

Posted by Matthew Brookhart via TVM Discuss <no...@discuss.tvm.ai>.


Hmm. Okay. I'll add a check callback to the partition pass.

I think that leaves us with three levels of possible user control in level of priority:
1. Use the Pattern Language for any constraints possible. We might be able to add shape checking to the language, we can't use an external codegen type checker :slight_smile: 
2. If you need more control, use the Partition check if there are more complicated/external constraints
3. if you need even more control, use the Rewriter pass for more complicated graph rewriting.





---
[Visit Topic](https://discuss.tvm.ai/t/byoc-use-pattern-language-to-create-composite-functions/6727/25) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/ed968ef16932ab0a9d7b3af17a4e15e51a19c66d041cc0ca9b7256220923857d).

[TVM Discuss] [Development/RFC] [BYOC] Use pattern language to create composite functions

Posted by Matt Barrett via TVM Discuss <no...@discuss.tvm.ai>.


For our codegen, the check functions are actually quite complex. They're not just attribute checks, there's also things relating to tensor sizes, number of dimensions and quantisation parameters. Additionally, our codegen library exposes an API to check whether a given operator is supported for a particular set of parameters and we query that function in the check. Therefore, in my view the check function needs to be able to support an arbitrary packed function.





---
[Visit Topic](https://discuss.tvm.ai/t/byoc-use-pattern-language-to-create-composite-functions/6727/24) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/8665f626649d270a0ddde9e501f1476ffcd7f077981972ba6cd1874063407bbd).

[TVM Discuss] [Development/RFC] [BYOC] Use pattern language to create composite functions

Posted by Matthew Brookhart via TVM Discuss <no...@discuss.tvm.ai>.


The currently implementation wont, thinking over it more, but I'll work on an update that adds that functionality and a user-specified set of attributes





---
[Visit Topic](https://discuss.tvm.ai/t/byoc-use-pattern-language-to-create-composite-functions/6727/4) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/415f42675d6d11d430dfd25750911b84844577c5b55885ec3f25efb577ab9bf9).

[TVM Discuss] [Development/RFC] [BYOC] Use pattern language to create composite functions

Posted by "Cody H. Yu via TVM Discuss" <no...@discuss.tvm.ai>.


Thanks for pointing out. I missed this part in the original post. Currently `MergeComposite` pass doesn't deal with the overlapping issue but let users decide the pattern priority. It means that when performing pattern matching, it should ignore existing composite functions. Is this functionality can also be achieved using `pattern.partition()`?





---
[Visit Topic](https://discuss.tvm.ai/t/byoc-use-pattern-language-to-create-composite-functions/6727/3) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/4387f64111a288fb2a387533fe412e40dd958bce17dc3731ccf8b243e494c89c).

[TVM Discuss] [Development/RFC] [BYOC] Use pattern language to create composite functions

Posted by Matthew Brookhart via TVM Discuss <no...@discuss.tvm.ai>.


I'd be happy to extend the parition pass to accept an optional map of attributes. I think it's a relatively small change to the pass, if you guys think that would be the easiest option.

Something I thought of over the last few days: If some patterns in your list are larger (say conv-bias-relu), and later patterns in the list are subgraphs of previous patterns (say conv-bias), you may end up with a case where you have a composite function called from inside another composite function, which is probably not the goal.

It might be a good idea to introduce a default "Partitioned" attribute and not re-partition inside the "Partitioned" functions in later passes.

CC @jroesch @tqchen





---
[Visit Topic](https://discuss.tvm.ai/t/byoc-use-pattern-language-to-create-composite-functions/6727/2) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/83483bc78911591307a46f0608f05d482761d27020ed9ac75d9821d44e861c64).

[TVM Discuss] [Development/RFC] [BYOC] Use pattern language to create composite functions

Posted by "Cody H. Yu via TVM Discuss" <no...@discuss.tvm.ai>.


@mbrookhart this sounds great!

Then I can use the second method to implement `MergeComposite` to allow optional check functions as it currently does. Meanwhile, if a user's pattern only needs to check attributes and shapes, the user can use the first method to specify the pattern.





---
[Visit Topic](https://discuss.tvm.ai/t/byoc-use-pattern-language-to-create-composite-functions/6727/26) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/daf70eb16cf13d6aa0ab6388ebf9de348a9b3951a39da5acde08f932f20689f8).

[TVM Discuss] [Development/RFC] [BYOC] Use pattern language to create composite functions

Posted by Matthew Brookhart via TVM Discuss <no...@discuss.tvm.ai>.


@comaniac @matt-arm

I've extending the AttrPattern to handle attributes of CallNodes and FunctionNodes here: https://github.com/apache/incubator-tvm/pull/5637

Would you mind taking a look and letting me know if you think that would be sufficiently powerful for solution B?





---
[Visit Topic](https://discuss.tvm.ai/t/byoc-use-pattern-language-to-create-composite-functions/6727/22) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/9ca1122456b70d81b95e19067047fea76615cd3571028153d761d9c8d34fdd8b).