You are viewing a plain text version of this content. The canonical link for it is here.
Posted to discuss-archive@tvm.apache.org by moderato via Apache TVM Discuss <no...@discuss.tvm.ai> on 2020/10/26 08:25:51 UTC
[Apache TVM Discuss] [Questions] How to detect the pattern of a
function?
def @main(%data: Tensor[(1, 112, 112, 32), float32]) -> Tensor[(1, 112, 112, 64), float32] {
%3 = fn (%p0: Tensor[(1, 112, 112, 32), float32], %p1: Tensor[(3, 3, 32, 1), float32], %p2: Tensor[(1, 1, 1, 32), float32], %p3: Tensor[(1, 1, 1, 32), float32], Primitive=1) -> Tensor[(1, 112, 112, 32), float32] {
%0 = nn.conv2d(%p0, %p1, padding=[1, 1, 1, 1], groups=32, channels=32, kernel_size=[3, 3], data_layout="NHWC", kernel_layout="HWIO") /* ty=Tensor[(1, 112, 112, 32), float32] */;
%1 = multiply(%0, %p2) /* ty=Tensor[(1, 112, 112, 32), float32] */;
%2 = add(%1, %p3) /* ty=Tensor[(1, 112, 112, 32), float32] */;
nn.relu(%2) /* ty=Tensor[(1, 112, 112, 32), float32] */
};
%4 = %3(%data, meta[relay.Constant][0] /* ty=Tensor[(3, 3, 32, 1), float32] */ /* ty=Tensor[(3, 3, 32, 1), float32] */, meta[relay.Constant][1] /* ty=Tensor[(1, 1, 1, 32), float32] */ /* ty=Tensor[(1, 1, 1, 32), float32] */, meta[relay.Constant][2] /* ty=Tensor[(1, 1, 1, 32), float32] */ /* ty=Tensor[(1, 1, 1, 32), float32] */) /* ty=Tensor[(1, 112, 112, 32), float32] */;
%8 = fn (%p01: Tensor[(1, 112, 112, 32), float32], %p11: Tensor[(1, 1, 32, 64), float32], %p21: Tensor[(1, 1, 1, 64), float32], %p31: Tensor[(1, 1, 1, 64), float32], Primitive=1) -> Tensor[(1, 112, 112, 64), float32] {
%5 = nn.conv2d(%p01, %p11, padding=[0, 0, 0, 0], channels=64, kernel_size=[1, 1], data_layout="NHWC", kernel_layout="HWIO") /* ty=Tensor[(1, 112, 112, 64), float32] */;
%6 = multiply(%5, %p21) /* ty=Tensor[(1, 112, 112, 64), float32] */;
%7 = add(%6, %p31) /* ty=Tensor[(1, 112, 112, 64), float32] */;
nn.relu(%7) /* ty=Tensor[(1, 112, 112, 64), float32] */
};
%8(%4, meta[relay.Constant][3] /* ty=Tensor[(1, 1, 32, 64), float32] */ /* ty=Tensor[(1, 1, 32, 64), float32] */, meta[relay.Constant][4] /* ty=Tensor[(1, 1, 1, 64), float32] */ /* ty=Tensor[(1, 1, 1, 64), float32] */, meta[relay.Constant][5] /* ty=Tensor[(1, 1, 1, 64), float32] */ /* ty=Tensor[(1, 1, 1, 64), float32] */) /* ty=Tensor[(1, 112, 112, 64), float32] */
}
Hello, I'm just start learning how to use Relay's pattern matching. I wonder if there's a way I can match a function pattern (together with its internal pattern, e.g. `conv+mul+add+relu`), just like the two shown in the above example? Thanks in advance!
---
[Visit Topic](https://discuss.tvm.apache.org/t/how-to-detect-the-pattern-of-a-function/8283/1) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/5399e2a8eed927aceeea834ed43a678d29b51523cec87633b4cef221a7c97e86).
[Apache TVM Discuss] [Questions] How to match the pattern of a
function in Relay?
Posted by moderato via Apache TVM Discuss <no...@discuss.tvm.ai>.
Thanks! I'll try that after I fix up some of the bugs I'm having.
---
[Visit Topic](https://discuss.tvm.apache.org/t/how-to-match-the-pattern-of-a-function-in-relay/8283/19) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/90e7cae60731febd16ea5d1be753da09e10b73cab49b00685c11aaa1e25c81f8).
[Apache TVM Discuss] [Questions] How to match the pattern of a
function in Relay?
Posted by Matthew Brookhart via Apache TVM Discuss <no...@discuss.tvm.ai>.
The way the API is written, the matcher doesn't traverse you're input graph, it will return false unless the exact node you supply matches the pattern.
The partition and rewriting functions, however, will traverse the relay expression, go into functions, and find the match inside your function. If you partition instead of matching, you should see the extraction of your pattern into a function.
---
[Visit Topic](https://discuss.tvm.apache.org/t/how-to-match-the-pattern-of-a-function-in-relay/8283/18) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/533bc1063e381edfe03d8e558e6df64b74b70e0bfd9b95220223bad330f65af5).
[Apache TVM Discuss] [Questions] How to match the pattern of a
function in Relay?
Posted by moderato via Apache TVM Discuss <no...@discuss.tvm.ai>.
Thanks! I had checked that out, but seems it doesn't show a way to match a function. In my case conv+mul+add+relu is already wrapped into a function, so I failed to match them directly. One example in the tutorial related to function matching uses function attr, but it looks like the function I have above has a None attr.
Any thoughts?
---
[Visit Topic](https://discuss.tvm.apache.org/t/how-to-match-the-pattern-of-a-function-in-relay/8283/3) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/c6b38494c9ccd331ca4454ddde43948446191d63db5ca7fd2b962ce2c5306f02).
[Apache TVM Discuss] [Questions] How to match the pattern of a
function in Relay?
Posted by "Cody H. Yu via Apache TVM Discuss" <no...@discuss.tvm.ai>.
Check this out: https://tvm.apache.org/docs/langref/relay_pattern.html
---
[Visit Topic](https://discuss.tvm.apache.org/t/how-to-match-the-pattern-of-a-function-in-relay/8283/2) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/233f8ca21947a4ccabca500c412bdde493b419690286d3b2e094a6f504f6c183).
[Apache TVM Discuss] [Questions] How to match the pattern of a
function in Relay?
Posted by Matt Barrett via Apache TVM Discuss <no...@discuss.tvm.ai>.
I'd like to be able to rewrite a composite function into a single op. Obviously can be done with an ExprMutator but it'd be more convenient if the pattern language could do it.
---
[Visit Topic](https://discuss.tvm.apache.org/t/how-to-match-the-pattern-of-a-function-in-relay/8283/9) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/9ad4b469a5f3ac864b5908da0d566081ea62528663e6ccd437d6e90589be03fb).
[Apache TVM Discuss] [Questions] How to match the pattern of a
function in Relay?
Posted by moderato via Apache TVM Discuss <no...@discuss.tvm.ai>.
Thank you all for the discussion!
When a module is built, do the pattern matching and the MergeComposite pass currently happen before FuseOps or after? Is it possible to make it configurable, as in some use cases (like mine) the matching and merging takes the results of some previous passes, e.g. SimplifyInference, etc, as the inputs?
---
[Visit Topic](https://discuss.tvm.apache.org/t/how-to-match-the-pattern-of-a-function-in-relay/8283/13) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/34f1f949772802730072d9c66eef9ebfb1a22d13f1f9b21dd087a8a2f323877c).
[Apache TVM Discuss] [Questions] How to match the pattern of a
function in Relay?
Posted by "Cody H. Yu via Apache TVM Discuss" <no...@discuss.tvm.ai>.
Thanks @mbrookhart @matt-arm for the valuable information! Here is a short summary of this issue.
- The problem OP has should be resolved by simply matching and partitioning patterns before `FuseOps`.
- In addition to that, it would still be better to have `FunctionPattern` that matches and rewrites Relay functions.
I'm interested in adding the node but I may not have bandwidth in these days either. @mbrookhart I could ping you when I get a chance to work on it to avoid duplicated work :)
---
[Visit Topic](https://discuss.tvm.apache.org/t/how-to-match-the-pattern-of-a-function-in-relay/8283/10) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/a7fc6948b8927a9ea2406f5126f229250d4e37c31b8d9f2824e97605e21e1010).
[Apache TVM Discuss] [Questions] How to match the pattern of a
function in Relay?
Posted by moderato via Apache TVM Discuss <no...@discuss.tvm.ai>.
@comaniac OK I see. I think FunctionPattern would solve it all for me here. Glad to see the team has a plan on this!
@mbrookhart
[quote="mbrookhart, post:5, topic:8283, full:true"]
It will definitely go inside the function to match patterns, but you're right, we don't have a Function Pattern right now, we should probably add one.
This seems to be a function created by the FuseOps pass. Typically we'd do pattern rewriting/partitioning before that, maybe there's a simpler way to get what you're looking for?
[/quote]
I have tried using this to match
```
pattern = is_op('nn.conv2d')(wildcard(), wildcard())
pattern = is_op('multiply')(pattern, wildcard())
pattern = is_op('add')(pattern, wildcard())
tuple_get_item_node = TupleGetItemPattern(pattern, 0)
pattern = is_op('nn.relu')(tuple_get_item_node)
```
but it returns false. Looks like it doesn't go inside the function. I also tried disabling FuseOps and insert
MergeComposite after Optimize() has finished, but it throws an error saying something like "`add` is detected and will be removed by future passes."
Just to see if there's any luck here. Is there a way to get it work with the current code base?
---
[Visit Topic](https://discuss.tvm.apache.org/t/how-to-match-the-pattern-of-a-function-in-relay/8283/15) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/5879ca7f328d363d2931553693b51f73a3891bc1c610ce843130a48d2a9b6d44).
[Apache TVM Discuss] [Questions] How to match the pattern of a
function in Relay?
Posted by Matthew Brookhart via Apache TVM Discuss <no...@discuss.tvm.ai>.
It will definitely go inside the function to match patterns, but you're right, we don't have a Function Pattern right now, we should probably add one.
This seems to be a function created by the FuseOps pass. Typically we'd do pattern rewriting/partitioning before that, maybe there's a simpler way to get what you're looking for?
---
[Visit Topic](https://discuss.tvm.apache.org/t/how-to-match-the-pattern-of-a-function-in-relay/8283/5) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/6af1cec6d50a0514e0a6a23aab08685d832184693246ea65daa47e75ed8de39c).
[Apache TVM Discuss] [Questions] How to match the pattern of a
function in Relay?
Posted by moderato via Apache TVM Discuss <no...@discuss.tvm.ai>.
Yeah... You're right. I copied it from conv+bn+relu pattern and forgot to delete the line for tuple. But removing it still doesn't make it work though.
---
[Visit Topic](https://discuss.tvm.apache.org/t/how-to-match-the-pattern-of-a-function-in-relay/8283/17) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/73c36efe8ecc224f63c63f04f94966e1ad8312ca1ac9471b42df3ad81013fd92).
[Apache TVM Discuss] [Questions] How to match the pattern of a
function in Relay?
Posted by "Cody H. Yu via Apache TVM Discuss" <no...@discuss.tvm.ai>.
Why your pattern has a tuple node? It seems to me that it tries to match
```
%0 = nn.conv2d(%p0, %p1, padding=[1, 1, 1, 1], groups=32, channels=32, kernel_size=[3, 3], data_layout="NHWC", kernel_layout="HWIO") /* ty=Tensor[(1, 112, 112, 32), float32] */;
%1 = multiply(%0, %p2);
%2 = add(%1, %p3);
%3 = %2.0; /* This looks incorrect? */
nn.relu(%3)
```
---
[Visit Topic](https://discuss.tvm.apache.org/t/how-to-match-the-pattern-of-a-function-in-relay/8283/16) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/9ef30f2ae0ada32d342d0622ff8c18452eed055de9a337c2b7ef76a9f3ceb32e).
[Apache TVM Discuss] [Questions] How to match the pattern of a
function in Relay?
Posted by "Cody H. Yu via Apache TVM Discuss" <no...@discuss.tvm.ai>.
It should be configurable but depends on your use case. You could first figure out when FuseOps is invoked (and by which API) in your script, and we can see if that makes sense to run pattern matching and merge composite beforehand, Otherwise, you may still need FunctionPattern.
---
[Visit Topic](https://discuss.tvm.apache.org/t/how-to-match-the-pattern-of-a-function-in-relay/8283/14) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/14c9117625ed95684889743ae555a07d2ece372c11fe2d8a9fe05061815842e3).
[Apache TVM Discuss] [Questions] How to match the pattern of a
function in Relay?
Posted by cronos via Apache TVM Discuss <no...@discuss.tvm.ai>.
I also +1 this feature.
It seems that what you want to do is very similar to a previous post I had https://discuss.tvm.apache.org/t/relay-pattern-replacing-with-custom-relay-ops/7531/13
The biggest fear I have of introducing a new Relay operator (which I guess is what would eventually happen if we follow this path) was that it would have to work with all the Relay transformation passes.
The example in the TVM documentation https://tvm.apache.org/docs/dev/relay_add_op.html seems to trivial and not sufficient to answer this question.
Any guidance you could give would be gladly received :slight_smile:
---
[Visit Topic](https://discuss.tvm.apache.org/t/how-to-match-the-pattern-of-a-function-in-relay/8283/12) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/a0c15d76339ea9301e6a4ec961840dbf98fc699659375ffd78273ab6eec221cf).
[Apache TVM Discuss] [Questions] How to match the pattern of a
function in Relay?
Posted by aca88 via Apache TVM Discuss <no...@discuss.tvm.ai>.
I have to agree with @mbrookhart with his statement that pattern rewriting/partitioning comes before FuseOps. Could you provide more information as to why this would be useful?
---
[Visit Topic](https://discuss.tvm.apache.org/t/how-to-match-the-pattern-of-a-function-in-relay/8283/8) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/7e71684ccbf561350b1d31d7a6da0ddb3238d3e88b76219ffeafa837527241cd).
[Apache TVM Discuss] [Questions] How to match the pattern of a
function in Relay?
Posted by Matt Barrett via Apache TVM Discuss <no...@discuss.tvm.ai>.
+1 for this feature - I have some use-cases where it would be valuable to match composite functions.
---
[Visit Topic](https://discuss.tvm.apache.org/t/how-to-match-the-pattern-of-a-function-in-relay/8283/6) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/c4218d29f76bd3834a7e93614ed0dcfd40cf85550e91424edccdbaaf29c13c99).
[Apache TVM Discuss] [Questions] How to match the pattern of a
function in Relay?
Posted by "Cody H. Yu via Apache TVM Discuss" <no...@discuss.tvm.ai>.
Hmm, I’m not quite sure if pattern matcher will go into Relay functions for matching. I’ll check it later but maybe @mbrookhart could comment.
Meanwhile, maybe we can make a new FunctionPattern that matches function nodes.
---
[Visit Topic](https://discuss.tvm.apache.org/t/how-to-match-the-pattern-of-a-function-in-relay/8283/4) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/812c90f346669d6da5449a3e29f57fdc46d7af85ada89ab2df260089ad4339a0).
[Apache TVM Discuss] [Questions] How to match the pattern of a
function in Relay?
Posted by Matthew Brookhart via Apache TVM Discuss <no...@discuss.tvm.ai>.
Like a pattern that exists across multiple functions? Yes, we'll need a FunctionPattern.
---
[Visit Topic](https://discuss.tvm.apache.org/t/how-to-match-the-pattern-of-a-function-in-relay/8283/21) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/0c9dadb384c9f0208a0aea0cd2804ccc4651f5d4bf9775a12e80f96d886e4c72).
[Apache TVM Discuss] [Questions] How to match the pattern of a
function in Relay?
Posted by moderato via Apache TVM Discuss <no...@discuss.tvm.ai>.
@mbrookhart So I tried the way you suggested and I'm able to rewrite the pattern inside a function. I wonder if it's also possible to partition and rewrite a pattern across multiple functions? I suspect if this would need the support of the potential `FunctionPattern`.
---
[Visit Topic](https://discuss.tvm.apache.org/t/how-to-match-the-pattern-of-a-function-in-relay/8283/20) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/09b6a887a143d06a79de4a38ac67318ec9cc8c402f2b2101515a4bf0b36e7288).
[Apache TVM Discuss] [Questions] How to match the pattern of a
function in Relay?
Posted by Matthew Brookhart via Apache TVM Discuss <no...@discuss.tvm.ai>.
I'm happy to add it, but it will be a couple of days before I can get to it. Any one else interested in adding the node and some matching tests?
---
[Visit Topic](https://discuss.tvm.apache.org/t/how-to-match-the-pattern-of-a-function-in-relay/8283/7) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/7e66ab050956ade71029e51c9a80632f10ab651a85a78acb0edd010be028f09e).
[Apache TVM Discuss] [Questions] How to match the pattern of a
function in Relay?
Posted by Matthew Brookhart via Apache TVM Discuss <no...@discuss.tvm.ai>.
@comaniac sounds good, we'll see who gets there first :)
---
[Visit Topic](https://discuss.tvm.apache.org/t/how-to-match-the-pattern-of-a-function-in-relay/8283/11) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/f570a6d6a68070b11960c8125a6cf645d08bdb7f02554e414298afcf1f5ec8e5).