You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2023/01/09 09:51:54 UTC
[GitHub] [tvm] Alexey-Yazev opened a new pull request, #13732: [microNPU] Add support for TFLite PAD
Alexey-Yazev opened a new pull request, #13732:
URL: https://github.com/apache/tvm/pull/13732
A separate nn.pad relay operator is legalized to an Ethos-U depthwise_conv2d operator. For ethosu_depthwise_conv2d the hardware only supports padding up to 31, 31, 32, 32, 32, so the pad size for legalization on the NPU is within these limits.
cc @leandron, @ekalda, @lhutton1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] arina-grovety commented on pull request #13732: [microNPU] Add support for TFLite PAD
Posted by GitBox <gi...@apache.org>.
arina-grovety commented on PR #13732:
URL: https://github.com/apache/tvm/pull/13732#issuecomment-1377029980
Hello @lhutton1, thanks for the review!
> * With the current implementation `nn.pad` does not get offloaded if the provided padding exceeds [31, 31, 32, 32]. If these dimensions are exceeded, we might be able to use multiple average pooling operations similar to https://git.mlplatform.org/ml/ethos-u/ethos-u-vela.git/tree/ethosu/vela/tflite_graph_optimiser.py#n1500
Yes, this was the first option that we tried to implement. But in the Vela implementation, this is done by **"copies IFM to the right place inside the OFM"** using **write_offset** attribute of the created AvgPool operation.
In the TVM, VelaAPI operations are derived from the NpuOperation class, which does not have a write_offset attribute, so we cannot replicate Vela convert_pad() function.
We tried to implement PAD legalization using the Concatenate operation but encountered an error. Seems like Cascader must be turned off for Concatenate to work. For example, Cascader is disabled in test_tflite_concat() (if Cascader is enabled, there is the same error as we have with the Concatenate)
So far, the most feasible option seems to use several depthwise_conv2d operators, if padding exceeds [31, 31, 32, 32].
But of course, I do not have all the knowledge about this, maybe there are other options?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] lhutton1 merged pull request #13732: [microNPU] Add support for TFLite PAD
Posted by GitBox <gi...@apache.org>.
lhutton1 merged PR #13732:
URL: https://github.com/apache/tvm/pull/13732
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] lhutton1 commented on pull request #13732: [microNPU] Add support for TFLite PAD
Posted by GitBox <gi...@apache.org>.
lhutton1 commented on PR #13732:
URL: https://github.com/apache/tvm/pull/13732#issuecomment-1376339728
Thanks @Alexey-Yazev, @ekalda!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] ekalda commented on a diff in pull request #13732: [microNPU] Add support for TFLite PAD
Posted by GitBox <gi...@apache.org>.
ekalda commented on code in PR #13732:
URL: https://github.com/apache/tvm/pull/13732#discussion_r1064825326
##########
python/tvm/relay/backend/contrib/ethosu/legalize.py:
##########
@@ -1355,6 +1355,62 @@ def callback(self, pre, post, node_map):
return ethosu_fc
+class PadRewriter(DFPatternCallback):
+ """Convert ethos-u.pad2d composite function to ethosu_depthwise_conv2d
+ operator"""
+
+ def __init__(self):
+ super().__init__(require_type=True)
+ self.pattern = (
+ wildcard().has_attr({"Composite": ethosu_patterns.PadParams.composite_name})
+ )(wildcard())
+
+ def callback(
+ self, pre: tvm.relay.Expr, post: tvm.relay.Expr, node_map: tvm.ir.container.Map
+ ) -> tvm.relay.Expr:
+ params = ethosu_patterns.PadParams(post.op.body)
+ params.ifm.tensor = post.args[0]
+ channels_map = {
+ "NHWC": 3,
+ }
Review Comment:
IIRC this one entry `channels_map` is a historic relic that can go and so simplify the code little bit, but as it is present in other operators, it's probably a clean up task for some other time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tvm-bot commented on pull request #13732: [microNPU] Add support for TFLite PAD
Posted by GitBox <gi...@apache.org>.
tvm-bot commented on PR #13732:
URL: https://github.com/apache/tvm/pull/13732#issuecomment-1375351744
<!---bot-comment-->
Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from [Reviewers](https://github.com/apache/incubator-tvm/blob/master/CONTRIBUTORS.md#reviewers) by @-ing them in a comment.
<!--bot-comment-ccs-start-->
* cc @Mousius, @leandron, @lhutton1 <sub>See [#10317](https://github.com/apache/tvm/issues/10317) for details</sub><!--bot-comment-ccs-end-->
<sub>Generated by [tvm-bot](https://github.com/apache/tvm/blob/main/ci/README.md#github-actions)</sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] lhutton1 commented on pull request #13732: [microNPU] Add support for TFLite PAD
Posted by GitBox <gi...@apache.org>.
lhutton1 commented on PR #13732:
URL: https://github.com/apache/tvm/pull/13732#issuecomment-1376371642
Oops I forgot to ask, would it be possible to add a legalization test under: `tests/python/contrib/test_ethosu/test_legalize.py` in a follow-up? Apologies for missing this before!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] lhutton1 commented on pull request #13732: [microNPU] Add support for TFLite PAD
Posted by GitBox <gi...@apache.org>.
lhutton1 commented on PR #13732:
URL: https://github.com/apache/tvm/pull/13732#issuecomment-1380066522
Thanks @arina-grovety for the explanation, just following up on some of the questions...
I suspect this is a case of needing to expose this functionality from within Vela, I'll see if we can make this happen for a future Vela release.
The concatenate error does indeed sound like a separate issue in itself. It might be worth investigating the reason for that at some point.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] Alexey-Yazev commented on pull request #13732: [microNPU] Add support for TFLite PAD
Posted by GitBox <gi...@apache.org>.
Alexey-Yazev commented on PR #13732:
URL: https://github.com/apache/tvm/pull/13732#issuecomment-1377280850
> Oops I forgot to ask, would it be possible to add a legalization test under: tests/python/contrib/test_ethosu/test_legalize.py in a follow-up? Apologies for missing this before!
Test was added in [PR](https://github.com/apache/tvm/pull/13750)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org