[GitHub] [tvm] lhutton1 commented on a change in pull request #9209: Arm(R) Ethos(TM)-U NPU Depthwise2d operator support

[GitBox <gi...@apache.org>]

lhutton1 commented on a change in pull request #9209:
URL: https://github.com/apache/tvm/pull/9209#discussion_r723689306



##########
File path: python/tvm/relay/backend/contrib/ethosu/legalize.py
##########
@@ -208,6 +208,96 @@ def __call__(self, *args, **kwargs):
         pass
 
 
+class EthosuDepthwise2DRewriter(DFPatternCallback):
+    """Convert ethosu.qnn_depthwise2d composite functions to ethosu_depthwise2d operators"""
+
+    def __init__(self):
+        super().__init__(require_type=True)
+        self.pattern = (
+            wildcard().has_attr({"Composite": ethosu_patterns.QnnDepthwise2DParams.composite_name})
+        )(wildcard())
+
+    def callback(
+        self, pre: tvm.relay.Expr, post: tvm.relay.Expr, node_map: tvm.ir.container.Map
+    ) -> tvm.relay.Expr:
+        params = ethosu_patterns.QnnDepthwise2DParams(post.op.body)
+        params.ifm.tensor = post.args[0]
+        channels_map = {
+            "NHWC": 3,
+        }
+        if str(params.ofm.layout) not in channels_map.keys():
+            raise UnsupportedLayout(str(params.ofm.layout))
+        kernel_shape_map = {
+            "HWOI": params.weights.shape[0:2],

Review comment:
       Is it worth supporting OHWI weights here?

##########
File path: python/tvm/relay/op/contrib/ethosu.py
##########
@@ -223,12 +223,13 @@ def __init__(self, func_body: tvm.relay.Function):
         self.strides = qnn_conv2d.attrs.strides
         self.dilation = qnn_conv2d.attrs.dilation
         self.activation = activation
+        self.channels = qnn_conv2d.attrs.channels

Review comment:
       Better to access attrs once here i.e,
   ```
   attrs = qnn_conv2d.attrs
   self.padding = attrs.padding
   ...
   self.channels = attrs.channels
   ```

##########
File path: python/tvm/relay/op/contrib/ethosu.py
##########
@@ -272,14 +315,34 @@ def qnn_conv2d_pattern() -> tvm.relay.dataflow_pattern.DFPattern:
     return clip_or_req
 
 
+def qnn_depthwise2d_pattern() -> tvm.relay.dataflow_pattern.DFPattern:
+    """
+    This function creates the pattern for depthwise qnn.conv2D with optional fused RELU activation.
+    """
+    qnn_conv2d = is_op("qnn.conv2d")(
+        wildcard(), is_constant(), is_constant(), is_constant(), is_constant(), is_constant()
+    ).has_attr({"kernel_layout": "HWOI"})
+    bias_add = is_op("nn.bias_add")(qnn_conv2d, is_constant())
+    req = is_op("qnn.requantize")(
+        qnn_conv2d | bias_add, is_constant(), is_constant(), is_constant(), is_constant()

Review comment:
       Remove optional bias here? Then we can follow up with separate PR for conv2d




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org