[GitHub] [tvm] Mousius commented on a diff in pull request #12671: [microTVM] Allow int8 operations for Cortex-M cores

[GitBox <gi...@apache.org>]

Mousius commented on code in PR #12671:
URL: https://github.com/apache/tvm/pull/12671#discussion_r961506187


##########
python/tvm/relay/qnn/op/legalizations.py:
##########
@@ -417,6 +417,16 @@ def is_aarch64_arm():
     return "aarch64" in target.attrs.get("mtriple", "")
 
 
+def is_cortexm_arm():
+    """Checks whether we are compiling for a Cortex-M target. We want to preserve
+    int8 operations for these boards, because it halves the number of memory loads
+    needed for dense layers and convolutions. It comes at a cost of needing more
+    micro kernels and sign extension instructions to load these values, but this
+    trade-off seems to be worth it on Cortex-M."""
+    target = tvm.target.Target.current(allow_none=False)
+    return "cortex-m" in target.attrs.get("mcpu", "")

Review Comment:
   Can we add this to https://github.com/apache/tvm/blob/main/src/target/parsers/mprofile.c as `is_mprofile` ?  We're about to remove all of these helpers in https://github.com/apache/tvm/pull/12454



##########
python/tvm/relay/qnn/op/legalizations.py:
##########
@@ -433,15 +443,15 @@ def _qnn_conv2d_legalize_arm_cpu(attrs, inputs, types):
         attrs["groups"],
     )
     use_int8_on_arm = (not is_depthwise) and is_aarch64_arm() and attrs["data_layout"] == "NHWC"
-    if use_int8_on_arm or is_fast_int8_on_arm():
+    if use_int8_on_arm or is_fast_int8_on_arm() or is_cortexm_arm():
         return helper_change_dtypes_to_be_same(attrs, inputs, types, relay.qnn.op.conv2d)
     return helper_no_fast_int8_hw_legalization(attrs, inputs, types, relay.nn.conv2d)

Review Comment:
   I think this is actually the same as an issue awhile back (https://github.com/apache/tvm/issues/8717#issuecomment-976400930), instead of special casing with `is_cortexm` you should be able to instead use something like:
   
   ```suggestion
       has_asimd = (is_aarch64_arm() or "+neon" in target.mattr)
       if has_asimd and not is_fast_int8_on_arm():
           return helper_no_fast_int8_hw_legalization(attrs, inputs, types, relay.nn.conv2d)
       return helper_change_dtypes_to_be_same(attrs, inputs, types, relay.qnn.op.conv2d)
   ```
   
   Which means it will only do the casting when the specific architecture features are available (`has_asimd` will become `target.features.has_asimd` after https://github.com/apache/tvm/pull/12454 so is fine temporarily)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org