[GitHub] [tvm] anwang2009 commented on a change in pull request #10450: [ONNX] Add MatMulInteger importer

[GitBox <gi...@apache.org>]

anwang2009 commented on a change in pull request #10450:
URL: https://github.com/apache/tvm/pull/10450#discussion_r818410945



##########
File path: python/tvm/relay/frontend/onnx.py
##########
@@ -3935,6 +3935,58 @@ def try_resolve_to_const(x, dtype_override=None):
         return y
 
 
+class MatMulInteger(OnnxOpConverter):
+    """Operator converter for MatMulInteger."""
+
+    @classmethod
+    def _impl_v10(cls, inputs, attr, params):
+        a = inputs[0]
+        b = inputs[1]
+
+        a_dtype = infer_type(a).checked_type.dtype
+        b_dtype = infer_type(b).checked_type.dtype
+
+        assert a_dtype in ("int8", "uint8"), "MatMulInteger: invalid dtype for first input"
+        assert b_dtype in ("int8", "uint8"), "MatMulInteger: invalid dtype for second input"
+
+        assert a_dtype == b_dtype, "MatMulInteger: input dtypes must match"
+
+        a_scale = _op.const(1.0, dtype="float32")
+        b_scale = _op.const(1.0, dtype="float32")
+        out_scale = _op.const(1.0, dtype="float32")
+
+        a_zero_point = _op.const(0.0, dtype=a_dtype)

Review comment:
       yeah zp isn't necessarily fixed to 0, but the docs say 0 should be the default. I do this a few lines down: 
   
   ```
           if len(inputs) == 4:
               a_zero_point = inputs[2]
               b_zero_point = inputs[3]
   ```
   From the example it looks like the scale factors are 1 by default.

##########
File path: python/tvm/relay/frontend/onnx.py
##########
@@ -3935,6 +3935,58 @@ def try_resolve_to_const(x, dtype_override=None):
         return y
 
 
+class MatMulInteger(OnnxOpConverter):
+    """Operator converter for MatMulInteger."""
+
+    @classmethod
+    def _impl_v10(cls, inputs, attr, params):
+        a = inputs[0]

Review comment:
       modified `QLinearMatMul` converter to skip requantize in the MatMulInteger case




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org