You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2020/01/15 18:25:24 UTC
[GitHub] [incubator-tvm] shoubhik opened a new pull request #4714: Mxnet
parser for Qnn dialect
shoubhik opened a new pull request #4714: Mxnet parser for Qnn dialect
URL: https://github.com/apache/incubator-tvm/pull/4714
In this parser, I am including the changes needed for QNN dialect for Mxnet. It has been tested with most of the open-source Mxnet models.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] jackwish commented on a change in pull request
#4714: Mxnet parser for Qnn dialect
Posted by GitBox <gi...@apache.org>.
jackwish commented on a change in pull request #4714: Mxnet parser for Qnn dialect
URL: https://github.com/apache/incubator-tvm/pull/4714#discussion_r368292387
##########
File path: python/tvm/relay/frontend/mxnet.py
##########
@@ -1075,6 +1097,378 @@ def _mx_cond(inputs, attrs, subgraphs):
return ret
+def _qnn_quantize(inputs, attrs):
+ out_dtype = 'int8'
+ out_type = attrs.get_str('out_type')
+ if out_type == 'auto':
+ if attrs.has_attr('min_calib_range') and attrs.has_attr('max_calib_range'):
+ if attrs.get_float('min_calib_range') >= 0:
+ out_dtype = 'uint8'
+ else:
+ out_dtype = 'int8'
+ else:
+ out_dtype = out_type
+ if out_dtype not in {'int8', 'uint8'}:
Review comment:
It seems that this assert checks `out_type` actually? If that is the case, can we move it to where we obtained the `out_type` value?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] shoubhik commented on a change in pull request
#4714: Mxnet parser for Qnn dialect
Posted by GitBox <gi...@apache.org>.
shoubhik commented on a change in pull request #4714: Mxnet parser for Qnn dialect
URL: https://github.com/apache/incubator-tvm/pull/4714#discussion_r368705876
##########
File path: python/tvm/relay/frontend/mxnet.py
##########
@@ -44,8 +58,9 @@
"relu" : _op.nn.relu
}
+
def _mx_fully_connected(inputs, attrs):
- import mxnet as mx
+ import mxnet as mx #pylint: disable=import-outside-toplevel
Review comment:
running the pylint locally i was getting this warning.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] masahi commented on a change in pull request #4714:
Mxnet parser for Qnn dialect
Posted by GitBox <gi...@apache.org>.
masahi commented on a change in pull request #4714: Mxnet parser for Qnn dialect
URL: https://github.com/apache/incubator-tvm/pull/4714#discussion_r367655953
##########
File path: python/tvm/relay/frontend/mxnet.py
##########
@@ -180,6 +190,17 @@ def _mx_conv2d(inputs, attrs):
new_attrs["groups"] = attrs.get_int("num_group", 1)
new_attrs["data_layout"] = data_layout
new_attrs["kernel_layout"] = kernel_layout
+ return new_attrs
+
+def _mx_conv2d(inputs, attrs):
+ kernel_size = attrs.get_int_tuple("kernel")
+ data_layout = attrs.get_str("layout", "NCHW")
+ if len(kernel_size) != 2:
+ raise tvm.error.OpAttributeInvalid(
+ 'Non 1D or 2D kernels are not supported for operator Convolution')
Review comment:
actually we do have support for 1d and 3d conv in topi now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] jackwish commented on a change in pull request
#4714: Mxnet parser for Qnn dialect
Posted by GitBox <gi...@apache.org>.
jackwish commented on a change in pull request #4714: Mxnet parser for Qnn dialect
URL: https://github.com/apache/incubator-tvm/pull/4714#discussion_r368292112
##########
File path: python/tvm/relay/frontend/mxnet.py
##########
@@ -44,8 +58,9 @@
"relu" : _op.nn.relu
}
+
def _mx_fully_connected(inputs, attrs):
- import mxnet as mx
+ import mxnet as mx #pylint: disable=import-outside-toplevel
Review comment:
Why we need this change?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] shoubhik commented on a change in pull request
#4714: Mxnet parser for Qnn dialect
Posted by GitBox <gi...@apache.org>.
shoubhik commented on a change in pull request #4714: Mxnet parser for Qnn dialect
URL: https://github.com/apache/incubator-tvm/pull/4714#discussion_r367651581
##########
File path: python/tvm/relay/frontend/mxnet_qnn_op_utils.py
##########
@@ -21,52 +21,95 @@
import numpy as np
from tvm import relay
-from tvm.relay.qnn.op.qnn import dequantize
+from tvm.relay.qnn.op.qnn import quantize, dequantize
-zero_centered_uint8_quantized_range = np.float32(255)
-zero_centered_int8_quantized_range = np.float32(127)
+zero_centered_uint8_quantized_range = np.float32(255.5)
Review comment:
Added reference.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] jackwish commented on a change in pull request
#4714: Mxnet parser for Qnn dialect
Posted by GitBox <gi...@apache.org>.
jackwish commented on a change in pull request #4714: Mxnet parser for Qnn dialect
URL: https://github.com/apache/incubator-tvm/pull/4714#discussion_r368292864
##########
File path: python/tvm/relay/frontend/mxnet_qnn_op_utils.py
##########
@@ -21,52 +21,95 @@
import numpy as np
from tvm import relay
-from tvm.relay.qnn.op.qnn import dequantize
+from tvm.relay.qnn.op.qnn import quantize, dequantize
-zero_centered_uint8_quantized_range = np.float32(255)
-zero_centered_int8_quantized_range = np.float32(127)
+zero_centered_uint8_quantized_range = np.float32(255.5)
Review comment:
> Can you explain why are we adding an extra 0.5 here?
I guess this is related to the rounding policy and the quantization arithmetic?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] shoubhik commented on a change in pull request
#4714: Mxnet parser for Qnn dialect
Posted by GitBox <gi...@apache.org>.
shoubhik commented on a change in pull request #4714: Mxnet parser for Qnn dialect
URL: https://github.com/apache/incubator-tvm/pull/4714#discussion_r368707578
##########
File path: python/tvm/relay/frontend/mxnet.py
##########
@@ -1075,6 +1097,378 @@ def _mx_cond(inputs, attrs, subgraphs):
return ret
+def _qnn_quantize(inputs, attrs):
+ out_dtype = 'int8'
+ out_type = attrs.get_str('out_type')
+ if out_type == 'auto':
+ if attrs.has_attr('min_calib_range') and attrs.has_attr('max_calib_range'):
+ if attrs.get_float('min_calib_range') >= 0:
+ out_dtype = 'uint8'
+ else:
+ out_dtype = 'int8'
+ else:
+ out_dtype = out_type
+ if out_dtype not in {'int8', 'uint8'}:
+ raise ValueError('Unsupported out_dtype: %s' % out_dtype)
+ min_calib_range = attrs.get_float('min_calib_range', 0.0)
+ max_calib_range = attrs.get_float('max_calib_range', 0.0)
Review comment:
You are correct. The zeros should not be there. In fact there min and max operators. i'll fix this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] anijain2305 edited a comment on issue #4714: Mxnet
parser for Qnn dialect
Posted by GitBox <gi...@apache.org>.
anijain2305 edited a comment on issue #4714: Mxnet parser for Qnn dialect
URL: https://github.com/apache/incubator-tvm/pull/4714#issuecomment-576905072
With the latest commit, we have good accuracy for all MxNet-MKLDNN quantized models
@tmoreau89 @yzhliu @tqchen @jackwish @FrozenGene @liangfu
| MxNet | TVM | Degradation
-- | -- | -- | --
| Top1 | Top5 | Top1 | Top5 | Top1 | Top5
Resnet18_v1 | 69.76 | 89.02 | 69.85 | 89.09 | -0.09 | -0.07
Resnet50_v1 | 76.13 | 92.6 | 75.9 | 92.66 | 0.23 | -0.06
Resnet50_v1b | 76.66 | 92.6 | 76.45 | 92.57 | 0.21 | 0.03
Resnet101_v1 | 77.13 | 93.06 | 77 | 93.06 | 0.13 | 0
Resnet152_v2 | 75.99 | 92.52 | 75.32 | 92.26 | 0.67 | 0.26
Inception-V3 | 77.84 | 93.52 | 77.28 | 93.32 | 0.56 | 0.2
Inception-BN | 71.96 | 90.38 | 71.79 | 90.25 | 0.17 | 0.13
MobileNetV1 | 71.27 | 90.09 | 71.13 | 90.16 | 0.14 | -0.07
MobileNetV2 | 70.35 | 89.45 | 70.19 | 89.5 | 0.16 | -0.05
Quantized models are generated using this - https://github.com/apache/incubator-mxnet/tree/master/example/quantization
Accuracy is collected over 10,000 inputs with target = 'llvm'
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] jackwish commented on a change in pull request
#4714: Mxnet parser for Qnn dialect
Posted by GitBox <gi...@apache.org>.
jackwish commented on a change in pull request #4714: Mxnet parser for Qnn dialect
URL: https://github.com/apache/incubator-tvm/pull/4714#discussion_r368292444
##########
File path: python/tvm/relay/frontend/mxnet.py
##########
@@ -1075,6 +1097,378 @@ def _mx_cond(inputs, attrs, subgraphs):
return ret
+def _qnn_quantize(inputs, attrs):
+ out_dtype = 'int8'
+ out_type = attrs.get_str('out_type')
+ if out_type == 'auto':
+ if attrs.has_attr('min_calib_range') and attrs.has_attr('max_calib_range'):
+ if attrs.get_float('min_calib_range') >= 0:
+ out_dtype = 'uint8'
+ else:
+ out_dtype = 'int8'
+ else:
+ out_dtype = out_type
+ if out_dtype not in {'int8', 'uint8'}:
+ raise ValueError('Unsupported out_dtype: %s' % out_dtype)
+ min_calib_range = attrs.get_float('min_calib_range', 0.0)
+ max_calib_range = attrs.get_float('max_calib_range', 0.0)
Review comment:
That's interesting, if there is not `min/max_calib_range` provided, we are expecting the range to be `[0.0, 0.0)`? I am not very familiar with MxNet calibration method, but it appears to me that the *detault* min/max can derive from `out_dtype`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] shoubhik commented on a change in pull request
#4714: Mxnet parser for Qnn dialect
Posted by GitBox <gi...@apache.org>.
shoubhik commented on a change in pull request #4714: Mxnet parser for Qnn dialect
URL: https://github.com/apache/incubator-tvm/pull/4714#discussion_r368708311
##########
File path: python/tvm/relay/frontend/mxnet.py
##########
@@ -1075,6 +1097,378 @@ def _mx_cond(inputs, attrs, subgraphs):
return ret
+def _qnn_quantize(inputs, attrs):
+ out_dtype = 'int8'
+ out_type = attrs.get_str('out_type')
+ if out_type == 'auto':
+ if attrs.has_attr('min_calib_range') and attrs.has_attr('max_calib_range'):
+ if attrs.get_float('min_calib_range') >= 0:
+ out_dtype = 'uint8'
+ else:
+ out_dtype = 'int8'
+ else:
+ out_dtype = out_type
+ if out_dtype not in {'int8', 'uint8'}:
Review comment:
out_dtype can be auto too. in which case we infer it based on the min and max values.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] jackwish commented on a change in pull request
#4714: Mxnet parser for Qnn dialect
Posted by GitBox <gi...@apache.org>.
jackwish commented on a change in pull request #4714: Mxnet parser for Qnn dialect
URL: https://github.com/apache/incubator-tvm/pull/4714#discussion_r368292195
##########
File path: python/tvm/relay/frontend/mxnet.py
##########
@@ -1033,6 +1054,7 @@ def _mx_contrib_fifo_buffer(inputs, attrs):
new_attrs['axis'] = attrs.get_int('axis')
return _op.nn.fifo_buffer(*inputs, **new_attrs)
+
Review comment:
not needed?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] anijain2305 edited a comment on issue #4714: Mxnet
parser for Qnn dialect
Posted by GitBox <gi...@apache.org>.
anijain2305 edited a comment on issue #4714: Mxnet parser for Qnn dialect
URL: https://github.com/apache/incubator-tvm/pull/4714#issuecomment-576905072
With the latest commit, we have good accuracy for all MxNet-MKLDNN quantized models
@tmoreau89 @yzhliu @tqchen @jackwish @FrozenGene @liangfu
-- | Mxnet-Top1 | Mxnet-Top5 | TVM-Top1 | TVM-Top5 | Degradation-Top1 | Degradation-Top5
-- | -- | -- | -- | -- | -- | --
Resnet18_v1 | 69.76 | 89.02 | 69.85 | 89.09 | -0.09 | -0.07
Resnet50_v1 | 76.13 | 92.6 | 75.9 | 92.66 | 0.23 | -0.06
Resnet50_v1b | 76.66 | 92.6 | 76.45 | 92.57 | 0.21 | 0.03
Resnet101_v1 | 77.13 | 93.06 | 77 | 93.06 | 0.13 | 0
Resnet152_v2 | 75.99 | 92.52 | 75.32 | 92.26 | 0.67 | 0.26
Inception-V3 | 77.84 | 93.52 | 77.28 | 93.32 | 0.56 | 0.2
Inception-BN | 71.96 | 90.38 | 71.79 | 90.25 | 0.17 | 0.13
MobileNetV1 | 71.27 | 90.09 | 71.13 | 90.16 | 0.14 | -0.07
MobileNetV2 | 70.35 | 89.45 | 70.19 | 89.5 | 0.16 | -0.05
Quantized models are generated using this - https://github.com/apache/incubator-mxnet/tree/master/example/quantization
Accuracy is collected over 10,000 inputs with target = 'llvm'
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] masahi commented on issue #4714: Mxnet parser for
Qnn dialect
Posted by GitBox <gi...@apache.org>.
masahi commented on issue #4714: Mxnet parser for Qnn dialect
URL: https://github.com/apache/incubator-tvm/pull/4714#issuecomment-582140804
@shoubhik I'll have a look and merge. I'm interested in this PR.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] anijain2305 commented on issue #4714: Mxnet parser
for Qnn dialect
Posted by GitBox <gi...@apache.org>.
anijain2305 commented on issue #4714: Mxnet parser for Qnn dialect
URL: https://github.com/apache/incubator-tvm/pull/4714#issuecomment-581769804
Given the constraints, I am ok with MxNet-parser code w/o any MxNet-MKLDNN CI testing, as long as we have a test script linked in the PR, that somebody can locally use to test the changes.
Once Mxnet-MKLDNN CI/accuracy issues are resolved, we can re-write that script as a test.
Is that ok - @tqchen @icemelon9
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] FrozenGene commented on issue #4714: Mxnet parser
for Qnn dialect
Posted by GitBox <gi...@apache.org>.
FrozenGene commented on issue #4714: Mxnet parser for Qnn dialect
URL: https://github.com/apache/incubator-tvm/pull/4714#issuecomment-581800500
> @tmoreau89 @yzhliu @tqchen @jackwish @FrozenGene @liangfu @vinx13 @anijain2305 I want to discuss this PR from testability POV. Mxnet suggests to use MKLDNN as backend for quantization. The operators for MKLDNN quantization in Mxnet is different from stock Mxnet quantization operators. Also, for some ops the implementation of these ops are also different.
> Due to this PR #4753 and #4764, it is not possible to merge the Mxnet-MKLDNN installation in TVM docker. One potential solution is to test stock quantization operators but that would be extra work that may not be too useful in future. All the testing we have done and the edge cases we have covered are for the MKLDNN operators.
> My suggestion is at this point we check in the parser code along with the the test scripts we have for benchmarking the QNN networks. When MKLDNN issues are fixed and we can safely upgrade we can add test cases at that point.
I am OK with this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] shoubhik commented on issue #4714: Mxnet parser for
Qnn dialect
Posted by GitBox <gi...@apache.org>.
shoubhik commented on issue #4714: Mxnet parser for Qnn dialect
URL: https://github.com/apache/incubator-tvm/pull/4714#issuecomment-574790244
@anijain2305 , @jackwish can you take a look at this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] liangfu commented on a change in pull request
#4714: Mxnet parser for Qnn dialect
Posted by GitBox <gi...@apache.org>.
liangfu commented on a change in pull request #4714: Mxnet parser for Qnn dialect
URL: https://github.com/apache/incubator-tvm/pull/4714#discussion_r367195032
##########
File path: python/tvm/relay/frontend/mxnet_qnn_op_utils.py
##########
@@ -21,52 +21,95 @@
import numpy as np
from tvm import relay
-from tvm.relay.qnn.op.qnn import dequantize
+from tvm.relay.qnn.op.qnn import quantize, dequantize
-zero_centered_uint8_quantized_range = np.float32(255)
-zero_centered_int8_quantized_range = np.float32(127)
+zero_centered_uint8_quantized_range = np.float32(255.5)
Review comment:
Can you explain why are we adding an extra 0.5 here?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] shoubhik commented on issue #4714: Mxnet parser for
Qnn dialect
Posted by GitBox <gi...@apache.org>.
shoubhik commented on issue #4714: Mxnet parser for Qnn dialect
URL: https://github.com/apache/incubator-tvm/pull/4714#issuecomment-576458551
> it would be great if you could add an test case on end to end network translation like in our tflite test case.
I have created a post on discuss on how we can do this - https://discuss.tvm.ai/t/use-mxnet-mkldnn-distribution-in-ci-instead-of-stock-mxnet-distribution/5474
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] shoubhik commented on issue #4714: Mxnet parser for
Qnn dialect
Posted by GitBox <gi...@apache.org>.
shoubhik commented on issue #4714: Mxnet parser for Qnn dialect
URL: https://github.com/apache/incubator-tvm/pull/4714#issuecomment-582138102
@tqchen @yzhliu I think the code can be merged now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] shoubhik commented on issue #4714: Mxnet parser for
Qnn dialect
Posted by GitBox <gi...@apache.org>.
shoubhik commented on issue #4714: Mxnet parser for Qnn dialect
URL: https://github.com/apache/incubator-tvm/pull/4714#issuecomment-581672885
@tmoreau89 @yzhliu @tqchen @jackwish @FrozenGene @liangfu @vinx13 @anijain2305 I want to discuss this PR from testability POV. Mxnet suggests to use MKLDNN as backend for quantization. The operators for MKLDNN quantization in Mxnet is different from stock Mxnet quantization operators. Also, for some ops the implementation of these ops are also different.
Due to this PR #4753 and #4764, it is not possible to merge the Mxnet-MKLDNN installation in TVM docker. One potential solution is to test stock quantization operators but that would be extra work that may not be too useful in future. All the testing we have done and the edge cases we have covered are for the MKLDNN operators.
My suggestion is at this point we check in the parser code along with the the test scripts we have for benchmarking the QNN networks. When MKLDNN issues are fixed and we can safely upgrade we can add test cases at that point.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] liangfu commented on a change in pull request
#4714: Mxnet parser for Qnn dialect
Posted by GitBox <gi...@apache.org>.
liangfu commented on a change in pull request #4714: Mxnet parser for Qnn dialect
URL: https://github.com/apache/incubator-tvm/pull/4714#discussion_r367197041
##########
File path: python/tvm/relay/frontend/mxnet.py
##########
@@ -180,6 +190,17 @@ def _mx_conv2d(inputs, attrs):
new_attrs["groups"] = attrs.get_int("num_group", 1)
new_attrs["data_layout"] = data_layout
new_attrs["kernel_layout"] = kernel_layout
+ return new_attrs
+
+def _mx_conv2d(inputs, attrs):
+ kernel_size = attrs.get_int_tuple("kernel")
+ data_layout = attrs.get_str("layout", "NCHW")
+ if len(kernel_size) != 2:
+ raise tvm.error.OpAttributeInvalid(
+ 'Non 1D or 2D kernels are not supported for operator Convolution')
Review comment:
Only 2D kernels are supported.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] masahi commented on issue #4714: Mxnet parser for
Qnn dialect
Posted by GitBox <gi...@apache.org>.
masahi commented on issue #4714: Mxnet parser for Qnn dialect
URL: https://github.com/apache/incubator-tvm/pull/4714#issuecomment-575442459
it would be great if you could an test case on end to end network translation like in our tflite test case.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] liangfu commented on a change in pull request
#4714: Mxnet parser for Qnn dialect
Posted by GitBox <gi...@apache.org>.
liangfu commented on a change in pull request #4714: Mxnet parser for Qnn dialect
URL: https://github.com/apache/incubator-tvm/pull/4714#discussion_r367192729
##########
File path: python/tvm/relay/frontend/mxnet.py
##########
@@ -14,12 +14,14 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
-# pylint: disable=invalid-name, import-self, len-as-condition, no-else-return
+# pylint: disable=invalid-name, import-self, len-as-condition, no-else-return, too-many-lines
Review comment:
Avoid disabling linter in head of a file. Instead, you can disable them in specific functions or classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] masahi commented on a change in pull request #4714:
Mxnet parser for Qnn dialect
Posted by GitBox <gi...@apache.org>.
masahi commented on a change in pull request #4714: Mxnet parser for Qnn dialect
URL: https://github.com/apache/incubator-tvm/pull/4714#discussion_r374951699
##########
File path: tests/python/frontend/mxnet/test_forward.py
##########
@@ -28,68 +28,173 @@
import model_zoo
+def get_gluon_output(name, x, dtype):
+ net = vision.get_model(name)
+ net.collect_params().initialize(mx.init.Xavier())
+ net_sym = gluon.nn.SymbolBlock(outputs=net(mx.sym.var('data')),
+ inputs=mx.sym.var('data'),
+ params=net.collect_params())
+ out = net_sym(mx.nd.array(x.astype(dtype))).asnumpy()
+ return out, net_sym
+
+
+def get_mxnet_output(symbol, x, dtype='float32', arg_params=None, aux_params=None):
+ from collections import namedtuple
+ Batch = namedtuple('Batch', ['data'])
+ mod = mx.mod.Module(symbol, label_names=None)
+ mod.bind(data_shapes=[('data', x.shape)], for_training=False)
+ if aux_params is not None and arg_params is not None:
+ mod.set_params(arg_params, aux_params)
+ else:
+ mod.init_params()
+ mod.forward(Batch([mx.nd.array(x.astype(dtype))]))
+ out = mod.get_outputs()[0].asnumpy()
+ args, auxs = mod.get_params()
+ return out, args, auxs
+
+
+def get_tvm_output(symbol, x, args, auxs, target, ctx, out_shape, dtype='float32', gluon_impl=False):
+ shape_dict = {"data": x.shape}
+ if gluon_impl:
+ mod, params = relay.frontend.from_mxnet(symbol, shape_dict)
+ else:
+ mod, params = relay.frontend.from_mxnet(symbol,
+ shape_dict,
+ arg_params=args,
+ aux_params=auxs)
+ with relay.build_config(opt_level=3):
+ graph, lib, params = relay.build(mod, target, params=params)
+ m = graph_runtime.create(graph, lib, ctx)
+ # set inputs
+ m.set_input("data", tvm.nd.array(x.astype(dtype)))
+ m.set_input(**params)
+ m.run()
+ # get outputs
+ out = m.get_output(0, tvm.nd.empty(out_shape, dtype))
+ return out.asnumpy()
+
+
def verify_mxnet_frontend_impl(mx_symbol,
data_shape=(1, 3, 224, 224),
out_shape=(1, 1000),
gluon_impl=False,
name=None,
dtype='float32'):
"""Use name different from test to avoid pytest picking it up"""
- if gluon_impl:
- def get_gluon_output(name, x):
- net = vision.get_model(name)
- net.collect_params().initialize(mx.init.Xavier())
- net_sym = gluon.nn.SymbolBlock(outputs=net(mx.sym.var('data')),
- inputs=mx.sym.var('data'),
- params=net.collect_params())
- out = net_sym(mx.nd.array(x.astype(dtype))).asnumpy()
- return out, net_sym
- else:
- def get_mxnet_output(symbol, x, dtype='float32'):
- from collections import namedtuple
- Batch = namedtuple('Batch', ['data'])
- mod = mx.mod.Module(symbol, label_names=None)
- mod.bind(data_shapes=[('data', x.shape)], for_training=False)
- mod.init_params()
- mod.forward(Batch([mx.nd.array(x.astype(dtype))]))
- out = mod.get_outputs()[0].asnumpy()
- args, auxs = mod.get_params()
- return out, args, auxs
-
- def get_tvm_output(symbol, x, args, auxs, target, ctx, dtype='float32'):
- shape_dict = {"data": x.shape}
- if gluon_impl:
- mod, params = relay.frontend.from_mxnet(symbol, shape_dict)
- else:
- mod, params = relay.frontend.from_mxnet(symbol,
- shape_dict,
- arg_params=args,
- aux_params=auxs)
- with relay.build_config(opt_level=3):
- graph, lib, params = relay.build(mod, target, params=params)
- m = graph_runtime.create(graph, lib, ctx)
- # set inputs
- m.set_input("data", tvm.nd.array(x.astype(dtype)))
- m.set_input(**params)
- m.run()
- # get outputs
- out = m.get_output(0, tvm.nd.empty(out_shape, dtype))
- return out.asnumpy()
-
# random input
x = np.random.uniform(size=data_shape)
if gluon_impl:
- gluon_out, gluon_sym = get_gluon_output(name, x)
+ gluon_out, gluon_sym = get_gluon_output(name, x, dtype)
for target, ctx in ctx_list():
tvm_out = get_tvm_output(gluon_sym, x, None, None, target, ctx, dtype)
tvm.testing.assert_allclose(gluon_out, tvm_out, rtol=1e-5, atol=1e-5)
else:
mx_out, args, auxs = get_mxnet_output(mx_symbol, x, dtype)
assert "data" not in args
for target, ctx in ctx_list():
- tvm_out = get_tvm_output(mx_symbol, x, args, auxs, target, ctx, dtype)
+ tvm_out = get_tvm_output(mx_symbol, x, args, auxs, target, ctx, out_shape, dtype)
tvm.testing.assert_allclose(mx_out, tvm_out, rtol=1e-5, atol=1e-5)
+
+def verify_mxnet_quantized_forward_impl(mx_symbol,
+ data_shape=(1, 3, 224, 224),
+ out_shape=(1, 1000),
+ name=None,
+ dtype='float32'):
+
+ def use_old_mxnet_quantization_api():
+ mx_version = float(mx.__version__[:3])
+ return mx_version <= 1.5
+
+ def quantize_with_old_api(sym, _arg_params, _aux_params, quantized_dtype, _data_shape, logger):
+ """Calibrate and quantize the network with dummy data"""
+ ctx = mx.cpu(0)
+ batch_size = 1
+ num_calib_batches = 5
+ num_calib_examples = num_calib_batches * batch_size
+ calibration_data = [mx.random.uniform(-1.0, 1.0, shape=_data_shape, ctx=ctx, dtype='float32')]
+ data = mx.io.NDArrayIter(data=calibration_data, label=[], batch_size=batch_size,
+ shuffle=True, last_batch_handle='discard')
+ qsym = sym.get_backend_symbol('MKLDNN_QUANTIZE')
+ qsym, qarg_params, qaux_params = mx.contrib.quantization.quantize_model(sym=qsym,
+ arg_params=_arg_params,
+ aux_params=_aux_params,
+ ctx=ctx,
+ excluded_sym_names=[],
+ calib_data=data,
+ num_calib_examples=num_calib_examples,
+ calib_mode='none',
+ quantized_dtype=quantized_dtype,
+ label_names=None,
+ logger=logger)
+ qsym = sym.get_backend_symbol('MKLDNN_QUANTIZE')
+ return qsym, qarg_params, qaux_params
+
+ def quantized_with_new_api(sym, _arg_params, _aux_params, quantized_dtype, _data_shape, logger):
+ from mxnet.contrib.quantization import *
+
+ sym = sym.get_backend_symbol('MKLDNN_QUANTIZE')
+ # set exclude layers
+ excluded_names = []
+ # set calib mode.
+ calib_mode = 'naive'
+ # set calib_layer
+ calib_layer = None
+ # set quantized_dtype
+ ctx = mx.cpu(0)
+ cqsym, cqarg_params, _aux_params, collector = \
+ quantize_graph(sym=sym, arg_params=_arg_params, aux_params=_aux_params,
+ excluded_sym_names=excluded_names,
+ calib_mode=calib_mode, calib_layer=calib_layer,
+ quantized_dtype=quantized_dtype, logger=logger)
+ batch_size = 16
+ calibration_data = [mx.random.uniform(-1.0, 1.0, shape=_data_shape, ctx=ctx, dtype='float32')]
+
+ data = mx.io.NDArrayIter(data=calibration_data, label=[], batch_size=batch_size,
+ shuffle=True, last_batch_handle='discard')
+ # create module
+ mod = mx.mod.Module(symbol=sym, label_names=None, context=mx.cpu())
+ mod.bind(for_training=False, data_shapes=data.provide_data, label_shapes=None)
+ mod.set_params(_arg_params, _aux_params)
+ # calibration configs
+ # set num_calib_batches
+ num_calib_batches = 5
+ max_num_examples = num_calib_batches * batch_size
+ # monitor FP32 Inference
+ mod._exec_group.execs[0].set_monitor_callback(collector.collect, monitor_all=True)
+ num_batches = 0
+ num_examples = 0
+ for batch in data:
+ mod.forward(data_batch=batch, is_train=False)
+ num_batches += 1
+ num_examples += batch_size
+ if num_examples >= max_num_examples:
+ break
+ if logger is not None:
+ logger.info("Collected statistics from %d batches with batch_size=%d"
+ % (num_batches, batch_size))
+ cqsym, cqarg_params, _aux_params = calib_graph(qsym=cqsym,
+ arg_params=_arg_params,
+ aux_params=_aux_params,
+ collector=collector,
+ calib_mode=calib_mode,
+ quantized_dtype=quantized_dtype,
+ logger=logger)
+ return cqsym, cqarg_params, _aux_params
+
+ ctx = mx.cpu(0)
+ x = mx.random.uniform(-1.0, 1.0, shape=data_shape, ctx=ctx, dtype='float32')
+ _, arg_params, aux_params = get_mxnet_output(mx_symbol, x)
+ quantize_api = quantize_with_old_api if use_old_mxnet_quantization_api() else quantized_with_new_api
+
+
Review comment:
remove this white spaces
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] masahi merged pull request #4714: Mxnet parser for
Qnn dialect
Posted by GitBox <gi...@apache.org>.
masahi merged pull request #4714: Mxnet parser for Qnn dialect
URL: https://github.com/apache/incubator-tvm/pull/4714
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] anijain2305 edited a comment on issue #4714: Mxnet
parser for Qnn dialect
Posted by GitBox <gi...@apache.org>.
anijain2305 edited a comment on issue #4714: Mxnet parser for Qnn dialect
URL: https://github.com/apache/incubator-tvm/pull/4714#issuecomment-576905072
With the latest commit, we have good accuracy for all MxNet-MKLDNN quantized models
@tmoreau89 @yzhliu @tqchen @jackwish @FrozenGene @liangfu @vinx13
-- | Mxnet-Top1 | Mxnet-Top5 | TVM-Top1 | TVM-Top5 | Degradation-Top1 | Degradation-Top5
-- | -- | -- | -- | -- | -- | --
Resnet18_v1 | 69.76 | 89.02 | 69.85 | 89.09 | -0.09 | -0.07
Resnet50_v1 | 76.13 | 92.6 | 75.9 | 92.66 | 0.23 | -0.06
Resnet50_v1b | 76.66 | 92.6 | 76.45 | 92.57 | 0.21 | 0.03
Resnet101_v1 | 77.13 | 93.06 | 77 | 93.06 | 0.13 | 0
Resnet152_v2 | 75.99 | 92.52 | 75.32 | 92.26 | 0.67 | 0.26
Inception-V3 | 77.84 | 93.52 | 77.28 | 93.32 | 0.56 | 0.2
Inception-BN | 71.96 | 90.38 | 71.79 | 90.25 | 0.17 | 0.13
MobileNetV1 | 71.27 | 90.09 | 71.13 | 90.16 | 0.14 | -0.07
MobileNetV2 | 70.35 | 89.45 | 70.19 | 89.5 | 0.16 | -0.05
Quantized models are generated using this - https://github.com/apache/incubator-mxnet/tree/master/example/quantization
Accuracy is collected over 10,000 inputs with target = 'llvm'
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] shoubhik commented on a change in pull request
#4714: Mxnet parser for Qnn dialect
Posted by GitBox <gi...@apache.org>.
shoubhik commented on a change in pull request #4714: Mxnet parser for Qnn dialect
URL: https://github.com/apache/incubator-tvm/pull/4714#discussion_r367724657
##########
File path: python/tvm/relay/frontend/mxnet.py
##########
@@ -180,6 +190,17 @@ def _mx_conv2d(inputs, attrs):
new_attrs["groups"] = attrs.get_int("num_group", 1)
new_attrs["data_layout"] = data_layout
new_attrs["kernel_layout"] = kernel_layout
+ return new_attrs
+
+def _mx_conv2d(inputs, attrs):
+ kernel_size = attrs.get_int_tuple("kernel")
+ data_layout = attrs.get_str("layout", "NCHW")
+ if len(kernel_size) != 2:
+ raise tvm.error.OpAttributeInvalid(
+ 'Non 1D or 2D kernels are not supported for operator Convolution')
Review comment:
This condition seems to be from an older version of mxnet. It makes more sense to put this change in a separate PR after more extensive testing.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] shoubhik commented on a change in pull request
#4714: Mxnet parser for Qnn dialect
Posted by GitBox <gi...@apache.org>.
shoubhik commented on a change in pull request #4714: Mxnet parser for Qnn dialect
URL: https://github.com/apache/incubator-tvm/pull/4714#discussion_r367201345
##########
File path: python/tvm/relay/frontend/mxnet.py
##########
@@ -14,12 +14,14 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
-# pylint: disable=invalid-name, import-self, len-as-condition, no-else-return
+# pylint: disable=invalid-name, import-self, len-as-condition, no-else-return, too-many-lines
Review comment:
This particular one is about the file, where pylint complains about too many lines in the file.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] anijain2305 commented on a change in pull request
#4714: Mxnet parser for Qnn dialect
Posted by GitBox <gi...@apache.org>.
anijain2305 commented on a change in pull request #4714: Mxnet parser for Qnn dialect
URL: https://github.com/apache/incubator-tvm/pull/4714#discussion_r367814763
##########
File path: src/relay/qnn/op/convolution.cc
##########
@@ -57,7 +57,7 @@ bool QnnConv2DRel(const Array<Type>& types, int num_inputs, const Attrs& attrs,
CHECK(IsScalarType(types[2], DataType::Int(32))); // input_zero_point
CHECK(IsScalarType(types[3], DataType::Int(32))); // kernel_zero_point
CHECK(IsScalarType(types[4], DataType::Float(32))); // input_scale
- CHECK(IsScalarType(types[5], DataType::Float(32))); // kernel_scale
+ // AssignType(types[1], DataType::Float(32), data->shape[axis], reporter); // kernel_scale
Review comment:
Handled in a separate PR - https://github.com/apache/incubator-tvm/pull/4732
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] anijain2305 commented on issue #4714: Mxnet parser
for Qnn dialect
Posted by GitBox <gi...@apache.org>.
anijain2305 commented on issue #4714: Mxnet parser for Qnn dialect
URL: https://github.com/apache/incubator-tvm/pull/4714#issuecomment-576905072
With the latest commit, we have good accuracy for all MxNet-MKLDNN quantized models
@tmoreau89 @yzhliu @tqchen @jackwish @FrozenGene @liangfu
| MxNet | TVM | Degradation
-- | -- | -- | --
| Top1 | Top5 | Top1 | Top5 | Top1 | Top5
Resnet18_v1 | 69.76 | 89.02 | 69.85 | 89.09 | -0.09 | -0.07
Resnet50_v1 | 76.13 | 92.6 | 75.9 | 92.66 | 0.23 | -0.06
Resnet50_v1b | 76.66 | 92.6 | 76.45 | 92.57 | 0.21 | 0.03
Resnet101_v1 | 77.13 | 93.06 | 77 | 93.06 | 0.13 | 0
Resnet152_v2 | 75.99 | 92.52 | 75.32 | 92.26 | 0.67 | 0.26
Inception-V3 | 77.84 | 93.52 | 77.28 | 93.32 | 0.56 | 0.2
Inception-BN | 71.96 | 90.38 | 71.79 | 90.25 | 0.17 | 0.13
MobileNetV1 | 71.27 | 90.09 | 71.13 | 90.16 | 0.14 | -0.07
MobileNetV2 | 70.35 | 89.45 | 70.19 | 89.5 | 0.16 | -0.05
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] shoubhik commented on a change in pull request
#4714: Mxnet parser for Qnn dialect
Posted by GitBox <gi...@apache.org>.
shoubhik commented on a change in pull request #4714: Mxnet parser for Qnn dialect
URL: https://github.com/apache/incubator-tvm/pull/4714#discussion_r367651664
##########
File path: python/tvm/relay/frontend/mxnet.py
##########
@@ -180,6 +190,17 @@ def _mx_conv2d(inputs, attrs):
new_attrs["groups"] = attrs.get_int("num_group", 1)
new_attrs["data_layout"] = data_layout
new_attrs["kernel_layout"] = kernel_layout
+ return new_attrs
+
+def _mx_conv2d(inputs, attrs):
+ kernel_size = attrs.get_int_tuple("kernel")
+ data_layout = attrs.get_str("layout", "NCHW")
+ if len(kernel_size) != 2:
+ raise tvm.error.OpAttributeInvalid(
+ 'Non 1D or 2D kernels are not supported for operator Convolution')
Review comment:
Fixed the message
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] masahi commented on issue #4714: Mxnet parser for
Qnn dialect
Posted by GitBox <gi...@apache.org>.
masahi commented on issue #4714: Mxnet parser for Qnn dialect
URL: https://github.com/apache/incubator-tvm/pull/4714#issuecomment-582407047
thanks @shoubhik @anijain2305 @jackwish @FrozenGene @liangfu this is merged.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] masahi commented on issue #4714: Mxnet parser for
Qnn dialect
Posted by GitBox <gi...@apache.org>.
masahi commented on issue #4714: Mxnet parser for Qnn dialect
URL: https://github.com/apache/incubator-tvm/pull/4714#issuecomment-583791983
> My suggestion is at this point we check in the parser code along with the the test scripts we have for benchmarking the QNN networks
Where are that test scripts? I want to run them @shoubhik @anijain2305
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] shoubhik commented on a change in pull request
#4714: Mxnet parser for Qnn dialect
Posted by GitBox <gi...@apache.org>.
shoubhik commented on a change in pull request #4714: Mxnet parser for Qnn dialect
URL: https://github.com/apache/incubator-tvm/pull/4714#discussion_r367724657
##########
File path: python/tvm/relay/frontend/mxnet.py
##########
@@ -180,6 +190,17 @@ def _mx_conv2d(inputs, attrs):
new_attrs["groups"] = attrs.get_int("num_group", 1)
new_attrs["data_layout"] = data_layout
new_attrs["kernel_layout"] = kernel_layout
+ return new_attrs
+
+def _mx_conv2d(inputs, attrs):
+ kernel_size = attrs.get_int_tuple("kernel")
+ data_layout = attrs.get_str("layout", "NCHW")
+ if len(kernel_size) != 2:
+ raise tvm.error.OpAttributeInvalid(
+ 'Non 1D or 2D kernels are not supported for operator Convolution')
Review comment:
This condition seems to be from an older version of mxnet. It makes more sense to put this change in a separate PR after more extensive testing.
Latest mxnet https://github.com/apache/incubator-mxnet/blob/master/src/operator/nn/convolution.cc#L364-L368 does support 1,2,3 dim kernels
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] liangfu commented on a change in pull request
#4714: Mxnet parser for Qnn dialect
Posted by GitBox <gi...@apache.org>.
liangfu commented on a change in pull request #4714: Mxnet parser for Qnn dialect
URL: https://github.com/apache/incubator-tvm/pull/4714#discussion_r367196432
##########
File path: src/relay/qnn/op/convolution.cc
##########
@@ -57,7 +57,7 @@ bool QnnConv2DRel(const Array<Type>& types, int num_inputs, const Attrs& attrs,
CHECK(IsScalarType(types[2], DataType::Int(32))); // input_zero_point
CHECK(IsScalarType(types[3], DataType::Int(32))); // kernel_zero_point
CHECK(IsScalarType(types[4], DataType::Float(32))); // input_scale
- CHECK(IsScalarType(types[5], DataType::Float(32))); // kernel_scale
+ // AssignType(types[1], DataType::Float(32), data->shape[axis], reporter); // kernel_scale
Review comment:
Why are we removing type check of `types[5]` ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] shoubhik commented on a change in pull request
#4714: Mxnet parser for Qnn dialect
Posted by GitBox <gi...@apache.org>.
shoubhik commented on a change in pull request #4714: Mxnet parser for Qnn dialect
URL: https://github.com/apache/incubator-tvm/pull/4714#discussion_r367651907
##########
File path: python/tvm/relay/frontend/mxnet.py
##########
@@ -1075,6 +1098,377 @@ def _mx_cond(inputs, attrs, subgraphs):
return ret
+def _qnn_mx_contrib_quantize(inputs, attrs):
Review comment:
Renamed the ops so that they no longer have _mx_ prefix in them.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] shoubhik commented on a change in pull request
#4714: Mxnet parser for Qnn dialect
Posted by GitBox <gi...@apache.org>.
shoubhik commented on a change in pull request #4714: Mxnet parser for Qnn dialect
URL: https://github.com/apache/incubator-tvm/pull/4714#discussion_r367724657
##########
File path: python/tvm/relay/frontend/mxnet.py
##########
@@ -180,6 +190,17 @@ def _mx_conv2d(inputs, attrs):
new_attrs["groups"] = attrs.get_int("num_group", 1)
new_attrs["data_layout"] = data_layout
new_attrs["kernel_layout"] = kernel_layout
+ return new_attrs
+
+def _mx_conv2d(inputs, attrs):
+ kernel_size = attrs.get_int_tuple("kernel")
+ data_layout = attrs.get_str("layout", "NCHW")
+ if len(kernel_size) != 2:
+ raise tvm.error.OpAttributeInvalid(
+ 'Non 1D or 2D kernels are not supported for operator Convolution')
Review comment:
This condition seems to be from an older version of mxnet. It makes more sense to put this change in a separate PR after more extensive testing.
Latest mxnet https://github.com/apache/incubator-mxnet/blob/master/src/operator/nn/convolution.cc#L364-L368 does support 1,2,3 dic kernels
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] masahi edited a comment on issue #4714: Mxnet
parser for Qnn dialect
Posted by GitBox <gi...@apache.org>.
masahi edited a comment on issue #4714: Mxnet parser for Qnn dialect
URL: https://github.com/apache/incubator-tvm/pull/4714#issuecomment-575442459
it would be great if you could add an test case on end to end network translation like in our tflite test case.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] liangfu commented on a change in pull request
#4714: Mxnet parser for Qnn dialect
Posted by GitBox <gi...@apache.org>.
liangfu commented on a change in pull request #4714: Mxnet parser for Qnn dialect
URL: https://github.com/apache/incubator-tvm/pull/4714#discussion_r367198080
##########
File path: python/tvm/relay/frontend/mxnet.py
##########
@@ -1075,6 +1098,377 @@ def _mx_cond(inputs, attrs, subgraphs):
return ret
+def _qnn_mx_contrib_quantize(inputs, attrs):
Review comment:
Can we put functions with `_qnn_mx` prefix into mxnet_qnn_op_utils.py file as well?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services