You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2020/03/13 04:12:11 UTC
[GitHub] [incubator-tvm] masahi opened a new pull request #5061: [Torch,
QNN] Remove FP32 piggy back and use QNN add/mul/concatenate
masahi opened a new pull request #5061: [Torch, QNN] Remove FP32 piggy back and use QNN add/mul/concatenate
URL: https://github.com/apache/incubator-tvm/pull/5061
@anijain2305
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] masahi merged pull request #5061: [Torch,
QNN] Remove FP32 piggy back and use QNN add/mul/concatenate
Posted by GitBox <gi...@apache.org>.
masahi merged pull request #5061: [Torch, QNN] Remove FP32 piggy back and use QNN add/mul/concatenate
URL: https://github.com/apache/incubator-tvm/pull/5061
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] masahi edited a comment on issue #5061: [Torch,
QNN] Remove FP32 piggy back and use QNN add/mul/concatenate
Posted by GitBox <gi...@apache.org>.
masahi edited a comment on issue #5061: [Torch, QNN] Remove FP32 piggy back and use QNN add/mul/concatenate
URL: https://github.com/apache/incubator-tvm/pull/5061#issuecomment-598902899
@anijain2305 @jwfromm @jjohnson-arm
Here is the current result on mobilenet v2, using QNN `add` and post training calibration (**which is wrong**).
```
Model name: mobilenet_v2, per channel quantization
PyTorch accuracy: Top1 = 67.87, Top5 = 88.15
TVM accuracy: Top1 = 62.47, Top5 = 84.67
PyTorch top5 label: [101 386 51 385 69]
TVM top5 label: [101 386 51 385 340]
PyTorch top5 raw output: [18.233843 16.314491 15.674707 13.115572 12.795679]
TVM top5 raw output: [27.510712 26.231144 21.752655 20.153194 17.274168]
max abs diff: 9.916653
mean abs_diff: 2.0649028
50 in 1000 raw outputs correct.
```
We lost 5 point accuracy compared to Torch.
And here is **without** post training calibration, also using QNN `add`. Now the top1 accuracy is much better and almost the same as Torch. Moreover, the raw output of the network, 1000 floating point values, are much closer to Torch. The former has only 50 out of 1000 outputs identical, while the latter, correct one has 376/1000.
```
Model name: mobilenet_v2, per channel quantization
PyTorch accuracy: Top1 = 71.32, Top5 = 89.86
TVM accuracy: Top1 = 71.27, Top5 = 89.86
PyTorch top5 label: [101 386 385 51 340]
TVM top5 label: [101 386 385 51 340]
PyTorch top5 raw output: [20.168097 18.80845 17.222195 13.59647 9.290921]
TVM top5 raw output: [19.941488 18.581842 16.995586 13.823077 9.064313]
max abs diff: 0.9064312
mean abs_diff: 0.17562106
376 in 1000 raw outputs correct.
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] masahi commented on a change in pull request #5061:
[Torch, QNN] Remove FP32 piggy back and use QNN add/mul/concatenate
Posted by GitBox <gi...@apache.org>.
masahi commented on a change in pull request #5061: [Torch, QNN] Remove FP32 piggy back and use QNN add/mul/concatenate
URL: https://github.com/apache/incubator-tvm/pull/5061#discussion_r392444554
##########
File path: python/tvm/relay/frontend/qnn_torch.py
##########
@@ -574,30 +577,69 @@ def _impl(inputs, _):
output_zero_point,
axis=-1,
out_dtype="uint8")
+
+ def _impl(inputs, _):
+ lhs = inputs[0]
+ rhs = inputs[1]
+ output_scale = _expr.const(inputs[2])
+ output_zero_point = _expr.const(inputs[3])
+ assert len(inputs) == 8, "Input quant params not found in op inputs"
+ # Manually added by add_input_quant_params_to_op_inputs above
+ input_scale_lhs = _expr.const(inputs[4])
+ input_zero_point_lhs = _expr.const(inputs[5])
+ input_scale_rhs = _expr.const(inputs[6])
+ input_zero_point_rhs = _expr.const(inputs[7])
+
+ if fp32_piggy_back:
+ logging.info("Piggy backing to FP32 op (PyTorch way)")
+ return torch_impl(lhs, rhs, input_scale_lhs, input_zero_point_lhs,
+ input_scale_rhs, input_zero_point_rhs,
+ output_scale, output_zero_point)
+
+ logging.info("Using QNN add/mul")
Review comment:
done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] masahi commented on issue #5061: [Torch,
QNN] Remove FP32 piggy back and use QNN add/mul/concatenate
Posted by GitBox <gi...@apache.org>.
masahi commented on issue #5061: [Torch, QNN] Remove FP32 piggy back and use QNN add/mul/concatenate
URL: https://github.com/apache/incubator-tvm/pull/5061#issuecomment-598948308
Thanks @anijain2305
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] anijain2305 commented on a change in pull request
#5061: [Torch, QNN] Remove FP32 piggy back and use QNN add/mul/concatenate
Posted by GitBox <gi...@apache.org>.
anijain2305 commented on a change in pull request #5061: [Torch, QNN] Remove FP32 piggy back and use QNN add/mul/concatenate
URL: https://github.com/apache/incubator-tvm/pull/5061#discussion_r392441508
##########
File path: python/tvm/relay/frontend/qnn_torch.py
##########
@@ -574,30 +577,69 @@ def _impl(inputs, _):
output_zero_point,
axis=-1,
out_dtype="uint8")
+
+ def _impl(inputs, _):
+ lhs = inputs[0]
+ rhs = inputs[1]
+ output_scale = _expr.const(inputs[2])
+ output_zero_point = _expr.const(inputs[3])
+ assert len(inputs) == 8, "Input quant params not found in op inputs"
+ # Manually added by add_input_quant_params_to_op_inputs above
+ input_scale_lhs = _expr.const(inputs[4])
+ input_zero_point_lhs = _expr.const(inputs[5])
+ input_scale_rhs = _expr.const(inputs[6])
+ input_zero_point_rhs = _expr.const(inputs[7])
+
+ if fp32_piggy_back:
+ logging.info("Piggy backing to FP32 op (PyTorch way)")
+ return torch_impl(lhs, rhs, input_scale_lhs, input_zero_point_lhs,
+ input_scale_rhs, input_zero_point_rhs,
+ output_scale, output_zero_point)
+
+ logging.info("Using QNN add/mul")
Review comment:
Remove this?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] masahi commented on issue #5061: [Torch,
QNN] Remove FP32 piggy back and use QNN add/mul/concatenate
Posted by GitBox <gi...@apache.org>.
masahi commented on issue #5061: [Torch, QNN] Remove FP32 piggy back and use QNN add/mul/concatenate
URL: https://github.com/apache/incubator-tvm/pull/5061#issuecomment-598902899
@anijain2305 @jwfromm @jjohnson-arm
Here is the current result on mobilenet v2, using QNN `add` and post training calibration (which is wrong).
```
Model name: mobilenet_v2, per channel quantization
PyTorch accuracy: Top1 = 67.87, Top5 = 88.15
TVM accuracy: Top1 = 62.47, Top5 = 84.67
PyTorch top5 label: [101 386 51 385 69]
TVM top5 label: [101 386 51 385 340]
PyTorch top5 raw output: [18.233843 16.314491 15.674707 13.115572 12.795679]
TVM top5 raw output: [27.510712 26.231144 21.752655 20.153194 17.274168]
max abs diff: 9.916653
mean abs_diff: 2.0649028
50 in 1000 raw outputs correct.
```
We lost 5 point accuracy compared to Torch.
And here is without post training calibration, also using QNN `add`. Now the top1 accuracy is much better and almost the same as Torch. Moreover, the raw output of the network, 1000 floating point values, are much closer to Torch. The former has only 50 out of 1000 outputs identical, while the latter, correct one has 376/1000.
```
Model name: mobilenet_v2, per channel quantization
PyTorch accuracy: Top1 = 71.32, Top5 = 89.86
TVM accuracy: Top1 = 71.27, Top5 = 89.86
PyTorch top5 label: [101 386 385 51 340]
TVM top5 label: [101 386 385 51 340]
PyTorch top5 raw output: [20.168097 18.80845 17.222195 13.59647 9.290921]
TVM top5 raw output: [19.941488 18.581842 16.995586 13.823077 9.064313]
max abs diff: 0.9064312
mean abs_diff: 0.17562106
376 in 1000 raw outputs correct.
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services