You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2021/03/24 13:36:33 UTC
[GitHub] [tvm] euntaik opened a new pull request #7736: [frontend][tflite] float16 quant support
euntaik opened a new pull request #7736:
URL: https://github.com/apache/tvm/pull/7736
add float16 quant support for fc and transpose_conv
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] anijain2305 commented on pull request #7736: [frontend][tflite] float16 quant support
Posted by GitBox <gi...@apache.org>.
anijain2305 commented on pull request #7736:
URL: https://github.com/apache/tvm/pull/7736#issuecomment-821890958
Thanks @euntaik This is merged!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] euntaik commented on a change in pull request #7736: [frontend][tflite] float16 quant support
Posted by GitBox <gi...@apache.org>.
euntaik commented on a change in pull request #7736:
URL: https://github.com/apache/tvm/pull/7736#discussion_r614278332
##########
File path: python/tvm/relay/frontend/tflite.py
##########
@@ -2852,11 +2855,18 @@ def convert_transpose_conv(self, op):
# weights tensor type should be UINT8 (quantization) or FLOAT32
assert weights_tensor_type in (TensorType.INT8, TensorType.UINT8, TensorType.FLOAT32)
Review comment:
No we don't. It is FP32. Weights are dequantized and then passed to this op.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] anijain2305 commented on a change in pull request #7736: [frontend][tflite] float16 quant support
Posted by GitBox <gi...@apache.org>.
anijain2305 commented on a change in pull request #7736:
URL: https://github.com/apache/tvm/pull/7736#discussion_r601910828
##########
File path: tests/python/frontend/tflite/test_forward.py
##########
@@ -1043,74 +1049,118 @@ def _test_convolution(
def test_forward_convolution():
for quantized in [False, True]:
- _test_convolution(
- [4, 8, 8, 176], [1, 1, 176, 32], [1, 1], [1, 1], "SAME", "NHWC", quantized=quantized
- )
- _test_convolution(
- [4, 17, 17, 19], [3, 3, 19, 19], [1, 1], [2, 2], "VALID", "NHWC", quantized=quantized
- )
- _test_convolution(
- [4, 17, 17, 124], [1, 1, 124, 19], [1, 1], [1, 1], "SAME", "NHWC", quantized=quantized
- )
- _test_convolution(
- [4, 17, 17, 12], [3, 3, 12, 32], [1, 1], [2, 2], "VALID", "NHWC", quantized=quantized
- )
+ for fp16_quantized in [False, True]:
+ if quantized and fp16_quantized:
+ continue
Review comment:
Why are we skipping the fp16_quantized test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] anijain2305 merged pull request #7736: [frontend][tflite] float16 quant support
Posted by GitBox <gi...@apache.org>.
anijain2305 merged pull request #7736:
URL: https://github.com/apache/tvm/pull/7736
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] anijain2305 commented on pull request #7736: [frontend][tflite] float16 quant support
Posted by GitBox <gi...@apache.org>.
anijain2305 commented on pull request #7736:
URL: https://github.com/apache/tvm/pull/7736#issuecomment-817996155
@euntaik What are the input and output dtypes for the float16 quantized graph? I do not see any `cast` operations in your PR. Is it possible to paste the TFLite `float16` quantized graph here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] euntaik commented on pull request #7736: [frontend][tflite] float16 quant support
Posted by GitBox <gi...@apache.org>.
euntaik commented on pull request #7736:
URL: https://github.com/apache/tvm/pull/7736#issuecomment-818487016
> @euntaik What are the input and output dtypes for the float16 quantized graph? I do not see any `cast` operations in your PR. Is it possible to paste the TFLite `float16` quantized graph here?
Cast(float16 to float32) is already done in the Dequantize node and the resulting expression is passed to the FullyConnected op as weight and bias.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] euntaik commented on a change in pull request #7736: [frontend][tflite] float16 quant support
Posted by GitBox <gi...@apache.org>.
euntaik commented on a change in pull request #7736:
URL: https://github.com/apache/tvm/pull/7736#discussion_r602017703
##########
File path: tests/python/frontend/tflite/test_forward.py
##########
@@ -1043,74 +1049,118 @@ def _test_convolution(
def test_forward_convolution():
for quantized in [False, True]:
- _test_convolution(
- [4, 8, 8, 176], [1, 1, 176, 32], [1, 1], [1, 1], "SAME", "NHWC", quantized=quantized
- )
- _test_convolution(
- [4, 17, 17, 19], [3, 3, 19, 19], [1, 1], [2, 2], "VALID", "NHWC", quantized=quantized
- )
- _test_convolution(
- [4, 17, 17, 124], [1, 1, 124, 19], [1, 1], [1, 1], "SAME", "NHWC", quantized=quantized
- )
- _test_convolution(
- [4, 17, 17, 12], [3, 3, 12, 32], [1, 1], [2, 2], "VALID", "NHWC", quantized=quantized
- )
+ for fp16_quantized in [False, True]:
+ if quantized and fp16_quantized:
+ continue
Review comment:
fp16 quantized path doesn't go through quantized path.
quantized=True and fp16_quantzied=True goes through same code as quantized=False and fp16_quantize=True. So I just wanted to reduce the test time here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] euntaik commented on a change in pull request #7736: [frontend][tflite] float16 quant support
Posted by GitBox <gi...@apache.org>.
euntaik commented on a change in pull request #7736:
URL: https://github.com/apache/tvm/pull/7736#discussion_r614276096
##########
File path: tests/python/frontend/tflite/test_forward.py
##########
@@ -1043,74 +1049,118 @@ def _test_convolution(
def test_forward_convolution():
for quantized in [False, True]:
- _test_convolution(
- [4, 8, 8, 176], [1, 1, 176, 32], [1, 1], [1, 1], "SAME", "NHWC", quantized=quantized
- )
- _test_convolution(
- [4, 17, 17, 19], [3, 3, 19, 19], [1, 1], [2, 2], "VALID", "NHWC", quantized=quantized
- )
- _test_convolution(
- [4, 17, 17, 124], [1, 1, 124, 19], [1, 1], [1, 1], "SAME", "NHWC", quantized=quantized
- )
- _test_convolution(
- [4, 17, 17, 12], [3, 3, 12, 32], [1, 1], [2, 2], "VALID", "NHWC", quantized=quantized
- )
+ for fp16_quantized in [False, True]:
+ if quantized and fp16_quantized:
+ continue
Review comment:
I have addressed your request.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] anijain2305 commented on a change in pull request #7736: [frontend][tflite] float16 quant support
Posted by GitBox <gi...@apache.org>.
anijain2305 commented on a change in pull request #7736:
URL: https://github.com/apache/tvm/pull/7736#discussion_r611824514
##########
File path: python/tvm/relay/frontend/tflite.py
##########
@@ -2852,11 +2855,18 @@ def convert_transpose_conv(self, op):
# weights tensor type should be UINT8 (quantization) or FLOAT32
assert weights_tensor_type in (TensorType.INT8, TensorType.UINT8, TensorType.FLOAT32)
Review comment:
Do you need to change these? Is bias now in FP16?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] anijain2305 commented on a change in pull request #7736: [frontend][tflite] float16 quant support
Posted by GitBox <gi...@apache.org>.
anijain2305 commented on a change in pull request #7736:
URL: https://github.com/apache/tvm/pull/7736#discussion_r611824736
##########
File path: tests/python/frontend/tflite/test_forward.py
##########
@@ -1043,74 +1049,118 @@ def _test_convolution(
def test_forward_convolution():
for quantized in [False, True]:
- _test_convolution(
- [4, 8, 8, 176], [1, 1, 176, 32], [1, 1], [1, 1], "SAME", "NHWC", quantized=quantized
- )
- _test_convolution(
- [4, 17, 17, 19], [3, 3, 19, 19], [1, 1], [2, 2], "VALID", "NHWC", quantized=quantized
- )
- _test_convolution(
- [4, 17, 17, 124], [1, 1, 124, 19], [1, 1], [1, 1], "SAME", "NHWC", quantized=quantized
- )
- _test_convolution(
- [4, 17, 17, 12], [3, 3, 12, 32], [1, 1], [2, 2], "VALID", "NHWC", quantized=quantized
- )
+ for fp16_quantized in [False, True]:
+ if quantized and fp16_quantized:
+ continue
Review comment:
Sorry for the delay. Yes, please. Can you restructure the test such that there is no need of skipping code. One might be confused while reading the code otherwise.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] euntaik commented on a change in pull request #7736: [frontend][tflite] float16 quant support
Posted by GitBox <gi...@apache.org>.
euntaik commented on a change in pull request #7736:
URL: https://github.com/apache/tvm/pull/7736#discussion_r602017703
##########
File path: tests/python/frontend/tflite/test_forward.py
##########
@@ -1043,74 +1049,118 @@ def _test_convolution(
def test_forward_convolution():
for quantized in [False, True]:
- _test_convolution(
- [4, 8, 8, 176], [1, 1, 176, 32], [1, 1], [1, 1], "SAME", "NHWC", quantized=quantized
- )
- _test_convolution(
- [4, 17, 17, 19], [3, 3, 19, 19], [1, 1], [2, 2], "VALID", "NHWC", quantized=quantized
- )
- _test_convolution(
- [4, 17, 17, 124], [1, 1, 124, 19], [1, 1], [1, 1], "SAME", "NHWC", quantized=quantized
- )
- _test_convolution(
- [4, 17, 17, 12], [3, 3, 12, 32], [1, 1], [2, 2], "VALID", "NHWC", quantized=quantized
- )
+ for fp16_quantized in [False, True]:
+ if quantized and fp16_quantized:
+ continue
Review comment:
fp16 quantized path doesn't go through quantized path.
quantized=True and fp16_quantzied=True goes through same code as quantized=False and fp16_quantize=True. So I just wanted to reduce the time test time here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] anijain2305 commented on a change in pull request #7736: [frontend][tflite] float16 quant support
Posted by GitBox <gi...@apache.org>.
anijain2305 commented on a change in pull request #7736:
URL: https://github.com/apache/tvm/pull/7736#discussion_r611823253
##########
File path: tests/python/frontend/tflite/test_forward.py
##########
@@ -1043,74 +1049,118 @@ def _test_convolution(
def test_forward_convolution():
for quantized in [False, True]:
- _test_convolution(
- [4, 8, 8, 176], [1, 1, 176, 32], [1, 1], [1, 1], "SAME", "NHWC", quantized=quantized
- )
- _test_convolution(
- [4, 17, 17, 19], [3, 3, 19, 19], [1, 1], [2, 2], "VALID", "NHWC", quantized=quantized
- )
- _test_convolution(
- [4, 17, 17, 124], [1, 1, 124, 19], [1, 1], [1, 1], "SAME", "NHWC", quantized=quantized
- )
- _test_convolution(
- [4, 17, 17, 12], [3, 3, 12, 32], [1, 1], [2, 2], "VALID", "NHWC", quantized=quantized
- )
+ for fp16_quantized in [False, True]:
+ if quantized and fp16_quantized:
+ continue
Review comment:
Sorry for the delay. Yes, please. Can you restructure the test such that there is no need of skipping code. One might be confused while reading the code otherwise.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] euntaik commented on a change in pull request #7736: [frontend][tflite] float16 quant support
Posted by GitBox <gi...@apache.org>.
euntaik commented on a change in pull request #7736:
URL: https://github.com/apache/tvm/pull/7736#discussion_r611527105
##########
File path: tests/python/frontend/tflite/test_forward.py
##########
@@ -1043,74 +1049,118 @@ def _test_convolution(
def test_forward_convolution():
for quantized in [False, True]:
- _test_convolution(
- [4, 8, 8, 176], [1, 1, 176, 32], [1, 1], [1, 1], "SAME", "NHWC", quantized=quantized
- )
- _test_convolution(
- [4, 17, 17, 19], [3, 3, 19, 19], [1, 1], [2, 2], "VALID", "NHWC", quantized=quantized
- )
- _test_convolution(
- [4, 17, 17, 124], [1, 1, 124, 19], [1, 1], [1, 1], "SAME", "NHWC", quantized=quantized
- )
- _test_convolution(
- [4, 17, 17, 12], [3, 3, 12, 32], [1, 1], [2, 2], "VALID", "NHWC", quantized=quantized
- )
+ for fp16_quantized in [False, True]:
+ if quantized and fp16_quantized:
+ continue
Review comment:
Should I remove the skipping code?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org