You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2020/01/29 12:41:20 UTC
[GitHub] [incubator-tvm] inadob opened a new pull request #4789:
[Frontend][TFLite] Dynamically calculate input_stats of any fake_quant range
inadob opened a new pull request #4789: [Frontend][TFLite] Dynamically calculate input_stats of any fake_quant range
URL: https://github.com/apache/incubator-tvm/pull/4789
* pass the input range to the convertor and caclulate (mean, scale) there
* change the range of the second tensor in elemwise operations
so that we test inputs with different quant params
* change the possible output range for elemwise ops wrt the updated ranges
* update the comments for (m, s) calculations
* add input range dict to tflite reduce_mean operation
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] kevinthesun merged pull request #4789:
[Frontend][TFLite] Dynamically calculate input_stats of any fake_quant range
Posted by GitBox <gi...@apache.org>.
kevinthesun merged pull request #4789: [Frontend][TFLite] Dynamically calculate input_stats of any fake_quant range
URL: https://github.com/apache/incubator-tvm/pull/4789
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] inadob commented on a change in pull request #4789:
[Frontend][TFLite] Dynamically calculate input_stats of any fake_quant range
Posted by GitBox <gi...@apache.org>.
inadob commented on a change in pull request #4789: [Frontend][TFLite] Dynamically calculate input_stats of any fake_quant range
URL: https://github.com/apache/incubator-tvm/pull/4789#discussion_r374629200
##########
File path: tests/python/frontend/tflite/test_forward.py
##########
@@ -143,11 +144,13 @@ def compare_tflite_with_tvm(in_data, in_name, input_tensors,
converter.inference_type = tf.lite.constants.QUANTIZED_UINT8
input_arrays = converter.get_input_arrays()
input_stats = {}
- # hardcode the mean_values and std_dev_values (m,s) to be the same
- # if all inputs are in (float_min; float_max) == (-100, 100)
+ # calculate the mean and quantization scale for every input tensor,
+ # with respect to its fp32 input range, defined in fake_quant.
# s = 255/(fmax-fmin); m = -fmin*s (the zero point)
for i in input_arrays:
- input_stats[i] = (128., 1.275)
+ quant_scale = 255 / (input_range[i][1] - input_range[i][0])
Review comment:
Thanks for reviewing! Adding a check makes sense - I will do it.
Here you can see the documentation I used for developing the qnn tests
- at 'Quantizing models for integer-only execution': **_std_dev is 255 / (float_max - float_min)_** https://www.tensorflow.org/lite/convert/quantization
- at 'Model flags': **_real_value = (quantized_input_value - mean_value) / std_dev_value_**
https://www.tensorflow.org/lite/convert/cmdline_reference
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] inadob commented on a change in pull request #4789:
[Frontend][TFLite] Dynamically calculate input_stats of any fake_quant range
Posted by GitBox <gi...@apache.org>.
inadob commented on a change in pull request #4789: [Frontend][TFLite] Dynamically calculate input_stats of any fake_quant range
URL: https://github.com/apache/incubator-tvm/pull/4789#discussion_r374629200
##########
File path: tests/python/frontend/tflite/test_forward.py
##########
@@ -143,11 +144,13 @@ def compare_tflite_with_tvm(in_data, in_name, input_tensors,
converter.inference_type = tf.lite.constants.QUANTIZED_UINT8
input_arrays = converter.get_input_arrays()
input_stats = {}
- # hardcode the mean_values and std_dev_values (m,s) to be the same
- # if all inputs are in (float_min; float_max) == (-100, 100)
+ # calculate the mean and quantization scale for every input tensor,
+ # with respect to its fp32 input range, defined in fake_quant.
# s = 255/(fmax-fmin); m = -fmin*s (the zero point)
for i in input_arrays:
- input_stats[i] = (128., 1.275)
+ quant_scale = 255 / (input_range[i][1] - input_range[i][0])
Review comment:
Thanks for reviewing! Adding a check makes sense - I will do it.
Here you can see the documentation I used for developing the qnn tests
- at 'Quantizing models for integer-only execution': **_std_dev is 255 / (float_max - float_min)_**
https://www.tensorflow.org/lite/convert/quantization
- at 'Model flags': **_real_value = (quantized_input_value - mean_value) / std_dev_value_**
https://www.tensorflow.org/lite/convert/cmdline_reference
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] anijain2305 commented on a change in pull request
#4789: [Frontend][TFLite] Dynamically calculate input_stats of any
fake_quant range
Posted by GitBox <gi...@apache.org>.
anijain2305 commented on a change in pull request #4789: [Frontend][TFLite] Dynamically calculate input_stats of any fake_quant range
URL: https://github.com/apache/incubator-tvm/pull/4789#discussion_r374350076
##########
File path: tests/python/frontend/tflite/test_forward.py
##########
@@ -143,11 +144,13 @@ def compare_tflite_with_tvm(in_data, in_name, input_tensors,
converter.inference_type = tf.lite.constants.QUANTIZED_UINT8
input_arrays = converter.get_input_arrays()
input_stats = {}
- # hardcode the mean_values and std_dev_values (m,s) to be the same
- # if all inputs are in (float_min; float_max) == (-100, 100)
+ # calculate the mean and quantization scale for every input tensor,
+ # with respect to its fp32 input range, defined in fake_quant.
# s = 255/(fmax-fmin); m = -fmin*s (the zero point)
for i in input_arrays:
- input_stats[i] = (128., 1.275)
+ quant_scale = 255 / (input_range[i][1] - input_range[i][0])
Review comment:
Minor
* Lets add a check to ensure that denominator is not zero.
* Typically QNN_scale = FP32_range/QNN_range, so suggest to double-check if TFLite accepts inverse scale.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] inadob commented on issue #4789: [Frontend][TFLite]
Dynamically calculate input_stats of any fake_quant range
Posted by GitBox <gi...@apache.org>.
inadob commented on issue #4789: [Frontend][TFLite] Dynamically calculate input_stats of any fake_quant range
URL: https://github.com/apache/incubator-tvm/pull/4789#issuecomment-579739378
@kevinthesun @anijain2305 can you please take a look
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] inadob commented on a change in pull request #4789:
[Frontend][TFLite] Dynamically calculate input_stats of any fake_quant range
Posted by GitBox <gi...@apache.org>.
inadob commented on a change in pull request #4789: [Frontend][TFLite] Dynamically calculate input_stats of any fake_quant range
URL: https://github.com/apache/incubator-tvm/pull/4789#discussion_r374622386
##########
File path: tests/python/frontend/tflite/test_forward.py
##########
@@ -793,13 +799,14 @@ def _test_elemwise(math_op, data, fused_activation_function=None, quantized=Fals
if quantized:
inq_const = tf.quantization.fake_quant_with_min_max_args(data[0], min=-100, max=100, name="const_tensor")
- inq_data = [tf.quantization.fake_quant_with_min_max_args(in_data[0], min=-100, max=100, name="inq_1")]
+ inq_data = [tf.quantization.fake_quant_with_min_max_args(in_data[0], min=-50, max=50, name="inq_1")]
+ input_range = {'inq_1': (-100, 100)}
Review comment:
Thanks for spotting the mistake, I will fix it
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] kevinthesun commented on issue #4789:
[Frontend][TFLite] Dynamically calculate input_stats of any fake_quant range
Posted by GitBox <gi...@apache.org>.
kevinthesun commented on issue #4789: [Frontend][TFLite] Dynamically calculate input_stats of any fake_quant range
URL: https://github.com/apache/incubator-tvm/pull/4789#issuecomment-582582023
Thanks @inadob @wyc-ruiker @anijain2305
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] wyc-ruiker commented on a change in pull request
#4789: [Frontend][TFLite] Dynamically calculate input_stats of any
fake_quant range
Posted by GitBox <gi...@apache.org>.
wyc-ruiker commented on a change in pull request #4789: [Frontend][TFLite] Dynamically calculate input_stats of any fake_quant range
URL: https://github.com/apache/incubator-tvm/pull/4789#discussion_r374618096
##########
File path: tests/python/frontend/tflite/test_forward.py
##########
@@ -793,13 +799,14 @@ def _test_elemwise(math_op, data, fused_activation_function=None, quantized=Fals
if quantized:
inq_const = tf.quantization.fake_quant_with_min_max_args(data[0], min=-100, max=100, name="const_tensor")
- inq_data = [tf.quantization.fake_quant_with_min_max_args(in_data[0], min=-100, max=100, name="inq_1")]
+ inq_data = [tf.quantization.fake_quant_with_min_max_args(in_data[0], min=-50, max=50, name="inq_1")]
+ input_range = {'inq_1': (-100, 100)}
Review comment:
I'm a little confused here. I think the `input_range` of `inq_1` is (-50,50)? Because you modify the range of `inq_data`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services