[GitHub] [incubator-mxnet] ZhennanQin commented on issue #14801: mxnet.ndarray.contrib.requantize introduces too much error

[GitBox <gi...@apache.org>]

ZhennanQin commented on issue #14801: mxnet.ndarray.contrib.requantize introduces too much error
URL: https://github.com/apache/incubator-mxnet/issues/14801#issuecomment-486896785
 
 
   Hi @shoubhik, the problem of your case is,
   
   ```
   requantized_data, requantized_data_min, requantized_data_max = requantize(
       quantized_data.astype('int32'), quantized_data_min, quantized_data_max)
   ```
   
   For `requantize` op, its input `data_min` and `data_max` are not the same as the float32 min/max value before quantization.
   
   So for your case, the expected `data_min` and `data_max` of requantize are,
   
   ```
   data_min = (quantized_data_max - quantized_data_min) / (INT8_MAX_VALUE - INT8_MIN_VALUE) * INT32_MIN_VALUE
   data_max = (quantized_data_max - quantized_data_min) / (INT8_MAX_VALUE - INT8_MIN_VALUE) * INT32_MAX_VALUE
   ```
   Here INT8_MIN_VALUE is the minimum value of int8 type, thus is -128. INT8_MAX_VALUE is the maximum value of int8 type, thus is 127. INT32_MIN_VALUE and INT32_MAX_VALUE are in similar definitions.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services