[GitHub] reminisce commented on a change in pull request #9552: [MXNET-133] Model Quantization with Calibration

[GitBox <gi...@apache.org>]

reminisce commented on a change in pull request #9552: [MXNET-133] Model Quantization with Calibration
URL: https://github.com/apache/incubator-mxnet/pull/9552#discussion_r176749839
 
 

 ##########
 File path: src/operator/quantization/dequantize.cc
 ##########
 @@ -30,27 +30,32 @@ DMLC_REGISTER_PARAMETER(DequantizeParam);
 
 NNVM_REGISTER_OP(_contrib_dequantize)
 .describe(R"code(Dequantize the input tensor into a float tensor.
-[min_range, max_range] are scalar floats that spcify the range for
+min_range and max_range are scalar floats that specify the range for
 the output data.
 
-Each value of the tensor will undergo the following:
+When input data type is `uint8`, the output is calculated using the following equation:
 
-`out[i] = min_range + (in[i] * (max_range - min_range) / range(INPUT_TYPE))`
+`out[i] = in[i] * (max_range - min_range) / 255.0`,
 
-here `range(T) = numeric_limits<T>::max() - numeric_limits<T>::min()`
+When input data type is `int8`, the output is calculate using the following equation
+by keep zero centered for the quantized value:
+
+`out[i] = in[i] * MaxAbs(min_range, max_range) / 127.0`,
+
+.. Note::
+    This operator only supports forward propogation. DO NOT use it in training.
 
 Review comment:
   Same as other operators with no backward computation function implemented used in training. It will complain that the attribute `FGradient` is not implemented for this operator.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services