You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2022/08/31 19:54:13 UTC
[GitHub] [tvm] mbrookhart opened a new pull request, #12666: [FQ2I] Quantized constant bias
mbrookhart opened a new pull request, #12666:
URL: https://github.com/apache/tvm/pull/12666
Saw this on a QAT model
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] AndrewZhaoLuo commented on a diff in pull request #12666: [FQ2I] Quantized constant bias
Posted by GitBox <gi...@apache.org>.
AndrewZhaoLuo commented on code in PR #12666:
URL: https://github.com/apache/tvm/pull/12666#discussion_r960212897
##########
python/tvm/relay/transform/fake_quantization_to_integer.py:
##########
@@ -158,23 +157,29 @@ def bias_add(expr, type_map):
"""Rewrite a bias_add op"""
x, b = expr.args
x_t = type_map[x]
- b_t = type_map[b]
- in_scale = fold_constant(x_t.scale)
- in_zero_point = fold_constant(x_t.zero_point)
- if not (
- approx_equal(x_t.scale, b_t.scale)
- and approx_equal(x_t.zero_point, b_t.zero_point)
- and tvm.ir.structural_equal(x_t.dtype, b_t.dtype)
- ):
- b = relay.qnn.op.requantize(
- b,
- b_t.scale,
- b_t.zero_point,
- in_scale,
- in_zero_point,
- out_dtype=x_t.dtype,
- axis=0,
- )
+ if b in type_map:
+ # Ensure bias matches the previous op
+ b_t = type_map[b]
+ in_scale = fold_constant(x_t.scale)
+ in_zero_point = fold_constant(x_t.zero_point)
+ if not (
+ approx_equal(x_t.scale, b_t.scale)
+ and approx_equal(x_t.zero_point, b_t.zero_point)
+ and tvm.ir.structural_equal(x_t.dtype, b_t.dtype)
+ ):
+ b = relay.qnn.op.requantize(
+ b,
+ b_t.scale,
+ b_t.zero_point,
+ in_scale,
+ in_zero_point,
+ out_dtype=x_t.dtype,
+ axis=0,
+ )
+ else:
Review Comment:
Can we just do something like
`type_map.get(b, x_t) # assume is x_t's type if b is not fake quantized`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] AndrewZhaoLuo merged pull request #12666: [FQ2I] Quantized constant bias
Posted by GitBox <gi...@apache.org>.
AndrewZhaoLuo merged PR #12666:
URL: https://github.com/apache/tvm/pull/12666
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] AndrewZhaoLuo commented on a diff in pull request #12666: [FQ2I] Quantized constant bias
Posted by GitBox <gi...@apache.org>.
AndrewZhaoLuo commented on code in PR #12666:
URL: https://github.com/apache/tvm/pull/12666#discussion_r960212897
##########
python/tvm/relay/transform/fake_quantization_to_integer.py:
##########
@@ -158,23 +157,29 @@ def bias_add(expr, type_map):
"""Rewrite a bias_add op"""
x, b = expr.args
x_t = type_map[x]
- b_t = type_map[b]
- in_scale = fold_constant(x_t.scale)
- in_zero_point = fold_constant(x_t.zero_point)
- if not (
- approx_equal(x_t.scale, b_t.scale)
- and approx_equal(x_t.zero_point, b_t.zero_point)
- and tvm.ir.structural_equal(x_t.dtype, b_t.dtype)
- ):
- b = relay.qnn.op.requantize(
- b,
- b_t.scale,
- b_t.zero_point,
- in_scale,
- in_zero_point,
- out_dtype=x_t.dtype,
- axis=0,
- )
+ if b in type_map:
+ # Ensure bias matches the previous op
+ b_t = type_map[b]
+ in_scale = fold_constant(x_t.scale)
+ in_zero_point = fold_constant(x_t.zero_point)
+ if not (
+ approx_equal(x_t.scale, b_t.scale)
+ and approx_equal(x_t.zero_point, b_t.zero_point)
+ and tvm.ir.structural_equal(x_t.dtype, b_t.dtype)
+ ):
+ b = relay.qnn.op.requantize(
+ b,
+ b_t.scale,
+ b_t.zero_point,
+ in_scale,
+ in_zero_point,
+ out_dtype=x_t.dtype,
+ axis=0,
+ )
+ else:
Review Comment:
Can we just do something like
type_map.get(b, x_t) # assume is x_t's type if b is not fake quantized
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org