You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2021/12/01 16:40:14 UTC
[GitHub] [tvm] ekalda opened a new pull request #9627: [microNPU] Add support for SIGMOID
ekalda opened a new pull request #9627:
URL: https://github.com/apache/tvm/pull/9627
Add support for SIGMOID activation function using the lookup
table mechanism in the NPU.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] ekalda commented on pull request #9627: [microNPU] Add support for SIGMOID
Posted by GitBox <gi...@apache.org>.
ekalda commented on pull request #9627:
URL: https://github.com/apache/tvm/pull/9627#issuecomment-986714139
Thanks for the review @lhutton1!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] lhutton1 commented on a change in pull request #9627: [microNPU] Add support for SIGMOID
Posted by GitBox <gi...@apache.org>.
lhutton1 commented on a change in pull request #9627:
URL: https://github.com/apache/tvm/pull/9627#discussion_r762971609
##########
File path: python/tvm/relay/op/contrib/ethosu.py
##########
@@ -1035,6 +1035,35 @@ def mean_pattern() -> tvm.relay.dataflow_pattern.DFPattern:
return pattern
+class SigmoidParams:
Review comment:
Should we also create a subclass for this and `TanhParams`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] ekalda removed a comment on pull request #9627: [microNPU] Add support for SIGMOID
Posted by GitBox <gi...@apache.org>.
ekalda removed a comment on pull request #9627:
URL: https://github.com/apache/tvm/pull/9627#issuecomment-984560008
https://github.com/apache/tvm/pull/9627
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] lhutton1 commented on a change in pull request #9627: [microNPU] Add support for SIGMOID
Posted by GitBox <gi...@apache.org>.
lhutton1 commented on a change in pull request #9627:
URL: https://github.com/apache/tvm/pull/9627#discussion_r761119882
##########
File path: tests/python/contrib/test_ethosu/test_codegen.py
##########
@@ -1038,5 +1038,70 @@ def representative_dataset():
infra.verify_source(compiled_models, accel_type)
+@pytest.mark.parametrize("accel_type", ACCEL_TYPES)
+@pytest.mark.parametrize("ifm_shape", [[1, 115, 32, 7], [1, 4, 5, 2]])
+def test_tflite_sigmoid(accel_type, ifm_shape):
+ dtype = "int8"
+
+ def create_tflite_graph():
+ tf.config.run_functions_eagerly(True)
+
+ class Model(tf.Module):
+ @tf.function
+ def tanh_function(self, x):
Review comment:
```suggestion
def sigmoid_function(self, x):
```
##########
File path: python/tvm/relay/op/contrib/ethosu.py
##########
@@ -944,6 +944,35 @@ def tanh_pattern():
return quant
+class SigmoidParams:
+ """
+ This class will parse a call to a ethos-u.sigmoid composite function
+ and extract the parameter information.
+ """
+
+ composite_name = "ethos-u.sigmoid"
+
+ def __init__(self, func_body: Call):
+ self.ofm = TensorParams(func_body)
+ self.ifm = TensorParams(func_body.args[0].args[0].args[0])
+
+ def is_valid(self):
+ """
+ This function checks whether reshape has compatible attributes with the NPU
Review comment:
```suggestion
This function checks whether sigmoid has compatible attributes with the NPU
```
##########
File path: python/tvm/relay/backend/contrib/ethosu/legalize.py
##########
@@ -194,6 +194,76 @@ def __call__(self, *args, **kwargs):
pass
+def sigmoid_calc_func(x):
+ """Function to calculate the values for sigmoid"""
+ # Thse limits are inherited from TFLite
+ upper_limit = 8.0
+ lower_limit = -8.0
+
+ if x <= lower_limit:
+ y = 0.0
+ elif x >= upper_limit:
+ y = 1.0
+ else:
+ y = 1 / (1 + math.exp(-x))
+ return y
+
+
+class SigmoidRewriter(DFPatternCallback):
Review comment:
This looks very similar to `TanhRewriter`, is it worth creating a subclass?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] ekalda commented on a change in pull request #9627: [microNPU] Add support for SIGMOID
Posted by GitBox <gi...@apache.org>.
ekalda commented on a change in pull request #9627:
URL: https://github.com/apache/tvm/pull/9627#discussion_r762947023
##########
File path: python/tvm/relay/backend/contrib/ethosu/legalize.py
##########
@@ -194,6 +194,76 @@ def __call__(self, *args, **kwargs):
pass
+def sigmoid_calc_func(x):
+ """Function to calculate the values for sigmoid"""
+ # Thse limits are inherited from TFLite
+ upper_limit = 8.0
+ lower_limit = -8.0
+
+ if x <= lower_limit:
+ y = 0.0
+ elif x >= upper_limit:
+ y = 1.0
+ else:
+ y = 1 / (1 + math.exp(-x))
+ return y
+
+
+class SigmoidRewriter(DFPatternCallback):
Review comment:
Good idea! I made the change
##########
File path: python/tvm/relay/op/contrib/ethosu.py
##########
@@ -944,6 +944,35 @@ def tanh_pattern():
return quant
+class SigmoidParams:
+ """
+ This class will parse a call to a ethos-u.sigmoid composite function
+ and extract the parameter information.
+ """
+
+ composite_name = "ethos-u.sigmoid"
+
+ def __init__(self, func_body: Call):
+ self.ofm = TensorParams(func_body)
+ self.ifm = TensorParams(func_body.args[0].args[0].args[0])
+
+ def is_valid(self):
+ """
+ This function checks whether reshape has compatible attributes with the NPU
Review comment:
Done
##########
File path: tests/python/contrib/test_ethosu/test_codegen.py
##########
@@ -1038,5 +1038,70 @@ def representative_dataset():
infra.verify_source(compiled_models, accel_type)
+@pytest.mark.parametrize("accel_type", ACCEL_TYPES)
+@pytest.mark.parametrize("ifm_shape", [[1, 115, 32, 7], [1, 4, 5, 2]])
+def test_tflite_sigmoid(accel_type, ifm_shape):
+ dtype = "int8"
+
+ def create_tflite_graph():
+ tf.config.run_functions_eagerly(True)
+
+ class Model(tf.Module):
+ @tf.function
+ def tanh_function(self, x):
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] ekalda commented on a change in pull request #9627: [microNPU] Add support for SIGMOID
Posted by GitBox <gi...@apache.org>.
ekalda commented on a change in pull request #9627:
URL: https://github.com/apache/tvm/pull/9627#discussion_r764004004
##########
File path: python/tvm/relay/backend/contrib/ethosu/legalize.py
##########
@@ -194,6 +205,48 @@ def __call__(self, *args, **kwargs):
pass
+def sigmoid_calc_func(x):
Review comment:
Done
##########
File path: python/tvm/relay/backend/contrib/ethosu/legalize.py
##########
@@ -125,30 +125,30 @@ def __call__(self, *args, **kwargs):
pass
-def find_tanh_values(ifm_scale, ifm_zp, ofm_scale, ofm_zp):
- """Method to calculate the values of the tanh lookup table"""
+def get_lut_from_func(ifm_scale, ifm_zp, ofm_scale, ofm_zp, func):
+ """Method to calculate the values of the lookup table based on the calculation function"""
lut_values = list()
# Only int8 is currently supported
dtype = np.int8
qmin, qmax = np.iinfo(dtype).min, np.iinfo(dtype).max
for x in range(qmin, qmax + 1):
x_real = ifm_scale * (x - ifm_zp)
- out_real = math.tanh(x_real)
+ out_real = func(x_real)
lut_result = int(util.round_away_zero(ofm_zp + out_real / ofm_scale))
lut_result = min(qmax, max(qmin, lut_result))
lut_values.append(lut_result)
return lut_values
-class TanhRewriter(DFPatternCallback):
- """This pass adds tanh as a LUT to the identity operator"""
+class LutActivationRewriter(DFPatternCallback):
+ """A class to create an identity operator with the LUT"""
- def __init__(self):
+ def __init__(self, params_class, activation_type, calc_func):
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] ekalda commented on a change in pull request #9627: [microNPU] Add support for SIGMOID
Posted by GitBox <gi...@apache.org>.
ekalda commented on a change in pull request #9627:
URL: https://github.com/apache/tvm/pull/9627#discussion_r764004404
##########
File path: python/tvm/relay/op/contrib/ethosu.py
##########
@@ -1035,6 +1035,35 @@ def mean_pattern() -> tvm.relay.dataflow_pattern.DFPattern:
return pattern
+class SigmoidParams:
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] ekalda commented on pull request #9627: [microNPU] Add support for SIGMOID
Posted by GitBox <gi...@apache.org>.
ekalda commented on pull request #9627:
URL: https://github.com/apache/tvm/pull/9627#issuecomment-987938459
Thanks for the reviews @NicolaLancellotti and @lhutton1! :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] NicolaLancellotti commented on a change in pull request #9627: [microNPU] Add support for SIGMOID
Posted by GitBox <gi...@apache.org>.
NicolaLancellotti commented on a change in pull request #9627:
URL: https://github.com/apache/tvm/pull/9627#discussion_r761296602
##########
File path: tests/python/contrib/test_ethosu/test_codegen.py
##########
@@ -1038,5 +1038,70 @@ def representative_dataset():
infra.verify_source(compiled_models, accel_type)
+@pytest.mark.parametrize("accel_type", ACCEL_TYPES)
+@pytest.mark.parametrize("ifm_shape", [[1, 115, 32, 7], [1, 4, 5, 2]])
+def test_tflite_sigmoid(accel_type, ifm_shape):
+ dtype = "int8"
+
+ def create_tflite_graph():
+ tf.config.run_functions_eagerly(True)
Review comment:
```suggestion
```
We don't need it, do we?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] ekalda commented on a change in pull request #9627: [microNPU] Add support for SIGMOID
Posted by GitBox <gi...@apache.org>.
ekalda commented on a change in pull request #9627:
URL: https://github.com/apache/tvm/pull/9627#discussion_r764003912
##########
File path: python/tvm/relay/backend/contrib/ethosu/legalize.py
##########
@@ -125,30 +125,30 @@ def __call__(self, *args, **kwargs):
pass
-def find_tanh_values(ifm_scale, ifm_zp, ofm_scale, ofm_zp):
- """Method to calculate the values of the tanh lookup table"""
+def get_lut_from_func(ifm_scale, ifm_zp, ofm_scale, ofm_zp, func):
+ """Method to calculate the values of the lookup table based on the calculation function"""
lut_values = list()
# Only int8 is currently supported
dtype = np.int8
qmin, qmax = np.iinfo(dtype).min, np.iinfo(dtype).max
for x in range(qmin, qmax + 1):
x_real = ifm_scale * (x - ifm_zp)
- out_real = math.tanh(x_real)
+ out_real = func(x_real)
lut_result = int(util.round_away_zero(ofm_zp + out_real / ofm_scale))
lut_result = min(qmax, max(qmin, lut_result))
lut_values.append(lut_result)
return lut_values
-class TanhRewriter(DFPatternCallback):
- """This pass adds tanh as a LUT to the identity operator"""
+class LutActivationRewriter(DFPatternCallback):
+ """A class to create an identity operator with the LUT"""
- def __init__(self):
+ def __init__(self, params_class, activation_type, calc_func):
super().__init__(require_type=True, rewrite_once=True)
- self.pattern = (
- wildcard().has_attr({"Composite": ethosu_patterns.TanhParams.composite_name})
- )(wildcard())
+ self.pattern = (wildcard().has_attr({"Composite": params_class.composite_name}))(wildcard())
+ self.activation_type = activation_type
+ self.calc_func = calc_func
def callback(self, pre, post, node_map):
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] ekalda commented on a change in pull request #9627: [microNPU] Add support for SIGMOID
Posted by GitBox <gi...@apache.org>.
ekalda commented on a change in pull request #9627:
URL: https://github.com/apache/tvm/pull/9627#discussion_r762947391
##########
File path: tests/python/contrib/test_ethosu/test_codegen.py
##########
@@ -1038,5 +1038,70 @@ def representative_dataset():
infra.verify_source(compiled_models, accel_type)
+@pytest.mark.parametrize("accel_type", ACCEL_TYPES)
+@pytest.mark.parametrize("ifm_shape", [[1, 115, 32, 7], [1, 4, 5, 2]])
+def test_tflite_sigmoid(accel_type, ifm_shape):
+ dtype = "int8"
+
+ def create_tflite_graph():
+ tf.config.run_functions_eagerly(True)
Review comment:
No we don't :D
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] ekalda commented on a change in pull request #9627: [microNPU] Add support for SIGMOID
Posted by GitBox <gi...@apache.org>.
ekalda commented on a change in pull request #9627:
URL: https://github.com/apache/tvm/pull/9627#discussion_r764004208
##########
File path: python/tvm/relay/backend/contrib/ethosu/legalize.py
##########
@@ -125,30 +125,30 @@ def __call__(self, *args, **kwargs):
pass
-def find_tanh_values(ifm_scale, ifm_zp, ofm_scale, ofm_zp):
- """Method to calculate the values of the tanh lookup table"""
+def get_lut_from_func(ifm_scale, ifm_zp, ofm_scale, ofm_zp, func):
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] manupa-arm commented on pull request #9627: [microNPU] Add support for SIGMOID
Posted by GitBox <gi...@apache.org>.
manupa-arm commented on pull request #9627:
URL: https://github.com/apache/tvm/pull/9627#issuecomment-988728906
Thanks all!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] NicolaLancellotti commented on a change in pull request #9627: [microNPU] Add support for SIGMOID
Posted by GitBox <gi...@apache.org>.
NicolaLancellotti commented on a change in pull request #9627:
URL: https://github.com/apache/tvm/pull/9627#discussion_r762957516
##########
File path: python/tvm/relay/backend/contrib/ethosu/legalize.py
##########
@@ -194,6 +205,48 @@ def __call__(self, *args, **kwargs):
pass
+def sigmoid_calc_func(x):
Review comment:
```suggestion
def sigmoid_calc_func(x: float) -> float:
```
##########
File path: python/tvm/relay/backend/contrib/ethosu/legalize.py
##########
@@ -125,30 +125,30 @@ def __call__(self, *args, **kwargs):
pass
-def find_tanh_values(ifm_scale, ifm_zp, ofm_scale, ofm_zp):
- """Method to calculate the values of the tanh lookup table"""
+def get_lut_from_func(ifm_scale, ifm_zp, ofm_scale, ofm_zp, func):
Review comment:
```suggestion
def get_lut_from_func(ifm_scale: float, ifm_zp: int, ofm_scale: float, ofm_zp: int, func: Callable[[float], float]) -> list[int]:
```
##########
File path: python/tvm/relay/backend/contrib/ethosu/legalize.py
##########
@@ -125,30 +125,30 @@ def __call__(self, *args, **kwargs):
pass
-def find_tanh_values(ifm_scale, ifm_zp, ofm_scale, ofm_zp):
- """Method to calculate the values of the tanh lookup table"""
+def get_lut_from_func(ifm_scale, ifm_zp, ofm_scale, ofm_zp, func):
+ """Method to calculate the values of the lookup table based on the calculation function"""
lut_values = list()
# Only int8 is currently supported
dtype = np.int8
qmin, qmax = np.iinfo(dtype).min, np.iinfo(dtype).max
for x in range(qmin, qmax + 1):
x_real = ifm_scale * (x - ifm_zp)
- out_real = math.tanh(x_real)
+ out_real = func(x_real)
lut_result = int(util.round_away_zero(ofm_zp + out_real / ofm_scale))
lut_result = min(qmax, max(qmin, lut_result))
lut_values.append(lut_result)
return lut_values
-class TanhRewriter(DFPatternCallback):
- """This pass adds tanh as a LUT to the identity operator"""
+class LutActivationRewriter(DFPatternCallback):
+ """A class to create an identity operator with the LUT"""
- def __init__(self):
+ def __init__(self, params_class, activation_type, calc_func):
super().__init__(require_type=True, rewrite_once=True)
- self.pattern = (
- wildcard().has_attr({"Composite": ethosu_patterns.TanhParams.composite_name})
- )(wildcard())
+ self.pattern = (wildcard().has_attr({"Composite": params_class.composite_name}))(wildcard())
+ self.activation_type = activation_type
+ self.calc_func = calc_func
def callback(self, pre, post, node_map):
Review comment:
```suggestion
def callback(self, pre: tvm.relay.Expr, post: tvm.relay.Expr, node_map: tvm.ir.container.Map):
```
##########
File path: python/tvm/relay/backend/contrib/ethosu/legalize.py
##########
@@ -125,30 +125,30 @@ def __call__(self, *args, **kwargs):
pass
-def find_tanh_values(ifm_scale, ifm_zp, ofm_scale, ofm_zp):
- """Method to calculate the values of the tanh lookup table"""
+def get_lut_from_func(ifm_scale, ifm_zp, ofm_scale, ofm_zp, func):
+ """Method to calculate the values of the lookup table based on the calculation function"""
lut_values = list()
# Only int8 is currently supported
dtype = np.int8
qmin, qmax = np.iinfo(dtype).min, np.iinfo(dtype).max
for x in range(qmin, qmax + 1):
x_real = ifm_scale * (x - ifm_zp)
- out_real = math.tanh(x_real)
+ out_real = func(x_real)
lut_result = int(util.round_away_zero(ofm_zp + out_real / ofm_scale))
lut_result = min(qmax, max(qmin, lut_result))
lut_values.append(lut_result)
return lut_values
-class TanhRewriter(DFPatternCallback):
- """This pass adds tanh as a LUT to the identity operator"""
+class LutActivationRewriter(DFPatternCallback):
+ """A class to create an identity operator with the LUT"""
- def __init__(self):
+ def __init__(self, params_class, activation_type, calc_func):
Review comment:
```suggestion
def __init__(self, params_class: Type, activation_type: string, calc_func: Callable[[float], float]):
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] manupa-arm merged pull request #9627: [microNPU] Add support for SIGMOID
Posted by GitBox <gi...@apache.org>.
manupa-arm merged pull request #9627:
URL: https://github.com/apache/tvm/pull/9627
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] ekalda commented on pull request #9627: [microNPU] Add support for SIGMOID
Posted by GitBox <gi...@apache.org>.
ekalda commented on pull request #9627:
URL: https://github.com/apache/tvm/pull/9627#issuecomment-984560008
https://github.com/apache/tvm/pull/9627
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] ekalda commented on pull request #9627: [microNPU] Add support for SIGMOID
Posted by GitBox <gi...@apache.org>.
ekalda commented on pull request #9627:
URL: https://github.com/apache/tvm/pull/9627#issuecomment-983825280
cc @manupa-arm @lhutton1 @NicolaLancellotti @dchauhan-arm
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org