You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by "Aleksei-grovety (via GitHub)" <gi...@apache.org> on 2023/04/14 14:28:14 UTC
[GitHub] [tvm] Aleksei-grovety opened a new pull request, #14629: [microNPU][ETHOSU] Softmax int8 legalization support
Aleksei-grovety opened a new pull request, #14629:
URL: https://github.com/apache/tvm/pull/14629
Add support for legalizing Softmax int8 to list NPU operations as implemented in Vela.
cc @neildhickey, @lhutton1, @ekalda, @leandron, @ilyag-grovety
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] ekalda commented on pull request #14629: [microNPU][ETHOSU] Softmax int8 legalization support
Posted by "ekalda (via GitHub)" <gi...@apache.org>.
ekalda commented on PR #14629:
URL: https://github.com/apache/tvm/pull/14629#issuecomment-1523060180
Thanks @Aleksei-grovety and @ilyag-grovety, this is now merged!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] Aleksei-grovety commented on pull request #14629: [microNPU][ETHOSU] Softmax int8 legalization support
Posted by "Aleksei-grovety (via GitHub)" <gi...@apache.org>.
Aleksei-grovety commented on PR #14629:
URL: https://github.com/apache/tvm/pull/14629#issuecomment-1514708044
There is a problem with run microTVM demos, which consists in a greatly increased time to execute lower_to_te, I'm trying to find out the reason for this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] Aleksei-grovety commented on a diff in pull request #14629: [microNPU][ETHOSU] Softmax int8 legalization support
Posted by "Aleksei-grovety (via GitHub)" <gi...@apache.org>.
Aleksei-grovety commented on code in PR #14629:
URL: https://github.com/apache/tvm/pull/14629#discussion_r1168696585
##########
tests/python/contrib/test_ethosu/test_legalize.py:
##########
@@ -3076,5 +3076,116 @@ def representative_dataset():
assert tuple(func_body.args[1].checked_type.shape) == (256,)
+@pytest.mark.parametrize("ifm_shape", [(1, 12), (1, 12, 32)])
+def test_tflite_softmax(ifm_shape):
+ dtype = "int8"
+
+ def create_tflite_graph():
+ @tf.function
+ def softmax(x):
+ return tf.nn.softmax(x)
+
+ concrete_func = softmax.get_concrete_function(tf.TensorSpec(ifm_shape, dtype=tf.float32))
+ # Convert the model
+ def representative_dataset():
+ for _ in range(100):
+ data = np.random.rand(*tuple(ifm_shape))
+ yield [data.astype(np.float32)]
+
+ converter = tf.lite.TFLiteConverter.from_concrete_functions([concrete_func])
+ converter.optimizations = [tf.lite.Optimize.DEFAULT]
+ converter.representative_dataset = representative_dataset
+ converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
+ converter.inference_input_type = tf.int8
+ converter.inference_output_type = tf.int8
+ tflite_model = converter.convert()
+ return tflite_model
+
+ def verify(ext_func):
+ out_op = ext_func.body
+ ops = []
+ expected_op_names = [
+ "reshape",
+ "reshape",
+ "contrib.ethosu.pooling",
+ "contrib.ethosu.binary_elementwise",
Review Comment:
I'll add it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tvm-bot commented on pull request #14629: [microNPU][ETHOSU] Softmax int8 legalization support
Posted by "tvm-bot (via GitHub)" <gi...@apache.org>.
tvm-bot commented on PR #14629:
URL: https://github.com/apache/tvm/pull/14629#issuecomment-1508646677
<!---bot-comment-->
Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from [Reviewers](https://github.com/apache/incubator-tvm/blob/master/CONTRIBUTORS.md#reviewers) by @-ing them in a comment.
<!--bot-comment-ccs-start-->
* cc @Mousius, @leandron, @lhutton1 <sub>See [#10317](https://github.com/apache/tvm/issues/10317) for details</sub><!--bot-comment-ccs-end-->
<sub>Generated by [tvm-bot](https://github.com/apache/tvm/blob/main/ci/README.md#github-actions)</sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] ilyag-grovety commented on a diff in pull request #14629: [microNPU][ETHOSU] Softmax int8 legalization support
Posted by "ilyag-grovety (via GitHub)" <gi...@apache.org>.
ilyag-grovety commented on code in PR #14629:
URL: https://github.com/apache/tvm/pull/14629#discussion_r1168503842
##########
tests/python/contrib/test_ethosu/test_legalize.py:
##########
@@ -3076,5 +3076,116 @@ def representative_dataset():
assert tuple(func_body.args[1].checked_type.shape) == (256,)
+@pytest.mark.parametrize("ifm_shape", [(1, 12), (1, 12, 32)])
+def test_tflite_softmax(ifm_shape):
+ dtype = "int8"
+
+ def create_tflite_graph():
+ @tf.function
+ def softmax(x):
+ return tf.nn.softmax(x)
+
+ concrete_func = softmax.get_concrete_function(tf.TensorSpec(ifm_shape, dtype=tf.float32))
+ # Convert the model
+ def representative_dataset():
+ for _ in range(100):
+ data = np.random.rand(*tuple(ifm_shape))
+ yield [data.astype(np.float32)]
+
+ converter = tf.lite.TFLiteConverter.from_concrete_functions([concrete_func])
+ converter.optimizations = [tf.lite.Optimize.DEFAULT]
+ converter.representative_dataset = representative_dataset
+ converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
+ converter.inference_input_type = tf.int8
+ converter.inference_output_type = tf.int8
+ tflite_model = converter.convert()
+ return tflite_model
+
+ def verify(ext_func):
+ out_op = ext_func.body
+ ops = []
+ expected_op_names = [
+ "reshape",
+ "reshape",
+ "contrib.ethosu.pooling",
+ "contrib.ethosu.binary_elementwise",
Review Comment:
To make a test more precise, I think it's also better to verify `operator_type` of each `contrib.ethosu.binary_elementwise` op. And maybe a corresponding parameter for `pooling`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] Aleksei-grovety commented on pull request #14629: [microNPU][ETHOSU] Softmax int8 legalization support
Posted by "Aleksei-grovety (via GitHub)" <gi...@apache.org>.
Aleksei-grovety commented on PR #14629:
URL: https://github.com/apache/tvm/pull/14629#issuecomment-1511355241
> Thanks @Aleksei-grovety, looks good to me in general (bar @ilyag-grovety suggestion). Since it is a 500 line beast of an operator, I wonder if it would make sense to break it out into a separate file?
I will add a separate file for Softmax legalization.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] ekalda merged pull request #14629: [microNPU][ETHOSU] Softmax int8 legalization support
Posted by "ekalda (via GitHub)" <gi...@apache.org>.
ekalda merged PR #14629:
URL: https://github.com/apache/tvm/pull/14629
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org