You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2022/04/08 14:39:06 UTC
[GitHub] [tvm] ashutosh-arm opened a new pull request, #10939: [CMSIS-NN] Moved TFLite model making to common area
ashutosh-arm opened a new pull request, #10939:
URL: https://github.com/apache/tvm/pull/10939
Moving out TFLite model creation from CMSIS-NN area to common area of python/tvm/testing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] ashutosh-arm commented on a diff in pull request #10939: [CMSIS-NN] Moved TFLite model making to common area
Posted by GitBox <gi...@apache.org>.
ashutosh-arm commented on code in PR #10939:
URL: https://github.com/apache/tvm/pull/10939#discussion_r856117243
##########
python/tvm/relay/testing/tflite.py:
##########
@@ -0,0 +1,152 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Common utilities for creating TFLite models"""
+from distutils.version import LooseVersion
+import numpy as np
+import pytest
+import tvm
+
+pytest.importorskip("tflite")
+pytest.importorskip("tensorflow")
+import tflite.Model # pylint: disable=wrong-import-position
+import tensorflow as tf # pylint: disable=wrong-import-position
+
+
+class TFLiteModel:
+ """Creates TFLite Model and facilitates reference data generation"""
+
+ def __init__(self, dtype):
+ self.serial_model = None # This is what TFLite convert() provides
+ self.dtype = dtype # This is the dtype of graph inputs
+ self.shape_dict = {}
+ self.dtype_dict = {}
+
+ @tf.function
+ def conv2d_single_function(self, ifm_tensor, args):
+ """Returns TFLite Conv2d layer"""
+ assert len(args) == 6, "Conv2D needs (ifm_shape, kernel_shape, strides, padding, dilation)"
+ _, kernel_shape, strides, padding, dilation, activation = args
+ op = tf.nn.conv2d(
+ ifm_tensor,
+ filters=tf.constant(
+ np.random.uniform(size=[kernel_shape[0], kernel_shape[1], 3, 3]),
+ dtype=tf.float32,
+ ),
+ strides=[1, strides[0], strides[1], 1],
+ padding=padding,
+ dilations=dilation,
+ )
+ if activation == "RELU":
+ op = tf.nn.relu(op)
+ elif activation == "NONE":
+ pass
+ else:
+ assert False, "Unsupported activation {}".format(activation)
+ return op
+
+ def create_tflite_model(self, op_type, *args):
+ """Returns TFLite serial graph, Relay module, Relay params based on op_type"""
+ concrete_func = None
+ input_shape = None
+ if op_type == "conv2d_single":
+ input_shape = args[0]
+ ifm_tensor = tf.TensorSpec(input_shape, dtype=tf.float32, name="input")
+ concrete_func = self.conv2d_single_function.get_concrete_function(ifm_tensor, args)
+ else:
+ assert False, "Unsupported op_type {}".format(op_type)
+
+ def representative_dataset():
+ for _ in range(100):
+ data = np.random.rand(*tuple(input_shape))
Review Comment:
ACK.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] ashutosh-arm commented on pull request #10939: [CMSIS-NN] Moved TFLite model making to common area
Posted by GitBox <gi...@apache.org>.
ashutosh-arm commented on PR #10939:
URL: https://github.com/apache/tvm/pull/10939#issuecomment-1103828264
Thanks @lhutton1. Since there are different types of test graphs in use by different backends, I got a lot of conflicts/failures the first time getting it right. So, I have added the infrastructure for now and we can keep moving individual tests one after the other gradually if its okay with you. For now, I would just like to add this infrastructure, but the idea is to use this common implementation for all tests eventually.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] ashutosh-arm commented on a diff in pull request #10939: [CMSIS-NN] Moved TFLite model making to common area
Posted by GitBox <gi...@apache.org>.
ashutosh-arm commented on code in PR #10939:
URL: https://github.com/apache/tvm/pull/10939#discussion_r856122901
##########
python/tvm/relay/testing/tflite.py:
##########
@@ -0,0 +1,152 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Common utilities for creating TFLite models"""
+from distutils.version import LooseVersion
+import numpy as np
+import pytest
+import tvm
+
+pytest.importorskip("tflite")
+pytest.importorskip("tensorflow")
+import tflite.Model # pylint: disable=wrong-import-position
+import tensorflow as tf # pylint: disable=wrong-import-position
+
+
+class TFLiteModel:
+ """Creates TFLite Model and facilitates reference data generation"""
+
+ def __init__(self, dtype):
+ self.serial_model = None # This is what TFLite convert() provides
+ self.dtype = dtype # This is the dtype of graph inputs
+ self.shape_dict = {}
+ self.dtype_dict = {}
+
+ @tf.function
+ def conv2d_single_function(self, ifm_tensor, args):
Review Comment:
Yeah, I was thinking about it too. But wanted to wait until I had seen more cases that might be potential users of it. The problem with exposing this function is that the end user could chain this with other functions and would expect create_model() to work, but it might not because of the tf.function pragmas.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] leandron merged pull request #10939: [CMSIS-NN] Moved TFLite model making to common area
Posted by GitBox <gi...@apache.org>.
leandron merged PR #10939:
URL: https://github.com/apache/tvm/pull/10939
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] ashutosh-arm commented on pull request #10939: [CMSIS-NN] Moved TFLite model making to common area
Posted by GitBox <gi...@apache.org>.
ashutosh-arm commented on PR #10939:
URL: https://github.com/apache/tvm/pull/10939#issuecomment-1103961788
It seems all checks have passed after resolving the conflicts. Are you okay with the current changes and future plans @lhutton1 and @Mousius?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] ashutosh-arm commented on a diff in pull request #10939: [CMSIS-NN] Moved TFLite model making to common area
Posted by GitBox <gi...@apache.org>.
ashutosh-arm commented on code in PR #10939:
URL: https://github.com/apache/tvm/pull/10939#discussion_r856121051
##########
tests/python/contrib/test_cmsisnn/test_conv2d.py:
##########
@@ -314,25 +312,29 @@ def test_conv2d_int8_tflite(ifm_shape, kernel_shape, strides, dilation, padding,
interface_api = "c"
use_unpacked_api = True
test_runner = AOT_USMP_CORSTONE300_RUNNER
-
dtype = "int8"
- tflite_model, relay_mod, params = create_conv2d_tflite_relay_models(
- ifm_shape, kernel_shape, strides, dilation, padding, activation, dtype
+
+ from tvm.relay.testing.tflite import TFLiteModel
+
+ tfl_model = TFLiteModel(dtype)
+ tfl_model.create_tflite_model(
+ "conv2d_single", ifm_shape, kernel_shape, strides, padding, dilation, activation
)
+ relay_mod, relay_params = tfl_model.convert_to_relay()
- cmsisnn_mod = cmsisnn.partition_for_cmsisnn(relay_mod, params)
+ cmsisnn_mod = cmsisnn.partition_for_cmsisnn(relay_mod, relay_params)
# validate pattern matching
assert_partitioned_function(relay_mod, cmsisnn_mod)
# validate CMSIS-NN output against TFLite output
- input_map, output_map, output_tolerance = generate_ref_data_tflite(tflite_model)
Review Comment:
No, it should be removed in this commit itself. Why I backtracked I can't recall.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] ashutosh-arm commented on pull request #10939: [CMSIS-NN] Moved TFLite model making to common area
Posted by GitBox <gi...@apache.org>.
ashutosh-arm commented on PR #10939:
URL: https://github.com/apache/tvm/pull/10939#issuecomment-1095432489
cc: @manupa-arm @Mousius @grant-arm @lhutton1 for code review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] lhutton1 commented on a diff in pull request #10939: [CMSIS-NN] Moved TFLite model making to common area
Posted by GitBox <gi...@apache.org>.
lhutton1 commented on code in PR #10939:
URL: https://github.com/apache/tvm/pull/10939#discussion_r856022562
##########
tests/python/contrib/test_cmsisnn/test_conv2d.py:
##########
@@ -314,25 +312,29 @@ def test_conv2d_int8_tflite(ifm_shape, kernel_shape, strides, dilation, padding,
interface_api = "c"
use_unpacked_api = True
test_runner = AOT_USMP_CORSTONE300_RUNNER
-
dtype = "int8"
- tflite_model, relay_mod, params = create_conv2d_tflite_relay_models(
- ifm_shape, kernel_shape, strides, dilation, padding, activation, dtype
+
+ from tvm.relay.testing.tflite import TFLiteModel
+
+ tfl_model = TFLiteModel(dtype)
+ tfl_model.create_tflite_model(
+ "conv2d_single", ifm_shape, kernel_shape, strides, padding, dilation, activation
)
+ relay_mod, relay_params = tfl_model.convert_to_relay()
- cmsisnn_mod = cmsisnn.partition_for_cmsisnn(relay_mod, params)
+ cmsisnn_mod = cmsisnn.partition_for_cmsisnn(relay_mod, relay_params)
# validate pattern matching
assert_partitioned_function(relay_mod, cmsisnn_mod)
# validate CMSIS-NN output against TFLite output
- input_map, output_map, output_tolerance = generate_ref_data_tflite(tflite_model)
Review Comment:
Is the plan to remove these functions from utils after the rest of the tests have been converted?
##########
tests/python/contrib/test_cmsisnn/test_conv2d.py:
##########
@@ -314,25 +312,29 @@ def test_conv2d_int8_tflite(ifm_shape, kernel_shape, strides, dilation, padding,
interface_api = "c"
use_unpacked_api = True
test_runner = AOT_USMP_CORSTONE300_RUNNER
-
dtype = "int8"
- tflite_model, relay_mod, params = create_conv2d_tflite_relay_models(
- ifm_shape, kernel_shape, strides, dilation, padding, activation, dtype
+
+ from tvm.relay.testing.tflite import TFLiteModel
+
+ tfl_model = TFLiteModel(dtype)
+ tfl_model.create_tflite_model(
+ "conv2d_single", ifm_shape, kernel_shape, strides, padding, dilation, activation
)
+ relay_mod, relay_params = tfl_model.convert_to_relay()
- cmsisnn_mod = cmsisnn.partition_for_cmsisnn(relay_mod, params)
+ cmsisnn_mod = cmsisnn.partition_for_cmsisnn(relay_mod, relay_params)
# validate pattern matching
assert_partitioned_function(relay_mod, cmsisnn_mod)
# validate CMSIS-NN output against TFLite output
- input_map, output_map, output_tolerance = generate_ref_data_tflite(tflite_model)
Review Comment:
Is the plan to remove these functions after the rest of the tests have been converted?
##########
python/tvm/relay/testing/tflite.py:
##########
@@ -0,0 +1,152 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Common utilities for creating TFLite models"""
+from distutils.version import LooseVersion
+import numpy as np
+import pytest
+import tvm
+
+pytest.importorskip("tflite")
+pytest.importorskip("tensorflow")
+import tflite.Model # pylint: disable=wrong-import-position
+import tensorflow as tf # pylint: disable=wrong-import-position
+
+
+class TFLiteModel:
+ """Creates TFLite Model and facilitates reference data generation"""
+
+ def __init__(self, dtype):
+ self.serial_model = None # This is what TFLite convert() provides
+ self.dtype = dtype # This is the dtype of graph inputs
+ self.shape_dict = {}
+ self.dtype_dict = {}
+
+ @tf.function
+ def conv2d_single_function(self, ifm_tensor, args):
Review Comment:
I feel like creation of this function should be part of the test itself so its clear what the test stimulus is, rather than hidden away in common testing utilities
##########
tests/python/contrib/test_cmsisnn/test_conv2d.py:
##########
@@ -314,25 +312,29 @@ def test_conv2d_int8_tflite(ifm_shape, kernel_shape, strides, dilation, padding,
interface_api = "c"
use_unpacked_api = True
test_runner = AOT_USMP_CORSTONE300_RUNNER
-
dtype = "int8"
- tflite_model, relay_mod, params = create_conv2d_tflite_relay_models(
- ifm_shape, kernel_shape, strides, dilation, padding, activation, dtype
+
+ from tvm.relay.testing.tflite import TFLiteModel
+
+ tfl_model = TFLiteModel(dtype)
+ tfl_model.create_tflite_model(
+ "conv2d_single", ifm_shape, kernel_shape, strides, padding, dilation, activation
)
+ relay_mod, relay_params = tfl_model.convert_to_relay()
- cmsisnn_mod = cmsisnn.partition_for_cmsisnn(relay_mod, params)
+ cmsisnn_mod = cmsisnn.partition_for_cmsisnn(relay_mod, relay_params)
# validate pattern matching
assert_partitioned_function(relay_mod, cmsisnn_mod)
# validate CMSIS-NN output against TFLite output
- input_map, output_map, output_tolerance = generate_ref_data_tflite(tflite_model)
+ input_map, output_map, output_tolerance = tfl_model.generate_reference_data()
compile_and_run(
Review Comment:
Could we wrap this part in a `compare_cmsisnn_with_reference` function similar to `compare_ethosu_with_reference`?
##########
python/tvm/relay/testing/tflite.py:
##########
@@ -0,0 +1,152 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Common utilities for creating TFLite models"""
+from distutils.version import LooseVersion
+import numpy as np
+import pytest
+import tvm
+
+pytest.importorskip("tflite")
+pytest.importorskip("tensorflow")
+import tflite.Model # pylint: disable=wrong-import-position
+import tensorflow as tf # pylint: disable=wrong-import-position
+
+
+class TFLiteModel:
+ """Creates TFLite Model and facilitates reference data generation"""
+
+ def __init__(self, dtype):
+ self.serial_model = None # This is what TFLite convert() provides
+ self.dtype = dtype # This is the dtype of graph inputs
+ self.shape_dict = {}
+ self.dtype_dict = {}
+
+ @tf.function
+ def conv2d_single_function(self, ifm_tensor, args):
Review Comment:
I feel like creation of this function should be part of the test itself so its clear what's being tested, rather than hidden away in common testing utilities
##########
python/tvm/relay/testing/tflite.py:
##########
@@ -0,0 +1,152 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Common utilities for creating TFLite models"""
+from distutils.version import LooseVersion
+import numpy as np
+import pytest
+import tvm
+
+pytest.importorskip("tflite")
+pytest.importorskip("tensorflow")
+import tflite.Model # pylint: disable=wrong-import-position
+import tensorflow as tf # pylint: disable=wrong-import-position
+
+
+class TFLiteModel:
+ """Creates TFLite Model and facilitates reference data generation"""
+
+ def __init__(self, dtype):
+ self.serial_model = None # This is what TFLite convert() provides
+ self.dtype = dtype # This is the dtype of graph inputs
+ self.shape_dict = {}
+ self.dtype_dict = {}
+
+ @tf.function
+ def conv2d_single_function(self, ifm_tensor, args):
+ """Returns TFLite Conv2d layer"""
+ assert len(args) == 6, "Conv2D needs (ifm_shape, kernel_shape, strides, padding, dilation)"
+ _, kernel_shape, strides, padding, dilation, activation = args
+ op = tf.nn.conv2d(
+ ifm_tensor,
+ filters=tf.constant(
+ np.random.uniform(size=[kernel_shape[0], kernel_shape[1], 3, 3]),
+ dtype=tf.float32,
+ ),
+ strides=[1, strides[0], strides[1], 1],
+ padding=padding,
+ dilations=dilation,
+ )
+ if activation == "RELU":
+ op = tf.nn.relu(op)
+ elif activation == "NONE":
+ pass
+ else:
+ assert False, "Unsupported activation {}".format(activation)
+ return op
+
+ def create_tflite_model(self, op_type, *args):
+ """Returns TFLite serial graph, Relay module, Relay params based on op_type"""
+ concrete_func = None
+ input_shape = None
+ if op_type == "conv2d_single":
+ input_shape = args[0]
+ ifm_tensor = tf.TensorSpec(input_shape, dtype=tf.float32, name="input")
+ concrete_func = self.conv2d_single_function.get_concrete_function(ifm_tensor, args)
+ else:
+ assert False, "Unsupported op_type {}".format(op_type)
+
+ def representative_dataset():
+ for _ in range(100):
+ data = np.random.rand(*tuple(input_shape))
Review Comment:
It might be worth including negative values here as well as this exposed a couple of issues in the microNPU integration
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] ashutosh-arm commented on a diff in pull request #10939: [CMSIS-NN] Moved TFLite model making to common area
Posted by GitBox <gi...@apache.org>.
ashutosh-arm commented on code in PR #10939:
URL: https://github.com/apache/tvm/pull/10939#discussion_r858562502
##########
python/tvm/relay/testing/tflite.py:
##########
@@ -0,0 +1,152 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Common utilities for creating TFLite models"""
+from distutils.version import LooseVersion
+import numpy as np
+import pytest
+import tvm
+
+pytest.importorskip("tflite")
+pytest.importorskip("tensorflow")
+import tflite.Model # pylint: disable=wrong-import-position
+import tensorflow as tf # pylint: disable=wrong-import-position
+
+
+class TFLiteModel:
+ """Creates TFLite Model and facilitates reference data generation"""
+
+ def __init__(self, dtype):
+ self.serial_model = None # This is what TFLite convert() provides
+ self.dtype = dtype # This is the dtype of graph inputs
+ self.shape_dict = {}
+ self.dtype_dict = {}
+
+ @tf.function
+ def conv2d_single_function(self, ifm_tensor, args):
Review Comment:
As discussed offline, we will keep single layer creation as part of the infra to avoid duplication of this bit across multiple tests. If necessary, to include complex cases, user can write their own @tf.function and still create a model using this infra.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org