You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2021/10/04 12:54:01 UTC
[GitHub] [tvm] Mousius commented on a change in pull request #8922: Arm(R) Cortex(R)-M55 CPU and Arm(R) Ethos(TM)-U55 NPU Demo App
Mousius commented on a change in pull request #8922:
URL: https://github.com/apache/tvm/pull/8922#discussion_r721320883
##########
File path: apps/microtvm/ethosu/Makefile
##########
@@ -0,0 +1,110 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Makefile to build demo
+
+# Setup build environment
+build_dir := build
+STANDALONE_CRT_PATH := $(shell python3 -c "import tvm.micro; print(tvm.micro.get_standalone_crt_dir())")
+
+ARM_CPU=ARMCM55
+ETHOSU_PATH=/opt/arm/ethosu
+DRIVER_PATH=${ETHOSU_PATH}/core_driver
+CMSIS_PATH=${ETHOSU_PATH}/cmsis
+PLATFORM_PATH=${ETHOSU_PATH}/core_platform/targets/corstone-300
+PKG_COMPILE_OPTS = -g -Wall -O2 -Wno-incompatible-pointer-types -Wno-format -mcpu=cortex-m55 -mthumb -mfloat-abi=hard -std=gnu99
+CMAKE = /opt/arm/cmake/bin/cmake
+CC = arm-none-eabi-gcc
+AR = arm-none-eabi-ar
+RANLIB = arm-none-eabi-ranlib
+CC_OPTS = CC=$(CC) AR=$(AR) RANLIB=$(RANLIB)
+PKG_CFLAGS = ${PKG_COMPILE_OPTS} \
+ -I${STANDALONE_CRT_PATH}/include \
+ -I${STANDALONE_CRT_PATH}/src/runtime/crt/include \
+ -Iinclude \
+ -I${PLATFORM_PATH} \
+ -I${ETHOSU_PATH}/core_driver/include \
+ -I${CMSIS_PATH}/Device/ARM/${ARM_CPU}/Include/ \
+ -I${CMSIS_PATH}/CMSIS/Core/Include \
+ -I$(abspath $(build_dir))/codegen/host/include \
+ -DETHOSU_TEST_RUNNER_TOL=${ETHOSU_TEST_RUNNER_TOL}
+DRIVER_CMAKE_FLAGS = -DCMAKE_TOOLCHAIN_FILE=$(abspath $(build_dir))/../arm-none-eabi-gcc.cmake \
+ -DETHOSU_LOG_SEVERITY=debug \
+ -DCMAKE_SYSTEM_PROCESSOR=cortex-m55
+PKG_LDFLAGS = -lm -specs=nosys.specs -static -T corstone300.ld
+
+$(ifeq VERBOSE,1)
+QUIET ?=
+$(else)
+QUIET ?= @
+$(endif)
+
+CODEGEN_SRCS = $(shell find $(abspath $(build_dir))/codegen/host/src/*.c)
+CODEGEN_OBJS = $(subst .c,.o,$(CODEGEN_SRCS))
+CMSIS_STARTUP_SRCS = $(shell find ${CMSIS_PATH}/Device/ARM/${ARM_CPU}/Source/*.c)
Review comment:
```suggestion
CMSIS_STARTUP_SRCS = $(wildcard ${CMSIS_PATH}/Device/ARM/${ARM_CPU}/Source/*.c)
```
##########
File path: apps/microtvm/ethosu/README.md
##########
@@ -0,0 +1,78 @@
+<!--- Licensed to the Apache Software Foundation (ASF) under one -->
+<!--- or more contributor license agreements. See the NOTICE file -->
+<!--- distributed with this work for additional information -->
+<!--- regarding copyright ownership. The ASF licenses this file -->
+<!--- to you under the Apache License, Version 2.0 (the -->
+<!--- "License"); you may not use this file except in compliance -->
+<!--- with the License. You may obtain a copy of the License at -->
+
+<!--- http://www.apache.org/licenses/LICENSE-2.0 -->
+
+<!--- Unless required by applicable law or agreed to in writing, -->
+<!--- software distributed under the License is distributed on an -->
+<!--- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -->
+<!--- KIND, either express or implied. See the License for the -->
+<!--- specific language governing permissions and limitations -->
+<!--- under the License. -->
+
+
+Running TVM on bare metal Arm(R) Cortex(R)-M55 CPU and Ethos(TM)-U55 NPU
+========================================================================
+
+This folder contains an example of how to use TVM to run a model
+on bare metal Cortex(R)-M55 CPU and Ethos(TM)-U55 NPU.
+
+Prerequisites
+-------------
+If the demo is run in the ci_cpu Docker container provided with TVM, then the following
+software will already be installed.
+
+If the demo is not run in the ci_cpu Docker container, then you will need the following:
+- Software required to build the Ethos(TM)-U driver stack and run the demo (These can all be
+ installed by running tvm/docker/install/ubuntu_install_ethosu_driver_stack.sh)
+ - [ Fixed Virtual Platform (FVP) based on Arm(R) Corstone(TM)-300 software](https://developer.arm.com/tools-and-software/open-source-software/arm-platforms-software/arm-ecosystem-fvps)
+ - [cmake 3.19.5](https://github.com/Kitware/CMake/releases/)
+ - [GCC toolchain from Arm(R)](https://developer.arm.com/-/media/Files/downloads/gnu-rm/10-2020q4/gcc-arm-none-eabi-10-2020-q4-major-x86_64-linux.tar.bz2)
+ - [Arm(R) Ethos(TM)-U NPU driver stack](https://review.mlplatform.org)
+ - [CMSIS](https://github.com/ARM-software/CMSIS_5)
+- The python libraries listed in the requirements.txt of this directory
+ - These can be installed by running the following from the current directory:
+ ```bash
+ pip install -r ./requirements.txt
+ ```
+
+You will also need TVM which can either be:
+ - Built from source (see [Install from Source](https://tvm.apache.org/docs/install/from_source.html))
+ - Installed from TLCPACK (see [TLCPack](https://tlcpack.ai/))
+
+
+Running the demo application
+----------------------------
+Type the following command to run the demo application:
+
+```bash
+./run_demo.sh
+```
+
+This will:
+- Download a quantized mobilenet v1 model
+- Use tvmc to compile the model for Cortex(R)-M55 CPU and Ethos(TM)-U55 NPU
+- Download an image of a kitten to run the model on
+- Create a C header file inputs.c containing the image data as a C array
+- Create a C header file outputs.c containing a C array where the output of inference will be stored
+- Build the Ethos(TM)-U55 core driver
+- Build the demo application
+- Run the demo application on a Fixed Virtual Platform (FVP) based on Arm(R) Corstone(TM)-300 software
+- The application will display what the image has been classified as e.g. "The image has been classified as 'tabby'"
+
+Using your own image
+--------------------
+The create_image.py script takes a single argument on the command line which is the path of the
+image to be converted into an array of bytes for consumption by the model.
+
+The demo can be modified to use an image of your choice by changing the following lines in run_demo.sh
+
+```bash
+curl -sSL https://s3.amazonaws.com/model-server/inputs/kitten.jpg > ./kitten.jpg
Review comment:
```suggestion
curl -sS https://s3.amazonaws.com/model-server/inputs/kitten.jpg -o ./kitten.jpg
```
`-L` is unnessary and can introduce security issues by blindly following redirects
`-o` is just the `curl` native way of outputting a file rather than piping
##########
File path: apps/microtvm/ethosu/Makefile
##########
@@ -0,0 +1,110 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Makefile to build demo
+
+# Setup build environment
+build_dir := build
Review comment:
This should be `BUILD_DIR` to match other variables?
##########
File path: apps/microtvm/ethosu/Makefile
##########
@@ -0,0 +1,110 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Makefile to build demo
+
+# Setup build environment
+build_dir := build
+STANDALONE_CRT_PATH := $(shell python3 -c "import tvm.micro; print(tvm.micro.get_standalone_crt_dir())")
+
+ARM_CPU=ARMCM55
+ETHOSU_PATH=/opt/arm/ethosu
+DRIVER_PATH=${ETHOSU_PATH}/core_driver
+CMSIS_PATH=${ETHOSU_PATH}/cmsis
+PLATFORM_PATH=${ETHOSU_PATH}/core_platform/targets/corstone-300
+PKG_COMPILE_OPTS = -g -Wall -O2 -Wno-incompatible-pointer-types -Wno-format -mcpu=cortex-m55 -mthumb -mfloat-abi=hard -std=gnu99
+CMAKE = /opt/arm/cmake/bin/cmake
Review comment:
For this to be portable this should be just `cmake`, this path is specific to the TVM containers.
##########
File path: apps/microtvm/ethosu/Makefile
##########
@@ -0,0 +1,110 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Makefile to build demo
+
+# Setup build environment
+build_dir := build
+STANDALONE_CRT_PATH := $(shell python3 -c "import tvm.micro; print(tvm.micro.get_standalone_crt_dir())")
+
+ARM_CPU=ARMCM55
+ETHOSU_PATH=/opt/arm/ethosu
+DRIVER_PATH=${ETHOSU_PATH}/core_driver
+CMSIS_PATH=${ETHOSU_PATH}/cmsis
+PLATFORM_PATH=${ETHOSU_PATH}/core_platform/targets/corstone-300
+PKG_COMPILE_OPTS = -g -Wall -O2 -Wno-incompatible-pointer-types -Wno-format -mcpu=cortex-m55 -mthumb -mfloat-abi=hard -std=gnu99
+CMAKE = /opt/arm/cmake/bin/cmake
+CC = arm-none-eabi-gcc
+AR = arm-none-eabi-ar
+RANLIB = arm-none-eabi-ranlib
+CC_OPTS = CC=$(CC) AR=$(AR) RANLIB=$(RANLIB)
Review comment:
I don't think we need `CC_OPTS` ?
##########
File path: apps/microtvm/ethosu/Makefile
##########
@@ -0,0 +1,110 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Makefile to build demo
+
+# Setup build environment
+build_dir := build
+STANDALONE_CRT_PATH := $(shell python3 -c "import tvm.micro; print(tvm.micro.get_standalone_crt_dir())")
+
+ARM_CPU=ARMCM55
+ETHOSU_PATH=/opt/arm/ethosu
+DRIVER_PATH=${ETHOSU_PATH}/core_driver
+CMSIS_PATH=${ETHOSU_PATH}/cmsis
+PLATFORM_PATH=${ETHOSU_PATH}/core_platform/targets/corstone-300
+PKG_COMPILE_OPTS = -g -Wall -O2 -Wno-incompatible-pointer-types -Wno-format -mcpu=cortex-m55 -mthumb -mfloat-abi=hard -std=gnu99
+CMAKE = /opt/arm/cmake/bin/cmake
+CC = arm-none-eabi-gcc
+AR = arm-none-eabi-ar
+RANLIB = arm-none-eabi-ranlib
+CC_OPTS = CC=$(CC) AR=$(AR) RANLIB=$(RANLIB)
+PKG_CFLAGS = ${PKG_COMPILE_OPTS} \
+ -I${STANDALONE_CRT_PATH}/include \
+ -I${STANDALONE_CRT_PATH}/src/runtime/crt/include \
+ -Iinclude \
+ -I${PLATFORM_PATH} \
+ -I${ETHOSU_PATH}/core_driver/include \
+ -I${CMSIS_PATH}/Device/ARM/${ARM_CPU}/Include/ \
+ -I${CMSIS_PATH}/CMSIS/Core/Include \
+ -I$(abspath $(build_dir))/codegen/host/include \
Review comment:
Does this work without `abspath` ?
##########
File path: apps/microtvm/ethosu/Makefile
##########
@@ -0,0 +1,110 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Makefile to build demo
+
+# Setup build environment
+build_dir := build
+STANDALONE_CRT_PATH := $(shell python3 -c "import tvm.micro; print(tvm.micro.get_standalone_crt_dir())")
+
+ARM_CPU=ARMCM55
+ETHOSU_PATH=/opt/arm/ethosu
+DRIVER_PATH=${ETHOSU_PATH}/core_driver
+CMSIS_PATH=${ETHOSU_PATH}/cmsis
+PLATFORM_PATH=${ETHOSU_PATH}/core_platform/targets/corstone-300
+PKG_COMPILE_OPTS = -g -Wall -O2 -Wno-incompatible-pointer-types -Wno-format -mcpu=cortex-m55 -mthumb -mfloat-abi=hard -std=gnu99
+CMAKE = /opt/arm/cmake/bin/cmake
+CC = arm-none-eabi-gcc
+AR = arm-none-eabi-ar
+RANLIB = arm-none-eabi-ranlib
+CC_OPTS = CC=$(CC) AR=$(AR) RANLIB=$(RANLIB)
+PKG_CFLAGS = ${PKG_COMPILE_OPTS} \
+ -I${STANDALONE_CRT_PATH}/include \
+ -I${STANDALONE_CRT_PATH}/src/runtime/crt/include \
+ -Iinclude \
+ -I${PLATFORM_PATH} \
+ -I${ETHOSU_PATH}/core_driver/include \
+ -I${CMSIS_PATH}/Device/ARM/${ARM_CPU}/Include/ \
+ -I${CMSIS_PATH}/CMSIS/Core/Include \
+ -I$(abspath $(build_dir))/codegen/host/include \
+ -DETHOSU_TEST_RUNNER_TOL=${ETHOSU_TEST_RUNNER_TOL}
+DRIVER_CMAKE_FLAGS = -DCMAKE_TOOLCHAIN_FILE=$(abspath $(build_dir))/../arm-none-eabi-gcc.cmake \
+ -DETHOSU_LOG_SEVERITY=debug \
+ -DCMAKE_SYSTEM_PROCESSOR=cortex-m55
+PKG_LDFLAGS = -lm -specs=nosys.specs -static -T corstone300.ld
+
+$(ifeq VERBOSE,1)
+QUIET ?=
+$(else)
+QUIET ?= @
+$(endif)
+
+CODEGEN_SRCS = $(shell find $(abspath $(build_dir))/codegen/host/src/*.c)
Review comment:
```suggestion
CODEGEN_SRCS = $(wildcard ${BUILD_DIR}/codegen/host/src/*.c)
```
To minimise reliance on a specific shell util.
##########
File path: apps/microtvm/ethosu/Makefile
##########
@@ -0,0 +1,110 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Makefile to build demo
+
+# Setup build environment
+build_dir := build
+STANDALONE_CRT_PATH := $(shell python3 -c "import tvm.micro; print(tvm.micro.get_standalone_crt_dir())")
+
+ARM_CPU=ARMCM55
+ETHOSU_PATH=/opt/arm/ethosu
Review comment:
```suggestion
ETHOSU_PATH ?= /opt/arm/ethosu
```
So users can specify the Ethos U path when building the demo.
##########
File path: apps/microtvm/ethosu/README.md
##########
@@ -0,0 +1,78 @@
+<!--- Licensed to the Apache Software Foundation (ASF) under one -->
+<!--- or more contributor license agreements. See the NOTICE file -->
+<!--- distributed with this work for additional information -->
+<!--- regarding copyright ownership. The ASF licenses this file -->
+<!--- to you under the Apache License, Version 2.0 (the -->
+<!--- "License"); you may not use this file except in compliance -->
+<!--- with the License. You may obtain a copy of the License at -->
+
+<!--- http://www.apache.org/licenses/LICENSE-2.0 -->
+
+<!--- Unless required by applicable law or agreed to in writing, -->
+<!--- software distributed under the License is distributed on an -->
+<!--- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -->
+<!--- KIND, either express or implied. See the License for the -->
+<!--- specific language governing permissions and limitations -->
+<!--- under the License. -->
+
+
+Running TVM on bare metal Arm(R) Cortex(R)-M55 CPU and Ethos(TM)-U55 NPU
+========================================================================
+
+This folder contains an example of how to use TVM to run a model
+on bare metal Cortex(R)-M55 CPU and Ethos(TM)-U55 NPU.
+
+Prerequisites
+-------------
+If the demo is run in the ci_cpu Docker container provided with TVM, then the following
+software will already be installed.
+
+If the demo is not run in the ci_cpu Docker container, then you will need the following:
+- Software required to build the Ethos(TM)-U driver stack and run the demo (These can all be
+ installed by running tvm/docker/install/ubuntu_install_ethosu_driver_stack.sh)
+ - [ Fixed Virtual Platform (FVP) based on Arm(R) Corstone(TM)-300 software](https://developer.arm.com/tools-and-software/open-source-software/arm-platforms-software/arm-ecosystem-fvps)
Review comment:
```suggestion
- [Fixed Virtual Platform (FVP) based on Arm(R) Corstone(TM)-300 software](https://developer.arm.com/tools-and-software/open-source-software/arm-platforms-software/arm-ecosystem-fvps)
```
##########
File path: apps/microtvm/ethosu/Makefile
##########
@@ -0,0 +1,110 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Makefile to build demo
+
+# Setup build environment
+build_dir := build
+STANDALONE_CRT_PATH := $(shell python3 -c "import tvm.micro; print(tvm.micro.get_standalone_crt_dir())")
+
+ARM_CPU=ARMCM55
+ETHOSU_PATH=/opt/arm/ethosu
+DRIVER_PATH=${ETHOSU_PATH}/core_driver
+CMSIS_PATH=${ETHOSU_PATH}/cmsis
+PLATFORM_PATH=${ETHOSU_PATH}/core_platform/targets/corstone-300
+PKG_COMPILE_OPTS = -g -Wall -O2 -Wno-incompatible-pointer-types -Wno-format -mcpu=cortex-m55 -mthumb -mfloat-abi=hard -std=gnu99
+CMAKE = /opt/arm/cmake/bin/cmake
+CC = arm-none-eabi-gcc
+AR = arm-none-eabi-ar
+RANLIB = arm-none-eabi-ranlib
+CC_OPTS = CC=$(CC) AR=$(AR) RANLIB=$(RANLIB)
+PKG_CFLAGS = ${PKG_COMPILE_OPTS} \
+ -I${STANDALONE_CRT_PATH}/include \
+ -I${STANDALONE_CRT_PATH}/src/runtime/crt/include \
+ -Iinclude \
+ -I${PLATFORM_PATH} \
+ -I${ETHOSU_PATH}/core_driver/include \
+ -I${CMSIS_PATH}/Device/ARM/${ARM_CPU}/Include/ \
+ -I${CMSIS_PATH}/CMSIS/Core/Include \
+ -I$(abspath $(build_dir))/codegen/host/include \
+ -DETHOSU_TEST_RUNNER_TOL=${ETHOSU_TEST_RUNNER_TOL}
+DRIVER_CMAKE_FLAGS = -DCMAKE_TOOLCHAIN_FILE=$(abspath $(build_dir))/../arm-none-eabi-gcc.cmake \
+ -DETHOSU_LOG_SEVERITY=debug \
+ -DCMAKE_SYSTEM_PROCESSOR=cortex-m55
+PKG_LDFLAGS = -lm -specs=nosys.specs -static -T corstone300.ld
+
+$(ifeq VERBOSE,1)
+QUIET ?=
+$(else)
+QUIET ?= @
+$(endif)
+
+CODEGEN_SRCS = $(shell find $(abspath $(build_dir))/codegen/host/src/*.c)
+CODEGEN_OBJS = $(subst .c,.o,$(CODEGEN_SRCS))
+CMSIS_STARTUP_SRCS = $(shell find ${CMSIS_PATH}/Device/ARM/${ARM_CPU}/Source/*.c)
+UART_SRCS = $(shell find ${PLATFORM_PATH}/*.c)
Review comment:
```suggestion
UART_SRCS = $(wildcard ${PLATFORM_PATH}/*.c)
```
##########
File path: apps/microtvm/ethosu/README.md
##########
@@ -0,0 +1,78 @@
+<!--- Licensed to the Apache Software Foundation (ASF) under one -->
+<!--- or more contributor license agreements. See the NOTICE file -->
+<!--- distributed with this work for additional information -->
+<!--- regarding copyright ownership. The ASF licenses this file -->
+<!--- to you under the Apache License, Version 2.0 (the -->
+<!--- "License"); you may not use this file except in compliance -->
+<!--- with the License. You may obtain a copy of the License at -->
+
+<!--- http://www.apache.org/licenses/LICENSE-2.0 -->
+
+<!--- Unless required by applicable law or agreed to in writing, -->
+<!--- software distributed under the License is distributed on an -->
+<!--- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -->
+<!--- KIND, either express or implied. See the License for the -->
+<!--- specific language governing permissions and limitations -->
+<!--- under the License. -->
+
+
+Running TVM on bare metal Arm(R) Cortex(R)-M55 CPU and Ethos(TM)-U55 NPU
+========================================================================
+
+This folder contains an example of how to use TVM to run a model
+on bare metal Cortex(R)-M55 CPU and Ethos(TM)-U55 NPU.
+
+Prerequisites
+-------------
+If the demo is run in the ci_cpu Docker container provided with TVM, then the following
+software will already be installed.
+
+If the demo is not run in the ci_cpu Docker container, then you will need the following:
+- Software required to build the Ethos(TM)-U driver stack and run the demo (These can all be
+ installed by running tvm/docker/install/ubuntu_install_ethosu_driver_stack.sh)
+ - [ Fixed Virtual Platform (FVP) based on Arm(R) Corstone(TM)-300 software](https://developer.arm.com/tools-and-software/open-source-software/arm-platforms-software/arm-ecosystem-fvps)
+ - [cmake 3.19.5](https://github.com/Kitware/CMake/releases/)
+ - [GCC toolchain from Arm(R)](https://developer.arm.com/-/media/Files/downloads/gnu-rm/10-2020q4/gcc-arm-none-eabi-10-2020-q4-major-x86_64-linux.tar.bz2)
+ - [Arm(R) Ethos(TM)-U NPU driver stack](https://review.mlplatform.org)
+ - [CMSIS](https://github.com/ARM-software/CMSIS_5)
+- The python libraries listed in the requirements.txt of this directory
+ - These can be installed by running the following from the current directory:
+ ```bash
+ pip install -r ./requirements.txt
+ ```
+
+You will also need TVM which can either be:
+ - Built from source (see [Install from Source](https://tvm.apache.org/docs/install/from_source.html))
+ - Installed from TLCPACK (see [TLCPack](https://tlcpack.ai/))
+
+
+Running the demo application
+----------------------------
+Type the following command to run the demo application:
+
+```bash
+./run_demo.sh
+```
+
+This will:
+- Download a quantized mobilenet v1 model
+- Use tvmc to compile the model for Cortex(R)-M55 CPU and Ethos(TM)-U55 NPU
+- Download an image of a kitten to run the model on
+- Create a C header file inputs.c containing the image data as a C array
+- Create a C header file outputs.c containing a C array where the output of inference will be stored
+- Build the Ethos(TM)-U55 core driver
+- Build the demo application
+- Run the demo application on a Fixed Virtual Platform (FVP) based on Arm(R) Corstone(TM)-300 software
+- The application will display what the image has been classified as e.g. "The image has been classified as 'tabby'"
+
+Using your own image
+--------------------
+The create_image.py script takes a single argument on the command line which is the path of the
+image to be converted into an array of bytes for consumption by the model.
+
+The demo can be modified to use an image of your choice by changing the following lines in run_demo.sh
+
+```bash
+curl -sSL https://s3.amazonaws.com/model-server/inputs/kitten.jpg > ./kitten.jpg
+python3 ./convert_image.py ./build/kitten.jpg
Review comment:
```suggestion
python3 ./convert_image.py ./kitten.jpg
```
To match above download.
##########
File path: apps/microtvm/ethosu/README.md
##########
@@ -0,0 +1,78 @@
+<!--- Licensed to the Apache Software Foundation (ASF) under one -->
+<!--- or more contributor license agreements. See the NOTICE file -->
+<!--- distributed with this work for additional information -->
+<!--- regarding copyright ownership. The ASF licenses this file -->
+<!--- to you under the Apache License, Version 2.0 (the -->
+<!--- "License"); you may not use this file except in compliance -->
+<!--- with the License. You may obtain a copy of the License at -->
+
+<!--- http://www.apache.org/licenses/LICENSE-2.0 -->
+
+<!--- Unless required by applicable law or agreed to in writing, -->
+<!--- software distributed under the License is distributed on an -->
+<!--- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -->
+<!--- KIND, either express or implied. See the License for the -->
+<!--- specific language governing permissions and limitations -->
+<!--- under the License. -->
+
+
+Running TVM on bare metal Arm(R) Cortex(R)-M55 CPU and Ethos(TM)-U55 NPU
+========================================================================
+
+This folder contains an example of how to use TVM to run a model
+on bare metal Cortex(R)-M55 CPU and Ethos(TM)-U55 NPU.
+
+Prerequisites
+-------------
+If the demo is run in the ci_cpu Docker container provided with TVM, then the following
+software will already be installed.
+
+If the demo is not run in the ci_cpu Docker container, then you will need the following:
+- Software required to build the Ethos(TM)-U driver stack and run the demo (These can all be
+ installed by running tvm/docker/install/ubuntu_install_ethosu_driver_stack.sh)
+ - [ Fixed Virtual Platform (FVP) based on Arm(R) Corstone(TM)-300 software](https://developer.arm.com/tools-and-software/open-source-software/arm-platforms-software/arm-ecosystem-fvps)
+ - [cmake 3.19.5](https://github.com/Kitware/CMake/releases/)
+ - [GCC toolchain from Arm(R)](https://developer.arm.com/-/media/Files/downloads/gnu-rm/10-2020q4/gcc-arm-none-eabi-10-2020-q4-major-x86_64-linux.tar.bz2)
+ - [Arm(R) Ethos(TM)-U NPU driver stack](https://review.mlplatform.org)
+ - [CMSIS](https://github.com/ARM-software/CMSIS_5)
+- The python libraries listed in the requirements.txt of this directory
+ - These can be installed by running the following from the current directory:
+ ```bash
+ pip install -r ./requirements.txt
+ ```
+
+You will also need TVM which can either be:
+ - Built from source (see [Install from Source](https://tvm.apache.org/docs/install/from_source.html))
+ - Installed from TLCPACK (see [TLCPack](https://tlcpack.ai/))
Review comment:
```suggestion
- Built from source (see [Install from Source](https://tvm.apache.org/docs/install/from_source.html))
- Installed from TLCPack (see [TLCPack](https://tlcpack.ai/))
```
##########
File path: apps/microtvm/ethosu/Makefile
##########
@@ -0,0 +1,110 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Makefile to build demo
+
+# Setup build environment
+build_dir := build
+STANDALONE_CRT_PATH := $(shell python3 -c "import tvm.micro; print(tvm.micro.get_standalone_crt_dir())")
+
+ARM_CPU=ARMCM55
+ETHOSU_PATH=/opt/arm/ethosu
+DRIVER_PATH=${ETHOSU_PATH}/core_driver
+CMSIS_PATH=${ETHOSU_PATH}/cmsis
+PLATFORM_PATH=${ETHOSU_PATH}/core_platform/targets/corstone-300
Review comment:
In fact all of these should be specifiable and not rely on a specific set of directories matching a TVM build container?
##########
File path: apps/microtvm/ethosu/run_demo.sh
##########
@@ -0,0 +1,64 @@
+#!/bin/bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+set -e
+set -u
+set -o pipefail
+
+# Directories
+script_dir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"
+driver_dir="${script_dir}/build/ethosu_core_driver/"
+arm_dir="/opt/arm/"
Review comment:
Should be an argument to specify the directories? See https://github.com/apache/tvm/blob/main/docker/bash.sh#L153-L244
##########
File path: apps/microtvm/ethosu/run_demo.sh
##########
@@ -0,0 +1,64 @@
+#!/bin/bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+set -e
+set -u
+set -o pipefail
+
+# Directories
+script_dir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"
+driver_dir="${script_dir}/build/ethosu_core_driver/"
+arm_dir="/opt/arm/"
+
+# Make build directory
+mkdir -p build
+cd build
+
+# Get mobilenet_v1 tflite model
+mobilenet_url='https://storage.googleapis.com/download.tensorflow.org/models/mobilenet_v1_2018_08_02/mobilenet_v1_1.0_224_quant.tgz'
+curl --retry 64 -sSL ${mobilenet_url} | gunzip | tar -xvf - ./mobilenet_v1_1.0_224_quant.tflite
+
+# Compile model for Arm(R) Cortex(R)-M55 CPU and Ethos(TM)-U55 NPU
+tvmc compile --target="ethos-u -accelerator_config=ethos-u55-256, \
+ c -runtime=c --link-params -mcpu=cortex-m55 --executor=aot --interface-api=c --unpacked-api=1" \
+ --pass-config tir.disable_vectorize=1 ./mobilenet_v1_1.0_224_quant.tflite --output-format=mlf
+tar -xvf module.tar
+
+# Get ImageNet labels
+curl -sSL https://raw.githubusercontent.com/tensorflow/tensorflow/master/tensorflow/lite/java/demo/app/src/main/assets/labels_mobilenet_quant_v1_224.txt \
+ > ./labels_mobilenet_quant_v1_224.txt
+
+# Get input image
+curl -sSL https://s3.amazonaws.com/model-server/inputs/kitten.jpg > ./kitten.jpg
Review comment:
```suggestion
curl -sS https://s3.amazonaws.com/model-server/inputs/kitten.jpg -o ./kitten.jpg
```
##########
File path: apps/microtvm/ethosu/requirements.txt
##########
@@ -0,0 +1,22 @@
+attrs>=21.2.0
Review comment:
Can you export this file with hashes to ensure integrity of the packages?
##########
File path: apps/microtvm/ethosu/include/tvm_runtime.h
##########
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+#include <stdarg.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <tvm/runtime/c_runtime_api.h>
+#include <tvm/runtime/crt/stack_allocator.h>
+
+#define WORKSPACE_SIZE (16384 * 1024)
+__attribute__((section("ethosu_scratch"))) static uint8_t g_aot_memory[WORKSPACE_SIZE];
+
+tvm_workspace_t app_workspace;
+
+void __attribute__((noreturn)) TVMPlatformAbort(tvm_crt_error_t error_code) {
+ printf("TVMPlatformAbort: %d\n", error_code);
+ printf("EXITTHESIM\n");
+ exit(-1);
+}
+
+tvm_crt_error_t TVMPlatformMemoryAllocate(size_t num_bytes, DLDevice dev, void** out_ptr) {
+ return StackMemoryManager_Allocate(&app_workspace, num_bytes, out_ptr);
+}
+
+tvm_crt_error_t TVMPlatformMemoryFree(void* ptr, DLDevice dev) {
+ return StackMemoryManager_Free(&app_workspace, ptr);
+}
+
+void TVMLogf(const char* msg, ...) {
+ va_list args;
+ va_start(args, msg);
+ vfprintf(stdout, msg, args);
+ va_end(args);
+}
+
+TVM_DLL int TVMFuncRegisterGlobal(const char* name, TVMFunctionHandle f, int override) { return 0; }
Review comment:
Please add some header guards - although I think this could also just be a `.c`
##########
File path: apps/microtvm/ethosu/convert_image.py
##########
@@ -0,0 +1,95 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import os
+import pathlib
+import re
+import sys
+from PIL import Image
+import numpy as np
+
+
+def create_header_file(name, section, npy_data, output_path):
+ """
+ This function generates a header file containing the data from the numpy array provided.
+ """
+ file_path = pathlib.Path(f"{output_path}/" + name).resolve()
+
+ # Create header file with npy_data as a C array
+ raw_path = file_path.with_suffix(".h").resolve()
+ with open(raw_path, "w") as header_file:
+ header_file.write("#include <tvmgen_default.h>\n")
+ for tensor_name in npy_data.keys():
+ sanitized_tensor_name = re.sub(r"\W+", "_", tensor_name)
+ header_file.write(
+ f"const size_t {sanitized_tensor_name}_len = {npy_data[tensor_name].size};\n"
+ )
+
+ # Convert numpy data type to C data type
+ if npy_data[tensor_name].dtype == np.uint8:
+ c_type = "uint8_t"
+ elif npy_data[tensor_name].dtype == np.int8:
+ c_type = "int8_t"
+ else:
+ raise RuntimeError(f"Data type {str(npy_data[tensor_name].dtype)} not supported")
+
+ header_file.write(
+ f'{c_type} {sanitized_tensor_name}[] __attribute__((section("{section}"), aligned(16))) = "'
+ )
+
+ data_hexstr = npy_data[tensor_name].tobytes().hex()
+ for i in range(0, len(data_hexstr), 2):
+ header_file.write(f"\\x{data_hexstr[i:i+2]}")
+ header_file.write('";\n\n')
+
+ # Generate code to initialize the struct used by the C API
+ header_file.write(f"struct tvmgen_default_{name} {name} = {{")
Review comment:
We should minimize code generation in the app where possible, we know the inputs to mobilenet ahead of time. It'd be good to only generate out the binary data and everything else be static.
##########
File path: apps/microtvm/ethosu/run_demo.sh
##########
@@ -0,0 +1,64 @@
+#!/bin/bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+set -e
+set -u
+set -o pipefail
+
+# Directories
+script_dir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"
+driver_dir="${script_dir}/build/ethosu_core_driver/"
+arm_dir="/opt/arm/"
+
+# Make build directory
+mkdir -p build
+cd build
+
+# Get mobilenet_v1 tflite model
+mobilenet_url='https://storage.googleapis.com/download.tensorflow.org/models/mobilenet_v1_2018_08_02/mobilenet_v1_1.0_224_quant.tgz'
+curl --retry 64 -sSL ${mobilenet_url} | gunzip | tar -xvf - ./mobilenet_v1_1.0_224_quant.tflite
+
+# Compile model for Arm(R) Cortex(R)-M55 CPU and Ethos(TM)-U55 NPU
+tvmc compile --target="ethos-u -accelerator_config=ethos-u55-256, \
+ c -runtime=c --link-params -mcpu=cortex-m55 --executor=aot --interface-api=c --unpacked-api=1" \
+ --pass-config tir.disable_vectorize=1 ./mobilenet_v1_1.0_224_quant.tflite --output-format=mlf
+tar -xvf module.tar
+
+# Get ImageNet labels
+curl -sSL https://raw.githubusercontent.com/tensorflow/tensorflow/master/tensorflow/lite/java/demo/app/src/main/assets/labels_mobilenet_quant_v1_224.txt \
+ > ./labels_mobilenet_quant_v1_224.txt
Review comment:
```suggestion
curl -sS https://raw.githubusercontent.com/tensorflow/tensorflow/master/tensorflow/lite/java/demo/app/src/main/assets/labels_mobilenet_quant_v1_224.txt \
-o ./labels_mobilenet_quant_v1_224.txt
```
##########
File path: apps/microtvm/ethosu/run_demo.sh
##########
@@ -0,0 +1,64 @@
+#!/bin/bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+set -e
+set -u
+set -o pipefail
+
+# Directories
+script_dir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"
+driver_dir="${script_dir}/build/ethosu_core_driver/"
+arm_dir="/opt/arm/"
+
+# Make build directory
+mkdir -p build
+cd build
+
+# Get mobilenet_v1 tflite model
+mobilenet_url='https://storage.googleapis.com/download.tensorflow.org/models/mobilenet_v1_2018_08_02/mobilenet_v1_1.0_224_quant.tgz'
+curl --retry 64 -sSL ${mobilenet_url} | gunzip | tar -xvf - ./mobilenet_v1_1.0_224_quant.tflite
Review comment:
```suggestion
curl --retry 64 -sS ${mobilenet_url} | gunzip | tar -xvf - ./mobilenet_v1_1.0_224_quant.tflite
```
(Remove `-L` if we can)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org