You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by lu...@apache.org on 2023/03/03 13:20:24 UTC
[tvm] branch arm/rebase created (now fff751d69d)
This is an automated email from the ASF dual-hosted git repository.
lukhut pushed a change to branch arm/rebase
in repository https://gitbox.apache.org/repos/asf/tvm.git
at fff751d69d [Docker] Add script to build llvm from source
This branch includes the following new commits:
new f6ebb84aba [INTERNAL][ETHOSN] CI for downstream build
new 0eabe1fd3b [INTERNAL][ETHOSU] Test script for downstream
new fff751d69d [Docker] Add script to build llvm from source
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
[tvm] 01/03: [INTERNAL][ETHOSN] CI for downstream build
Posted by lu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
lukhut pushed a commit to branch arm/rebase
in repository https://gitbox.apache.org/repos/asf/tvm.git
commit f6ebb84aba33994c6ac9bb38f797d8c3e27c9949
Author: Matthew Barrett <ma...@arm.com>
AuthorDate: Mon Jul 27 15:45:25 2020 +0100
[INTERNAL][ETHOSN] CI for downstream build
This patch includes the necessary changes to docker to
support building a downstream version of the NPU
software stack. This is for internal use only.
Co-authored-by: Leo Blonk <Le...@arm.com>
Co-authored-by: Leandro Nunes <le...@arm.com>
Co-authored-by: Ramana Radhakrishnan <ra...@arm.com>
Co-authored-by: Tristan O'Connor <tr...@arm.com>
Change-Id: I0f123ab2de37a982fb14dabe22b839be89d12015
---
docker/Dockerfile.ci_cpu | 13 ++++++++
.../install/ubuntu_install_arm_ethos_n_driver.sh | 37 ++++++++++++++++++++++
2 files changed, 50 insertions(+)
diff --git a/docker/Dockerfile.ci_cpu b/docker/Dockerfile.ci_cpu
index c75bf31361..46ce34e616 100644
--- a/docker/Dockerfile.ci_cpu
+++ b/docker/Dockerfile.ci_cpu
@@ -141,3 +141,16 @@ ENV PATH /opt/sccache:$PATH
# Libxsmm deps
COPY install/ubuntu_install_libxsmm.sh /install
RUN bash /install/ubuntu_install_libxsmm.sh
+
+# Development Ethos-N driver from internal Artifactory
+# NOTE: ethosn_cap will be removed from future versions of
+# tvm-tools, conditionally copying ethosn_cap to maintain
+# backwards compatibility with different versions of tvm-tools.
+COPY install/ethos*.tar.gz install/ethosn_cap? /install/
+COPY install/gcc-linaro-*x86_64_aarch64-linux-gnu.tar.xz /install/
+COPY install/ubuntu_install_arm_ethos_n_driver.sh /install/ubuntu_install_arm_ethos_n_driver.sh
+RUN bash /install/ubuntu_install_arm_ethos_n_driver.sh
+
+ENV NPU_MODEL_FASTMODEL_LIB=/opt/arm/fast-model/cadi_system_Linux64-Release-GCC-5.4.so
+ENV ARMLMD_LICENSE_FILE=7010@euhpc-lic07.euhpc.arm.com:7010@euhpc-lic03.euhpc.arm.com
+ENV PATH $PATH:/opt/arm/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu/bin
diff --git a/docker/install/ubuntu_install_arm_ethos_n_driver.sh b/docker/install/ubuntu_install_arm_ethos_n_driver.sh
new file mode 100644
index 0000000000..8f4343c88f
--- /dev/null
+++ b/docker/install/ubuntu_install_arm_ethos_n_driver.sh
@@ -0,0 +1,37 @@
+#!/bin/bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+set -e
+set -u
+set -o pipefail
+
+driver_package_file=$(ls -Art /install/ethos*.tar.gz | tail -n 1)
+cross_compiler_package_file=$(ls -Art /install/gcc-linaro-*x86_64_aarch64-linux-gnu.tar.xz)
+install_dir="/opt/arm"
+
+cleanup()
+{
+ rm -rf "$driver_package_file"
+}
+
+trap cleanup 0
+
+mkdir -p "$install_dir"
+tar -xvzf "$driver_package_file" -C "$install_dir"
+tar -xvf "$cross_compiler_package_file" -C "$install_dir"
+chmod -R 555 "$install_dir"
[tvm] 03/03: [Docker] Add script to build llvm from source
Posted by lu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
lukhut pushed a commit to branch arm/rebase
in repository https://gitbox.apache.org/repos/asf/tvm.git
commit fff751d69da5521d1b518b81db7dfa2ada1fc7e9
Author: Neil Hickey <ne...@arm.com>
AuthorDate: Wed Dec 14 09:05:03 2022 +0000
[Docker] Add script to build llvm from source
This allows us to use the latest llvm and make sure it builds with the same configuration as the rest of tvm
Change-Id: I004ec89700498167c632d25f4fdf667922ba4943
---
docker/Dockerfile.ci_arm | 4 +-
docker/install/ubuntu_install_llvm_from_source.sh | 99 +++++++++++++++++++++++
tests/scripts/task_config_build_arm.sh | 2 +-
3 files changed, 102 insertions(+), 3 deletions(-)
diff --git a/docker/Dockerfile.ci_arm b/docker/Dockerfile.ci_arm
index bd2b2d8fb1..238a8c2c1b 100644
--- a/docker/Dockerfile.ci_arm
+++ b/docker/Dockerfile.ci_arm
@@ -50,8 +50,8 @@ COPY install/ubuntu_install_sccache.sh /install/ubuntu_install_sccache.sh
RUN bash /install/ubuntu_install_sccache.sh
ENV PATH /opt/sccache:$PATH
-COPY install/ubuntu_install_llvm.sh /install/ubuntu_install_llvm.sh
-RUN bash /install/ubuntu_install_llvm.sh
+COPY install/ubuntu_install_llvm_from_source.sh /install/ubuntu_install_llvm_from_source.sh
+RUN bash /install/ubuntu_install_llvm_from_source.sh 15.0
ENV TVM_VENV /venv/apache-tvm-py3.7
COPY python/bootstrap/lockfiles /install/python/bootstrap/lockfiles
diff --git a/docker/install/ubuntu_install_llvm_from_source.sh b/docker/install/ubuntu_install_llvm_from_source.sh
new file mode 100644
index 0000000000..e03a2a426a
--- /dev/null
+++ b/docker/install/ubuntu_install_llvm_from_source.sh
@@ -0,0 +1,99 @@
+#!/bin/bash
+
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+# This script builds LLVM and clang from the llvm-project tarball
+# using CMake. It is tested with LLVM from version 15.
+
+set -e
+
+LLVM_VERSION_MAJOR=$1
+
+detect_llvm_version() {
+ curl -sL "https://api.github.com/repos/llvm/llvm-project/releases?per_page=100" | \
+ grep tag_name | \
+ grep -o "llvmorg-${LLVM_VERSION_MAJOR}[^\"]*" | \
+ grep -v rc | \
+ sed -e "s/^llvmorg-//g" | \
+ head -n 1
+}
+
+LLVM_VERSION=$(detect_llvm_version)
+echo ${LLVM_VERSION}
+
+curl -sL \
+ https://github.com/llvm/llvm-project/releases/download/llvmorg-${LLVM_VERSION}/llvm-project-${LLVM_VERSION}.src.tar.xz \
+ -o llvm-project-${LLVM_VERSION}.src.tar.xz
+unxz llvm-project-${LLVM_VERSION}.src.tar.xz
+tar xf llvm-project-${LLVM_VERSION}.src.tar
+pushd llvm-project-${LLVM_VERSION}.src
+
+pushd llvm
+mkdir build
+pushd build
+cmake \
+ -DCMAKE_BUILD_TYPE=Release \
+ -DCMAKE_MODULE_PATH="/llvm-project-${LLVM_VERSION}.src/cmake/Modules" \
+ -DCMAKE_INSTALL_PREFIX=/usr \
+ -DLLVM_TARGETS_TO_BUILD="AArch64;ARM;X86" \
+ -DLLVM_INCLUDE_DOCS=OFF \
+ -DLLVM_INCLUDE_EXAMPLES=OFF \
+ -DLLVM_INCLUDE_TESTS=OFF \
+ -DLLVM_INCLUDE_UTILS=OFF \
+ -DLLVM_ENABLE_TERMINFO=OFF \
+ -DLLVM_ENABLE_ASSERTIONS=ON \
+ -DLLVM_ENABLE_RTTI=ON \
+ -DLLVM_ENABLE_OCAMLDOC=OFF \
+ -DLLVM_USE_INTEL_JITEVENTS=ON \
+ -DLLVM_TEMPORARILY_ALLOW_OLD_TOOLCHAIN=ON \
+ -DPYTHON_EXECUTABLE="$(cpython_path 3.7)/bin/python" \
+ -GNinja \
+ ..
+ninja install
+popd
+popd
+rm -rf llvm-${LLVM_VERSION}.src.tar.xz llvm-${LLVM_VERSION}.src.tar llvm-${LLVM_VERSION}.src
+
+# clang is only used to precompile Gandiva bitcode
+if [ ${LLVM_VERSION_MAJOR} -lt 9 ]; then
+ clang_package_name=cfe
+else
+ clang_package_name=clang
+fi
+
+pushd ${clang_package_name}
+mkdir build
+pushd build
+cmake \
+ -DCMAKE_BUILD_TYPE=Release \
+ -DCMAKE_INSTALL_PREFIX=/usr \
+ -DCMAKE_MODULE_PATH="/llvm-project-${LLVM_VERSION}.src/cmake/Modules" \
+ -DCLANG_INCLUDE_TESTS=OFF \
+ -DCLANG_INCLUDE_DOCS=OFF \
+ -DLLVM_INCLUDE_TESTS=OFF \
+ -DLLVM_INCLUDE_DOCS=OFF \
+ -Wno-dev \
+ -GNinja \
+ ..
+
+ninja -w dupbuild=warn install # both clang and llvm builds generate llvm-config file
+popd
+popd
+
+# out of llvm-project-${LLVM_VERSION}.src
+popd
+rm -rf ${clang_package_name}-${LLVM_VERSION}.src.tar.xz ${clang_package_name}-${LLVM_VERSION}.src.tar ${clang_package_name}-${LLVM_VERSION}.src
diff --git a/tests/scripts/task_config_build_arm.sh b/tests/scripts/task_config_build_arm.sh
index 516e6ac867..50b1a9553f 100755
--- a/tests/scripts/task_config_build_arm.sh
+++ b/tests/scripts/task_config_build_arm.sh
@@ -28,7 +28,7 @@ echo set\(USE_RPC ON\) >> config.cmake
echo set\(USE_MICRO ON\) >> config.cmake
echo set\(USE_MICRO_STANDALONE_RUNTIME ON\) >> config.cmake
echo set\(USE_PROFILER ON\) >> config.cmake
-echo set\(USE_LLVM llvm-config-8\) >> config.cmake
+echo set\(USE_LLVM llvm-config\) >> config.cmake
echo set\(CMAKE_CXX_FLAGS -Werror\) >> config.cmake
echo set\(USE_VTA_FSIM ON\) >> config.cmake
echo set\(USE_ARM_COMPUTE_LIB ON\) >> config.cmake
[tvm] 02/03: [INTERNAL][ETHOSU] Test script for downstream
Posted by lu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
lukhut pushed a commit to branch arm/rebase
in repository https://gitbox.apache.org/repos/asf/tvm.git
commit 0eabe1fd3be692dbece4aa8a1cef8c9a273b8417
Author: Luke Hutton <lu...@arm.com>
AuthorDate: Mon Dec 6 13:28:35 2021 +0000
[INTERNAL][ETHOSU] Test script for downstream
A script to run the Ethos-U tests in isolation.
Change-Id: I86be834f42597ef436ed5b9be769b15242f5ce84
---
tests/scripts/task_python_ethosu_tests.sh | 32 +++++++++++++++++++++++++++++++
1 file changed, 32 insertions(+)
diff --git a/tests/scripts/task_python_ethosu_tests.sh b/tests/scripts/task_python_ethosu_tests.sh
new file mode 100755
index 0000000000..997628b1a6
--- /dev/null
+++ b/tests/scripts/task_python_ethosu_tests.sh
@@ -0,0 +1,32 @@
+#!/bin/bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+set -e
+set -u
+source tests/scripts/setup-pytest-env.sh
+
+
+# Rebuild cython
+
+find . -type f -path "*.pyc" | xargs rm -f
+make cython3
+
+TVM_FFI=ctypes python3 -m pytest tests/python/contrib/test_ethosu
+
+# Run tvmc tests, as they are critical for the Ethos-U enablement
+TVM_FFI=ctypes python3 -m pytest tests/python/driver