You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2022/04/27 08:32:22 UTC
[GitHub] [tvm] yangulei opened a new pull request, #11140: [CI] Enable CI to get and build the latest oneDNN release
yangulei opened a new pull request, #11140:
URL: https://github.com/apache/tvm/pull/11140
I submitted the PR to enable bfloat16 in DNNL BYOC(https://github.com/apache/tvm/pull/11111). But the PR failed to be built due to the version of the oneDNN in CI is limited to v2.2, which is a relative old one. So I update the CI script to get and build the latest oneDNN release.
I had tested the script on my own PC and it works fine, but I don't know how to test the script in CI environment. So let me know if I did something inappropriate, thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] yangulei commented on pull request #11140: [CI] Enable CI to get and build the latest oneDNN release
Posted by GitBox <gi...@apache.org>.
yangulei commented on PR #11140:
URL: https://github.com/apache/tvm/pull/11140#issuecomment-1111636166
> CI tests your changes automatically so no worries there, I would prefer that we just upgrade the oneDNN version manually (and keep the version pinned) though instead of grabbing the latest (imagine that there is a breaking update, the Docker image build would just stop working one day)
Thanks for your comments and suggestions. I had modified the script to pin oneDNN to the latest release v2.6, and it's easy to change the pinned version or set to build the latest release.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] areusch commented on pull request #11140: [CI] Enable CI to get and build the latest oneDNN release
Posted by GitBox <gi...@apache.org>.
areusch commented on PR #11140:
URL: https://github.com/apache/tvm/pull/11140#issuecomment-1124197882
@yangulei i think this means the CI's quota to PyPI was exceeded. perhaps retry now that it's been a while?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] yangulei commented on pull request #11140: [CI] update oneDNN to v2.6
Posted by GitBox <gi...@apache.org>.
yangulei commented on PR #11140:
URL: https://github.com/apache/tvm/pull/11140#issuecomment-1125592600
@areusch @driazati
PyPi works fine, thanks!
Looks like we should install the package to /usr/lib instead of /usr/local in the latest main branch. I have modified my script (follow the installation of gtest) and it pass all the CI now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] areusch merged pull request #11140: [CI] update oneDNN to v2.6
Posted by GitBox <gi...@apache.org>.
areusch merged PR #11140:
URL: https://github.com/apache/tvm/pull/11140
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] driazati commented on a diff in pull request #11140: [CI] Enable CI to get and build the latest oneDNN release
Posted by GitBox <gi...@apache.org>.
driazati commented on code in PR #11140:
URL: https://github.com/apache/tvm/pull/11140#discussion_r860120569
##########
docker/install/ubuntu_install_dnnl.sh:
##########
@@ -20,10 +20,38 @@ set -e
set -u
set -o pipefail
-cd /usr/local/
-wget -q https://github.com/oneapi-src/oneDNN/releases/download/v2.2/dnnl_lnx_2.2.0_cpu_gomp.tgz
-tar -xzf dnnl_lnx_2.2.0_cpu_gomp.tgz
-mv dnnl_lnx_2.2.0_cpu_gomp/include/* /usr/local/include/
-mv dnnl_lnx_2.2.0_cpu_gomp/lib/libdnnl* /usr/local/lib/
-rm -rf dnnl_lnx_2.2.0_cpu_gomp.tgz dnnl_lnx_2.2.0_cpu_gomp
-cd -
+pre_dir=`pwd`
+
+build_dir="/usr/local/"
+install_dir="/usr/local/"
+
+cd ${build_dir}
+rls_tag=$(curl -s https://github.com/oneapi-src/oneDNN/releases/latest \
+ | cut -d '"' -f 2 \
+ | grep -o '[^\/]*$')
+dnnl_ver=`echo ${rls_tag} | sed 's/v//g'`
+echo "The latest oneDNN release is version ${dnnl_ver} with tag '${rls_tag}'"
+
+tar_file="${rls_tag}.tar.gz"
+src_dir="${build_dir}/oneDNN-${dnnl_ver}"
+
+if [ -d ${src_dir} ]; then
+ echo "source files exist."
+else
+ if [ -f ${tar_file} ]; then
+ echo "${tar_file} exists, skip downloading."
+ else
+ echo "downloading ${tar_file}."
+ tar_url="https://github.com/oneapi-src/oneDNN/archive/refs/tags/${tar_file}"
+ wget ${tar_url}
+ fi
+ tar -xzvf ${tar_file}
+fi
+
+cd ${src_dir}
+cmake . -DCMAKE_INSTALL_PREFIX=${install_dir} && make -j16 && make install
Review Comment:
nit: use all available CPUs
```suggestion
NPROC=$(nproc)
cmake . -GNinja -DCMAKE_INSTALL_PREFIX=${install_dir}
make -j"$NPROC"
make install
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] yangulei commented on pull request #11140: [CI] Enable CI to get and build the latest oneDNN release
Posted by GitBox <gi...@apache.org>.
yangulei commented on PR #11140:
URL: https://github.com/apache/tvm/pull/11140#issuecomment-1115510883
The CI failed due to connection error during the installation of tensorflow-aarch64 v2.6.2:
> [2022-04-28T01:43:42.770Z]` WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0xffff827a3050>, 'Connection to snapshots.linaro.org timed out. (connect timeout=15)')': /ldcg/python-cache/tensorflow-aarch64/
> [2022-04-28T01:43:55.126Z] ERROR: Could not find a version that satisfies the requirement tensorflow-aarch64==2.6.2 (from versions: 1.2, 2.7.0rc0, 2.7.0rc1, 2.7.0, 2.7.1, 2.8.0rc0, 2.8.0rc1, 2.8.0, 2.9.0rc0, 2.9.0rc1)
> [2022-04-28T01:43:55.126Z] ERROR: No matching distribution found for tensorflow-aarch64==2.6.2
> [2022-04-28T01:44:00.406Z] WARNING: You are using pip version 19.3.1; however, version 22.0.4 is available.
> [2022-04-28T01:44:00.406Z] You should consider upgrading via the 'pip install --upgrade pip' command.
> [2022-04-28T01:44:00.406Z] The command '/bin/sh -c bash /install/ubuntu_install_tensorflow_aarch64.sh' returned a non-zero code: 1
> [2022-04-28T01:44:00.406Z] ERROR: docker build failed.
> script returned exit code 1
Should I do something or just ignore this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] yangulei commented on pull request #11140: [CI] update oneDNN to v2.6
Posted by GitBox <gi...@apache.org>.
yangulei commented on PR #11140:
URL: https://github.com/apache/tvm/pull/11140#issuecomment-1128595121
Hi @areusch @driazati,
PR #11111 depends on this PR and had waited for 3 weeks, so I want this PR to be merged if it's ready.
Could you please take a look at this script and approve it if no more modifications are required?
Thanks a lot.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] yangulei commented on pull request #11140: [CI] update oneDNN to v2.6
Posted by GitBox <gi...@apache.org>.
yangulei commented on PR #11140:
URL: https://github.com/apache/tvm/pull/11140#issuecomment-1131106519
Thank you @areusch, I will rebase #11111 once the docker image is ready.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] driazati commented on pull request #11140: [CI] Enable CI to get and build the latest oneDNN release
Posted by GitBox <gi...@apache.org>.
driazati commented on PR #11140:
URL: https://github.com/apache/tvm/pull/11140#issuecomment-1124199573
Yeah that index was down for a while last week (https://github.com/apache/tvm/pull/11156#issuecomment-1112015086) but it should be working now. If this happens again we might want to look into rehosting some of those assets
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] yangulei commented on pull request #11140: [CI] update oneDNN to v2.6
Posted by GitBox <gi...@apache.org>.
yangulei commented on PR #11140:
URL: https://github.com/apache/tvm/pull/11140#issuecomment-1128595120
Hi @areusch @driazati,
PR #11111 depends on this PR and had waited for 3 weeks, so I want this PR to be merged if it's ready.
Could you please take a look at this script and approve it if no more modifications are required?
Thanks a lot.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on pull request #11140: [CI] update oneDNN to v2.6
Posted by GitBox <gi...@apache.org>.
masahi commented on PR #11140:
URL: https://github.com/apache/tvm/pull/11140#issuecomment-1130561215
@areusch We still need to go through the actual update process for the cpu image right? I've got another request for gpu update in https://github.com/apache/tvm/pull/11346, I can do the cpu update as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org