You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mxnet.apache.org by GitBox <gi...@apache.org> on 2020/06/29 19:36:04 UTC
[GitHub] [incubator-mxnet] ChaiBapchya opened a new issue #18637: [Nvidia][CI] Jetson toolchain files public acess
ChaiBapchya opened a new issue #18637:
URL: https://github.com/apache/incubator-mxnet/issues/18637
## Description
Currently, CI for Jetson builds uses aarch64 cross depedencies package based on Jetpack 4.3
However, it needs to be manually downloaded using SDK Manager tool and placed in a private S3 bucket because we don't have license to redistribute them.
Fix would be to make it publicly available under nvidia machine learning apt repository http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64/
For reference
https://github.com/apache/incubator-mxnet/blob/master/ci/docker/Dockerfile.build.jetson#L61-L65
@leezu @ptrendx
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] leezu commented on issue #18637: [Nvidia][CI] Jetson toolchain files public acess
Posted by GitBox <gi...@apache.org>.
leezu commented on issue #18637:
URL: https://github.com/apache/incubator-mxnet/issues/18637#issuecomment-655643843
Thank you @ChaiBapchya!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] ChaiBapchya commented on issue #18637: [Nvidia][CI] Jetson toolchain files public acess
Posted by GitBox <gi...@apache.org>.
ChaiBapchya commented on issue #18637:
URL: https://github.com/apache/incubator-mxnet/issues/18637#issuecomment-651318233
Currently, MXNet devs won't be able to reproduce CI issue for Jetson via
```
ci/build.py --docker-registry mxnetci --platform jetson --docker-build-retries 0 --shm-size 500m /work/runtime_functions.sh build_jetson
```
As it fails for
```
Step 11/17 : RUN aws s3 cp s3://mxnet-ci-prod-private-slave-data/nvidia/sdkm_downloads/cuda-repo-ubuntu1804-10-0-local-10.0.326-410.108_1.0-1_amd64.deb . && dpkg -i cuda-repo-ubuntu1804-10-0-local-10.0.326-410.108_1.0-1_amd64.deb && rm cuda-repo-ubuntu1804-10-0-local-10.0.326-410.108_1.0-1_amd64.deb && apt-key add /var/cuda-repo-10-0-local-10.0.326-410.108/7fa2af80.pub && aws s3 cp s3://mxnet-ci-prod-private-slave-data/nvidia/sdkm_downloads/cuda-repo-cross-aarch64-10-0-local-10.0.326_1.0-1_all.deb . && dpkg -i cuda-repo-cross-aarch64-10-0-local-10.0.326_1.0-1_all.deb && rm cuda-repo-cross-aarch64-10-0-local-10.0.326_1.0-1_all.deb && apt-get update && apt-get install -y -f && apt-get install -y cuda-cross-aarch64 cuda-cross-aarch64-10-0 && rm -rf /var/lib/apt/lists/*
---> Running in 0ee2a3406e2c
fatal error: Unable to locate credentials
```
This needs access to the private mxnet CI prod account bucket which isn't feasible.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] leezu commented on issue #18637: [Nvidia][CI] Jetson toolchain files public acess
Posted by GitBox <gi...@apache.org>.
leezu commented on issue #18637:
URL: https://github.com/apache/incubator-mxnet/issues/18637#issuecomment-651320760
Thanks @ChaiBapchya for opening the tracking issue! Previously I was given an early-June 2020 ETA for public availability of the files. I just followed up with the owner. Ideally the files are available already / will be soon and we just need to update the Dockerfile.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] ChaiBapchya commented on issue #18637: [Nvidia][CI] Jetson toolchain files public acess
Posted by GitBox <gi...@apache.org>.
ChaiBapchya commented on issue #18637:
URL: https://github.com/apache/incubator-mxnet/issues/18637#issuecomment-655643337
Yes working on that fix.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] josefhernandez commented on issue #18637: [Nvidia][CI] Jetson toolchain files public acess
Posted by GitBox <gi...@apache.org>.
josefhernandez commented on issue #18637:
URL: https://github.com/apache/incubator-mxnet/issues/18637#issuecomment-917425575
I'm getting back this error on versions master, 1.8.0 and 1.7.0
Last week everything compiled ok
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For additional commands, e-mail: issues-help@mxnet.apache.org
[GitHub] [incubator-mxnet] leezu closed issue #18637: [Nvidia][CI] Jetson toolchain files public acess
Posted by GitBox <gi...@apache.org>.
leezu closed issue #18637:
URL: https://github.com/apache/incubator-mxnet/issues/18637
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] leezu commented on issue #18637: [Nvidia][CI] Jetson toolchain files public acess
Posted by GitBox <gi...@apache.org>.
leezu commented on issue #18637:
URL: https://github.com/apache/incubator-mxnet/issues/18637#issuecomment-655641151
Nvidia helped to make the files available publicly:
```
https://repo.download.nvidia.com/jetson/x86_64/pool/r32.4/c/cuda/cuda-cross-aarch64-10-2_10.2.89-1_all.deb
https://repo.download.nvidia.com/jetson/x86_64/pool/r32.4/c/cuda/cuda-cross-aarch64_10.2.89-1_all.deb
https://repo.download.nvidia.com/jetson/x86_64/pool/r32.4/c/cuda/cuda-cudart-cross-aarch64-10-2_10.2.89-1_all.deb
https://repo.download.nvidia.com/jetson/x86_64/pool/r32.4/c/cuda/cuda-cufft-cross-aarch64-10-2_10.2.89-1_all.deb
https://repo.download.nvidia.com/jetson/x86_64/pool/r32.4/c/cuda/cuda-cupti-cross-aarch64-10-2_10.2.89-1_all.deb
https://repo.download.nvidia.com/jetson/x86_64/pool/r32.4/c/cuda/cuda-curand-cross-aarch64-10-2_10.2.89-1_all.deb
https://repo.download.nvidia.com/jetson/x86_64/pool/r32.4/c/cuda/cuda-cusolver-cross-aarch64-10-2_10.2.89-1_all.deb
https://repo.download.nvidia.com/jetson/x86_64/pool/r32.4/c/cuda/cuda-cusparse-cross-aarch64-10-2_10.2.89-1_all.deb
https://repo.download.nvidia.com/jetson/x86_64/pool/r32.4/c/cuda/cuda-driver-cross-aarch64-10-2_10.2.89-1_all.deb
https://repo.download.nvidia.com/jetson/x86_64/pool/r32.4/c/cuda/cuda-misc-headers-cross-aarch64-10-2_10.2.89-1_all.deb
https://repo.download.nvidia.com/jetson/x86_64/pool/r32.4/c/cuda/cuda-npp-cross-aarch64-10-2_10.2.89-1_all.deb
https://repo.download.nvidia.com/jetson/x86_64/pool/r32.4/c/cuda/cuda-nsight-compute-addon-l4t-10-2_10.2.89-1_all.deb
https://repo.download.nvidia.com/jetson/x86_64/pool/r32.4/c/cuda/cuda-nvgraph-cross-aarch64-10-2_10.2.89-1_all.deb
https://repo.download.nvidia.com/jetson/x86_64/pool/r32.4/c/cuda/cuda-nvml-cross-aarch64-10-2_10.2.89-1_all.deb
https://repo.download.nvidia.com/jetson/x86_64/pool/r32.4/c/cuda/cuda-nvrtc-cross-aarch64-10-2_10.2.89-1_all.deb
https://repo.download.nvidia.com/jetson/x86_64/pool/r32.4/c/cublas/libcublas-cross-aarch64_10.2.2.89-1_all.deb
https://repo.download.nvidia.com/jetson/x86_64/pool/r32.4/n/nsight-compute/nsight-compute-addon-l4t-2019.5.0_2019.5.0.14-1_all.deb
```
Now the Dockerfile can be updated to make use of the public files and the private S3 bucket can be deleted.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org