You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2020/06/30 12:27:27 UTC
[GitHub] [arrow] kszucs opened a new pull request #7589: ARROW-9276: [Release] Enforce CUDA device for updating the api documentations
kszucs opened a new pull request #7589:
URL: https://github.com/apache/arrow/pull/7589
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #7589: ARROW-9276: [Dev] Enable ARROW_CUDA when generating API documentations
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #7589:
URL: https://github.com/apache/arrow/pull/7589#issuecomment-655985904
Revision: 0d645d5260adc639a6fa6cc7a0adab879254f1d6
Submitted crossbow builds: [ursa-labs/crossbow @ actions-393](https://github.com/ursa-labs/crossbow/branches/all?query=actions-393)
|Task|Status|
|----|------|
|test-ubuntu-18.04-docs|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-393-azure-test-ubuntu-18.04-docs)|
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a change in pull request #7589: ARROW-9276: [Release] Enforce CUDA device for updating the api documentations
Posted by GitBox <gi...@apache.org>.
pitrou commented on a change in pull request #7589:
URL: https://github.com/apache/arrow/pull/7589#discussion_r447741353
##########
File path: dev/release/post-09-docs.sh
##########
@@ -42,20 +47,20 @@ popd
pushd "${ARROW_DIR}"
git checkout "${release_tag}"
Review comment:
@kszucs Is a CUDA machine needed? Do you want me to try?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on a change in pull request #7589: ARROW-9276: [Release] Enforce CUDA device for updating the api documentations
Posted by GitBox <gi...@apache.org>.
kou commented on a change in pull request #7589:
URL: https://github.com/apache/arrow/pull/7589#discussion_r448188490
##########
File path: dev/release/post-09-docs.sh
##########
@@ -42,20 +47,20 @@ popd
pushd "${ARROW_DIR}"
git checkout "${release_tag}"
Review comment:
FYI: We don't need a CUDA machine if we don't need to run CUDA enabled programs. We're building CUDA enabled deb packages on a no CUDA machine.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on a change in pull request #7589: ARROW-9276: [Release] Enforce CUDA device for updating the api documentations
Posted by GitBox <gi...@apache.org>.
kou commented on a change in pull request #7589:
URL: https://github.com/apache/arrow/pull/7589#discussion_r451154810
##########
File path: dev/release/post-09-docs.sh
##########
@@ -42,20 +47,20 @@ popd
pushd "${ARROW_DIR}"
git checkout "${release_tag}"
Review comment:
Could you try the following change? I confirmed that this can build C++ and GLib with CUDA but Java build is failed... (I think that the Java build error is unrelated to this.)
```diff
diff --git a/ci/docker/linux-apt-docs.dockerfile b/ci/docker/linux-apt-docs.dockerfile
index 55ce2b87c..d0d98d5cd 100644
--- a/ci/docker/linux-apt-docs.dockerfile
+++ b/ci/docker/linux-apt-docs.dockerfile
@@ -43,6 +43,7 @@ RUN apt-get update -y && \
libtool \
libxml2-dev \
ninja-build \
+ nvidia-cuda-toolkit \
openjdk-${jdk}-jdk-headless \
pandoc \
r-base=${r}* \
diff --git a/dev/release/post-09-docs.sh b/dev/release/post-09-docs.sh
index e81147a0e..cf983d066 100755
--- a/dev/release/post-09-docs.sh
+++ b/dev/release/post-09-docs.sh
@@ -50,7 +50,7 @@ git checkout "${release_tag}"
archery docker run \
-v "${ARROW_SITE_DIR}/docs:/build/docs" \
-e ARROW_DOCS_VERSION="${version}" \
- ubuntu-cuda-docs
+ ubuntu-docs
: ${PUSH:=1}
diff --git a/docker-compose.yml b/docker-compose.yml
index f124f28fb..37e0b054d 100644
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -65,7 +65,6 @@ x-ccache: &ccache
x-with-gpus:
- ubuntu-cuda-cpp
- ubuntu-cuda-python
- - ubuntu-cuda-docs
x-hierarchy:
# This section is used by the archery tool to enable building nested images,
@@ -116,8 +115,7 @@ x-hierarchy:
- ubuntu-docs
- ubuntu-r
- ubuntu-cuda-cpp:
- - ubuntu-cuda-python:
- - ubuntu-cuda-docs
+ - ubuntu-cuda-python
- ubuntu-csharp
- ubuntu-cpp-sanitizer
- ubuntu-r-sanitizer
@@ -1125,6 +1123,7 @@ services:
base: ${REPO}:${ARCH}-ubuntu-${UBUNTU}-python-3
environment:
<<: *ccache
+ ARROW_CUDA: "ON"
ARROW_GLIB_GTK_DOC: "true"
volumes: *ubuntu-volumes
command: &docs-command >
@@ -1137,30 +1136,6 @@ services:
/arrow/ci/scripts/r_build.sh /arrow true &&
/arrow/ci/scripts/docs_build.sh /arrow /build"
- ubuntu-cuda-docs:
- # Usage:
- # docker-compose build ubuntu-cuda-cpp
- # docker-compose build ubuntu-cuda-python
- # docker-compose build ubuntu-cuda-docs
- # docker-compose run --rm ubuntu-cuda-docs
- # Notes about --runtime and --gpus and docker-compose incompatibility
- image: ${REPO}:${ARCH}-ubuntu-${UBUNTU}-cuda-${CUDA}-python-3-docs
- build:
- context: .
- dockerfile: ci/docker/linux-apt-docs.dockerfile
- cache_from:
- - ${REPO}:${ARCH}-ubuntu-${UBUNTU}-docs
- args:
- jdk: ${JDK}
- node: ${NODE}
- base: ${REPO}:${ARCH}-ubuntu-${UBUNTU}-cuda-${CUDA}-python-3
- environment:
- <<: *ccache
- ARROW_CUDA: "ON"
- ARROW_GLIB_GTK_DOC: "true"
- volumes: *ubuntu-volumes
- command: *docs-command
-
################################# Tools #####################################
ubuntu-lint:
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on a change in pull request #7589: ARROW-9276: [Release] Enforce CUDA device for updating the api documentations
Posted by GitBox <gi...@apache.org>.
kszucs commented on a change in pull request #7589:
URL: https://github.com/apache/arrow/pull/7589#discussion_r450783001
##########
File path: dev/release/post-09-docs.sh
##########
@@ -42,20 +47,20 @@ popd
pushd "${ARROW_DIR}"
git checkout "${release_tag}"
Review comment:
> FYI: We don't need a CUDA machine if we don't need to run CUDA enabled programs. We're building CUDA enabled deb packages on a no CUDA machine.
Didn't know that, testing it locally and updating the release script.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #7589: ARROW-9276: [Release] Enforce CUDA device for updating the api documentations
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #7589:
URL: https://github.com/apache/arrow/pull/7589#issuecomment-651761179
https://issues.apache.org/jira/browse/ARROW-9276
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on a change in pull request #7589: ARROW-9276: [Release] Enforce CUDA device for updating the api documentations
Posted by GitBox <gi...@apache.org>.
kszucs commented on a change in pull request #7589:
URL: https://github.com/apache/arrow/pull/7589#discussion_r450786728
##########
File path: dev/release/post-09-docs.sh
##########
@@ -42,20 +47,20 @@ popd
pushd "${ARROW_DIR}"
git checkout "${release_tag}"
Review comment:
@kou it doesn't work, without `--with-gpus` I'm getting link error:
```
/usr/bin/ld: cannot find -lCUDA_CUDA_LIBRARY-NOTFOUND
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on pull request #7589: ARROW-9276: [Dev] Enable ARROW_CUDA when generating API documentations
Posted by GitBox <gi...@apache.org>.
kszucs commented on pull request #7589:
URL: https://github.com/apache/arrow/pull/7589#issuecomment-655984986
@github-actions crossbow submit test-ubuntu-18.04-docs
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on a change in pull request #7589: ARROW-9276: [Release] Enforce CUDA device for updating the api documentations
Posted by GitBox <gi...@apache.org>.
kszucs commented on a change in pull request #7589:
URL: https://github.com/apache/arrow/pull/7589#discussion_r451362423
##########
File path: dev/release/post-09-docs.sh
##########
@@ -42,20 +47,20 @@ popd
pushd "${ARROW_DIR}"
git checkout "${release_tag}"
Review comment:
So `nvidia-cuda-toolkit` does the trick! I'm updating the PR, thanks Kou!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on a change in pull request #7589: ARROW-9276: [Dev] Enable ARROW_CUDA when generating API documentations
Posted by GitBox <gi...@apache.org>.
kou commented on a change in pull request #7589:
URL: https://github.com/apache/arrow/pull/7589#discussion_r452480967
##########
File path: dev/release/post-09-docs.sh
##########
@@ -42,20 +42,20 @@ popd
pushd "${ARROW_DIR}"
git checkout "${release_tag}"
-docker-compose build ubuntu-cpp
-docker-compose build ubuntu-python
-docker-compose build ubuntu-docs
-docker-compose run --rm \
+archery docker run \
-v "${ARROW_SITE_DIR}/docs:/build/docs" \
-e ARROW_DOCS_VERSION="${version}" \
ubuntu-docs
-popd
-pushd "${ARROW_SITE_DIR}"
-git add docs
-git commit -m "[Website] Update documentations for ${version}"
-git push -u origin ${branch_name}
-popd
+: ${PUSH:=1}
+
+if [ ${PUSH} -gt 0 ]; then
+ pushd "${ARROW_SITE_DIR}"
+ git add docs
+ git commit -m "[Website] Update documentations for ${version}"
+ git push -u origin ${branch_name}
+ popd
+fi
Review comment:
Could you also put the last `echo "Success!" ...` block into this `if ...fi`?
If we don't push, the message is meaningless.
Could define `github_url` that is used at the last line? `github_url` isn't defined in this file. (Or we can replace `${github_url}` with `https://github.com/apache/arrow`.)
(This is not related to this change.)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] wesm commented on a change in pull request #7589: ARROW-9276: [Dev] Enable ARROW_CUDA when generating API documentations
Posted by GitBox <gi...@apache.org>.
wesm commented on a change in pull request #7589:
URL: https://github.com/apache/arrow/pull/7589#discussion_r453241713
##########
File path: dev/release/post-09-docs.sh
##########
@@ -42,20 +42,20 @@ popd
pushd "${ARROW_DIR}"
git checkout "${release_tag}"
-docker-compose build ubuntu-cpp
-docker-compose build ubuntu-python
-docker-compose build ubuntu-docs
-docker-compose run --rm \
+archery docker run \
-v "${ARROW_SITE_DIR}/docs:/build/docs" \
-e ARROW_DOCS_VERSION="${version}" \
ubuntu-docs
-popd
-pushd "${ARROW_SITE_DIR}"
-git add docs
-git commit -m "[Website] Update documentations for ${version}"
-git push -u origin ${branch_name}
-popd
+: ${PUSH:=1}
+
+if [ ${PUSH} -gt 0 ]; then
+ pushd "${ARROW_SITE_DIR}"
+ git add docs
+ git commit -m "[Website] Update documentations for ${version}"
+ git push -u origin ${branch_name}
+ popd
+fi
Review comment:
done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on a change in pull request #7589: ARROW-9276: [Release] Enforce CUDA device for updating the api documentations
Posted by GitBox <gi...@apache.org>.
kszucs commented on a change in pull request #7589:
URL: https://github.com/apache/arrow/pull/7589#discussion_r447643080
##########
File path: dev/release/post-09-docs.sh
##########
@@ -42,20 +47,20 @@ popd
pushd "${ARROW_DIR}"
git checkout "${release_tag}"
Review comment:
@wesm could you try running it after commenting the checkout line above (unless you have the tag locally):
```bash
PUSH=0 dev/release/post-09-docs.sh 0.17.1
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on a change in pull request #7589: ARROW-9276: [Release] Enforce CUDA device for updating the api documentations
Posted by GitBox <gi...@apache.org>.
kszucs commented on a change in pull request #7589:
URL: https://github.com/apache/arrow/pull/7589#discussion_r450777633
##########
File path: dev/release/post-09-docs.sh
##########
@@ -42,20 +47,20 @@ popd
pushd "${ARROW_DIR}"
git checkout "${release_tag}"
Review comment:
> $ LANG=C PUSH=0 dev/release/post-09-docs.sh 0.17.1
> dev/release/post-09-docs.sh: line 41: pushd: /home/antoine/arrow/dev/dev/release/../../../arrow-site: No such file or directory
You need to have arrow-site cloned next to arrow - it is documented in the release management guide.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] wesm closed pull request #7589: ARROW-9276: [Dev] Enable ARROW_CUDA when generating API documentations
Posted by GitBox <gi...@apache.org>.
wesm closed pull request #7589:
URL: https://github.com/apache/arrow/pull/7589
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org