You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/05/12 16:05:01 UTC
[GitHub] [arrow] assignUser opened a new pull request, #13131: ARROW-16402: [R][CI] Create new Archery Tasks
assignUser opened a new pull request, #13131:
URL: https://github.com/apache/arrow/pull/13131
This PR introduces to new archery docker task for future use in building the R nightlies in Crossbow.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] amol- closed pull request #13131: ARROW-16402: [R][CI] Create new Archery Tasks
Posted by GitBox <gi...@apache.org>.
amol- closed pull request #13131: ARROW-16402: [R][CI] Create new Archery Tasks
URL: https://github.com/apache/arrow/pull/13131
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on a diff in pull request #13131: ARROW-16402: [R][CI] Create new Archery Tasks
Posted by GitBox <gi...@apache.org>.
kszucs commented on code in PR #13131:
URL: https://github.com/apache/arrow/pull/13131#discussion_r874098108
##########
docker-compose.yml:
##########
@@ -351,7 +353,65 @@ services:
- .:/arrow:delegated
- ${DOCKER_VOLUME_PREFIX}ubuntu-ccache:/ccache:delegated
command: *cpp-command
+
+ ubuntu-cpp-static:
+ # Usage:
+ # docker-compose build ubuntu-cpp-static
+ # docker-compose run --rm ubuntu-cpp-static
+ # Parameters:
+ # ARCH: amd64, arm64v8, s390x, ...
+ # UBUNTU: 18.04, 20.04
+ image: ${REPO}:${ARCH}-ubuntu-${UBUNTU}-cpp-static
+ build:
+ context: .
+ dockerfile: ci/docker/ubuntu-${UBUNTU}-cpp.dockerfile
+ cache_from:
+ - ${REPO}:${ARCH}-ubuntu-${UBUNTU}-cpp-static
+ args:
+ arch: ${ARCH}
+ base: "${ARCH}/ubuntu:${UBUNTU}"
+ clang_tools: ${CLANG_TOOLS}
+ llvm: ${LLVM}
+ gcc_version: ${GCC_VERSION}
+ shm_size: *shm-size
+ cap_add:
+ - SYS_ADMIN
+ devices:
+ - "/dev/fuse:/dev/fuse"
+ security_opt:
+ - apparmor:unconfined
+ ulimits: *ulimits
+ environment:
+ <<: *ccache
+ ARROW_HOME: /arrow
+ ARROW_DEPENDENCY_SOURCE: BUNDLED
+ LIBARROW_MINIMAL: "false"
+ ARROW_MIMALLOC: "ON"
+ volumes: *ubuntu-volumes
+ command: /bin/bash -c "
+ cd /arrow && r/inst/build_arrow_static.sh"
Review Comment:
It could be defined in tasks.yml like:
```yml
test-ubuntu-18.04-r-static: # or something related to R
ci: github
template: docker-tests/github.linux.yml
params:
env:
UBUNTU: 18.04
flags: "-e ARROW_DEPENDENCY_SOURCE=BUNDLED -e ARROW_HOME=/arrow -e LIBARROW_MINIMAL=false"
command: "/arrow/r/inst/build_arrow_static.sh"
image: ubuntu-cpp
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] assignUser commented on a diff in pull request #13131: ARROW-16402: [R][CI] Create new Archery Tasks
Posted by GitBox <gi...@apache.org>.
assignUser commented on code in PR #13131:
URL: https://github.com/apache/arrow/pull/13131#discussion_r871703266
##########
ci/docker/centos-7-cpp.dockerfile:
##########
@@ -0,0 +1,37 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+FROM centos:centos7
+
+RUN yum install -y \
+ diffutils \
+ gcc-c++ \
+ libcurl-devel \
+ make \
+ openssl-devel \
+ wget \
+ which
+
+# yum install cmake version is too old
+
+ARG cmake=3.18.1
Review Comment:
```suggestion
ARG cmake=3.23.1
```
I have made a note to look into the other cmake versions & places but it is out of scope for this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] assignUser commented on a diff in pull request #13131: ARROW-16402: [R][CI] Create new Archery Tasks
Posted by GitBox <gi...@apache.org>.
assignUser commented on code in PR #13131:
URL: https://github.com/apache/arrow/pull/13131#discussion_r874478345
##########
docker-compose.yml:
##########
@@ -351,7 +353,65 @@ services:
- .:/arrow:delegated
- ${DOCKER_VOLUME_PREFIX}ubuntu-ccache:/ccache:delegated
command: *cpp-command
+
+ ubuntu-cpp-static:
+ # Usage:
+ # docker-compose build ubuntu-cpp-static
+ # docker-compose run --rm ubuntu-cpp-static
+ # Parameters:
+ # ARCH: amd64, arm64v8, s390x, ...
+ # UBUNTU: 18.04, 20.04
+ image: ${REPO}:${ARCH}-ubuntu-${UBUNTU}-cpp-static
+ build:
+ context: .
+ dockerfile: ci/docker/ubuntu-${UBUNTU}-cpp.dockerfile
+ cache_from:
+ - ${REPO}:${ARCH}-ubuntu-${UBUNTU}-cpp-static
+ args:
+ arch: ${ARCH}
+ base: "${ARCH}/ubuntu:${UBUNTU}"
+ clang_tools: ${CLANG_TOOLS}
+ llvm: ${LLVM}
+ gcc_version: ${GCC_VERSION}
+ shm_size: *shm-size
+ cap_add:
+ - SYS_ADMIN
+ devices:
+ - "/dev/fuse:/dev/fuse"
+ security_opt:
+ - apparmor:unconfined
+ ulimits: *ulimits
+ environment:
+ <<: *ccache
+ ARROW_HOME: /arrow
+ ARROW_DEPENDENCY_SOURCE: BUNDLED
Review Comment:
I named them like this as they run `r/inst/build_arrow_static.sh` which does set `ARROW_BUILD_STATIC=ON`. But we can rename them. Maybe `*-cpp-r-static` or something like you suggested in another comment?
As this is part of moving over the builds from https://github.com/ursa-labs/arrow-r-nightly I first wanted to move the jobs over as 1:1 as possible and optimize in a follow up.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on a diff in pull request #13131: ARROW-16402: [R][CI] Create new Archery Tasks
Posted by GitBox <gi...@apache.org>.
kszucs commented on code in PR #13131:
URL: https://github.com/apache/arrow/pull/13131#discussion_r874092444
##########
docker-compose.yml:
##########
@@ -351,7 +353,65 @@ services:
- .:/arrow:delegated
- ${DOCKER_VOLUME_PREFIX}ubuntu-ccache:/ccache:delegated
command: *cpp-command
+
+ ubuntu-cpp-static:
+ # Usage:
+ # docker-compose build ubuntu-cpp-static
+ # docker-compose run --rm ubuntu-cpp-static
+ # Parameters:
+ # ARCH: amd64, arm64v8, s390x, ...
+ # UBUNTU: 18.04, 20.04
+ image: ${REPO}:${ARCH}-ubuntu-${UBUNTU}-cpp-static
+ build:
+ context: .
+ dockerfile: ci/docker/ubuntu-${UBUNTU}-cpp.dockerfile
+ cache_from:
+ - ${REPO}:${ARCH}-ubuntu-${UBUNTU}-cpp-static
+ args:
+ arch: ${ARCH}
+ base: "${ARCH}/ubuntu:${UBUNTU}"
+ clang_tools: ${CLANG_TOOLS}
+ llvm: ${LLVM}
+ gcc_version: ${GCC_VERSION}
+ shm_size: *shm-size
+ cap_add:
+ - SYS_ADMIN
+ devices:
+ - "/dev/fuse:/dev/fuse"
+ security_opt:
+ - apparmor:unconfined
+ ulimits: *ulimits
+ environment:
+ <<: *ccache
+ ARROW_HOME: /arrow
+ ARROW_DEPENDENCY_SOURCE: BUNDLED
+ LIBARROW_MINIMAL: "false"
+ ARROW_MIMALLOC: "ON"
+ volumes: *ubuntu-volumes
+ command: /bin/bash -c "
+ cd /arrow && r/inst/build_arrow_static.sh"
Review Comment:
This change seems to introduce new `R` builds, not `C++` builds.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot commented on pull request #13131: ARROW-16402: [R][CI] Create new Archery Tasks
Posted by GitBox <gi...@apache.org>.
ursabot commented on PR #13131:
URL: https://github.com/apache/arrow/pull/13131#issuecomment-1126825039
Benchmark runs are scheduled for baseline = ebfbb08d8e62fc788caab2c3a9958f34cd17a26a and contender = d9346927a0cb79c53464c43c9f991907fca63f10. d9346927a0cb79c53464c43c9f991907fca63f10 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/0fff84fc32b142c093cc237864a8eceb...a1bbafdfd3ec4d149f9f4d1cfebaea06/)
[Failed :arrow_down:0.31% :arrow_up:0.0%] [test-mac-arm](https://conbench.ursa.dev/compare/runs/39e1a6a3a1274ee5b78fced4f9dcd623...d90d26403e1f424f94409593cf2fb235/)
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/fabe8f1e57db4973b01e04bd2c1d58c8...f8db3985ea5d48a68b33ec4e5163a1d8/)
[Finished :arrow_down:0.12% :arrow_up:0.0%] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/b745eeb64966426992d3ba577760bd2f...8b8014c8858e47b4940e533e50d65a40/)
Buildkite builds:
[Finished] [`d9346927` ec2-t3-xlarge-us-east-2](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/754)
[Finished] [`d9346927` test-mac-arm](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/751)
[Finished] [`d9346927` ursa-i9-9960x](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/741)
[Finished] [`d9346927` ursa-thinkcentre-m75q](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/756)
[Finished] [`ebfbb08d` ec2-t3-xlarge-us-east-2](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/753)
[Failed] [`ebfbb08d` test-mac-arm](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/750)
[Finished] [`ebfbb08d` ursa-i9-9960x](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/740)
[Finished] [`ebfbb08d` ursa-thinkcentre-m75q](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/755)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] assignUser commented on a diff in pull request #13131: ARROW-16402: [R][CI] Create new Archery Tasks
Posted by GitBox <gi...@apache.org>.
assignUser commented on code in PR #13131:
URL: https://github.com/apache/arrow/pull/13131#discussion_r871703266
##########
ci/docker/centos-7-cpp.dockerfile:
##########
@@ -0,0 +1,37 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+FROM centos:centos7
+
+RUN yum install -y \
+ diffutils \
+ gcc-c++ \
+ libcurl-devel \
+ make \
+ openssl-devel \
+ wget \
+ which
+
+# yum install cmake version is too old
+
+ARG cmake=3.18.1
Review Comment:
```suggestion
ARG cmake=3.23.1
```
I have made a note to look into the other cmake versions & places but it is out of scope for this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] assignUser commented on a diff in pull request #13131: ARROW-16402: [R][CI] Create new Archery Tasks
Posted by GitBox <gi...@apache.org>.
assignUser commented on code in PR #13131:
URL: https://github.com/apache/arrow/pull/13131#discussion_r874485300
##########
docker-compose.yml:
##########
@@ -351,7 +353,65 @@ services:
- .:/arrow:delegated
- ${DOCKER_VOLUME_PREFIX}ubuntu-ccache:/ccache:delegated
command: *cpp-command
+
+ ubuntu-cpp-static:
+ # Usage:
+ # docker-compose build ubuntu-cpp-static
+ # docker-compose run --rm ubuntu-cpp-static
+ # Parameters:
+ # ARCH: amd64, arm64v8, s390x, ...
+ # UBUNTU: 18.04, 20.04
+ image: ${REPO}:${ARCH}-ubuntu-${UBUNTU}-cpp-static
+ build:
+ context: .
+ dockerfile: ci/docker/ubuntu-${UBUNTU}-cpp.dockerfile
+ cache_from:
+ - ${REPO}:${ARCH}-ubuntu-${UBUNTU}-cpp-static
+ args:
+ arch: ${ARCH}
+ base: "${ARCH}/ubuntu:${UBUNTU}"
+ clang_tools: ${CLANG_TOOLS}
+ llvm: ${LLVM}
+ gcc_version: ${GCC_VERSION}
+ shm_size: *shm-size
+ cap_add:
+ - SYS_ADMIN
+ devices:
+ - "/dev/fuse:/dev/fuse"
+ security_opt:
+ - apparmor:unconfined
+ ulimits: *ulimits
+ environment:
+ <<: *ccache
+ ARROW_HOME: /arrow
+ ARROW_DEPENDENCY_SOURCE: BUNDLED
+ LIBARROW_MINIMAL: "false"
+ ARROW_MIMALLOC: "ON"
+ volumes: *ubuntu-volumes
+ command: /bin/bash -c "
+ cd /arrow && r/inst/build_arrow_static.sh"
Review Comment:
> This change seems to introduce new `R` builds, not `C++` builds.
Well it is a libarrow build for use in the R package, so both in a way?
The task itself needs a bit more involved [template](https://github.com/assignUser/arrow/blob/ebf0e00db702bef2f00621318df92f50cf3711e9/dev/tasks/r/github.nightly.yml#L1) than most jobs (at least currently moving it over 1:1 as mentioned above, we can probably find a way to be more efficent later on)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on a diff in pull request #13131: ARROW-16402: [R][CI] Create new Archery Tasks
Posted by GitBox <gi...@apache.org>.
kszucs commented on code in PR #13131:
URL: https://github.com/apache/arrow/pull/13131#discussion_r874092942
##########
docker-compose.yml:
##########
@@ -351,7 +353,65 @@ services:
- .:/arrow:delegated
- ${DOCKER_VOLUME_PREFIX}ubuntu-ccache:/ccache:delegated
command: *cpp-command
+
+ ubuntu-cpp-static:
+ # Usage:
+ # docker-compose build ubuntu-cpp-static
+ # docker-compose run --rm ubuntu-cpp-static
+ # Parameters:
+ # ARCH: amd64, arm64v8, s390x, ...
+ # UBUNTU: 18.04, 20.04
+ image: ${REPO}:${ARCH}-ubuntu-${UBUNTU}-cpp-static
+ build:
+ context: .
+ dockerfile: ci/docker/ubuntu-${UBUNTU}-cpp.dockerfile
+ cache_from:
+ - ${REPO}:${ARCH}-ubuntu-${UBUNTU}-cpp-static
+ args:
+ arch: ${ARCH}
+ base: "${ARCH}/ubuntu:${UBUNTU}"
+ clang_tools: ${CLANG_TOOLS}
+ llvm: ${LLVM}
+ gcc_version: ${GCC_VERSION}
+ shm_size: *shm-size
+ cap_add:
+ - SYS_ADMIN
+ devices:
+ - "/dev/fuse:/dev/fuse"
+ security_opt:
+ - apparmor:unconfined
+ ulimits: *ulimits
+ environment:
+ <<: *ccache
+ ARROW_HOME: /arrow
+ ARROW_DEPENDENCY_SOURCE: BUNDLED
Review Comment:
We define static builds in `tasks.yml` like the following:
```
test-ubuntu-18.04-cpp-static:
ci: github
template: docker-tests/github.linux.yml
params:
env:
UBUNTU: 18.04
flags: "-e ARROW_BUILD_SHARED=OFF -e ARROW_BUILD_STATIC=ON -e ARROW_TEST_LINKAGE=static"
image: ubuntu-cpp
```
##########
docker-compose.yml:
##########
@@ -351,7 +353,65 @@ services:
- .:/arrow:delegated
- ${DOCKER_VOLUME_PREFIX}ubuntu-ccache:/ccache:delegated
command: *cpp-command
+
+ ubuntu-cpp-static:
+ # Usage:
+ # docker-compose build ubuntu-cpp-static
+ # docker-compose run --rm ubuntu-cpp-static
+ # Parameters:
+ # ARCH: amd64, arm64v8, s390x, ...
+ # UBUNTU: 18.04, 20.04
+ image: ${REPO}:${ARCH}-ubuntu-${UBUNTU}-cpp-static
+ build:
+ context: .
+ dockerfile: ci/docker/ubuntu-${UBUNTU}-cpp.dockerfile
+ cache_from:
+ - ${REPO}:${ARCH}-ubuntu-${UBUNTU}-cpp-static
+ args:
+ arch: ${ARCH}
+ base: "${ARCH}/ubuntu:${UBUNTU}"
+ clang_tools: ${CLANG_TOOLS}
+ llvm: ${LLVM}
+ gcc_version: ${GCC_VERSION}
+ shm_size: *shm-size
+ cap_add:
+ - SYS_ADMIN
+ devices:
+ - "/dev/fuse:/dev/fuse"
+ security_opt:
+ - apparmor:unconfined
+ ulimits: *ulimits
+ environment:
+ <<: *ccache
+ ARROW_HOME: /arrow
+ ARROW_DEPENDENCY_SOURCE: BUNDLED
Review Comment:
We define static builds in `tasks.yml` like the following:
```yaml
test-ubuntu-18.04-cpp-static:
ci: github
template: docker-tests/github.linux.yml
params:
env:
UBUNTU: 18.04
flags: "-e ARROW_BUILD_SHARED=OFF -e ARROW_BUILD_STATIC=ON -e ARROW_TEST_LINKAGE=static"
image: ubuntu-cpp
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on a diff in pull request #13131: ARROW-16402: [R][CI] Create new Archery Tasks
Posted by GitBox <gi...@apache.org>.
kszucs commented on code in PR #13131:
URL: https://github.com/apache/arrow/pull/13131#discussion_r874090728
##########
ci/docker/centos-7-cpp.dockerfile:
##########
@@ -0,0 +1,37 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+FROM centos:centos7
+
+RUN yum install -y \
+ diffutils \
+ gcc-c++ \
+ libcurl-devel \
+ make \
+ openssl-devel \
+ wget \
+ which
+
+# yum install cmake version is too old
+
+ARG cmake=3.23.1
+RUN wget -nv -O - https://github.com/Kitware/CMake/releases/download/v${cmake}/cmake-${cmake}-Linux-x86_64.tar.gz | tar -xzf - -C /opt
+ENV PATH=/opt/cmake-${cmake}-Linux-x86_64/bin:$PATH
Review Comment:
I suggest to set these env variables in a single step, at least that's the convention in the rest of the dockerfiles.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] nealrichardson commented on a diff in pull request #13131: ARROW-16402: [R][CI] Create new Archery Tasks
Posted by GitBox <gi...@apache.org>.
nealrichardson commented on code in PR #13131:
URL: https://github.com/apache/arrow/pull/13131#discussion_r871656786
##########
ci/docker/centos-7-cpp.dockerfile:
##########
@@ -0,0 +1,37 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+FROM centos:centos7
+
+RUN yum install -y \
+ diffutils \
+ gcc-c++ \
+ libcurl-devel \
+ make \
+ openssl-devel \
+ wget \
+ which
+
+# yum install cmake version is too old
+
+ARG cmake=3.18.1
Review Comment:
3.23.1 is newest, no reason to keep it pinned to something old.
I'm actually surprised this isn't defined in .env because this incantation exists in several dockerfiles, should we set the version there?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on a diff in pull request #13131: ARROW-16402: [R][CI] Create new Archery Tasks
Posted by GitBox <gi...@apache.org>.
kszucs commented on code in PR #13131:
URL: https://github.com/apache/arrow/pull/13131#discussion_r874091439
##########
docker-compose.yml:
##########
@@ -351,7 +353,65 @@ services:
- .:/arrow:delegated
- ${DOCKER_VOLUME_PREFIX}ubuntu-ccache:/ccache:delegated
command: *cpp-command
+
+ ubuntu-cpp-static:
Review Comment:
Why are these called `ubuntu|centos-cpp-static`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] nealrichardson commented on a diff in pull request #13131: ARROW-16402: [R][CI] Create new Archery Tasks
Posted by GitBox <gi...@apache.org>.
nealrichardson commented on code in PR #13131:
URL: https://github.com/apache/arrow/pull/13131#discussion_r871689730
##########
ci/docker/centos-7-cpp.dockerfile:
##########
@@ -0,0 +1,37 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+FROM centos:centos7
+
+RUN yum install -y \
+ diffutils \
+ gcc-c++ \
+ libcurl-devel \
+ make \
+ openssl-devel \
+ wget \
+ which
+
+# yum install cmake version is too old
+
+ARG cmake=3.18.1
Review Comment:
You could upgrade and let CI tell you if it's fine ðŸ¤
Searching for `ARG cmake` I find three other instances: One with a comment pointing to an issue that has been fixed, one using the same version as that but with no comment, and another older one with no comment. We could trace those back through but I'd be willing to guess if there's no note explaining why it needs to be pinned on an older version, it's probably because that was the newest version at the time. Specifically for this case, coming from arrow-r-nightly, I can confirm that that's why it is on this version.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] assignUser commented on a diff in pull request #13131: ARROW-16402: [R][CI] Create new Archery Tasks
Posted by GitBox <gi...@apache.org>.
assignUser commented on code in PR #13131:
URL: https://github.com/apache/arrow/pull/13131#discussion_r871680770
##########
ci/docker/centos-7-cpp.dockerfile:
##########
@@ -0,0 +1,37 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+FROM centos:centos7
+
+RUN yum install -y \
+ diffutils \
+ gcc-c++ \
+ libcurl-devel \
+ make \
+ openssl-devel \
+ wget \
+ which
+
+# yum install cmake version is too old
+
+ARG cmake=3.18.1
Review Comment:
I was not sure if there was a reason to pin this version, that's why I did not update it, can do though.
The ubuntu 18.04 actually uses 3.10.2 as that is what apt has.
Yes I noticed that too, there are several different cmake versions pinned all over the project but I am not familiar enough with possible repercussions/advantages of bumping the version up to make a call on that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13131: ARROW-16402: [R][CI] Create new Archery Tasks
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13131:
URL: https://github.com/apache/arrow/pull/13131#issuecomment-1125178491
https://issues.apache.org/jira/browse/ARROW-16402
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on a diff in pull request #13131: ARROW-16402: [R][CI] Create new Archery Tasks
Posted by GitBox <gi...@apache.org>.
kszucs commented on code in PR #13131:
URL: https://github.com/apache/arrow/pull/13131#discussion_r874091653
##########
docker-compose.yml:
##########
@@ -351,7 +353,65 @@ services:
- .:/arrow:delegated
- ${DOCKER_VOLUME_PREFIX}ubuntu-ccache:/ccache:delegated
command: *cpp-command
+
+ ubuntu-cpp-static:
+ # Usage:
+ # docker-compose build ubuntu-cpp-static
+ # docker-compose run --rm ubuntu-cpp-static
+ # Parameters:
+ # ARCH: amd64, arm64v8, s390x, ...
+ # UBUNTU: 18.04, 20.04
+ image: ${REPO}:${ARCH}-ubuntu-${UBUNTU}-cpp-static
+ build:
+ context: .
+ dockerfile: ci/docker/ubuntu-${UBUNTU}-cpp.dockerfile
+ cache_from:
+ - ${REPO}:${ARCH}-ubuntu-${UBUNTU}-cpp-static
+ args:
+ arch: ${ARCH}
+ base: "${ARCH}/ubuntu:${UBUNTU}"
+ clang_tools: ${CLANG_TOOLS}
+ llvm: ${LLVM}
+ gcc_version: ${GCC_VERSION}
+ shm_size: *shm-size
+ cap_add:
+ - SYS_ADMIN
+ devices:
+ - "/dev/fuse:/dev/fuse"
+ security_opt:
+ - apparmor:unconfined
+ ulimits: *ulimits
+ environment:
+ <<: *ccache
+ ARROW_HOME: /arrow
+ ARROW_DEPENDENCY_SOURCE: BUNDLED
Review Comment:
I can't see anything here which indicates a static build.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on a diff in pull request #13131: ARROW-16402: [R][CI] Create new Archery Tasks
Posted by GitBox <gi...@apache.org>.
kszucs commented on code in PR #13131:
URL: https://github.com/apache/arrow/pull/13131#discussion_r874147761
##########
docker-compose.yml:
##########
@@ -351,7 +353,65 @@ services:
- .:/arrow:delegated
- ${DOCKER_VOLUME_PREFIX}ubuntu-ccache:/ccache:delegated
command: *cpp-command
+
+ ubuntu-cpp-static:
+ # Usage:
+ # docker-compose build ubuntu-cpp-static
+ # docker-compose run --rm ubuntu-cpp-static
+ # Parameters:
+ # ARCH: amd64, arm64v8, s390x, ...
+ # UBUNTU: 18.04, 20.04
+ image: ${REPO}:${ARCH}-ubuntu-${UBUNTU}-cpp-static
+ build:
+ context: .
+ dockerfile: ci/docker/ubuntu-${UBUNTU}-cpp.dockerfile
+ cache_from:
+ - ${REPO}:${ARCH}-ubuntu-${UBUNTU}-cpp-static
+ args:
+ arch: ${ARCH}
+ base: "${ARCH}/ubuntu:${UBUNTU}"
+ clang_tools: ${CLANG_TOOLS}
+ llvm: ${LLVM}
+ gcc_version: ${GCC_VERSION}
+ shm_size: *shm-size
+ cap_add:
+ - SYS_ADMIN
+ devices:
+ - "/dev/fuse:/dev/fuse"
+ security_opt:
+ - apparmor:unconfined
+ ulimits: *ulimits
+ environment:
+ <<: *ccache
+ ARROW_HOME: /arrow
+ ARROW_DEPENDENCY_SOURCE: BUNDLED
+ LIBARROW_MINIMAL: "false"
+ ARROW_MIMALLOC: "ON"
+ volumes: *ubuntu-volumes
+ command: /bin/bash -c "
+ cd /arrow && r/inst/build_arrow_static.sh"
+ centos-cpp-static:
Review Comment:
If we would like to keep these builds runnable locally without archery commands then we should add a presets feature to `archery docker run` and keep docker-compose.yml mostly as a template.
Ideally we should move away from docker-compose at some point and either use our own implementation (e.g. [inspiration from ursabot](https://github.com/ursa-labs/ursabot/blob/master/ursabot/docker.py)) or preferably https://earthly.dev with better code reuse.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org