You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2020/08/12 21:56:24 UTC
[GitHub] [arrow] kou commented on a change in pull request #7938: ARROW-9701: [CI][Java] Add a job for s390x Java on TravisCI
kou commented on a change in pull request #7938:
URL: https://github.com/apache/arrow/pull/7938#discussion_r469565710
##########
File path: .travis.yml
##########
@@ -81,24 +137,6 @@ before_install:
install:
- pip3 install -e dev/archery[docker]
-script:
Review comment:
How about using a common `script` for all jobs like the following?
```diff
diff --git a/.travis.yml b/.travis.yml
index fa5d84c82..551d0a029 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -47,15 +47,23 @@ jobs:
env:
ARCH: s390x
ARROW_CI_MODULES: "CPP"
- ARROW_FLIGHT: "ON"
- ARROW_PARQUET: "OFF"
DOCKER_IMAGE_ID: ubuntu-cpp
- PARQUET_BUILD_EXAMPLES: "OFF"
- PARQUET_BUILD_EXECUTABLES: "OFF"
- Protobuf_SOURCE: "BUNDLED"
+ DOCKER_RUN_ARGS: >-
+ -e ARROW_FLIGHT=ON
+ -e ARROW_PARQUET=OFF
+ -e PARQUET_BUILD_EXAMPLES=OFF
+ -e PARQUET_BUILD_EXECUTABLES=OFF
+ -e Protobuf_SOURCE=BUNDLED
+ -e cares_SOURCE=BUNDLED
+ -e gRPC_SOURCE=BUNDLED
UBUNTU: "20.04"
- cares_SOURCE: "BUNDLED"
- gRPC_SOURCE: "BUNDLED"
+ - name: "Java on s390x"
+ os: linux
+ arch: s390x
+ env:
+ ARCH: s390x
+ ARROW_CI_MODULES: "JAVA"
+ DOCKER_IMAGE_ID: debian-java
allow_failures:
- arch: s390x
@@ -89,13 +97,7 @@ script:
ulimit -c unlimited || :
- |
archery docker run \
- -e ARROW_FLIGHT=${ARROW_FLIGHT:-OFF} \
- -e ARROW_PARQUET=${ARROW_PARQUET:-ON} \
- -e PARQUET_BUILD_EXAMPLES=${PARQUET_BUILD_EXAMPLES:-ON} \
- -e PARQUET_BUILD_EXECUTABLES=${PARQUET_BUILD_EXECUTABLES:-ON} \
- -e Protobuf_SOURCE=${Protobuf_SOURCE:-} \
- -e cares_SOURCE=${cares_SOURCE:-} \
- -e gRPC_SOURCE=${gRPC_SOURCE:-} \
+ ${DOCKER_RUN_ARGS} \
--volume ${PWD}/build:/build \
${DOCKER_IMAGE_ID}
```
##########
File path: .travis.yml
##########
@@ -56,6 +74,44 @@ jobs:
UBUNTU: "20.04"
cares_SOURCE: "BUNDLED"
gRPC_SOURCE: "BUNDLED"
+ script:
+ - sudo sysctl -w kernel.core_pattern="core.%e.%p"
+ # This isn't allowed on Travis CI:
+ # /home/travis/.travis/functions: line 109: ulimit: core file size: cannot modify limit: Operation not permitted
+ - |
+ ulimit -c unlimited || :
+ - |
+ archery docker run \
+ -e ARROW_FLIGHT=${ARROW_FLIGHT:-OFF} \
+ -e ARROW_PARQUET=${ARROW_PARQUET:-ON} \
+ -e PARQUET_BUILD_EXAMPLES=${PARQUET_BUILD_EXAMPLES:-ON} \
+ -e PARQUET_BUILD_EXECUTABLES=${PARQUET_BUILD_EXECUTABLES:-ON} \
+ -e Protobuf_SOURCE=${Protobuf_SOURCE:-} \
+ -e cares_SOURCE=${cares_SOURCE:-} \
+ -e gRPC_SOURCE=${gRPC_SOURCE:-} \
+ --volume ${PWD}/build:/build \
+ ${DOCKER_IMAGE_ID}
+
+ - name: "Java on s390x"
+ os: linux
+ arch: s390x
+ env:
+ ARCH: s390x
+ ARROW_CI_MODULES: "JAVA"
+ DOCKER_IMAGE_ID: debian-java
+ UBUNTU: "20.04"
Review comment:
This is needless because we don't use Ubuntu based docker image.
##########
File path: ci/scripts/java_build.sh
##########
@@ -23,6 +23,61 @@ source_dir=${1}/java
cpp_build_dir=${2}/cpp/${ARROW_BUILD_TYPE:-debug}
with_docs=${3:-false}
+if [[ "$(uname -s)" == "Linux" ]] && [[ "$(uname -m)" == "s390x" ]]; then
+ # Since some files for s390_64 are not available at maven central,
+ # download pre-build files from bintray and install them explicitly
+ mvn_install="mvn install:install-file"
+ wget="wget"
+ bintray_base_url="https://dl.bintray.com/apache/arrow"
+
+ bintray_dir="flatc-binary"
+ grp="com.github.icexelloss"
+ artifact="flatc-linux-s390_64"
+ ver="1.9.0"
+ extension="exe"
+ target=${artifact}-${ver}.${extension}
+ ${wget} ${bintray_base_url}/${bintray_dir}/${ver}/${target}
+ ${mvn_install} -DgroupId=${grp} -DartifactId=${artifact} -Dversion=${ver} -Dpackaging=${extension} -Dfile=`pwd`/${target}
+
+ bintray_dir="protoc-binary"
+ grp="com.google.protobuf"
+ artifact="protoc"
+ ver="3.7.1"
+ cls="linux-s390_64"
Review comment:
```suggestion
classifier="linux-s390_64"
```
##########
File path: ci/scripts/java_build.sh
##########
@@ -23,6 +23,61 @@ source_dir=${1}/java
cpp_build_dir=${2}/cpp/${ARROW_BUILD_TYPE:-debug}
with_docs=${3:-false}
+if [[ "$(uname -s)" == "Linux" ]] && [[ "$(uname -m)" == "s390x" ]]; then
+ # Since some files for s390_64 are not available at maven central,
+ # download pre-build files from bintray and install them explicitly
+ mvn_install="mvn install:install-file"
+ wget="wget"
+ bintray_base_url="https://dl.bintray.com/apache/arrow"
+
+ bintray_dir="flatc-binary"
+ grp="com.github.icexelloss"
Review comment:
```suggestion
group="com.github.icexelloss"
```
##########
File path: ci/scripts/java_build.sh
##########
@@ -23,6 +23,61 @@ source_dir=${1}/java
cpp_build_dir=${2}/cpp/${ARROW_BUILD_TYPE:-debug}
with_docs=${3:-false}
+if [[ "$(uname -s)" == "Linux" ]] && [[ "$(uname -m)" == "s390x" ]]; then
+ # Since some files for s390_64 are not available at maven central,
+ # download pre-build files from bintray and install them explicitly
+ mvn_install="mvn install:install-file"
+ wget="wget"
+ bintray_base_url="https://dl.bintray.com/apache/arrow"
+
+ bintray_dir="flatc-binary"
+ grp="com.github.icexelloss"
+ artifact="flatc-linux-s390_64"
+ ver="1.9.0"
+ extension="exe"
+ target=${artifact}-${ver}.${extension}
+ ${wget} ${bintray_base_url}/${bintray_dir}/${ver}/${target}
+ ${mvn_install} -DgroupId=${grp} -DartifactId=${artifact} -Dversion=${ver} -Dpackaging=${extension} -Dfile=`pwd`/${target}
Review comment:
Generally, `$(...)` is preferred than `` `...` ``:
```suggestion
${mvn_install} -DgroupId=${grp} -DartifactId=${artifact} -Dversion=${ver} -Dpackaging=${extension} -Dfile=$(pwd)/${target}
```
##########
File path: .travis.yml
##########
@@ -56,6 +74,44 @@ jobs:
UBUNTU: "20.04"
cares_SOURCE: "BUNDLED"
gRPC_SOURCE: "BUNDLED"
+ script:
+ - sudo sysctl -w kernel.core_pattern="core.%e.%p"
+ # This isn't allowed on Travis CI:
+ # /home/travis/.travis/functions: line 109: ulimit: core file size: cannot modify limit: Operation not permitted
+ - |
+ ulimit -c unlimited || :
+ - |
+ archery docker run \
+ -e ARROW_FLIGHT=${ARROW_FLIGHT:-OFF} \
+ -e ARROW_PARQUET=${ARROW_PARQUET:-ON} \
+ -e PARQUET_BUILD_EXAMPLES=${PARQUET_BUILD_EXAMPLES:-ON} \
+ -e PARQUET_BUILD_EXECUTABLES=${PARQUET_BUILD_EXECUTABLES:-ON} \
+ -e Protobuf_SOURCE=${Protobuf_SOURCE:-} \
+ -e cares_SOURCE=${cares_SOURCE:-} \
+ -e gRPC_SOURCE=${gRPC_SOURCE:-} \
+ --volume ${PWD}/build:/build \
+ ${DOCKER_IMAGE_ID}
+
+ - name: "Java on s390x"
+ os: linux
+ arch: s390x
+ env:
+ ARCH: s390x
+ ARROW_CI_MODULES: "JAVA"
+ DOCKER_IMAGE_ID: debian-java
+ UBUNTU: "20.04"
+ script:
+ - sudo sysctl -w kernel.core_pattern="core.%e.%p"
+ # This isn't allowed on Travis CI:
+ # /home/travis/.travis/functions: line 109: ulimit: core file size: cannot modify limit: Operation not permitted
+ - |
+ ulimit -c unlimited || :
+ - |
+ archery docker run \
+ -e ARCH=${ARCH} \
Review comment:
This is needless because `ARCH` is already passed as environment variable.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org