[GitHub] [arrow] kou commented on a change in pull request #8995: ARROW-11015: [CI][Gandiva] Move gandiva nightly build from travis to github action

[GitBox <gi...@apache.org>]

kou commented on a change in pull request #8995:
URL: https://github.com/apache/arrow/pull/8995#discussion_r548216878



##########
File path: dev/tasks/gandiva-jars/github.osx.yml
##########
@@ -0,0 +1,71 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# NOTE: must set "Crossbow" as name to have the badge links working in the
+# github comment reports!
+name: Crossbow
+
+on:
+  push:
+    branches:
+      - "*-github-*"
+
+jobs:
+  package:
+    name: Package Gandiva
+    runs-on: macos-latest
+    steps:
+      - name: Checkout Arrow
+        run: |
+          git clone --no-checkout {{ arrow.remote }} arrow
+          git -C arrow fetch -t {{ arrow.remote }} {{ arrow.branch }}
+          if [ $CROSSBOW_USE_COMMIT_ID = true ]; then git -C arrow checkout {{ arrow.head }}; else git -C arrow checkout FETCH_HEAD; fi
+          git -C arrow submodule update --init --recursive
+      - name: Build Gandiva
+        run: |
+          cd arrow
+          mkdir -p dist
+          export ARROW_TEST_DATA=$GITHUB_WORKSPACE/arrow/testing/data

Review comment:
       
   How about using `$PWD` instead of `$GITHUB_WORKSPACE/arrow`?
   ```suggestion
             export ARROW_TEST_DATA=$PWD/testing/data
   ```

##########
File path: dev/tasks/gandiva-jars/github.linux.yml
##########
@@ -0,0 +1,72 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# NOTE: must set "Crossbow" as name to have the badge links working in the
+# github comment reports!
+name: Crossbow
+
+on:
+  push:
+    branches:
+      - "*-github-*"
+
+jobs:
+  package:
+    name: Package Gandiva
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout Arrow
+        run: |
+          git clone --no-checkout {{ arrow.remote }} arrow
+          git -C arrow fetch -t {{ arrow.remote }} {{ arrow.branch }}
+          if [ $CROSSBOW_USE_COMMIT_ID = true ]; then git -C arrow checkout {{ arrow.head }}; else git -C arrow checkout FETCH_HEAD; fi
+          git -C arrow submodule update --init --recursive
+      - name: Build Gandiva
+        run: |
+          python3 -VV
+          cd arrow
+          mkdir -p dist
+          export CC="gcc-4.9" CXX="g++-4.9"
+          ulimit -c unlimited -S
+          docker run -v $PWD:/io -v $PWD:/arrow quay.io/projjal/arrow:gandivadocker /io/dev/tasks/gandiva-jars/build-cpp-linux.sh || exit 1
+          dev/tasks/gandiva-jars/build-java.sh || exit 1

Review comment:
       How about using `set -e` instead of `|| exit 1`?

##########
File path: dev/tasks/gandiva-jars/github.osx.yml
##########
@@ -0,0 +1,71 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# NOTE: must set "Crossbow" as name to have the badge links working in the
+# github comment reports!
+name: Crossbow
+
+on:
+  push:
+    branches:
+      - "*-github-*"
+
+jobs:
+  package:
+    name: Package Gandiva
+    runs-on: macos-latest
+    steps:
+      - name: Checkout Arrow
+        run: |
+          git clone --no-checkout {{ arrow.remote }} arrow
+          git -C arrow fetch -t {{ arrow.remote }} {{ arrow.branch }}
+          if [ $CROSSBOW_USE_COMMIT_ID = true ]; then git -C arrow checkout {{ arrow.head }}; else git -C arrow checkout FETCH_HEAD; fi
+          git -C arrow submodule update --init --recursive
+      - name: Build Gandiva
+        run: |
+          cd arrow
+          mkdir -p dist
+          export ARROW_TEST_DATA=$GITHUB_WORKSPACE/arrow/testing/data
+          dev/tasks/gandiva-jars/build-cpp-osx.sh || exit 1
+          dev/tasks/gandiva-jars/build-java.sh || exit 1
+        env:
+          OS_NAME: "osx"
+          CHECK_SHARED_DEPENDENCIES: true
+          MACOSX_DEPLOYMENT_TARGET: "10.11"
+      - name: Set up Crossbow
+        run: |
+          brew install libgit2
+          pip3 install \
+            click \
+            github3.py \
+            jinja2 \
+            jira \
+            pygit2 \
+            ruamel.yaml \
+            setuptools_scm \
+            toolz
+      - name: Upload artifacts
+        run: |
+          python3 $GITHUB_WORKSPACE/arrow/dev/tasks/crossbow.py \
+            --queue-path $GITHUB_WORKSPACE/arrow \

Review comment:
       ```suggestion
             python3 arrow/dev/tasks/crossbow.py \
               --queue-path arrow \
   ```

##########
File path: dev/tasks/gandiva-jars/github.linux.yml
##########
@@ -0,0 +1,72 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# NOTE: must set "Crossbow" as name to have the badge links working in the
+# github comment reports!
+name: Crossbow
+
+on:
+  push:
+    branches:
+      - "*-github-*"
+
+jobs:
+  package:
+    name: Package Gandiva
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout Arrow
+        run: |
+          git clone --no-checkout {{ arrow.remote }} arrow
+          git -C arrow fetch -t {{ arrow.remote }} {{ arrow.branch }}
+          if [ $CROSSBOW_USE_COMMIT_ID = true ]; then git -C arrow checkout {{ arrow.head }}; else git -C arrow checkout FETCH_HEAD; fi
+          git -C arrow submodule update --init --recursive
+      - name: Build Gandiva
+        run: |
+          python3 -VV
+          cd arrow
+          mkdir -p dist
+          export CC="gcc-4.9" CXX="g++-4.9"
+          ulimit -c unlimited -S
+          docker run -v $PWD:/io -v $PWD:/arrow quay.io/projjal/arrow:gandivadocker /io/dev/tasks/gandiva-jars/build-cpp-linux.sh || exit 1

Review comment:
       Can we simplify volume usage?
   
   ```suggestion
             docker run -v $PWD:/arrow quay.io/projjal/arrow:gandivadocker /arrow/dev/tasks/gandiva-jars/build-cpp-linux.sh || exit 1
   ```

##########
File path: dev/tasks/gandiva-jars/github.linux.yml
##########
@@ -0,0 +1,72 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# NOTE: must set "Crossbow" as name to have the badge links working in the
+# github comment reports!
+name: Crossbow
+
+on:
+  push:
+    branches:
+      - "*-github-*"
+
+jobs:
+  package:
+    name: Package Gandiva
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout Arrow
+        run: |
+          git clone --no-checkout {{ arrow.remote }} arrow
+          git -C arrow fetch -t {{ arrow.remote }} {{ arrow.branch }}
+          if [ $CROSSBOW_USE_COMMIT_ID = true ]; then git -C arrow checkout {{ arrow.head }}; else git -C arrow checkout FETCH_HEAD; fi
+          git -C arrow submodule update --init --recursive
+      - name: Build Gandiva
+        run: |
+          python3 -VV
+          cd arrow
+          mkdir -p dist
+          export CC="gcc-4.9" CXX="g++-4.9"
+          ulimit -c unlimited -S
+          docker run -v $PWD:/io -v $PWD:/arrow quay.io/projjal/arrow:gandivadocker /io/dev/tasks/gandiva-jars/build-cpp-linux.sh || exit 1
+          dev/tasks/gandiva-jars/build-java.sh || exit 1
+        env:
+          OS_NAME: "linux"
+          CHECK_SHARED_DEPENDENCIES: true
+      - name: Set up Crossbow
+        run: |
+          sudo apt-get install -y libgit2-dev
+          pip3 install \
+            click \
+            github3.py \
+            jinja2 \
+            jira \
+            pygit2==1.0.3 \
+            ruamel.yaml \
+            setuptools_scm \
+            toolz
+      - name: Upload artifacts
+        run: |
+          python3 $GITHUB_WORKSPACE/arrow/dev/tasks/crossbow.py \
+            --queue-path $GITHUB_WORKSPACE/arrow \

Review comment:
       Can we remove `$GITHUB_WORKSPACE/`?
   
   ```suggestion
             python3 arrow/dev/tasks/crossbow.py \
               --queue-path arrow \
   ```

##########
File path: dev/tasks/tasks.yml
##########
@@ -43,7 +43,8 @@ groups:
     - centos-*-aarch64
 
   gandiva:
-    - gandiva-*
+    - gandiva-jar-ubuntu
+    - gandiva-jar-osx

Review comment:
       Could you revert this change?
   `gandiva-*` is expanded to `gandiva-ja-ubuntu` and `gandiva-jar-osx` automatically.
   




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org