You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "sunchao (via GitHub)" <gi...@apache.org> on 2024/02/13 22:03:22 UTC
[PR] build: Add basic CI test pipelines [arrow-datafusion-comet]
sunchao opened a new pull request, #18:
URL: https://github.com/apache/arrow-datafusion-comet/pull/18
This adds some basic CI test pipelines for the project. Basically run tests in the Rust and Java/Scala side within the repo.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] build: Add basic CI test pipelines [arrow-datafusion-comet]
Posted by "sunchao (via GitHub)" <gi...@apache.org>.
sunchao commented on PR #18:
URL: https://github.com/apache/arrow-datafusion-comet/pull/18#issuecomment-1943117428
@viirya @andygrove this is ready now. Tested [here](https://github.com/sunchao/arrow-datafusion-comet/actions/runs/7896769061).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] build: Add basic CI test pipelines [arrow-datafusion-comet]
Posted by "viirya (via GitHub)" <gi...@apache.org>.
viirya commented on code in PR #18:
URL: https://github.com/apache/arrow-datafusion-comet/pull/18#discussion_r1488940242
##########
.github/workflows/pr_build.yml:
##########
@@ -0,0 +1,112 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: PR Build
+
+concurrency:
+ group: ${{ github.repository }}-${{ github.head_ref || github.sha }}-${{ github.workflow }}
+ cancel-in-progress: true
+
+on:
+ push:
+ paths-ignore:
+ - "doc/**"
+ - "**.md"
+ pull_request:
+ paths-ignore:
+ - "doc/**"
+ - "**.md"
+ # manual trigger
+ # https://docs.github.com/en/actions/managing-workflow-runs/manually-running-a-workflow
+ workflow_dispatch:
+
+env:
+ JAVA_VERSION: 17
+
+jobs:
+ linux-rust-test:
+ name: Rust test (amd64)
+ runs-on: ubuntu-latest
+ container:
+ image: amd64/rust
+ steps:
+ - uses: actions/checkout@v4
+ - name: Setup Rust & Java toolchain
+ uses: ./.github/actions/setup-builder
+ with:
+ rust-version: nightly
+ jdk-version: ${{env.JAVA_VERSION}}
+
+ - name: Check cargo fmt
+ run: |
+ cd core
+ cargo fmt --all -- --check --color=never
+
+ - name: Check cargo clippy
+ run: |
+ cd core
+ cargo clippy --color=never -- -D warnings
+
+ - name: Check compilation
+ run: |
+ cd core
+ cargo check --benches
+
+ - name: Setup JAVA_HOME
+ run: |
+ echo "JAVA_HOME=/usr/lib/jvm/java-${{env.JAVA_VERSION}}-openjdk-amd64" >> $GITHUB_ENV
+
+ - name: Build common module (pre-requisite for Rust tests)
+ run: |
+ cd common
+ ../mvnw clean compile -DskipTests
+
+ - name: Run cargo test
+ run: |
+ cd core
+ # This is required to run some JNI related tests on the Rust side
+ LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$JAVA_HOME/lib:$JAVA_HOME/lib/server cargo test
Review Comment:
RUST_BACKTRACE=1?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] build: Add basic CI test pipelines [arrow-datafusion-comet]
Posted by "sunchao (via GitHub)" <gi...@apache.org>.
sunchao commented on PR #18:
URL: https://github.com/apache/arrow-datafusion-comet/pull/18#issuecomment-1942717513
Need to find a way to test this. Also this only run tests in Linux for Spark 3.4 atm, and we need more combinations.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] build: Add basic CI test pipelines [arrow-datafusion-comet]
Posted by "viirya (via GitHub)" <gi...@apache.org>.
viirya commented on PR #18:
URL: https://github.com/apache/arrow-datafusion-comet/pull/18#issuecomment-1943148045
There are a log message like this:
```
Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.0.22/plexus-utils-3.0.22.jar (245 kB at 2.3 MB/s)
Progress (1): 1.0/2.3 MB
Progress (1): 1.1/2.3 MB
Progress (1): 1.1/2.3 MB
...
```
It is really distracting for looking at the logs. Can we silent it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] build: Add basic CI test pipelines [arrow-datafusion-comet]
Posted by "sunchao (via GitHub)" <gi...@apache.org>.
sunchao commented on PR #18:
URL: https://github.com/apache/arrow-datafusion-comet/pull/18#issuecomment-1943152654
> It is really distracting for looking at the logs. Can we silent it?
Hmm perhaps we can cache the Maven dependencies so these will only show up in the first time. Let me check
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] build: Add basic CI test pipelines [arrow-datafusion-comet]
Posted by "sunchao (via GitHub)" <gi...@apache.org>.
sunchao commented on code in PR #18:
URL: https://github.com/apache/arrow-datafusion-comet/pull/18#discussion_r1488959279
##########
.github/workflows/pr_build.yml:
##########
@@ -0,0 +1,112 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: PR Build
+
+concurrency:
+ group: ${{ github.repository }}-${{ github.head_ref || github.sha }}-${{ github.workflow }}
+ cancel-in-progress: true
+
+on:
+ push:
+ paths-ignore:
+ - "doc/**"
+ - "**.md"
+ pull_request:
+ paths-ignore:
+ - "doc/**"
+ - "**.md"
+ # manual trigger
+ # https://docs.github.com/en/actions/managing-workflow-runs/manually-running-a-workflow
+ workflow_dispatch:
+
+env:
+ JAVA_VERSION: 17
+
+jobs:
+ linux-rust-test:
+ name: Rust test (amd64)
+ runs-on: ubuntu-latest
+ container:
+ image: amd64/rust
+ steps:
+ - uses: actions/checkout@v4
+ - name: Setup Rust & Java toolchain
+ uses: ./.github/actions/setup-builder
+ with:
+ rust-version: nightly
+ jdk-version: ${{env.JAVA_VERSION}}
+
+ - name: Check cargo fmt
+ run: |
+ cd core
+ cargo fmt --all -- --check --color=never
+
+ - name: Check cargo clippy
+ run: |
+ cd core
+ cargo clippy --color=never -- -D warnings
+
+ - name: Check compilation
+ run: |
+ cd core
+ cargo check --benches
+
+ - name: Setup JAVA_HOME
+ run: |
+ echo "JAVA_HOME=/usr/lib/jvm/java-${{env.JAVA_VERSION}}-openjdk-amd64" >> $GITHUB_ENV
+
+ - name: Build common module (pre-requisite for Rust tests)
+ run: |
+ cd common
+ ../mvnw clean compile -DskipTests
+
+ - name: Run cargo test
+ run: |
+ cd core
+ # This is required to run some JNI related tests on the Rust side
+ LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$JAVA_HOME/lib:$JAVA_HOME/lib/server cargo test
Review Comment:
yea added
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] build: Add basic CI test pipelines [arrow-datafusion-comet]
Posted by "sunchao (via GitHub)" <gi...@apache.org>.
sunchao merged PR #18:
URL: https://github.com/apache/arrow-datafusion-comet/pull/18
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] build: Add basic CI test pipelines [arrow-datafusion-comet]
Posted by "sunchao (via GitHub)" <gi...@apache.org>.
sunchao commented on code in PR #18:
URL: https://github.com/apache/arrow-datafusion-comet/pull/18#discussion_r1488948626
##########
.github/workflows/pr_build.yml:
##########
@@ -0,0 +1,112 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: PR Build
+
+concurrency:
+ group: ${{ github.repository }}-${{ github.head_ref || github.sha }}-${{ github.workflow }}
+ cancel-in-progress: true
+
+on:
+ push:
+ paths-ignore:
+ - "doc/**"
+ - "**.md"
+ pull_request:
+ paths-ignore:
+ - "doc/**"
+ - "**.md"
+ # manual trigger
+ # https://docs.github.com/en/actions/managing-workflow-runs/manually-running-a-workflow
+ workflow_dispatch:
+
+env:
+ JAVA_VERSION: 17
+
+jobs:
+ linux-rust-test:
+ name: Rust test (amd64)
+ runs-on: ubuntu-latest
+ container:
+ image: amd64/rust
+ steps:
+ - uses: actions/checkout@v4
+ - name: Setup Rust & Java toolchain
+ uses: ./.github/actions/setup-builder
+ with:
+ rust-version: nightly
+ jdk-version: ${{env.JAVA_VERSION}}
+
+ - name: Check cargo fmt
+ run: |
+ cd core
+ cargo fmt --all -- --check --color=never
+
+ - name: Check cargo clippy
+ run: |
+ cd core
+ cargo clippy --color=never -- -D warnings
+
+ - name: Check compilation
+ run: |
+ cd core
+ cargo check --benches
+
+ - name: Setup JAVA_HOME
+ run: |
+ echo "JAVA_HOME=/usr/lib/jvm/java-${{env.JAVA_VERSION}}-openjdk-amd64" >> $GITHUB_ENV
+
+ - name: Build common module (pre-requisite for Rust tests)
+ run: |
+ cd common
+ ../mvnw clean compile -DskipTests
+
+ - name: Run cargo test
+ run: |
+ cd core
+ # This is required to run some JNI related tests on the Rust side
+ LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$JAVA_HOME/lib:$JAVA_HOME/lib/server cargo test
+
+ linux-java-test:
+ name: Java test (amd64)
+ runs-on: ubuntu-latest
+ container:
+ image: amd64/rust
+ steps:
+ - uses: actions/checkout@v4
+ - name: Setup Rust & Java toolchain
+ uses: ./.github/actions/setup-builder
+ with:
+ rust-version: nightly
+ jdk-version: ${{env.JAVA_VERSION}}
+
+ - name: Run cargo build
+ run: |
+ cd core
+ cargo build
+
+ - name: Setup JAVA_HOME
+ run: |
+ echo "JAVA_HOME=/usr/lib/jvm/java-${{env.JAVA_VERSION}}-openjdk-amd64" >> $GITHUB_ENV
+
+ - name: Run Maven compile
+ run: |
+ ./mvnw compile test-compile scalafix:scalafix -Psemanticdb
+
+ - name: Run tests
+ run: |
+ SPARK_HOME=`pwd` ./mvnw clean install
Review Comment:
`install` covers `verify`: https://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html#a-build-lifecycle-is-made-up-of-phases
`COMET_CONF_DIR` is optional and by default, Rust test outputs is directed to stdout.
##########
.github/workflows/pr_build.yml:
##########
@@ -0,0 +1,112 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: PR Build
+
+concurrency:
+ group: ${{ github.repository }}-${{ github.head_ref || github.sha }}-${{ github.workflow }}
+ cancel-in-progress: true
+
+on:
+ push:
+ paths-ignore:
+ - "doc/**"
+ - "**.md"
+ pull_request:
+ paths-ignore:
+ - "doc/**"
+ - "**.md"
+ # manual trigger
+ # https://docs.github.com/en/actions/managing-workflow-runs/manually-running-a-workflow
+ workflow_dispatch:
+
+env:
+ JAVA_VERSION: 17
+
+jobs:
+ linux-rust-test:
+ name: Rust test (amd64)
+ runs-on: ubuntu-latest
+ container:
+ image: amd64/rust
+ steps:
+ - uses: actions/checkout@v4
+ - name: Setup Rust & Java toolchain
+ uses: ./.github/actions/setup-builder
+ with:
+ rust-version: nightly
+ jdk-version: ${{env.JAVA_VERSION}}
+
+ - name: Check cargo fmt
+ run: |
+ cd core
+ cargo fmt --all -- --check --color=never
+
+ - name: Check cargo clippy
+ run: |
+ cd core
+ cargo clippy --color=never -- -D warnings
+
+ - name: Check compilation
+ run: |
+ cd core
+ cargo check --benches
+
+ - name: Setup JAVA_HOME
+ run: |
+ echo "JAVA_HOME=/usr/lib/jvm/java-${{env.JAVA_VERSION}}-openjdk-amd64" >> $GITHUB_ENV
+
+ - name: Build common module (pre-requisite for Rust tests)
+ run: |
+ cd common
+ ../mvnw clean compile -DskipTests
+
+ - name: Run cargo test
+ run: |
+ cd core
+ # This is required to run some JNI related tests on the Rust side
+ LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$JAVA_HOME/lib:$JAVA_HOME/lib/server cargo test
+
+ linux-java-test:
+ name: Java test (amd64)
+ runs-on: ubuntu-latest
+ container:
+ image: amd64/rust
+ steps:
+ - uses: actions/checkout@v4
+ - name: Setup Rust & Java toolchain
+ uses: ./.github/actions/setup-builder
+ with:
+ rust-version: nightly
+ jdk-version: ${{env.JAVA_VERSION}}
+
+ - name: Run cargo build
+ run: |
+ cd core
+ cargo build
+
+ - name: Setup JAVA_HOME
+ run: |
+ echo "JAVA_HOME=/usr/lib/jvm/java-${{env.JAVA_VERSION}}-openjdk-amd64" >> $GITHUB_ENV
+
+ - name: Run Maven compile
+ run: |
+ ./mvnw compile test-compile scalafix:scalafix -Psemanticdb
+
+ - name: Run tests
+ run: |
+ SPARK_HOME=`pwd` ./mvnw clean install
Review Comment:
`install` covers `verify`: https://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html#a-build-lifecycle-is-made-up-of-phases
`COMET_CONF_DIR` is optional and by default, Rust test outputs are directed to stdout.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] build: Add basic CI test pipelines [arrow-datafusion-comet]
Posted by "viirya (via GitHub)" <gi...@apache.org>.
viirya commented on code in PR #18:
URL: https://github.com/apache/arrow-datafusion-comet/pull/18#discussion_r1488941277
##########
.github/workflows/pr_build.yml:
##########
@@ -0,0 +1,112 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: PR Build
+
+concurrency:
+ group: ${{ github.repository }}-${{ github.head_ref || github.sha }}-${{ github.workflow }}
+ cancel-in-progress: true
+
+on:
+ push:
+ paths-ignore:
+ - "doc/**"
+ - "**.md"
+ pull_request:
+ paths-ignore:
+ - "doc/**"
+ - "**.md"
+ # manual trigger
+ # https://docs.github.com/en/actions/managing-workflow-runs/manually-running-a-workflow
+ workflow_dispatch:
+
+env:
+ JAVA_VERSION: 17
+
+jobs:
+ linux-rust-test:
+ name: Rust test (amd64)
+ runs-on: ubuntu-latest
+ container:
+ image: amd64/rust
+ steps:
+ - uses: actions/checkout@v4
+ - name: Setup Rust & Java toolchain
+ uses: ./.github/actions/setup-builder
+ with:
+ rust-version: nightly
+ jdk-version: ${{env.JAVA_VERSION}}
+
+ - name: Check cargo fmt
+ run: |
+ cd core
+ cargo fmt --all -- --check --color=never
+
+ - name: Check cargo clippy
+ run: |
+ cd core
+ cargo clippy --color=never -- -D warnings
+
+ - name: Check compilation
+ run: |
+ cd core
+ cargo check --benches
+
+ - name: Setup JAVA_HOME
+ run: |
+ echo "JAVA_HOME=/usr/lib/jvm/java-${{env.JAVA_VERSION}}-openjdk-amd64" >> $GITHUB_ENV
+
+ - name: Build common module (pre-requisite for Rust tests)
+ run: |
+ cd common
+ ../mvnw clean compile -DskipTests
+
+ - name: Run cargo test
+ run: |
+ cd core
+ # This is required to run some JNI related tests on the Rust side
+ LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$JAVA_HOME/lib:$JAVA_HOME/lib/server cargo test
+
+ linux-java-test:
+ name: Java test (amd64)
+ runs-on: ubuntu-latest
+ container:
+ image: amd64/rust
+ steps:
+ - uses: actions/checkout@v4
+ - name: Setup Rust & Java toolchain
+ uses: ./.github/actions/setup-builder
+ with:
+ rust-version: nightly
+ jdk-version: ${{env.JAVA_VERSION}}
+
+ - name: Run cargo build
+ run: |
+ cd core
+ cargo build
+
+ - name: Setup JAVA_HOME
+ run: |
+ echo "JAVA_HOME=/usr/lib/jvm/java-${{env.JAVA_VERSION}}-openjdk-amd64" >> $GITHUB_ENV
+
+ - name: Run Maven compile
+ run: |
+ ./mvnw compile test-compile scalafix:scalafix -Psemanticdb
+
+ - name: Run tests
+ run: |
+ SPARK_HOME=`pwd` ./mvnw clean install
Review Comment:
Hmm, we use `./mvnw verify` in Makefile, do we need to do that?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] build: Add basic CI test pipelines [arrow-datafusion-comet]
Posted by "sunchao (via GitHub)" <gi...@apache.org>.
sunchao commented on PR #18:
URL: https://github.com/apache/arrow-datafusion-comet/pull/18#issuecomment-1944088119
Thanks! merged. Let me check if the Maven cache did work. I'll create followups if it doesn't.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] build: Add basic CI test pipelines [arrow-datafusion-comet]
Posted by "sunchao (via GitHub)" <gi...@apache.org>.
sunchao commented on code in PR #18:
URL: https://github.com/apache/arrow-datafusion-comet/pull/18#discussion_r1488915595
##########
spark/src/test/scala/org/apache/spark/sql/comet/CometPlanStabilitySuite.scala:
##########
@@ -297,7 +297,7 @@ class CometTPCDSV1_4_PlanStabilitySuite extends CometPlanStabilitySuite {
new File(baseResourcePath, "approved-plans-v1_4").getAbsolutePath
tpcdsQueries.foreach { q =>
- test(s"check simplified (tpcds-v1.4/$q)") {
+ ignore(s"check simplified (tpcds-v1.4/$q)") {
Review Comment:
depends on #10
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] build: Add basic CI test pipelines [arrow-datafusion-comet]
Posted by "sunchao (via GitHub)" <gi...@apache.org>.
sunchao commented on code in PR #18:
URL: https://github.com/apache/arrow-datafusion-comet/pull/18#discussion_r1488915230
##########
core/src/errors.rs:
##########
@@ -651,6 +651,7 @@ mod tests {
/// See [`object_panic_exception`] for a test which involves generating a panic and verifying
/// that the resulting stack trace includes the offending call.
#[test]
+ #[ignore]
Review Comment:
the golden file for this test is not added yet, so ignore for now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] build: Add basic CI test pipelines [arrow-datafusion-comet]
Posted by "viirya (via GitHub)" <gi...@apache.org>.
viirya commented on code in PR #18:
URL: https://github.com/apache/arrow-datafusion-comet/pull/18#discussion_r1488941787
##########
.github/workflows/pr_build.yml:
##########
@@ -0,0 +1,112 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: PR Build
+
+concurrency:
+ group: ${{ github.repository }}-${{ github.head_ref || github.sha }}-${{ github.workflow }}
+ cancel-in-progress: true
+
+on:
+ push:
+ paths-ignore:
+ - "doc/**"
+ - "**.md"
+ pull_request:
+ paths-ignore:
+ - "doc/**"
+ - "**.md"
+ # manual trigger
+ # https://docs.github.com/en/actions/managing-workflow-runs/manually-running-a-workflow
+ workflow_dispatch:
+
+env:
+ JAVA_VERSION: 17
+
+jobs:
+ linux-rust-test:
+ name: Rust test (amd64)
+ runs-on: ubuntu-latest
+ container:
+ image: amd64/rust
+ steps:
+ - uses: actions/checkout@v4
+ - name: Setup Rust & Java toolchain
+ uses: ./.github/actions/setup-builder
+ with:
+ rust-version: nightly
+ jdk-version: ${{env.JAVA_VERSION}}
+
+ - name: Check cargo fmt
+ run: |
+ cd core
+ cargo fmt --all -- --check --color=never
+
+ - name: Check cargo clippy
+ run: |
+ cd core
+ cargo clippy --color=never -- -D warnings
+
+ - name: Check compilation
+ run: |
+ cd core
+ cargo check --benches
+
+ - name: Setup JAVA_HOME
+ run: |
+ echo "JAVA_HOME=/usr/lib/jvm/java-${{env.JAVA_VERSION}}-openjdk-amd64" >> $GITHUB_ENV
+
+ - name: Build common module (pre-requisite for Rust tests)
+ run: |
+ cd common
+ ../mvnw clean compile -DskipTests
+
+ - name: Run cargo test
+ run: |
+ cd core
+ # This is required to run some JNI related tests on the Rust side
+ LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$JAVA_HOME/lib:$JAVA_HOME/lib/server cargo test
+
+ linux-java-test:
+ name: Java test (amd64)
+ runs-on: ubuntu-latest
+ container:
+ image: amd64/rust
+ steps:
+ - uses: actions/checkout@v4
+ - name: Setup Rust & Java toolchain
+ uses: ./.github/actions/setup-builder
+ with:
+ rust-version: nightly
+ jdk-version: ${{env.JAVA_VERSION}}
+
+ - name: Run cargo build
+ run: |
+ cd core
+ cargo build
+
+ - name: Setup JAVA_HOME
+ run: |
+ echo "JAVA_HOME=/usr/lib/jvm/java-${{env.JAVA_VERSION}}-openjdk-amd64" >> $GITHUB_ENV
+
+ - name: Run Maven compile
+ run: |
+ ./mvnw compile test-compile scalafix:scalafix -Psemanticdb
+
+ - name: Run tests
+ run: |
+ SPARK_HOME=`pwd` ./mvnw clean install
Review Comment:
Don't we need `BOSON_CONF_DIR` and `SPARK_HOME`?
##########
.github/workflows/pr_build.yml:
##########
@@ -0,0 +1,112 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: PR Build
+
+concurrency:
+ group: ${{ github.repository }}-${{ github.head_ref || github.sha }}-${{ github.workflow }}
+ cancel-in-progress: true
+
+on:
+ push:
+ paths-ignore:
+ - "doc/**"
+ - "**.md"
+ pull_request:
+ paths-ignore:
+ - "doc/**"
+ - "**.md"
+ # manual trigger
+ # https://docs.github.com/en/actions/managing-workflow-runs/manually-running-a-workflow
+ workflow_dispatch:
+
+env:
+ JAVA_VERSION: 17
+
+jobs:
+ linux-rust-test:
+ name: Rust test (amd64)
+ runs-on: ubuntu-latest
+ container:
+ image: amd64/rust
+ steps:
+ - uses: actions/checkout@v4
+ - name: Setup Rust & Java toolchain
+ uses: ./.github/actions/setup-builder
+ with:
+ rust-version: nightly
+ jdk-version: ${{env.JAVA_VERSION}}
+
+ - name: Check cargo fmt
+ run: |
+ cd core
+ cargo fmt --all -- --check --color=never
+
+ - name: Check cargo clippy
+ run: |
+ cd core
+ cargo clippy --color=never -- -D warnings
+
+ - name: Check compilation
+ run: |
+ cd core
+ cargo check --benches
+
+ - name: Setup JAVA_HOME
+ run: |
+ echo "JAVA_HOME=/usr/lib/jvm/java-${{env.JAVA_VERSION}}-openjdk-amd64" >> $GITHUB_ENV
+
+ - name: Build common module (pre-requisite for Rust tests)
+ run: |
+ cd common
+ ../mvnw clean compile -DskipTests
+
+ - name: Run cargo test
+ run: |
+ cd core
+ # This is required to run some JNI related tests on the Rust side
+ LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$JAVA_HOME/lib:$JAVA_HOME/lib/server cargo test
+
+ linux-java-test:
+ name: Java test (amd64)
+ runs-on: ubuntu-latest
+ container:
+ image: amd64/rust
+ steps:
+ - uses: actions/checkout@v4
+ - name: Setup Rust & Java toolchain
+ uses: ./.github/actions/setup-builder
+ with:
+ rust-version: nightly
+ jdk-version: ${{env.JAVA_VERSION}}
+
+ - name: Run cargo build
+ run: |
+ cd core
+ cargo build
+
+ - name: Setup JAVA_HOME
+ run: |
+ echo "JAVA_HOME=/usr/lib/jvm/java-${{env.JAVA_VERSION}}-openjdk-amd64" >> $GITHUB_ENV
+
+ - name: Run Maven compile
+ run: |
+ ./mvnw compile test-compile scalafix:scalafix -Psemanticdb
+
+ - name: Run tests
+ run: |
+ SPARK_HOME=`pwd` ./mvnw clean install
Review Comment:
Don't we need `BOSON_CONF_DIR`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org