You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2025/09/12 23:49:11 UTC
(spark-connect-swift) branch main updated: [SPARK-53571] Add `integration-test-mac-spark4-iceberg` GitHub Action job
This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/spark-connect-swift.git
The following commit(s) were added to refs/heads/main by this push:
new 4bbaa9d [SPARK-53571] Add `integration-test-mac-spark4-iceberg` GitHub Action job
4bbaa9d is described below
commit 4bbaa9db182536beed3c67a9320eb751eb6aeb9d
Author: Dongjoon Hyun <do...@apache.org>
AuthorDate: Fri Sep 12 16:49:04 2025 -0700
[SPARK-53571] Add `integration-test-mac-spark4-iceberg` GitHub Action job
### What changes were proposed in this pull request?
This PR aims to add `integration-test-mac-spark4-iceberg` GitHub Action job. In addition, the existing GitHub Action job `integration-test-mac-iceberg` is renamed to `integration-test-mac-spark3-iceberg` to be clear.
### Why are the changes needed?
Apache Iceberg 1.10 is released with Apache Spark 4 support. We need to have a test coverage for this combination.
https://github.com/apache/iceberg/releases/tag/apache-iceberg-1.10.0
### Does this PR introduce _any_ user-facing change?
No, this is a test infra change.
### How was this patch tested?
Pass the CIs.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #239 from dongjoon-hyun/SPARK-53571.
Authored-by: Dongjoon Hyun <do...@apache.org>
Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
.github/workflows/build_and_test.yml | 31 ++++++++++++++++++++++++++++++-
1 file changed, 30 insertions(+), 1 deletion(-)
diff --git a/.github/workflows/build_and_test.yml b/.github/workflows/build_and_test.yml
index 4226988..9b2fbf5 100644
--- a/.github/workflows/build_and_test.yml
+++ b/.github/workflows/build_and_test.yml
@@ -225,7 +225,7 @@ jobs:
cd -
swift test --no-parallel -c release
- integration-test-mac-iceberg:
+ integration-test-mac-spark3-iceberg:
runs-on: macos-15
timeout-minutes: 20
env:
@@ -254,6 +254,35 @@ jobs:
swift test --filter DataFrameWriterV2Tests -c release
swift test --filter IcebergTest -c release
+ integration-test-mac-spark4-iceberg:
+ runs-on: macos-15
+ timeout-minutes: 20
+ env:
+ SPARK_LOCAL_IP: localhost
+ SPARK_ICEBERG_TEST_ENABLED: "true"
+ steps:
+ - uses: actions/checkout@v5
+ - uses: swift-actions/setup-swift@d10500c1ac8822132eebbd74c48c3372c71d7ff5
+ with:
+ swift-version: "6.1"
+ - name: Install Java
+ uses: actions/setup-java@v4
+ with:
+ distribution: zulu
+ java-version: 17
+ - name: Build
+ run: swift test --filter NOTHING -c release
+ - name: Test
+ run: |
+ curl -LO https://www.apache.org/dyn/closer.lua/spark/spark-4.0.1/spark-4.0.1-bin-hadoop3.tgz?action=download
+ tar xvfz spark-4.0.1-bin-hadoop3.tgz && rm spark-4.0.1-bin-hadoop3.tgz
+ mv spark-4.0.1-bin-hadoop3 /tmp/spark
+ cd /tmp/spark/sbin
+ ./start-connect-server.sh --packages org.apache.spark:spark-connect_2.13:4.0.1,org.apache.iceberg:iceberg-spark-runtime-4.0_2.13:1.10.0 -c spark.sql.catalog.local=org.apache.iceberg.spark.SparkCatalog -c spark.sql.catalog.local.type=hadoop -c spark.sql.catalog.local.warehouse=/tmp/spark/warehouse -c spark.sql.defaultCatalog=local
+ cd -
+ swift test --filter DataFrameWriterV2Tests -c release
+ swift test --filter IcebergTest -c release
+
linter:
runs-on: ubuntu-latest
timeout-minutes: 20
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org