You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by ru...@apache.org on 2025/04/14 04:14:25 UTC
(spark) branch master updated: [SPARK-51788][INFRA] Add a PySpark test that runs every 3 days using the Ubuntu Arm Runner
This is an automated email from the ASF dual-hosted git repository.
ruifengz pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new c3af3dc0b70e [SPARK-51788][INFRA] Add a PySpark test that runs every 3 days using the Ubuntu Arm Runner
c3af3dc0b70e is described below
commit c3af3dc0b70e18d83a74efeec6250c911259738a
Author: yangjie01 <ya...@baidu.com>
AuthorDate: Mon Apr 14 12:14:05 2025 +0800
[SPARK-51788][INFRA] Add a PySpark test that runs every 3 days using the Ubuntu Arm Runner
### What changes were proposed in this pull request?
Similar to SPARK-51761, this pr adds a PySpark test that runs every 3 days using the Ubuntu Arm Runner》
### Why are the changes needed?
Check the availability of PySpark on `Linux + Arm64`.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Monitor Daily test after merged.
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #50574 from LuciferYang/python-linux-arm.
Lead-authored-by: yangjie01 <ya...@baidu.com>
Co-authored-by: YangJie <ya...@baidu.com>
Signed-off-by: Ruifeng Zheng <ru...@apache.org>
---
...hon_3.11_macos.yml => build_python_3.11_arm.yml} | 8 +++++---
.github/workflows/build_python_3.11_macos.yml | 2 +-
...macos_test.yml => python_hosted_runner_test.yml} | 21 ++++++++++++++++++---
3 files changed, 24 insertions(+), 7 deletions(-)
diff --git a/.github/workflows/build_python_3.11_macos.yml b/.github/workflows/build_python_3.11_arm.yml
similarity index 84%
copy from .github/workflows/build_python_3.11_macos.yml
copy to .github/workflows/build_python_3.11_arm.yml
index 57902e4871ff..f0a1b467703c 100644
--- a/.github/workflows/build_python_3.11_macos.yml
+++ b/.github/workflows/build_python_3.11_arm.yml
@@ -17,11 +17,11 @@
# under the License.
#
-name: "Build / Python-only (master, Python 3.11, MacOS)"
+name: "Build / Python-only (master, Python 3.11, ARM)"
on:
schedule:
- - cron: '0 21 * * *'
+ - cron: '0 22 */3 * *'
workflow_dispatch:
jobs:
@@ -29,5 +29,7 @@ jobs:
permissions:
packages: write
name: Run
- uses: ./.github/workflows/python_macos_test.yml
+ uses: ./.github/workflows/python_hosted_runner_test.yml
if: github.repository == 'apache/spark'
+ with:
+ os: ubuntu-24.04-arm
diff --git a/.github/workflows/build_python_3.11_macos.yml b/.github/workflows/build_python_3.11_macos.yml
index 57902e4871ff..9566bfd8271d 100644
--- a/.github/workflows/build_python_3.11_macos.yml
+++ b/.github/workflows/build_python_3.11_macos.yml
@@ -29,5 +29,5 @@ jobs:
permissions:
packages: write
name: Run
- uses: ./.github/workflows/python_macos_test.yml
+ uses: ./.github/workflows/python_hosted_runner_test.yml
if: github.repository == 'apache/spark'
diff --git a/.github/workflows/python_macos_test.yml b/.github/workflows/python_hosted_runner_test.yml
similarity index 88%
rename from .github/workflows/python_macos_test.yml
rename to .github/workflows/python_hosted_runner_test.yml
index 2cb3d8dd23b7..03d423acca71 100644
--- a/.github/workflows/python_macos_test.yml
+++ b/.github/workflows/python_hosted_runner_test.yml
@@ -40,6 +40,16 @@ on:
required: false
type: string
default: hadoop3
+ os:
+ description: OS to run this build.
+ required: false
+ type: string
+ default: macos-15
+ arch:
+ description: The target architecture (x86, x64, arm64) of the Python or PyPy interpreter.
+ required: false
+ type: string
+ default: arm64
envs:
description: Additional environment variables to set when running the tests. Should be in JSON format.
required: false
@@ -48,7 +58,7 @@ on:
jobs:
build:
name: "PySpark test on macos: ${{ matrix.modules }}"
- runs-on: macos-15
+ runs-on: ${{ inputs.os }}
strategy:
fail-fast: false
matrix:
@@ -129,6 +139,11 @@ jobs:
with:
distribution: zulu
java-version: ${{ matrix.java }}
+ - name: Install Python ${{matrix.python}}
+ uses: actions/setup-python@v5
+ with:
+ python-version: ${{matrix.python}}
+ architecture: ${{ inputs.arch }}
- name: Install Python packages (Python ${{matrix.python}})
run: |
python${{matrix.python}} -m pip install --ignore-installed 'blinker>=1.6.2'
@@ -152,12 +167,12 @@ jobs:
if: always()
uses: actions/upload-artifact@v4
with:
- name: test-results-${{ matrix.modules }}--${{ matrix.java }}-${{ inputs.hadoop }}-hive2.3-${{ env.PYTHON_TO_TEST }}
+ name: test-results-${{ inputs.os }}-${{ matrix.modules }}--${{ matrix.java }}-${{ inputs.hadoop }}-hive2.3-${{ env.PYTHON_TO_TEST }}
path: "**/target/test-reports/*.xml"
- name: Upload unit tests log files
env: ${{ fromJSON(inputs.envs) }}
if: ${{ !success() }}
uses: actions/upload-artifact@v4
with:
- name: unit-tests-log-${{ matrix.modules }}--${{ matrix.java }}-${{ inputs.hadoop }}-hive2.3-${{ env.PYTHON_TO_TEST }}
+ name: unit-tests-log-${{ inputs.os }}-${{ matrix.modules }}--${{ matrix.java }}-${{ inputs.hadoop }}-hive2.3-${{ env.PYTHON_TO_TEST }}
path: "**/target/unit-tests.log"
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org