You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by ru...@apache.org on 2025/02/12 05:11:15 UTC
(spark) branch branch-4.0 updated: [MINOR][INFRA] List pip installation before test in python macos test and python connect test
This is an automated email from the ASF dual-hosted git repository.
ruifengz pushed a commit to branch branch-4.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.0 by this push:
new c9d1ece83ba2 [MINOR][INFRA] List pip installation before test in python macos test and python connect test
c9d1ece83ba2 is described below
commit c9d1ece83ba2fc8c77b81ff208ebae673395c4e2
Author: Ruifeng Zheng <ru...@apache.org>
AuthorDate: Wed Feb 12 13:10:43 2025 +0800
[MINOR][INFRA] List pip installation before test in python macos test and python connect test
### What changes were proposed in this pull request?
List pip installation before test in python macos test and python connect test
### Why are the changes needed?
to improve the debugability, no need to check the installation log
### Does this PR introduce _any_ user-facing change?
no
### How was this patch tested?
ci
### Was this patch authored or co-authored using generative AI tooling?
no
Closes #49901 from zhengruifeng/infra_list_pip.
Authored-by: Ruifeng Zheng <ru...@apache.org>
Signed-off-by: Ruifeng Zheng <ru...@apache.org>
(cherry picked from commit d6fe024ac4693c370f823f3ef8d6ef79dbe76343)
Signed-off-by: Ruifeng Zheng <ru...@apache.org>
---
.github/workflows/build_python_connect.yml | 4 +++-
.github/workflows/build_python_connect35.yml | 2 ++
.github/workflows/python_macos_test.yml | 5 +++--
3 files changed, 8 insertions(+), 3 deletions(-)
diff --git a/.github/workflows/build_python_connect.yml b/.github/workflows/build_python_connect.yml
index c2a10bfedc44..5aaa1aa4713d 100644
--- a/.github/workflows/build_python_connect.yml
+++ b/.github/workflows/build_python_connect.yml
@@ -72,7 +72,9 @@ jobs:
python packaging/connect/setup.py sdist
cd dist
pip install pyspark*connect-*.tar.gz
- pip install 'six==1.16.0' 'pandas==2.2.3' scipy 'plotly<6.0.0' 'mlflow>=2.8.1' coverage matplotlib openpyxl 'memory-profiler>=0.61.0' 'scikit-learn>=1.3.2' 'graphviz==0.20.3' 'torch<2.6.0' torchvision torcheval deepspeed unittest-xml-reporting 'plotly>=4.8'
+ pip install 'six==1.16.0' 'pandas==2.2.3' scipy 'plotly<6.0.0' 'mlflow>=2.8.1' coverage matplotlib openpyxl 'memory-profiler>=0.61.0' 'scikit-learn>=1.3.2' 'graphviz==0.20.3' 'torch<2.6.0' torchvision torcheval deepspeed unittest-xml-reporting
+ - name: List Python packages
+ run: python -m pip list
- name: Run tests (local)
env:
SPARK_TESTING: 1
diff --git a/.github/workflows/build_python_connect35.yml b/.github/workflows/build_python_connect35.yml
index 7b854f8b1a28..66bd816d39be 100644
--- a/.github/workflows/build_python_connect35.yml
+++ b/.github/workflows/build_python_connect35.yml
@@ -75,6 +75,8 @@ jobs:
# Add torch as a testing dependency for TorchDistributor
pip install 'torch==2.0.1' 'torchvision==0.15.2' torcheval
+ - name: List Python packages
+ run: python -m pip list
- name: Run tests
env:
SPARK_TESTING: 1
diff --git a/.github/workflows/python_macos_test.yml b/.github/workflows/python_macos_test.yml
index cb6b33fb2a50..2cffb68419e8 100644
--- a/.github/workflows/python_macos_test.yml
+++ b/.github/workflows/python_macos_test.yml
@@ -135,8 +135,9 @@ jobs:
python${{matrix.python}} -m pip install --ignore-installed 'six==1.16.0'
python${{matrix.python}} -m pip install numpy 'pyarrow>=15.0.0' 'six==1.16.0' 'pandas==2.2.3' scipy 'plotly<6.0.0' 'mlflow>=2.8.1' coverage matplotlib openpyxl 'memory-profiler>=0.61.0' 'scikit-learn>=1.3.2' unittest-xml-reporting && \
python${{matrix.python}} -m pip install 'grpcio==1.67.0' 'grpcio-status==1.67.0' 'protobuf==5.29.1' 'googleapis-common-protos==1.65.0' 'graphviz==0.20.3' && \
- python${{matrix.python}} -m pip cache purge && \
- python${{matrix.python}} -m pip list
+ python${{matrix.python}} -m pip cache purge
+ - name: List Python packages
+ run: python${{matrix.python}} -m pip list
# Run the tests.
- name: Run tests
env: ${{ fromJSON(inputs.envs) }}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org