You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by gu...@apache.org on 2025/02/25 00:22:04 UTC
(spark) branch master updated: [SPARK-50914][PYTHON][CONNECT] Match GRPC dependencies for Python-only master scheduled job
This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 529e887b794e [SPARK-50914][PYTHON][CONNECT] Match GRPC dependencies for Python-only master scheduled job
529e887b794e is described below
commit 529e887b794e67385ef98fa43be6fe000776db99
Author: Hyukjin Kwon <gu...@apache.org>
AuthorDate: Tue Feb 25 09:21:53 2025 +0900
[SPARK-50914][PYTHON][CONNECT] Match GRPC dependencies for Python-only master scheduled job
### What changes were proposed in this pull request?
This PR matches the GRPC dependencies for "Build / Spark Connect Python-only (master, Python 3.11)" build: https://github.com/apache/spark/blob/d4b4cfc79491fb8a39ccb0b3d0ca2e1d1c8b7a70/dev/infra/Dockerfile#L99
### Why are the changes needed?
To fix the broken build up https://github.com/apache/spark/actions/workflows/build_python_connect.yml
### Does this PR introduce _any_ user-facing change?
No, test-only.
### How was this patch tested?
Tested in https://github.com/HyukjinKwon/spark/actions/runs/13496429113
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #50058 from HyukjinKwon/debug-python-job.
Authored-by: Hyukjin Kwon <gu...@apache.org>
Signed-off-by: Hyukjin Kwon <gu...@apache.org>
---
.github/workflows/build_python_connect.yml | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/.github/workflows/build_python_connect.yml b/.github/workflows/build_python_connect.yml
index 62c7d0582214..67428d6af0de 100644
--- a/.github/workflows/build_python_connect.yml
+++ b/.github/workflows/build_python_connect.yml
@@ -72,7 +72,7 @@ jobs:
python packaging/client/setup.py sdist
cd dist
pip install pyspark*client-*.tar.gz
- pip install 'six==1.16.0' 'pandas==2.2.3' scipy 'plotly<6.0.0' 'mlflow>=2.8.1' coverage matplotlib openpyxl 'memory-profiler>=0.61.0' 'scikit-learn>=1.3.2' 'graphviz==0.20.3' 'torch<2.6.0' torchvision torcheval deepspeed unittest-xml-reporting
+ pip install 'grpcio==1.67.0' 'grpcio-status==1.67.0' 'protobuf==5.29.1' 'googleapis-common-protos==1.65.0' 'graphviz==0.20.3' 'six==1.16.0' 'pandas==2.2.3' scipy 'plotly<6.0.0' 'mlflow>=2.8.1' coverage matplotlib openpyxl 'memory-profiler>=0.61.0' 'scikit-learn>=1.3.2' 'graphviz==0.20.3' 'torch<2.6.0' torchvision torcheval deepspeed unittest-xml-reporting
- name: List Python packages
run: python -m pip list
- name: Run tests (local)
@@ -116,7 +116,7 @@ jobs:
# Remove Py4J and PySpark zipped library to make sure there is no JVM connection
mv python/lib lib.back
- mv python/pyspark lib.back
+ mv python/pyspark pyspark.back
./python/run-tests --parallelism=1 --python-executables=python3 --testnames "pyspark.resource.tests.test_connect_resources,pyspark.sql.tests.connect.client.test_artifact,pyspark.sql.tests.connect.client.test_artifact_localcluster,pyspark.sql.tests.connect.test_resources"
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org