You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by gu...@apache.org on 2023/10/27 12:39:35 UTC
(spark) tag v3.5.0 created (now 270861a3cd6)
This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a change to tag v3.5.0
in repository https://gitbox.apache.org/repos/asf/spark.git
at 270861a3cd6 (commit)
This tag includes the following new commits:
new 270861a3cd6 [SPARK-45706][PYTHON][DOCS] Makes entire Binder build fails fast during setting up
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
(spark) 01/01: [SPARK-45706][PYTHON][DOCS] Makes entire Binder build fails fast during setting up
Posted by gu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to tag v3.5.0
in repository https://gitbox.apache.org/repos/asf/spark.git
commit 270861a3cd6c21b3685a6702a6682672f7720d4a
Author: Hyukjin Kwon <gu...@apache.org>
AuthorDate: Fri Oct 27 21:20:40 2023 +0900
[SPARK-45706][PYTHON][DOCS] Makes entire Binder build fails fast during setting up
### What changes were proposed in this pull request?
This PR proposes to make entire Binder build fails fast during setting up to prevent the Binder image to be successfully built which it cannot be rebuilt later on the same commit.
### Why are the changes needed?
Binder build is currently broken for Spark 3.5.0:
https://mybinder.org/v2/gh/apache/spark/ce5ddad9903?filepath=python%2Fdocs%2Fsource%2Fgetting_started%2Fquickstart_df.ipynb
Seems like we uploaded PySpark late into PyPI, and the installation steps just slightly ignored the failure (a user triggered the first docker image for Binder, and that's being reused at that time PySpark wasn't uploaded to PyPI).
### Does this PR introduce _any_ user-facing change?
Yes, it fixes the user-facing live notebooks (at https://spark.apache.org/docs/latest/api/python/index.html).
### How was this patch tested?
Manually tested in my fork:
- https://mybinder.org/v2/gh/HyukjinKwon/spark/ce5ddad9903?filepath=python%2Fdocs%2Fsource%2Fgetting_started%2Fquickstart_df.ipynb
- https://mybinder.org/v2/gh/HyukjinKwon/spark/ce5ddad9903?filepath=python%2Fdocs%2Fsource%2Fgetting_started%2Fquickstart_connect.ipynb
- https://mybinder.org/v2/gh/HyukjinKwon/spark/ce5ddad9903?filepath=python%2Fdocs%2Fsource%2Fgetting_started%2Fquickstart_ps.ipynb
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #43553 from HyukjinKwon/SPARK-45706.
Authored-by: Hyukjin Kwon <gu...@apache.org>
Signed-off-by: Hyukjin Kwon <gu...@apache.org>
(cherry picked from commit f1d1dc1f87a7e4accd1c3c2d824f39df05465906)
Signed-off-by: Hyukjin Kwon <gu...@apache.org>
---
binder/postBuild | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/binder/postBuild b/binder/postBuild
index 70ae23b3937..fec233f8c8c 100644
--- a/binder/postBuild
+++ b/binder/postBuild
@@ -20,6 +20,11 @@
# This file is used for Binder integration to install PySpark available in
# Jupyter notebook.
+# SPARK-45706: Should fail fast. Otherwise, the Binder image is successfully
+# built, and it cannot be rebuilt.
+set -o pipefail
+set -e
+
VERSION=$(python -c "exec(open('python/pyspark/version.py').read()); print(__version__)")
TAG=$(git describe --tags --exact-match 2>/dev/null)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org