(spark) branch branch-4.0 updated: [SPARK-52454][BUILD] Automatically remove old releases from the mirror

[gu...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

gurwls223 pushed a commit to branch branch-4.0
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-4.0 by this push:
     new 774db408247e [SPARK-52454][BUILD] Automatically remove old releases from the mirror
774db408247e is described below

commit 774db408247e9cb8cf446271389cf1bf62b1e868
Author: Hyukjin Kwon <gu...@apache.org>
AuthorDate: Thu Jun 12 04:55:49 2025 -0700

    [SPARK-52454][BUILD] Automatically remove old releases from the mirror
    
    ### What changes were proposed in this pull request?
    
    This PR adds the automated step for "Remove old releases from Mirror Network" in https://spark.apache.org/release-process.html
    
    ### Why are the changes needed?
    
    To make the release easier.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No, dev-only.
    
    ### How was this patch tested?
    
    Manually tested the commands but it needs to be tested in the official release.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    No.
    
    Closes #51160 from HyukjinKwon/SPARK-52454.
    
    Authored-by: Hyukjin Kwon <gu...@apache.org>
    Signed-off-by: Hyukjin Kwon <gu...@apache.org>
    (cherry picked from commit bf9237396a8981b0ce96bf172755065e35f1b5b1)
    Signed-off-by: Hyukjin Kwon <gu...@apache.org>
---
 dev/create-release/release-build.sh | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/dev/create-release/release-build.sh b/dev/create-release/release-build.sh
index 1245b414fb0c..c78c6b463068 100755
--- a/dev/create-release/release-build.sh
+++ b/dev/create-release/release-build.sh
@@ -230,6 +230,26 @@ if [[ "$1" == "finalize" ]]; then
     -d "{\"data\": {\"stagedRepositoryIds\": [\"$REPO_ID\"], \"description\": \"Dropped after release\"}}"
 
   echo "Done."
+
+  # TODO: Test it in the actual official release
+  # Remove old releases from the mirror
+  # Extract major.minor prefix
+  RELEASE_SERIES=$(echo "$RELEASE_VERSION" | cut -d. -f1-2)
+  
+  # Fetch existing dist URLs
+  OLD_VERSION=$(svn ls https://dist.apache.org/repos/dist/release/spark/ | \
+    grep "^spark-$RELEASE_SERIES" | \
+    grep -v "^spark-$RELEASE_VERSION/" | \
+    sed 's#/##' | sed 's/^spark-//' | \
+    sort -V | tail -n 1)
+  
+  if [[ -n "$OLD_VERSION" ]]; then
+    echo "Removing old version: spark-$OLD_VERSION"
+    svn rm "https://dist.apache.org/repos/dist/release/spark/spark-$OLD_VERSION" -m "Remove older $RELEASE_SERIES release after $RELEASE_VERSION"
+  else
+    echo "No previous $RELEASE_SERIES version found to remove. Manually remove it if there is."
+  fi
+
   exit 0
 fi
 


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org