[spark] branch master updated: [SPARK-44103][INFRA] Make pending container jobs cancelable

[ru...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

ruifengz pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new 526831b8f07 [SPARK-44103][INFRA] Make pending container jobs cancelable
526831b8f07 is described below

commit 526831b8f072109df464d503076fda32f8fd5981
Author: Ruifeng Zheng <ru...@foxmail.com>
AuthorDate: Wed Jun 21 09:48:26 2023 +0800

    [SPARK-44103][INFRA] Make pending container jobs cancelable
    
    ### What changes were proposed in this pull request?
    Make pending container jobs (pyspark/sparkr/linter) cancelable, to release the resources ASAP.
    
    ### Why are the changes needed?
    to release the resources ASAP.
    
    before this PR when we click the `Cancel Workflow`,  container jobs can not be cancelled, no matter the status (running or pending).
    
    In this PR, we can cancel the pending container jobs. Note that the running ones still can not be cancelled.
    
    ### Does this PR introduce _any_ user-facing change?
    no, test-only
    
    ### How was this patch tested?
    manually check
    
    Closes #41668 from zhengruifeng/infra_docker_stop.
    
    Lead-authored-by: Ruifeng Zheng <ru...@foxmail.com>
    Co-authored-by: Ruifeng Zheng <ru...@apache.org>
    Signed-off-by: Ruifeng Zheng <ru...@apache.org>
---
 .github/workflows/build_and_test.yml | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/.github/workflows/build_and_test.yml b/.github/workflows/build_and_test.yml
index 93d69f18740..a03aa53dc88 100644
--- a/.github/workflows/build_and_test.yml
+++ b/.github/workflows/build_and_test.yml
@@ -333,7 +333,7 @@ jobs:
   pyspark:
     needs: [precondition, infra-image]
     # always run if pyspark == 'true', even infra-image is skip (such as non-master job)
-    if: always() && fromJson(needs.precondition.outputs.required).pyspark == 'true'
+    if: (!cancelled()) && fromJson(needs.precondition.outputs.required).pyspark == 'true'
     name: "Build modules: ${{ matrix.modules }}"
     runs-on: ubuntu-22.04
     container:
@@ -446,7 +446,7 @@ jobs:
   sparkr:
     needs: [precondition, infra-image]
     # always run if sparkr == 'true', even infra-image is skip (such as non-master job)
-    if: always() && fromJson(needs.precondition.outputs.required).sparkr == 'true'
+    if: (!cancelled()) && fromJson(needs.precondition.outputs.required).sparkr == 'true'
     name: "Build modules: sparkr"
     runs-on: ubuntu-22.04
     container:
@@ -548,7 +548,7 @@ jobs:
   lint:
     needs: [precondition, infra-image]
     # always run if lint == 'true', even infra-image is skip (such as non-master job)
-    if: always() && fromJson(needs.precondition.outputs.required).lint == 'true'
+    if: (!cancelled()) && fromJson(needs.precondition.outputs.required).lint == 'true'
     name: Linters, licenses, dependencies and documentation generation
     runs-on: ubuntu-22.04
     env:


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org