You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "gaogaotiantian (via GitHub)" <gi...@apache.org> on 2026/06/11 23:02:50 UTC

[PR] [SPARK-57398][INFRA][4.0] Make branch-4.0 scheduled build workflows self-contained [spark]

gaogaotiantian opened a new pull request, #56461:
URL: https://github.com/apache/spark/pull/56461

   ### What changes were proposed in this pull request?
   
   This PR makes the `branch-4.0` scheduled build workflow files self-contained, following the same approach as the `branch-4.1` work (SPARK-57267) and SPARK-57115 (which did this for `build_java17.yml`):
   
   - Rewrites `build_java21.yml`, `build_non_ansi.yml`, `build_maven.yml`, `build_maven_java21.yml`, and `build_python_pypy3.10.yml` so they are triggered by `workflow_dispatch` only (the dormant `schedule:` triggers are removed; scheduled runs on a non-default branch never fire anyway), drop the inherited `branch: master` (so they build `branch-4.0`), and use generic, non-branch-tagged names.
   - Adds `build_python_3.11.yml`, the equivalent of `build_branch40_python.yml`.
   
   The per-build configurations are relocated from the active `build_branch40_*.yml` files on `master` so the coverage matches what `branch-4.0` schedules today.
   
   Notes:
   - `build_and_test.yml` already defaults `branch` to `branch-4.0`, so those callers omit `branch`. `maven_test.yml` defaulted `branch` to `master`, so its default is bumped to `branch-4.0`; with that fixed, `build_maven.yml` / `build_maven_java21.yml` no longer need an explicit `branch`. This also corrects `build_maven_java21_macos15.yml`, which omits `branch` and was likewise defaulting to master.
   - A pre-existing invalid trailing comma in the non-ANSI `envs` JSON was fixed during relocation, and `build-core-utils: false` was added to match `build_branch40_non_ansi.yml`.
   
   ### Why are the changes needed?
   
   This is the `branch-4.0` side of decoupling our scheduled CIs (cf. the `branch-4.x` and `branch-4.1` efforts). Scheduled workflows only fire from the default branch, so `branch-4.0` CI should consist of self-contained, dispatchable workflow files on `branch-4.0` that a single scheduler on `master` can trigger. This PR prepares those targets; a follow-up on `master` will add the scheduler and remove the `build_branch40_*.yml` files.
   
   ### Does this PR introduce _any_ user-facing change?
   
   No. CI only.
   
   ### How was this patch tested?
   
   These workflows can be triggered manually via `workflow_dispatch` once merged.
   
   ### Was this patch authored or co-authored using generative AI tooling?
   
   Generated-by: Claude Code (Claude Opus 4.8)
   
   This pull request and its description were written by Isaac.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-57398][INFRA][4.0] Make branch-4.0 scheduled build workflows self-contained [spark]

Posted by "gaogaotiantian (via GitHub)" <gi...@apache.org>.
gaogaotiantian closed pull request #56461: [SPARK-57398][INFRA][4.0] Make branch-4.0 scheduled build workflows self-contained
URL: https://github.com/apache/spark/pull/56461


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-57398][INFRA][4.0] Make branch-4.0 scheduled build workflows self-contained [spark]

Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on PR #56461:
URL: https://github.com/apache/spark/pull/56461#issuecomment-4687899314

   Yes, @gaogaotiantian !


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-57398][INFRA][4.0] Make branch-4.0 scheduled build workflows self-contained [spark]

Posted by "gaogaotiantian (via GitHub)" <gi...@apache.org>.
gaogaotiantian commented on PR #56461:
URL: https://github.com/apache/spark/pull/56461#issuecomment-4685841325

   @dongjoon-hyun , I waited for the release of 4.0.3 to apply this CI change. Is branch-4.0 open for commit now?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-57398][INFRA][4.0] Make branch-4.0 scheduled build workflows self-contained [spark]

Posted by "gaogaotiantian (via GitHub)" <gi...@apache.org>.
gaogaotiantian commented on PR #56461:
URL: https://github.com/apache/spark/pull/56461#issuecomment-4695474411

   Thanks all, merged to branch-4.0


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-57398][INFRA][4.0] Make branch-4.0 scheduled build workflows self-contained [spark]

Posted by "gaogaotiantian (via GitHub)" <gi...@apache.org>.
gaogaotiantian commented on code in PR #56461:
URL: https://github.com/apache/spark/pull/56461#discussion_r3405978250


##########
.github/workflows/build_non_ansi.yml:
##########
@@ -33,17 +31,17 @@ jobs:
     if: github.repository == 'apache/spark'
     with:
       java: 17
-      branch: master
       hadoop: hadoop3
       envs: >-
         {
           "PYSPARK_IMAGE_TO_TEST": "python-311",
           "PYTHON_TO_TEST": "python3.11",
-          "SPARK_ANSI_SQL_MODE": "false",

Review Comment:
   I believe the trailing comma is actually wrong. This is a JSON string and JSON does not allow trailing comma. `fromJSON` would crash with it. It was not caught because this file was not ever used.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-57398][INFRA][4.0] Make branch-4.0 scheduled build workflows self-contained [spark]

Posted by "gaogaotiantian (via GitHub)" <gi...@apache.org>.
gaogaotiantian commented on code in PR #56461:
URL: https://github.com/apache/spark/pull/56461#discussion_r3405978250


##########
.github/workflows/build_non_ansi.yml:
##########
@@ -33,17 +31,17 @@ jobs:
     if: github.repository == 'apache/spark'
     with:
       java: 17
-      branch: master
       hadoop: hadoop3
       envs: >-
         {
           "PYSPARK_IMAGE_TO_TEST": "python-311",
           "PYTHON_TO_TEST": "python3.11",
-          "SPARK_ANSI_SQL_MODE": "false",

Review Comment:
   I believe the trailing comma is actually wrong. This is a JSON string and JSON does not allow trailing comma. `fromJSON` would crash with it. It was not caught because this file was never used.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-57398][INFRA][4.0] Make branch-4.0 scheduled build workflows self-contained [spark]

Posted by "Yicong-Huang (via GitHub)" <gi...@apache.org>.
Yicong-Huang commented on code in PR #56461:
URL: https://github.com/apache/spark/pull/56461#discussion_r3405884332


##########
.github/workflows/build_non_ansi.yml:
##########
@@ -33,17 +31,17 @@ jobs:
     if: github.repository == 'apache/spark'
     with:
       java: 17
-      branch: master
       hadoop: hadoop3
       envs: >-
         {
           "PYSPARK_IMAGE_TO_TEST": "python-311",
           "PYTHON_TO_TEST": "python3.11",
-          "SPARK_ANSI_SQL_MODE": "false",

Review Comment:
   nit: the trailing comma better to be kept, so that new env can introduce clean line change?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-57398][INFRA][4.0] Make branch-4.0 scheduled build workflows self-contained [spark]

Posted by "Yicong-Huang (via GitHub)" <gi...@apache.org>.
Yicong-Huang commented on code in PR #56461:
URL: https://github.com/apache/spark/pull/56461#discussion_r3406027817


##########
.github/workflows/build_non_ansi.yml:
##########
@@ -33,17 +31,17 @@ jobs:
     if: github.repository == 'apache/spark'
     with:
       java: 17
-      branch: master
       hadoop: hadoop3
       envs: >-
         {
           "PYSPARK_IMAGE_TO_TEST": "python-311",
           "PYTHON_TO_TEST": "python3.11",
-          "SPARK_ANSI_SQL_MODE": "false",

Review Comment:
   Ah, that's too bad



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org