[jira] [Resolved] (SPARK-58537) Scheduled Maven workflows on branch-4.x build master instead of the branch

["Jungtaek Lim (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-58537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jungtaek Lim resolved SPARK-58537.
----------------------------------
    Fix Version/s: 4.4.0
       Resolution: Fixed

Issue resolved by pull request 57739
[https://github.com/apache/spark/pull/57739]

> Scheduled Maven workflows on branch-4.x build master instead of the branch
> --------------------------------------------------------------------------
>
>                 Key: SPARK-58537
>                 URL: https://issues.apache.org/jira/browse/SPARK-58537
>             Project: Spark
>          Issue Type: Bug
>          Components: Project Infra
>    Affects Versions: 4.4.0
>            Reporter: Jungtaek Lim
>            Assignee: Jungtaek Lim
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 4.4.0
>
>
> The scheduled Maven workflows on {{branch-4.x}} are named for {{branch-4.x}} but actually build {{master}}, because they invoke the reusable {{maven_test.yml}} without passing the {{branch}} input, and that workflow's default is {{master}}.
> h3. The bug
> {{.github/workflows/build_maven.yml}} on {{branch-4.x}} is titled:
> {code}
> name: "Build / Maven (branch-4.x, Scala 2.13, Hadoop 3, JDK 17)"
> {code}
> but its job body supplies no inputs:
> {code}
> jobs:
>   run-build:
>     permissions:
>       packages: write
>     name: Run
>     uses: ./.github/workflows/maven_test.yml
>     if: github.repository == 'apache/spark'
> {code}
> So {{maven_test.yml}} falls back to its declared default:
> {code}
>       branch:
>         description: Branch to run the build against
>         required: false
>         type: string
>         default: master
> {code}
> and it then checks out {{ref: ${{ inputs.branch }}}} — i.e. {{master}}.
> h3. Affected workflows
> The five consumers of {{maven_test.yml}} on {{branch-4.x}} that pass no {{branch:}} input:
> * {{build_maven.yml}}
> * {{build_maven_java21.yml}}
> * {{build_maven_java25.yml}}
> * {{build_maven_java21_arm.yml}}
> * {{build_maven_java21_macos26.yml}}
> The last two are additionally titled {{"Build / Maven (master, ...)"}}, which looks like deliberate master-targeting but is a stale artifact of being copied from {{master}} -- the same two names are stale on {{branch-4.2}}, where the {{maven_test.yml}} default is correctly {{branch-4.2}}. Their titles should be corrected along with the default.
> Callers that target another branch pass {{branch:}} explicitly (e.g. {{build_branch41_maven.yml}} passes {{branch-4.1}}) and are unaffected.
> The SBT/Python workflows on the branch ({{build_java17/21/25}}, {{build_non_ansi}}, {{build_python_3.11/3.14}}) also omit {{branch:}}, but they use {{build_and_test.yml}}, whose default on this branch is already {{branch-4.x}}. They are correct and need no change.
> The fix is therefore to change the {{branch}} default in {{maven_test.yml}} on {{branch-4.x}} from {{master}} to {{branch-4.x}}, matching {{build_and_test.yml}} on the same branch and both reusable workflows on {{branch-4.2}} / {{branch-4.1}} (each defaulting to their own branch, which is why their callers correctly pass no input).
> The same bug exists on {{branch-4.3}}, which inherited these files when it was cut; it is fixed there as part of SPARK-58538. {{master}} is unaffected: {{default: master}} is correct there.
> h3. Evidence
> The most recent green "Build / Maven (branch-4.x, ...)" run (30840288268, 2026-08-03) produced {{master}}'s artifacts. Its Precompile log contains:
> {code}
> spark-connect-client-jdbc_2.13-5.0.0-SNAPSHOT.jar, unused-1.0.0.jar define 1 overlapping resource
> {code}
> At the time, {{branch-4.x}} was at {{4.3.0-SNAPSHOT}} and {{master}} at {{5.0.0-SNAPSHOT}}, so those coordinates show {{master}} had been checked out.
> For contrast, the "Build (branch-4.x, ...)" JDK 17 run (30818354111), which goes through {{build_and_test.yml}}, logs only {{4.3.0-SNAPSHOT}} — correctly building the branch.
> h3. Impact
> {{branch-4.x}} has had no scheduled Maven coverage, and the green checks from these three workflows are misleading.
> This masked SPARK-57897, where {{branch-4.x}} could not load its Maven project model at all — {{mvn}} failed with {{'dependencies.dependency.version' for io.grpc:grpc-api:jar is missing}} before any test could run — while the branch's Maven workflow kept reporting success. The breakage was found only by building the branch locally.
> Note the release branches are handled correctly elsewhere: the {{build_branch42_*.yml}} / {{build_branch4x_*.yml}} workflows on {{master}} do pass {{branch:}} explicitly. The bug is specific to the copies of these workflows that live on {{branch-4.x}} itself, and would be inherited by any newly cut branch that copies them (e.g. {{branch-4.3}}).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org