You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2026/08/20 12:19:58 UTC

[PR] [SPARK-58905][INFRA] Allow merge script to update resolved JIRA fix versions [spark]

zhengruifeng opened a new pull request, #58165:
URL: https://github.com/apache/spark/pull/58165

   ### What changes were proposed in this pull request?
   
   This PR allows `dev/merge_spark_pr.py` to update Fix Versions when the linked JIRA
   issue is already Resolved or Closed during the normal merge cleanup path.
   
   The script already supports adding additional Fix Versions to a resolved issue; this
   change passes `allow_resolved=True` from the regular post-merge JIRA update path, matching
   the existing backport cleanup path.
   
   ### Why are the changes needed?
   
   When merging a PR whose JIRA issue was already resolved, the merge script printed that
   the issue already had status `Resolved` and returned before prompting for Fix Versions.
   This prevented committers from correcting or adding the Fix Version during merge.
   
   ### Does this PR introduce _any_ user-facing change?
   
   No.
   
   ### How was this patch tested?
   
   Manually verified with:
   
   ```
   git diff --check
   conda run -n spark-dev-313 python -m py_compile dev/merge_spark_pr.py
   ```
   
   ### Was this patch authored or co-authored using generative AI tooling?
   
   Generated-by: OpenAI Codex GPT-5
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-58905][INFRA] Allow merge script to update resolved JIRA fix versions [spark]

Posted by "szehon-ho (via GitHub)" <gi...@apache.org>.
szehon-ho commented on PR #58165:
URL: https://github.com/apache/spark/pull/58165#issuecomment-5375701550

   Thanks for the writeup on the three flows -- that clears up my scope question, and you're right that requiring empty Fix Versions breaks your cases 2 and 3, so drop that half of my earlier suggestion. Two suggestions below that I think work best together: the first covers the cases where only the committer can judge, the second removes the cases where there's nothing to judge. With both in place `allow_resolved` can be deleted from all four functions, since both merge paths would then get the same treatment.
   
   Assume JIRA has `5.0.0` (master) and `4.3.0` (branch-4.x) unreleased, and the PR is merged to master only unless noted.
   
   - **Add a new explicit prompt before any Fix Version is added.** Prompt once at line 1179-1189, where `default_fix_list` is already narrowed to what would actually be added, so the prompt can show the decision instead of asking blind. Merging a PR whose ticket is Resolved with `['4.3.0']`:
   
     ```
     Does SPARK-58880 match this PR? (y/N):
   
     Proceed with merging pull request #58150? (y/N):
   
     Would you like to pick abc1234 into another branch? (y/N):
   
     Would you like to update an associated JIRA? (y/N):
   
     Enter a JIRA id [SPARK-58880]:
   
     JIRA issue SPARK-58880 already has status 'Resolved'
   
     Check if the JIRA information is as expected (y/N):              <- newly reached by this PR
   
     JIRA issue SPARK-58880 has fix version(s) ['4.3.0']; inferred addition(s): ['5.0.0']
     Add these fix version(s)? (y/N):                                <- PROPOSED
   
     Enter comma-separated additional fix version(s) [5.0.0]:        <- newly reached by this PR
     ```
   
     The added prompt gates the existing one rather than replacing it, so the committer can still edit the list after answering `y`:
   
     ```python
     if is_resolved:
         default_fix_list = additional_fix_versions(default_fix_list, existing_fix_version_names)
         if not default_fix_list:
             print("JIRA issue %s already contains all inferred fix versions; no update needed." % issue.key)
             return
         print(
             "JIRA issue %s has fix version(s) %s; inferred addition(s): %s"
             % (issue.key, existing_fix_version_names, default_fix_list)
         )
         if get_input("Add these fix version(s)? (y/N): ", ["y", "n", ""]) != "y":
             return
     ```
   
     Nothing is added unless the committer says yes, and the ticket's current Fix Versions are printed right next to the proposed ones, so the answer is visible at the moment it's asked. This is also what covers the `[FOLLOWUP]` case @Yicong-Huang raised, and revert PRs: both leave the ticket genuinely `Fixed`, so no automatic rule can tell them apart from an interrupted merge -- only the committer can.
   
   - **Gate on `resolution == "Fixed"`.** Skip the resolved ticket entirely, keeping today's behavior of printing the status and moving on, unless its resolution is `Fixed`. That excludes Duplicate, Won't Fix, Invalid, Incomplete, and Cannot Reproduce, while letting every ticket from cases 1-3 through, since your resolve path sets resolution `Fixed` at line 1241. Nothing currently surfaces the resolution either -- `print_jira_issue_summary` (line 1015) and `format_jira_verification` (line 920) both print status only -- so a committer sees "Closed" with no hint that it was closed as a duplicate. The check belongs as its own early return next to the existing status check at line 1122-1125, not folded into `is_resolved` at line 1152-1153, since that flag also controls skipping the assignee and component steps and would send a Duplicate ticket down the resolve path and try to transition it to Resolved:
   
     ```python
     status = issue.fields.status.name
     if status == "Resolved" or status == "Closed":
         resolution = issue.fields.resolution
         resolution_name = resolution.name if resolution is not None else None
         print("JIRA issue %s already has status '%s' (%s)" % (jira_id, status, resolution_name))
         # Only a ticket an earlier merge resolved as Fixed can legitimately gain another Fix
         # Version. Duplicate / Won't Fix / Invalid tickets must not be touched.
         if resolution_name != "Fixed":
             return None
     ```
   
   | Ticket already Resolved/Closed because... | After this PR | With the gate | Desired |
   |---|---|---|---|
   | earlier interrupted merge, no Fix Versions yet | `additional fix version(s) [5.0.0]` | `additional fix version(s) [5.0.0]` | same, the gap being closed |
   | `[FOLLOWUP]`, ticket already has `5.0.0` | `no update needed` | `no update needed` | same |
   | `[FOLLOWUP]` merged to master only, ticket has `4.3.0` | `additional fix version(s) [5.0.0]` | `additional fix version(s) [5.0.0]` | no prompt |
   | the PR reverts the fix, ticket has `4.3.0` | `additional fix version(s) [5.0.0]` | `additional fix version(s) [5.0.0]` | no prompt |
   | closed as Duplicate / Won't Fix | `additional fix version(s) [5.0.0]` | no prompt | no prompt |
   | fix released long ago in `3.5.0` | `additional fix version(s) [5.0.0]` | `additional fix version(s) [5.0.0]` | no prompt |
   | case 3: separate `branch-4.x` PR, ticket has `5.0.0` | `additional fix version(s) [4.3.0]` | `additional fix version(s) [4.3.0]` | same, target case |
   | case 2: later backport-mode run, ticket has `5.0.0` | `[4.3.0]`, unchanged | unchanged | same |
   
   The gate on its own settles the Duplicate / Won't Fix row and costs nothing on cases 1-3, but the three `Fixed`-but-shouldn't rows still reach the prompt -- those are the ones the first suggestion catches. Conversely the gate keeps the first suggestion's prompt off merges where the answer is never yes.
   
   What do you think about these two? Happy to be talked out of either if I'm misreading how often the resolved-ticket cases come up in practice.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-58905][INFRA] Allow merge script to update resolved JIRA fix versions [spark]

Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng closed pull request #58165: [SPARK-58905][INFRA] Allow merge script to update resolved JIRA fix versions
URL: https://github.com/apache/spark/pull/58165


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-58905][INFRA] Allow merge script to update resolved JIRA fix versions [spark]

Posted by "szehon-ho (via GitHub)" <gi...@apache.org>.
szehon-ho commented on PR #58165:
URL: https://github.com/apache/spark/pull/58165#issuecomment-5401004221

   Thanks!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-58905][INFRA] Allow merge script to update resolved JIRA fix versions [spark]

Posted by "uros-b (via GitHub)" <gi...@apache.org>.
uros-b commented on PR #58165:
URL: https://github.com/apache/spark/pull/58165#issuecomment-5362584066

   Thank you @zhengruifeng!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-58905][INFRA] Allow merge script to update resolved JIRA fix versions [spark]

Posted by "szehon-ho (via GitHub)" <gi...@apache.org>.
szehon-ho commented on code in PR #58165:
URL: https://github.com/apache/spark/pull/58165#discussion_r3825782839


##########
dev/merge_spark_pr.py:
##########
@@ -1351,7 +1351,7 @@ def resolve_jira_issues(title, merge_branches, comment, title_components=(), all
         )
 
 
-def update_jira_for_pr(pr_num, title, merge_branches, title_components, allow_resolved=False):
+def update_jira_for_pr(pr_num, title, merge_branches, title_components, allow_resolved=True):

Review Comment:
   If both paths keep the same behavior, all callers now use the default, so `allow_resolved` can be dropped from these four functions. (If the scope question in my other comment leads to the two paths differing, it can stay.)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-58905][INFRA] Allow merge script to update resolved JIRA fix versions [spark]

Posted by "Yicong-Huang (via GitHub)" <gi...@apache.org>.
Yicong-Huang commented on PR #58165:
URL: https://github.com/apache/spark/pull/58165#issuecomment-5367533374

   I think we can let the committer decide whether to add a new version to fixVersion. We can also distinguish `[FOLLOWUP]` cases from merge-interrupted cases, since the latter will have a post-merge message on the PR.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-58905][INFRA] Allow merge script to update resolved JIRA fix versions [spark]

Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on PR #58165:
URL: https://github.com/apache/spark/pull/58165#issuecomment-5403474565

   **Merge Summary:**
   - merged into master https://github.com/apache/spark/commit/53c26da1cbfaf7133bb596146323c44bd49a0c0a
   
   *Posted by `merge_spark_pr.py`*


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-58905][INFRA] Allow merge script to update resolved JIRA fix versions [spark]

Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on PR #58165:
URL: https://github.com/apache/spark/pull/58165#issuecomment-5391023956

   Thanks, that makes sense. I updated the PR with both changes: resolved/closed JIRAs now continue only when the resolution is Fixed, and already-resolved Fixed tickets now show the existing Fix Versions plus inferred additions and require an explicit confirmation before the editable additional-Fix-Version prompt. I also removed the allow_resolved plumbing so normal merge and backport mode use the same path, and surfaced the resolution in the JIRA summary/verification output so cases like Closed (Duplicate) are visible. Verified with python3 -m py_compile dev/merge_spark_pr.py and git diff --check.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-58905][INFRA] Allow merge script to update resolved JIRA fix versions [spark]

Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on PR #58165:
URL: https://github.com/apache/spark/pull/58165#issuecomment-5363727688

   Thanks, this is a good point. I think there are three different backport flows here:
   
   1. Happy path: we merge a PR to master and backport it in the same
      `merge_spark_pr.py` invocation. In this case the script sees all merged refs
      together and can set all Fix Versions correctly.
   
   2. The master PR was merged first, and later we decide to backport the same
      merged PR to more branches using `merge_spark_pr.py` backport mode. In this
      case the JIRA is already resolved, but the later backport merge still needs
      to add the newly inferred Fix Versions. This is the case fixed by
      84a309d91f702b2434e8525ff36e77d88f6a61cc / SPARK-58754.
   
   3. The PR cannot be backported cleanly while merging to master because of code
      conflicts, so we merge the master PR first and then open a separate PR
      targeting the release branch with the conflict resolution. This is becoming
      more common as branches diverge; I hit this twice this week. In that case
      the branch PR is a normal open PR, not the "already merged PR backport mode"
      from case 2, but the linked JIRA is already resolved from the master merge
      and we still need to add the release-branch Fix Version while merging the
      branch PR. PR #58147 -> #58184 is one example. Another is #58129 -> #58150:
      the master-side PR was merged first, then a separate release-branch PR was
      needed because the backport could not be completed directly from the original
      merge flow.
   
   So I agree we should not blindly update every resolved/closed JIRA. But checking
   for empty Fix Versions is too narrow, because cases 2 and 3 both need to add
   additional Fix Versions to an already resolved issue that may already have the
   master Fix Version.
   
   One related issue is that branch-4.x-only merges currently do not infer the 4.x
   Fix Version unless master is also in `merge_branches`, so I may need to adjust
   the inference for separate release-branch PRs too.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org