You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by we...@apache.org on 2026/04/17 02:12:18 UTC
(spark) branch master updated: [SPARK-56506][INFRA] Use PR author's GitHub identity as the commit author in merge script
This is an automated email from the ASF dual-hosted git repository.
wenchen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 0a7483ec4069 [SPARK-56506][INFRA] Use PR author's GitHub identity as the commit author in merge script
0a7483ec4069 is described below
commit 0a7483ec40695040c4311c782099115a6e24f290
Author: Wenchen Fan <we...@databricks.com>
AuthorDate: Fri Apr 17 10:12:02 2026 +0800
[SPARK-56506][INFRA] Use PR author's GitHub identity as the commit author in merge script
### What changes were proposed in this pull request?
This PR changes the merge script (`dev/merge_spark_pr.py`) to use the PR author's GitHub profile (name + email) as the commit author, instead of extracting the author from the Git commit history.
Previously, the script used `git log --pretty=format:%an <%ae>` to pick the primary author from the PR's commits. This uses whatever name/email the contributor configured in their local `git config`, which may not be linked to their GitHub account. As a result, GitHub cannot map the merge commit back to the PR author's GitHub profile — the `author` field in the commit API returns `null`.
Now the script fetches the PR author's GitHub profile via the `/users/{login}` API and uses their profile name and email. If the author has no public email, it falls back to GitHub's noreply email (`{id}+{login}users.noreply.github.com`). Other commit authors (from multi-author PRs) are preserved as co-authors, with the PR author filtered out to avoid duplication.
### Why are the changes needed?
When the merge commit's author email isn't linked to a GitHub account, GitHub's API returns `null` for `author.login` on the commit. This breaks downstream tooling that relies on the commit author's GitHub identity (e.g., identifying whether a commit author belongs to a specific organization). Using the PR author's GitHub profile ensures the merge commit is always properly attributed and linkable on GitHub.
### Does this PR introduce _any_ user-facing change?
No. This only affects the internal merge script used by committers.
### How was this patch tested?
Manual review of the script changes. The interactive prompt still allows committers to override the default author if needed.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code
Closes #55366 from cloud-fan/merge-script-pr-author.
Authored-by: Wenchen Fan <we...@databricks.com>
Signed-off-by: Wenchen Fan <we...@databricks.com>
---
AGENTS.md | 23 ++++++++++++++++---
dev/merge_spark_pr.py | 62 ++++++++++++++++++++++++++++++++++-----------------
2 files changed, 62 insertions(+), 23 deletions(-)
diff --git a/AGENTS.md b/AGENTS.md
index 4485e7f7ce62..fedc4be1ace7 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -92,15 +92,32 @@ Each annotation contains the test class, test name, and failure message.
## Pull Request Workflow
-PR title format is `[SPARK-xxxx][Component] Title`. Infer the PR title from the changes. If no ticket ID is given, create one using `dev/create_spark_jira.py`, using the PR title (without the JIRA ID and component tag) as the ticket title.
+PR title format is `[SPARK-xxxx][COMPONENT] Title`. The component tag is derived from the JIRA component name: take the last word and uppercase it (e.g. `Project Infra` → `[INFRA]`, `Spark Core` → `[CORE]`, `Structured Streaming` → `[STREAMING]`, `SQL` → `[SQL]`).
+
+Infer the PR title from the changes. If no ticket ID is given, create one using `dev/create_spark_jira.py`, using the PR title (without the JIRA ID and component tag) as the ticket title.
python3 dev/create_spark_jira.py "<title>" -c <component> { -t <type> | -p <parent-jira-id> }
-- **Component** (`-c`): e.g. "SQL", "Spark Core", "PySpark", "Connect". Run `python3 dev/create_spark_jira.py --list-components` for the full list.
+- **Component** (`-c`): the exact JIRA component name (not the PR title shorthand), e.g. "SQL", "Spark Core", "PySpark", "Connect". Run `python3 dev/create_spark_jira.py --list-components` for the full list.
- **Issue type** (`-t`): "Bug", "Improvement", "New Feature", "Test", "Documentation", or "Dependency upgrade".
- **Parent** (`-p`): if the user mentions a parent JIRA ticket (e.g., "this is a subtask of SPARK-12345"), pass it instead of `-t`. The issue type is automatically "Sub-task".
-The script sets the latest unreleased version as the default affected version. Ask the user to review and adjust versions and other fields on the JIRA ticket after creation.
+The script sets the latest unreleased version as the default affected version.
+
+After creating a JIRA ticket, print a prominent notice so the user does not miss it:
+
+ ============================================================
+ JIRA ticket created: SPARK-XXXXX
+ https://issues.apache.org/jira/browse/SPARK-XXXXX
+
+ Title: <title>
+ Component(s): <component>
+ Issue type: <type>
+ Affected version(s): <version>
+ Priority: <priority>
+
+ Please review and adjust these fields if needed.
+ ============================================================
Before writing the PR description, read `.github/PULL_REQUEST_TEMPLATE` and fill in every section from that file.
diff --git a/dev/merge_spark_pr.py b/dev/merge_spark_pr.py
index ff7a95fbaf53..fbd64e816179 100755
--- a/dev/merge_spark_pr.py
+++ b/dev/merge_spark_pr.py
@@ -142,7 +142,7 @@ def clean_up():
# merge the requested PR and return the merge hash
-def merge_pr(pr_num, target_ref, title, body, pr_repo_desc):
+def merge_pr(pr_num, target_ref, title, body, pr_repo_desc, pr_author, co_authors):
pr_branch_name = "%s_MERGE_PR_%s" % (BRANCH_PREFIX, pr_num)
target_branch_name = "%s_MERGE_PR_%s_%s" % (BRANCH_PREFIX, pr_num, target_ref.upper())
run_cmd("git fetch %s pull/%s/head:%s" % (PR_REMOTE_NAME, pr_num, pr_branch_name))
@@ -159,23 +159,12 @@ def merge_pr(pr_num, target_ref, title, body, pr_repo_desc):
continue_maybe(msg)
had_conflicts = True
- # First commit author should be considered as the primary author when the rank is the same
- commit_authors = run_cmd(
- ["git", "log", "HEAD..%s" % pr_branch_name, "--pretty=format:%an <%ae>", "--reverse"]
- ).split("\n")
- distinct_authors = sorted(
- list(dict.fromkeys(commit_authors)), key=lambda x: commit_authors.count(x), reverse=True
- )
+ # Use PR author as the primary author for proper GitHub attribution
primary_author = bold_input(
- 'Enter primary author in the format of "name <email>" [%s]: ' % distinct_authors[0]
+ 'Enter primary author in the format of "name <email>" [%s]: ' % pr_author
)
if primary_author == "":
- primary_author = distinct_authors[0]
- else:
- # When primary author is specified manually, de-dup it from author list and
- # put it at the head of author list.
- distinct_authors = list(filter(lambda x: x != primary_author, distinct_authors))
- distinct_authors.insert(0, primary_author)
+ primary_author = pr_author
merge_message_flags = []
@@ -198,10 +187,10 @@ def merge_pr(pr_num, target_ref, title, body, pr_repo_desc):
# The string "Closes #%s" string is required for GitHub to correctly close the PR
merge_message_flags += ["-m", "Closes #%s from %s." % (pr_num, pr_repo_desc)]
- authors = "Authored-by:" if len(distinct_authors) == 1 else "Lead-authored-by:"
- authors += " %s" % (distinct_authors.pop(0))
- if len(distinct_authors) > 0:
- authors += "\n" + "\n".join(["Co-authored-by: %s" % a for a in distinct_authors])
+ authors = "Authored-by:" if len(co_authors) == 0 else "Lead-authored-by:"
+ authors += " %s" % primary_author
+ if len(co_authors) > 0:
+ authors += "\n" + "\n".join(["Co-authored-by: %s" % a for a in co_authors])
authors += "\n" + "Signed-off-by: %s <%s>" % (committer_name, committer_email)
merge_message_flags += ["-m", authors]
@@ -668,6 +657,39 @@ def main():
base_ref = pr["head"]["ref"]
pr_repo_desc = "%s/%s" % (user_login, base_ref)
+ # Fetch PR author's GitHub profile and PR commits for commit attribution
+ pr_author_info = get_json("https://api.github.com/users/%s" % user_login)
+ pr_author_name = pr_author_info.get("name") or user_login
+ pr_author_email = pr_author_info.get("email")
+ pr_commits = get_json("%s/pulls/%s/commits" % (GITHUB_API_BASE, pr_num))
+ if not pr_author_email:
+ # If the GitHub profile has no public email, find the author's email from the
+ # PR's git commits. Only use it if GitHub links the commit to the PR author's
+ # account (meaning the email is verified on their GitHub account).
+ for c in pr_commits:
+ commit_author = c.get("author")
+ if commit_author and commit_author.get("login") == user_login:
+ pr_author_email = c["commit"]["author"]["email"]
+ break
+ if not pr_author_email:
+ pr_author_email = "%s+%s@users.noreply.github.com" % (
+ pr_author_info["id"], user_login
+ )
+ pr_author = "%s <%s>" % (pr_author_name, pr_author_email)
+
+ # Collect co-authors: commit authors whose GitHub login differs from the PR author.
+ # This deduplicates the PR author (who may use multiple emails across commits) by login.
+ co_authors = []
+ seen = set()
+ for c in pr_commits:
+ gh_author = c.get("author")
+ if gh_author and gh_author.get("login") == user_login:
+ continue
+ raw = "%s <%s>" % (c["commit"]["author"]["name"], c["commit"]["author"]["email"])
+ if raw not in seen:
+ seen.add(raw)
+ co_authors.append(raw)
+
# Merged pull requests don't appear as merged in the GitHub API;
# Instead, they're closed by committers.
merge_commits = [e for e in pr_events if e["event"] == "closed" and e["commit_id"] is not None]
@@ -714,7 +736,7 @@ def main():
merged_refs = [target_ref]
- merge_hash = merge_pr(pr_num, target_ref, title, body, pr_repo_desc)
+ merge_hash = merge_pr(pr_num, target_ref, title, body, pr_repo_desc, pr_author, co_authors)
pick_prompt = "Would you like to pick %s into another branch?" % merge_hash
while bold_input("\n%s (y/N): " % pick_prompt).lower() == "y":
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org