You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by gi...@apache.org on 2026/04/15 15:40:28 UTC
(datafusion) 01/01: Add release management page to the documentation (#21001)
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch gh-readonly-queue/main/pr-21001-edf8ad3930d57b435d6a2acbe7d39be918232148
in repository https://gitbox.apache.org/repos/asf/datafusion.git
commit f99ba69c618947a21cd0b9af282cbddda20394c4
Author: Andrew Lamb <an...@nerdnetworks.org>
AuthorDate: Wed Apr 15 11:40:05 2026 -0400
Add release management page to the documentation (#21001)
## Which issue does this PR close?
- Related to #19692
## Rationale for this change
As we see more people helping with releases (thank you @timsaucer and
@comphead ❤️ ) I think having the process documented more will help
Also, I spent quite a while back porting PRs and wanted to document it
better
## What changes are included in this PR?
1. Add a release management page
2. Leave links to relevant sections
## Are these changes tested?
By CI
## Are there any user-facing changes?
Docs only
---
dev/release/README.md | 166 ++++++++-------------
docs/source/contributor-guide/index.md | 5 +
.../source/contributor-guide/release_management.md | 123 +++++++++++++++
docs/source/index.rst | 1 +
4 files changed, 195 insertions(+), 100 deletions(-)
diff --git a/dev/release/README.md b/dev/release/README.md
index 533bfc20ee..2f99d52b20 100644
--- a/dev/release/README.md
+++ b/dev/release/README.md
@@ -19,59 +19,11 @@
# Release Process
-DataFusion typically has major releases around once per month, including breaking API changes.
+For contributor-facing guidance on release branches and backports, see the
+[Contributor Guide Release Management page](../../docs/source/contributor-guide/release_management.md).
-Patch releases are made on an adhoc basis, but we try and avoid them given the frequent major releases.
-
-## Release Process Overview
-
-New development happens on the `main` branch.
-Releases are made from branches, e.g. `branch-50` for the `50.x.y` release series.
-
-To prepare for a new release series, we:
-
-- Create a new branch from `main`, such as `branch-50` in the Apache repository (not in a fork)
-- Continue merging new features changes to `main` branch
-- Prepare the release branch for release:
- - Update version numbers in `Cargo.toml` files and create `CHANGELOG.md`
- - Add additional changes to the release branch as needed
-- When the code is ready, create GitHub tags release candidate (rc) artifacts from the release branch.
-- After the release is approved, publish to [crates.io], the ASF distribution servers, and GitHub tags.
-
-To add changes to the release branch, depending on the change we either:
-
-- Fix the issue on `main` and then backport the change to the release branch (e.g. [#18129])
-- Fix the issue on the release branch and then forward-port the change back to `main` (e.g.[#18057])
-
-[crates.io]: https://crates.io/crates/datafusion
-[#18129]: https://github.com/apache/datafusion/pull/18129
-[#18057]: https://github.com/apache/datafusion/pull/18057
-
-## Backporting (add changes) to `branch-*` branch
-
-If you would like to propose your change for inclusion in a patch release, the
-change must be applied to the relevant release branch. To do so please follow
-these steps:
-
-1. Find (or create) the issue for the incremental release ([example release issue]) and discuss the proposed change there with the maintainers.
-2. Follow normal workflow to create PR to `main` branch and wait for its approval and merge.
-3. After PR is squash merged to `main`, branch from most recent release branch (e.g. `branch-50`), cherry-pick the commit and create a PR targeting the release branch [example backport PR].
-
-For example, to backport commit `12345` from `main` to `branch-50`:
-
-```shell
-git checkout branch-50
-git checkout -b backport_to_50
-git cherry-pick 12345 # your git commit hash
-git push -u <your fork>
-# make a PR as normal targeting branch-50, prefixed with [branch-50]
-```
-
-It is also acceptable to fix the issue directly on the release branch first
-and then cherry-pick the change back to `main` branch in a new PR.
-
-[example release issue]: https://github.com/apache/datafusion/issues/18072
-[example backport pr]: https://github.com/apache/datafusion/pull/18131
+This guide is for maintainers to create release candidates and run the release
+process.
## Release Prerequisites
@@ -160,33 +112,35 @@ branch-50:
First, prepare a PR to update the changelog and versions to reflect the planned
release. See [#18173](https://github.com/apache/datafusion/pull/18173) for an example.
-#### Update Version Numbers
-
-Manually update the DataFusion version in the root `Cargo.toml` to reflect the new release version.
+- This PR should be the last one merged before creating a release
+ candidate, so the changelog includes all changes.
+- If there are code changes between RCs, create and merge a new PR to update
+ the changelog before creating the next RC.
-Ensure Cargo.lock is updated accordingly by running:
+**Step 1**: Manually update the DataFusion version in the root `Cargo.toml` to
+reflect the new release version. Ensure `Cargo.lock` is updated accordingly by
+running:
```shell
cargo check -p datafusion
```
-#### Changelog Generation
-
-We maintain a [changelog] so our users know what has been changed between releases.
-
-[changelog]: ../changelog
-
-The changelog is generated using a Python script.
+**Step 2**: Update the [changelog] in `dev/changelog/`. Each release has its
+own file, such as `dev/changelog/50.0.0.md`, which should include all changes
+since the previous release.
-To run the script, you will need a GitHub Personal Access Token (described in the prerequisites section) and the `PyGitHub` library. First install the dev dependencies via `uv`:
+The changelog is generated using a Python script, which requires a GitHub
+Personal Access Token (described in the prerequisites section) and the
+`PyGitHub` library. First install the dev dependencies via `uv`:
```shell
uv sync
```
-To generate the changelog, set the `GITHUB_TOKEN` environment variable and then run `./dev/release/generate-changelog.py`
-providing two commit ids or tags followed by the version number of the release being created. For example,
-to generate a change log of all changes between the `50.3.0` tag and `branch-51`, in preparation for release `51.0.0`:
+To generate the changelog, set the `GITHUB_TOKEN` environment variable and run
+`./dev/release/generate-changelog.py` with two commit IDs or tags followed by
+the release version. For example, to generate a changelog of all changes
+between the `50.3.0` tag and `branch-51` for release `51.0.0`:
> [!NOTE]
>
@@ -205,52 +159,52 @@ uv run ./dev/release/generate-changelog.py 50.3.0 branch-51 51.0.0 > dev/changel
This script creates a changelog from GitHub PRs based on the labels associated with them as well as looking for
titles starting with `feat:`, `fix:`, or `docs:`.
-Once the change log is generated, run `prettier` to format the document:
+Once the changelog is generated, run `prettier` to format the document:
```shell
prettier -w dev/changelog/51.0.0.md
```
-#### Commit and PR
-
Then commit the changes and create a PR targeting the release branch.
```shell
git commit -a -m 'Update version'
```
-Remember to merge any fixes back to `main` branch as well.
+Remember to merge any fixes back to the `main` branch as well.
### 4. Prepare Release Candidate Artifacts
After the PR gets merged, you are ready to create release artifacts based off the
merged commit.
-(Note you need to be a committer to run these scripts as they upload to the apache svn distribution servers)
+- You must be a committer to run these scripts because they upload to the
+ Apache SVN distribution servers.
+- If there are code changes between RCs, create and merge a changelog PR before
+ creating the next RC.
#### Pick a Release Candidate (RC) number
Pick numbers in sequential order, with `1` for `rc1`, `2` for `rc2`, etc.
-#### Create git Tag for the Release:
+#### Create Git tag for the Release Candidate
While the official release artifacts are signed tarballs and zip files, we also
-tag the commit it was created for convenience and code archaeology. Release tags
-have the format `<version>` (e.g. `38.0.0`), and release candidates have the
-format `<version>-rc<rc>` (e.g. `38.0.0-rc0`). See [the list of existing
+tag the commit it was created from for convenience and code archaeology. Release tags
+look like `38.0.0`, and release candidate tags look like `38.0.0-rc0`. See [the list of existing
tags].
[the list of existing tags]: https://github.com/apache/datafusion/tags
-Using a string such as `38.0.0` as the `<version>`, create and push the rc tag by running these commands:
+Create and push the RC tag:
```shell
git fetch apache
-git tag <version>-<rc> apache/branch-X # create tag from the release branch
-git push apache <version>-<rc> # push tag to Github remote
+git tag <version>-rc<rc> apache/branch-X
+git push apache <version>-rc<rc>
```
-For example, to create the `50.3.0-rc1 tag from `branch-50`:
+For example, to create the `50.3.0-rc1` tag from `branch-50`, use:
```shell
git fetch apache
@@ -260,7 +214,7 @@ git push apache 50.3.0-rc1
#### Create, Sign, and Upload Artifacts
-Run the `create-tarball.sh` script with the `<version>` tag and `<rc>` and you determined in previous steps:
+Run the `create-tarball.sh` script with the `<version>` tag and `<rc>` number you determined in previous steps:
For example, to create the `50.3.0-rc1` artifacts:
@@ -275,18 +229,20 @@ The `create-tarball.sh` script
apache distribution SVN server
2. Provides you an email template to
- send to dev@datafusion.apache.org for release voting.
+ send to `dev@datafusion.apache.org` for release voting.
### 5. Vote on Release Candidate Artifacts
-Send the email output from the script to dev@datafusion.apache.org.
+Send the email output from the script to `dev@datafusion.apache.org`.
-In order to publish the release on crates.io, it must be "official". To become
-official it needs at least three PMC members to vote +1 on it.
+In order to publish the release on crates.io, it must be "official." To become
+official, it needs at least three PMC members to vote +1 on it and no -1 votes.
+The vote must remain open for at least 72 hours to give everyone a chance to
+review the release candidate.
#### Verifying Release Candidates
-The `dev/release/verify-release-candidate.sh` is a script in this repository that can assist in the verification process. Run it like:
+`dev/release/verify-release-candidate.sh` is a script in this repository that can assist in the verification process. Run it like this:
```shell
./dev/release/verify-release-candidate.sh 50.3.0 1
@@ -294,15 +250,18 @@ The `dev/release/verify-release-candidate.sh` is a script in this repository tha
#### If the Release is not Approved
-If the release is not approved, fix whatever the problem is, merge changelog
-changes into the release branch and try again with the next RC number.
+If the release is not approved, fix whatever the problem is, make a new release
+candidate and try again.
+
+Reminders:
-Remember to merge any fixes back to `main` branch as well.
+1. Update the changelog and create a new PR if there are any code changes, and merge it before creating the next RC.
+2. Merge any fixes back to the `main` branch as well.
-#### If the Release is Approved: Call the Vote
+#### If the Vote Passes: Announce the Result
Call the vote on the Arrow dev list by replying to the RC voting thread. The
-reply should have a new subject constructed by adding `[RESULT]` prefix to the
+reply should have a new subject constructed by adding the `[RESULT]` prefix to the
old subject line.
Sample announcement template:
@@ -415,39 +374,46 @@ the examples from previous releases.
The release information is used to generate a template for a board report (see example from Apache Arrow project
[here](https://github.com/apache/arrow/pull/14357)).
-### 10: Delete old RCs and Releases
+### 10: Delete Old RCs and Releases
See the ASF documentation on [when to archive](https://www.apache.org/legal/release-policy.html#when-to-archive)
for more information.
-#### Deleting old release candidates from `dev` svn
-
Release candidates should be deleted once the release is published.
-Get a list of DataFusion release candidates:
+To get a list of DataFusion release candidates:
```shell
svn ls https://dist.apache.org/repos/dist/dev/datafusion
```
-Delete a release candidate:
+To delete a release candidate:
```shell
svn delete -m "delete old DataFusion RC" https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-50.0.0-rc1/
```
-#### Deleting old releases from `release` svn
+#### Delete old releases from `release` SVN
-Only the latest release should be available. Delete old releases after publishing the new release.
+Only the latest release should be available. Delete old releases after
+publishing the new release.
-Get a list of DataFusion releases:
+To get a list of DataFusion releases:
```shell
svn ls https://dist.apache.org/repos/dist/release/datafusion
```
-Delete a release:
+To delete a release:
```shell
svn delete -m "delete old DataFusion release" https://dist.apache.org/repos/dist/release/datafusion/datafusion-50.0.0
```
+
+### 11. Forward-port changelog to `main`
+
+After the release is published, forward port the version update and changelog to
+`main` so that it is up to date for the next release. See [#21053] for an
+example PR that forward-ports the changelog to `main`.
+
+[#21053]: https://github.com/apache/datafusion/pull/21053
diff --git a/docs/source/contributor-guide/index.md b/docs/source/contributor-guide/index.md
index 4ace4be494..6ec1efa4d9 100644
--- a/docs/source/contributor-guide/index.md
+++ b/docs/source/contributor-guide/index.md
@@ -101,6 +101,11 @@ If you are concerned that a larger design will be lost in a string of small PRs,
Note all commits in a PR are squashed when merged to the `main` branch so there is one commit per PR after merge.
+## Release Management and Backports
+
+Contributor-facing guidance for release branches, patch releases, and backports
+is documented in the [Release Management](release_management.md) guide.
+
## Before Submitting a PR
Before submitting a PR, run the standard non-functional checks. PRs must pass
diff --git a/docs/source/contributor-guide/release_management.md b/docs/source/contributor-guide/release_management.md
new file mode 100644
index 0000000000..0515204a5e
--- /dev/null
+++ b/docs/source/contributor-guide/release_management.md
@@ -0,0 +1,123 @@
+<!---
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# Release Management
+
+This page describes DataFusion release branches and backports. For the
+maintainer release guide, including release candidate artifacts, voting, and
+publication, see [the release process README in `dev/release`].
+
+## Overview
+
+DataFusion typically has a major release about once per month, including
+breaking API changes. Patch releases are made on an ad hoc basis, but we try to
+avoid them because major releases are frequent.
+
+New development happens on the [`main` branch]. Releases are made from release
+branches named `branch-NN`, such as [`branch-50`] for the `50.x.y` release
+series.
+
+In general:
+
+- New features land on [`main`]
+- Patch releases are cut from the corresponding `branch-NN`
+- Only targeted, low-risk fixes should be added to a release branch
+
+Changes reach a release branch in one of two ways:
+
+- (Most common) Fix the issue on `main` and then backport the merged change to the release branch
+- Fix the issue on the release branch and then forward-port the change to `main`
+
+Releases are coordinated in a GitHub issue, such as the
+[release issue for 50.3.0]. If you think a fix should be included in a patch
+release, discuss it on the relevant tracking issue first. You can also open the
+backport PR first and then link it from the tracking issue.
+
+To prepare for a new release series, maintainers:
+
+- Create a new branch from `main`, such as `branch-50`, in the Apache repository
+- Continue merging new features to `main`
+- Prepare the release branch for release by updating versions, changelog content,
+ and any additional release-specific fixes via the
+ [Backport Workflow](#backport-workflow)
+- Create release candidate artifacts from the release branch
+- After approval, publish to crates.io, ASF distribution servers, and Git tags
+
+## Backport Workflow
+
+The usual workflow is:
+
+1. Fix on `main` first, and merge the fix via a normal PR workflow.
+2. Cherry-pick the merged commit onto the release branch.
+3. Open a backport PR targeting the release branch (examples below).
+
+- [Example backport PR]
+- [Additional backport PR example]
+
+### Inputs
+
+To backport a change, gather the following information:
+
+- Target branch, such as `apache/branch-52`
+- The release tracking issue URL, such as https://github.com/apache/datafusion/issues/19692
+- The original PR URL, such as https://github.com/apache/datafusion/pull/20192
+- Optional explicit commit SHA to backport
+
+### Apply the Backport
+
+Start from the target release branch, create a dedicated backport branch, and
+use `git cherry-pick`. For example, to backport PR #1234 to `branch-52` when
+the commit SHA is `abc123`, run:
+
+```bash
+git checkout apache/branch-52
+git checkout -b alamb/backport_1234
+git cherry-pick abc123
+```
+
+### Test
+
+Run tests as described in the [testing documentation].
+
+### Open the PR
+
+Create a PR against the release branch, not `main`, and prefix it with
+`[branch-NN]` to show which release branch the backport targets. For example:
+
+- `[branch-52] fix: validate inter-file ordering in eq_properties() (#20329)`
+
+Use a PR description that links the tracking issue, original PR, and target
+branch, for example:
+
+```markdown
+- Part of <tracking-issue-url>
+- Closes <backport-issue-url> on <branch-name>
+
+This PR:
+
+- Backports <original-pr-url> from @<author> to the <branch-name> line
+```
+
+[`main` branch]: https://github.com/apache/datafusion/tree/main
+[`branch-50`]: https://github.com/apache/datafusion/tree/branch-50
+[the release process readme in `dev/release`]: https://github.com/apache/datafusion/blob/main/dev/release/README.md
+[release issue for 50.3.0]: https://github.com/apache/datafusion/issues/18072
+[example backport pr]: https://github.com/apache/datafusion/pull/18131
+[additional backport pr example]: https://github.com/apache/datafusion/pull/20792
+[testing documentation]: testing.md
diff --git a/docs/source/index.rst b/docs/source/index.rst
index 4d57faa0cb..cc0da3c444 100644
--- a/docs/source/index.rst
+++ b/docs/source/index.rst
@@ -163,6 +163,7 @@ To get started, see
contributor-guide/testing
contributor-guide/api-health
contributor-guide/howtos
+ contributor-guide/release_management
contributor-guide/roadmap
contributor-guide/governance
contributor-guide/inviting
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org