[arrow-ballista] branch main updated: implement new release process (#623)

[ag...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

agrove pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/arrow-ballista.git


The following commit(s) were added to refs/heads/main by this push:
     new a5f306e5 implement new release process (#623)
a5f306e5 is described below

commit a5f306e5315471100065ddf252b6f85d9835abbe
Author: Andy Grove <an...@gmail.com>
AuthorDate: Mon Jan 23 17:33:04 2023 -0700

    implement new release process (#623)
---
 dev/release/README.md | 112 ++++++++++++++++++--------------------------------
 1 file changed, 40 insertions(+), 72 deletions(-)

diff --git a/dev/release/README.md b/dev/release/README.md
index f7a9039f..2f5d02f1 100644
--- a/dev/release/README.md
+++ b/dev/release/README.md
@@ -19,23 +19,32 @@
 
 # Release Process
 
-## Branching
+Development happens on the `main` branch, and most of the time, we depend on DataFusion using GitHub dependencies
+rather than using an official release from crates.io. This allows us to pick up new features and bug fixes frequently
+by creating PRs to move to a later revision of the code. It also means we can incrementally make updates that are
+required due to changes in DataFusion rather than having a large amount of work to do when the next official release
+is available.
 
-### Major Release
+When there is a new official release of DataFusion, we update the `main` branch to point to that, update the version
+number, and create a new release branch, such as `branch-0.11`. Once this branch is created, we switch the `main` branch
+back to using GitHub dependencies. The release activity (such as generating the changelog) can then happen on the
+release branch without blocking ongoing development in the `main` branch.
 
-Ballista typically has major releases from the `main` branch every 4 weeks.
+We can cherry-pick commits from the `main` branch into `branch-0.11` as needed and then create new patch releases
+from that branch.
 
-## Prerequisite
+## Detailed Guide
 
+### Prerequisite
+
+- You will need a GitHub Personal Access Token with "repo" access. Follow
+  [these instructions](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token)
+  to generate one if you do not already have one.
 - Have upstream git repo `git@github.com:apache/arrow-ballista.git` add as git remote `apache`.
-- Created a peronal access token in Github for changelog automation script.
-  - Github PAT should be created with `repo` access
 - Make sure your signing key is added to the following files in SVN:
   - https://dist.apache.org/repos/dist/dev/arrow/KEYS
   - https://dist.apache.org/repos/dist/release/arrow/KEYS
 
-### How to add signing key
-
 See instructions at https://infra.apache.org/release-signing.html#generate for generating keys.
 
 Committers can add signing keys in Subversion client with their ASF account. e.g.:
@@ -54,64 +63,25 @@ Follow the instructions in the header of the KEYS file to append your key. Here
 svn commit KEYS -m "Add key for John Doe"
 ```
 
-## Process Overview
-
-As part of the Apache governance model, official releases consist of signed
-source tarballs approved by the PMC.
-
-We then use the code in the approved artifacts to release to crates.io and
-PyPI.
-
-### Change Log
-
-We maintain `CHANGELOG.md` for each sub project so our users know what has been
-changed between releases.
-
-The CHANGELOG is managed automatically using
-[update_change_log.sh](https://github.com/apache/arrow-ballista/blob/main/dev/release/update_change_log.sh)
-
-This script creates a changelog using github PRs and issues based on the labels
-associated with them.
-
-## Prepare release commits and PR
-
-Prepare a PR to update `CHANGELOG.md` and versions to reflect the planned
-release.
-
-See [#801](https://github.com/apache/arrow-ballista/pull/801) for an example.
+### Preparing the `main` Branch
 
-Here are the commands that could be used to prepare the `0.8.0` release:
+Before creating a new release:
 
-### Update Version
+- We need to ensure that the main branch does not have any GitHub dependencies
+- a PR should be created and merged to update the major version number of the project. There is a script to automate
+  updating the version number: `./dev/update_ballista_versions.py 0.11.0`
+- A new release branch should be created, such as `branch-0.11`
 
-Checkout the master commit to be released
+Once the release branch has been created, the `main` branch can immediately go back to depending on DataFusion with a
+GitHub dependency.
 
-```
-git fetch apache
-git checkout apache/main
-```
-
-Update version in `ballista/Cargo.toml` to `0.8.0`:
-
-```
-./dev/update_ballista_versions.py 0.8.0
-```
-
-Lastly commit the version change:
-
-```
-git commit -a -m 'Update version'
-```
+### Change Log
 
 ### Update CHANGELOG.md
 
-Define release branch (e.g. `main`), base version tag (e.g. `0.8.0`) and future version tag (e.g. `0.9.0`). Commits
+Define release branch (e.g. `branch-0.11`), base version tag (e.g. `0.7.0`) and future version tag (e.g. `0.9.0`). Commits
 between the base version tag and the release branch will be used to populate the changelog content.
 
-You will need a GitHub Personal Access Token for the following steps. Follow
-[these instructions](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token)
-to generate one if you do not already have one.
-
 ```bash
 # create the changelog
 CHANGELOG_GITHUB_TOKEN=<TOKEN> ./dev/release/update_change_log-ballista.sh main 0.8.0 0.7.0
@@ -127,10 +97,9 @@ You can add `invalid` or `development-process` label to exclude items from
 release notes. Add `datafusion`, `ballista` and `python` labels to group items
 into each sub-project's change log.
 
-Send a PR to get these changes merged into `main` branch. If new commits that
-could change the change log content landed in the `main` branch before you
-could merge the PR, you need to rerun the changelog update script to regenerate
-the changelog and update the PR accordingly.
+Send a PR to get these changes merged into the release branch (e.g. `branch-0.11`). If new commits that could change the
+change log content landed in the release branch before you could merge the PR, you need to rerun the changelog update
+script to regenerate the changelog and update the PR accordingly.
 
 ## Prepare release candidate artifacts
 
@@ -148,11 +117,10 @@ Pick numbers in sequential order, with `0` for `rc0`, `1` for `rc1`, etc.
 While the official release artifacts are signed tarballs and zip files, we also
 tag the commit it was created for convenience and code archaeology.
 
-Using a string such as `0.8.0` as the `<version>`, create and push the tag by running these commands:
+Using a string such as `0.11.0` as the `<version>`, create and push the tag by running these commands:
 
 ```shell
-git fetch apache
-git tag <version>-<rc> apache/main
+git tag <version>-<rc>
 # push tag to Github remote
 git push apache <version>
 ```
@@ -162,7 +130,7 @@ git push apache <version>
 Run `create-tarball.sh` with the `<version>` tag and `<rc>` and you found in previous steps:
 
 ```shell
-GH_TOKEN=<TOKEN> ./dev/release/create-tarball.sh 0.8.0 0
+GH_TOKEN=<TOKEN> ./dev/release/create-tarball.sh 0.11.0 1
 ```
 
 The `create-tarball.sh` script
@@ -212,7 +180,7 @@ For the release to become "official" it needs at least three PMC members to vote
 The `dev/release/verify-release-candidate.sh` is a script in this repository that can assist in the verification process. Run it like:
 
 ```
-./dev/release/verify-release-candidate.sh 0.8.0 0
+./dev/release/verify-release-candidate.sh 0.11.0 0
 ```
 
 #### If the release is not approved
@@ -231,7 +199,7 @@ https://dist.apache.org/repos/dist/release/arrow/arrow-ballista-0.8.0/, using
 the `release-tarball.sh` script:
 
 ```shell
-./dev/release/release-tarball.sh 0.8.0 0
+./dev/release/release-tarball.sh 0.11.0 1
 ```
 
 Congratulations! The release is now official!
@@ -241,9 +209,9 @@ Congratulations! The release is now official!
 Tag the same release candidate commit with the final release tag
 
 ```
-git co apache 0.8.0-rc1
-git tag 0.8.0
-git push apache 0.8.0
+git checkout 0.11.0-rc1
+git tag 0.11.0
+git push apache 0.11.0
 ```
 
 ### Publish on Crates.io
@@ -364,8 +332,8 @@ with a copy of the previous release announcement.
 Run the following commands to get the number of commits and number of unique contributors for inclusion in the blog post.
 
 ```bash
-git log --pretty=oneline 0.9.0..0.8.0 ballista ballista-cli examples | wc -l
-git shortlog -sn 0.9.0..0.8.0 ballista ballista-cli examples | wc -l
+git log --pretty=oneline 0.11.0..0.10.0 ballista ballista-cli examples | wc -l
+git shortlog -sn 0.11.0..0.10.0 ballista ballista-cli examples | wc -l
 ```
 
 Once there is consensus on the contents of the post, create a PR to add a blog post to the