You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by GitBox <gi...@apache.org> on 2018/10/17 17:03:29 UTC
[GitHub] john-bodley opened a new issue #6131: [SIP-12] Proposal for Branch
Management and Release Process
john-bodley opened a new issue #6131: [SIP-12] Proposal for Branch Management and Release Process
URL: https://github.com/apache/incubator-superset/issues/6131
## Motivation
Historically both [Airbnb](http://github.com/airbnb/incubator-superset) and [Lyft](http://github.com/lyft/incubator-superset) have managed their own fork on incubator-superset which is deployed in production via a Git submodule. Both companies have been actively developing Superset and deploy internally (features and bug fixes) at a higher cadence than the public PyPI releases. These forks came about because of the potential instability of the master branch (see SIP-12) and provided a mechanism to cherry-pick specific commits thus ensuring an increased stability.
Airbnb and Lyft have discussed ways of trying to fold these forked branches back into the incubator-superset repo to better ensure consistency and to serve as a mechanism for formalizing the release process which is somewhat sporadic. Though ideas have been discussed between both Airbnb and Lyft, this SIP is Airbnb’s perspective on a possible viable proposal.
## Proposed Change
The proposed solution is based primarily on having release branches where Airbnb and Lyft are the primary beta testers for stabilizing the branch.
### Master branch
There will be no changes to how the `master` branch currently works. We will do our best to keep the `master` branch healthy in part by the initiatives outlined in SIP-13. If you create a pull request, please do it against the `master` branch. We will maintain stable branches for releases separately but we don’t accept pull requests to them directly. Instead, we will cherry-pick non-breaking changes from `master` to the two latest releases.
### Release Cadence
We propose a regular release cadence of every **two weeks**. This period is a tradeoff between stability and flexibility ensuring that new features can be deployed within a reasonable time frame. Furthermore the longer the release cadence the greater the potential conflict when cherry-picking commits. A release will be deployed two weeks after it was cut.
We propose that each release branch is fully managed by a single organization, i.e., initially both Airbnb and Lyft would alternate between releases and be responsible for managing one branch every four weeks.
### Branch Naming
We propose that all Git branches follow the `<prefix>/<suffix>` naming convention:
- Personal: `<username>/<issue>`, i.e., `jane-doe/issue-123`
- Feature: `feature/<name>`, i.e., `feature/dashboard-v2`
- Release: `release/<version>`, i.e., `release/1.1`
### Semantic Versioning
Superset follows semantic versioning (`<major>.<minor>.<patch>`) although the major version has remained fixed at zero. We should release patch versions for bug fixes, minor versions for new features, and the major version for breaking changes. When we make breaking changes, we should also introduce deprecation warnings in a minor version so that our users learn about the upcoming changes and migrate their code in advance.
Git release branches represent the `<major>.<minor>` level whereas Git tags (associated with a specific commit) represent the fully defined `<major>.<minor>.<patch>` version. Note tags may contain an optional release candidate suffix (`-rc`).
### The Git Ecosystem
The following diagram describes the incubator-superset Git ecosystem and how we plan to couple this with Airbnb’s internal deployment (superset-internal which uses a Git submodule) which will help serve as a mechanism for stabilizing a release.
#### Day 0
On day 0 the next release branch (`release/1.1`) will be cut from `master` which references the latest commit. The commit will be tagged with `v1.1.0-rc` (the v1.1.0 release candidate) and deployed to PyPI.
#### Day 0 - 13
Only commits associated which resolve bugs present in the release candidate will be cherry-picked onto the release branch. Fixes to features added after the cut will not be cherry picked (†). Note there may be potential merge conflicts (which will have to be resolved) when cherry-picking a fix due to an upstream commit which is not in the release (‡). If this is too complex the cherry-pick should be skipped (---) as it will be included in the next release. New features can be committed on `master` **if and only if** they’re defined in their entirety within the two week cycle. Larger features should be developed on a feature specific branch (which is then merged into `master`) or using feature flags.
The following diagram shows the use of a feature branch which spans multiple releases.
#### Day 7
Airbnb and Lyft will update the Git submodule reference to point to the release (`release/1.1`). This seven day lag provides a soak period of ensuring that critical bugs are not deployed to our production environments. The update is made to the `.gitmodules` file,
```
[submodule "incubator-superset"]
path = incubator-superset
url = https://github.com/apache/incubator-superset
branch = release/1.1
```
which ensures it tracks the correct release branch. Like the two week release process the submodule branch will be updated every two weeks.
#### Day 7 - 20
Periodically Airbnb and Lyft will ensure that the submodule in their internal deployment will reference the latest commit from the remote branch (which contains all the cherries) via,
```
git submodule update --remote --recursive --merge
```
which will fetch the latest changes for each submodule, merge them in, and check out the latest revision of the submodule. This step allows us to further test and stabilize the branch in a production environment.
#### Day 14
The first (non release candidate) version will be tagged (`v1.1.0`) and deployed to PyPI which references the latest commit on the branch (*). Simultaneously the next release branch (`release/1.2`) will be cut from `master` which will mimic `v1.1.0` except it will also include any feature commits which were merged in the past 14 days. Note a new release branch is cut if and only if feature commits were added, to prevent duplication.
The following diagram shows an example where on day 14 a new release branch is not cut as there were no new features added since day 0. The next viable release cut date will then be day 28 to preserve the release cadence.
#### Day 14 - 27
Additional bug fixes to commits in the release candidate will be cherry-picked onto the _n_ most recent releases (where _n_ = 2, i.e.,` release/1.1` and `release/1.2`). Throughout this period the tag can be updated (incrementing the patch) as desired based on the severity of the fix.
#### Day 28 -
Assuming there is a new release branch (`release/1.3`) we will no longer be picking cherries onto `release/1.1` since we only support two releases (`release/1.2` and `release/1.3`). In summary most release branches will have a four week active lifespan.
Branches should exist for infinitum to ensure critical security fixes can be patched to the necessary branch.
## Action Items
Currently we plan on manually cherry-picking the commits from `master` to the desired release branch. This could be automated in the future based on PR tags.
We also should check with Apache whether there are any concerns/feasibility issues with releasing on a two week cadence.
@beto @mistercrunch
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org