Effort in doing a release

[Bolke de Bruin <bd...@gmail.com>]

(Migrating this from the vote result thread).

The git-fu part is quite limited. With the branching for release (test/stable) and then tagging it has become quite easy to get right thing is. In the beginning it was often syncing to master, now it is cherry-picking from master and making sure it still builds properly as master and test/stable might have deviated. The airflow-pr tool makes life quite easy too as it allows applying patches to multiple branches. For code it has just become applying it thoughtfully.

What you really start to appreciate is unit tests. Many of the last minute issues that we now see, we would have caught during unit tests as they would express the intention of what we expected from a certain function. Now we are much more reliant on people reporting issues and then understanding their issues. You will start hating the time that is being spend on the (integration) tests - I think we went from 10 minutes per run to around 20 minutes. I don’t think that is required and I think we are a bit lazy in creating good tests.

What you also start appreciating is atomic PRs. Those large PRs that address multiple issues are really hard to review and to holistically understand. This is even harder when having to review multiple PRs to get to a release. Problems with PRs also get harder to track down because of them.

A bit harder is keeping the momentum going. For myself, but also others. The voting takes quite a bit of time and at the moment we are required to do two. Chasing people to get enough testing or when they found an issue and promised a PR for it but they weren't forthcoming. I ended up writing quite a lot of small things (or sometimes larger) ones by myself. I understand why it happens, but still it puts a lot on one person. Going forward I assume it will be easier when releases happen faster.

Great that you want to volunteer, I think Chris also wanted to pick up 1.8.1. It is possible to start already: v1-8-test will be the next release and as soon as we get 1.8.0 out can can start bringing both branches into sync until we hit a release candidate for 1.8.1 at which time the stable branch gets kind of frozen.

All in all it isn’t / wasn’t a unpleasant experience and your (plural) support really helped. Doing it entirely by yourself really wouldn’t be fun :)

Getting back to you question:

* Assuming a 20/80 rule on PRs in complexity 20%=complex 80%=simple
* Complex PR full integration: 1 hour
* Simple: 10 minutes
* Point release (1.8.0 -> 1.8.1): 30 PRs (6 complex, 24 simple)
* Integration patches: ~20 simple: 30 minutes each, 2 complex: 2 hours each

Point release: 240min + 360min for PRs + 600min simple patches + 240 minutes complex = 24h, add 20% overhead (Voting, Gitter etc): ~28,8h to spend.

Make sense?

Bolke

> What I meant to ask is "how much engineering effort it takes to bake a
> single RC?", I guess it depends on how much git-fu is necessary plus some
> overhead cost of doing the series of actions/commands/emails/jira.
> 
> I can volunteer for 1.8.1 (hopefully I can get do it along another Airbnb
> engineer/volunteer to tag along) and will try to document/automate
> everything I can as I go through the process. The goal of 1.8.1 could be to
> basically package 1.8.0 + Dan's bugfix, and for Airbnb to get familiar with
> the process.
> 
> It'd be great if you can dump your whole process on the wiki, and we'll
> improve it on this next pass.
> 
> Thanks again for the mountain of work that went into packaging this release.
> 
> Max