[DISCUSS] Forthcoming Hadoop releases

[Steve Loughran <st...@cloudera.com.INVALID>]

I want to start a quick discussion on a plan for hadoop releases this
summer. I am willing to do the release manager work. Mukund and Mehakmeet
have have already volunteered to help even if they don't know that yet.

I've got two goals

   1. minor followup to 3.3.3
   2. feature release of new stuff


*Followup to 3.3.3, working title "3.3.4"*

I've a PR up on github to add those change to the 3.3.2/3.3.3 line which
have shipped elsewhere and/or we consider critical.

https://github.com/apache/hadoop/pull/4345

This is for critical data integrity/service availability patches; things
like test values we will just triage.

I can start a new release of this at the end of the week, with an RC up
next week ready for review. With the wonderful docker based build and some
extra automation I've been adding for validating releases
(validate-hadoop-client-artifacts), getting that RC out is not that
problematic; issuing git commands is the heavy lifting.

What does take effort is the testing by everybody else; the smaller the set
of changes the more this is limited to validating the artifacts and the
maven publishing.

As it is a follow up to hadoop 3.3.3 then it needs the version number
3.3.4. This raises the question "what about branch-3.3", which brings me to
the next deliverable.

*branch-3.3 => branch-3.4, targeting hadoop 3.4.0 in 3Q22*

With the 3.3.x line being maintained for critical fixes only, make the
hadoop version in branch-3.3 "hadoop-3.4.0" and release later this year.

A release schedule which is probably doable despite people taking time off
over the summer could be

   - feature complete by July/August
   - RC(s) sept/oct with goal of shipping by October


I volunteer to be release manager, albeit with critical help from
colleagues. For people who haven't worked with me on a project release
before, know that I'm fairly ruthless about getting changes in once the
branch is locked down. So get those features in now.

hadoop trunk gets its version number incremented to 3.5.0-SNAPSHOT

It's probably time we think about what a release off trunk would mean -but
t I would like to get a branch-3.3 release out rather than later.

What do people think of this? And is there anyone else willing to get
involved with the release process?

-Steve