You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@superset.apache.org by Maxime Beauchemin <ma...@gmail.com> on 2019/03/19 04:52:50 UTC
The state of releases
Hi all,
I wanted to send an email explaining the current state of releases and
start a thread about what's ahead.
Early on in the project lifecycle, I used to package releases and push to
Pypi without much process. I'd package internal releases internally for
Airbnb. We'd roll out to staging, stabilize the release, and launch in
production. After a some time without major issues and regressions, I'd
push a new release out to Pypi and make it available to the community. That
process worked ok when I reviewed or wrote most PRs and had a handle on
everything that was going on. The community has grown much, and clearly
that process has not been appropriate for quite some time now.
Later on, we joined the Apache Software Foundation's incubator, and the ASF
has clear process around releases
<http://www.apache.org/dev/release-publishing.html>. As an ASF project, it
was not reasonable for us to push to Pypi until we sorted out the process
and followed the "Apache Way". Since `0.29` we've still been cutting
release branches and labeling "release candidates", but have refrained from
publishing to Pypi. Those release branches contain commits that have been
put into production at Lyft and/or Airbnb and elsewhere, but didn't meet
the ASF's standards, so could not be published as releases.
Also with the growing community, we clearly need a more structured process
around releases. John Bodley wrote a solid SIP
<https://github.com/apache/incubator-superset/issues/6131> proposing a
clear structure and process around releases, and details much of what the
ASF release process does not.
All this to say, *I'm starting some work on our first ASF-approved source
code release*. For context, the ASF makes a distinction between "source
code releases" and "convenience releases". The former contains the source
code and build instructions, and the latter contains binaries and is made
available in places like Pypi (or Maven / npm / RubyGem / ...).
While the source code release should be pretty straight forward, it's not
really, well, convenient. You can't just "pip install" that... The
convenience release on the other hand is great, but may require more work
and guidance from our mentors, ASF legal, license experts as the binaries
may contain our Javascript bundles (or maybe they don't?) but then have to
have a gigantic LICENSE.txt file detailing the licenses of the 500+
Javascript libs that have accumulated in our `package-lock.json` over time.
Or maybe we ship without the bundles and add a `superset compile` command
that does the fetching/build of the JS packages. Unclear to me. I'll need
help figuring this out. For context here's some work I did trying to
automate the LICENSE.txt creation
<https://github.com/apache/incubator-superset/pull/5801>
Anyhow, wanted people to know what's up since it's been so long since the
last Pypi release, and give the opportunity for contributors to jump in and
help out.
Let's get the [community] release process going!
Max
Re: The state of releases
Posted by Maxime Beauchemin <ma...@gmail.com>.
I'm still wondering what to do with my Apache release attempt, we're now a
few RCs behind as `0.31.0.rc20` is on top of branch `release--0.31`
I'd love if someone can follow the steps I described in
https://github.com/apache/incubator-superset/blob/master/RELEASING.md (I
need a volunteer please!) to validate them and release push the latest RC.
Some of these steps are one-offs and were already setup ahead of time in my
case. This second attempt would make sure a committer that has never done
an Apache release can make it through.
Also let's make the suffix RC an Apache release specific thing, so that RCs
become an Apache release concept that follows a sequence that matches
what's in Apache's SVN. If you need different version strings, maybe suffix
with another scheme? 0.31.0devN maybe? Though this may be a python-specific
/ not semver string, I'm not sure and personally don't care about version
strings as long as they make sense and are consistent.
I'm also assuming it's ok to Github-tag RCs that go onto Apache's SVN. A
few months back I was told to not push RC tags to Github, but now that
we're actually trying to do Apache releases it should be ok.
Max
On Tue, Mar 26, 2019 at 12:55 PM David Smith <da...@gmail.com> wrote:
> Thanks for the context, ya'll. Since SIP-12 had not been voted on for
> adoption I was hoping to chime in before there was a lot of "sunk cost" and
> before it was locked. Your background was helpful, thanks.
>
> I do want to note that the motivation for my comments was not to make
> running git commands and creating branches easy -- that is always easy. :-)
> It's about the flow of code and features through branches and how you
> manage those along with customer/support SLAs. The way you do that can
> make the things that ARE hard and potentially very time consuming --
> validating, testing, fixing bugs, releasing, and supporting -- much, much
> more efficient.
>
> I'll take some time to absorb how folks are currently working day to day as
> I ramp up, then revisit later. I'll also take some time to dive into the
> Apache processes. I definitely should gain more understanding of that
> aspect.
>
> Thanks again!
> Dave
>
>
> On Tue, Mar 26, 2019 at 11:43 AM Krist Wongsuphasawat <
> krist.wongz@gmail.com>
> wrote:
>
> > Hi Dave,
> >
> >
> > We appreciate your comments on the release process and are glad to have
> > someone with experience about this topic on board. Before going deep into
> > the discussion about gitflow, we would like to clarify our stance on the
> > topic.
> >
> >
> > First of all, git workflow and cherry-pick are not the pain point in the
> > current release process. As you can verify with Christine and others who
> > have worked on the past releases that the git commands and creating
> > branches are easy parts of the release. The time-consuming parts are
> > verifying features, finding bugs and fixing issues. Therefore, changing
> the
> > git workflow is not the main item on the top of our wishlist at the
> moment.
> > If you would like to provide command line tools for git tagging, creating
> > change logs, that would be greatly welcomed. However, please keep in mind
> > that the git tasks probably accounts for <10% of the work that actually
> > happens in each release, and the 90% are the parts we hope can be more
> > efficient.
> >
> >
> > Secondly, we do not object if you would like to improve the git process.
> > However, we have invested a lot of effort into drafting SIP-12 and
> shifting
> > our dev environment to that workflow already. The SIP has been posted
> since
> > last year. We would like to save our bandwidth to focus on other topics
> > rather than changing the git process. If you would like to come up with a
> > new process, please also think of a smooth transition between the current
> > plan to that plan and what are the benefits we all would gain.
> >
> >
> > Third, making the release process compatible with Apache release
> protocols.
> > This is the part where we have not carefully thought about and would be a
> > great area where your expertise could be helpful. We do not have any
> > problem with having 18 RCs for internal releases and testing. But if
> having
> > frequent version bump like this is not a viable option for Apache
> > “vote-before-release” protocol, we need to come up with some solutions.
> >
> >
> > Thank you again for your initiation. We are happy to have you on board
> and
> > hope your experience can help all of us make the releases better.
> >
> >
> > Best regards,
> >
> >
> > MIchelle and Krist
> >
> >
> >
> > On Fri, Mar 22, 2019 at 10:32 AM David Smith <da...@gmail.com>
> > wrote:
> >
> > > Thanks, Michelle. The reliance on cherry-picking as part of a work flow
> > can
> > > definitely bring the process into conflict with other tools and
> processes
> > > in the git ecosystem. I've left a fairly lengthy comment on SIP-12
> > > proposing that the mechanics be handled using gitflow, which does not
> use
> > > cherry-picking, just for discussion.
> > >
> > >
> >
> https://github.com/apache/incubator-superset/issues/6131#issuecomment-475691949
> > >
> > > On Wed, Mar 20, 2019 at 1:04 PM Michelle Thomas
> > > <mi...@airbnb.com.invalid> wrote:
> > >
> > > > To answer the question about using a workflow for managing
> > > > releases/branches/tags, I previously evaluated using conventional
> > > commits:
> > > > https://www.conventionalcommits.org/en/v1.0.0-beta.3/, which is a
> > system
> > > > for tagging commits to generate the changelog. At the time it seemed
> > like
> > > > it may have issues creating the Changelog with the way we commit
> > > everything
> > > > to master then cherry-pick fixes onto a branch, but the consistency
> in
> > > > tagging PRs with fix, feature, docs seemed useful. I don't think
> we've
> > > > heavily discussed other workflows. Sounds good to get suggestions for
> > > > improvement on SIP-12.
> > > >
> > > > On Tue, Mar 19, 2019 at 9:27 PM David Smith <da...@gmail.com>
> > > > wrote:
> > > >
> > > > > I'm new to this project, so I apologize if this has been discussed
> > > > > previously, but... Have we considered using a widely adopted
> workflow
> > > for
> > > > > managing releases/branches/tags, and using tools that the community
> > has
> > > > > already built for them? For example Git Flow or some variant
> > thereof?
> > > > For
> > > > > anyone who is unfamiliar, see overview and tooling--in the form of
> > git
> > > > > extensions--here: https://github.com/nvie/gitflow
> > > > >
> > > > > I saw SIP-12 and have a few comments that I will try to document
> > > tomorrow
> > > > > on that issue, but I'm really curious whether other common
> > > > processes/flows
> > > > > were rejected explicitly or if they just didn't come up?
> > > > >
> > > > > Dave
> > > > >
> > > > >
> > > > >
> > > > > On Tue, Mar 19, 2019 at 8:45 PM Maxime Beauchemin <
> > > > > maximebeauchemin@gmail.com> wrote:
> > > > >
> > > > > > Wondering what to do next, Justin, is it ok for me to push the
> > > related
> > > > > tag
> > > > > > to Github?
> > > > > >
> > > > > > Should I trigger a [VOTE] thread?
> > > > > >
> > > > > > Max
> > > > > >
> > > > > > On Tue, Mar 19, 2019 at 12:04 AM Maxime Beauchemin <
> > > > > > maximebeauchemin@gmail.com> wrote:
> > > > > >
> > > > > > > My svn is a bit rusty but I made some progress tonight:
> > > > > > > https://github.com/apache/incubator-superset/pull/7054
> > > > > > >
> > > > > > > Pushed some files to SVN
> > > > > > >
> > > >
> https://dist.apache.org/repos/dist/dev/incubator/superset/0.31.0rc18/
> > > > > > >
> > > > > > > It's not intended to be an actual real RC, but it's a start.
> > > > > > >
> > > > > > > An early question is around RC numbers. Lyft & Airbnb have been
> > > > doing a
> > > > > > > lot of back and forth on the release branch already and
> `0.31.0`
> > is
> > > > > > already
> > > > > > > up to rc18. I was internally debating what to do here, keeping
> > the
> > > > > > > numbering scheme, or starting anew. Probably the right thing to
> > do
> > > is
> > > > > to
> > > > > > > pick a number (say 0.32.0) and start clean at 0.32.0rc1, and
> > stick
> > > to
> > > > > the
> > > > > > > Apache process, and use continuous numbers. I'm assuming the
> > > handoff
> > > > is
> > > > > > at
> > > > > > > 0.31.0rc18.
> > > > > > >
> > > > > > > Now people that would want to work on non-Apache release
> branches
> > > > would
> > > > > > do
> > > > > > > so in their fork, and in their own namespace. Maybe their local
> > > build
> > > > > > > instructions can become recipes for an upcoming release, but I
> > > > suggest
> > > > > > that
> > > > > > > this happens in forks from now on.
> > > > > > >
> > > > > > > Clearly this Apache process here with signing + svn commit +
> > voting
> > > > is
> > > > > > > more involved than the usual git cherry-pick + git tag + git
> > push,
> > > so
> > > > > > that
> > > > > > > should probably lead to less RCs (I can't imagine doing 18
> votes
> > > > > around a
> > > > > > > single release).
> > > > > > >
> > > > > > > Not to mention the fact that the real complexity around
> releasing
> > > is
> > > > > > > raising and labeling issues with a proper target version,
> gather
> > > PRs,
> > > > > > > cherry-pick fixes and resolve conflicts if any. I'm hoping we
> can
> > > > build
> > > > > > > tooling to help with all this. Hugh started something a while
> > back,
> > > > but
> > > > > > > there's lots to be done still in that area.
> > > > > > >
> > > > > > > Max
> > > > > > >
> > > > > > > On Mon, Mar 18, 2019 at 11:39 PM Maxime Beauchemin <
> > > > > > > maximebeauchemin@gmail.com> wrote:
> > > > > > >
> > > > > > >> Not officially approved yet, I think we should start a
> [DISCUSS]
> > > > > thread
> > > > > > >> and eventually [VOTE]
> > > > > > >>
> > > > > > >> On Mon, Mar 18, 2019 at 10:13 PM David Smith <
> > > > dave.a.smith@gmail.com>
> > > > > > >> wrote:
> > > > > > >>
> > > > > > >>> Thanks Max! Out of curiosity, has that release SIP-12 been
> > > approved
> > > > > > yet?
> > > > > > >>> I
> > > > > > >>> have some thoughts but if it is already a done deal I'll wait
> > > until
> > > > > > >>> another
> > > > > > >>> time. :-)
> > > > > > >>>
> > > > > > >>> On Mon, Mar 18, 2019 at 9:53 PM Maxime Beauchemin <
> > > > > > >>> maximebeauchemin@gmail.com> wrote:
> > > > > > >>>
> > > > > > >>> > Hi all,
> > > > > > >>> >
> > > > > > >>> > I wanted to send an email explaining the current state of
> > > > releases
> > > > > > and
> > > > > > >>> > start a thread about what's ahead.
> > > > > > >>> >
> > > > > > >>> > Early on in the project lifecycle, I used to package
> releases
> > > and
> > > > > > push
> > > > > > >>> to
> > > > > > >>> > Pypi without much process. I'd package internal releases
> > > > internally
> > > > > > for
> > > > > > >>> > Airbnb. We'd roll out to staging, stabilize the release,
> and
> > > > launch
> > > > > > in
> > > > > > >>> > production. After a some time without major issues and
> > > > regressions,
> > > > > > I'd
> > > > > > >>> > push a new release out to Pypi and make it available to the
> > > > > > community.
> > > > > > >>> That
> > > > > > >>> > process worked ok when I reviewed or wrote most PRs and
> had a
> > > > > handle
> > > > > > on
> > > > > > >>> > everything that was going on. The community has grown much,
> > and
> > > > > > clearly
> > > > > > >>> > that process has not been appropriate for quite some time
> > now.
> > > > > > >>> >
> > > > > > >>> > Later on, we joined the Apache Software Foundation's
> > incubator,
> > > > and
> > > > > > >>> the ASF
> > > > > > >>> > has clear process around releases
> > > > > > >>> > <http://www.apache.org/dev/release-publishing.html>. As an
> > ASF
> > > > > > >>> project, it
> > > > > > >>> > was not reasonable for us to push to Pypi until we sorted
> out
> > > the
> > > > > > >>> process
> > > > > > >>> > and followed the "Apache Way". Since `0.29` we've still
> been
> > > > > cutting
> > > > > > >>> > release branches and labeling "release candidates", but
> have
> > > > > > refrained
> > > > > > >>> from
> > > > > > >>> > publishing to Pypi. Those release branches contain commits
> > that
> > > > > have
> > > > > > >>> been
> > > > > > >>> > put into production at Lyft and/or Airbnb and elsewhere,
> but
> > > > didn't
> > > > > > >>> meet
> > > > > > >>> > the ASF's standards, so could not be published as releases.
> > > > > > >>> >
> > > > > > >>> > Also with the growing community, we clearly need a more
> > > > structured
> > > > > > >>> process
> > > > > > >>> > around releases. John Bodley wrote a solid SIP
> > > > > > >>> > <https://github.com/apache/incubator-superset/issues/6131>
> > > > > > proposing a
> > > > > > >>> > clear structure and process around releases, and details
> much
> > > of
> > > > > what
> > > > > > >>> the
> > > > > > >>> > ASF release process does not.
> > > > > > >>> >
> > > > > > >>> > All this to say, *I'm starting some work on our first
> > > > ASF-approved
> > > > > > >>> source
> > > > > > >>> > code release*. For context, the ASF makes a distinction
> > between
> > > > > > "source
> > > > > > >>> > code releases" and "convenience releases". The former
> > contains
> > > > the
> > > > > > >>> source
> > > > > > >>> > code and build instructions, and the latter contains
> binaries
> > > and
> > > > > is
> > > > > > >>> made
> > > > > > >>> > available in places like Pypi (or Maven / npm / RubyGem /
> > ...).
> > > > > > >>> >
> > > > > > >>> > While the source code release should be pretty straight
> > > forward,
> > > > > it's
> > > > > > >>> not
> > > > > > >>> > really, well, convenient. You can't just "pip install"
> > that...
> > > > The
> > > > > > >>> > convenience release on the other hand is great, but may
> > require
> > > > > more
> > > > > > >>> work
> > > > > > >>> > and guidance from our mentors, ASF legal, license experts
> as
> > > the
> > > > > > >>> binaries
> > > > > > >>> > may contain our Javascript bundles (or maybe they don't?)
> but
> > > > then
> > > > > > >>> have to
> > > > > > >>> > have a gigantic LICENSE.txt file detailing the licenses of
> > the
> > > > 500+
> > > > > > >>> > Javascript libs that have accumulated in our
> > > `package-lock.json`
> > > > > over
> > > > > > >>> time.
> > > > > > >>> > Or maybe we ship without the bundles and add a `superset
> > > compile`
> > > > > > >>> command
> > > > > > >>> > that does the fetching/build of the JS packages. Unclear to
> > me.
> > > > > I'll
> > > > > > >>> need
> > > > > > >>> > help figuring this out. For context here's some work I did
> > > trying
> > > > > to
> > > > > > >>> > automate the LICENSE.txt creation
> > > > > > >>> > <https://github.com/apache/incubator-superset/pull/5801>
> > > > > > >>> >
> > > > > > >>> > Anyhow, wanted people to know what's up since it's been so
> > long
> > > > > since
> > > > > > >>> the
> > > > > > >>> > last Pypi release, and give the opportunity for
> contributors
> > to
> > > > > jump
> > > > > > >>> in and
> > > > > > >>> > help out.
> > > > > > >>> >
> > > > > > >>> > Let's get the [community] release process going!
> > > > > > >>> >
> > > > > > >>> > Max
> > > > > > >>> >
> > > > > > >>>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> >
> > --
> >
> > *Krist Wongsuphasawat, PhD*
> > http://kristw.yellowpigz.com
> >
>
Re: The state of releases
Posted by David Smith <da...@gmail.com>.
Thanks for the context, ya'll. Since SIP-12 had not been voted on for
adoption I was hoping to chime in before there was a lot of "sunk cost" and
before it was locked. Your background was helpful, thanks.
I do want to note that the motivation for my comments was not to make
running git commands and creating branches easy -- that is always easy. :-)
It's about the flow of code and features through branches and how you
manage those along with customer/support SLAs. The way you do that can
make the things that ARE hard and potentially very time consuming --
validating, testing, fixing bugs, releasing, and supporting -- much, much
more efficient.
I'll take some time to absorb how folks are currently working day to day as
I ramp up, then revisit later. I'll also take some time to dive into the
Apache processes. I definitely should gain more understanding of that
aspect.
Thanks again!
Dave
On Tue, Mar 26, 2019 at 11:43 AM Krist Wongsuphasawat <kr...@gmail.com>
wrote:
> Hi Dave,
>
>
> We appreciate your comments on the release process and are glad to have
> someone with experience about this topic on board. Before going deep into
> the discussion about gitflow, we would like to clarify our stance on the
> topic.
>
>
> First of all, git workflow and cherry-pick are not the pain point in the
> current release process. As you can verify with Christine and others who
> have worked on the past releases that the git commands and creating
> branches are easy parts of the release. The time-consuming parts are
> verifying features, finding bugs and fixing issues. Therefore, changing the
> git workflow is not the main item on the top of our wishlist at the moment.
> If you would like to provide command line tools for git tagging, creating
> change logs, that would be greatly welcomed. However, please keep in mind
> that the git tasks probably accounts for <10% of the work that actually
> happens in each release, and the 90% are the parts we hope can be more
> efficient.
>
>
> Secondly, we do not object if you would like to improve the git process.
> However, we have invested a lot of effort into drafting SIP-12 and shifting
> our dev environment to that workflow already. The SIP has been posted since
> last year. We would like to save our bandwidth to focus on other topics
> rather than changing the git process. If you would like to come up with a
> new process, please also think of a smooth transition between the current
> plan to that plan and what are the benefits we all would gain.
>
>
> Third, making the release process compatible with Apache release protocols.
> This is the part where we have not carefully thought about and would be a
> great area where your expertise could be helpful. We do not have any
> problem with having 18 RCs for internal releases and testing. But if having
> frequent version bump like this is not a viable option for Apache
> “vote-before-release” protocol, we need to come up with some solutions.
>
>
> Thank you again for your initiation. We are happy to have you on board and
> hope your experience can help all of us make the releases better.
>
>
> Best regards,
>
>
> MIchelle and Krist
>
>
>
> On Fri, Mar 22, 2019 at 10:32 AM David Smith <da...@gmail.com>
> wrote:
>
> > Thanks, Michelle. The reliance on cherry-picking as part of a work flow
> can
> > definitely bring the process into conflict with other tools and processes
> > in the git ecosystem. I've left a fairly lengthy comment on SIP-12
> > proposing that the mechanics be handled using gitflow, which does not use
> > cherry-picking, just for discussion.
> >
> >
> https://github.com/apache/incubator-superset/issues/6131#issuecomment-475691949
> >
> > On Wed, Mar 20, 2019 at 1:04 PM Michelle Thomas
> > <mi...@airbnb.com.invalid> wrote:
> >
> > > To answer the question about using a workflow for managing
> > > releases/branches/tags, I previously evaluated using conventional
> > commits:
> > > https://www.conventionalcommits.org/en/v1.0.0-beta.3/, which is a
> system
> > > for tagging commits to generate the changelog. At the time it seemed
> like
> > > it may have issues creating the Changelog with the way we commit
> > everything
> > > to master then cherry-pick fixes onto a branch, but the consistency in
> > > tagging PRs with fix, feature, docs seemed useful. I don't think we've
> > > heavily discussed other workflows. Sounds good to get suggestions for
> > > improvement on SIP-12.
> > >
> > > On Tue, Mar 19, 2019 at 9:27 PM David Smith <da...@gmail.com>
> > > wrote:
> > >
> > > > I'm new to this project, so I apologize if this has been discussed
> > > > previously, but... Have we considered using a widely adopted workflow
> > for
> > > > managing releases/branches/tags, and using tools that the community
> has
> > > > already built for them? For example Git Flow or some variant
> thereof?
> > > For
> > > > anyone who is unfamiliar, see overview and tooling--in the form of
> git
> > > > extensions--here: https://github.com/nvie/gitflow
> > > >
> > > > I saw SIP-12 and have a few comments that I will try to document
> > tomorrow
> > > > on that issue, but I'm really curious whether other common
> > > processes/flows
> > > > were rejected explicitly or if they just didn't come up?
> > > >
> > > > Dave
> > > >
> > > >
> > > >
> > > > On Tue, Mar 19, 2019 at 8:45 PM Maxime Beauchemin <
> > > > maximebeauchemin@gmail.com> wrote:
> > > >
> > > > > Wondering what to do next, Justin, is it ok for me to push the
> > related
> > > > tag
> > > > > to Github?
> > > > >
> > > > > Should I trigger a [VOTE] thread?
> > > > >
> > > > > Max
> > > > >
> > > > > On Tue, Mar 19, 2019 at 12:04 AM Maxime Beauchemin <
> > > > > maximebeauchemin@gmail.com> wrote:
> > > > >
> > > > > > My svn is a bit rusty but I made some progress tonight:
> > > > > > https://github.com/apache/incubator-superset/pull/7054
> > > > > >
> > > > > > Pushed some files to SVN
> > > > > >
> > > https://dist.apache.org/repos/dist/dev/incubator/superset/0.31.0rc18/
> > > > > >
> > > > > > It's not intended to be an actual real RC, but it's a start.
> > > > > >
> > > > > > An early question is around RC numbers. Lyft & Airbnb have been
> > > doing a
> > > > > > lot of back and forth on the release branch already and `0.31.0`
> is
> > > > > already
> > > > > > up to rc18. I was internally debating what to do here, keeping
> the
> > > > > > numbering scheme, or starting anew. Probably the right thing to
> do
> > is
> > > > to
> > > > > > pick a number (say 0.32.0) and start clean at 0.32.0rc1, and
> stick
> > to
> > > > the
> > > > > > Apache process, and use continuous numbers. I'm assuming the
> > handoff
> > > is
> > > > > at
> > > > > > 0.31.0rc18.
> > > > > >
> > > > > > Now people that would want to work on non-Apache release branches
> > > would
> > > > > do
> > > > > > so in their fork, and in their own namespace. Maybe their local
> > build
> > > > > > instructions can become recipes for an upcoming release, but I
> > > suggest
> > > > > that
> > > > > > this happens in forks from now on.
> > > > > >
> > > > > > Clearly this Apache process here with signing + svn commit +
> voting
> > > is
> > > > > > more involved than the usual git cherry-pick + git tag + git
> push,
> > so
> > > > > that
> > > > > > should probably lead to less RCs (I can't imagine doing 18 votes
> > > > around a
> > > > > > single release).
> > > > > >
> > > > > > Not to mention the fact that the real complexity around releasing
> > is
> > > > > > raising and labeling issues with a proper target version, gather
> > PRs,
> > > > > > cherry-pick fixes and resolve conflicts if any. I'm hoping we can
> > > build
> > > > > > tooling to help with all this. Hugh started something a while
> back,
> > > but
> > > > > > there's lots to be done still in that area.
> > > > > >
> > > > > > Max
> > > > > >
> > > > > > On Mon, Mar 18, 2019 at 11:39 PM Maxime Beauchemin <
> > > > > > maximebeauchemin@gmail.com> wrote:
> > > > > >
> > > > > >> Not officially approved yet, I think we should start a [DISCUSS]
> > > > thread
> > > > > >> and eventually [VOTE]
> > > > > >>
> > > > > >> On Mon, Mar 18, 2019 at 10:13 PM David Smith <
> > > dave.a.smith@gmail.com>
> > > > > >> wrote:
> > > > > >>
> > > > > >>> Thanks Max! Out of curiosity, has that release SIP-12 been
> > approved
> > > > > yet?
> > > > > >>> I
> > > > > >>> have some thoughts but if it is already a done deal I'll wait
> > until
> > > > > >>> another
> > > > > >>> time. :-)
> > > > > >>>
> > > > > >>> On Mon, Mar 18, 2019 at 9:53 PM Maxime Beauchemin <
> > > > > >>> maximebeauchemin@gmail.com> wrote:
> > > > > >>>
> > > > > >>> > Hi all,
> > > > > >>> >
> > > > > >>> > I wanted to send an email explaining the current state of
> > > releases
> > > > > and
> > > > > >>> > start a thread about what's ahead.
> > > > > >>> >
> > > > > >>> > Early on in the project lifecycle, I used to package releases
> > and
> > > > > push
> > > > > >>> to
> > > > > >>> > Pypi without much process. I'd package internal releases
> > > internally
> > > > > for
> > > > > >>> > Airbnb. We'd roll out to staging, stabilize the release, and
> > > launch
> > > > > in
> > > > > >>> > production. After a some time without major issues and
> > > regressions,
> > > > > I'd
> > > > > >>> > push a new release out to Pypi and make it available to the
> > > > > community.
> > > > > >>> That
> > > > > >>> > process worked ok when I reviewed or wrote most PRs and had a
> > > > handle
> > > > > on
> > > > > >>> > everything that was going on. The community has grown much,
> and
> > > > > clearly
> > > > > >>> > that process has not been appropriate for quite some time
> now.
> > > > > >>> >
> > > > > >>> > Later on, we joined the Apache Software Foundation's
> incubator,
> > > and
> > > > > >>> the ASF
> > > > > >>> > has clear process around releases
> > > > > >>> > <http://www.apache.org/dev/release-publishing.html>. As an
> ASF
> > > > > >>> project, it
> > > > > >>> > was not reasonable for us to push to Pypi until we sorted out
> > the
> > > > > >>> process
> > > > > >>> > and followed the "Apache Way". Since `0.29` we've still been
> > > > cutting
> > > > > >>> > release branches and labeling "release candidates", but have
> > > > > refrained
> > > > > >>> from
> > > > > >>> > publishing to Pypi. Those release branches contain commits
> that
> > > > have
> > > > > >>> been
> > > > > >>> > put into production at Lyft and/or Airbnb and elsewhere, but
> > > didn't
> > > > > >>> meet
> > > > > >>> > the ASF's standards, so could not be published as releases.
> > > > > >>> >
> > > > > >>> > Also with the growing community, we clearly need a more
> > > structured
> > > > > >>> process
> > > > > >>> > around releases. John Bodley wrote a solid SIP
> > > > > >>> > <https://github.com/apache/incubator-superset/issues/6131>
> > > > > proposing a
> > > > > >>> > clear structure and process around releases, and details much
> > of
> > > > what
> > > > > >>> the
> > > > > >>> > ASF release process does not.
> > > > > >>> >
> > > > > >>> > All this to say, *I'm starting some work on our first
> > > ASF-approved
> > > > > >>> source
> > > > > >>> > code release*. For context, the ASF makes a distinction
> between
> > > > > "source
> > > > > >>> > code releases" and "convenience releases". The former
> contains
> > > the
> > > > > >>> source
> > > > > >>> > code and build instructions, and the latter contains binaries
> > and
> > > > is
> > > > > >>> made
> > > > > >>> > available in places like Pypi (or Maven / npm / RubyGem /
> ...).
> > > > > >>> >
> > > > > >>> > While the source code release should be pretty straight
> > forward,
> > > > it's
> > > > > >>> not
> > > > > >>> > really, well, convenient. You can't just "pip install"
> that...
> > > The
> > > > > >>> > convenience release on the other hand is great, but may
> require
> > > > more
> > > > > >>> work
> > > > > >>> > and guidance from our mentors, ASF legal, license experts as
> > the
> > > > > >>> binaries
> > > > > >>> > may contain our Javascript bundles (or maybe they don't?) but
> > > then
> > > > > >>> have to
> > > > > >>> > have a gigantic LICENSE.txt file detailing the licenses of
> the
> > > 500+
> > > > > >>> > Javascript libs that have accumulated in our
> > `package-lock.json`
> > > > over
> > > > > >>> time.
> > > > > >>> > Or maybe we ship without the bundles and add a `superset
> > compile`
> > > > > >>> command
> > > > > >>> > that does the fetching/build of the JS packages. Unclear to
> me.
> > > > I'll
> > > > > >>> need
> > > > > >>> > help figuring this out. For context here's some work I did
> > trying
> > > > to
> > > > > >>> > automate the LICENSE.txt creation
> > > > > >>> > <https://github.com/apache/incubator-superset/pull/5801>
> > > > > >>> >
> > > > > >>> > Anyhow, wanted people to know what's up since it's been so
> long
> > > > since
> > > > > >>> the
> > > > > >>> > last Pypi release, and give the opportunity for contributors
> to
> > > > jump
> > > > > >>> in and
> > > > > >>> > help out.
> > > > > >>> >
> > > > > >>> > Let's get the [community] release process going!
> > > > > >>> >
> > > > > >>> > Max
> > > > > >>> >
> > > > > >>>
> > > > > >>
> > > > >
> > > >
> > >
> >
>
>
> --
>
> *Krist Wongsuphasawat, PhD*
> http://kristw.yellowpigz.com
>
Re: The state of releases
Posted by Krist Wongsuphasawat <kr...@gmail.com>.
Hi Dave,
We appreciate your comments on the release process and are glad to have
someone with experience about this topic on board. Before going deep into
the discussion about gitflow, we would like to clarify our stance on the
topic.
First of all, git workflow and cherry-pick are not the pain point in the
current release process. As you can verify with Christine and others who
have worked on the past releases that the git commands and creating
branches are easy parts of the release. The time-consuming parts are
verifying features, finding bugs and fixing issues. Therefore, changing the
git workflow is not the main item on the top of our wishlist at the moment.
If you would like to provide command line tools for git tagging, creating
change logs, that would be greatly welcomed. However, please keep in mind
that the git tasks probably accounts for <10% of the work that actually
happens in each release, and the 90% are the parts we hope can be more
efficient.
Secondly, we do not object if you would like to improve the git process.
However, we have invested a lot of effort into drafting SIP-12 and shifting
our dev environment to that workflow already. The SIP has been posted since
last year. We would like to save our bandwidth to focus on other topics
rather than changing the git process. If you would like to come up with a
new process, please also think of a smooth transition between the current
plan to that plan and what are the benefits we all would gain.
Third, making the release process compatible with Apache release protocols.
This is the part where we have not carefully thought about and would be a
great area where your expertise could be helpful. We do not have any
problem with having 18 RCs for internal releases and testing. But if having
frequent version bump like this is not a viable option for Apache
“vote-before-release” protocol, we need to come up with some solutions.
Thank you again for your initiation. We are happy to have you on board and
hope your experience can help all of us make the releases better.
Best regards,
MIchelle and Krist
On Fri, Mar 22, 2019 at 10:32 AM David Smith <da...@gmail.com> wrote:
> Thanks, Michelle. The reliance on cherry-picking as part of a work flow can
> definitely bring the process into conflict with other tools and processes
> in the git ecosystem. I've left a fairly lengthy comment on SIP-12
> proposing that the mechanics be handled using gitflow, which does not use
> cherry-picking, just for discussion.
>
> https://github.com/apache/incubator-superset/issues/6131#issuecomment-475691949
>
> On Wed, Mar 20, 2019 at 1:04 PM Michelle Thomas
> <mi...@airbnb.com.invalid> wrote:
>
> > To answer the question about using a workflow for managing
> > releases/branches/tags, I previously evaluated using conventional
> commits:
> > https://www.conventionalcommits.org/en/v1.0.0-beta.3/, which is a system
> > for tagging commits to generate the changelog. At the time it seemed like
> > it may have issues creating the Changelog with the way we commit
> everything
> > to master then cherry-pick fixes onto a branch, but the consistency in
> > tagging PRs with fix, feature, docs seemed useful. I don't think we've
> > heavily discussed other workflows. Sounds good to get suggestions for
> > improvement on SIP-12.
> >
> > On Tue, Mar 19, 2019 at 9:27 PM David Smith <da...@gmail.com>
> > wrote:
> >
> > > I'm new to this project, so I apologize if this has been discussed
> > > previously, but... Have we considered using a widely adopted workflow
> for
> > > managing releases/branches/tags, and using tools that the community has
> > > already built for them? For example Git Flow or some variant thereof?
> > For
> > > anyone who is unfamiliar, see overview and tooling--in the form of git
> > > extensions--here: https://github.com/nvie/gitflow
> > >
> > > I saw SIP-12 and have a few comments that I will try to document
> tomorrow
> > > on that issue, but I'm really curious whether other common
> > processes/flows
> > > were rejected explicitly or if they just didn't come up?
> > >
> > > Dave
> > >
> > >
> > >
> > > On Tue, Mar 19, 2019 at 8:45 PM Maxime Beauchemin <
> > > maximebeauchemin@gmail.com> wrote:
> > >
> > > > Wondering what to do next, Justin, is it ok for me to push the
> related
> > > tag
> > > > to Github?
> > > >
> > > > Should I trigger a [VOTE] thread?
> > > >
> > > > Max
> > > >
> > > > On Tue, Mar 19, 2019 at 12:04 AM Maxime Beauchemin <
> > > > maximebeauchemin@gmail.com> wrote:
> > > >
> > > > > My svn is a bit rusty but I made some progress tonight:
> > > > > https://github.com/apache/incubator-superset/pull/7054
> > > > >
> > > > > Pushed some files to SVN
> > > > >
> > https://dist.apache.org/repos/dist/dev/incubator/superset/0.31.0rc18/
> > > > >
> > > > > It's not intended to be an actual real RC, but it's a start.
> > > > >
> > > > > An early question is around RC numbers. Lyft & Airbnb have been
> > doing a
> > > > > lot of back and forth on the release branch already and `0.31.0` is
> > > > already
> > > > > up to rc18. I was internally debating what to do here, keeping the
> > > > > numbering scheme, or starting anew. Probably the right thing to do
> is
> > > to
> > > > > pick a number (say 0.32.0) and start clean at 0.32.0rc1, and stick
> to
> > > the
> > > > > Apache process, and use continuous numbers. I'm assuming the
> handoff
> > is
> > > > at
> > > > > 0.31.0rc18.
> > > > >
> > > > > Now people that would want to work on non-Apache release branches
> > would
> > > > do
> > > > > so in their fork, and in their own namespace. Maybe their local
> build
> > > > > instructions can become recipes for an upcoming release, but I
> > suggest
> > > > that
> > > > > this happens in forks from now on.
> > > > >
> > > > > Clearly this Apache process here with signing + svn commit + voting
> > is
> > > > > more involved than the usual git cherry-pick + git tag + git push,
> so
> > > > that
> > > > > should probably lead to less RCs (I can't imagine doing 18 votes
> > > around a
> > > > > single release).
> > > > >
> > > > > Not to mention the fact that the real complexity around releasing
> is
> > > > > raising and labeling issues with a proper target version, gather
> PRs,
> > > > > cherry-pick fixes and resolve conflicts if any. I'm hoping we can
> > build
> > > > > tooling to help with all this. Hugh started something a while back,
> > but
> > > > > there's lots to be done still in that area.
> > > > >
> > > > > Max
> > > > >
> > > > > On Mon, Mar 18, 2019 at 11:39 PM Maxime Beauchemin <
> > > > > maximebeauchemin@gmail.com> wrote:
> > > > >
> > > > >> Not officially approved yet, I think we should start a [DISCUSS]
> > > thread
> > > > >> and eventually [VOTE]
> > > > >>
> > > > >> On Mon, Mar 18, 2019 at 10:13 PM David Smith <
> > dave.a.smith@gmail.com>
> > > > >> wrote:
> > > > >>
> > > > >>> Thanks Max! Out of curiosity, has that release SIP-12 been
> approved
> > > > yet?
> > > > >>> I
> > > > >>> have some thoughts but if it is already a done deal I'll wait
> until
> > > > >>> another
> > > > >>> time. :-)
> > > > >>>
> > > > >>> On Mon, Mar 18, 2019 at 9:53 PM Maxime Beauchemin <
> > > > >>> maximebeauchemin@gmail.com> wrote:
> > > > >>>
> > > > >>> > Hi all,
> > > > >>> >
> > > > >>> > I wanted to send an email explaining the current state of
> > releases
> > > > and
> > > > >>> > start a thread about what's ahead.
> > > > >>> >
> > > > >>> > Early on in the project lifecycle, I used to package releases
> and
> > > > push
> > > > >>> to
> > > > >>> > Pypi without much process. I'd package internal releases
> > internally
> > > > for
> > > > >>> > Airbnb. We'd roll out to staging, stabilize the release, and
> > launch
> > > > in
> > > > >>> > production. After a some time without major issues and
> > regressions,
> > > > I'd
> > > > >>> > push a new release out to Pypi and make it available to the
> > > > community.
> > > > >>> That
> > > > >>> > process worked ok when I reviewed or wrote most PRs and had a
> > > handle
> > > > on
> > > > >>> > everything that was going on. The community has grown much, and
> > > > clearly
> > > > >>> > that process has not been appropriate for quite some time now.
> > > > >>> >
> > > > >>> > Later on, we joined the Apache Software Foundation's incubator,
> > and
> > > > >>> the ASF
> > > > >>> > has clear process around releases
> > > > >>> > <http://www.apache.org/dev/release-publishing.html>. As an ASF
> > > > >>> project, it
> > > > >>> > was not reasonable for us to push to Pypi until we sorted out
> the
> > > > >>> process
> > > > >>> > and followed the "Apache Way". Since `0.29` we've still been
> > > cutting
> > > > >>> > release branches and labeling "release candidates", but have
> > > > refrained
> > > > >>> from
> > > > >>> > publishing to Pypi. Those release branches contain commits that
> > > have
> > > > >>> been
> > > > >>> > put into production at Lyft and/or Airbnb and elsewhere, but
> > didn't
> > > > >>> meet
> > > > >>> > the ASF's standards, so could not be published as releases.
> > > > >>> >
> > > > >>> > Also with the growing community, we clearly need a more
> > structured
> > > > >>> process
> > > > >>> > around releases. John Bodley wrote a solid SIP
> > > > >>> > <https://github.com/apache/incubator-superset/issues/6131>
> > > > proposing a
> > > > >>> > clear structure and process around releases, and details much
> of
> > > what
> > > > >>> the
> > > > >>> > ASF release process does not.
> > > > >>> >
> > > > >>> > All this to say, *I'm starting some work on our first
> > ASF-approved
> > > > >>> source
> > > > >>> > code release*. For context, the ASF makes a distinction between
> > > > "source
> > > > >>> > code releases" and "convenience releases". The former contains
> > the
> > > > >>> source
> > > > >>> > code and build instructions, and the latter contains binaries
> and
> > > is
> > > > >>> made
> > > > >>> > available in places like Pypi (or Maven / npm / RubyGem / ...).
> > > > >>> >
> > > > >>> > While the source code release should be pretty straight
> forward,
> > > it's
> > > > >>> not
> > > > >>> > really, well, convenient. You can't just "pip install" that...
> > The
> > > > >>> > convenience release on the other hand is great, but may require
> > > more
> > > > >>> work
> > > > >>> > and guidance from our mentors, ASF legal, license experts as
> the
> > > > >>> binaries
> > > > >>> > may contain our Javascript bundles (or maybe they don't?) but
> > then
> > > > >>> have to
> > > > >>> > have a gigantic LICENSE.txt file detailing the licenses of the
> > 500+
> > > > >>> > Javascript libs that have accumulated in our
> `package-lock.json`
> > > over
> > > > >>> time.
> > > > >>> > Or maybe we ship without the bundles and add a `superset
> compile`
> > > > >>> command
> > > > >>> > that does the fetching/build of the JS packages. Unclear to me.
> > > I'll
> > > > >>> need
> > > > >>> > help figuring this out. For context here's some work I did
> trying
> > > to
> > > > >>> > automate the LICENSE.txt creation
> > > > >>> > <https://github.com/apache/incubator-superset/pull/5801>
> > > > >>> >
> > > > >>> > Anyhow, wanted people to know what's up since it's been so long
> > > since
> > > > >>> the
> > > > >>> > last Pypi release, and give the opportunity for contributors to
> > > jump
> > > > >>> in and
> > > > >>> > help out.
> > > > >>> >
> > > > >>> > Let's get the [community] release process going!
> > > > >>> >
> > > > >>> > Max
> > > > >>> >
> > > > >>>
> > > > >>
> > > >
> > >
> >
>
--
*Krist Wongsuphasawat, PhD*
http://kristw.yellowpigz.com
Re: The state of releases
Posted by David Smith <da...@gmail.com>.
Thanks, Michelle. The reliance on cherry-picking as part of a work flow can
definitely bring the process into conflict with other tools and processes
in the git ecosystem. I've left a fairly lengthy comment on SIP-12
proposing that the mechanics be handled using gitflow, which does not use
cherry-picking, just for discussion.
https://github.com/apache/incubator-superset/issues/6131#issuecomment-475691949
On Wed, Mar 20, 2019 at 1:04 PM Michelle Thomas
<mi...@airbnb.com.invalid> wrote:
> To answer the question about using a workflow for managing
> releases/branches/tags, I previously evaluated using conventional commits:
> https://www.conventionalcommits.org/en/v1.0.0-beta.3/, which is a system
> for tagging commits to generate the changelog. At the time it seemed like
> it may have issues creating the Changelog with the way we commit everything
> to master then cherry-pick fixes onto a branch, but the consistency in
> tagging PRs with fix, feature, docs seemed useful. I don't think we've
> heavily discussed other workflows. Sounds good to get suggestions for
> improvement on SIP-12.
>
> On Tue, Mar 19, 2019 at 9:27 PM David Smith <da...@gmail.com>
> wrote:
>
> > I'm new to this project, so I apologize if this has been discussed
> > previously, but... Have we considered using a widely adopted workflow for
> > managing releases/branches/tags, and using tools that the community has
> > already built for them? For example Git Flow or some variant thereof?
> For
> > anyone who is unfamiliar, see overview and tooling--in the form of git
> > extensions--here: https://github.com/nvie/gitflow
> >
> > I saw SIP-12 and have a few comments that I will try to document tomorrow
> > on that issue, but I'm really curious whether other common
> processes/flows
> > were rejected explicitly or if they just didn't come up?
> >
> > Dave
> >
> >
> >
> > On Tue, Mar 19, 2019 at 8:45 PM Maxime Beauchemin <
> > maximebeauchemin@gmail.com> wrote:
> >
> > > Wondering what to do next, Justin, is it ok for me to push the related
> > tag
> > > to Github?
> > >
> > > Should I trigger a [VOTE] thread?
> > >
> > > Max
> > >
> > > On Tue, Mar 19, 2019 at 12:04 AM Maxime Beauchemin <
> > > maximebeauchemin@gmail.com> wrote:
> > >
> > > > My svn is a bit rusty but I made some progress tonight:
> > > > https://github.com/apache/incubator-superset/pull/7054
> > > >
> > > > Pushed some files to SVN
> > > >
> https://dist.apache.org/repos/dist/dev/incubator/superset/0.31.0rc18/
> > > >
> > > > It's not intended to be an actual real RC, but it's a start.
> > > >
> > > > An early question is around RC numbers. Lyft & Airbnb have been
> doing a
> > > > lot of back and forth on the release branch already and `0.31.0` is
> > > already
> > > > up to rc18. I was internally debating what to do here, keeping the
> > > > numbering scheme, or starting anew. Probably the right thing to do is
> > to
> > > > pick a number (say 0.32.0) and start clean at 0.32.0rc1, and stick to
> > the
> > > > Apache process, and use continuous numbers. I'm assuming the handoff
> is
> > > at
> > > > 0.31.0rc18.
> > > >
> > > > Now people that would want to work on non-Apache release branches
> would
> > > do
> > > > so in their fork, and in their own namespace. Maybe their local build
> > > > instructions can become recipes for an upcoming release, but I
> suggest
> > > that
> > > > this happens in forks from now on.
> > > >
> > > > Clearly this Apache process here with signing + svn commit + voting
> is
> > > > more involved than the usual git cherry-pick + git tag + git push, so
> > > that
> > > > should probably lead to less RCs (I can't imagine doing 18 votes
> > around a
> > > > single release).
> > > >
> > > > Not to mention the fact that the real complexity around releasing is
> > > > raising and labeling issues with a proper target version, gather PRs,
> > > > cherry-pick fixes and resolve conflicts if any. I'm hoping we can
> build
> > > > tooling to help with all this. Hugh started something a while back,
> but
> > > > there's lots to be done still in that area.
> > > >
> > > > Max
> > > >
> > > > On Mon, Mar 18, 2019 at 11:39 PM Maxime Beauchemin <
> > > > maximebeauchemin@gmail.com> wrote:
> > > >
> > > >> Not officially approved yet, I think we should start a [DISCUSS]
> > thread
> > > >> and eventually [VOTE]
> > > >>
> > > >> On Mon, Mar 18, 2019 at 10:13 PM David Smith <
> dave.a.smith@gmail.com>
> > > >> wrote:
> > > >>
> > > >>> Thanks Max! Out of curiosity, has that release SIP-12 been approved
> > > yet?
> > > >>> I
> > > >>> have some thoughts but if it is already a done deal I'll wait until
> > > >>> another
> > > >>> time. :-)
> > > >>>
> > > >>> On Mon, Mar 18, 2019 at 9:53 PM Maxime Beauchemin <
> > > >>> maximebeauchemin@gmail.com> wrote:
> > > >>>
> > > >>> > Hi all,
> > > >>> >
> > > >>> > I wanted to send an email explaining the current state of
> releases
> > > and
> > > >>> > start a thread about what's ahead.
> > > >>> >
> > > >>> > Early on in the project lifecycle, I used to package releases and
> > > push
> > > >>> to
> > > >>> > Pypi without much process. I'd package internal releases
> internally
> > > for
> > > >>> > Airbnb. We'd roll out to staging, stabilize the release, and
> launch
> > > in
> > > >>> > production. After a some time without major issues and
> regressions,
> > > I'd
> > > >>> > push a new release out to Pypi and make it available to the
> > > community.
> > > >>> That
> > > >>> > process worked ok when I reviewed or wrote most PRs and had a
> > handle
> > > on
> > > >>> > everything that was going on. The community has grown much, and
> > > clearly
> > > >>> > that process has not been appropriate for quite some time now.
> > > >>> >
> > > >>> > Later on, we joined the Apache Software Foundation's incubator,
> and
> > > >>> the ASF
> > > >>> > has clear process around releases
> > > >>> > <http://www.apache.org/dev/release-publishing.html>. As an ASF
> > > >>> project, it
> > > >>> > was not reasonable for us to push to Pypi until we sorted out the
> > > >>> process
> > > >>> > and followed the "Apache Way". Since `0.29` we've still been
> > cutting
> > > >>> > release branches and labeling "release candidates", but have
> > > refrained
> > > >>> from
> > > >>> > publishing to Pypi. Those release branches contain commits that
> > have
> > > >>> been
> > > >>> > put into production at Lyft and/or Airbnb and elsewhere, but
> didn't
> > > >>> meet
> > > >>> > the ASF's standards, so could not be published as releases.
> > > >>> >
> > > >>> > Also with the growing community, we clearly need a more
> structured
> > > >>> process
> > > >>> > around releases. John Bodley wrote a solid SIP
> > > >>> > <https://github.com/apache/incubator-superset/issues/6131>
> > > proposing a
> > > >>> > clear structure and process around releases, and details much of
> > what
> > > >>> the
> > > >>> > ASF release process does not.
> > > >>> >
> > > >>> > All this to say, *I'm starting some work on our first
> ASF-approved
> > > >>> source
> > > >>> > code release*. For context, the ASF makes a distinction between
> > > "source
> > > >>> > code releases" and "convenience releases". The former contains
> the
> > > >>> source
> > > >>> > code and build instructions, and the latter contains binaries and
> > is
> > > >>> made
> > > >>> > available in places like Pypi (or Maven / npm / RubyGem / ...).
> > > >>> >
> > > >>> > While the source code release should be pretty straight forward,
> > it's
> > > >>> not
> > > >>> > really, well, convenient. You can't just "pip install" that...
> The
> > > >>> > convenience release on the other hand is great, but may require
> > more
> > > >>> work
> > > >>> > and guidance from our mentors, ASF legal, license experts as the
> > > >>> binaries
> > > >>> > may contain our Javascript bundles (or maybe they don't?) but
> then
> > > >>> have to
> > > >>> > have a gigantic LICENSE.txt file detailing the licenses of the
> 500+
> > > >>> > Javascript libs that have accumulated in our `package-lock.json`
> > over
> > > >>> time.
> > > >>> > Or maybe we ship without the bundles and add a `superset compile`
> > > >>> command
> > > >>> > that does the fetching/build of the JS packages. Unclear to me.
> > I'll
> > > >>> need
> > > >>> > help figuring this out. For context here's some work I did trying
> > to
> > > >>> > automate the LICENSE.txt creation
> > > >>> > <https://github.com/apache/incubator-superset/pull/5801>
> > > >>> >
> > > >>> > Anyhow, wanted people to know what's up since it's been so long
> > since
> > > >>> the
> > > >>> > last Pypi release, and give the opportunity for contributors to
> > jump
> > > >>> in and
> > > >>> > help out.
> > > >>> >
> > > >>> > Let's get the [community] release process going!
> > > >>> >
> > > >>> > Max
> > > >>> >
> > > >>>
> > > >>
> > >
> >
>
Re: The state of releases
Posted by Michelle Thomas <mi...@airbnb.com.INVALID>.
To answer the question about using a workflow for managing
releases/branches/tags, I previously evaluated using conventional commits:
https://www.conventionalcommits.org/en/v1.0.0-beta.3/, which is a system
for tagging commits to generate the changelog. At the time it seemed like
it may have issues creating the Changelog with the way we commit everything
to master then cherry-pick fixes onto a branch, but the consistency in
tagging PRs with fix, feature, docs seemed useful. I don't think we've
heavily discussed other workflows. Sounds good to get suggestions for
improvement on SIP-12.
On Tue, Mar 19, 2019 at 9:27 PM David Smith <da...@gmail.com> wrote:
> I'm new to this project, so I apologize if this has been discussed
> previously, but... Have we considered using a widely adopted workflow for
> managing releases/branches/tags, and using tools that the community has
> already built for them? For example Git Flow or some variant thereof? For
> anyone who is unfamiliar, see overview and tooling--in the form of git
> extensions--here: https://github.com/nvie/gitflow
>
> I saw SIP-12 and have a few comments that I will try to document tomorrow
> on that issue, but I'm really curious whether other common processes/flows
> were rejected explicitly or if they just didn't come up?
>
> Dave
>
>
>
> On Tue, Mar 19, 2019 at 8:45 PM Maxime Beauchemin <
> maximebeauchemin@gmail.com> wrote:
>
> > Wondering what to do next, Justin, is it ok for me to push the related
> tag
> > to Github?
> >
> > Should I trigger a [VOTE] thread?
> >
> > Max
> >
> > On Tue, Mar 19, 2019 at 12:04 AM Maxime Beauchemin <
> > maximebeauchemin@gmail.com> wrote:
> >
> > > My svn is a bit rusty but I made some progress tonight:
> > > https://github.com/apache/incubator-superset/pull/7054
> > >
> > > Pushed some files to SVN
> > > https://dist.apache.org/repos/dist/dev/incubator/superset/0.31.0rc18/
> > >
> > > It's not intended to be an actual real RC, but it's a start.
> > >
> > > An early question is around RC numbers. Lyft & Airbnb have been doing a
> > > lot of back and forth on the release branch already and `0.31.0` is
> > already
> > > up to rc18. I was internally debating what to do here, keeping the
> > > numbering scheme, or starting anew. Probably the right thing to do is
> to
> > > pick a number (say 0.32.0) and start clean at 0.32.0rc1, and stick to
> the
> > > Apache process, and use continuous numbers. I'm assuming the handoff is
> > at
> > > 0.31.0rc18.
> > >
> > > Now people that would want to work on non-Apache release branches would
> > do
> > > so in their fork, and in their own namespace. Maybe their local build
> > > instructions can become recipes for an upcoming release, but I suggest
> > that
> > > this happens in forks from now on.
> > >
> > > Clearly this Apache process here with signing + svn commit + voting is
> > > more involved than the usual git cherry-pick + git tag + git push, so
> > that
> > > should probably lead to less RCs (I can't imagine doing 18 votes
> around a
> > > single release).
> > >
> > > Not to mention the fact that the real complexity around releasing is
> > > raising and labeling issues with a proper target version, gather PRs,
> > > cherry-pick fixes and resolve conflicts if any. I'm hoping we can build
> > > tooling to help with all this. Hugh started something a while back, but
> > > there's lots to be done still in that area.
> > >
> > > Max
> > >
> > > On Mon, Mar 18, 2019 at 11:39 PM Maxime Beauchemin <
> > > maximebeauchemin@gmail.com> wrote:
> > >
> > >> Not officially approved yet, I think we should start a [DISCUSS]
> thread
> > >> and eventually [VOTE]
> > >>
> > >> On Mon, Mar 18, 2019 at 10:13 PM David Smith <da...@gmail.com>
> > >> wrote:
> > >>
> > >>> Thanks Max! Out of curiosity, has that release SIP-12 been approved
> > yet?
> > >>> I
> > >>> have some thoughts but if it is already a done deal I'll wait until
> > >>> another
> > >>> time. :-)
> > >>>
> > >>> On Mon, Mar 18, 2019 at 9:53 PM Maxime Beauchemin <
> > >>> maximebeauchemin@gmail.com> wrote:
> > >>>
> > >>> > Hi all,
> > >>> >
> > >>> > I wanted to send an email explaining the current state of releases
> > and
> > >>> > start a thread about what's ahead.
> > >>> >
> > >>> > Early on in the project lifecycle, I used to package releases and
> > push
> > >>> to
> > >>> > Pypi without much process. I'd package internal releases internally
> > for
> > >>> > Airbnb. We'd roll out to staging, stabilize the release, and launch
> > in
> > >>> > production. After a some time without major issues and regressions,
> > I'd
> > >>> > push a new release out to Pypi and make it available to the
> > community.
> > >>> That
> > >>> > process worked ok when I reviewed or wrote most PRs and had a
> handle
> > on
> > >>> > everything that was going on. The community has grown much, and
> > clearly
> > >>> > that process has not been appropriate for quite some time now.
> > >>> >
> > >>> > Later on, we joined the Apache Software Foundation's incubator, and
> > >>> the ASF
> > >>> > has clear process around releases
> > >>> > <http://www.apache.org/dev/release-publishing.html>. As an ASF
> > >>> project, it
> > >>> > was not reasonable for us to push to Pypi until we sorted out the
> > >>> process
> > >>> > and followed the "Apache Way". Since `0.29` we've still been
> cutting
> > >>> > release branches and labeling "release candidates", but have
> > refrained
> > >>> from
> > >>> > publishing to Pypi. Those release branches contain commits that
> have
> > >>> been
> > >>> > put into production at Lyft and/or Airbnb and elsewhere, but didn't
> > >>> meet
> > >>> > the ASF's standards, so could not be published as releases.
> > >>> >
> > >>> > Also with the growing community, we clearly need a more structured
> > >>> process
> > >>> > around releases. John Bodley wrote a solid SIP
> > >>> > <https://github.com/apache/incubator-superset/issues/6131>
> > proposing a
> > >>> > clear structure and process around releases, and details much of
> what
> > >>> the
> > >>> > ASF release process does not.
> > >>> >
> > >>> > All this to say, *I'm starting some work on our first ASF-approved
> > >>> source
> > >>> > code release*. For context, the ASF makes a distinction between
> > "source
> > >>> > code releases" and "convenience releases". The former contains the
> > >>> source
> > >>> > code and build instructions, and the latter contains binaries and
> is
> > >>> made
> > >>> > available in places like Pypi (or Maven / npm / RubyGem / ...).
> > >>> >
> > >>> > While the source code release should be pretty straight forward,
> it's
> > >>> not
> > >>> > really, well, convenient. You can't just "pip install" that... The
> > >>> > convenience release on the other hand is great, but may require
> more
> > >>> work
> > >>> > and guidance from our mentors, ASF legal, license experts as the
> > >>> binaries
> > >>> > may contain our Javascript bundles (or maybe they don't?) but then
> > >>> have to
> > >>> > have a gigantic LICENSE.txt file detailing the licenses of the 500+
> > >>> > Javascript libs that have accumulated in our `package-lock.json`
> over
> > >>> time.
> > >>> > Or maybe we ship without the bundles and add a `superset compile`
> > >>> command
> > >>> > that does the fetching/build of the JS packages. Unclear to me.
> I'll
> > >>> need
> > >>> > help figuring this out. For context here's some work I did trying
> to
> > >>> > automate the LICENSE.txt creation
> > >>> > <https://github.com/apache/incubator-superset/pull/5801>
> > >>> >
> > >>> > Anyhow, wanted people to know what's up since it's been so long
> since
> > >>> the
> > >>> > last Pypi release, and give the opportunity for contributors to
> jump
> > >>> in and
> > >>> > help out.
> > >>> >
> > >>> > Let's get the [community] release process going!
> > >>> >
> > >>> > Max
> > >>> >
> > >>>
> > >>
> >
>
Re: The state of releases
Posted by David Smith <da...@gmail.com>.
Fair, and you are correct, pypi explicitly forbids that representation, but I
think there are separable concerns here:
1) what is the Superset version?
2) how is that version represented in published artifacts across
potentially multiple consuming tech stacks?
I realize opinions may vary, but for the I personally would avoid coupling
the two questions. Publication artifacts may include tarballs, pypi
packages, npm packages, could even someday include client libs for other
things like java/maven (who knows). If and when necessary, the Superset
release version can be transformed in a manner that is required for
specific consumers (ie you can turn 1.2.3-rc1 into 1.2.3rc1 when you push
to pypi).
I think there are a lot of advantages to opting into semver, in terms of
existing documentation and tooling, git workflows, etc.
On Wed, Mar 20, 2019 at 10:43 AM Don Bowman <do...@agilicus.com> wrote:
> On Wed, 20 Mar 2019 at 13:40, Maxime Beauchemin <
> maximebeauchemin@gmail.com>
> wrote:
>
> > Python's PEP-440 says otherwise:
> > https://www.python.org/dev/peps/pep-0440/#pre-releases
> >
> > I'm not 100% on this but if I remember correctly Pypi will reject that
> > specific flavor of semver-compliant string. (ie: 0.32.0-rc1)
> >
> >
> >
> This is a specific area where semver is incompatible w/ pep440 and one
> should use pep440 w/ python.
> https://github.com/pypa/pipenv/issues/3220
>
> you can see how other projects have tried to rationalise, eg..
> https://docs.openstack.org/pbr/3.1.1/semver.html
>
Re: The state of releases
Posted by Don Bowman <do...@agilicus.com>.
On Wed, 20 Mar 2019 at 13:40, Maxime Beauchemin <ma...@gmail.com>
wrote:
> Python's PEP-440 says otherwise:
> https://www.python.org/dev/peps/pep-0440/#pre-releases
>
> I'm not 100% on this but if I remember correctly Pypi will reject that
> specific flavor of semver-compliant string. (ie: 0.32.0-rc1)
>
>
>
This is a specific area where semver is incompatible w/ pep440 and one
should use pep440 w/ python.
https://github.com/pypa/pipenv/issues/3220
you can see how other projects have tried to rationalise, eg..
https://docs.openstack.org/pbr/3.1.1/semver.html
Re: The state of releases
Posted by Maxime Beauchemin <ma...@gmail.com>.
Python's PEP-440 says otherwise:
https://www.python.org/dev/peps/pep-0440/#pre-releases
I'm not 100% on this but if I remember correctly Pypi will reject that
specific flavor of semver-compliant string. (ie: 0.32.0-rc1)
On Tue, Mar 19, 2019 at 9:37 PM David Smith <da...@gmail.com> wrote:
> One more note: to be semver compliant, the version string "0.32.0rc1"
> should be changed to "0.32.0-rc1" (see rule #9 at https://semver.org)
>
> On Tue, Mar 19, 2019 at 9:26 PM David Smith <da...@gmail.com>
> wrote:
>
> > I'm new to this project, so I apologize if this has been discussed
> > previously, but... Have we considered using a widely adopted workflow for
> > managing releases/branches/tags, and using tools that the community has
> > already built for them? For example Git Flow or some variant thereof?
> For
> > anyone who is unfamiliar, see overview and tooling--in the form of git
> > extensions--here: https://github.com/nvie/gitflow
> >
> > I saw SIP-12 and have a few comments that I will try to document tomorrow
> > on that issue, but I'm really curious whether other common
> processes/flows
> > were rejected explicitly or if they just didn't come up?
> >
> > Dave
> >
> >
> >
> > On Tue, Mar 19, 2019 at 8:45 PM Maxime Beauchemin <
> > maximebeauchemin@gmail.com> wrote:
> >
> >> Wondering what to do next, Justin, is it ok for me to push the related
> tag
> >> to Github?
> >>
> >> Should I trigger a [VOTE] thread?
> >>
> >> Max
> >>
> >> On Tue, Mar 19, 2019 at 12:04 AM Maxime Beauchemin <
> >> maximebeauchemin@gmail.com> wrote:
> >>
> >> > My svn is a bit rusty but I made some progress tonight:
> >> > https://github.com/apache/incubator-superset/pull/7054
> >> >
> >> > Pushed some files to SVN
> >> > https://dist.apache.org/repos/dist/dev/incubator/superset/0.31.0rc18/
> >> >
> >> > It's not intended to be an actual real RC, but it's a start.
> >> >
> >> > An early question is around RC numbers. Lyft & Airbnb have been doing
> a
> >> > lot of back and forth on the release branch already and `0.31.0` is
> >> already
> >> > up to rc18. I was internally debating what to do here, keeping the
> >> > numbering scheme, or starting anew. Probably the right thing to do is
> to
> >> > pick a number (say 0.32.0) and start clean at 0.32.0rc1, and stick to
> >> the
> >> > Apache process, and use continuous numbers. I'm assuming the handoff
> is
> >> at
> >> > 0.31.0rc18.
> >> >
> >> > Now people that would want to work on non-Apache release branches
> would
> >> do
> >> > so in their fork, and in their own namespace. Maybe their local build
> >> > instructions can become recipes for an upcoming release, but I suggest
> >> that
> >> > this happens in forks from now on.
> >> >
> >> > Clearly this Apache process here with signing + svn commit + voting is
> >> > more involved than the usual git cherry-pick + git tag + git push, so
> >> that
> >> > should probably lead to less RCs (I can't imagine doing 18 votes
> around
> >> a
> >> > single release).
> >> >
> >> > Not to mention the fact that the real complexity around releasing is
> >> > raising and labeling issues with a proper target version, gather PRs,
> >> > cherry-pick fixes and resolve conflicts if any. I'm hoping we can
> build
> >> > tooling to help with all this. Hugh started something a while back,
> but
> >> > there's lots to be done still in that area.
> >> >
> >> > Max
> >> >
> >> > On Mon, Mar 18, 2019 at 11:39 PM Maxime Beauchemin <
> >> > maximebeauchemin@gmail.com> wrote:
> >> >
> >> >> Not officially approved yet, I think we should start a [DISCUSS]
> thread
> >> >> and eventually [VOTE]
> >> >>
> >> >> On Mon, Mar 18, 2019 at 10:13 PM David Smith <dave.a.smith@gmail.com
> >
> >> >> wrote:
> >> >>
> >> >>> Thanks Max! Out of curiosity, has that release SIP-12 been approved
> >> yet?
> >> >>> I
> >> >>> have some thoughts but if it is already a done deal I'll wait until
> >> >>> another
> >> >>> time. :-)
> >> >>>
> >> >>> On Mon, Mar 18, 2019 at 9:53 PM Maxime Beauchemin <
> >> >>> maximebeauchemin@gmail.com> wrote:
> >> >>>
> >> >>> > Hi all,
> >> >>> >
> >> >>> > I wanted to send an email explaining the current state of releases
> >> and
> >> >>> > start a thread about what's ahead.
> >> >>> >
> >> >>> > Early on in the project lifecycle, I used to package releases and
> >> push
> >> >>> to
> >> >>> > Pypi without much process. I'd package internal releases
> internally
> >> for
> >> >>> > Airbnb. We'd roll out to staging, stabilize the release, and
> launch
> >> in
> >> >>> > production. After a some time without major issues and
> regressions,
> >> I'd
> >> >>> > push a new release out to Pypi and make it available to the
> >> community.
> >> >>> That
> >> >>> > process worked ok when I reviewed or wrote most PRs and had a
> >> handle on
> >> >>> > everything that was going on. The community has grown much, and
> >> clearly
> >> >>> > that process has not been appropriate for quite some time now.
> >> >>> >
> >> >>> > Later on, we joined the Apache Software Foundation's incubator,
> and
> >> >>> the ASF
> >> >>> > has clear process around releases
> >> >>> > <http://www.apache.org/dev/release-publishing.html>. As an ASF
> >> >>> project, it
> >> >>> > was not reasonable for us to push to Pypi until we sorted out the
> >> >>> process
> >> >>> > and followed the "Apache Way". Since `0.29` we've still been
> cutting
> >> >>> > release branches and labeling "release candidates", but have
> >> refrained
> >> >>> from
> >> >>> > publishing to Pypi. Those release branches contain commits that
> have
> >> >>> been
> >> >>> > put into production at Lyft and/or Airbnb and elsewhere, but
> didn't
> >> >>> meet
> >> >>> > the ASF's standards, so could not be published as releases.
> >> >>> >
> >> >>> > Also with the growing community, we clearly need a more structured
> >> >>> process
> >> >>> > around releases. John Bodley wrote a solid SIP
> >> >>> > <https://github.com/apache/incubator-superset/issues/6131>
> >> proposing a
> >> >>> > clear structure and process around releases, and details much of
> >> what
> >> >>> the
> >> >>> > ASF release process does not.
> >> >>> >
> >> >>> > All this to say, *I'm starting some work on our first ASF-approved
> >> >>> source
> >> >>> > code release*. For context, the ASF makes a distinction between
> >> "source
> >> >>> > code releases" and "convenience releases". The former contains the
> >> >>> source
> >> >>> > code and build instructions, and the latter contains binaries and
> is
> >> >>> made
> >> >>> > available in places like Pypi (or Maven / npm / RubyGem / ...).
> >> >>> >
> >> >>> > While the source code release should be pretty straight forward,
> >> it's
> >> >>> not
> >> >>> > really, well, convenient. You can't just "pip install" that... The
> >> >>> > convenience release on the other hand is great, but may require
> more
> >> >>> work
> >> >>> > and guidance from our mentors, ASF legal, license experts as the
> >> >>> binaries
> >> >>> > may contain our Javascript bundles (or maybe they don't?) but then
> >> >>> have to
> >> >>> > have a gigantic LICENSE.txt file detailing the licenses of the
> 500+
> >> >>> > Javascript libs that have accumulated in our `package-lock.json`
> >> over
> >> >>> time.
> >> >>> > Or maybe we ship without the bundles and add a `superset compile`
> >> >>> command
> >> >>> > that does the fetching/build of the JS packages. Unclear to me.
> I'll
> >> >>> need
> >> >>> > help figuring this out. For context here's some work I did trying
> to
> >> >>> > automate the LICENSE.txt creation
> >> >>> > <https://github.com/apache/incubator-superset/pull/5801>
> >> >>> >
> >> >>> > Anyhow, wanted people to know what's up since it's been so long
> >> since
> >> >>> the
> >> >>> > last Pypi release, and give the opportunity for contributors to
> jump
> >> >>> in and
> >> >>> > help out.
> >> >>> >
> >> >>> > Let's get the [community] release process going!
> >> >>> >
> >> >>> > Max
> >> >>> >
> >> >>>
> >> >>
> >>
> >
>
Re: The state of releases
Posted by David Smith <da...@gmail.com>.
One more note: to be semver compliant, the version string "0.32.0rc1"
should be changed to "0.32.0-rc1" (see rule #9 at https://semver.org)
On Tue, Mar 19, 2019 at 9:26 PM David Smith <da...@gmail.com> wrote:
> I'm new to this project, so I apologize if this has been discussed
> previously, but... Have we considered using a widely adopted workflow for
> managing releases/branches/tags, and using tools that the community has
> already built for them? For example Git Flow or some variant thereof? For
> anyone who is unfamiliar, see overview and tooling--in the form of git
> extensions--here: https://github.com/nvie/gitflow
>
> I saw SIP-12 and have a few comments that I will try to document tomorrow
> on that issue, but I'm really curious whether other common processes/flows
> were rejected explicitly or if they just didn't come up?
>
> Dave
>
>
>
> On Tue, Mar 19, 2019 at 8:45 PM Maxime Beauchemin <
> maximebeauchemin@gmail.com> wrote:
>
>> Wondering what to do next, Justin, is it ok for me to push the related tag
>> to Github?
>>
>> Should I trigger a [VOTE] thread?
>>
>> Max
>>
>> On Tue, Mar 19, 2019 at 12:04 AM Maxime Beauchemin <
>> maximebeauchemin@gmail.com> wrote:
>>
>> > My svn is a bit rusty but I made some progress tonight:
>> > https://github.com/apache/incubator-superset/pull/7054
>> >
>> > Pushed some files to SVN
>> > https://dist.apache.org/repos/dist/dev/incubator/superset/0.31.0rc18/
>> >
>> > It's not intended to be an actual real RC, but it's a start.
>> >
>> > An early question is around RC numbers. Lyft & Airbnb have been doing a
>> > lot of back and forth on the release branch already and `0.31.0` is
>> already
>> > up to rc18. I was internally debating what to do here, keeping the
>> > numbering scheme, or starting anew. Probably the right thing to do is to
>> > pick a number (say 0.32.0) and start clean at 0.32.0rc1, and stick to
>> the
>> > Apache process, and use continuous numbers. I'm assuming the handoff is
>> at
>> > 0.31.0rc18.
>> >
>> > Now people that would want to work on non-Apache release branches would
>> do
>> > so in their fork, and in their own namespace. Maybe their local build
>> > instructions can become recipes for an upcoming release, but I suggest
>> that
>> > this happens in forks from now on.
>> >
>> > Clearly this Apache process here with signing + svn commit + voting is
>> > more involved than the usual git cherry-pick + git tag + git push, so
>> that
>> > should probably lead to less RCs (I can't imagine doing 18 votes around
>> a
>> > single release).
>> >
>> > Not to mention the fact that the real complexity around releasing is
>> > raising and labeling issues with a proper target version, gather PRs,
>> > cherry-pick fixes and resolve conflicts if any. I'm hoping we can build
>> > tooling to help with all this. Hugh started something a while back, but
>> > there's lots to be done still in that area.
>> >
>> > Max
>> >
>> > On Mon, Mar 18, 2019 at 11:39 PM Maxime Beauchemin <
>> > maximebeauchemin@gmail.com> wrote:
>> >
>> >> Not officially approved yet, I think we should start a [DISCUSS] thread
>> >> and eventually [VOTE]
>> >>
>> >> On Mon, Mar 18, 2019 at 10:13 PM David Smith <da...@gmail.com>
>> >> wrote:
>> >>
>> >>> Thanks Max! Out of curiosity, has that release SIP-12 been approved
>> yet?
>> >>> I
>> >>> have some thoughts but if it is already a done deal I'll wait until
>> >>> another
>> >>> time. :-)
>> >>>
>> >>> On Mon, Mar 18, 2019 at 9:53 PM Maxime Beauchemin <
>> >>> maximebeauchemin@gmail.com> wrote:
>> >>>
>> >>> > Hi all,
>> >>> >
>> >>> > I wanted to send an email explaining the current state of releases
>> and
>> >>> > start a thread about what's ahead.
>> >>> >
>> >>> > Early on in the project lifecycle, I used to package releases and
>> push
>> >>> to
>> >>> > Pypi without much process. I'd package internal releases internally
>> for
>> >>> > Airbnb. We'd roll out to staging, stabilize the release, and launch
>> in
>> >>> > production. After a some time without major issues and regressions,
>> I'd
>> >>> > push a new release out to Pypi and make it available to the
>> community.
>> >>> That
>> >>> > process worked ok when I reviewed or wrote most PRs and had a
>> handle on
>> >>> > everything that was going on. The community has grown much, and
>> clearly
>> >>> > that process has not been appropriate for quite some time now.
>> >>> >
>> >>> > Later on, we joined the Apache Software Foundation's incubator, and
>> >>> the ASF
>> >>> > has clear process around releases
>> >>> > <http://www.apache.org/dev/release-publishing.html>. As an ASF
>> >>> project, it
>> >>> > was not reasonable for us to push to Pypi until we sorted out the
>> >>> process
>> >>> > and followed the "Apache Way". Since `0.29` we've still been cutting
>> >>> > release branches and labeling "release candidates", but have
>> refrained
>> >>> from
>> >>> > publishing to Pypi. Those release branches contain commits that have
>> >>> been
>> >>> > put into production at Lyft and/or Airbnb and elsewhere, but didn't
>> >>> meet
>> >>> > the ASF's standards, so could not be published as releases.
>> >>> >
>> >>> > Also with the growing community, we clearly need a more structured
>> >>> process
>> >>> > around releases. John Bodley wrote a solid SIP
>> >>> > <https://github.com/apache/incubator-superset/issues/6131>
>> proposing a
>> >>> > clear structure and process around releases, and details much of
>> what
>> >>> the
>> >>> > ASF release process does not.
>> >>> >
>> >>> > All this to say, *I'm starting some work on our first ASF-approved
>> >>> source
>> >>> > code release*. For context, the ASF makes a distinction between
>> "source
>> >>> > code releases" and "convenience releases". The former contains the
>> >>> source
>> >>> > code and build instructions, and the latter contains binaries and is
>> >>> made
>> >>> > available in places like Pypi (or Maven / npm / RubyGem / ...).
>> >>> >
>> >>> > While the source code release should be pretty straight forward,
>> it's
>> >>> not
>> >>> > really, well, convenient. You can't just "pip install" that... The
>> >>> > convenience release on the other hand is great, but may require more
>> >>> work
>> >>> > and guidance from our mentors, ASF legal, license experts as the
>> >>> binaries
>> >>> > may contain our Javascript bundles (or maybe they don't?) but then
>> >>> have to
>> >>> > have a gigantic LICENSE.txt file detailing the licenses of the 500+
>> >>> > Javascript libs that have accumulated in our `package-lock.json`
>> over
>> >>> time.
>> >>> > Or maybe we ship without the bundles and add a `superset compile`
>> >>> command
>> >>> > that does the fetching/build of the JS packages. Unclear to me. I'll
>> >>> need
>> >>> > help figuring this out. For context here's some work I did trying to
>> >>> > automate the LICENSE.txt creation
>> >>> > <https://github.com/apache/incubator-superset/pull/5801>
>> >>> >
>> >>> > Anyhow, wanted people to know what's up since it's been so long
>> since
>> >>> the
>> >>> > last Pypi release, and give the opportunity for contributors to jump
>> >>> in and
>> >>> > help out.
>> >>> >
>> >>> > Let's get the [community] release process going!
>> >>> >
>> >>> > Max
>> >>> >
>> >>>
>> >>
>>
>
Re: The state of releases
Posted by David Smith <da...@gmail.com>.
I'm new to this project, so I apologize if this has been discussed
previously, but... Have we considered using a widely adopted workflow for
managing releases/branches/tags, and using tools that the community has
already built for them? For example Git Flow or some variant thereof? For
anyone who is unfamiliar, see overview and tooling--in the form of git
extensions--here: https://github.com/nvie/gitflow
I saw SIP-12 and have a few comments that I will try to document tomorrow
on that issue, but I'm really curious whether other common processes/flows
were rejected explicitly or if they just didn't come up?
Dave
On Tue, Mar 19, 2019 at 8:45 PM Maxime Beauchemin <
maximebeauchemin@gmail.com> wrote:
> Wondering what to do next, Justin, is it ok for me to push the related tag
> to Github?
>
> Should I trigger a [VOTE] thread?
>
> Max
>
> On Tue, Mar 19, 2019 at 12:04 AM Maxime Beauchemin <
> maximebeauchemin@gmail.com> wrote:
>
> > My svn is a bit rusty but I made some progress tonight:
> > https://github.com/apache/incubator-superset/pull/7054
> >
> > Pushed some files to SVN
> > https://dist.apache.org/repos/dist/dev/incubator/superset/0.31.0rc18/
> >
> > It's not intended to be an actual real RC, but it's a start.
> >
> > An early question is around RC numbers. Lyft & Airbnb have been doing a
> > lot of back and forth on the release branch already and `0.31.0` is
> already
> > up to rc18. I was internally debating what to do here, keeping the
> > numbering scheme, or starting anew. Probably the right thing to do is to
> > pick a number (say 0.32.0) and start clean at 0.32.0rc1, and stick to the
> > Apache process, and use continuous numbers. I'm assuming the handoff is
> at
> > 0.31.0rc18.
> >
> > Now people that would want to work on non-Apache release branches would
> do
> > so in their fork, and in their own namespace. Maybe their local build
> > instructions can become recipes for an upcoming release, but I suggest
> that
> > this happens in forks from now on.
> >
> > Clearly this Apache process here with signing + svn commit + voting is
> > more involved than the usual git cherry-pick + git tag + git push, so
> that
> > should probably lead to less RCs (I can't imagine doing 18 votes around a
> > single release).
> >
> > Not to mention the fact that the real complexity around releasing is
> > raising and labeling issues with a proper target version, gather PRs,
> > cherry-pick fixes and resolve conflicts if any. I'm hoping we can build
> > tooling to help with all this. Hugh started something a while back, but
> > there's lots to be done still in that area.
> >
> > Max
> >
> > On Mon, Mar 18, 2019 at 11:39 PM Maxime Beauchemin <
> > maximebeauchemin@gmail.com> wrote:
> >
> >> Not officially approved yet, I think we should start a [DISCUSS] thread
> >> and eventually [VOTE]
> >>
> >> On Mon, Mar 18, 2019 at 10:13 PM David Smith <da...@gmail.com>
> >> wrote:
> >>
> >>> Thanks Max! Out of curiosity, has that release SIP-12 been approved
> yet?
> >>> I
> >>> have some thoughts but if it is already a done deal I'll wait until
> >>> another
> >>> time. :-)
> >>>
> >>> On Mon, Mar 18, 2019 at 9:53 PM Maxime Beauchemin <
> >>> maximebeauchemin@gmail.com> wrote:
> >>>
> >>> > Hi all,
> >>> >
> >>> > I wanted to send an email explaining the current state of releases
> and
> >>> > start a thread about what's ahead.
> >>> >
> >>> > Early on in the project lifecycle, I used to package releases and
> push
> >>> to
> >>> > Pypi without much process. I'd package internal releases internally
> for
> >>> > Airbnb. We'd roll out to staging, stabilize the release, and launch
> in
> >>> > production. After a some time without major issues and regressions,
> I'd
> >>> > push a new release out to Pypi and make it available to the
> community.
> >>> That
> >>> > process worked ok when I reviewed or wrote most PRs and had a handle
> on
> >>> > everything that was going on. The community has grown much, and
> clearly
> >>> > that process has not been appropriate for quite some time now.
> >>> >
> >>> > Later on, we joined the Apache Software Foundation's incubator, and
> >>> the ASF
> >>> > has clear process around releases
> >>> > <http://www.apache.org/dev/release-publishing.html>. As an ASF
> >>> project, it
> >>> > was not reasonable for us to push to Pypi until we sorted out the
> >>> process
> >>> > and followed the "Apache Way". Since `0.29` we've still been cutting
> >>> > release branches and labeling "release candidates", but have
> refrained
> >>> from
> >>> > publishing to Pypi. Those release branches contain commits that have
> >>> been
> >>> > put into production at Lyft and/or Airbnb and elsewhere, but didn't
> >>> meet
> >>> > the ASF's standards, so could not be published as releases.
> >>> >
> >>> > Also with the growing community, we clearly need a more structured
> >>> process
> >>> > around releases. John Bodley wrote a solid SIP
> >>> > <https://github.com/apache/incubator-superset/issues/6131>
> proposing a
> >>> > clear structure and process around releases, and details much of what
> >>> the
> >>> > ASF release process does not.
> >>> >
> >>> > All this to say, *I'm starting some work on our first ASF-approved
> >>> source
> >>> > code release*. For context, the ASF makes a distinction between
> "source
> >>> > code releases" and "convenience releases". The former contains the
> >>> source
> >>> > code and build instructions, and the latter contains binaries and is
> >>> made
> >>> > available in places like Pypi (or Maven / npm / RubyGem / ...).
> >>> >
> >>> > While the source code release should be pretty straight forward, it's
> >>> not
> >>> > really, well, convenient. You can't just "pip install" that... The
> >>> > convenience release on the other hand is great, but may require more
> >>> work
> >>> > and guidance from our mentors, ASF legal, license experts as the
> >>> binaries
> >>> > may contain our Javascript bundles (or maybe they don't?) but then
> >>> have to
> >>> > have a gigantic LICENSE.txt file detailing the licenses of the 500+
> >>> > Javascript libs that have accumulated in our `package-lock.json` over
> >>> time.
> >>> > Or maybe we ship without the bundles and add a `superset compile`
> >>> command
> >>> > that does the fetching/build of the JS packages. Unclear to me. I'll
> >>> need
> >>> > help figuring this out. For context here's some work I did trying to
> >>> > automate the LICENSE.txt creation
> >>> > <https://github.com/apache/incubator-superset/pull/5801>
> >>> >
> >>> > Anyhow, wanted people to know what's up since it's been so long since
> >>> the
> >>> > last Pypi release, and give the opportunity for contributors to jump
> >>> in and
> >>> > help out.
> >>> >
> >>> > Let's get the [community] release process going!
> >>> >
> >>> > Max
> >>> >
> >>>
> >>
>
Re: The state of releases
Posted by Maxime Beauchemin <ma...@gmail.com>.
Wondering what to do next, Justin, is it ok for me to push the related tag
to Github?
Should I trigger a [VOTE] thread?
Max
On Tue, Mar 19, 2019 at 12:04 AM Maxime Beauchemin <
maximebeauchemin@gmail.com> wrote:
> My svn is a bit rusty but I made some progress tonight:
> https://github.com/apache/incubator-superset/pull/7054
>
> Pushed some files to SVN
> https://dist.apache.org/repos/dist/dev/incubator/superset/0.31.0rc18/
>
> It's not intended to be an actual real RC, but it's a start.
>
> An early question is around RC numbers. Lyft & Airbnb have been doing a
> lot of back and forth on the release branch already and `0.31.0` is already
> up to rc18. I was internally debating what to do here, keeping the
> numbering scheme, or starting anew. Probably the right thing to do is to
> pick a number (say 0.32.0) and start clean at 0.32.0rc1, and stick to the
> Apache process, and use continuous numbers. I'm assuming the handoff is at
> 0.31.0rc18.
>
> Now people that would want to work on non-Apache release branches would do
> so in their fork, and in their own namespace. Maybe their local build
> instructions can become recipes for an upcoming release, but I suggest that
> this happens in forks from now on.
>
> Clearly this Apache process here with signing + svn commit + voting is
> more involved than the usual git cherry-pick + git tag + git push, so that
> should probably lead to less RCs (I can't imagine doing 18 votes around a
> single release).
>
> Not to mention the fact that the real complexity around releasing is
> raising and labeling issues with a proper target version, gather PRs,
> cherry-pick fixes and resolve conflicts if any. I'm hoping we can build
> tooling to help with all this. Hugh started something a while back, but
> there's lots to be done still in that area.
>
> Max
>
> On Mon, Mar 18, 2019 at 11:39 PM Maxime Beauchemin <
> maximebeauchemin@gmail.com> wrote:
>
>> Not officially approved yet, I think we should start a [DISCUSS] thread
>> and eventually [VOTE]
>>
>> On Mon, Mar 18, 2019 at 10:13 PM David Smith <da...@gmail.com>
>> wrote:
>>
>>> Thanks Max! Out of curiosity, has that release SIP-12 been approved yet?
>>> I
>>> have some thoughts but if it is already a done deal I'll wait until
>>> another
>>> time. :-)
>>>
>>> On Mon, Mar 18, 2019 at 9:53 PM Maxime Beauchemin <
>>> maximebeauchemin@gmail.com> wrote:
>>>
>>> > Hi all,
>>> >
>>> > I wanted to send an email explaining the current state of releases and
>>> > start a thread about what's ahead.
>>> >
>>> > Early on in the project lifecycle, I used to package releases and push
>>> to
>>> > Pypi without much process. I'd package internal releases internally for
>>> > Airbnb. We'd roll out to staging, stabilize the release, and launch in
>>> > production. After a some time without major issues and regressions, I'd
>>> > push a new release out to Pypi and make it available to the community.
>>> That
>>> > process worked ok when I reviewed or wrote most PRs and had a handle on
>>> > everything that was going on. The community has grown much, and clearly
>>> > that process has not been appropriate for quite some time now.
>>> >
>>> > Later on, we joined the Apache Software Foundation's incubator, and
>>> the ASF
>>> > has clear process around releases
>>> > <http://www.apache.org/dev/release-publishing.html>. As an ASF
>>> project, it
>>> > was not reasonable for us to push to Pypi until we sorted out the
>>> process
>>> > and followed the "Apache Way". Since `0.29` we've still been cutting
>>> > release branches and labeling "release candidates", but have refrained
>>> from
>>> > publishing to Pypi. Those release branches contain commits that have
>>> been
>>> > put into production at Lyft and/or Airbnb and elsewhere, but didn't
>>> meet
>>> > the ASF's standards, so could not be published as releases.
>>> >
>>> > Also with the growing community, we clearly need a more structured
>>> process
>>> > around releases. John Bodley wrote a solid SIP
>>> > <https://github.com/apache/incubator-superset/issues/6131> proposing a
>>> > clear structure and process around releases, and details much of what
>>> the
>>> > ASF release process does not.
>>> >
>>> > All this to say, *I'm starting some work on our first ASF-approved
>>> source
>>> > code release*. For context, the ASF makes a distinction between "source
>>> > code releases" and "convenience releases". The former contains the
>>> source
>>> > code and build instructions, and the latter contains binaries and is
>>> made
>>> > available in places like Pypi (or Maven / npm / RubyGem / ...).
>>> >
>>> > While the source code release should be pretty straight forward, it's
>>> not
>>> > really, well, convenient. You can't just "pip install" that... The
>>> > convenience release on the other hand is great, but may require more
>>> work
>>> > and guidance from our mentors, ASF legal, license experts as the
>>> binaries
>>> > may contain our Javascript bundles (or maybe they don't?) but then
>>> have to
>>> > have a gigantic LICENSE.txt file detailing the licenses of the 500+
>>> > Javascript libs that have accumulated in our `package-lock.json` over
>>> time.
>>> > Or maybe we ship without the bundles and add a `superset compile`
>>> command
>>> > that does the fetching/build of the JS packages. Unclear to me. I'll
>>> need
>>> > help figuring this out. For context here's some work I did trying to
>>> > automate the LICENSE.txt creation
>>> > <https://github.com/apache/incubator-superset/pull/5801>
>>> >
>>> > Anyhow, wanted people to know what's up since it's been so long since
>>> the
>>> > last Pypi release, and give the opportunity for contributors to jump
>>> in and
>>> > help out.
>>> >
>>> > Let's get the [community] release process going!
>>> >
>>> > Max
>>> >
>>>
>>
Re: The state of releases
Posted by Maxime Beauchemin <ma...@gmail.com>.
My svn is a bit rusty but I made some progress tonight:
https://github.com/apache/incubator-superset/pull/7054
Pushed some files to SVN
https://dist.apache.org/repos/dist/dev/incubator/superset/0.31.0rc18/
It's not intended to be an actual real RC, but it's a start.
An early question is around RC numbers. Lyft & Airbnb have been doing a lot
of back and forth on the release branch already and `0.31.0` is already up
to rc18. I was internally debating what to do here, keeping the numbering
scheme, or starting anew. Probably the right thing to do is to pick a
number (say 0.32.0) and start clean at 0.32.0rc1, and stick to the Apache
process, and use continuous numbers. I'm assuming the handoff is at
0.31.0rc18.
Now people that would want to work on non-Apache release branches would do
so in their fork, and in their own namespace. Maybe their local build
instructions can become recipes for an upcoming release, but I suggest that
this happens in forks from now on.
Clearly this Apache process here with signing + svn commit + voting is more
involved than the usual git cherry-pick + git tag + git push, so that
should probably lead to less RCs (I can't imagine doing 18 votes around a
single release).
Not to mention the fact that the real complexity around releasing is
raising and labeling issues with a proper target version, gather PRs,
cherry-pick fixes and resolve conflicts if any. I'm hoping we can build
tooling to help with all this. Hugh started something a while back, but
there's lots to be done still in that area.
Max
On Mon, Mar 18, 2019 at 11:39 PM Maxime Beauchemin <
maximebeauchemin@gmail.com> wrote:
> Not officially approved yet, I think we should start a [DISCUSS] thread
> and eventually [VOTE]
>
> On Mon, Mar 18, 2019 at 10:13 PM David Smith <da...@gmail.com>
> wrote:
>
>> Thanks Max! Out of curiosity, has that release SIP-12 been approved yet? I
>> have some thoughts but if it is already a done deal I'll wait until
>> another
>> time. :-)
>>
>> On Mon, Mar 18, 2019 at 9:53 PM Maxime Beauchemin <
>> maximebeauchemin@gmail.com> wrote:
>>
>> > Hi all,
>> >
>> > I wanted to send an email explaining the current state of releases and
>> > start a thread about what's ahead.
>> >
>> > Early on in the project lifecycle, I used to package releases and push
>> to
>> > Pypi without much process. I'd package internal releases internally for
>> > Airbnb. We'd roll out to staging, stabilize the release, and launch in
>> > production. After a some time without major issues and regressions, I'd
>> > push a new release out to Pypi and make it available to the community.
>> That
>> > process worked ok when I reviewed or wrote most PRs and had a handle on
>> > everything that was going on. The community has grown much, and clearly
>> > that process has not been appropriate for quite some time now.
>> >
>> > Later on, we joined the Apache Software Foundation's incubator, and the
>> ASF
>> > has clear process around releases
>> > <http://www.apache.org/dev/release-publishing.html>. As an ASF
>> project, it
>> > was not reasonable for us to push to Pypi until we sorted out the
>> process
>> > and followed the "Apache Way". Since `0.29` we've still been cutting
>> > release branches and labeling "release candidates", but have refrained
>> from
>> > publishing to Pypi. Those release branches contain commits that have
>> been
>> > put into production at Lyft and/or Airbnb and elsewhere, but didn't meet
>> > the ASF's standards, so could not be published as releases.
>> >
>> > Also with the growing community, we clearly need a more structured
>> process
>> > around releases. John Bodley wrote a solid SIP
>> > <https://github.com/apache/incubator-superset/issues/6131> proposing a
>> > clear structure and process around releases, and details much of what
>> the
>> > ASF release process does not.
>> >
>> > All this to say, *I'm starting some work on our first ASF-approved
>> source
>> > code release*. For context, the ASF makes a distinction between "source
>> > code releases" and "convenience releases". The former contains the
>> source
>> > code and build instructions, and the latter contains binaries and is
>> made
>> > available in places like Pypi (or Maven / npm / RubyGem / ...).
>> >
>> > While the source code release should be pretty straight forward, it's
>> not
>> > really, well, convenient. You can't just "pip install" that... The
>> > convenience release on the other hand is great, but may require more
>> work
>> > and guidance from our mentors, ASF legal, license experts as the
>> binaries
>> > may contain our Javascript bundles (or maybe they don't?) but then have
>> to
>> > have a gigantic LICENSE.txt file detailing the licenses of the 500+
>> > Javascript libs that have accumulated in our `package-lock.json` over
>> time.
>> > Or maybe we ship without the bundles and add a `superset compile`
>> command
>> > that does the fetching/build of the JS packages. Unclear to me. I'll
>> need
>> > help figuring this out. For context here's some work I did trying to
>> > automate the LICENSE.txt creation
>> > <https://github.com/apache/incubator-superset/pull/5801>
>> >
>> > Anyhow, wanted people to know what's up since it's been so long since
>> the
>> > last Pypi release, and give the opportunity for contributors to jump in
>> and
>> > help out.
>> >
>> > Let's get the [community] release process going!
>> >
>> > Max
>> >
>>
>
Re: The state of releases
Posted by Maxime Beauchemin <ma...@gmail.com>.
Not officially approved yet, I think we should start a [DISCUSS] thread and
eventually [VOTE]
On Mon, Mar 18, 2019 at 10:13 PM David Smith <da...@gmail.com> wrote:
> Thanks Max! Out of curiosity, has that release SIP-12 been approved yet? I
> have some thoughts but if it is already a done deal I'll wait until another
> time. :-)
>
> On Mon, Mar 18, 2019 at 9:53 PM Maxime Beauchemin <
> maximebeauchemin@gmail.com> wrote:
>
> > Hi all,
> >
> > I wanted to send an email explaining the current state of releases and
> > start a thread about what's ahead.
> >
> > Early on in the project lifecycle, I used to package releases and push to
> > Pypi without much process. I'd package internal releases internally for
> > Airbnb. We'd roll out to staging, stabilize the release, and launch in
> > production. After a some time without major issues and regressions, I'd
> > push a new release out to Pypi and make it available to the community.
> That
> > process worked ok when I reviewed or wrote most PRs and had a handle on
> > everything that was going on. The community has grown much, and clearly
> > that process has not been appropriate for quite some time now.
> >
> > Later on, we joined the Apache Software Foundation's incubator, and the
> ASF
> > has clear process around releases
> > <http://www.apache.org/dev/release-publishing.html>. As an ASF project,
> it
> > was not reasonable for us to push to Pypi until we sorted out the process
> > and followed the "Apache Way". Since `0.29` we've still been cutting
> > release branches and labeling "release candidates", but have refrained
> from
> > publishing to Pypi. Those release branches contain commits that have been
> > put into production at Lyft and/or Airbnb and elsewhere, but didn't meet
> > the ASF's standards, so could not be published as releases.
> >
> > Also with the growing community, we clearly need a more structured
> process
> > around releases. John Bodley wrote a solid SIP
> > <https://github.com/apache/incubator-superset/issues/6131> proposing a
> > clear structure and process around releases, and details much of what the
> > ASF release process does not.
> >
> > All this to say, *I'm starting some work on our first ASF-approved source
> > code release*. For context, the ASF makes a distinction between "source
> > code releases" and "convenience releases". The former contains the source
> > code and build instructions, and the latter contains binaries and is made
> > available in places like Pypi (or Maven / npm / RubyGem / ...).
> >
> > While the source code release should be pretty straight forward, it's not
> > really, well, convenient. You can't just "pip install" that... The
> > convenience release on the other hand is great, but may require more work
> > and guidance from our mentors, ASF legal, license experts as the binaries
> > may contain our Javascript bundles (or maybe they don't?) but then have
> to
> > have a gigantic LICENSE.txt file detailing the licenses of the 500+
> > Javascript libs that have accumulated in our `package-lock.json` over
> time.
> > Or maybe we ship without the bundles and add a `superset compile` command
> > that does the fetching/build of the JS packages. Unclear to me. I'll need
> > help figuring this out. For context here's some work I did trying to
> > automate the LICENSE.txt creation
> > <https://github.com/apache/incubator-superset/pull/5801>
> >
> > Anyhow, wanted people to know what's up since it's been so long since the
> > last Pypi release, and give the opportunity for contributors to jump in
> and
> > help out.
> >
> > Let's get the [community] release process going!
> >
> > Max
> >
>
Re: The state of releases
Posted by David Smith <da...@gmail.com>.
Thanks Max! Out of curiosity, has that release SIP-12 been approved yet? I
have some thoughts but if it is already a done deal I'll wait until another
time. :-)
On Mon, Mar 18, 2019 at 9:53 PM Maxime Beauchemin <
maximebeauchemin@gmail.com> wrote:
> Hi all,
>
> I wanted to send an email explaining the current state of releases and
> start a thread about what's ahead.
>
> Early on in the project lifecycle, I used to package releases and push to
> Pypi without much process. I'd package internal releases internally for
> Airbnb. We'd roll out to staging, stabilize the release, and launch in
> production. After a some time without major issues and regressions, I'd
> push a new release out to Pypi and make it available to the community. That
> process worked ok when I reviewed or wrote most PRs and had a handle on
> everything that was going on. The community has grown much, and clearly
> that process has not been appropriate for quite some time now.
>
> Later on, we joined the Apache Software Foundation's incubator, and the ASF
> has clear process around releases
> <http://www.apache.org/dev/release-publishing.html>. As an ASF project, it
> was not reasonable for us to push to Pypi until we sorted out the process
> and followed the "Apache Way". Since `0.29` we've still been cutting
> release branches and labeling "release candidates", but have refrained from
> publishing to Pypi. Those release branches contain commits that have been
> put into production at Lyft and/or Airbnb and elsewhere, but didn't meet
> the ASF's standards, so could not be published as releases.
>
> Also with the growing community, we clearly need a more structured process
> around releases. John Bodley wrote a solid SIP
> <https://github.com/apache/incubator-superset/issues/6131> proposing a
> clear structure and process around releases, and details much of what the
> ASF release process does not.
>
> All this to say, *I'm starting some work on our first ASF-approved source
> code release*. For context, the ASF makes a distinction between "source
> code releases" and "convenience releases". The former contains the source
> code and build instructions, and the latter contains binaries and is made
> available in places like Pypi (or Maven / npm / RubyGem / ...).
>
> While the source code release should be pretty straight forward, it's not
> really, well, convenient. You can't just "pip install" that... The
> convenience release on the other hand is great, but may require more work
> and guidance from our mentors, ASF legal, license experts as the binaries
> may contain our Javascript bundles (or maybe they don't?) but then have to
> have a gigantic LICENSE.txt file detailing the licenses of the 500+
> Javascript libs that have accumulated in our `package-lock.json` over time.
> Or maybe we ship without the bundles and add a `superset compile` command
> that does the fetching/build of the JS packages. Unclear to me. I'll need
> help figuring this out. For context here's some work I did trying to
> automate the LICENSE.txt creation
> <https://github.com/apache/incubator-superset/pull/5801>
>
> Anyhow, wanted people to know what's up since it's been so long since the
> last Pypi release, and give the opportunity for contributors to jump in and
> help out.
>
> Let's get the [community] release process going!
>
> Max
>