You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Jason Gustafson <ja...@confluent.io> on 2016/09/07 03:22:25 UTC
Re: [DISCUSS] Time-based releases for Apache Kafka
Hey All,
It sounds like the general consensus is in favor of time-based releases. We
can continue the discussion about LTS, but I wanted to go ahead and get
things moving forward by volunteering to manage the next release, which is
currently slated for October. If that sounds OK, I'll draft a release plan
and send it out to the community for feedback and a vote.
Thanks,
Jason
On Thu, Aug 25, 2016 at 2:03 PM, Ofir Manor <of...@equalum.io> wrote:
> I happily agree that Kafka is a solid and the community is great :)
> But I think there is a gap in perception here.
> For me, LTS means that someone is actively taking care of a release -
> actively backporting critical fixes (security, stability, data loss,
> corruption, hangs etc) from trunk to that LTS version periodically for an
> extended period of time, for example 18-36 months... So people can really
> rely on the same Kafka version for a long time.
> Is someone doing it today for 0.9.0? When is 0.9.0.2 expected? When is
> 0.8.2.3 expected? Will they cover all known critical issues for whoever
> relies on them in production?
> In other words, what is the scope of support that the community want to
> commit for older versions? (upgrade compatibility? investigating bug
> reports? proactively backporting fixes?)
> BTW, another legit option is that the Apache Kafka project won't commit to
> LTS releases. It could let commercial vendors compete on supporting very
> old versions. I find that actually quite reasonable as well.
>
> Ofir Manor
>
> Co-Founder & CTO | Equalum
>
> Mobile: +972-54-7801286 | Email: ofir.manor@equalum.io
>
> On Thu, Aug 25, 2016 at 8:19 PM, Andrew Schofield <
> andrew_schofield_jira@outlook.com> wrote:
>
> > I agree that the Kafka community has managed to maintain a very high
> > quality level, so I'm not concerned
> > about the quality of non-LTS releases. If the principle is that every
> > release is supported for 2 years, that
> > would be good. I suppose that if the burden of having that many
> in-support
> > releases proves too heavy,
> > as you say we could reconsider.
> >
> > Andrew Schofield
> >
> > ----------------------------------------
> > > From: gwen@confluent.io
> > > Date: Thu, 25 Aug 2016 09:57:30 -0700
> > > Subject: Re: [DISCUSS] Time-based releases for Apache Kafka
> > > To: dev@kafka.apache.org
> > >
> > > I prefer Ismael's suggestion for supporting 2-years (6 releases)
> > > rather than have designated LTS releases.
> > >
> > > The LTS model seems to work well when some releases are high quality
> > > (LTS) and the rest are a bit more questionable. It is great for
> > > companies like Redhat, where they have to invest less to support few
> > > releases and let the community deal with everything else.
> > >
> > > Until now the Kafka community has managed to maintain very high
> > > quality level. Not just for releases, our trunk is often of better
> > > quality than other project's releases - we don't think of stability as
> > > something you tuck into a release (and just some releases) but rather
> > > as an on-going concern. There are costs to doing things that way, but
> > > in general, I think it has served us well - allowing even conservative
> > > companies to run on the latest released version.
> > >
> > > I hope we can agree to at least try maintaining last 6 releases as LTS
> > > (i.e. every single release is supported for 2 years) rather than
> > > designate some releases as better than others. Of course, if this
> > > totally fails, we can reconsider.
> > >
> > > Gwen
> > >
> > > On Thu, Aug 25, 2016 at 9:51 AM, Andrew Schofield
> > > <an...@outlook.com> wrote:
> > >> The proposal sounds pretty good, but the main thing currently missing
> > is a proper long-term support release.
> > >>
> > >> Having 3 releases a year sounds OK, but if they're all equivalent and
> > bugfix releases are produced for the most
> > >> recent 2 or 3 releases, anyone wanting to run on an "in support"
> > release of Kafka has to upgrade every 8-12 months.
> > >> If you don't actually want anything specific from the newer releases,
> > it's just unnecessary churn.
> > >>
> > >> Wouldn't it be better to designate one release every 12-18 months as a
> > long-term support release with bugfix releases
> > >> produced for those for a longer period of say 24 months. That halves
> > the upgrade work for people just wanting to keep
> > >> "in support". Now that adoption is increasing, there are plenty of
> > users that just want a dependable messaging system
> > >> without having to be deeply knowledgeable about its innards.
> > >>
> > >> LTS works nicely for plenty of open-source projects. I think it would
> > work well for Kafka too.
> > >>
> > >> Andrew Schofield
> > >>
> > >> ----------------------------------------
> > >>> From: ofir.manor@equalum.io
> > >>> Date: Thu, 25 Aug 2016 16:07:07 +0300
> > >>> Subject: Re: [DISCUSS] Time-based releases for Apache Kafka
> > >>> To: dev@kafka.apache.org
> > >>>
> > >>> Regarding bug fixes, you may want to consider to have an LTS release
> > once a
> > >>> year - designating it for longer-term support / better for the
> masses.
> > >>> If you like that - then fix bugs in trunk, backport important ones to
> > >>> latest release + the last two LTS releases.
> > >>> Even if you don't - if a downstream distribution picks a Kafka
> version
> > and
> > >>> plans to support it over a few years, it could be nice of them to
> "own"
> > >>> that older release - volunteer to be a release manager for bug
> > backports to
> > >>> that version over a longer period...
> > >>> Just my two cents :)
> > >>>
> > >>> Ofir Manor
> > >>>
> > >>> Co-Founder & CTO | Equalum
> > >>>
> > >>> Mobile: +972-54-7801286 | Email: ofir.manor@equalum.io
> > >>>
> > >>> On Thu, Aug 25, 2016 at 12:32 PM, Ismael Juma <is...@juma.me.uk>
> > wrote:
> > >>>
> > >>>> Thanks for putting this together Gwen. I think it sounds reasonable
> > and
> > >>>> instead of trying to optimise every aspect of it ahead of time
> (which
> > is
> > >>>> hard, subjective and time-consuming), I am happy to try what is
> being
> > >>>> proposed and tweak based on experience. One thing we should pay
> > particular
> > >>>> attention to is how the stabilisation process works out in practice.
> > >>>>
> > >>>> A couple of comments:
> > >>>>
> > >>>> "Given 3 releases a year and the fact that no one upgrades three
> > times a
> > >>>> year, we propose making sure (by testing!) that rolling upgrade can
> > be done
> > >>>> from each release in the past year (i.e. last 3 releases) to the
> > latest
> > >>>> version."
> > >>>>
> > >>>> Because the cost of doing this for a larger number of releases is
> > >>>> relatively low, I still think we should go for 6 here (our code
> > currently
> > >>>> supports 5 versions as I said in a previous message, so we're close
> > to that
> > >>>> target already). I'm generally very keen to make upgrades as easy as
> > >>>> possible so that people have no reason not to upgrade. :)
> > >>>>
> > >>>> "We will also attempt, as a community to do bugfix releases as
> needed
> > for
> > >>>> the last 3 releases."
> > >>>>
> > >>>> I would suggest 2, personally, but since this is a bit fuzzy, I am
> OK
> > with
> > >>>> 3 if people prefer that.
> > >>>>
> > >>>> Ismael
> > >>>>
> > >>>> On Thu, Aug 25, 2016 at 6:22 AM, Gwen Shapira <gw...@confluent.io>
> > wrote:
> > >>>>
> > >>>>> Hi Team Kafka,
> > >>>>>
> > >>>>> As per the KIP meeting, I created a wiki:
> > >>>>> https://cwiki.apache.org/confluence/display/KAFKA/Time+
> > >>>> Based+Release+Plan
> > >>>>> Summarizing most of the discussion so far.
> > >>>>>
> > >>>>> Comments and additional discussion is welcome :)
> > >>>>>
> > >>>>> Gwen
> > >>>>>
> > >>>>> On Wed, Aug 17, 2016 at 12:31 PM, Vahid S Hashemian
> > >>>>> <va...@us.ibm.com> wrote:
> > >>>>>> Time-based releases is a good idea and something that has proved
> to
> > be
> > >>>>>> working in a number of open source projects. One successful
> example
> > is
> > >>>>>> Node.js, that goes through two major releases a year. The
> > interesting
> > >>>>> fact
> > >>>>>> about the two releases is that only one (the even-number release)
> > comes
> > >>>>>> with a long term support (LTS) plan (30 months). More can be read
> > here:
> > >>>>>> https://github.com/nodejs/LTS. The odd-number releases still come
> > with
> > >>>>>> major changes and help build the ecosystem, but as far as LTS
> goes,
> > >>>> there
> > >>>>>> is only one per year. This LTS plan makes most enterprises want to
> > >>>> stick
> > >>>>>> to even-number releases, which is okay since frequent upgrades is
> > not
> > >>>>>> something they are normally interested in anyway.
> > >>>>>>
> > >>>>>> There could be several minor releases (non-breaking) in between
> > major
> > >>>>>> releases. A major release contains all the features / bug fixes in
> > the
> > >>>>>> master branch a month before the release date, with the potential
> > >>>>> addition
> > >>>>>> of (non-breaking) bug fixes until the release day. The deprecation
> > >>>> cycle
> > >>>>>> is one major release: any functionality that is decided to be
> > removed
> > >>>> is
> > >>>>>> deprecated in the next major release, and removed in the major
> > release
> > >>>>>> after that.
> > >>>>>>
> > >>>>>> Because of the success of LTS models in this and other open source
> > >>>>>> projects, I would suggest implementing a formal LTS plan for Kafka
> > too.
> > >>>>>>
> > >>>>>> Regards,
> > >>>>>> --Vahid
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> From: Gwen Shapira <gw...@confluent.io>
> > >>>>>> To: dev@kafka.apache.org
> > >>>>>> Date: 08/09/2016 04:49 PM
> > >>>>>> Subject: [DISCUSS] Time-based releases for Apache Kafka
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> Dear Kafka Developers and Users,
> > >>>>>>
> > >>>>>> In the past, our releases have been quite unpredictable. We'll
> > notice
> > >>>>>> that a large number of nice features made it in (or are close),
> > >>>>>> someone would suggest a release and we'd do it. This is fun, but
> > makes
> > >>>>>> planning really hard - we saw it during the last release which we
> > >>>>>> decided to delay by a few weeks to allow more features to "land".
> > >>>>>>
> > >>>>>> Many other communities have adopted time-based releases
> successfully
> > >>>>>> (Cassandra, GCC, LLVM, Fedora, Gnome, Ubuntu, etc.). And I thought
> > it
> > >>>>>> will make sense for the Apache Kafka community to try doing the
> > same.
> > >>>>>>
> > >>>>>> The benefits of this approach are:
> > >>>>>>
> > >>>>>> 1. A quicker feedback cycle and users can benefit from features
> > >>>>>> quicker (assuming for reasonably short time between releases - I
> was
> > >>>>>> thinking 4 months)
> > >>>>>>
> > >>>>>> 2. Predictability for contributors and users:
> > >>>>>> * Developers and reviewers can decide in advance what release they
> > are
> > >>>>>> aiming for with specific features.
> > >>>>>> * If a feature misses a release we have a good idea of when it
> will
> > >>>> show
> > >>>>>> up.
> > >>>>>> * Users know when to expect their features
> > >>>>>>
> > >>>>>> 3. Transparency - There will be a published cut-off date (AKA
> > feature
> > >>>>>> freeze) for the release and people will know about it in advance.
> > >>>>>> Hopefully this will remove the contention around which features
> make
> > >>>>>> it.
> > >>>>>>
> > >>>>>> 4. Quality - we've seen issues pop up in release candidates due to
> > >>>>>> last-minute features that didn't have proper time to bake in. More
> > >>>>>> time between feature freeze and release will let us test more,
> > >>>>>> document more and resolve more issues.
> > >>>>>>
> > >>>>>> Since nothing is ever perfect, there will be some downsides:
> > >>>>>>
> > >>>>>> 1. Most notably, features that miss the feature-freeze date for a
> > >>>>>> release will have to wait few month for the next release. Features
> > >>>>>> will reach users faster overall as per benefit #1, but individual
> > >>>>>> features that just miss the cut will lose out
> > >>>>>>
> > >>>>>> 2. More releases a year mean that being a committer is more work -
> > >>>>>> release management is still some headache and we'll have more of
> > >>>>>> those. Hopefully we'll get better at it. Also, the committer list
> is
> > >>>>>> growing and hopefully it will be less than once-a-year effort for
> > each
> > >>>>>> committer.
> > >>>>>>
> > >>>>>> 3. For users, figuring out which release to use and having
> frequent
> > >>>>>> new releases to upgrade to may be a bit confusing.
> > >>>>>>
> > >>>>>> 4. Frequent releases mean we need to do bugfix releases for older
> > >>>>>> branches. Right now we only do bugfix releases to latest release.
> > >>>>>>
> > >>>>>> I think the benefits outweigh the drawbacks. Or at least suggest
> > that
> > >>>>>> its worth trying - we can have another discussion in few releases
> to
> > >>>>>> see if we want to keep it that way or try something else.
> > >>>>>>
> > >>>>>> My suggestion for the process:
> > >>>>>>
> > >>>>>> 1. We decide on a reasonable release cadence
> > >>>>>> 2. We decide on release dates (even rough estimate such as "end of
> > >>>>>> February" or something) and work back feature freeze dates.
> > >>>>>> 3. Committers volunteer to be "release managers" for specific
> > >>>>>> releases. We can coordinate on the list or on a wiki. If no
> > committer
> > >>>>>> volunteers, we assume the community doesn't need a release and
> skip
> > >>>>>> it.
> > >>>>>> 4. At the "feature freeze" date, the release manager announces the
> > >>>>>> contents of the release (which KIPs made it in on time), creates
> the
> > >>>>>> release branch and starts the release process as usual. From this
> > >>>>>> point onwards, only bug fixes should be double-committed to the
> > >>>>>> release branch while trunk can start collecting features for the
> > >>>>>> subsequent release.
> > >>>>>>
> > >>>>>> Comments and improvements are appreciated.
> > >>>>>>
> > >>>>>> Gwen Shapira
> > >>>>>> Former-release-manager
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> --
> > >>>>> Gwen Shapira
> > >>>>> Product Manager | Confluent
> > >>>>> 650.450.2760 | @gwenshap
> > >>>>> Follow us: Twitter | blog
> > >>>>>
> > >>>>
> > >>
> > >
> > >
> > >
> > > --
> > > Gwen Shapira
> > > Product Manager | Confluent
> > > 650.450.2760 | @gwenshap
> > > Follow us: Twitter | blog
> >
> >
>
Re: [DISCUSS] Time-based releases for Apache Kafka
Posted by Neha Narkhede <ne...@confluent.io>.
Thanks for volunteering, Jason!
On Wed, Sep 7, 2016 at 1:59 AM Ismael Juma <is...@juma.me.uk> wrote:
> Thanks for volunteering Jason. Sounds good to me,
>
> Ismael
>
> On Wed, Sep 7, 2016 at 4:22 AM, Jason Gustafson <ja...@confluent.io>
> wrote:
>
> > Hey All,
> >
> > It sounds like the general consensus is in favor of time-based releases.
> We
> > can continue the discussion about LTS, but I wanted to go ahead and get
> > things moving forward by volunteering to manage the next release, which
> is
> > currently slated for October. If that sounds OK, I'll draft a release
> plan
> > and send it out to the community for feedback and a vote.
> >
> > Thanks,
> > Jason
> >
> > On Thu, Aug 25, 2016 at 2:03 PM, Ofir Manor <of...@equalum.io>
> wrote:
> >
> > > I happily agree that Kafka is a solid and the community is great :)
> > > But I think there is a gap in perception here.
> > > For me, LTS means that someone is actively taking care of a release -
> > > actively backporting critical fixes (security, stability, data loss,
> > > corruption, hangs etc) from trunk to that LTS version periodically for
> an
> > > extended period of time, for example 18-36 months... So people can
> really
> > > rely on the same Kafka version for a long time.
> > > Is someone doing it today for 0.9.0? When is 0.9.0.2 expected? When is
> > > 0.8.2.3 expected? Will they cover all known critical issues for whoever
> > > relies on them in production?
> > > In other words, what is the scope of support that the community want to
> > > commit for older versions? (upgrade compatibility? investigating bug
> > > reports? proactively backporting fixes?)
> > > BTW, another legit option is that the Apache Kafka project won't commit
> > to
> > > LTS releases. It could let commercial vendors compete on supporting
> very
> > > old versions. I find that actually quite reasonable as well.
> > >
> > > Ofir Manor
> > >
> > > Co-Founder & CTO | Equalum
> > >
> > > Mobile: +972-54-7801286 | Email: ofir.manor@equalum.io
> > >
> > > On Thu, Aug 25, 2016 at 8:19 PM, Andrew Schofield <
> > > andrew_schofield_jira@outlook.com> wrote:
> > >
> > > > I agree that the Kafka community has managed to maintain a very high
> > > > quality level, so I'm not concerned
> > > > about the quality of non-LTS releases. If the principle is that every
> > > > release is supported for 2 years, that
> > > > would be good. I suppose that if the burden of having that many
> > > in-support
> > > > releases proves too heavy,
> > > > as you say we could reconsider.
> > > >
> > > > Andrew Schofield
> > > >
> > > > ----------------------------------------
> > > > > From: gwen@confluent.io
> > > > > Date: Thu, 25 Aug 2016 09:57:30 -0700
> > > > > Subject: Re: [DISCUSS] Time-based releases for Apache Kafka
> > > > > To: dev@kafka.apache.org
> > > > >
> > > > > I prefer Ismael's suggestion for supporting 2-years (6 releases)
> > > > > rather than have designated LTS releases.
> > > > >
> > > > > The LTS model seems to work well when some releases are high
> quality
> > > > > (LTS) and the rest are a bit more questionable. It is great for
> > > > > companies like Redhat, where they have to invest less to support
> few
> > > > > releases and let the community deal with everything else.
> > > > >
> > > > > Until now the Kafka community has managed to maintain very high
> > > > > quality level. Not just for releases, our trunk is often of better
> > > > > quality than other project's releases - we don't think of stability
> > as
> > > > > something you tuck into a release (and just some releases) but
> rather
> > > > > as an on-going concern. There are costs to doing things that way,
> but
> > > > > in general, I think it has served us well - allowing even
> > conservative
> > > > > companies to run on the latest released version.
> > > > >
> > > > > I hope we can agree to at least try maintaining last 6 releases as
> > LTS
> > > > > (i.e. every single release is supported for 2 years) rather than
> > > > > designate some releases as better than others. Of course, if this
> > > > > totally fails, we can reconsider.
> > > > >
> > > > > Gwen
> > > > >
> > > > > On Thu, Aug 25, 2016 at 9:51 AM, Andrew Schofield
> > > > > <an...@outlook.com> wrote:
> > > > >> The proposal sounds pretty good, but the main thing currently
> > missing
> > > > is a proper long-term support release.
> > > > >>
> > > > >> Having 3 releases a year sounds OK, but if they're all equivalent
> > and
> > > > bugfix releases are produced for the most
> > > > >> recent 2 or 3 releases, anyone wanting to run on an "in support"
> > > > release of Kafka has to upgrade every 8-12 months.
> > > > >> If you don't actually want anything specific from the newer
> > releases,
> > > > it's just unnecessary churn.
> > > > >>
> > > > >> Wouldn't it be better to designate one release every 12-18 months
> > as a
> > > > long-term support release with bugfix releases
> > > > >> produced for those for a longer period of say 24 months. That
> halves
> > > > the upgrade work for people just wanting to keep
> > > > >> "in support". Now that adoption is increasing, there are plenty of
> > > > users that just want a dependable messaging system
> > > > >> without having to be deeply knowledgeable about its innards.
> > > > >>
> > > > >> LTS works nicely for plenty of open-source projects. I think it
> > would
> > > > work well for Kafka too.
> > > > >>
> > > > >> Andrew Schofield
> > > > >>
> > > > >> ----------------------------------------
> > > > >>> From: ofir.manor@equalum.io
> > > > >>> Date: Thu, 25 Aug 2016 16:07:07 +0300
> > > > >>> Subject: Re: [DISCUSS] Time-based releases for Apache Kafka
> > > > >>> To: dev@kafka.apache.org
> > > > >>>
> > > > >>> Regarding bug fixes, you may want to consider to have an LTS
> > release
> > > > once a
> > > > >>> year - designating it for longer-term support / better for the
> > > masses.
> > > > >>> If you like that - then fix bugs in trunk, backport important
> ones
> > to
> > > > >>> latest release + the last two LTS releases.
> > > > >>> Even if you don't - if a downstream distribution picks a Kafka
> > > version
> > > > and
> > > > >>> plans to support it over a few years, it could be nice of them to
> > > "own"
> > > > >>> that older release - volunteer to be a release manager for bug
> > > > backports to
> > > > >>> that version over a longer period...
> > > > >>> Just my two cents :)
> > > > >>>
> > > > >>> Ofir Manor
> > > > >>>
> > > > >>> Co-Founder & CTO | Equalum
> > > > >>>
> > > > >>> Mobile: +972-54-7801286 | Email: ofir.manor@equalum.io
> > > > >>>
> > > > >>> On Thu, Aug 25, 2016 at 12:32 PM, Ismael Juma <ismael@juma.me.uk
> >
> > > > wrote:
> > > > >>>
> > > > >>>> Thanks for putting this together Gwen. I think it sounds
> > reasonable
> > > > and
> > > > >>>> instead of trying to optimise every aspect of it ahead of time
> > > (which
> > > > is
> > > > >>>> hard, subjective and time-consuming), I am happy to try what is
> > > being
> > > > >>>> proposed and tweak based on experience. One thing we should pay
> > > > particular
> > > > >>>> attention to is how the stabilisation process works out in
> > practice.
> > > > >>>>
> > > > >>>> A couple of comments:
> > > > >>>>
> > > > >>>> "Given 3 releases a year and the fact that no one upgrades three
> > > > times a
> > > > >>>> year, we propose making sure (by testing!) that rolling upgrade
> > can
> > > > be done
> > > > >>>> from each release in the past year (i.e. last 3 releases) to the
> > > > latest
> > > > >>>> version."
> > > > >>>>
> > > > >>>> Because the cost of doing this for a larger number of releases
> is
> > > > >>>> relatively low, I still think we should go for 6 here (our code
> > > > currently
> > > > >>>> supports 5 versions as I said in a previous message, so we're
> > close
> > > > to that
> > > > >>>> target already). I'm generally very keen to make upgrades as
> easy
> > as
> > > > >>>> possible so that people have no reason not to upgrade. :)
> > > > >>>>
> > > > >>>> "We will also attempt, as a community to do bugfix releases as
> > > needed
> > > > for
> > > > >>>> the last 3 releases."
> > > > >>>>
> > > > >>>> I would suggest 2, personally, but since this is a bit fuzzy, I
> am
> > > OK
> > > > with
> > > > >>>> 3 if people prefer that.
> > > > >>>>
> > > > >>>> Ismael
> > > > >>>>
> > > > >>>> On Thu, Aug 25, 2016 at 6:22 AM, Gwen Shapira <
> gwen@confluent.io>
> > > > wrote:
> > > > >>>>
> > > > >>>>> Hi Team Kafka,
> > > > >>>>>
> > > > >>>>> As per the KIP meeting, I created a wiki:
> > > > >>>>> https://cwiki.apache.org/confluence/display/KAFKA/Time+
> > > > >>>> Based+Release+Plan
> > > > >>>>> Summarizing most of the discussion so far.
> > > > >>>>>
> > > > >>>>> Comments and additional discussion is welcome :)
> > > > >>>>>
> > > > >>>>> Gwen
> > > > >>>>>
> > > > >>>>> On Wed, Aug 17, 2016 at 12:31 PM, Vahid S Hashemian
> > > > >>>>> <va...@us.ibm.com> wrote:
> > > > >>>>>> Time-based releases is a good idea and something that has
> proved
> > > to
> > > > be
> > > > >>>>>> working in a number of open source projects. One successful
> > > example
> > > > is
> > > > >>>>>> Node.js, that goes through two major releases a year. The
> > > > interesting
> > > > >>>>> fact
> > > > >>>>>> about the two releases is that only one (the even-number
> > release)
> > > > comes
> > > > >>>>>> with a long term support (LTS) plan (30 months). More can be
> > read
> > > > here:
> > > > >>>>>> https://github.com/nodejs/LTS. The odd-number releases still
> > come
> > > > with
> > > > >>>>>> major changes and help build the ecosystem, but as far as LTS
> > > goes,
> > > > >>>> there
> > > > >>>>>> is only one per year. This LTS plan makes most enterprises
> want
> > to
> > > > >>>> stick
> > > > >>>>>> to even-number releases, which is okay since frequent upgrades
> > is
> > > > not
> > > > >>>>>> something they are normally interested in anyway.
> > > > >>>>>>
> > > > >>>>>> There could be several minor releases (non-breaking) in
> between
> > > > major
> > > > >>>>>> releases. A major release contains all the features / bug
> fixes
> > in
> > > > the
> > > > >>>>>> master branch a month before the release date, with the
> > potential
> > > > >>>>> addition
> > > > >>>>>> of (non-breaking) bug fixes until the release day. The
> > deprecation
> > > > >>>> cycle
> > > > >>>>>> is one major release: any functionality that is decided to be
> > > > removed
> > > > >>>> is
> > > > >>>>>> deprecated in the next major release, and removed in the major
> > > > release
> > > > >>>>>> after that.
> > > > >>>>>>
> > > > >>>>>> Because of the success of LTS models in this and other open
> > source
> > > > >>>>>> projects, I would suggest implementing a formal LTS plan for
> > Kafka
> > > > too.
> > > > >>>>>>
> > > > >>>>>> Regards,
> > > > >>>>>> --Vahid
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>> From: Gwen Shapira <gw...@confluent.io>
> > > > >>>>>> To: dev@kafka.apache.org
> > > > >>>>>> Date: 08/09/2016 04:49 PM
> > > > >>>>>> Subject: [DISCUSS] Time-based releases for Apache Kafka
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>> Dear Kafka Developers and Users,
> > > > >>>>>>
> > > > >>>>>> In the past, our releases have been quite unpredictable. We'll
> > > > notice
> > > > >>>>>> that a large number of nice features made it in (or are
> close),
> > > > >>>>>> someone would suggest a release and we'd do it. This is fun,
> but
> > > > makes
> > > > >>>>>> planning really hard - we saw it during the last release which
> > we
> > > > >>>>>> decided to delay by a few weeks to allow more features to
> > "land".
> > > > >>>>>>
> > > > >>>>>> Many other communities have adopted time-based releases
> > > successfully
> > > > >>>>>> (Cassandra, GCC, LLVM, Fedora, Gnome, Ubuntu, etc.). And I
> > thought
> > > > it
> > > > >>>>>> will make sense for the Apache Kafka community to try doing
> the
> > > > same.
> > > > >>>>>>
> > > > >>>>>> The benefits of this approach are:
> > > > >>>>>>
> > > > >>>>>> 1. A quicker feedback cycle and users can benefit from
> features
> > > > >>>>>> quicker (assuming for reasonably short time between releases
> - I
> > > was
> > > > >>>>>> thinking 4 months)
> > > > >>>>>>
> > > > >>>>>> 2. Predictability for contributors and users:
> > > > >>>>>> * Developers and reviewers can decide in advance what release
> > they
> > > > are
> > > > >>>>>> aiming for with specific features.
> > > > >>>>>> * If a feature misses a release we have a good idea of when it
> > > will
> > > > >>>> show
> > > > >>>>>> up.
> > > > >>>>>> * Users know when to expect their features
> > > > >>>>>>
> > > > >>>>>> 3. Transparency - There will be a published cut-off date (AKA
> > > > feature
> > > > >>>>>> freeze) for the release and people will know about it in
> > advance.
> > > > >>>>>> Hopefully this will remove the contention around which
> features
> > > make
> > > > >>>>>> it.
> > > > >>>>>>
> > > > >>>>>> 4. Quality - we've seen issues pop up in release candidates
> due
> > to
> > > > >>>>>> last-minute features that didn't have proper time to bake in.
> > More
> > > > >>>>>> time between feature freeze and release will let us test more,
> > > > >>>>>> document more and resolve more issues.
> > > > >>>>>>
> > > > >>>>>> Since nothing is ever perfect, there will be some downsides:
> > > > >>>>>>
> > > > >>>>>> 1. Most notably, features that miss the feature-freeze date
> for
> > a
> > > > >>>>>> release will have to wait few month for the next release.
> > Features
> > > > >>>>>> will reach users faster overall as per benefit #1, but
> > individual
> > > > >>>>>> features that just miss the cut will lose out
> > > > >>>>>>
> > > > >>>>>> 2. More releases a year mean that being a committer is more
> > work -
> > > > >>>>>> release management is still some headache and we'll have more
> of
> > > > >>>>>> those. Hopefully we'll get better at it. Also, the committer
> > list
> > > is
> > > > >>>>>> growing and hopefully it will be less than once-a-year effort
> > for
> > > > each
> > > > >>>>>> committer.
> > > > >>>>>>
> > > > >>>>>> 3. For users, figuring out which release to use and having
> > > frequent
> > > > >>>>>> new releases to upgrade to may be a bit confusing.
> > > > >>>>>>
> > > > >>>>>> 4. Frequent releases mean we need to do bugfix releases for
> > older
> > > > >>>>>> branches. Right now we only do bugfix releases to latest
> > release.
> > > > >>>>>>
> > > > >>>>>> I think the benefits outweigh the drawbacks. Or at least
> suggest
> > > > that
> > > > >>>>>> its worth trying - we can have another discussion in few
> > releases
> > > to
> > > > >>>>>> see if we want to keep it that way or try something else.
> > > > >>>>>>
> > > > >>>>>> My suggestion for the process:
> > > > >>>>>>
> > > > >>>>>> 1. We decide on a reasonable release cadence
> > > > >>>>>> 2. We decide on release dates (even rough estimate such as
> "end
> > of
> > > > >>>>>> February" or something) and work back feature freeze dates.
> > > > >>>>>> 3. Committers volunteer to be "release managers" for specific
> > > > >>>>>> releases. We can coordinate on the list or on a wiki. If no
> > > > committer
> > > > >>>>>> volunteers, we assume the community doesn't need a release and
> > > skip
> > > > >>>>>> it.
> > > > >>>>>> 4. At the "feature freeze" date, the release manager announces
> > the
> > > > >>>>>> contents of the release (which KIPs made it in on time),
> creates
> > > the
> > > > >>>>>> release branch and starts the release process as usual. From
> > this
> > > > >>>>>> point onwards, only bug fixes should be double-committed to
> the
> > > > >>>>>> release branch while trunk can start collecting features for
> the
> > > > >>>>>> subsequent release.
> > > > >>>>>>
> > > > >>>>>> Comments and improvements are appreciated.
> > > > >>>>>>
> > > > >>>>>> Gwen Shapira
> > > > >>>>>> Former-release-manager
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>
> > > > >>>>>
> > > > >>>>>
> > > > >>>>> --
> > > > >>>>> Gwen Shapira
> > > > >>>>> Product Manager | Confluent
> > > > >>>>> 650.450.2760 | @gwenshap
> > > > >>>>> Follow us: Twitter | blog
> > > > >>>>>
> > > > >>>>
> > > > >>
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Gwen Shapira
> > > > > Product Manager | Confluent
> > > > > 650.450.2760 | @gwenshap
> > > > > Follow us: Twitter | blog
> > > >
> > > >
> > >
> >
>
--
Thanks,
Neha
Re: [DISCUSS] Time-based releases for Apache Kafka
Posted by Ismael Juma <is...@juma.me.uk>.
Thanks for volunteering Jason. Sounds good to me,
Ismael
On Wed, Sep 7, 2016 at 4:22 AM, Jason Gustafson <ja...@confluent.io> wrote:
> Hey All,
>
> It sounds like the general consensus is in favor of time-based releases. We
> can continue the discussion about LTS, but I wanted to go ahead and get
> things moving forward by volunteering to manage the next release, which is
> currently slated for October. If that sounds OK, I'll draft a release plan
> and send it out to the community for feedback and a vote.
>
> Thanks,
> Jason
>
> On Thu, Aug 25, 2016 at 2:03 PM, Ofir Manor <of...@equalum.io> wrote:
>
> > I happily agree that Kafka is a solid and the community is great :)
> > But I think there is a gap in perception here.
> > For me, LTS means that someone is actively taking care of a release -
> > actively backporting critical fixes (security, stability, data loss,
> > corruption, hangs etc) from trunk to that LTS version periodically for an
> > extended period of time, for example 18-36 months... So people can really
> > rely on the same Kafka version for a long time.
> > Is someone doing it today for 0.9.0? When is 0.9.0.2 expected? When is
> > 0.8.2.3 expected? Will they cover all known critical issues for whoever
> > relies on them in production?
> > In other words, what is the scope of support that the community want to
> > commit for older versions? (upgrade compatibility? investigating bug
> > reports? proactively backporting fixes?)
> > BTW, another legit option is that the Apache Kafka project won't commit
> to
> > LTS releases. It could let commercial vendors compete on supporting very
> > old versions. I find that actually quite reasonable as well.
> >
> > Ofir Manor
> >
> > Co-Founder & CTO | Equalum
> >
> > Mobile: +972-54-7801286 | Email: ofir.manor@equalum.io
> >
> > On Thu, Aug 25, 2016 at 8:19 PM, Andrew Schofield <
> > andrew_schofield_jira@outlook.com> wrote:
> >
> > > I agree that the Kafka community has managed to maintain a very high
> > > quality level, so I'm not concerned
> > > about the quality of non-LTS releases. If the principle is that every
> > > release is supported for 2 years, that
> > > would be good. I suppose that if the burden of having that many
> > in-support
> > > releases proves too heavy,
> > > as you say we could reconsider.
> > >
> > > Andrew Schofield
> > >
> > > ----------------------------------------
> > > > From: gwen@confluent.io
> > > > Date: Thu, 25 Aug 2016 09:57:30 -0700
> > > > Subject: Re: [DISCUSS] Time-based releases for Apache Kafka
> > > > To: dev@kafka.apache.org
> > > >
> > > > I prefer Ismael's suggestion for supporting 2-years (6 releases)
> > > > rather than have designated LTS releases.
> > > >
> > > > The LTS model seems to work well when some releases are high quality
> > > > (LTS) and the rest are a bit more questionable. It is great for
> > > > companies like Redhat, where they have to invest less to support few
> > > > releases and let the community deal with everything else.
> > > >
> > > > Until now the Kafka community has managed to maintain very high
> > > > quality level. Not just for releases, our trunk is often of better
> > > > quality than other project's releases - we don't think of stability
> as
> > > > something you tuck into a release (and just some releases) but rather
> > > > as an on-going concern. There are costs to doing things that way, but
> > > > in general, I think it has served us well - allowing even
> conservative
> > > > companies to run on the latest released version.
> > > >
> > > > I hope we can agree to at least try maintaining last 6 releases as
> LTS
> > > > (i.e. every single release is supported for 2 years) rather than
> > > > designate some releases as better than others. Of course, if this
> > > > totally fails, we can reconsider.
> > > >
> > > > Gwen
> > > >
> > > > On Thu, Aug 25, 2016 at 9:51 AM, Andrew Schofield
> > > > <an...@outlook.com> wrote:
> > > >> The proposal sounds pretty good, but the main thing currently
> missing
> > > is a proper long-term support release.
> > > >>
> > > >> Having 3 releases a year sounds OK, but if they're all equivalent
> and
> > > bugfix releases are produced for the most
> > > >> recent 2 or 3 releases, anyone wanting to run on an "in support"
> > > release of Kafka has to upgrade every 8-12 months.
> > > >> If you don't actually want anything specific from the newer
> releases,
> > > it's just unnecessary churn.
> > > >>
> > > >> Wouldn't it be better to designate one release every 12-18 months
> as a
> > > long-term support release with bugfix releases
> > > >> produced for those for a longer period of say 24 months. That halves
> > > the upgrade work for people just wanting to keep
> > > >> "in support". Now that adoption is increasing, there are plenty of
> > > users that just want a dependable messaging system
> > > >> without having to be deeply knowledgeable about its innards.
> > > >>
> > > >> LTS works nicely for plenty of open-source projects. I think it
> would
> > > work well for Kafka too.
> > > >>
> > > >> Andrew Schofield
> > > >>
> > > >> ----------------------------------------
> > > >>> From: ofir.manor@equalum.io
> > > >>> Date: Thu, 25 Aug 2016 16:07:07 +0300
> > > >>> Subject: Re: [DISCUSS] Time-based releases for Apache Kafka
> > > >>> To: dev@kafka.apache.org
> > > >>>
> > > >>> Regarding bug fixes, you may want to consider to have an LTS
> release
> > > once a
> > > >>> year - designating it for longer-term support / better for the
> > masses.
> > > >>> If you like that - then fix bugs in trunk, backport important ones
> to
> > > >>> latest release + the last two LTS releases.
> > > >>> Even if you don't - if a downstream distribution picks a Kafka
> > version
> > > and
> > > >>> plans to support it over a few years, it could be nice of them to
> > "own"
> > > >>> that older release - volunteer to be a release manager for bug
> > > backports to
> > > >>> that version over a longer period...
> > > >>> Just my two cents :)
> > > >>>
> > > >>> Ofir Manor
> > > >>>
> > > >>> Co-Founder & CTO | Equalum
> > > >>>
> > > >>> Mobile: +972-54-7801286 | Email: ofir.manor@equalum.io
> > > >>>
> > > >>> On Thu, Aug 25, 2016 at 12:32 PM, Ismael Juma <is...@juma.me.uk>
> > > wrote:
> > > >>>
> > > >>>> Thanks for putting this together Gwen. I think it sounds
> reasonable
> > > and
> > > >>>> instead of trying to optimise every aspect of it ahead of time
> > (which
> > > is
> > > >>>> hard, subjective and time-consuming), I am happy to try what is
> > being
> > > >>>> proposed and tweak based on experience. One thing we should pay
> > > particular
> > > >>>> attention to is how the stabilisation process works out in
> practice.
> > > >>>>
> > > >>>> A couple of comments:
> > > >>>>
> > > >>>> "Given 3 releases a year and the fact that no one upgrades three
> > > times a
> > > >>>> year, we propose making sure (by testing!) that rolling upgrade
> can
> > > be done
> > > >>>> from each release in the past year (i.e. last 3 releases) to the
> > > latest
> > > >>>> version."
> > > >>>>
> > > >>>> Because the cost of doing this for a larger number of releases is
> > > >>>> relatively low, I still think we should go for 6 here (our code
> > > currently
> > > >>>> supports 5 versions as I said in a previous message, so we're
> close
> > > to that
> > > >>>> target already). I'm generally very keen to make upgrades as easy
> as
> > > >>>> possible so that people have no reason not to upgrade. :)
> > > >>>>
> > > >>>> "We will also attempt, as a community to do bugfix releases as
> > needed
> > > for
> > > >>>> the last 3 releases."
> > > >>>>
> > > >>>> I would suggest 2, personally, but since this is a bit fuzzy, I am
> > OK
> > > with
> > > >>>> 3 if people prefer that.
> > > >>>>
> > > >>>> Ismael
> > > >>>>
> > > >>>> On Thu, Aug 25, 2016 at 6:22 AM, Gwen Shapira <gw...@confluent.io>
> > > wrote:
> > > >>>>
> > > >>>>> Hi Team Kafka,
> > > >>>>>
> > > >>>>> As per the KIP meeting, I created a wiki:
> > > >>>>> https://cwiki.apache.org/confluence/display/KAFKA/Time+
> > > >>>> Based+Release+Plan
> > > >>>>> Summarizing most of the discussion so far.
> > > >>>>>
> > > >>>>> Comments and additional discussion is welcome :)
> > > >>>>>
> > > >>>>> Gwen
> > > >>>>>
> > > >>>>> On Wed, Aug 17, 2016 at 12:31 PM, Vahid S Hashemian
> > > >>>>> <va...@us.ibm.com> wrote:
> > > >>>>>> Time-based releases is a good idea and something that has proved
> > to
> > > be
> > > >>>>>> working in a number of open source projects. One successful
> > example
> > > is
> > > >>>>>> Node.js, that goes through two major releases a year. The
> > > interesting
> > > >>>>> fact
> > > >>>>>> about the two releases is that only one (the even-number
> release)
> > > comes
> > > >>>>>> with a long term support (LTS) plan (30 months). More can be
> read
> > > here:
> > > >>>>>> https://github.com/nodejs/LTS. The odd-number releases still
> come
> > > with
> > > >>>>>> major changes and help build the ecosystem, but as far as LTS
> > goes,
> > > >>>> there
> > > >>>>>> is only one per year. This LTS plan makes most enterprises want
> to
> > > >>>> stick
> > > >>>>>> to even-number releases, which is okay since frequent upgrades
> is
> > > not
> > > >>>>>> something they are normally interested in anyway.
> > > >>>>>>
> > > >>>>>> There could be several minor releases (non-breaking) in between
> > > major
> > > >>>>>> releases. A major release contains all the features / bug fixes
> in
> > > the
> > > >>>>>> master branch a month before the release date, with the
> potential
> > > >>>>> addition
> > > >>>>>> of (non-breaking) bug fixes until the release day. The
> deprecation
> > > >>>> cycle
> > > >>>>>> is one major release: any functionality that is decided to be
> > > removed
> > > >>>> is
> > > >>>>>> deprecated in the next major release, and removed in the major
> > > release
> > > >>>>>> after that.
> > > >>>>>>
> > > >>>>>> Because of the success of LTS models in this and other open
> source
> > > >>>>>> projects, I would suggest implementing a formal LTS plan for
> Kafka
> > > too.
> > > >>>>>>
> > > >>>>>> Regards,
> > > >>>>>> --Vahid
> > > >>>>>>
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> From: Gwen Shapira <gw...@confluent.io>
> > > >>>>>> To: dev@kafka.apache.org
> > > >>>>>> Date: 08/09/2016 04:49 PM
> > > >>>>>> Subject: [DISCUSS] Time-based releases for Apache Kafka
> > > >>>>>>
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> Dear Kafka Developers and Users,
> > > >>>>>>
> > > >>>>>> In the past, our releases have been quite unpredictable. We'll
> > > notice
> > > >>>>>> that a large number of nice features made it in (or are close),
> > > >>>>>> someone would suggest a release and we'd do it. This is fun, but
> > > makes
> > > >>>>>> planning really hard - we saw it during the last release which
> we
> > > >>>>>> decided to delay by a few weeks to allow more features to
> "land".
> > > >>>>>>
> > > >>>>>> Many other communities have adopted time-based releases
> > successfully
> > > >>>>>> (Cassandra, GCC, LLVM, Fedora, Gnome, Ubuntu, etc.). And I
> thought
> > > it
> > > >>>>>> will make sense for the Apache Kafka community to try doing the
> > > same.
> > > >>>>>>
> > > >>>>>> The benefits of this approach are:
> > > >>>>>>
> > > >>>>>> 1. A quicker feedback cycle and users can benefit from features
> > > >>>>>> quicker (assuming for reasonably short time between releases - I
> > was
> > > >>>>>> thinking 4 months)
> > > >>>>>>
> > > >>>>>> 2. Predictability for contributors and users:
> > > >>>>>> * Developers and reviewers can decide in advance what release
> they
> > > are
> > > >>>>>> aiming for with specific features.
> > > >>>>>> * If a feature misses a release we have a good idea of when it
> > will
> > > >>>> show
> > > >>>>>> up.
> > > >>>>>> * Users know when to expect their features
> > > >>>>>>
> > > >>>>>> 3. Transparency - There will be a published cut-off date (AKA
> > > feature
> > > >>>>>> freeze) for the release and people will know about it in
> advance.
> > > >>>>>> Hopefully this will remove the contention around which features
> > make
> > > >>>>>> it.
> > > >>>>>>
> > > >>>>>> 4. Quality - we've seen issues pop up in release candidates due
> to
> > > >>>>>> last-minute features that didn't have proper time to bake in.
> More
> > > >>>>>> time between feature freeze and release will let us test more,
> > > >>>>>> document more and resolve more issues.
> > > >>>>>>
> > > >>>>>> Since nothing is ever perfect, there will be some downsides:
> > > >>>>>>
> > > >>>>>> 1. Most notably, features that miss the feature-freeze date for
> a
> > > >>>>>> release will have to wait few month for the next release.
> Features
> > > >>>>>> will reach users faster overall as per benefit #1, but
> individual
> > > >>>>>> features that just miss the cut will lose out
> > > >>>>>>
> > > >>>>>> 2. More releases a year mean that being a committer is more
> work -
> > > >>>>>> release management is still some headache and we'll have more of
> > > >>>>>> those. Hopefully we'll get better at it. Also, the committer
> list
> > is
> > > >>>>>> growing and hopefully it will be less than once-a-year effort
> for
> > > each
> > > >>>>>> committer.
> > > >>>>>>
> > > >>>>>> 3. For users, figuring out which release to use and having
> > frequent
> > > >>>>>> new releases to upgrade to may be a bit confusing.
> > > >>>>>>
> > > >>>>>> 4. Frequent releases mean we need to do bugfix releases for
> older
> > > >>>>>> branches. Right now we only do bugfix releases to latest
> release.
> > > >>>>>>
> > > >>>>>> I think the benefits outweigh the drawbacks. Or at least suggest
> > > that
> > > >>>>>> its worth trying - we can have another discussion in few
> releases
> > to
> > > >>>>>> see if we want to keep it that way or try something else.
> > > >>>>>>
> > > >>>>>> My suggestion for the process:
> > > >>>>>>
> > > >>>>>> 1. We decide on a reasonable release cadence
> > > >>>>>> 2. We decide on release dates (even rough estimate such as "end
> of
> > > >>>>>> February" or something) and work back feature freeze dates.
> > > >>>>>> 3. Committers volunteer to be "release managers" for specific
> > > >>>>>> releases. We can coordinate on the list or on a wiki. If no
> > > committer
> > > >>>>>> volunteers, we assume the community doesn't need a release and
> > skip
> > > >>>>>> it.
> > > >>>>>> 4. At the "feature freeze" date, the release manager announces
> the
> > > >>>>>> contents of the release (which KIPs made it in on time), creates
> > the
> > > >>>>>> release branch and starts the release process as usual. From
> this
> > > >>>>>> point onwards, only bug fixes should be double-committed to the
> > > >>>>>> release branch while trunk can start collecting features for the
> > > >>>>>> subsequent release.
> > > >>>>>>
> > > >>>>>> Comments and improvements are appreciated.
> > > >>>>>>
> > > >>>>>> Gwen Shapira
> > > >>>>>> Former-release-manager
> > > >>>>>>
> > > >>>>>>
> > > >>>>>>
> > > >>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>>>
> > > >>>>>
> > > >>>>> --
> > > >>>>> Gwen Shapira
> > > >>>>> Product Manager | Confluent
> > > >>>>> 650.450.2760 | @gwenshap
> > > >>>>> Follow us: Twitter | blog
> > > >>>>>
> > > >>>>
> > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > Gwen Shapira
> > > > Product Manager | Confluent
> > > > 650.450.2760 | @gwenshap
> > > > Follow us: Twitter | blog
> > >
> > >
> >
>
Re: [DISCUSS] Time-based releases for Apache Kafka
Posted by Sriram Subramanian <ra...@confluent.io>.
Sounds good to me.
> On Sep 6, 2016, at 8:22 PM, Jason Gustafson <ja...@confluent.io> wrote:
>
> Hey All,
>
> It sounds like the general consensus is in favor of time-based releases. We
> can continue the discussion about LTS, but I wanted to go ahead and get
> things moving forward by volunteering to manage the next release, which is
> currently slated for October. If that sounds OK, I'll draft a release plan
> and send it out to the community for feedback and a vote.
>
> Thanks,
> Jason
>
>> On Thu, Aug 25, 2016 at 2:03 PM, Ofir Manor <of...@equalum.io> wrote:
>>
>> I happily agree that Kafka is a solid and the community is great :)
>> But I think there is a gap in perception here.
>> For me, LTS means that someone is actively taking care of a release -
>> actively backporting critical fixes (security, stability, data loss,
>> corruption, hangs etc) from trunk to that LTS version periodically for an
>> extended period of time, for example 18-36 months... So people can really
>> rely on the same Kafka version for a long time.
>> Is someone doing it today for 0.9.0? When is 0.9.0.2 expected? When is
>> 0.8.2.3 expected? Will they cover all known critical issues for whoever
>> relies on them in production?
>> In other words, what is the scope of support that the community want to
>> commit for older versions? (upgrade compatibility? investigating bug
>> reports? proactively backporting fixes?)
>> BTW, another legit option is that the Apache Kafka project won't commit to
>> LTS releases. It could let commercial vendors compete on supporting very
>> old versions. I find that actually quite reasonable as well.
>>
>> Ofir Manor
>>
>> Co-Founder & CTO | Equalum
>>
>> Mobile: +972-54-7801286 | Email: ofir.manor@equalum.io
>>
>> On Thu, Aug 25, 2016 at 8:19 PM, Andrew Schofield <
>> andrew_schofield_jira@outlook.com> wrote:
>>
>>> I agree that the Kafka community has managed to maintain a very high
>>> quality level, so I'm not concerned
>>> about the quality of non-LTS releases. If the principle is that every
>>> release is supported for 2 years, that
>>> would be good. I suppose that if the burden of having that many
>> in-support
>>> releases proves too heavy,
>>> as you say we could reconsider.
>>>
>>> Andrew Schofield
>>>
>>> ----------------------------------------
>>>> From: gwen@confluent.io
>>>> Date: Thu, 25 Aug 2016 09:57:30 -0700
>>>> Subject: Re: [DISCUSS] Time-based releases for Apache Kafka
>>>> To: dev@kafka.apache.org
>>>>
>>>> I prefer Ismael's suggestion for supporting 2-years (6 releases)
>>>> rather than have designated LTS releases.
>>>>
>>>> The LTS model seems to work well when some releases are high quality
>>>> (LTS) and the rest are a bit more questionable. It is great for
>>>> companies like Redhat, where they have to invest less to support few
>>>> releases and let the community deal with everything else.
>>>>
>>>> Until now the Kafka community has managed to maintain very high
>>>> quality level. Not just for releases, our trunk is often of better
>>>> quality than other project's releases - we don't think of stability as
>>>> something you tuck into a release (and just some releases) but rather
>>>> as an on-going concern. There are costs to doing things that way, but
>>>> in general, I think it has served us well - allowing even conservative
>>>> companies to run on the latest released version.
>>>>
>>>> I hope we can agree to at least try maintaining last 6 releases as LTS
>>>> (i.e. every single release is supported for 2 years) rather than
>>>> designate some releases as better than others. Of course, if this
>>>> totally fails, we can reconsider.
>>>>
>>>> Gwen
>>>>
>>>> On Thu, Aug 25, 2016 at 9:51 AM, Andrew Schofield
>>>> <an...@outlook.com> wrote:
>>>>> The proposal sounds pretty good, but the main thing currently missing
>>> is a proper long-term support release.
>>>>>
>>>>> Having 3 releases a year sounds OK, but if they're all equivalent and
>>> bugfix releases are produced for the most
>>>>> recent 2 or 3 releases, anyone wanting to run on an "in support"
>>> release of Kafka has to upgrade every 8-12 months.
>>>>> If you don't actually want anything specific from the newer releases,
>>> it's just unnecessary churn.
>>>>>
>>>>> Wouldn't it be better to designate one release every 12-18 months as a
>>> long-term support release with bugfix releases
>>>>> produced for those for a longer period of say 24 months. That halves
>>> the upgrade work for people just wanting to keep
>>>>> "in support". Now that adoption is increasing, there are plenty of
>>> users that just want a dependable messaging system
>>>>> without having to be deeply knowledgeable about its innards.
>>>>>
>>>>> LTS works nicely for plenty of open-source projects. I think it would
>>> work well for Kafka too.
>>>>>
>>>>> Andrew Schofield
>>>>>
>>>>> ----------------------------------------
>>>>>> From: ofir.manor@equalum.io
>>>>>> Date: Thu, 25 Aug 2016 16:07:07 +0300
>>>>>> Subject: Re: [DISCUSS] Time-based releases for Apache Kafka
>>>>>> To: dev@kafka.apache.org
>>>>>>
>>>>>> Regarding bug fixes, you may want to consider to have an LTS release
>>> once a
>>>>>> year - designating it for longer-term support / better for the
>> masses.
>>>>>> If you like that - then fix bugs in trunk, backport important ones to
>>>>>> latest release + the last two LTS releases.
>>>>>> Even if you don't - if a downstream distribution picks a Kafka
>> version
>>> and
>>>>>> plans to support it over a few years, it could be nice of them to
>> "own"
>>>>>> that older release - volunteer to be a release manager for bug
>>> backports to
>>>>>> that version over a longer period...
>>>>>> Just my two cents :)
>>>>>>
>>>>>> Ofir Manor
>>>>>>
>>>>>> Co-Founder & CTO | Equalum
>>>>>>
>>>>>> Mobile: +972-54-7801286 | Email: ofir.manor@equalum.io
>>>>>>
>>>>>> On Thu, Aug 25, 2016 at 12:32 PM, Ismael Juma <is...@juma.me.uk>
>>> wrote:
>>>>>>
>>>>>>> Thanks for putting this together Gwen. I think it sounds reasonable
>>> and
>>>>>>> instead of trying to optimise every aspect of it ahead of time
>> (which
>>> is
>>>>>>> hard, subjective and time-consuming), I am happy to try what is
>> being
>>>>>>> proposed and tweak based on experience. One thing we should pay
>>> particular
>>>>>>> attention to is how the stabilisation process works out in practice.
>>>>>>>
>>>>>>> A couple of comments:
>>>>>>>
>>>>>>> "Given 3 releases a year and the fact that no one upgrades three
>>> times a
>>>>>>> year, we propose making sure (by testing!) that rolling upgrade can
>>> be done
>>>>>>> from each release in the past year (i.e. last 3 releases) to the
>>> latest
>>>>>>> version."
>>>>>>>
>>>>>>> Because the cost of doing this for a larger number of releases is
>>>>>>> relatively low, I still think we should go for 6 here (our code
>>> currently
>>>>>>> supports 5 versions as I said in a previous message, so we're close
>>> to that
>>>>>>> target already). I'm generally very keen to make upgrades as easy as
>>>>>>> possible so that people have no reason not to upgrade. :)
>>>>>>>
>>>>>>> "We will also attempt, as a community to do bugfix releases as
>> needed
>>> for
>>>>>>> the last 3 releases."
>>>>>>>
>>>>>>> I would suggest 2, personally, but since this is a bit fuzzy, I am
>> OK
>>> with
>>>>>>> 3 if people prefer that.
>>>>>>>
>>>>>>> Ismael
>>>>>>>
>>>>>>> On Thu, Aug 25, 2016 at 6:22 AM, Gwen Shapira <gw...@confluent.io>
>>> wrote:
>>>>>>>
>>>>>>>> Hi Team Kafka,
>>>>>>>>
>>>>>>>> As per the KIP meeting, I created a wiki:
>>>>>>>> https://cwiki.apache.org/confluence/display/KAFKA/Time+
>>>>>>> Based+Release+Plan
>>>>>>>> Summarizing most of the discussion so far.
>>>>>>>>
>>>>>>>> Comments and additional discussion is welcome :)
>>>>>>>>
>>>>>>>> Gwen
>>>>>>>>
>>>>>>>> On Wed, Aug 17, 2016 at 12:31 PM, Vahid S Hashemian
>>>>>>>> <va...@us.ibm.com> wrote:
>>>>>>>>> Time-based releases is a good idea and something that has proved
>> to
>>> be
>>>>>>>>> working in a number of open source projects. One successful
>> example
>>> is
>>>>>>>>> Node.js, that goes through two major releases a year. The
>>> interesting
>>>>>>>> fact
>>>>>>>>> about the two releases is that only one (the even-number release)
>>> comes
>>>>>>>>> with a long term support (LTS) plan (30 months). More can be read
>>> here:
>>>>>>>>> https://github.com/nodejs/LTS. The odd-number releases still come
>>> with
>>>>>>>>> major changes and help build the ecosystem, but as far as LTS
>> goes,
>>>>>>> there
>>>>>>>>> is only one per year. This LTS plan makes most enterprises want to
>>>>>>> stick
>>>>>>>>> to even-number releases, which is okay since frequent upgrades is
>>> not
>>>>>>>>> something they are normally interested in anyway.
>>>>>>>>>
>>>>>>>>> There could be several minor releases (non-breaking) in between
>>> major
>>>>>>>>> releases. A major release contains all the features / bug fixes in
>>> the
>>>>>>>>> master branch a month before the release date, with the potential
>>>>>>>> addition
>>>>>>>>> of (non-breaking) bug fixes until the release day. The deprecation
>>>>>>> cycle
>>>>>>>>> is one major release: any functionality that is decided to be
>>> removed
>>>>>>> is
>>>>>>>>> deprecated in the next major release, and removed in the major
>>> release
>>>>>>>>> after that.
>>>>>>>>>
>>>>>>>>> Because of the success of LTS models in this and other open source
>>>>>>>>> projects, I would suggest implementing a formal LTS plan for Kafka
>>> too.
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> --Vahid
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> From: Gwen Shapira <gw...@confluent.io>
>>>>>>>>> To: dev@kafka.apache.org
>>>>>>>>> Date: 08/09/2016 04:49 PM
>>>>>>>>> Subject: [DISCUSS] Time-based releases for Apache Kafka
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Dear Kafka Developers and Users,
>>>>>>>>>
>>>>>>>>> In the past, our releases have been quite unpredictable. We'll
>>> notice
>>>>>>>>> that a large number of nice features made it in (or are close),
>>>>>>>>> someone would suggest a release and we'd do it. This is fun, but
>>> makes
>>>>>>>>> planning really hard - we saw it during the last release which we
>>>>>>>>> decided to delay by a few weeks to allow more features to "land".
>>>>>>>>>
>>>>>>>>> Many other communities have adopted time-based releases
>> successfully
>>>>>>>>> (Cassandra, GCC, LLVM, Fedora, Gnome, Ubuntu, etc.). And I thought
>>> it
>>>>>>>>> will make sense for the Apache Kafka community to try doing the
>>> same.
>>>>>>>>>
>>>>>>>>> The benefits of this approach are:
>>>>>>>>>
>>>>>>>>> 1. A quicker feedback cycle and users can benefit from features
>>>>>>>>> quicker (assuming for reasonably short time between releases - I
>> was
>>>>>>>>> thinking 4 months)
>>>>>>>>>
>>>>>>>>> 2. Predictability for contributors and users:
>>>>>>>>> * Developers and reviewers can decide in advance what release they
>>> are
>>>>>>>>> aiming for with specific features.
>>>>>>>>> * If a feature misses a release we have a good idea of when it
>> will
>>>>>>> show
>>>>>>>>> up.
>>>>>>>>> * Users know when to expect their features
>>>>>>>>>
>>>>>>>>> 3. Transparency - There will be a published cut-off date (AKA
>>> feature
>>>>>>>>> freeze) for the release and people will know about it in advance.
>>>>>>>>> Hopefully this will remove the contention around which features
>> make
>>>>>>>>> it.
>>>>>>>>>
>>>>>>>>> 4. Quality - we've seen issues pop up in release candidates due to
>>>>>>>>> last-minute features that didn't have proper time to bake in. More
>>>>>>>>> time between feature freeze and release will let us test more,
>>>>>>>>> document more and resolve more issues.
>>>>>>>>>
>>>>>>>>> Since nothing is ever perfect, there will be some downsides:
>>>>>>>>>
>>>>>>>>> 1. Most notably, features that miss the feature-freeze date for a
>>>>>>>>> release will have to wait few month for the next release. Features
>>>>>>>>> will reach users faster overall as per benefit #1, but individual
>>>>>>>>> features that just miss the cut will lose out
>>>>>>>>>
>>>>>>>>> 2. More releases a year mean that being a committer is more work -
>>>>>>>>> release management is still some headache and we'll have more of
>>>>>>>>> those. Hopefully we'll get better at it. Also, the committer list
>> is
>>>>>>>>> growing and hopefully it will be less than once-a-year effort for
>>> each
>>>>>>>>> committer.
>>>>>>>>>
>>>>>>>>> 3. For users, figuring out which release to use and having
>> frequent
>>>>>>>>> new releases to upgrade to may be a bit confusing.
>>>>>>>>>
>>>>>>>>> 4. Frequent releases mean we need to do bugfix releases for older
>>>>>>>>> branches. Right now we only do bugfix releases to latest release.
>>>>>>>>>
>>>>>>>>> I think the benefits outweigh the drawbacks. Or at least suggest
>>> that
>>>>>>>>> its worth trying - we can have another discussion in few releases
>> to
>>>>>>>>> see if we want to keep it that way or try something else.
>>>>>>>>>
>>>>>>>>> My suggestion for the process:
>>>>>>>>>
>>>>>>>>> 1. We decide on a reasonable release cadence
>>>>>>>>> 2. We decide on release dates (even rough estimate such as "end of
>>>>>>>>> February" or something) and work back feature freeze dates.
>>>>>>>>> 3. Committers volunteer to be "release managers" for specific
>>>>>>>>> releases. We can coordinate on the list or on a wiki. If no
>>> committer
>>>>>>>>> volunteers, we assume the community doesn't need a release and
>> skip
>>>>>>>>> it.
>>>>>>>>> 4. At the "feature freeze" date, the release manager announces the
>>>>>>>>> contents of the release (which KIPs made it in on time), creates
>> the
>>>>>>>>> release branch and starts the release process as usual. From this
>>>>>>>>> point onwards, only bug fixes should be double-committed to the
>>>>>>>>> release branch while trunk can start collecting features for the
>>>>>>>>> subsequent release.
>>>>>>>>>
>>>>>>>>> Comments and improvements are appreciated.
>>>>>>>>>
>>>>>>>>> Gwen Shapira
>>>>>>>>> Former-release-manager
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Gwen Shapira
>>>>>>>> Product Manager | Confluent
>>>>>>>> 650.450.2760 | @gwenshap
>>>>>>>> Follow us: Twitter | blog
>>>>
>>>>
>>>>
>>>> --
>>>> Gwen Shapira
>>>> Product Manager | Confluent
>>>> 650.450.2760 | @gwenshap
>>>> Follow us: Twitter | blog
>>