About how features are integrated to different HBase versions

["张铎 (Duo Zhang)" <pa...@gmail.com>]

I think we have a good discussion on HBASE-21034, where a feature is back
ported to branch-1, but then folks think that we should not back port them
to branch-2.1 and branch-2.0, as usually we should not add new features to
minor release lines.

I think the reason why we do not want the feature in branch-2.1 and
branch-2.0 is reasonable, but this will introduce another problem. As
later, we will release a 1.5.0 which has the feature, but when a user later
upgrades from 1.5.0 to 2.1.x or 2.0.x, it will find that the feature is
gone, even though the 2.1.x or 2.0.x is released after 1.5.0 is released,
as we do not port the feature to these two branches. This will be very
confusing to users I'd say.

So I think we should guarantee that, a higher version of HBase release will
always contain all the features of a HBase release with a lower version
which is released earlier, unless explicitly mentioned(for example, DLR).

And this implies that, when we setup a new major release and make a new
release on the first minor release line, then the develop branch for the
previous major release will be useless, as said above, usually we do not
want to port any new features to the minor release line of the new major
release, then the new features should not be ported to previous major
release, otherwise we will break the guarantee above. And this also means
that, we could just use the 'develop' branch to make new releases.

Re: About how features are integrated to different HBase versions

["张铎 (Duo Zhang)" <pa...@gmail.com>]

There is a issue already for completing HBCK2

https://issues.apache.org/jira/browse/HBASE-21745

Andrew Purtell <ap...@apache.org> 于2019年1月22日周二 上午9:57写道：

> I think a fully functional hbck2 is a prerequisite for moving the stable
> pointer. See the other thread on that. Otherwise, it seems fine if there is
> consensus to move it.
>
>
> On Mon, Jan 21, 2019 at 5:50 PM Stack <st...@duboce.net> wrote:
>
> > Thanks Andrew. I think there are others too who have the same predicament
> > (Francis Liu for instance). Backports of perf improvements, bug fixes,
> and
> > operational tooling improvements makes sense while branch-1 is in active
> > use.
> >
> > I appreciate Allans' sentiment. It is time to move the stable pointer
> from
> > branch-1 to branch-2, say branch-2.1.2?
> >
> > S
> >
> >
> >
> > On Sun, Jan 20, 2019 at 10:14 PM Andrew Purtell <
> andrew.purtell@gmail.com>
> > wrote:
> >
> > > I can speak for myself and assume others backporting to branch-1 have
> > > similar motivations.  We are backporting interesting and useful and
> > > reasonable improvements, because branch-1 is still useful and in
> > > production, and will be for the foreseeable future. Changes like perf
> > > improvements and operability/tooling/UI improvements should not be
> denied
> > > to branch-1 users if some contributors and committers are putting in
> the
> > > effort to maintain it. I agree major new features should not be
> > backported.
> > > I doubt anyone will try to backport AMv2 or IMC. I won’t, for sure. Not
> > > interested in destabilizing change.
> > >
> > > Personally I work for an organization that runs branch-1 derived code
> in
> > > production. You can stop my maintenance activities in the community but
> > > that would just mean I make a fork and take the work internal. Why not
> > let
> > > me contribute these efforts back to the community? It is your choice. I
> > > don't see why we can't have activity both in branch-1 and branch-2. To
> > each
> > > their own. Let the community organically decide what is right by
> > do-ocracy.
> > >
> > > If you ask me why aren’t we using branch-2 now, I have to say
> > > unfortunately it’s not production ready for us. Refer to the recent
> > > discussion about hbck2 for one concern. Another is a colleague did a
> > > trivial test with head of branch-2 and measured a 50% perf degradation
> in
> > > scan performance. Now, that is just a one off test. But it causes
> > concern.
> > > We are swamped already with run the business concerns. We can’t take on
> > the
> > > additional risk at this time. Perhaps in the future we will have more
> > > bandwidth to contribute to branch-2 efforts.
> > >
> > > > On Jan 20, 2019, at 9:29 PM, Allan Yang <al...@apache.org> wrote:
> > > >
> > > > {quote}
> > > > For example, you release 2.1.0 yesterday, 1.5.0 today, and then 2.1.1
> > > > tomorrow, it is OK that 2.1.0 does not have all feature which 1.5.0
> > has,
> > > > but 2.1.1 should have all features which 1.5.0 has.
> > > > {quote}
> > > > I don't think can work, normal user mostly won't care about the
> release
> > > > time, they only know 2.1.1 > 2.1.0 > 1.5.0. They will think higher
> > > version
> > > > includes all the feature in lower version.
> > > >
> > > > I don't get the point why we are now still backporting new features
> > from
> > > > branch-2 to branch-1. Yes, there is many 1.x cluster in production,
> so
> > we
> > > > need to release 1.x versions to fix bugs and keep them stable, as the
> > > > stable point is still in 1.x.
> > > > And at the same time, we should try to move on to 2.x, making
> branch-1
> > > as a
> > > > bugfix branch for sometime before deprecating it. As far as I see,
> > > branch-1
> > > > is still very 'active', too active I think.
> > > > If we stop backport features from branch-2 to branch-1, then there is
> > no
> > > > problem, IMHO.
> > > > Best Regards
> > > > Allan Yang
> > > >
> > > >
> > > > Andrew Purtell <an...@gmail.com> 于2019年1月20日周日 上午5:27写道：
> > > >
> > > >> As branch RM for branch-1 I will always check to make sure a commit
> > > there
> > > >> has first been committed to branch-2. There will always be an
> upgrade
> > > path
> > > >> from a branch-1 based release to a branch-2 based release. The
> > relevant
> > > >> JIRAs will either have a 1.x and 2.x fixVersion or the backport JIRA
> > > will
> > > >> be linked to the one for the branch-2 commit. When making the
> release
> > > notes
> > > >> we will be looking at these things (or should, anyway). We can
> update
> > > the
> > > >> upgrade paths documentation whenever we find this kind of linkage.
> > > Perhaps
> > > >> we can describe this for future RMs in the how to release section of
> > the
> > > >> doc? Does this satisfy the concerns?
> > > >>
> > > >>> On Jan 18, 2019, at 11:47 PM, Sean Busbey <bu...@apache.org>
> wrote:
> > > >>>
> > > >>> I agree with Andrew that we can't both have maintenance releases
> and
> > > >> expect
> > > >>> every feature in ongoing branch-1 releases to be in branches-2.y.
> > > >>>
> > > >>> Tracking consideration for when features are available across major
> > > >>> versions fits in well with the "upgrade paths" section in the ref
> > > guide.
> > > >>>
> > > >>> We've just gotten in the habit of it only getting filled in when a
> > big
> > > >>> release is coming up.
> > > >>>
> > > >>>
> > > >>>
> > > >>>> On Fri, Jan 18, 2019, 23:46 张铎(Duo Zhang) <palomino219@gmail.com
> > > wrote:
> > > >>>>
> > > >>>> Then we must have a upgrade path, for example, 1.5.x can only be
> > > >> upgraded
> > > >>>> to 2.2.x if you want all the features still there?
> > > >>>>
> > > >>>> Maybe we should have a release timeline for the first release of
> all
> > > the
> > > >>>> minor releases? So when user want to upgrade, they can choose the
> > > minor
> > > >>>> release which is released later than the current one.
> > > >>>>
> > > >>>> Andrew Purtell <an...@gmail.com>于2019年1月19日 周六13:15写道：
> > > >>>>
> > > >>>>> Also I think branch-1 releases will be done on a monthly cadence
> > > >>>>> independent of any branch-2 releases. This is because there are
> > > >> different
> > > >>>>> RMs at work with different needs and schedules.
> > > >>>>>
> > > >>>>> I can certainly help out some with branch-2 releasing if you need
> > it,
> > > >>>>> FWIW.
> > > >>>>>
> > > >>>>> It may also help if we begin talking about 1.x and 2.x as
> separate
> > > >>>>> "products". This can help avoid confusion about features in 1.5
> not
> > > in
> > > >>>> 2.1
> > > >>>>> but in 2.2. For all practical purposes they are separate
> products.
> > > Some
> > > >>>> of
> > > >>>>> our community develop and run branch-1. Others develop and run
> > > >> branch-2.
> > > >>>>> There is some overlap but the overlap is not total. The concerns
> > will
> > > >>>>> diverge a bit. I think this is healthy. Everyone is attending to
> > what
> > > >>>> they
> > > >>>>> need. Let's figure out how to make it work.
> > > >>>>>
> > > >>>>>> On Jan 18, 2019, at 9:04 PM, Andrew Purtell <
> > > andrew.purtell@gmail.com
> > > >>>
> > > >>>>> wrote:
> > > >>>>>>
> > > >>>>>> Also please be prepared to support forward evolution and
> > maintenance
> > > >> of
> > > >>>>> branch-1 for, potentially, years. Because it is used in
> production
> > > and
> > > >>>> will
> > > >>>>> continue to do so for a long time. Features may end up in 1.6.0
> > that
> > > >> only
> > > >>>>> appear in 2.3 or 2.4. And in 1.7 that only appear in 2.5 or 2.6.
> > This
> > > >>>>> shouldn't be confusing. We just need to document it. JIRA helps
> > some,
> > > >>>>> release notes can help a lot more. Maybe in the future a feature
> to
> > > >>>> version
> > > >>>>> matrix in the book.
> > > >>>>>>
> > > >>>>>>> On Jan 18, 2019, at 8:59 PM, Andrew Purtell <
> > > >> andrew.purtell@gmail.com
> > > >>>>>
> > > >>>>> wrote:
> > > >>>>>>>
> > > >>>>>>> This can't work, because we can put things into a new minor
> that
> > > >>>> cannot
> > > >>>>> go into a patch relesse. If you say instead 2.2.0 must have
> > > everything
> > > >> in
> > > >>>>> 1.5.0, it can work. The alignment of features should happen at
> the
> > > >> minor
> > > >>>>> releases. If we can also have alignment in patch releases too,
> that
> > > >> would
> > > >>>>> be great, but can't be mandatory.
> > > >>>>>>>
> > > >>>>>>>> On Jan 18, 2019, at 7:12 PM, 张铎(Duo Zhang) <
> > palomino219@gmail.com
> > > >
> > > >>>>> wrote:
> > > >>>>>>>>
> > > >>>>>>>> Please see the red words carefully, I explicitly mentioned
> that,
> > > the
> > > >>>>> newer
> > > >>>>>>>> version should be released LATER, if you want to get all the
> > > >>>> features.
> > > >>>>>>>>
> > > >>>>>>>> For example, you release 2.1.0 yesterday, 1.5.0 today, and
> then
> > > >> 2.1.1
> > > >>>>>>>> tomorrow, it is OK that 2.1.0 does not have all feature which
> > > 1.5.0
> > > >>>>> has,
> > > >>>>>>>> but 2.1.1 should have all features which 1.5.0 has.
> > > >>>>>>>>
> > > >>>>>>>> Sergey Shelukhin <Se...@microsoft.com.invalid>
> > > >>>>> 于2019年1月19日周六
> > > >>>>>>>> 上午10:23写道：
> > > >>>>>>>>
> > > >>>>>>>>> Consider that we actually cannot guarantee this without a
> time
> > > >>>>> machine,
> > > >>>>>>>>> because some "newer" versions are already released.
> > > >>>>>>>>>
> > > >>>>>>>>> If we backport to 1.5 now, we cannot do anything about 2.0.0,
> > > >> 2.1.0,
> > > >>>>>>>>> 2.1.2, etc. because they are already released... if the user
> > > >>>> upgrades
> > > >>>>> from
> > > >>>>>>>>> 1.5 to 2.0.1 for example, they will lose the feature no
> matter
> > > >> what.
> > > >>>>>>>>> The only way to ensure is to
> > > >>>>>>>>> - always update to latest dot version,
> > > >>>>>>>>> - also for us to make sure we never release before releasing
> > > every
> > > >>>>> "later"
> > > >>>>>>>>> dot release (e.g. we cannot release 1.5 before 2.1.3, 2.0.N,
> > etc.
> > > >> so
> > > >>>>>>>>> there's the latest release for every line).
> > > >>>>>>>>> - and also for us to make sure that every single dot line
> > > actually
> > > >>>>> has a
> > > >>>>>>>>> release - when e.g. 2.0.X line is abandoned that may not
> > happen,
> > > so
> > > >>>>> the
> > > >>>>>>>>> latest version of 2.0.X will precede latest 1.Y because 1.Y
> may
> > > >>>> still
> > > >>>>> be
> > > >>>>>>>>> active (like as far as I recall 0.94 was getting dot releases
> > > even
> > > >>>>> when
> > > >>>>>>>>> 0.96 was abandoned) - so even if the user goes from 1.Y to
> the
> > > >>>> latest
> > > >>>>> 2.0.X
> > > >>>>>>>>> they will lose the feature.
> > > >>>>>>>>>
> > > >>>>>>>>> I think this is kind of expected... I agree that it needs to
> be
> > > >>>>>>>>> documented. To an extent it already is in JIRA where
> fixVersion
> > > may
> > > >>>> be
> > > >>>>>>>>> "3.0, 2.2, 1.5", but it makes sense to document explicitly.
> > > >>>>>>>>>
> > > >>>>>>>>> -----Original Message-----
> > > >>>>>>>>> From: 张铎(Duo Zhang) <pa...@gmail.com>
> > > >>>>>>>>> Sent: Friday, January 18, 2019 5:50 PM
> > > >>>>>>>>> To: HBase Dev List <de...@hbase.apache.org>
> > > >>>>>>>>> Subject: About how features are integrated to different HBase
> > > >>>> versions
> > > >>>>>>>>>
> > > >>>>>>>>> I think we have a good discussion on HBASE-21034, where a
> > feature
> > > >> is
> > > >>>>> back
> > > >>>>>>>>> ported to branch-1, but then folks think that we should not
> > back
> > > >>>> port
> > > >>>>> them
> > > >>>>>>>>> to branch-2.1 and branch-2.0, as usually we should not add
> new
> > > >>>>> features to
> > > >>>>>>>>> minor release lines.
> > > >>>>>>>>>
> > > >>>>>>>>> I think the reason why we do not want the feature in
> branch-2.1
> > > and
> > > >>>>>>>>> branch-2.0 is reasonable, but this will introduce another
> > > problem.
> > > >>>> As
> > > >>>>>>>>> later, we will release a 1.5.0 which has the feature, but
> when
> > a
> > > >>>> user
> > > >>>>> later
> > > >>>>>>>>> upgrades from 1.5.0 to 2.1.x or 2.0.x, it will find that the
> > > >> feature
> > > >>>>> is
> > > >>>>>>>>> gone, even though the 2.1.x or 2.0.x is released after 1.5.0
> is
> > > >>>>> released,
> > > >>>>>>>>> as we do not port the feature to these two branches. This
> will
> > be
> > > >>>> very
> > > >>>>>>>>> confusing to users I'd say.
> > > >>>>>>>>>
> > > >>>>>>>>> So I think we should guarantee that, a higher version of
> HBase
> > > >>>> release
> > > >>>>>>>>> will always contain all the features of a HBase release with
> a
> > > >> lower
> > > >>>>>>>>> version which is released earlier, unless explicitly
> > > mentioned(for
> > > >>>>> example,
> > > >>>>>>>>> DLR).
> > > >>>>>>>>>
> > > >>>>>>>>> And this implies that, when we setup a new major release and
> > > make a
> > > >>>>> new
> > > >>>>>>>>> release on the first minor release line, then the develop
> > branch
> > > >> for
> > > >>>>> the
> > > >>>>>>>>> previous major release will be useless, as said above,
> usually
> > we
> > > >> do
> > > >>>>> not
> > > >>>>>>>>> want to port any new features to the minor release line of
> the
> > > new
> > > >>>>> major
> > > >>>>>>>>> release, then the new features should not be ported to
> previous
> > > >>>> major
> > > >>>>>>>>> release, otherwise we will break the guarantee above. And
> this
> > > also
> > > >>>>> means
> > > >>>>>>>>> that, we could just use the 'develop' branch to make new
> > > releases.
> > > >>>>>>>>>
> > > >>>>>
> > > >>>>
> > > >>
> > >
> >
>
>
> --
> Best regards,
> Andrew
>
> Words like orphans lost among the crosstalk, meaning torn from truth's
> decrepit hands
>    - A23, Crosstalk
>

Re: About how features are integrated to different HBase versions

[Andrew Purtell <ap...@apache.org>]

I think a fully functional hbck2 is a prerequisite for moving the stable
pointer. See the other thread on that. Otherwise, it seems fine if there is
consensus to move it.


On Mon, Jan 21, 2019 at 5:50 PM Stack <st...@duboce.net> wrote:

> Thanks Andrew. I think there are others too who have the same predicament
> (Francis Liu for instance). Backports of perf improvements, bug fixes, and
> operational tooling improvements makes sense while branch-1 is in active
> use.
>
> I appreciate Allans' sentiment. It is time to move the stable pointer from
> branch-1 to branch-2, say branch-2.1.2?
>
> S
>
>
>
> On Sun, Jan 20, 2019 at 10:14 PM Andrew Purtell <an...@gmail.com>
> wrote:
>
> > I can speak for myself and assume others backporting to branch-1 have
> > similar motivations.  We are backporting interesting and useful and
> > reasonable improvements, because branch-1 is still useful and in
> > production, and will be for the foreseeable future. Changes like perf
> > improvements and operability/tooling/UI improvements should not be denied
> > to branch-1 users if some contributors and committers are putting in the
> > effort to maintain it. I agree major new features should not be
> backported.
> > I doubt anyone will try to backport AMv2 or IMC. I won’t, for sure. Not
> > interested in destabilizing change.
> >
> > Personally I work for an organization that runs branch-1 derived code in
> > production. You can stop my maintenance activities in the community but
> > that would just mean I make a fork and take the work internal. Why not
> let
> > me contribute these efforts back to the community? It is your choice. I
> > don't see why we can't have activity both in branch-1 and branch-2. To
> each
> > their own. Let the community organically decide what is right by
> do-ocracy.
> >
> > If you ask me why aren’t we using branch-2 now, I have to say
> > unfortunately it’s not production ready for us. Refer to the recent
> > discussion about hbck2 for one concern. Another is a colleague did a
> > trivial test with head of branch-2 and measured a 50% perf degradation in
> > scan performance. Now, that is just a one off test. But it causes
> concern.
> > We are swamped already with run the business concerns. We can’t take on
> the
> > additional risk at this time. Perhaps in the future we will have more
> > bandwidth to contribute to branch-2 efforts.
> >
> > > On Jan 20, 2019, at 9:29 PM, Allan Yang <al...@apache.org> wrote:
> > >
> > > {quote}
> > > For example, you release 2.1.0 yesterday, 1.5.0 today, and then 2.1.1
> > > tomorrow, it is OK that 2.1.0 does not have all feature which 1.5.0
> has,
> > > but 2.1.1 should have all features which 1.5.0 has.
> > > {quote}
> > > I don't think can work, normal user mostly won't care about the release
> > > time, they only know 2.1.1 > 2.1.0 > 1.5.0. They will think higher
> > version
> > > includes all the feature in lower version.
> > >
> > > I don't get the point why we are now still backporting new features
> from
> > > branch-2 to branch-1. Yes, there is many 1.x cluster in production, so
> we
> > > need to release 1.x versions to fix bugs and keep them stable, as the
> > > stable point is still in 1.x.
> > > And at the same time, we should try to move on to 2.x, making branch-1
> > as a
> > > bugfix branch for sometime before deprecating it. As far as I see,
> > branch-1
> > > is still very 'active', too active I think.
> > > If we stop backport features from branch-2 to branch-1, then there is
> no
> > > problem, IMHO.
> > > Best Regards
> > > Allan Yang
> > >
> > >
> > > Andrew Purtell <an...@gmail.com> 于2019年1月20日周日 上午5:27写道：
> > >
> > >> As branch RM for branch-1 I will always check to make sure a commit
> > there
> > >> has first been committed to branch-2. There will always be an upgrade
> > path
> > >> from a branch-1 based release to a branch-2 based release. The
> relevant
> > >> JIRAs will either have a 1.x and 2.x fixVersion or the backport JIRA
> > will
> > >> be linked to the one for the branch-2 commit. When making the release
> > notes
> > >> we will be looking at these things (or should, anyway). We can update
> > the
> > >> upgrade paths documentation whenever we find this kind of linkage.
> > Perhaps
> > >> we can describe this for future RMs in the how to release section of
> the
> > >> doc? Does this satisfy the concerns?
> > >>
> > >>> On Jan 18, 2019, at 11:47 PM, Sean Busbey <bu...@apache.org> wrote:
> > >>>
> > >>> I agree with Andrew that we can't both have maintenance releases and
> > >> expect
> > >>> every feature in ongoing branch-1 releases to be in branches-2.y.
> > >>>
> > >>> Tracking consideration for when features are available across major
> > >>> versions fits in well with the "upgrade paths" section in the ref
> > guide.
> > >>>
> > >>> We've just gotten in the habit of it only getting filled in when a
> big
> > >>> release is coming up.
> > >>>
> > >>>
> > >>>
> > >>>> On Fri, Jan 18, 2019, 23:46 张铎(Duo Zhang) <palomino219@gmail.com
> > wrote:
> > >>>>
> > >>>> Then we must have a upgrade path, for example, 1.5.x can only be
> > >> upgraded
> > >>>> to 2.2.x if you want all the features still there?
> > >>>>
> > >>>> Maybe we should have a release timeline for the first release of all
> > the
> > >>>> minor releases? So when user want to upgrade, they can choose the
> > minor
> > >>>> release which is released later than the current one.
> > >>>>
> > >>>> Andrew Purtell <an...@gmail.com>于2019年1月19日 周六13:15写道：
> > >>>>
> > >>>>> Also I think branch-1 releases will be done on a monthly cadence
> > >>>>> independent of any branch-2 releases. This is because there are
> > >> different
> > >>>>> RMs at work with different needs and schedules.
> > >>>>>
> > >>>>> I can certainly help out some with branch-2 releasing if you need
> it,
> > >>>>> FWIW.
> > >>>>>
> > >>>>> It may also help if we begin talking about 1.x and 2.x as separate
> > >>>>> "products". This can help avoid confusion about features in 1.5 not
> > in
> > >>>> 2.1
> > >>>>> but in 2.2. For all practical purposes they are separate products.
> > Some
> > >>>> of
> > >>>>> our community develop and run branch-1. Others develop and run
> > >> branch-2.
> > >>>>> There is some overlap but the overlap is not total. The concerns
> will
> > >>>>> diverge a bit. I think this is healthy. Everyone is attending to
> what
> > >>>> they
> > >>>>> need. Let's figure out how to make it work.
> > >>>>>
> > >>>>>> On Jan 18, 2019, at 9:04 PM, Andrew Purtell <
> > andrew.purtell@gmail.com
> > >>>
> > >>>>> wrote:
> > >>>>>>
> > >>>>>> Also please be prepared to support forward evolution and
> maintenance
> > >> of
> > >>>>> branch-1 for, potentially, years. Because it is used in production
> > and
> > >>>> will
> > >>>>> continue to do so for a long time. Features may end up in 1.6.0
> that
> > >> only
> > >>>>> appear in 2.3 or 2.4. And in 1.7 that only appear in 2.5 or 2.6.
> This
> > >>>>> shouldn't be confusing. We just need to document it. JIRA helps
> some,
> > >>>>> release notes can help a lot more. Maybe in the future a feature to
> > >>>> version
> > >>>>> matrix in the book.
> > >>>>>>
> > >>>>>>> On Jan 18, 2019, at 8:59 PM, Andrew Purtell <
> > >> andrew.purtell@gmail.com
> > >>>>>
> > >>>>> wrote:
> > >>>>>>>
> > >>>>>>> This can't work, because we can put things into a new minor that
> > >>>> cannot
> > >>>>> go into a patch relesse. If you say instead 2.2.0 must have
> > everything
> > >> in
> > >>>>> 1.5.0, it can work. The alignment of features should happen at the
> > >> minor
> > >>>>> releases. If we can also have alignment in patch releases too, that
> > >> would
> > >>>>> be great, but can't be mandatory.
> > >>>>>>>
> > >>>>>>>> On Jan 18, 2019, at 7:12 PM, 张铎(Duo Zhang) <
> palomino219@gmail.com
> > >
> > >>>>> wrote:
> > >>>>>>>>
> > >>>>>>>> Please see the red words carefully, I explicitly mentioned that,
> > the
> > >>>>> newer
> > >>>>>>>> version should be released LATER, if you want to get all the
> > >>>> features.
> > >>>>>>>>
> > >>>>>>>> For example, you release 2.1.0 yesterday, 1.5.0 today, and then
> > >> 2.1.1
> > >>>>>>>> tomorrow, it is OK that 2.1.0 does not have all feature which
> > 1.5.0
> > >>>>> has,
> > >>>>>>>> but 2.1.1 should have all features which 1.5.0 has.
> > >>>>>>>>
> > >>>>>>>> Sergey Shelukhin <Se...@microsoft.com.invalid>
> > >>>>> 于2019年1月19日周六
> > >>>>>>>> 上午10:23写道：
> > >>>>>>>>
> > >>>>>>>>> Consider that we actually cannot guarantee this without a time
> > >>>>> machine,
> > >>>>>>>>> because some "newer" versions are already released.
> > >>>>>>>>>
> > >>>>>>>>> If we backport to 1.5 now, we cannot do anything about 2.0.0,
> > >> 2.1.0,
> > >>>>>>>>> 2.1.2, etc. because they are already released... if the user
> > >>>> upgrades
> > >>>>> from
> > >>>>>>>>> 1.5 to 2.0.1 for example, they will lose the feature no matter
> > >> what.
> > >>>>>>>>> The only way to ensure is to
> > >>>>>>>>> - always update to latest dot version,
> > >>>>>>>>> - also for us to make sure we never release before releasing
> > every
> > >>>>> "later"
> > >>>>>>>>> dot release (e.g. we cannot release 1.5 before 2.1.3, 2.0.N,
> etc.
> > >> so
> > >>>>>>>>> there's the latest release for every line).
> > >>>>>>>>> - and also for us to make sure that every single dot line
> > actually
> > >>>>> has a
> > >>>>>>>>> release - when e.g. 2.0.X line is abandoned that may not
> happen,
> > so
> > >>>>> the
> > >>>>>>>>> latest version of 2.0.X will precede latest 1.Y because 1.Y may
> > >>>> still
> > >>>>> be
> > >>>>>>>>> active (like as far as I recall 0.94 was getting dot releases
> > even
> > >>>>> when
> > >>>>>>>>> 0.96 was abandoned) - so even if the user goes from 1.Y to the
> > >>>> latest
> > >>>>> 2.0.X
> > >>>>>>>>> they will lose the feature.
> > >>>>>>>>>
> > >>>>>>>>> I think this is kind of expected... I agree that it needs to be
> > >>>>>>>>> documented. To an extent it already is in JIRA where fixVersion
> > may
> > >>>> be
> > >>>>>>>>> "3.0, 2.2, 1.5", but it makes sense to document explicitly.
> > >>>>>>>>>
> > >>>>>>>>> -----Original Message-----
> > >>>>>>>>> From: 张铎(Duo Zhang) <pa...@gmail.com>
> > >>>>>>>>> Sent: Friday, January 18, 2019 5:50 PM
> > >>>>>>>>> To: HBase Dev List <de...@hbase.apache.org>
> > >>>>>>>>> Subject: About how features are integrated to different HBase
> > >>>> versions
> > >>>>>>>>>
> > >>>>>>>>> I think we have a good discussion on HBASE-21034, where a
> feature
> > >> is
> > >>>>> back
> > >>>>>>>>> ported to branch-1, but then folks think that we should not
> back
> > >>>> port
> > >>>>> them
> > >>>>>>>>> to branch-2.1 and branch-2.0, as usually we should not add new
> > >>>>> features to
> > >>>>>>>>> minor release lines.
> > >>>>>>>>>
> > >>>>>>>>> I think the reason why we do not want the feature in branch-2.1
> > and
> > >>>>>>>>> branch-2.0 is reasonable, but this will introduce another
> > problem.
> > >>>> As
> > >>>>>>>>> later, we will release a 1.5.0 which has the feature, but when
> a
> > >>>> user
> > >>>>> later
> > >>>>>>>>> upgrades from 1.5.0 to 2.1.x or 2.0.x, it will find that the
> > >> feature
> > >>>>> is
> > >>>>>>>>> gone, even though the 2.1.x or 2.0.x is released after 1.5.0 is
> > >>>>> released,
> > >>>>>>>>> as we do not port the feature to these two branches. This will
> be
> > >>>> very
> > >>>>>>>>> confusing to users I'd say.
> > >>>>>>>>>
> > >>>>>>>>> So I think we should guarantee that, a higher version of HBase
> > >>>> release
> > >>>>>>>>> will always contain all the features of a HBase release with a
> > >> lower
> > >>>>>>>>> version which is released earlier, unless explicitly
> > mentioned(for
> > >>>>> example,
> > >>>>>>>>> DLR).
> > >>>>>>>>>
> > >>>>>>>>> And this implies that, when we setup a new major release and
> > make a
> > >>>>> new
> > >>>>>>>>> release on the first minor release line, then the develop
> branch
> > >> for
> > >>>>> the
> > >>>>>>>>> previous major release will be useless, as said above, usually
> we
> > >> do
> > >>>>> not
> > >>>>>>>>> want to port any new features to the minor release line of the
> > new
> > >>>>> major
> > >>>>>>>>> release, then the new features should not be ported to previous
> > >>>> major
> > >>>>>>>>> release, otherwise we will break the guarantee above. And this
> > also
> > >>>>> means
> > >>>>>>>>> that, we could just use the 'develop' branch to make new
> > releases.
> > >>>>>>>>>
> > >>>>>
> > >>>>
> > >>
> >
>


-- 
Best regards,
Andrew

Words like orphans lost among the crosstalk, meaning torn from truth's
decrepit hands
   - A23, Crosstalk

Re: About how features are integrated to different HBase versions

[Stack <st...@duboce.net>]

Thanks Andrew. I think there are others too who have the same predicament
(Francis Liu for instance). Backports of perf improvements, bug fixes, and
operational tooling improvements makes sense while branch-1 is in active
use.

I appreciate Allans' sentiment. It is time to move the stable pointer from
branch-1 to branch-2, say branch-2.1.2?

S



On Sun, Jan 20, 2019 at 10:14 PM Andrew Purtell <an...@gmail.com>
wrote:

> I can speak for myself and assume others backporting to branch-1 have
> similar motivations.  We are backporting interesting and useful and
> reasonable improvements, because branch-1 is still useful and in
> production, and will be for the foreseeable future. Changes like perf
> improvements and operability/tooling/UI improvements should not be denied
> to branch-1 users if some contributors and committers are putting in the
> effort to maintain it. I agree major new features should not be backported.
> I doubt anyone will try to backport AMv2 or IMC. I won’t, for sure. Not
> interested in destabilizing change.
>
> Personally I work for an organization that runs branch-1 derived code in
> production. You can stop my maintenance activities in the community but
> that would just mean I make a fork and take the work internal. Why not let
> me contribute these efforts back to the community? It is your choice. I
> don't see why we can't have activity both in branch-1 and branch-2. To each
> their own. Let the community organically decide what is right by do-ocracy.
>
> If you ask me why aren’t we using branch-2 now, I have to say
> unfortunately it’s not production ready for us. Refer to the recent
> discussion about hbck2 for one concern. Another is a colleague did a
> trivial test with head of branch-2 and measured a 50% perf degradation in
> scan performance. Now, that is just a one off test. But it causes concern.
> We are swamped already with run the business concerns. We can’t take on the
> additional risk at this time. Perhaps in the future we will have more
> bandwidth to contribute to branch-2 efforts.
>
> > On Jan 20, 2019, at 9:29 PM, Allan Yang <al...@apache.org> wrote:
> >
> > {quote}
> > For example, you release 2.1.0 yesterday, 1.5.0 today, and then 2.1.1
> > tomorrow, it is OK that 2.1.0 does not have all feature which 1.5.0 has,
> > but 2.1.1 should have all features which 1.5.0 has.
> > {quote}
> > I don't think can work, normal user mostly won't care about the release
> > time, they only know 2.1.1 > 2.1.0 > 1.5.0. They will think higher
> version
> > includes all the feature in lower version.
> >
> > I don't get the point why we are now still backporting new features from
> > branch-2 to branch-1. Yes, there is many 1.x cluster in production, so we
> > need to release 1.x versions to fix bugs and keep them stable, as the
> > stable point is still in 1.x.
> > And at the same time, we should try to move on to 2.x, making branch-1
> as a
> > bugfix branch for sometime before deprecating it. As far as I see,
> branch-1
> > is still very 'active', too active I think.
> > If we stop backport features from branch-2 to branch-1, then there is no
> > problem, IMHO.
> > Best Regards
> > Allan Yang
> >
> >
> > Andrew Purtell <an...@gmail.com> 于2019年1月20日周日 上午5:27写道：
> >
> >> As branch RM for branch-1 I will always check to make sure a commit
> there
> >> has first been committed to branch-2. There will always be an upgrade
> path
> >> from a branch-1 based release to a branch-2 based release. The relevant
> >> JIRAs will either have a 1.x and 2.x fixVersion or the backport JIRA
> will
> >> be linked to the one for the branch-2 commit. When making the release
> notes
> >> we will be looking at these things (or should, anyway). We can update
> the
> >> upgrade paths documentation whenever we find this kind of linkage.
> Perhaps
> >> we can describe this for future RMs in the how to release section of the
> >> doc? Does this satisfy the concerns?
> >>
> >>> On Jan 18, 2019, at 11:47 PM, Sean Busbey <bu...@apache.org> wrote:
> >>>
> >>> I agree with Andrew that we can't both have maintenance releases and
> >> expect
> >>> every feature in ongoing branch-1 releases to be in branches-2.y.
> >>>
> >>> Tracking consideration for when features are available across major
> >>> versions fits in well with the "upgrade paths" section in the ref
> guide.
> >>>
> >>> We've just gotten in the habit of it only getting filled in when a big
> >>> release is coming up.
> >>>
> >>>
> >>>
> >>>> On Fri, Jan 18, 2019, 23:46 张铎(Duo Zhang) <palomino219@gmail.com
> wrote:
> >>>>
> >>>> Then we must have a upgrade path, for example, 1.5.x can only be
> >> upgraded
> >>>> to 2.2.x if you want all the features still there?
> >>>>
> >>>> Maybe we should have a release timeline for the first release of all
> the
> >>>> minor releases? So when user want to upgrade, they can choose the
> minor
> >>>> release which is released later than the current one.
> >>>>
> >>>> Andrew Purtell <an...@gmail.com>于2019年1月19日 周六13:15写道：
> >>>>
> >>>>> Also I think branch-1 releases will be done on a monthly cadence
> >>>>> independent of any branch-2 releases. This is because there are
> >> different
> >>>>> RMs at work with different needs and schedules.
> >>>>>
> >>>>> I can certainly help out some with branch-2 releasing if you need it,
> >>>>> FWIW.
> >>>>>
> >>>>> It may also help if we begin talking about 1.x and 2.x as separate
> >>>>> "products". This can help avoid confusion about features in 1.5 not
> in
> >>>> 2.1
> >>>>> but in 2.2. For all practical purposes they are separate products.
> Some
> >>>> of
> >>>>> our community develop and run branch-1. Others develop and run
> >> branch-2.
> >>>>> There is some overlap but the overlap is not total. The concerns will
> >>>>> diverge a bit. I think this is healthy. Everyone is attending to what
> >>>> they
> >>>>> need. Let's figure out how to make it work.
> >>>>>
> >>>>>> On Jan 18, 2019, at 9:04 PM, Andrew Purtell <
> andrew.purtell@gmail.com
> >>>
> >>>>> wrote:
> >>>>>>
> >>>>>> Also please be prepared to support forward evolution and maintenance
> >> of
> >>>>> branch-1 for, potentially, years. Because it is used in production
> and
> >>>> will
> >>>>> continue to do so for a long time. Features may end up in 1.6.0 that
> >> only
> >>>>> appear in 2.3 or 2.4. And in 1.7 that only appear in 2.5 or 2.6. This
> >>>>> shouldn't be confusing. We just need to document it. JIRA helps some,
> >>>>> release notes can help a lot more. Maybe in the future a feature to
> >>>> version
> >>>>> matrix in the book.
> >>>>>>
> >>>>>>> On Jan 18, 2019, at 8:59 PM, Andrew Purtell <
> >> andrew.purtell@gmail.com
> >>>>>
> >>>>> wrote:
> >>>>>>>
> >>>>>>> This can't work, because we can put things into a new minor that
> >>>> cannot
> >>>>> go into a patch relesse. If you say instead 2.2.0 must have
> everything
> >> in
> >>>>> 1.5.0, it can work. The alignment of features should happen at the
> >> minor
> >>>>> releases. If we can also have alignment in patch releases too, that
> >> would
> >>>>> be great, but can't be mandatory.
> >>>>>>>
> >>>>>>>> On Jan 18, 2019, at 7:12 PM, 张铎(Duo Zhang) <palomino219@gmail.com
> >
> >>>>> wrote:
> >>>>>>>>
> >>>>>>>> Please see the red words carefully, I explicitly mentioned that,
> the
> >>>>> newer
> >>>>>>>> version should be released LATER, if you want to get all the
> >>>> features.
> >>>>>>>>
> >>>>>>>> For example, you release 2.1.0 yesterday, 1.5.0 today, and then
> >> 2.1.1
> >>>>>>>> tomorrow, it is OK that 2.1.0 does not have all feature which
> 1.5.0
> >>>>> has,
> >>>>>>>> but 2.1.1 should have all features which 1.5.0 has.
> >>>>>>>>
> >>>>>>>> Sergey Shelukhin <Se...@microsoft.com.invalid>
> >>>>> 于2019年1月19日周六
> >>>>>>>> 上午10:23写道：
> >>>>>>>>
> >>>>>>>>> Consider that we actually cannot guarantee this without a time
> >>>>> machine,
> >>>>>>>>> because some "newer" versions are already released.
> >>>>>>>>>
> >>>>>>>>> If we backport to 1.5 now, we cannot do anything about 2.0.0,
> >> 2.1.0,
> >>>>>>>>> 2.1.2, etc. because they are already released... if the user
> >>>> upgrades
> >>>>> from
> >>>>>>>>> 1.5 to 2.0.1 for example, they will lose the feature no matter
> >> what.
> >>>>>>>>> The only way to ensure is to
> >>>>>>>>> - always update to latest dot version,
> >>>>>>>>> - also for us to make sure we never release before releasing
> every
> >>>>> "later"
> >>>>>>>>> dot release (e.g. we cannot release 1.5 before 2.1.3, 2.0.N, etc.
> >> so
> >>>>>>>>> there's the latest release for every line).
> >>>>>>>>> - and also for us to make sure that every single dot line
> actually
> >>>>> has a
> >>>>>>>>> release - when e.g. 2.0.X line is abandoned that may not happen,
> so
> >>>>> the
> >>>>>>>>> latest version of 2.0.X will precede latest 1.Y because 1.Y may
> >>>> still
> >>>>> be
> >>>>>>>>> active (like as far as I recall 0.94 was getting dot releases
> even
> >>>>> when
> >>>>>>>>> 0.96 was abandoned) - so even if the user goes from 1.Y to the
> >>>> latest
> >>>>> 2.0.X
> >>>>>>>>> they will lose the feature.
> >>>>>>>>>
> >>>>>>>>> I think this is kind of expected... I agree that it needs to be
> >>>>>>>>> documented. To an extent it already is in JIRA where fixVersion
> may
> >>>> be
> >>>>>>>>> "3.0, 2.2, 1.5", but it makes sense to document explicitly.
> >>>>>>>>>
> >>>>>>>>> -----Original Message-----
> >>>>>>>>> From: 张铎(Duo Zhang) <pa...@gmail.com>
> >>>>>>>>> Sent: Friday, January 18, 2019 5:50 PM
> >>>>>>>>> To: HBase Dev List <de...@hbase.apache.org>
> >>>>>>>>> Subject: About how features are integrated to different HBase
> >>>> versions
> >>>>>>>>>
> >>>>>>>>> I think we have a good discussion on HBASE-21034, where a feature
> >> is
> >>>>> back
> >>>>>>>>> ported to branch-1, but then folks think that we should not back
> >>>> port
> >>>>> them
> >>>>>>>>> to branch-2.1 and branch-2.0, as usually we should not add new
> >>>>> features to
> >>>>>>>>> minor release lines.
> >>>>>>>>>
> >>>>>>>>> I think the reason why we do not want the feature in branch-2.1
> and
> >>>>>>>>> branch-2.0 is reasonable, but this will introduce another
> problem.
> >>>> As
> >>>>>>>>> later, we will release a 1.5.0 which has the feature, but when a
> >>>> user
> >>>>> later
> >>>>>>>>> upgrades from 1.5.0 to 2.1.x or 2.0.x, it will find that the
> >> feature
> >>>>> is
> >>>>>>>>> gone, even though the 2.1.x or 2.0.x is released after 1.5.0 is
> >>>>> released,
> >>>>>>>>> as we do not port the feature to these two branches. This will be
> >>>> very
> >>>>>>>>> confusing to users I'd say.
> >>>>>>>>>
> >>>>>>>>> So I think we should guarantee that, a higher version of HBase
> >>>> release
> >>>>>>>>> will always contain all the features of a HBase release with a
> >> lower
> >>>>>>>>> version which is released earlier, unless explicitly
> mentioned(for
> >>>>> example,
> >>>>>>>>> DLR).
> >>>>>>>>>
> >>>>>>>>> And this implies that, when we setup a new major release and
> make a
> >>>>> new
> >>>>>>>>> release on the first minor release line, then the develop branch
> >> for
> >>>>> the
> >>>>>>>>> previous major release will be useless, as said above, usually we
> >> do
> >>>>> not
> >>>>>>>>> want to port any new features to the minor release line of the
> new
> >>>>> major
> >>>>>>>>> release, then the new features should not be ported to previous
> >>>> major
> >>>>>>>>> release, otherwise we will break the guarantee above. And this
> also
> >>>>> means
> >>>>>>>>> that, we could just use the 'develop' branch to make new
> releases.
> >>>>>>>>>
> >>>>>
> >>>>
> >>
>

Re: About how features are integrated to different HBase versions

[Andrew Purtell <an...@gmail.com>]

I can speak for myself and assume others backporting to branch-1 have similar motivations.  We are backporting interesting and useful and reasonable improvements, because branch-1 is still useful and in production, and will be for the foreseeable future. Changes like perf improvements and operability/tooling/UI improvements should not be denied to branch-1 users if some contributors and committers are putting in the effort to maintain it. I agree major new features should not be backported. I doubt anyone will try to backport AMv2 or IMC. I won’t, for sure. Not interested in destabilizing change. 

Personally I work for an organization that runs branch-1 derived code in production. You can stop my maintenance activities in the community but that would just mean I make a fork and take the work internal. Why not let me contribute these efforts back to the community? It is your choice. I don't see why we can't have activity both in branch-1 and branch-2. To each their own. Let the community organically decide what is right by do-ocracy. 

If you ask me why aren’t we using branch-2 now, I have to say unfortunately it’s not production ready for us. Refer to the recent discussion about hbck2 for one concern. Another is a colleague did a trivial test with head of branch-2 and measured a 50% perf degradation in scan performance. Now, that is just a one off test. But it causes concern. We are swamped already with run the business concerns. We can’t take on the additional risk at this time. Perhaps in the future we will have more bandwidth to contribute to branch-2 efforts.

> On Jan 20, 2019, at 9:29 PM, Allan Yang <al...@apache.org> wrote:
> 
> {quote}
> For example, you release 2.1.0 yesterday, 1.5.0 today, and then 2.1.1
> tomorrow, it is OK that 2.1.0 does not have all feature which 1.5.0 has,
> but 2.1.1 should have all features which 1.5.0 has.
> {quote}
> I don't think can work, normal user mostly won't care about the release
> time, they only know 2.1.1 > 2.1.0 > 1.5.0. They will think higher version
> includes all the feature in lower version.
> 
> I don't get the point why we are now still backporting new features from
> branch-2 to branch-1. Yes, there is many 1.x cluster in production, so we
> need to release 1.x versions to fix bugs and keep them stable, as the
> stable point is still in 1.x.
> And at the same time, we should try to move on to 2.x, making branch-1 as a
> bugfix branch for sometime before deprecating it. As far as I see, branch-1
> is still very 'active', too active I think.
> If we stop backport features from branch-2 to branch-1, then there is no
> problem, IMHO.
> Best Regards
> Allan Yang
> 
> 
> Andrew Purtell <an...@gmail.com> 于2019年1月20日周日 上午5:27写道：
> 
>> As branch RM for branch-1 I will always check to make sure a commit there
>> has first been committed to branch-2. There will always be an upgrade path
>> from a branch-1 based release to a branch-2 based release. The relevant
>> JIRAs will either have a 1.x and 2.x fixVersion or the backport JIRA will
>> be linked to the one for the branch-2 commit. When making the release notes
>> we will be looking at these things (or should, anyway). We can update the
>> upgrade paths documentation whenever we find this kind of linkage. Perhaps
>> we can describe this for future RMs in the how to release section of the
>> doc? Does this satisfy the concerns?
>> 
>>> On Jan 18, 2019, at 11:47 PM, Sean Busbey <bu...@apache.org> wrote:
>>> 
>>> I agree with Andrew that we can't both have maintenance releases and
>> expect
>>> every feature in ongoing branch-1 releases to be in branches-2.y.
>>> 
>>> Tracking consideration for when features are available across major
>>> versions fits in well with the "upgrade paths" section in the ref guide.
>>> 
>>> We've just gotten in the habit of it only getting filled in when a big
>>> release is coming up.
>>> 
>>> 
>>> 
>>>> On Fri, Jan 18, 2019, 23:46 张铎(Duo Zhang) <palomino219@gmail.com wrote:
>>>> 
>>>> Then we must have a upgrade path, for example, 1.5.x can only be
>> upgraded
>>>> to 2.2.x if you want all the features still there?
>>>> 
>>>> Maybe we should have a release timeline for the first release of all the
>>>> minor releases? So when user want to upgrade, they can choose the minor
>>>> release which is released later than the current one.
>>>> 
>>>> Andrew Purtell <an...@gmail.com>于2019年1月19日 周六13:15写道：
>>>> 
>>>>> Also I think branch-1 releases will be done on a monthly cadence
>>>>> independent of any branch-2 releases. This is because there are
>> different
>>>>> RMs at work with different needs and schedules.
>>>>> 
>>>>> I can certainly help out some with branch-2 releasing if you need it,
>>>>> FWIW.
>>>>> 
>>>>> It may also help if we begin talking about 1.x and 2.x as separate
>>>>> "products". This can help avoid confusion about features in 1.5 not in
>>>> 2.1
>>>>> but in 2.2. For all practical purposes they are separate products. Some
>>>> of
>>>>> our community develop and run branch-1. Others develop and run
>> branch-2.
>>>>> There is some overlap but the overlap is not total. The concerns will
>>>>> diverge a bit. I think this is healthy. Everyone is attending to what
>>>> they
>>>>> need. Let's figure out how to make it work.
>>>>> 
>>>>>> On Jan 18, 2019, at 9:04 PM, Andrew Purtell <andrew.purtell@gmail.com
>>> 
>>>>> wrote:
>>>>>> 
>>>>>> Also please be prepared to support forward evolution and maintenance
>> of
>>>>> branch-1 for, potentially, years. Because it is used in production and
>>>> will
>>>>> continue to do so for a long time. Features may end up in 1.6.0 that
>> only
>>>>> appear in 2.3 or 2.4. And in 1.7 that only appear in 2.5 or 2.6. This
>>>>> shouldn't be confusing. We just need to document it. JIRA helps some,
>>>>> release notes can help a lot more. Maybe in the future a feature to
>>>> version
>>>>> matrix in the book.
>>>>>> 
>>>>>>> On Jan 18, 2019, at 8:59 PM, Andrew Purtell <
>> andrew.purtell@gmail.com
>>>>> 
>>>>> wrote:
>>>>>>> 
>>>>>>> This can't work, because we can put things into a new minor that
>>>> cannot
>>>>> go into a patch relesse. If you say instead 2.2.0 must have everything
>> in
>>>>> 1.5.0, it can work. The alignment of features should happen at the
>> minor
>>>>> releases. If we can also have alignment in patch releases too, that
>> would
>>>>> be great, but can't be mandatory.
>>>>>>> 
>>>>>>>> On Jan 18, 2019, at 7:12 PM, 张铎(Duo Zhang) <pa...@gmail.com>
>>>>> wrote:
>>>>>>>> 
>>>>>>>> Please see the red words carefully, I explicitly mentioned that, the
>>>>> newer
>>>>>>>> version should be released LATER, if you want to get all the
>>>> features.
>>>>>>>> 
>>>>>>>> For example, you release 2.1.0 yesterday, 1.5.0 today, and then
>> 2.1.1
>>>>>>>> tomorrow, it is OK that 2.1.0 does not have all feature which 1.5.0
>>>>> has,
>>>>>>>> but 2.1.1 should have all features which 1.5.0 has.
>>>>>>>> 
>>>>>>>> Sergey Shelukhin <Se...@microsoft.com.invalid>
>>>>> 于2019年1月19日周六
>>>>>>>> 上午10:23写道：
>>>>>>>> 
>>>>>>>>> Consider that we actually cannot guarantee this without a time
>>>>> machine,
>>>>>>>>> because some "newer" versions are already released.
>>>>>>>>> 
>>>>>>>>> If we backport to 1.5 now, we cannot do anything about 2.0.0,
>> 2.1.0,
>>>>>>>>> 2.1.2, etc. because they are already released... if the user
>>>> upgrades
>>>>> from
>>>>>>>>> 1.5 to 2.0.1 for example, they will lose the feature no matter
>> what.
>>>>>>>>> The only way to ensure is to
>>>>>>>>> - always update to latest dot version,
>>>>>>>>> - also for us to make sure we never release before releasing every
>>>>> "later"
>>>>>>>>> dot release (e.g. we cannot release 1.5 before 2.1.3, 2.0.N, etc.
>> so
>>>>>>>>> there's the latest release for every line).
>>>>>>>>> - and also for us to make sure that every single dot line actually
>>>>> has a
>>>>>>>>> release - when e.g. 2.0.X line is abandoned that may not happen, so
>>>>> the
>>>>>>>>> latest version of 2.0.X will precede latest 1.Y because 1.Y may
>>>> still
>>>>> be
>>>>>>>>> active (like as far as I recall 0.94 was getting dot releases even
>>>>> when
>>>>>>>>> 0.96 was abandoned) - so even if the user goes from 1.Y to the
>>>> latest
>>>>> 2.0.X
>>>>>>>>> they will lose the feature.
>>>>>>>>> 
>>>>>>>>> I think this is kind of expected... I agree that it needs to be
>>>>>>>>> documented. To an extent it already is in JIRA where fixVersion may
>>>> be
>>>>>>>>> "3.0, 2.2, 1.5", but it makes sense to document explicitly.
>>>>>>>>> 
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: 张铎(Duo Zhang) <pa...@gmail.com>
>>>>>>>>> Sent: Friday, January 18, 2019 5:50 PM
>>>>>>>>> To: HBase Dev List <de...@hbase.apache.org>
>>>>>>>>> Subject: About how features are integrated to different HBase
>>>> versions
>>>>>>>>> 
>>>>>>>>> I think we have a good discussion on HBASE-21034, where a feature
>> is
>>>>> back
>>>>>>>>> ported to branch-1, but then folks think that we should not back
>>>> port
>>>>> them
>>>>>>>>> to branch-2.1 and branch-2.0, as usually we should not add new
>>>>> features to
>>>>>>>>> minor release lines.
>>>>>>>>> 
>>>>>>>>> I think the reason why we do not want the feature in branch-2.1 and
>>>>>>>>> branch-2.0 is reasonable, but this will introduce another problem.
>>>> As
>>>>>>>>> later, we will release a 1.5.0 which has the feature, but when a
>>>> user
>>>>> later
>>>>>>>>> upgrades from 1.5.0 to 2.1.x or 2.0.x, it will find that the
>> feature
>>>>> is
>>>>>>>>> gone, even though the 2.1.x or 2.0.x is released after 1.5.0 is
>>>>> released,
>>>>>>>>> as we do not port the feature to these two branches. This will be
>>>> very
>>>>>>>>> confusing to users I'd say.
>>>>>>>>> 
>>>>>>>>> So I think we should guarantee that, a higher version of HBase
>>>> release
>>>>>>>>> will always contain all the features of a HBase release with a
>> lower
>>>>>>>>> version which is released earlier, unless explicitly mentioned(for
>>>>> example,
>>>>>>>>> DLR).
>>>>>>>>> 
>>>>>>>>> And this implies that, when we setup a new major release and make a
>>>>> new
>>>>>>>>> release on the first minor release line, then the develop branch
>> for
>>>>> the
>>>>>>>>> previous major release will be useless, as said above, usually we
>> do
>>>>> not
>>>>>>>>> want to port any new features to the minor release line of the new
>>>>> major
>>>>>>>>> release, then the new features should not be ported to previous
>>>> major
>>>>>>>>> release, otherwise we will break the guarantee above. And this also
>>>>> means
>>>>>>>>> that, we could just use the 'develop' branch to make new releases.
>>>>>>>>> 
>>>>> 
>>>> 
>> 

Re: About how features are integrated to different HBase versions

[Allan Yang <al...@apache.org>]

{quote}
For example, you release 2.1.0 yesterday, 1.5.0 today, and then 2.1.1
tomorrow, it is OK that 2.1.0 does not have all feature which 1.5.0 has,
but 2.1.1 should have all features which 1.5.0 has.
{quote}
I don't think can work, normal user mostly won't care about the release
time, they only know 2.1.1 > 2.1.0 > 1.5.0. They will think higher version
includes all the feature in lower version.

I don't get the point why we are now still backporting new features from
branch-2 to branch-1. Yes, there is many 1.x cluster in production, so we
need to release 1.x versions to fix bugs and keep them stable, as the
stable point is still in 1.x.
And at the same time, we should try to move on to 2.x, making branch-1 as a
bugfix branch for sometime before deprecating it. As far as I see, branch-1
is still very 'active', too active I think.
If we stop backport features from branch-2 to branch-1, then there is no
problem, IMHO.
Best Regards
Allan Yang


Andrew Purtell <an...@gmail.com> 于2019年1月20日周日 上午5:27写道：

> As branch RM for branch-1 I will always check to make sure a commit there
> has first been committed to branch-2. There will always be an upgrade path
> from a branch-1 based release to a branch-2 based release. The relevant
> JIRAs will either have a 1.x and 2.x fixVersion or the backport JIRA will
> be linked to the one for the branch-2 commit. When making the release notes
> we will be looking at these things (or should, anyway). We can update the
> upgrade paths documentation whenever we find this kind of linkage. Perhaps
> we can describe this for future RMs in the how to release section of the
> doc? Does this satisfy the concerns?
>
> > On Jan 18, 2019, at 11:47 PM, Sean Busbey <bu...@apache.org> wrote:
> >
> > I agree with Andrew that we can't both have maintenance releases and
> expect
> > every feature in ongoing branch-1 releases to be in branches-2.y.
> >
> > Tracking consideration for when features are available across major
> > versions fits in well with the "upgrade paths" section in the ref guide.
> >
> > We've just gotten in the habit of it only getting filled in when a big
> > release is coming up.
> >
> >
> >
> >> On Fri, Jan 18, 2019, 23:46 张铎(Duo Zhang) <palomino219@gmail.com wrote:
> >>
> >> Then we must have a upgrade path, for example, 1.5.x can only be
> upgraded
> >> to 2.2.x if you want all the features still there?
> >>
> >> Maybe we should have a release timeline for the first release of all the
> >> minor releases? So when user want to upgrade, they can choose the minor
> >> release which is released later than the current one.
> >>
> >> Andrew Purtell <an...@gmail.com>于2019年1月19日 周六13:15写道：
> >>
> >>> Also I think branch-1 releases will be done on a monthly cadence
> >>> independent of any branch-2 releases. This is because there are
> different
> >>> RMs at work with different needs and schedules.
> >>>
> >>> I can certainly help out some with branch-2 releasing if you need it,
> >>> FWIW.
> >>>
> >>> It may also help if we begin talking about 1.x and 2.x as separate
> >>> "products". This can help avoid confusion about features in 1.5 not in
> >> 2.1
> >>> but in 2.2. For all practical purposes they are separate products. Some
> >> of
> >>> our community develop and run branch-1. Others develop and run
> branch-2.
> >>> There is some overlap but the overlap is not total. The concerns will
> >>> diverge a bit. I think this is healthy. Everyone is attending to what
> >> they
> >>> need. Let's figure out how to make it work.
> >>>
> >>>> On Jan 18, 2019, at 9:04 PM, Andrew Purtell <andrew.purtell@gmail.com
> >
> >>> wrote:
> >>>>
> >>>> Also please be prepared to support forward evolution and maintenance
> of
> >>> branch-1 for, potentially, years. Because it is used in production and
> >> will
> >>> continue to do so for a long time. Features may end up in 1.6.0 that
> only
> >>> appear in 2.3 or 2.4. And in 1.7 that only appear in 2.5 or 2.6. This
> >>> shouldn't be confusing. We just need to document it. JIRA helps some,
> >>> release notes can help a lot more. Maybe in the future a feature to
> >> version
> >>> matrix in the book.
> >>>>
> >>>>> On Jan 18, 2019, at 8:59 PM, Andrew Purtell <
> andrew.purtell@gmail.com
> >>>
> >>> wrote:
> >>>>>
> >>>>> This can't work, because we can put things into a new minor that
> >> cannot
> >>> go into a patch relesse. If you say instead 2.2.0 must have everything
> in
> >>> 1.5.0, it can work. The alignment of features should happen at the
> minor
> >>> releases. If we can also have alignment in patch releases too, that
> would
> >>> be great, but can't be mandatory.
> >>>>>
> >>>>>> On Jan 18, 2019, at 7:12 PM, 张铎(Duo Zhang) <pa...@gmail.com>
> >>> wrote:
> >>>>>>
> >>>>>> Please see the red words carefully, I explicitly mentioned that, the
> >>> newer
> >>>>>> version should be released LATER, if you want to get all the
> >> features.
> >>>>>>
> >>>>>> For example, you release 2.1.0 yesterday, 1.5.0 today, and then
> 2.1.1
> >>>>>> tomorrow, it is OK that 2.1.0 does not have all feature which 1.5.0
> >>> has,
> >>>>>> but 2.1.1 should have all features which 1.5.0 has.
> >>>>>>
> >>>>>> Sergey Shelukhin <Se...@microsoft.com.invalid>
> >>> 于2019年1月19日周六
> >>>>>> 上午10:23写道：
> >>>>>>
> >>>>>>> Consider that we actually cannot guarantee this without a time
> >>> machine,
> >>>>>>> because some "newer" versions are already released.
> >>>>>>>
> >>>>>>> If we backport to 1.5 now, we cannot do anything about 2.0.0,
> 2.1.0,
> >>>>>>> 2.1.2, etc. because they are already released... if the user
> >> upgrades
> >>> from
> >>>>>>> 1.5 to 2.0.1 for example, they will lose the feature no matter
> what.
> >>>>>>> The only way to ensure is to
> >>>>>>> - always update to latest dot version,
> >>>>>>> - also for us to make sure we never release before releasing every
> >>> "later"
> >>>>>>> dot release (e.g. we cannot release 1.5 before 2.1.3, 2.0.N, etc.
> so
> >>>>>>> there's the latest release for every line).
> >>>>>>> - and also for us to make sure that every single dot line actually
> >>> has a
> >>>>>>> release - when e.g. 2.0.X line is abandoned that may not happen, so
> >>> the
> >>>>>>> latest version of 2.0.X will precede latest 1.Y because 1.Y may
> >> still
> >>> be
> >>>>>>> active (like as far as I recall 0.94 was getting dot releases even
> >>> when
> >>>>>>> 0.96 was abandoned) - so even if the user goes from 1.Y to the
> >> latest
> >>> 2.0.X
> >>>>>>> they will lose the feature.
> >>>>>>>
> >>>>>>> I think this is kind of expected... I agree that it needs to be
> >>>>>>> documented. To an extent it already is in JIRA where fixVersion may
> >> be
> >>>>>>> "3.0, 2.2, 1.5", but it makes sense to document explicitly.
> >>>>>>>
> >>>>>>> -----Original Message-----
> >>>>>>> From: 张铎(Duo Zhang) <pa...@gmail.com>
> >>>>>>> Sent: Friday, January 18, 2019 5:50 PM
> >>>>>>> To: HBase Dev List <de...@hbase.apache.org>
> >>>>>>> Subject: About how features are integrated to different HBase
> >> versions
> >>>>>>>
> >>>>>>> I think we have a good discussion on HBASE-21034, where a feature
> is
> >>> back
> >>>>>>> ported to branch-1, but then folks think that we should not back
> >> port
> >>> them
> >>>>>>> to branch-2.1 and branch-2.0, as usually we should not add new
> >>> features to
> >>>>>>> minor release lines.
> >>>>>>>
> >>>>>>> I think the reason why we do not want the feature in branch-2.1 and
> >>>>>>> branch-2.0 is reasonable, but this will introduce another problem.
> >> As
> >>>>>>> later, we will release a 1.5.0 which has the feature, but when a
> >> user
> >>> later
> >>>>>>> upgrades from 1.5.0 to 2.1.x or 2.0.x, it will find that the
> feature
> >>> is
> >>>>>>> gone, even though the 2.1.x or 2.0.x is released after 1.5.0 is
> >>> released,
> >>>>>>> as we do not port the feature to these two branches. This will be
> >> very
> >>>>>>> confusing to users I'd say.
> >>>>>>>
> >>>>>>> So I think we should guarantee that, a higher version of HBase
> >> release
> >>>>>>> will always contain all the features of a HBase release with a
> lower
> >>>>>>> version which is released earlier, unless explicitly mentioned(for
> >>> example,
> >>>>>>> DLR).
> >>>>>>>
> >>>>>>> And this implies that, when we setup a new major release and make a
> >>> new
> >>>>>>> release on the first minor release line, then the develop branch
> for
> >>> the
> >>>>>>> previous major release will be useless, as said above, usually we
> do
> >>> not
> >>>>>>> want to port any new features to the minor release line of the new
> >>> major
> >>>>>>> release, then the new features should not be ported to previous
> >> major
> >>>>>>> release, otherwise we will break the guarantee above. And this also
> >>> means
> >>>>>>> that, we could just use the 'develop' branch to make new releases.
> >>>>>>>
> >>>
> >>
>

Re: About how features are integrated to different HBase versions

[Andrew Purtell <an...@gmail.com>]

As branch RM for branch-1 I will always check to make sure a commit there has first been committed to branch-2. There will always be an upgrade path from a branch-1 based release to a branch-2 based release. The relevant JIRAs will either have a 1.x and 2.x fixVersion or the backport JIRA will be linked to the one for the branch-2 commit. When making the release notes we will be looking at these things (or should, anyway). We can update the upgrade paths documentation whenever we find this kind of linkage. Perhaps we can describe this for future RMs in the how to release section of the doc? Does this satisfy the concerns? 

> On Jan 18, 2019, at 11:47 PM, Sean Busbey <bu...@apache.org> wrote:
> 
> I agree with Andrew that we can't both have maintenance releases and expect
> every feature in ongoing branch-1 releases to be in branches-2.y.
> 
> Tracking consideration for when features are available across major
> versions fits in well with the "upgrade paths" section in the ref guide.
> 
> We've just gotten in the habit of it only getting filled in when a big
> release is coming up.
> 
> 
> 
>> On Fri, Jan 18, 2019, 23:46 张铎(Duo Zhang) <palomino219@gmail.com wrote:
>> 
>> Then we must have a upgrade path, for example, 1.5.x can only be upgraded
>> to 2.2.x if you want all the features still there?
>> 
>> Maybe we should have a release timeline for the first release of all the
>> minor releases? So when user want to upgrade, they can choose the minor
>> release which is released later than the current one.
>> 
>> Andrew Purtell <an...@gmail.com>于2019年1月19日 周六13:15写道：
>> 
>>> Also I think branch-1 releases will be done on a monthly cadence
>>> independent of any branch-2 releases. This is because there are different
>>> RMs at work with different needs and schedules.
>>> 
>>> I can certainly help out some with branch-2 releasing if you need it,
>>> FWIW.
>>> 
>>> It may also help if we begin talking about 1.x and 2.x as separate
>>> "products". This can help avoid confusion about features in 1.5 not in
>> 2.1
>>> but in 2.2. For all practical purposes they are separate products. Some
>> of
>>> our community develop and run branch-1. Others develop and run branch-2.
>>> There is some overlap but the overlap is not total. The concerns will
>>> diverge a bit. I think this is healthy. Everyone is attending to what
>> they
>>> need. Let's figure out how to make it work.
>>> 
>>>> On Jan 18, 2019, at 9:04 PM, Andrew Purtell <an...@gmail.com>
>>> wrote:
>>>> 
>>>> Also please be prepared to support forward evolution and maintenance of
>>> branch-1 for, potentially, years. Because it is used in production and
>> will
>>> continue to do so for a long time. Features may end up in 1.6.0 that only
>>> appear in 2.3 or 2.4. And in 1.7 that only appear in 2.5 or 2.6. This
>>> shouldn't be confusing. We just need to document it. JIRA helps some,
>>> release notes can help a lot more. Maybe in the future a feature to
>> version
>>> matrix in the book.
>>>> 
>>>>> On Jan 18, 2019, at 8:59 PM, Andrew Purtell <andrew.purtell@gmail.com
>>> 
>>> wrote:
>>>>> 
>>>>> This can't work, because we can put things into a new minor that
>> cannot
>>> go into a patch relesse. If you say instead 2.2.0 must have everything in
>>> 1.5.0, it can work. The alignment of features should happen at the minor
>>> releases. If we can also have alignment in patch releases too, that would
>>> be great, but can't be mandatory.
>>>>> 
>>>>>> On Jan 18, 2019, at 7:12 PM, 张铎(Duo Zhang) <pa...@gmail.com>
>>> wrote:
>>>>>> 
>>>>>> Please see the red words carefully, I explicitly mentioned that, the
>>> newer
>>>>>> version should be released LATER, if you want to get all the
>> features.
>>>>>> 
>>>>>> For example, you release 2.1.0 yesterday, 1.5.0 today, and then 2.1.1
>>>>>> tomorrow, it is OK that 2.1.0 does not have all feature which 1.5.0
>>> has,
>>>>>> but 2.1.1 should have all features which 1.5.0 has.
>>>>>> 
>>>>>> Sergey Shelukhin <Se...@microsoft.com.invalid>
>>> 于2019年1月19日周六
>>>>>> 上午10:23写道：
>>>>>> 
>>>>>>> Consider that we actually cannot guarantee this without a time
>>> machine,
>>>>>>> because some "newer" versions are already released.
>>>>>>> 
>>>>>>> If we backport to 1.5 now, we cannot do anything about 2.0.0, 2.1.0,
>>>>>>> 2.1.2, etc. because they are already released... if the user
>> upgrades
>>> from
>>>>>>> 1.5 to 2.0.1 for example, they will lose the feature no matter what.
>>>>>>> The only way to ensure is to
>>>>>>> - always update to latest dot version,
>>>>>>> - also for us to make sure we never release before releasing every
>>> "later"
>>>>>>> dot release (e.g. we cannot release 1.5 before 2.1.3, 2.0.N, etc. so
>>>>>>> there's the latest release for every line).
>>>>>>> - and also for us to make sure that every single dot line actually
>>> has a
>>>>>>> release - when e.g. 2.0.X line is abandoned that may not happen, so
>>> the
>>>>>>> latest version of 2.0.X will precede latest 1.Y because 1.Y may
>> still
>>> be
>>>>>>> active (like as far as I recall 0.94 was getting dot releases even
>>> when
>>>>>>> 0.96 was abandoned) - so even if the user goes from 1.Y to the
>> latest
>>> 2.0.X
>>>>>>> they will lose the feature.
>>>>>>> 
>>>>>>> I think this is kind of expected... I agree that it needs to be
>>>>>>> documented. To an extent it already is in JIRA where fixVersion may
>> be
>>>>>>> "3.0, 2.2, 1.5", but it makes sense to document explicitly.
>>>>>>> 
>>>>>>> -----Original Message-----
>>>>>>> From: 张铎(Duo Zhang) <pa...@gmail.com>
>>>>>>> Sent: Friday, January 18, 2019 5:50 PM
>>>>>>> To: HBase Dev List <de...@hbase.apache.org>
>>>>>>> Subject: About how features are integrated to different HBase
>> versions
>>>>>>> 
>>>>>>> I think we have a good discussion on HBASE-21034, where a feature is
>>> back
>>>>>>> ported to branch-1, but then folks think that we should not back
>> port
>>> them
>>>>>>> to branch-2.1 and branch-2.0, as usually we should not add new
>>> features to
>>>>>>> minor release lines.
>>>>>>> 
>>>>>>> I think the reason why we do not want the feature in branch-2.1 and
>>>>>>> branch-2.0 is reasonable, but this will introduce another problem.
>> As
>>>>>>> later, we will release a 1.5.0 which has the feature, but when a
>> user
>>> later
>>>>>>> upgrades from 1.5.0 to 2.1.x or 2.0.x, it will find that the feature
>>> is
>>>>>>> gone, even though the 2.1.x or 2.0.x is released after 1.5.0 is
>>> released,
>>>>>>> as we do not port the feature to these two branches. This will be
>> very
>>>>>>> confusing to users I'd say.
>>>>>>> 
>>>>>>> So I think we should guarantee that, a higher version of HBase
>> release
>>>>>>> will always contain all the features of a HBase release with a lower
>>>>>>> version which is released earlier, unless explicitly mentioned(for
>>> example,
>>>>>>> DLR).
>>>>>>> 
>>>>>>> And this implies that, when we setup a new major release and make a
>>> new
>>>>>>> release on the first minor release line, then the develop branch for
>>> the
>>>>>>> previous major release will be useless, as said above, usually we do
>>> not
>>>>>>> want to port any new features to the minor release line of the new
>>> major
>>>>>>> release, then the new features should not be ported to previous
>> major
>>>>>>> release, otherwise we will break the guarantee above. And this also
>>> means
>>>>>>> that, we could just use the 'develop' branch to make new releases.
>>>>>>> 
>>> 
>> 

Re: About how features are integrated to different HBase versions

[Sean Busbey <bu...@apache.org>]

I agree with Andrew that we can't both have maintenance releases and expect
every feature in ongoing branch-1 releases to be in branches-2.y.

Tracking consideration for when features are available across major
versions fits in well with the "upgrade paths" section in the ref guide.

We've just gotten in the habit of it only getting filled in when a big
release is coming up.



On Fri, Jan 18, 2019, 23:46 张铎(Duo Zhang) <palomino219@gmail.com wrote:

> Then we must have a upgrade path, for example, 1.5.x can only be upgraded
> to 2.2.x if you want all the features still there?
>
> Maybe we should have a release timeline for the first release of all the
> minor releases? So when user want to upgrade, they can choose the minor
> release which is released later than the current one.
>
> Andrew Purtell <an...@gmail.com>于2019年1月19日 周六13:15写道：
>
> > Also I think branch-1 releases will be done on a monthly cadence
> > independent of any branch-2 releases. This is because there are different
> > RMs at work with different needs and schedules.
> >
> > I can certainly help out some with branch-2 releasing if you need it,
> > FWIW.
> >
> > It may also help if we begin talking about 1.x and 2.x as separate
> > "products". This can help avoid confusion about features in 1.5 not in
> 2.1
> > but in 2.2. For all practical purposes they are separate products. Some
> of
> > our community develop and run branch-1. Others develop and run branch-2.
> > There is some overlap but the overlap is not total. The concerns will
> > diverge a bit. I think this is healthy. Everyone is attending to what
> they
> > need. Let's figure out how to make it work.
> >
> > > On Jan 18, 2019, at 9:04 PM, Andrew Purtell <an...@gmail.com>
> > wrote:
> > >
> > > Also please be prepared to support forward evolution and maintenance of
> > branch-1 for, potentially, years. Because it is used in production and
> will
> > continue to do so for a long time. Features may end up in 1.6.0 that only
> > appear in 2.3 or 2.4. And in 1.7 that only appear in 2.5 or 2.6. This
> > shouldn't be confusing. We just need to document it. JIRA helps some,
> > release notes can help a lot more. Maybe in the future a feature to
> version
> > matrix in the book.
> > >
> > >> On Jan 18, 2019, at 8:59 PM, Andrew Purtell <andrew.purtell@gmail.com
> >
> > wrote:
> > >>
> > >> This can't work, because we can put things into a new minor that
> cannot
> > go into a patch relesse. If you say instead 2.2.0 must have everything in
> > 1.5.0, it can work. The alignment of features should happen at the minor
> > releases. If we can also have alignment in patch releases too, that would
> > be great, but can't be mandatory.
> > >>
> > >>> On Jan 18, 2019, at 7:12 PM, 张铎(Duo Zhang) <pa...@gmail.com>
> > wrote:
> > >>>
> > >>> Please see the red words carefully, I explicitly mentioned that, the
> > newer
> > >>> version should be released LATER, if you want to get all the
> features.
> > >>>
> > >>> For example, you release 2.1.0 yesterday, 1.5.0 today, and then 2.1.1
> > >>> tomorrow, it is OK that 2.1.0 does not have all feature which 1.5.0
> > has,
> > >>> but 2.1.1 should have all features which 1.5.0 has.
> > >>>
> > >>> Sergey Shelukhin <Se...@microsoft.com.invalid>
> > 于2019年1月19日周六
> > >>> 上午10:23写道：
> > >>>
> > >>>> Consider that we actually cannot guarantee this without a time
> > machine,
> > >>>> because some "newer" versions are already released.
> > >>>>
> > >>>> If we backport to 1.5 now, we cannot do anything about 2.0.0, 2.1.0,
> > >>>> 2.1.2, etc. because they are already released... if the user
> upgrades
> > from
> > >>>> 1.5 to 2.0.1 for example, they will lose the feature no matter what.
> > >>>> The only way to ensure is to
> > >>>> - always update to latest dot version,
> > >>>> - also for us to make sure we never release before releasing every
> > "later"
> > >>>> dot release (e.g. we cannot release 1.5 before 2.1.3, 2.0.N, etc. so
> > >>>> there's the latest release for every line).
> > >>>> - and also for us to make sure that every single dot line actually
> > has a
> > >>>> release - when e.g. 2.0.X line is abandoned that may not happen, so
> > the
> > >>>> latest version of 2.0.X will precede latest 1.Y because 1.Y may
> still
> > be
> > >>>> active (like as far as I recall 0.94 was getting dot releases even
> > when
> > >>>> 0.96 was abandoned) - so even if the user goes from 1.Y to the
> latest
> > 2.0.X
> > >>>> they will lose the feature.
> > >>>>
> > >>>> I think this is kind of expected... I agree that it needs to be
> > >>>> documented. To an extent it already is in JIRA where fixVersion may
> be
> > >>>> "3.0, 2.2, 1.5", but it makes sense to document explicitly.
> > >>>>
> > >>>> -----Original Message-----
> > >>>> From: 张铎(Duo Zhang) <pa...@gmail.com>
> > >>>> Sent: Friday, January 18, 2019 5:50 PM
> > >>>> To: HBase Dev List <de...@hbase.apache.org>
> > >>>> Subject: About how features are integrated to different HBase
> versions
> > >>>>
> > >>>> I think we have a good discussion on HBASE-21034, where a feature is
> > back
> > >>>> ported to branch-1, but then folks think that we should not back
> port
> > them
> > >>>> to branch-2.1 and branch-2.0, as usually we should not add new
> > features to
> > >>>> minor release lines.
> > >>>>
> > >>>> I think the reason why we do not want the feature in branch-2.1 and
> > >>>> branch-2.0 is reasonable, but this will introduce another problem.
> As
> > >>>> later, we will release a 1.5.0 which has the feature, but when a
> user
> > later
> > >>>> upgrades from 1.5.0 to 2.1.x or 2.0.x, it will find that the feature
> > is
> > >>>> gone, even though the 2.1.x or 2.0.x is released after 1.5.0 is
> > released,
> > >>>> as we do not port the feature to these two branches. This will be
> very
> > >>>> confusing to users I'd say.
> > >>>>
> > >>>> So I think we should guarantee that, a higher version of HBase
> release
> > >>>> will always contain all the features of a HBase release with a lower
> > >>>> version which is released earlier, unless explicitly mentioned(for
> > example,
> > >>>> DLR).
> > >>>>
> > >>>> And this implies that, when we setup a new major release and make a
> > new
> > >>>> release on the first minor release line, then the develop branch for
> > the
> > >>>> previous major release will be useless, as said above, usually we do
> > not
> > >>>> want to port any new features to the minor release line of the new
> > major
> > >>>> release, then the new features should not be ported to previous
> major
> > >>>> release, otherwise we will break the guarantee above. And this also
> > means
> > >>>> that, we could just use the 'develop' branch to make new releases.
> > >>>>
> >
>

Re: About how features are integrated to different HBase versions

["张铎 (Duo Zhang)" <pa...@gmail.com>]

Then we must have a upgrade path, for example, 1.5.x can only be upgraded
to 2.2.x if you want all the features still there?

Maybe we should have a release timeline for the first release of all the
minor releases? So when user want to upgrade, they can choose the minor
release which is released later than the current one.

Andrew Purtell <an...@gmail.com>于2019年1月19日 周六13:15写道：

> Also I think branch-1 releases will be done on a monthly cadence
> independent of any branch-2 releases. This is because there are different
> RMs at work with different needs and schedules.
>
> I can certainly help out some with branch-2 releasing if you need it,
> FWIW.
>
> It may also help if we begin talking about 1.x and 2.x as separate
> "products". This can help avoid confusion about features in 1.5 not in 2.1
> but in 2.2. For all practical purposes they are separate products. Some of
> our community develop and run branch-1. Others develop and run branch-2.
> There is some overlap but the overlap is not total. The concerns will
> diverge a bit. I think this is healthy. Everyone is attending to what they
> need. Let's figure out how to make it work.
>
> > On Jan 18, 2019, at 9:04 PM, Andrew Purtell <an...@gmail.com>
> wrote:
> >
> > Also please be prepared to support forward evolution and maintenance of
> branch-1 for, potentially, years. Because it is used in production and will
> continue to do so for a long time. Features may end up in 1.6.0 that only
> appear in 2.3 or 2.4. And in 1.7 that only appear in 2.5 or 2.6. This
> shouldn't be confusing. We just need to document it. JIRA helps some,
> release notes can help a lot more. Maybe in the future a feature to version
> matrix in the book.
> >
> >> On Jan 18, 2019, at 8:59 PM, Andrew Purtell <an...@gmail.com>
> wrote:
> >>
> >> This can't work, because we can put things into a new minor that cannot
> go into a patch relesse. If you say instead 2.2.0 must have everything in
> 1.5.0, it can work. The alignment of features should happen at the minor
> releases. If we can also have alignment in patch releases too, that would
> be great, but can't be mandatory.
> >>
> >>> On Jan 18, 2019, at 7:12 PM, 张铎(Duo Zhang) <pa...@gmail.com>
> wrote:
> >>>
> >>> Please see the red words carefully, I explicitly mentioned that, the
> newer
> >>> version should be released LATER, if you want to get all the features.
> >>>
> >>> For example, you release 2.1.0 yesterday, 1.5.0 today, and then 2.1.1
> >>> tomorrow, it is OK that 2.1.0 does not have all feature which 1.5.0
> has,
> >>> but 2.1.1 should have all features which 1.5.0 has.
> >>>
> >>> Sergey Shelukhin <Se...@microsoft.com.invalid>
> 于2019年1月19日周六
> >>> 上午10:23写道：
> >>>
> >>>> Consider that we actually cannot guarantee this without a time
> machine,
> >>>> because some "newer" versions are already released.
> >>>>
> >>>> If we backport to 1.5 now, we cannot do anything about 2.0.0, 2.1.0,
> >>>> 2.1.2, etc. because they are already released... if the user upgrades
> from
> >>>> 1.5 to 2.0.1 for example, they will lose the feature no matter what.
> >>>> The only way to ensure is to
> >>>> - always update to latest dot version,
> >>>> - also for us to make sure we never release before releasing every
> "later"
> >>>> dot release (e.g. we cannot release 1.5 before 2.1.3, 2.0.N, etc. so
> >>>> there's the latest release for every line).
> >>>> - and also for us to make sure that every single dot line actually
> has a
> >>>> release - when e.g. 2.0.X line is abandoned that may not happen, so
> the
> >>>> latest version of 2.0.X will precede latest 1.Y because 1.Y may still
> be
> >>>> active (like as far as I recall 0.94 was getting dot releases even
> when
> >>>> 0.96 was abandoned) - so even if the user goes from 1.Y to the latest
> 2.0.X
> >>>> they will lose the feature.
> >>>>
> >>>> I think this is kind of expected... I agree that it needs to be
> >>>> documented. To an extent it already is in JIRA where fixVersion may be
> >>>> "3.0, 2.2, 1.5", but it makes sense to document explicitly.
> >>>>
> >>>> -----Original Message-----
> >>>> From: 张铎(Duo Zhang) <pa...@gmail.com>
> >>>> Sent: Friday, January 18, 2019 5:50 PM
> >>>> To: HBase Dev List <de...@hbase.apache.org>
> >>>> Subject: About how features are integrated to different HBase versions
> >>>>
> >>>> I think we have a good discussion on HBASE-21034, where a feature is
> back
> >>>> ported to branch-1, but then folks think that we should not back port
> them
> >>>> to branch-2.1 and branch-2.0, as usually we should not add new
> features to
> >>>> minor release lines.
> >>>>
> >>>> I think the reason why we do not want the feature in branch-2.1 and
> >>>> branch-2.0 is reasonable, but this will introduce another problem. As
> >>>> later, we will release a 1.5.0 which has the feature, but when a user
> later
> >>>> upgrades from 1.5.0 to 2.1.x or 2.0.x, it will find that the feature
> is
> >>>> gone, even though the 2.1.x or 2.0.x is released after 1.5.0 is
> released,
> >>>> as we do not port the feature to these two branches. This will be very
> >>>> confusing to users I'd say.
> >>>>
> >>>> So I think we should guarantee that, a higher version of HBase release
> >>>> will always contain all the features of a HBase release with a lower
> >>>> version which is released earlier, unless explicitly mentioned(for
> example,
> >>>> DLR).
> >>>>
> >>>> And this implies that, when we setup a new major release and make a
> new
> >>>> release on the first minor release line, then the develop branch for
> the
> >>>> previous major release will be useless, as said above, usually we do
> not
> >>>> want to port any new features to the minor release line of the new
> major
> >>>> release, then the new features should not be ported to previous major
> >>>> release, otherwise we will break the guarantee above. And this also
> means
> >>>> that, we could just use the 'develop' branch to make new releases.
> >>>>
>

Re: About how features are integrated to different HBase versions

[Andrew Purtell <an...@gmail.com>]

Also I think branch-1 releases will be done on a monthly cadence independent of any branch-2 releases. This is because there are different RMs at work with different needs and schedules. 

I can certainly help out some with branch-2 releasing if you need it, FWIW. 

It may also help if we begin talking about 1.x and 2.x as separate "products". This can help avoid confusion about features in 1.5 not in 2.1 but in 2.2. For all practical purposes they are separate products. Some of our community develop and run branch-1. Others develop and run branch-2. There is some overlap but the overlap is not total. The concerns will diverge a bit. I think this is healthy. Everyone is attending to what they need. Let's figure out how to make it work. 

> On Jan 18, 2019, at 9:04 PM, Andrew Purtell <an...@gmail.com> wrote:
> 
> Also please be prepared to support forward evolution and maintenance of branch-1 for, potentially, years. Because it is used in production and will continue to do so for a long time. Features may end up in 1.6.0 that only appear in 2.3 or 2.4. And in 1.7 that only appear in 2.5 or 2.6. This shouldn't be confusing. We just need to document it. JIRA helps some, release notes can help a lot more. Maybe in the future a feature to version matrix in the book. 
> 
>> On Jan 18, 2019, at 8:59 PM, Andrew Purtell <an...@gmail.com> wrote:
>> 
>> This can't work, because we can put things into a new minor that cannot go into a patch relesse. If you say instead 2.2.0 must have everything in 1.5.0, it can work. The alignment of features should happen at the minor releases. If we can also have alignment in patch releases too, that would be great, but can't be mandatory. 
>> 
>>> On Jan 18, 2019, at 7:12 PM, 张铎(Duo Zhang) <pa...@gmail.com> wrote:
>>> 
>>> Please see the red words carefully, I explicitly mentioned that, the newer
>>> version should be released LATER, if you want to get all the features.
>>> 
>>> For example, you release 2.1.0 yesterday, 1.5.0 today, and then 2.1.1
>>> tomorrow, it is OK that 2.1.0 does not have all feature which 1.5.0 has,
>>> but 2.1.1 should have all features which 1.5.0 has.
>>> 
>>> Sergey Shelukhin <Se...@microsoft.com.invalid> 于2019年1月19日周六
>>> 上午10:23写道：
>>> 
>>>> Consider that we actually cannot guarantee this without a time machine,
>>>> because some "newer" versions are already released.
>>>> 
>>>> If we backport to 1.5 now, we cannot do anything about 2.0.0, 2.1.0,
>>>> 2.1.2, etc. because they are already released... if the user upgrades from
>>>> 1.5 to 2.0.1 for example, they will lose the feature no matter what.
>>>> The only way to ensure is to
>>>> - always update to latest dot version,
>>>> - also for us to make sure we never release before releasing every "later"
>>>> dot release (e.g. we cannot release 1.5 before 2.1.3, 2.0.N, etc. so
>>>> there's the latest release for every line).
>>>> - and also for us to make sure that every single dot line actually has a
>>>> release - when e.g. 2.0.X line is abandoned that may not happen, so the
>>>> latest version of 2.0.X will precede latest 1.Y because 1.Y may still be
>>>> active (like as far as I recall 0.94 was getting dot releases even when
>>>> 0.96 was abandoned) - so even if the user goes from 1.Y to the latest 2.0.X
>>>> they will lose the feature.
>>>> 
>>>> I think this is kind of expected... I agree that it needs to be
>>>> documented. To an extent it already is in JIRA where fixVersion may be
>>>> "3.0, 2.2, 1.5", but it makes sense to document explicitly.
>>>> 
>>>> -----Original Message-----
>>>> From: 张铎(Duo Zhang) <pa...@gmail.com>
>>>> Sent: Friday, January 18, 2019 5:50 PM
>>>> To: HBase Dev List <de...@hbase.apache.org>
>>>> Subject: About how features are integrated to different HBase versions
>>>> 
>>>> I think we have a good discussion on HBASE-21034, where a feature is back
>>>> ported to branch-1, but then folks think that we should not back port them
>>>> to branch-2.1 and branch-2.0, as usually we should not add new features to
>>>> minor release lines.
>>>> 
>>>> I think the reason why we do not want the feature in branch-2.1 and
>>>> branch-2.0 is reasonable, but this will introduce another problem. As
>>>> later, we will release a 1.5.0 which has the feature, but when a user later
>>>> upgrades from 1.5.0 to 2.1.x or 2.0.x, it will find that the feature is
>>>> gone, even though the 2.1.x or 2.0.x is released after 1.5.0 is released,
>>>> as we do not port the feature to these two branches. This will be very
>>>> confusing to users I'd say.
>>>> 
>>>> So I think we should guarantee that, a higher version of HBase release
>>>> will always contain all the features of a HBase release with a lower
>>>> version which is released earlier, unless explicitly mentioned(for example,
>>>> DLR).
>>>> 
>>>> And this implies that, when we setup a new major release and make a new
>>>> release on the first minor release line, then the develop branch for the
>>>> previous major release will be useless, as said above, usually we do not
>>>> want to port any new features to the minor release line of the new major
>>>> release, then the new features should not be ported to previous major
>>>> release, otherwise we will break the guarantee above. And this also means
>>>> that, we could just use the 'develop' branch to make new releases.
>>>> 

Re: About how features are integrated to different HBase versions

[Andrew Purtell <an...@gmail.com>]

Also please be prepared to support forward evolution and maintenance of branch-1 for, potentially, years. Because it is used in production and will continue to do so for a long time. Features may end up in 1.6.0 that only appear in 2.3 or 2.4. And in 1.7 that only appear in 2.5 or 2.6. This shouldn't be confusing. We just need to document it. JIRA helps some, release notes can help a lot more. Maybe in the future a feature to version matrix in the book. 

> On Jan 18, 2019, at 8:59 PM, Andrew Purtell <an...@gmail.com> wrote:
> 
> This can't work, because we can put things into a new minor that cannot go into a patch relesse. If you say instead 2.2.0 must have everything in 1.5.0, it can work. The alignment of features should happen at the minor releases. If we can also have alignment in patch releases too, that would be great, but can't be mandatory. 
> 
>> On Jan 18, 2019, at 7:12 PM, 张铎(Duo Zhang) <pa...@gmail.com> wrote:
>> 
>> Please see the red words carefully, I explicitly mentioned that, the newer
>> version should be released LATER, if you want to get all the features.
>> 
>> For example, you release 2.1.0 yesterday, 1.5.0 today, and then 2.1.1
>> tomorrow, it is OK that 2.1.0 does not have all feature which 1.5.0 has,
>> but 2.1.1 should have all features which 1.5.0 has.
>> 
>> Sergey Shelukhin <Se...@microsoft.com.invalid> 于2019年1月19日周六
>> 上午10:23写道：
>> 
>>> Consider that we actually cannot guarantee this without a time machine,
>>> because some "newer" versions are already released.
>>> 
>>> If we backport to 1.5 now, we cannot do anything about 2.0.0, 2.1.0,
>>> 2.1.2, etc. because they are already released... if the user upgrades from
>>> 1.5 to 2.0.1 for example, they will lose the feature no matter what.
>>> The only way to ensure is to
>>> - always update to latest dot version,
>>> - also for us to make sure we never release before releasing every "later"
>>> dot release (e.g. we cannot release 1.5 before 2.1.3, 2.0.N, etc. so
>>> there's the latest release for every line).
>>> - and also for us to make sure that every single dot line actually has a
>>> release - when e.g. 2.0.X line is abandoned that may not happen, so the
>>> latest version of 2.0.X will precede latest 1.Y because 1.Y may still be
>>> active (like as far as I recall 0.94 was getting dot releases even when
>>> 0.96 was abandoned) - so even if the user goes from 1.Y to the latest 2.0.X
>>> they will lose the feature.
>>> 
>>> I think this is kind of expected... I agree that it needs to be
>>> documented. To an extent it already is in JIRA where fixVersion may be
>>> "3.0, 2.2, 1.5", but it makes sense to document explicitly.
>>> 
>>> -----Original Message-----
>>> From: 张铎(Duo Zhang) <pa...@gmail.com>
>>> Sent: Friday, January 18, 2019 5:50 PM
>>> To: HBase Dev List <de...@hbase.apache.org>
>>> Subject: About how features are integrated to different HBase versions
>>> 
>>> I think we have a good discussion on HBASE-21034, where a feature is back
>>> ported to branch-1, but then folks think that we should not back port them
>>> to branch-2.1 and branch-2.0, as usually we should not add new features to
>>> minor release lines.
>>> 
>>> I think the reason why we do not want the feature in branch-2.1 and
>>> branch-2.0 is reasonable, but this will introduce another problem. As
>>> later, we will release a 1.5.0 which has the feature, but when a user later
>>> upgrades from 1.5.0 to 2.1.x or 2.0.x, it will find that the feature is
>>> gone, even though the 2.1.x or 2.0.x is released after 1.5.0 is released,
>>> as we do not port the feature to these two branches. This will be very
>>> confusing to users I'd say.
>>> 
>>> So I think we should guarantee that, a higher version of HBase release
>>> will always contain all the features of a HBase release with a lower
>>> version which is released earlier, unless explicitly mentioned(for example,
>>> DLR).
>>> 
>>> And this implies that, when we setup a new major release and make a new
>>> release on the first minor release line, then the develop branch for the
>>> previous major release will be useless, as said above, usually we do not
>>> want to port any new features to the minor release line of the new major
>>> release, then the new features should not be ported to previous major
>>> release, otherwise we will break the guarantee above. And this also means
>>> that, we could just use the 'develop' branch to make new releases.
>>> 

Re: About how features are integrated to different HBase versions

[Andrew Purtell <an...@gmail.com>]

This can't work, because we can put things into a new minor that cannot go into a patch relesse. If you say instead 2.2.0 must have everything in 1.5.0, it can work. The alignment of features should happen at the minor releases. If we can also have alignment in patch releases too, that would be great, but can't be mandatory. 

> On Jan 18, 2019, at 7:12 PM, 张铎(Duo Zhang) <pa...@gmail.com> wrote:
> 
> Please see the red words carefully, I explicitly mentioned that, the newer
> version should be released LATER, if you want to get all the features.
> 
> For example, you release 2.1.0 yesterday, 1.5.0 today, and then 2.1.1
> tomorrow, it is OK that 2.1.0 does not have all feature which 1.5.0 has,
> but 2.1.1 should have all features which 1.5.0 has.
> 
> Sergey Shelukhin <Se...@microsoft.com.invalid> 于2019年1月19日周六
> 上午10:23写道：
> 
>> Consider that we actually cannot guarantee this without a time machine,
>> because some "newer" versions are already released.
>> 
>> If we backport to 1.5 now, we cannot do anything about 2.0.0, 2.1.0,
>> 2.1.2, etc. because they are already released... if the user upgrades from
>> 1.5 to 2.0.1 for example, they will lose the feature no matter what.
>> The only way to ensure is to
>> - always update to latest dot version,
>> - also for us to make sure we never release before releasing every "later"
>> dot release (e.g. we cannot release 1.5 before 2.1.3, 2.0.N, etc. so
>> there's the latest release for every line).
>> - and also for us to make sure that every single dot line actually has a
>> release - when e.g. 2.0.X line is abandoned that may not happen, so the
>> latest version of 2.0.X will precede latest 1.Y because 1.Y may still be
>> active (like as far as I recall 0.94 was getting dot releases even when
>> 0.96 was abandoned) - so even if the user goes from 1.Y to the latest 2.0.X
>> they will lose the feature.
>> 
>> I think this is kind of expected... I agree that it needs to be
>> documented. To an extent it already is in JIRA where fixVersion may be
>> "3.0, 2.2, 1.5", but it makes sense to document explicitly.
>> 
>> -----Original Message-----
>> From: 张铎(Duo Zhang) <pa...@gmail.com>
>> Sent: Friday, January 18, 2019 5:50 PM
>> To: HBase Dev List <de...@hbase.apache.org>
>> Subject: About how features are integrated to different HBase versions
>> 
>> I think we have a good discussion on HBASE-21034, where a feature is back
>> ported to branch-1, but then folks think that we should not back port them
>> to branch-2.1 and branch-2.0, as usually we should not add new features to
>> minor release lines.
>> 
>> I think the reason why we do not want the feature in branch-2.1 and
>> branch-2.0 is reasonable, but this will introduce another problem. As
>> later, we will release a 1.5.0 which has the feature, but when a user later
>> upgrades from 1.5.0 to 2.1.x or 2.0.x, it will find that the feature is
>> gone, even though the 2.1.x or 2.0.x is released after 1.5.0 is released,
>> as we do not port the feature to these two branches. This will be very
>> confusing to users I'd say.
>> 
>> So I think we should guarantee that, a higher version of HBase release
>> will always contain all the features of a HBase release with a lower
>> version which is released earlier, unless explicitly mentioned(for example,
>> DLR).
>> 
>> And this implies that, when we setup a new major release and make a new
>> release on the first minor release line, then the develop branch for the
>> previous major release will be useless, as said above, usually we do not
>> want to port any new features to the minor release line of the new major
>> release, then the new features should not be ported to previous major
>> release, otherwise we will break the guarantee above. And this also means
>> that, we could just use the 'develop' branch to make new releases.
>> 

Re: About how features are integrated to different HBase versions

["张铎 (Duo Zhang)" <pa...@gmail.com>]

And yes we should document this.

And also we need to have a web page to list all the releases in timeline?
So user could know which version is safe to use when upgrading easily.

张铎(Duo Zhang) <pa...@gmail.com> 于2019年1月19日周六 上午11:12写道：

> Please see the red words carefully, I explicitly mentioned that, the newer
> version should be released LATER, if you want to get all the features.
>
> For example, you release 2.1.0 yesterday, 1.5.0 today, and then 2.1.1
> tomorrow, it is OK that 2.1.0 does not have all feature which 1.5.0 has,
> but 2.1.1 should have all features which 1.5.0 has.
>
> Sergey Shelukhin <Se...@microsoft.com.invalid> 于2019年1月19日周六
> 上午10:23写道：
>
>> Consider that we actually cannot guarantee this without a time machine,
>> because some "newer" versions are already released.
>>
>> If we backport to 1.5 now, we cannot do anything about 2.0.0, 2.1.0,
>> 2.1.2, etc. because they are already released... if the user upgrades from
>> 1.5 to 2.0.1 for example, they will lose the feature no matter what.
>> The only way to ensure is to
>> - always update to latest dot version,
>> - also for us to make sure we never release before releasing every
>> "later" dot release (e.g. we cannot release 1.5 before 2.1.3, 2.0.N, etc.
>> so there's the latest release for every line).
>> - and also for us to make sure that every single dot line actually has a
>> release - when e.g. 2.0.X line is abandoned that may not happen, so the
>> latest version of 2.0.X will precede latest 1.Y because 1.Y may still be
>> active (like as far as I recall 0.94 was getting dot releases even when
>> 0.96 was abandoned) - so even if the user goes from 1.Y to the latest 2.0.X
>> they will lose the feature.
>>
>> I think this is kind of expected... I agree that it needs to be
>> documented. To an extent it already is in JIRA where fixVersion may be
>> "3.0, 2.2, 1.5", but it makes sense to document explicitly.
>>
>> -----Original Message-----
>> From: 张铎(Duo Zhang) <pa...@gmail.com>
>> Sent: Friday, January 18, 2019 5:50 PM
>> To: HBase Dev List <de...@hbase.apache.org>
>> Subject: About how features are integrated to different HBase versions
>>
>> I think we have a good discussion on HBASE-21034, where a feature is back
>> ported to branch-1, but then folks think that we should not back port them
>> to branch-2.1 and branch-2.0, as usually we should not add new features to
>> minor release lines.
>>
>> I think the reason why we do not want the feature in branch-2.1 and
>> branch-2.0 is reasonable, but this will introduce another problem. As
>> later, we will release a 1.5.0 which has the feature, but when a user later
>> upgrades from 1.5.0 to 2.1.x or 2.0.x, it will find that the feature is
>> gone, even though the 2.1.x or 2.0.x is released after 1.5.0 is released,
>> as we do not port the feature to these two branches. This will be very
>> confusing to users I'd say.
>>
>> So I think we should guarantee that, a higher version of HBase release
>> will always contain all the features of a HBase release with a lower
>> version which is released earlier, unless explicitly mentioned(for example,
>> DLR).
>>
>> And this implies that, when we setup a new major release and make a new
>> release on the first minor release line, then the develop branch for the
>> previous major release will be useless, as said above, usually we do not
>> want to port any new features to the minor release line of the new major
>> release, then the new features should not be ported to previous major
>> release, otherwise we will break the guarantee above. And this also means
>> that, we could just use the 'develop' branch to make new releases.
>>
>

Re: About how features are integrated to different HBase versions

["张铎 (Duo Zhang)" <pa...@gmail.com>]

Please see the red words carefully, I explicitly mentioned that, the newer
version should be released LATER, if you want to get all the features.

For example, you release 2.1.0 yesterday, 1.5.0 today, and then 2.1.1
tomorrow, it is OK that 2.1.0 does not have all feature which 1.5.0 has,
but 2.1.1 should have all features which 1.5.0 has.

Sergey Shelukhin <Se...@microsoft.com.invalid> 于2019年1月19日周六
上午10:23写道：

> Consider that we actually cannot guarantee this without a time machine,
> because some "newer" versions are already released.
>
> If we backport to 1.5 now, we cannot do anything about 2.0.0, 2.1.0,
> 2.1.2, etc. because they are already released... if the user upgrades from
> 1.5 to 2.0.1 for example, they will lose the feature no matter what.
> The only way to ensure is to
> - always update to latest dot version,
> - also for us to make sure we never release before releasing every "later"
> dot release (e.g. we cannot release 1.5 before 2.1.3, 2.0.N, etc. so
> there's the latest release for every line).
> - and also for us to make sure that every single dot line actually has a
> release - when e.g. 2.0.X line is abandoned that may not happen, so the
> latest version of 2.0.X will precede latest 1.Y because 1.Y may still be
> active (like as far as I recall 0.94 was getting dot releases even when
> 0.96 was abandoned) - so even if the user goes from 1.Y to the latest 2.0.X
> they will lose the feature.
>
> I think this is kind of expected... I agree that it needs to be
> documented. To an extent it already is in JIRA where fixVersion may be
> "3.0, 2.2, 1.5", but it makes sense to document explicitly.
>
> -----Original Message-----
> From: 张铎(Duo Zhang) <pa...@gmail.com>
> Sent: Friday, January 18, 2019 5:50 PM
> To: HBase Dev List <de...@hbase.apache.org>
> Subject: About how features are integrated to different HBase versions
>
> I think we have a good discussion on HBASE-21034, where a feature is back
> ported to branch-1, but then folks think that we should not back port them
> to branch-2.1 and branch-2.0, as usually we should not add new features to
> minor release lines.
>
> I think the reason why we do not want the feature in branch-2.1 and
> branch-2.0 is reasonable, but this will introduce another problem. As
> later, we will release a 1.5.0 which has the feature, but when a user later
> upgrades from 1.5.0 to 2.1.x or 2.0.x, it will find that the feature is
> gone, even though the 2.1.x or 2.0.x is released after 1.5.0 is released,
> as we do not port the feature to these two branches. This will be very
> confusing to users I'd say.
>
> So I think we should guarantee that, a higher version of HBase release
> will always contain all the features of a HBase release with a lower
> version which is released earlier, unless explicitly mentioned(for example,
> DLR).
>
> And this implies that, when we setup a new major release and make a new
> release on the first minor release line, then the develop branch for the
> previous major release will be useless, as said above, usually we do not
> want to port any new features to the minor release line of the new major
> release, then the new features should not be ported to previous major
> release, otherwise we will break the guarantee above. And this also means
> that, we could just use the 'develop' branch to make new releases.
>

RE: About how features are integrated to different HBase versions

[Sergey Shelukhin <Se...@microsoft.com.INVALID>]

Consider that we actually cannot guarantee this without a time machine, because some "newer" versions are already released.

If we backport to 1.5 now, we cannot do anything about 2.0.0, 2.1.0, 2.1.2, etc. because they are already released... if the user upgrades from 1.5 to 2.0.1 for example, they will lose the feature no matter what. 
The only way to ensure is to
- always update to latest dot version,
- also for us to make sure we never release before releasing every "later" dot release (e.g. we cannot release 1.5 before 2.1.3, 2.0.N, etc. so there's the latest release for every line).
- and also for us to make sure that every single dot line actually has a release - when e.g. 2.0.X line is abandoned that may not happen, so the latest version of 2.0.X will precede latest 1.Y because 1.Y may still be active (like as far as I recall 0.94 was getting dot releases even when 0.96 was abandoned) - so even if the user goes from 1.Y to the latest 2.0.X they will lose the feature.

I think this is kind of expected... I agree that it needs to be documented. To an extent it already is in JIRA where fixVersion may be "3.0, 2.2, 1.5", but it makes sense to document explicitly.

-----Original Message-----
From: 张铎(Duo Zhang) <pa...@gmail.com> 
Sent: Friday, January 18, 2019 5:50 PM
To: HBase Dev List <de...@hbase.apache.org>
Subject: About how features are integrated to different HBase versions

I think we have a good discussion on HBASE-21034, where a feature is back ported to branch-1, but then folks think that we should not back port them to branch-2.1 and branch-2.0, as usually we should not add new features to minor release lines.

I think the reason why we do not want the feature in branch-2.1 and
branch-2.0 is reasonable, but this will introduce another problem. As later, we will release a 1.5.0 which has the feature, but when a user later upgrades from 1.5.0 to 2.1.x or 2.0.x, it will find that the feature is gone, even though the 2.1.x or 2.0.x is released after 1.5.0 is released, as we do not port the feature to these two branches. This will be very confusing to users I'd say.

So I think we should guarantee that, a higher version of HBase release will always contain all the features of a HBase release with a lower version which is released earlier, unless explicitly mentioned(for example, DLR).

And this implies that, when we setup a new major release and make a new release on the first minor release line, then the develop branch for the previous major release will be useless, as said above, usually we do not want to port any new features to the minor release line of the new major release, then the new features should not be ported to previous major release, otherwise we will break the guarantee above. And this also means that, we could just use the 'develop' branch to make new releases.