You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@hbase.apache.org by Enis Söztutar <en...@apache.org> on 2014/06/02 23:24:11 UTC

[VOTE] Merge branch HBASE-10070 to trunk

Hi,

Last week we started some discussion[4] for merging branch hbase-10070[1]
into trunk. It seems like the consensus there is to do the merge sooner
rather than later.


We had branched hbase-10070 in Feb out of trunk[5]. The branch contains 55
jiras committed[2]. Out of these 55, 15 has already been committed to trunk
and backported to hbase-10070 branch[3].

This VOTE is for merging back the remaining changes in branch to trunk. If
passes, we will rebase the branch on top of current trunk, in which we will
keep the commit-per-issue log history. After that we will do a git merge
for the branch keeping the history clean and not squashing the commits. I
expect rebasing to be straightforward, however with some manual conflict
resolution. After the merge we'll keep running the tests to make sure
everything is ok.

An overview of the changes, and the status of the work can be found under
[4], [6] and [7].In summary, with the code in branch, you can create tables
with region replicas, do gets / multi gets and scans using TIMELINE
consistency with high availability. Region replicas periodically scan the
files of the primary and pick up flushed / committed files. The RPC paths /
assignment, balancing etc are pretty stable. However some more performance
analysis and tuning is needed. Phase 2 work is being worked on under
HBASE-11183, and we have some working prototype for async-replicating and
region splits. However, we believe even without those features, this work
is useable (especially for read-only/bulk load tables) , and can be
released as an experimental feature in 1.0.

Please indicate your choice:

[ ] +1 on yes, merge branch hbase-10070 to trunk.
[ ] 0 on don't care
[ ] -1 don't merge, because ...

I'll keep the vote running for 7 days, and close it Mon 9th of June, PDT.

Here is my official +1.

Thanks,
Enis

[1]
https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=log;h=refs/heads/hbase-10070
[2]
https://issues.apache.org/jira/browse/HBASE-11214?jql=fixVersion%20%3D%20hbase-10070%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolved
[3]
https://issues.apache.org/jira/browse/HBASE-10792?jql=fixVersion%20%3D%20hbase-10070%20and%20fixversion%20%3D%200.99.0%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolved
[4] https://www.mail-archive.com/dev@hbase.apache.org/msg25795.html
[5]
https://github.com/apache/hbase/commit/e22c7efeac02efde3451a0c9ff9bdcd2725576d0
[6]
http://www.slideshare.net/enissoz/hbase-high-availability-for-reads-with-time
[7] https://issues.apache.org/jira/browse/HBASE-10070

Re: [VOTE] Merge branch HBASE-10070 to trunk

Posted by Stack <st...@duboce.net>.

Pardon me, what about this option: "I am fine with pushing the rebased
master branch for review to main repo
before the merge to another branch."  Is the +1 for this -- I wouldn't mind
having this option -- or for merge into master?
St.Ack




On Fri, Jun 6, 2014 at 1:46 PM, Enis Söztutar <en...@gmail.com> wrote:

> Sorry, I was mostly out for HadoopSummit.
>
> Yes, the git flow would be very similar to what you propose:
>
>      $ git checkout HBASE-10070
>      $ git rebase --ignore-date master
>      (fixups, git add, git rebase --continue, etc, etc, etc)
>      $ git checkout master
>
>      $ git push origin HBASE-10070 HBASE-10070-rebase-date   (optionally)
>      $ git merge HBASE-10070
>
> We can either go --ignore-date or not depending on what we want. If needed
> I am fine with pushing the rebased master branch for review to main repo
> before the merge to another branch. If not, I can just rebase the branch
> locally and merge + push to main repo.
>
> Creating final patches and attaching them to jira might be cumbersome. If
> we do the rebased-branch on repo, we might not need it. But if we need that
> for review, I can do it.
>
> Thanks,
> Enis
>
>
>
> On Wed, Jun 4, 2014 at 10:48 AM, Andrew Purtell <ap...@apache.org>
> wrote:
>
> > I realize this is a vote thread but I need a satisfactory answer to the
> > below inquiries before feeling comfortable casting a vote. Or perhaps
> that
> > means we need to cancel this vote and move back to discussion.
> >
> >
> > On Tue, Jun 3, 2014 at 11:17 AM, Andrew Purtell <ap...@apache.org>
> > wrote:
> >
> > > Also after the merge process is completed, do you plan to use git
> > > format-patch to break out the per-JIRA changes into updated patches for
> > > those JIRAs representing in effect the final commit?
> > >
> > >
> > > On Tue, Jun 3, 2014 at 11:16 AM, Andrew Purtell <ap...@apache.org>
> > > wrote:
> > >
> > >>  
> > >> On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <en...@apache.org>
> wrote:
> > >>
> > >> This VOTE is for merging back the remaining changes in branch to
> trunk.
> > If
> > >>> passes, we will rebase the branch on top of current trunk, in which
> we
> > >>> will
> > >>> keep the commit-per-issue log history. After that we will do a git
> > merge
> > >>> for the branch keeping the history clean and not squashing the
> > commits. I
> > >>> expect rebasing to be straightforward, however with some manual
> > conflict
> > >>> resolution. After the merge we'll keep running the tests to make sure
> > >>> everything is ok.
> > >>>
> > >>
> > >> Just to clarify that would look something like this:
> > >>
> > >>      $ git checkout HBASE-10070
> > >>      $ git rebase --ignore-date master
> > >>      (fixups, git add, git rebase --continue, etc, etc, etc)
> > >>      $ git checkout master
> > >>      $ git merge HBASE-10070
> > >>
> > >> ?
> > >>
> > >> That sounds good to me, the final merge should be a fast forward
> merge.
> > >>
> > >> Use of ' --ignore-date' could be mildly controversial. It's not
> strictly
> > >> necessary because the commits for 10070 will appear grouped in
> history,
> > but
> > >> then dates on commits will be discontiguous in that section of
> history.
> > I
> > >> suggest using that option so the order of commits and dates sort the
> > same
> > >> on master.
> > >>
> > >>
> > >> On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <en...@apache.org>
> wrote:
> > >>
> > >>> Hi,
> > >>>
> > >>> Last week we started some discussion[4] for merging branch
> > hbase-10070[1]
> > >>> into trunk. It seems like the consensus there is to do the merge
> sooner
> > >>> rather than later.
> > >>>
> > >>>
> > >>> We had branched hbase-10070 in Feb out of trunk[5]. The branch
> contains
> > >>> 55
> > >>> jiras committed[2]. Out of these 55, 15 has already been committed to
> > >>> trunk
> > >>> and backported to hbase-10070 branch[3].
> > >>>
> > >>> This VOTE is for merging back the remaining changes in branch to
> trunk.
> > >>> If
> > >>> passes, we will rebase the branch on top of current trunk, in which
> we
> > >>> will
> > >>> keep the commit-per-issue log history. After that we will do a git
> > merge
> > >>> for the branch keeping the history clean and not squashing the
> > commits. I
> > >>> expect rebasing to be straightforward, however with some manual
> > conflict
> > >>> resolution. After the merge we'll keep running the tests to make sure
> > >>> everything is ok.
> > >>>
> > >>> An overview of the changes, and the status of the work can be found
> > under
> > >>> [4], [6] and [7].In summary, with the code in branch, you can create
> > >>> tables
> > >>> with region replicas, do gets / multi gets and scans using TIMELINE
> > >>> consistency with high availability. Region replicas periodically scan
> > the
> > >>> files of the primary and pick up flushed / committed files. The RPC
> > >>> paths /
> > >>> assignment, balancing etc are pretty stable. However some more
> > >>> performance
> > >>> analysis and tuning is needed. Phase 2 work is being worked on under
> > >>> HBASE-11183, and we have some working prototype for async-replicating
> > and
> > >>> region splits. However, we believe even without those features, this
> > work
> > >>> is useable (especially for read-only/bulk load tables) , and can be
> > >>> released as an experimental feature in 1.0.
> > >>>
> > >>> Please indicate your choice:
> > >>>
> > >>> [ ] +1 on yes, merge branch hbase-10070 to trunk.
> > >>> [ ] 0 on don't care
> > >>> [ ] -1 don't merge, because ...
> > >>>
> > >>> I'll keep the vote running for 7 days, and close it Mon 9th of June,
> > PDT.
> > >>>
> > >>> Here is my official +1.
> > >>>
> > >>> Thanks,
> > >>> Enis
> > >>>
> > >>> [1]
> > >>>
> > >>>
> >
> https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=log;h=refs/heads/hbase-10070
> > >>> [2]
> > >>>
> > >>>
> >
> https://issues.apache.org/jira/browse/HBASE-11214?jql=fixVersion%20%3D%20hbase-10070%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolved
> > >>> [3]
> > >>>
> > >>>
> >
> https://issues.apache.org/jira/browse/HBASE-10792?jql=fixVersion%20%3D%20hbase-10070%20and%20fixversion%20%3D%200.99.0%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolved
> > >>> [4] https://www.mail-archive.com/dev@hbase.apache.org/msg25795.html
> > >>> [5]
> > >>>
> > >>>
> >
> https://github.com/apache/hbase/commit/e22c7efeac02efde3451a0c9ff9bdcd2725576d0
> > >>> [6]
> > >>>
> > >>>
> >
> http://www.slideshare.net/enissoz/hbase-high-availability-for-reads-with-time
> > >>> [7] https://issues.apache.org/jira/browse/HBASE-10070
> > >>>
> > >>
> > >>
> > >>
> > >> --
> > >> Best regards,
> > >>
> > >>    - Andy
> > >>
> > >> Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > >> (via Tom White)
> > >>
> > >
> > >
> > >
> > > --
> > > Best regards,
> > >
> > >    - Andy
> > >
> > > Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > > (via Tom White)
> > >
> >
> >
> >
> > --
> > Best regards,
> >
> >    - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
> >
>

Re: [VOTE] Merge branch HBASE-10070 to trunk

Posted by Jimmy Xiang <jx...@cloudera.com>.

Yes, the state for assignment is in the meta row.


On Fri, Jun 6, 2014 at 7:21 PM, Enis Söztutar <en...@gmail.com> wrote:

> On Fri, Jun 6, 2014 at 7:07 PM, Jimmy Xiang <jx...@cloudera.com> wrote:
>
> > First of all, I have not looked into the patches recently. I remember
> there
> > are some changes to the public interface. I was wondering if it is
> backward
> > compatible. Enis mentioned that it's rolling upgradable. Just want to
> > confirm if it is backward compatible. For existing applications, do they
> > need to recompile?
> >
>
> I think you are referring to the patch for
> https://issues.apache.org/jira/browse/HBASE-10347. It changes HRI to add
> the replicaId and deprecates all client-facing interfaces using HRL.
> RegionLocations replaces that, but locating regions are no longer
> client-facing interfaces. Existing applications might have to re-compile,
> but we do not have to have binary-compatibility guarantee between 0.98 and
> 1.0. AFAIK, non of the latest major releases had that guarantee to the
> previous major version (0.98 -> 0.96 -> 0.94).
>
>
> >
> > Second, there should be some conflict with HBASE-11059 I am working on
> now,
> > which is almost finished. How should we resolve this issue? ZK-less
> region
> > assignment doesn't know region replica yet.
> >
>
> I have not looked at your patch yet. The region replica assignment does not
> change the assignment mechanics on zookeeper. So assigning a secondary
> region works without any changes there. However region replicas do not have
> explicit meta locations (they share the same meta row). I am not sure where
> you keep the state for assignment. Is it in the meta row?
>
>
> >
> > Thanks,
> > Jimmy
> >
> >
> >
> >
> > On Fri, Jun 6, 2014 at 6:12 PM, Jeffrey Zhong <jz...@hortonworks.com>
> > wrote:
> >
> > >
> > > +1. It brings goodies such as region replica, cross region server
> > > scan&get, anti-affinity of regions, and pave the way for sync region
> > > replication.
> > >
> > > Thanks,
> > > -Jeffrey
> > >
> > > On 6/6/14 2:42 PM, "Devaraj Das" <dd...@hortonworks.com> wrote:
> > >
> > > >+1
> > > >
> > > >On Fri, Jun 6, 2014 at 2:13 PM, Andrew Purtell <ap...@apache.org>
> > > >wrote:
> > > >> +1, thanks Enis
> > > >>
> > > >>
> > > >> On Fri, Jun 6, 2014 at 1:46 PM, Enis Söztutar <en...@gmail.com>
> > > >>wrote:
> > > >>
> > > >>> Sorry, I was mostly out for HadoopSummit.
> > > >>>
> > > >>> Yes, the git flow would be very similar to what you propose:
> > > >>>
> > > >>>      $ git checkout HBASE-10070
> > > >>>      $ git rebase --ignore-date master
> > > >>>      (fixups, git add, git rebase --continue, etc, etc, etc)
> > > >>>      $ git checkout master
> > > >>>
> > > >>>      $ git push origin HBASE-10070 HBASE-10070-rebase-date
> > > >>>(optionally)
> > > >>>      $ git merge HBASE-10070
> > > >>>
> > > >>> We can either go --ignore-date or not depending on what we want. If
> > > >>>needed
> > > >>> I am fine with pushing the rebased master branch for review to main
> > > >>>repo
> > > >>> before the merge to another branch. If not, I can just rebase the
> > > >>>branch
> > > >>> locally and merge + push to main repo.
> > > >>>
> > > >>> Creating final patches and attaching them to jira might be
> > cumbersome.
> > > >>>If
> > > >>> we do the rebased-branch on repo, we might not need it. But if we
> > need
> > > >>>that
> > > >>> for review, I can do it.
> > > >>>
> > > >>> Thanks,
> > > >>> Enis
> > > >>>
> > > >>>
> > > >>>
> > > >>> On Wed, Jun 4, 2014 at 10:48 AM, Andrew Purtell <
> apurtell@apache.org
> > >
> > > >>> wrote:
> > > >>>
> > > >>> > I realize this is a vote thread but I need a satisfactory answer
> to
> > > >>>the
> > > >>> > below inquiries before feeling comfortable casting a vote. Or
> > perhaps
> > > >>> that
> > > >>> > means we need to cancel this vote and move back to discussion.
> > > >>> >
> > > >>> >
> > > >>> > On Tue, Jun 3, 2014 at 11:17 AM, Andrew Purtell <
> > apurtell@apache.org
> > > >
> > > >>> > wrote:
> > > >>> >
> > > >>> > > Also after the merge process is completed, do you plan to use
> git
> > > >>> > > format-patch to break out the per-JIRA changes into updated
> > > >>>patches for
> > > >>> > > those JIRAs representing in effect the final commit?
> > > >>> > >
> > > >>> > >
> > > >>> > > On Tue, Jun 3, 2014 at 11:16 AM, Andrew Purtell
> > > >>><ap...@apache.org>
> > > >>> > > wrote:
> > > >>> > >
> > > >>> > >>
> > > >>> > >> On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <
> enis@apache.org>
> > > >>> wrote:
> > > >>> > >>
> > > >>> > >> This VOTE is for merging back the remaining changes in branch
> to
> > > >>> trunk.
> > > >>> > If
> > > >>> > >>> passes, we will rebase the branch on top of current trunk, in
> > > >>>which
> > > >>> we
> > > >>> > >>> will
> > > >>> > >>> keep the commit-per-issue log history. After that we will do
> a
> > > >>>git
> > > >>> > merge
> > > >>> > >>> for the branch keeping the history clean and not squashing
> the
> > > >>> > commits. I
> > > >>> > >>> expect rebasing to be straightforward, however with some
> manual
> > > >>> > conflict
> > > >>> > >>> resolution. After the merge we'll keep running the tests to
> > make
> > > >>>sure
> > > >>> > >>> everything is ok.
> > > >>> > >>>
> > > >>> > >>
> > > >>> > >> Just to clarify that would look something like this:
> > > >>> > >>
> > > >>> > >>      $ git checkout HBASE-10070
> > > >>> > >>      $ git rebase --ignore-date master
> > > >>> > >>      (fixups, git add, git rebase --continue, etc, etc, etc)
> > > >>> > >>      $ git checkout master
> > > >>> > >>      $ git merge HBASE-10070
> > > >>> > >>
> > > >>> > >> ?
> > > >>> > >>
> > > >>> > >> That sounds good to me, the final merge should be a fast
> forward
> > > >>> merge.
> > > >>> > >>
> > > >>> > >> Use of ' --ignore-date' could be mildly controversial. It's
> not
> > > >>> strictly
> > > >>> > >> necessary because the commits for 10070 will appear grouped in
> > > >>> history,
> > > >>> > but
> > > >>> > >> then dates on commits will be discontiguous in that section of
> > > >>> history.
> > > >>> > I
> > > >>> > >> suggest using that option so the order of commits and dates
> sort
> > > >>>the
> > > >>> > same
> > > >>> > >> on master.
> > > >>> > >>
> > > >>> > >>
> > > >>> > >> On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <
> enis@apache.org>
> > > >>> wrote:
> > > >>> > >>
> > > >>> > >>> Hi,
> > > >>> > >>>
> > > >>> > >>> Last week we started some discussion[4] for merging branch
> > > >>> > hbase-10070[1]
> > > >>> > >>> into trunk. It seems like the consensus there is to do the
> > merge
> > > >>> sooner
> > > >>> > >>> rather than later.
> > > >>> > >>>
> > > >>> > >>>
> > > >>> > >>> We had branched hbase-10070 in Feb out of trunk[5]. The
> branch
> > > >>> contains
> > > >>> > >>> 55
> > > >>> > >>> jiras committed[2]. Out of these 55, 15 has already been
> > > >>>committed to
> > > >>> > >>> trunk
> > > >>> > >>> and backported to hbase-10070 branch[3].
> > > >>> > >>>
> > > >>> > >>> This VOTE is for merging back the remaining changes in branch
> > to
> > > >>> trunk.
> > > >>> > >>> If
> > > >>> > >>> passes, we will rebase the branch on top of current trunk, in
> > > >>>which
> > > >>> we
> > > >>> > >>> will
> > > >>> > >>> keep the commit-per-issue log history. After that we will do
> a
> > > >>>git
> > > >>> > merge
> > > >>> > >>> for the branch keeping the history clean and not squashing
> the
> > > >>> > commits. I
> > > >>> > >>> expect rebasing to be straightforward, however with some
> manual
> > > >>> > conflict
> > > >>> > >>> resolution. After the merge we'll keep running the tests to
> > make
> > > >>>sure
> > > >>> > >>> everything is ok.
> > > >>> > >>>
> > > >>> > >>> An overview of the changes, and the status of the work can be
> > > >>>found
> > > >>> > under
> > > >>> > >>> [4], [6] and [7].In summary, with the code in branch, you can
> > > >>>create
> > > >>> > >>> tables
> > > >>> > >>> with region replicas, do gets / multi gets and scans using
> > > >>>TIMELINE
> > > >>> > >>> consistency with high availability. Region replicas
> > periodically
> > > >>>scan
> > > >>> > the
> > > >>> > >>> files of the primary and pick up flushed / committed files.
> The
> > > >>>RPC
> > > >>> > >>> paths /
> > > >>> > >>> assignment, balancing etc are pretty stable. However some
> more
> > > >>> > >>> performance
> > > >>> > >>> analysis and tuning is needed. Phase 2 work is being worked
> on
> > > >>>under
> > > >>> > >>> HBASE-11183, and we have some working prototype for
> > > >>>async-replicating
> > > >>> > and
> > > >>> > >>> region splits. However, we believe even without those
> features,
> > > >>>this
> > > >>> > work
> > > >>> > >>> is useable (especially for read-only/bulk load tables) , and
> > can
> > > >>>be
> > > >>> > >>> released as an experimental feature in 1.0.
> > > >>> > >>>
> > > >>> > >>> Please indicate your choice:
> > > >>> > >>>
> > > >>> > >>> [ ] +1 on yes, merge branch hbase-10070 to trunk.
> > > >>> > >>> [ ] 0 on don't care
> > > >>> > >>> [ ] -1 don't merge, because ...
> > > >>> > >>>
> > > >>> > >>> I'll keep the vote running for 7 days, and close it Mon 9th
> of
> > > >>>June,
> > > >>> > PDT.
> > > >>> > >>>
> > > >>> > >>> Here is my official +1.
> > > >>> > >>>
> > > >>> > >>> Thanks,
> > > >>> > >>> Enis
> > > >>> > >>>
> > > >>> > >>> [1]
> > > >>> > >>>
> > > >>> > >>>
> > > >>> >
> > > >>>
> > > >>>
> > >
> https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=log;h=refs/heads/h
> > > >>>base-10070
> > > >>> > >>> [2]
> > > >>> > >>>
> > > >>> > >>>
> > > >>> >
> > > >>>
> > > >>>
> > >
> https://issues.apache.org/jira/browse/HBASE-11214?jql=fixVersion%20%3D%2
> > >
> >
> >>>0hbase-10070%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolv
> > > >>>ed
> > > >>> > >>> [3]
> > > >>> > >>>
> > > >>> > >>>
> > > >>> >
> > > >>>
> > > >>>
> > >
> https://issues.apache.org/jira/browse/HBASE-10792?jql=fixVersion%20%3D%2
> > >
> >
> >>>0hbase-10070%20and%20fixversion%20%3D%200.99.0%20AND%20project%20%3D%20H
> > > >>>BASE%20AND%20status%20%3D%20resolved
> > > >>> > >>> [4]
> > > >>>https://www.mail-archive.com/dev@hbase.apache.org/msg25795.html
> > > >>> > >>> [5]
> > > >>> > >>>
> > > >>> > >>>
> > > >>> >
> > > >>>
> > > >>>
> > >
> https://github.com/apache/hbase/commit/e22c7efeac02efde3451a0c9ff9bdcd27
> > > >>>25576d0
> > > >>> > >>> [6]
> > > >>> > >>>
> > > >>> > >>>
> > > >>> >
> > > >>>
> > > >>>
> > >
> http://www.slideshare.net/enissoz/hbase-high-availability-for-reads-with
> > > >>>-time
> > > >>> > >>> [7] https://issues.apache.org/jira/browse/HBASE-10070
> > > >>> > >>>
> > > >>> > >>
> > > >>> > >>
> > > >>> > >>
> > > >>> > >> --
> > > >>> > >> Best regards,
> > > >>> > >>
> > > >>> > >>    - Andy
> > > >>> > >>
> > > >>> > >> Problems worthy of attack prove their worth by hitting back. -
> > > >>>Piet
> > > >>> Hein
> > > >>> > >> (via Tom White)
> > > >>> > >>
> > > >>> > >
> > > >>> > >
> > > >>> > >
> > > >>> > > --
> > > >>> > > Best regards,
> > > >>> > >
> > > >>> > >    - Andy
> > > >>> > >
> > > >>> > > Problems worthy of attack prove their worth by hitting back. -
> > Piet
> > > >>> Hein
> > > >>> > > (via Tom White)
> > > >>> > >
> > > >>> >
> > > >>> >
> > > >>> >
> > > >>> > --
> > > >>> > Best regards,
> > > >>> >
> > > >>> >    - Andy
> > > >>> >
> > > >>> > Problems worthy of attack prove their worth by hitting back. -
> Piet
> > > >>>Hein
> > > >>> > (via Tom White)
> > > >>> >
> > > >>>
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> Best regards,
> > > >>
> > > >>    - Andy
> > > >>
> > > >> Problems worthy of attack prove their worth by hitting back. - Piet
> > Hein
> > > >> (via Tom White)
> > > >
> > > >--
> > > >CONFIDENTIALITY NOTICE
> > > >NOTICE: This message is intended for the use of the individual or
> entity
> > > >to
> > > >which it is addressed and may contain information that is
> confidential,
> > > >privileged and exempt from disclosure under applicable law. If the
> > reader
> > > >of this message is not the intended recipient, you are hereby notified
> > > >that
> > > >any printing, copying, dissemination, distribution, disclosure or
> > > >forwarding of this communication is strictly prohibited. If you have
> > > >received this communication in error, please contact the sender
> > > >immediately
> > > >and delete it from your system. Thank You.
> > >
> > >
> > >
> > > --
> > > CONFIDENTIALITY NOTICE
> > > NOTICE: This message is intended for the use of the individual or
> entity
> > to
> > > which it is addressed and may contain information that is confidential,
> > > privileged and exempt from disclosure under applicable law. If the
> reader
> > > of this message is not the intended recipient, you are hereby notified
> > that
> > > any printing, copying, dissemination, distribution, disclosure or
> > > forwarding of this communication is strictly prohibited. If you have
> > > received this communication in error, please contact the sender
> > immediately
> > > and delete it from your system. Thank You.
> > >
> >
>

Re: [VOTE] Merge branch HBASE-10070 to trunk

Posted by Enis Söztutar <en...@gmail.com>.

On Fri, Jun 6, 2014 at 7:07 PM, Jimmy Xiang <jx...@cloudera.com> wrote:

> First of all, I have not looked into the patches recently. I remember there
> are some changes to the public interface. I was wondering if it is backward
> compatible. Enis mentioned that it's rolling upgradable. Just want to
> confirm if it is backward compatible. For existing applications, do they
> need to recompile?
>

I think you are referring to the patch for
https://issues.apache.org/jira/browse/HBASE-10347. It changes HRI to add
the replicaId and deprecates all client-facing interfaces using HRL.
RegionLocations replaces that, but locating regions are no longer
client-facing interfaces. Existing applications might have to re-compile,
but we do not have to have binary-compatibility guarantee between 0.98 and
1.0. AFAIK, non of the latest major releases had that guarantee to the
previous major version (0.98 -> 0.96 -> 0.94).


>
> Second, there should be some conflict with HBASE-11059 I am working on now,
> which is almost finished. How should we resolve this issue? ZK-less region
> assignment doesn't know region replica yet.
>

I have not looked at your patch yet. The region replica assignment does not
change the assignment mechanics on zookeeper. So assigning a secondary
region works without any changes there. However region replicas do not have
explicit meta locations (they share the same meta row). I am not sure where
you keep the state for assignment. Is it in the meta row?


>
> Thanks,
> Jimmy
>
>
>
>
> On Fri, Jun 6, 2014 at 6:12 PM, Jeffrey Zhong <jz...@hortonworks.com>
> wrote:
>
> >
> > +1. It brings goodies such as region replica, cross region server
> > scan&get, anti-affinity of regions, and pave the way for sync region
> > replication.
> >
> > Thanks,
> > -Jeffrey
> >
> > On 6/6/14 2:42 PM, "Devaraj Das" <dd...@hortonworks.com> wrote:
> >
> > >+1
> > >
> > >On Fri, Jun 6, 2014 at 2:13 PM, Andrew Purtell <ap...@apache.org>
> > >wrote:
> > >> +1, thanks Enis
> > >>
> > >>
> > >> On Fri, Jun 6, 2014 at 1:46 PM, Enis Söztutar <en...@gmail.com>
> > >>wrote:
> > >>
> > >>> Sorry, I was mostly out for HadoopSummit.
> > >>>
> > >>> Yes, the git flow would be very similar to what you propose:
> > >>>
> > >>>      $ git checkout HBASE-10070
> > >>>      $ git rebase --ignore-date master
> > >>>      (fixups, git add, git rebase --continue, etc, etc, etc)
> > >>>      $ git checkout master
> > >>>
> > >>>      $ git push origin HBASE-10070 HBASE-10070-rebase-date
> > >>>(optionally)
> > >>>      $ git merge HBASE-10070
> > >>>
> > >>> We can either go --ignore-date or not depending on what we want. If
> > >>>needed
> > >>> I am fine with pushing the rebased master branch for review to main
> > >>>repo
> > >>> before the merge to another branch. If not, I can just rebase the
> > >>>branch
> > >>> locally and merge + push to main repo.
> > >>>
> > >>> Creating final patches and attaching them to jira might be
> cumbersome.
> > >>>If
> > >>> we do the rebased-branch on repo, we might not need it. But if we
> need
> > >>>that
> > >>> for review, I can do it.
> > >>>
> > >>> Thanks,
> > >>> Enis
> > >>>
> > >>>
> > >>>
> > >>> On Wed, Jun 4, 2014 at 10:48 AM, Andrew Purtell <apurtell@apache.org
> >
> > >>> wrote:
> > >>>
> > >>> > I realize this is a vote thread but I need a satisfactory answer to
> > >>>the
> > >>> > below inquiries before feeling comfortable casting a vote. Or
> perhaps
> > >>> that
> > >>> > means we need to cancel this vote and move back to discussion.
> > >>> >
> > >>> >
> > >>> > On Tue, Jun 3, 2014 at 11:17 AM, Andrew Purtell <
> apurtell@apache.org
> > >
> > >>> > wrote:
> > >>> >
> > >>> > > Also after the merge process is completed, do you plan to use git
> > >>> > > format-patch to break out the per-JIRA changes into updated
> > >>>patches for
> > >>> > > those JIRAs representing in effect the final commit?
> > >>> > >
> > >>> > >
> > >>> > > On Tue, Jun 3, 2014 at 11:16 AM, Andrew Purtell
> > >>><ap...@apache.org>
> > >>> > > wrote:
> > >>> > >
> > >>> > >>
> > >>> > >> On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <en...@apache.org>
> > >>> wrote:
> > >>> > >>
> > >>> > >> This VOTE is for merging back the remaining changes in branch to
> > >>> trunk.
> > >>> > If
> > >>> > >>> passes, we will rebase the branch on top of current trunk, in
> > >>>which
> > >>> we
> > >>> > >>> will
> > >>> > >>> keep the commit-per-issue log history. After that we will do a
> > >>>git
> > >>> > merge
> > >>> > >>> for the branch keeping the history clean and not squashing the
> > >>> > commits. I
> > >>> > >>> expect rebasing to be straightforward, however with some manual
> > >>> > conflict
> > >>> > >>> resolution. After the merge we'll keep running the tests to
> make
> > >>>sure
> > >>> > >>> everything is ok.
> > >>> > >>>
> > >>> > >>
> > >>> > >> Just to clarify that would look something like this:
> > >>> > >>
> > >>> > >>      $ git checkout HBASE-10070
> > >>> > >>      $ git rebase --ignore-date master
> > >>> > >>      (fixups, git add, git rebase --continue, etc, etc, etc)
> > >>> > >>      $ git checkout master
> > >>> > >>      $ git merge HBASE-10070
> > >>> > >>
> > >>> > >> ?
> > >>> > >>
> > >>> > >> That sounds good to me, the final merge should be a fast forward
> > >>> merge.
> > >>> > >>
> > >>> > >> Use of ' --ignore-date' could be mildly controversial. It's not
> > >>> strictly
> > >>> > >> necessary because the commits for 10070 will appear grouped in
> > >>> history,
> > >>> > but
> > >>> > >> then dates on commits will be discontiguous in that section of
> > >>> history.
> > >>> > I
> > >>> > >> suggest using that option so the order of commits and dates sort
> > >>>the
> > >>> > same
> > >>> > >> on master.
> > >>> > >>
> > >>> > >>
> > >>> > >> On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <en...@apache.org>
> > >>> wrote:
> > >>> > >>
> > >>> > >>> Hi,
> > >>> > >>>
> > >>> > >>> Last week we started some discussion[4] for merging branch
> > >>> > hbase-10070[1]
> > >>> > >>> into trunk. It seems like the consensus there is to do the
> merge
> > >>> sooner
> > >>> > >>> rather than later.
> > >>> > >>>
> > >>> > >>>
> > >>> > >>> We had branched hbase-10070 in Feb out of trunk[5]. The branch
> > >>> contains
> > >>> > >>> 55
> > >>> > >>> jiras committed[2]. Out of these 55, 15 has already been
> > >>>committed to
> > >>> > >>> trunk
> > >>> > >>> and backported to hbase-10070 branch[3].
> > >>> > >>>
> > >>> > >>> This VOTE is for merging back the remaining changes in branch
> to
> > >>> trunk.
> > >>> > >>> If
> > >>> > >>> passes, we will rebase the branch on top of current trunk, in
> > >>>which
> > >>> we
> > >>> > >>> will
> > >>> > >>> keep the commit-per-issue log history. After that we will do a
> > >>>git
> > >>> > merge
> > >>> > >>> for the branch keeping the history clean and not squashing the
> > >>> > commits. I
> > >>> > >>> expect rebasing to be straightforward, however with some manual
> > >>> > conflict
> > >>> > >>> resolution. After the merge we'll keep running the tests to
> make
> > >>>sure
> > >>> > >>> everything is ok.
> > >>> > >>>
> > >>> > >>> An overview of the changes, and the status of the work can be
> > >>>found
> > >>> > under
> > >>> > >>> [4], [6] and [7].In summary, with the code in branch, you can
> > >>>create
> > >>> > >>> tables
> > >>> > >>> with region replicas, do gets / multi gets and scans using
> > >>>TIMELINE
> > >>> > >>> consistency with high availability. Region replicas
> periodically
> > >>>scan
> > >>> > the
> > >>> > >>> files of the primary and pick up flushed / committed files. The
> > >>>RPC
> > >>> > >>> paths /
> > >>> > >>> assignment, balancing etc are pretty stable. However some more
> > >>> > >>> performance
> > >>> > >>> analysis and tuning is needed. Phase 2 work is being worked on
> > >>>under
> > >>> > >>> HBASE-11183, and we have some working prototype for
> > >>>async-replicating
> > >>> > and
> > >>> > >>> region splits. However, we believe even without those features,
> > >>>this
> > >>> > work
> > >>> > >>> is useable (especially for read-only/bulk load tables) , and
> can
> > >>>be
> > >>> > >>> released as an experimental feature in 1.0.
> > >>> > >>>
> > >>> > >>> Please indicate your choice:
> > >>> > >>>
> > >>> > >>> [ ] +1 on yes, merge branch hbase-10070 to trunk.
> > >>> > >>> [ ] 0 on don't care
> > >>> > >>> [ ] -1 don't merge, because ...
> > >>> > >>>
> > >>> > >>> I'll keep the vote running for 7 days, and close it Mon 9th of
> > >>>June,
> > >>> > PDT.
> > >>> > >>>
> > >>> > >>> Here is my official +1.
> > >>> > >>>
> > >>> > >>> Thanks,
> > >>> > >>> Enis
> > >>> > >>>
> > >>> > >>> [1]
> > >>> > >>>
> > >>> > >>>
> > >>> >
> > >>>
> > >>>
> > https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=log;h=refs/heads/h
> > >>>base-10070
> > >>> > >>> [2]
> > >>> > >>>
> > >>> > >>>
> > >>> >
> > >>>
> > >>>
> > https://issues.apache.org/jira/browse/HBASE-11214?jql=fixVersion%20%3D%2
> >
> >>>0hbase-10070%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolv
> > >>>ed
> > >>> > >>> [3]
> > >>> > >>>
> > >>> > >>>
> > >>> >
> > >>>
> > >>>
> > https://issues.apache.org/jira/browse/HBASE-10792?jql=fixVersion%20%3D%2
> >
> >>>0hbase-10070%20and%20fixversion%20%3D%200.99.0%20AND%20project%20%3D%20H
> > >>>BASE%20AND%20status%20%3D%20resolved
> > >>> > >>> [4]
> > >>>https://www.mail-archive.com/dev@hbase.apache.org/msg25795.html
> > >>> > >>> [5]
> > >>> > >>>
> > >>> > >>>
> > >>> >
> > >>>
> > >>>
> > https://github.com/apache/hbase/commit/e22c7efeac02efde3451a0c9ff9bdcd27
> > >>>25576d0
> > >>> > >>> [6]
> > >>> > >>>
> > >>> > >>>
> > >>> >
> > >>>
> > >>>
> > http://www.slideshare.net/enissoz/hbase-high-availability-for-reads-with
> > >>>-time
> > >>> > >>> [7] https://issues.apache.org/jira/browse/HBASE-10070
> > >>> > >>>
> > >>> > >>
> > >>> > >>
> > >>> > >>
> > >>> > >> --
> > >>> > >> Best regards,
> > >>> > >>
> > >>> > >>    - Andy
> > >>> > >>
> > >>> > >> Problems worthy of attack prove their worth by hitting back. -
> > >>>Piet
> > >>> Hein
> > >>> > >> (via Tom White)
> > >>> > >>
> > >>> > >
> > >>> > >
> > >>> > >
> > >>> > > --
> > >>> > > Best regards,
> > >>> > >
> > >>> > >    - Andy
> > >>> > >
> > >>> > > Problems worthy of attack prove their worth by hitting back. -
> Piet
> > >>> Hein
> > >>> > > (via Tom White)
> > >>> > >
> > >>> >
> > >>> >
> > >>> >
> > >>> > --
> > >>> > Best regards,
> > >>> >
> > >>> >    - Andy
> > >>> >
> > >>> > Problems worthy of attack prove their worth by hitting back. - Piet
> > >>>Hein
> > >>> > (via Tom White)
> > >>> >
> > >>>
> > >>
> > >>
> > >>
> > >> --
> > >> Best regards,
> > >>
> > >>    - Andy
> > >>
> > >> Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > >> (via Tom White)
> > >
> > >--
> > >CONFIDENTIALITY NOTICE
> > >NOTICE: This message is intended for the use of the individual or entity
> > >to
> > >which it is addressed and may contain information that is confidential,
> > >privileged and exempt from disclosure under applicable law. If the
> reader
> > >of this message is not the intended recipient, you are hereby notified
> > >that
> > >any printing, copying, dissemination, distribution, disclosure or
> > >forwarding of this communication is strictly prohibited. If you have
> > >received this communication in error, please contact the sender
> > >immediately
> > >and delete it from your system. Thank You.
> >
> >
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
> >
>

Re: [VOTE] Merge branch HBASE-10070 to trunk

Posted by Jimmy Xiang <jx...@cloudera.com>.

I will set zk-less assignment off by default so that your change is not
affected. But zk-less assignment won't support replica, at least for now.


On Mon, Jun 9, 2014 at 11:01 AM, Enis Söztutar <en...@gmail.com> wrote:

> BTW, I would really prefer to finish the merge before HBASE-11059. The
> replica assignment has been working with the new patches and zk-based
> assignment for many months.
>
> Enis
>
>
> On Fri, Jun 6, 2014 at 7:07 PM, Jimmy Xiang <jx...@cloudera.com> wrote:
>
> > First of all, I have not looked into the patches recently. I remember
> there
> > are some changes to the public interface. I was wondering if it is
> backward
> > compatible. Enis mentioned that it's rolling upgradable. Just want to
> > confirm if it is backward compatible. For existing applications, do they
> > need to recompile?
> >
> > Second, there should be some conflict with HBASE-11059 I am working on
> now,
> > which is almost finished. How should we resolve this issue? ZK-less
> region
> > assignment doesn't know region replica yet.
> >
> > Thanks,
> > Jimmy
> >
> >
> >
> >
> > On Fri, Jun 6, 2014 at 6:12 PM, Jeffrey Zhong <jz...@hortonworks.com>
> > wrote:
> >
> > >
> > > +1. It brings goodies such as region replica, cross region server
> > > scan&get, anti-affinity of regions, and pave the way for sync region
> > > replication.
> > >
> > > Thanks,
> > > -Jeffrey
> > >
> > > On 6/6/14 2:42 PM, "Devaraj Das" <dd...@hortonworks.com> wrote:
> > >
> > > >+1
> > > >
> > > >On Fri, Jun 6, 2014 at 2:13 PM, Andrew Purtell <ap...@apache.org>
> > > >wrote:
> > > >> +1, thanks Enis
> > > >>
> > > >>
> > > >> On Fri, Jun 6, 2014 at 1:46 PM, Enis Söztutar <en...@gmail.com>
> > > >>wrote:
> > > >>
> > > >>> Sorry, I was mostly out for HadoopSummit.
> > > >>>
> > > >>> Yes, the git flow would be very similar to what you propose:
> > > >>>
> > > >>>      $ git checkout HBASE-10070
> > > >>>      $ git rebase --ignore-date master
> > > >>>      (fixups, git add, git rebase --continue, etc, etc, etc)
> > > >>>      $ git checkout master
> > > >>>
> > > >>>      $ git push origin HBASE-10070 HBASE-10070-rebase-date
> > > >>>(optionally)
> > > >>>      $ git merge HBASE-10070
> > > >>>
> > > >>> We can either go --ignore-date or not depending on what we want. If
> > > >>>needed
> > > >>> I am fine with pushing the rebased master branch for review to main
> > > >>>repo
> > > >>> before the merge to another branch. If not, I can just rebase the
> > > >>>branch
> > > >>> locally and merge + push to main repo.
> > > >>>
> > > >>> Creating final patches and attaching them to jira might be
> > cumbersome.
> > > >>>If
> > > >>> we do the rebased-branch on repo, we might not need it. But if we
> > need
> > > >>>that
> > > >>> for review, I can do it.
> > > >>>
> > > >>> Thanks,
> > > >>> Enis
> > > >>>
> > > >>>
> > > >>>
> > > >>> On Wed, Jun 4, 2014 at 10:48 AM, Andrew Purtell <
> apurtell@apache.org
> > >
> > > >>> wrote:
> > > >>>
> > > >>> > I realize this is a vote thread but I need a satisfactory answer
> to
> > > >>>the
> > > >>> > below inquiries before feeling comfortable casting a vote. Or
> > perhaps
> > > >>> that
> > > >>> > means we need to cancel this vote and move back to discussion.
> > > >>> >
> > > >>> >
> > > >>> > On Tue, Jun 3, 2014 at 11:17 AM, Andrew Purtell <
> > apurtell@apache.org
> > > >
> > > >>> > wrote:
> > > >>> >
> > > >>> > > Also after the merge process is completed, do you plan to use
> git
> > > >>> > > format-patch to break out the per-JIRA changes into updated
> > > >>>patches for
> > > >>> > > those JIRAs representing in effect the final commit?
> > > >>> > >
> > > >>> > >
> > > >>> > > On Tue, Jun 3, 2014 at 11:16 AM, Andrew Purtell
> > > >>><ap...@apache.org>
> > > >>> > > wrote:
> > > >>> > >
> > > >>> > >>
> > > >>> > >> On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <
> enis@apache.org>
> > > >>> wrote:
> > > >>> > >>
> > > >>> > >> This VOTE is for merging back the remaining changes in branch
> to
> > > >>> trunk.
> > > >>> > If
> > > >>> > >>> passes, we will rebase the branch on top of current trunk, in
> > > >>>which
> > > >>> we
> > > >>> > >>> will
> > > >>> > >>> keep the commit-per-issue log history. After that we will do
> a
> > > >>>git
> > > >>> > merge
> > > >>> > >>> for the branch keeping the history clean and not squashing
> the
> > > >>> > commits. I
> > > >>> > >>> expect rebasing to be straightforward, however with some
> manual
> > > >>> > conflict
> > > >>> > >>> resolution. After the merge we'll keep running the tests to
> > make
> > > >>>sure
> > > >>> > >>> everything is ok.
> > > >>> > >>>
> > > >>> > >>
> > > >>> > >> Just to clarify that would look something like this:
> > > >>> > >>
> > > >>> > >>      $ git checkout HBASE-10070
> > > >>> > >>      $ git rebase --ignore-date master
> > > >>> > >>      (fixups, git add, git rebase --continue, etc, etc, etc)
> > > >>> > >>      $ git checkout master
> > > >>> > >>      $ git merge HBASE-10070
> > > >>> > >>
> > > >>> > >> ?
> > > >>> > >>
> > > >>> > >> That sounds good to me, the final merge should be a fast
> forward
> > > >>> merge.
> > > >>> > >>
> > > >>> > >> Use of ' --ignore-date' could be mildly controversial. It's
> not
> > > >>> strictly
> > > >>> > >> necessary because the commits for 10070 will appear grouped in
> > > >>> history,
> > > >>> > but
> > > >>> > >> then dates on commits will be discontiguous in that section of
> > > >>> history.
> > > >>> > I
> > > >>> > >> suggest using that option so the order of commits and dates
> sort
> > > >>>the
> > > >>> > same
> > > >>> > >> on master.
> > > >>> > >>
> > > >>> > >>
> > > >>> > >> On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <
> enis@apache.org>
> > > >>> wrote:
> > > >>> > >>
> > > >>> > >>> Hi,
> > > >>> > >>>
> > > >>> > >>> Last week we started some discussion[4] for merging branch
> > > >>> > hbase-10070[1]
> > > >>> > >>> into trunk. It seems like the consensus there is to do the
> > merge
> > > >>> sooner
> > > >>> > >>> rather than later.
> > > >>> > >>>
> > > >>> > >>>
> > > >>> > >>> We had branched hbase-10070 in Feb out of trunk[5]. The
> branch
> > > >>> contains
> > > >>> > >>> 55
> > > >>> > >>> jiras committed[2]. Out of these 55, 15 has already been
> > > >>>committed to
> > > >>> > >>> trunk
> > > >>> > >>> and backported to hbase-10070 branch[3].
> > > >>> > >>>
> > > >>> > >>> This VOTE is for merging back the remaining changes in branch
> > to
> > > >>> trunk.
> > > >>> > >>> If
> > > >>> > >>> passes, we will rebase the branch on top of current trunk, in
> > > >>>which
> > > >>> we
> > > >>> > >>> will
> > > >>> > >>> keep the commit-per-issue log history. After that we will do
> a
> > > >>>git
> > > >>> > merge
> > > >>> > >>> for the branch keeping the history clean and not squashing
> the
> > > >>> > commits. I
> > > >>> > >>> expect rebasing to be straightforward, however with some
> manual
> > > >>> > conflict
> > > >>> > >>> resolution. After the merge we'll keep running the tests to
> > make
> > > >>>sure
> > > >>> > >>> everything is ok.
> > > >>> > >>>
> > > >>> > >>> An overview of the changes, and the status of the work can be
> > > >>>found
> > > >>> > under
> > > >>> > >>> [4], [6] and [7].In summary, with the code in branch, you can
> > > >>>create
> > > >>> > >>> tables
> > > >>> > >>> with region replicas, do gets / multi gets and scans using
> > > >>>TIMELINE
> > > >>> > >>> consistency with high availability. Region replicas
> > periodically
> > > >>>scan
> > > >>> > the
> > > >>> > >>> files of the primary and pick up flushed / committed files.
> The
> > > >>>RPC
> > > >>> > >>> paths /
> > > >>> > >>> assignment, balancing etc are pretty stable. However some
> more
> > > >>> > >>> performance
> > > >>> > >>> analysis and tuning is needed. Phase 2 work is being worked
> on
> > > >>>under
> > > >>> > >>> HBASE-11183, and we have some working prototype for
> > > >>>async-replicating
> > > >>> > and
> > > >>> > >>> region splits. However, we believe even without those
> features,
> > > >>>this
> > > >>> > work
> > > >>> > >>> is useable (especially for read-only/bulk load tables) , and
> > can
> > > >>>be
> > > >>> > >>> released as an experimental feature in 1.0.
> > > >>> > >>>
> > > >>> > >>> Please indicate your choice:
> > > >>> > >>>
> > > >>> > >>> [ ] +1 on yes, merge branch hbase-10070 to trunk.
> > > >>> > >>> [ ] 0 on don't care
> > > >>> > >>> [ ] -1 don't merge, because ...
> > > >>> > >>>
> > > >>> > >>> I'll keep the vote running for 7 days, and close it Mon 9th
> of
> > > >>>June,
> > > >>> > PDT.
> > > >>> > >>>
> > > >>> > >>> Here is my official +1.
> > > >>> > >>>
> > > >>> > >>> Thanks,
> > > >>> > >>> Enis
> > > >>> > >>>
> > > >>> > >>> [1]
> > > >>> > >>>
> > > >>> > >>>
> > > >>> >
> > > >>>
> > > >>>
> > >
> https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=log;h=refs/heads/h
> > > >>>base-10070
> > > >>> > >>> [2]
> > > >>> > >>>
> > > >>> > >>>
> > > >>> >
> > > >>>
> > > >>>
> > >
> https://issues.apache.org/jira/browse/HBASE-11214?jql=fixVersion%20%3D%2
> > >
> >
> >>>0hbase-10070%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolv
> > > >>>ed
> > > >>> > >>> [3]
> > > >>> > >>>
> > > >>> > >>>
> > > >>> >
> > > >>>
> > > >>>
> > >
> https://issues.apache.org/jira/browse/HBASE-10792?jql=fixVersion%20%3D%2
> > >
> >
> >>>0hbase-10070%20and%20fixversion%20%3D%200.99.0%20AND%20project%20%3D%20H
> > > >>>BASE%20AND%20status%20%3D%20resolved
> > > >>> > >>> [4]
> > > >>>https://www.mail-archive.com/dev@hbase.apache.org/msg25795.html
> > > >>> > >>> [5]
> > > >>> > >>>
> > > >>> > >>>
> > > >>> >
> > > >>>
> > > >>>
> > >
> https://github.com/apache/hbase/commit/e22c7efeac02efde3451a0c9ff9bdcd27
> > > >>>25576d0
> > > >>> > >>> [6]
> > > >>> > >>>
> > > >>> > >>>
> > > >>> >
> > > >>>
> > > >>>
> > >
> http://www.slideshare.net/enissoz/hbase-high-availability-for-reads-with
> > > >>>-time
> > > >>> > >>> [7] https://issues.apache.org/jira/browse/HBASE-10070
> > > >>> > >>>
> > > >>> > >>
> > > >>> > >>
> > > >>> > >>
> > > >>> > >> --
> > > >>> > >> Best regards,
> > > >>> > >>
> > > >>> > >>    - Andy
> > > >>> > >>
> > > >>> > >> Problems worthy of attack prove their worth by hitting back. -
> > > >>>Piet
> > > >>> Hein
> > > >>> > >> (via Tom White)
> > > >>> > >>
> > > >>> > >
> > > >>> > >
> > > >>> > >
> > > >>> > > --
> > > >>> > > Best regards,
> > > >>> > >
> > > >>> > >    - Andy
> > > >>> > >
> > > >>> > > Problems worthy of attack prove their worth by hitting back. -
> > Piet
> > > >>> Hein
> > > >>> > > (via Tom White)
> > > >>> > >
> > > >>> >
> > > >>> >
> > > >>> >
> > > >>> > --
> > > >>> > Best regards,
> > > >>> >
> > > >>> >    - Andy
> > > >>> >
> > > >>> > Problems worthy of attack prove their worth by hitting back. -
> Piet
> > > >>>Hein
> > > >>> > (via Tom White)
> > > >>> >
> > > >>>
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> Best regards,
> > > >>
> > > >>    - Andy
> > > >>
> > > >> Problems worthy of attack prove their worth by hitting back. - Piet
> > Hein
> > > >> (via Tom White)
> > > >
> > > >--
> > > >CONFIDENTIALITY NOTICE
> > > >NOTICE: This message is intended for the use of the individual or
> entity
> > > >to
> > > >which it is addressed and may contain information that is
> confidential,
> > > >privileged and exempt from disclosure under applicable law. If the
> > reader
> > > >of this message is not the intended recipient, you are hereby notified
> > > >that
> > > >any printing, copying, dissemination, distribution, disclosure or
> > > >forwarding of this communication is strictly prohibited. If you have
> > > >received this communication in error, please contact the sender
> > > >immediately
> > > >and delete it from your system. Thank You.
> > >
> > >
> > >
> > > --
> > > CONFIDENTIALITY NOTICE
> > > NOTICE: This message is intended for the use of the individual or
> entity
> > to
> > > which it is addressed and may contain information that is confidential,
> > > privileged and exempt from disclosure under applicable law. If the
> reader
> > > of this message is not the intended recipient, you are hereby notified
> > that
> > > any printing, copying, dissemination, distribution, disclosure or
> > > forwarding of this communication is strictly prohibited. If you have
> > > received this communication in error, please contact the sender
> > immediately
> > > and delete it from your system. Thank You.
> > >
> >
>

Re: [VOTE] Merge branch HBASE-10070 to trunk

Posted by Enis Söztutar <en...@gmail.com>.

BTW, I would really prefer to finish the merge before HBASE-11059. The
replica assignment has been working with the new patches and zk-based
assignment for many months.

Enis


On Fri, Jun 6, 2014 at 7:07 PM, Jimmy Xiang <jx...@cloudera.com> wrote:

> First of all, I have not looked into the patches recently. I remember there
> are some changes to the public interface. I was wondering if it is backward
> compatible. Enis mentioned that it's rolling upgradable. Just want to
> confirm if it is backward compatible. For existing applications, do they
> need to recompile?
>
> Second, there should be some conflict with HBASE-11059 I am working on now,
> which is almost finished. How should we resolve this issue? ZK-less region
> assignment doesn't know region replica yet.
>
> Thanks,
> Jimmy
>
>
>
>
> On Fri, Jun 6, 2014 at 6:12 PM, Jeffrey Zhong <jz...@hortonworks.com>
> wrote:
>
> >
> > +1. It brings goodies such as region replica, cross region server
> > scan&get, anti-affinity of regions, and pave the way for sync region
> > replication.
> >
> > Thanks,
> > -Jeffrey
> >
> > On 6/6/14 2:42 PM, "Devaraj Das" <dd...@hortonworks.com> wrote:
> >
> > >+1
> > >
> > >On Fri, Jun 6, 2014 at 2:13 PM, Andrew Purtell <ap...@apache.org>
> > >wrote:
> > >> +1, thanks Enis
> > >>
> > >>
> > >> On Fri, Jun 6, 2014 at 1:46 PM, Enis Söztutar <en...@gmail.com>
> > >>wrote:
> > >>
> > >>> Sorry, I was mostly out for HadoopSummit.
> > >>>
> > >>> Yes, the git flow would be very similar to what you propose:
> > >>>
> > >>>      $ git checkout HBASE-10070
> > >>>      $ git rebase --ignore-date master
> > >>>      (fixups, git add, git rebase --continue, etc, etc, etc)
> > >>>      $ git checkout master
> > >>>
> > >>>      $ git push origin HBASE-10070 HBASE-10070-rebase-date
> > >>>(optionally)
> > >>>      $ git merge HBASE-10070
> > >>>
> > >>> We can either go --ignore-date or not depending on what we want. If
> > >>>needed
> > >>> I am fine with pushing the rebased master branch for review to main
> > >>>repo
> > >>> before the merge to another branch. If not, I can just rebase the
> > >>>branch
> > >>> locally and merge + push to main repo.
> > >>>
> > >>> Creating final patches and attaching them to jira might be
> cumbersome.
> > >>>If
> > >>> we do the rebased-branch on repo, we might not need it. But if we
> need
> > >>>that
> > >>> for review, I can do it.
> > >>>
> > >>> Thanks,
> > >>> Enis
> > >>>
> > >>>
> > >>>
> > >>> On Wed, Jun 4, 2014 at 10:48 AM, Andrew Purtell <apurtell@apache.org
> >
> > >>> wrote:
> > >>>
> > >>> > I realize this is a vote thread but I need a satisfactory answer to
> > >>>the
> > >>> > below inquiries before feeling comfortable casting a vote. Or
> perhaps
> > >>> that
> > >>> > means we need to cancel this vote and move back to discussion.
> > >>> >
> > >>> >
> > >>> > On Tue, Jun 3, 2014 at 11:17 AM, Andrew Purtell <
> apurtell@apache.org
> > >
> > >>> > wrote:
> > >>> >
> > >>> > > Also after the merge process is completed, do you plan to use git
> > >>> > > format-patch to break out the per-JIRA changes into updated
> > >>>patches for
> > >>> > > those JIRAs representing in effect the final commit?
> > >>> > >
> > >>> > >
> > >>> > > On Tue, Jun 3, 2014 at 11:16 AM, Andrew Purtell
> > >>><ap...@apache.org>
> > >>> > > wrote:
> > >>> > >
> > >>> > >>
> > >>> > >> On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <en...@apache.org>
> > >>> wrote:
> > >>> > >>
> > >>> > >> This VOTE is for merging back the remaining changes in branch to
> > >>> trunk.
> > >>> > If
> > >>> > >>> passes, we will rebase the branch on top of current trunk, in
> > >>>which
> > >>> we
> > >>> > >>> will
> > >>> > >>> keep the commit-per-issue log history. After that we will do a
> > >>>git
> > >>> > merge
> > >>> > >>> for the branch keeping the history clean and not squashing the
> > >>> > commits. I
> > >>> > >>> expect rebasing to be straightforward, however with some manual
> > >>> > conflict
> > >>> > >>> resolution. After the merge we'll keep running the tests to
> make
> > >>>sure
> > >>> > >>> everything is ok.
> > >>> > >>>
> > >>> > >>
> > >>> > >> Just to clarify that would look something like this:
> > >>> > >>
> > >>> > >>      $ git checkout HBASE-10070
> > >>> > >>      $ git rebase --ignore-date master
> > >>> > >>      (fixups, git add, git rebase --continue, etc, etc, etc)
> > >>> > >>      $ git checkout master
> > >>> > >>      $ git merge HBASE-10070
> > >>> > >>
> > >>> > >> ?
> > >>> > >>
> > >>> > >> That sounds good to me, the final merge should be a fast forward
> > >>> merge.
> > >>> > >>
> > >>> > >> Use of ' --ignore-date' could be mildly controversial. It's not
> > >>> strictly
> > >>> > >> necessary because the commits for 10070 will appear grouped in
> > >>> history,
> > >>> > but
> > >>> > >> then dates on commits will be discontiguous in that section of
> > >>> history.
> > >>> > I
> > >>> > >> suggest using that option so the order of commits and dates sort
> > >>>the
> > >>> > same
> > >>> > >> on master.
> > >>> > >>
> > >>> > >>
> > >>> > >> On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <en...@apache.org>
> > >>> wrote:
> > >>> > >>
> > >>> > >>> Hi,
> > >>> > >>>
> > >>> > >>> Last week we started some discussion[4] for merging branch
> > >>> > hbase-10070[1]
> > >>> > >>> into trunk. It seems like the consensus there is to do the
> merge
> > >>> sooner
> > >>> > >>> rather than later.
> > >>> > >>>
> > >>> > >>>
> > >>> > >>> We had branched hbase-10070 in Feb out of trunk[5]. The branch
> > >>> contains
> > >>> > >>> 55
> > >>> > >>> jiras committed[2]. Out of these 55, 15 has already been
> > >>>committed to
> > >>> > >>> trunk
> > >>> > >>> and backported to hbase-10070 branch[3].
> > >>> > >>>
> > >>> > >>> This VOTE is for merging back the remaining changes in branch
> to
> > >>> trunk.
> > >>> > >>> If
> > >>> > >>> passes, we will rebase the branch on top of current trunk, in
> > >>>which
> > >>> we
> > >>> > >>> will
> > >>> > >>> keep the commit-per-issue log history. After that we will do a
> > >>>git
> > >>> > merge
> > >>> > >>> for the branch keeping the history clean and not squashing the
> > >>> > commits. I
> > >>> > >>> expect rebasing to be straightforward, however with some manual
> > >>> > conflict
> > >>> > >>> resolution. After the merge we'll keep running the tests to
> make
> > >>>sure
> > >>> > >>> everything is ok.
> > >>> > >>>
> > >>> > >>> An overview of the changes, and the status of the work can be
> > >>>found
> > >>> > under
> > >>> > >>> [4], [6] and [7].In summary, with the code in branch, you can
> > >>>create
> > >>> > >>> tables
> > >>> > >>> with region replicas, do gets / multi gets and scans using
> > >>>TIMELINE
> > >>> > >>> consistency with high availability. Region replicas
> periodically
> > >>>scan
> > >>> > the
> > >>> > >>> files of the primary and pick up flushed / committed files. The
> > >>>RPC
> > >>> > >>> paths /
> > >>> > >>> assignment, balancing etc are pretty stable. However some more
> > >>> > >>> performance
> > >>> > >>> analysis and tuning is needed. Phase 2 work is being worked on
> > >>>under
> > >>> > >>> HBASE-11183, and we have some working prototype for
> > >>>async-replicating
> > >>> > and
> > >>> > >>> region splits. However, we believe even without those features,
> > >>>this
> > >>> > work
> > >>> > >>> is useable (especially for read-only/bulk load tables) , and
> can
> > >>>be
> > >>> > >>> released as an experimental feature in 1.0.
> > >>> > >>>
> > >>> > >>> Please indicate your choice:
> > >>> > >>>
> > >>> > >>> [ ] +1 on yes, merge branch hbase-10070 to trunk.
> > >>> > >>> [ ] 0 on don't care
> > >>> > >>> [ ] -1 don't merge, because ...
> > >>> > >>>
> > >>> > >>> I'll keep the vote running for 7 days, and close it Mon 9th of
> > >>>June,
> > >>> > PDT.
> > >>> > >>>
> > >>> > >>> Here is my official +1.
> > >>> > >>>
> > >>> > >>> Thanks,
> > >>> > >>> Enis
> > >>> > >>>
> > >>> > >>> [1]
> > >>> > >>>
> > >>> > >>>
> > >>> >
> > >>>
> > >>>
> > https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=log;h=refs/heads/h
> > >>>base-10070
> > >>> > >>> [2]
> > >>> > >>>
> > >>> > >>>
> > >>> >
> > >>>
> > >>>
> > https://issues.apache.org/jira/browse/HBASE-11214?jql=fixVersion%20%3D%2
> >
> >>>0hbase-10070%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolv
> > >>>ed
> > >>> > >>> [3]
> > >>> > >>>
> > >>> > >>>
> > >>> >
> > >>>
> > >>>
> > https://issues.apache.org/jira/browse/HBASE-10792?jql=fixVersion%20%3D%2
> >
> >>>0hbase-10070%20and%20fixversion%20%3D%200.99.0%20AND%20project%20%3D%20H
> > >>>BASE%20AND%20status%20%3D%20resolved
> > >>> > >>> [4]
> > >>>https://www.mail-archive.com/dev@hbase.apache.org/msg25795.html
> > >>> > >>> [5]
> > >>> > >>>
> > >>> > >>>
> > >>> >
> > >>>
> > >>>
> > https://github.com/apache/hbase/commit/e22c7efeac02efde3451a0c9ff9bdcd27
> > >>>25576d0
> > >>> > >>> [6]
> > >>> > >>>
> > >>> > >>>
> > >>> >
> > >>>
> > >>>
> > http://www.slideshare.net/enissoz/hbase-high-availability-for-reads-with
> > >>>-time
> > >>> > >>> [7] https://issues.apache.org/jira/browse/HBASE-10070
> > >>> > >>>
> > >>> > >>
> > >>> > >>
> > >>> > >>
> > >>> > >> --
> > >>> > >> Best regards,
> > >>> > >>
> > >>> > >>    - Andy
> > >>> > >>
> > >>> > >> Problems worthy of attack prove their worth by hitting back. -
> > >>>Piet
> > >>> Hein
> > >>> > >> (via Tom White)
> > >>> > >>
> > >>> > >
> > >>> > >
> > >>> > >
> > >>> > > --
> > >>> > > Best regards,
> > >>> > >
> > >>> > >    - Andy
> > >>> > >
> > >>> > > Problems worthy of attack prove their worth by hitting back. -
> Piet
> > >>> Hein
> > >>> > > (via Tom White)
> > >>> > >
> > >>> >
> > >>> >
> > >>> >
> > >>> > --
> > >>> > Best regards,
> > >>> >
> > >>> >    - Andy
> > >>> >
> > >>> > Problems worthy of attack prove their worth by hitting back. - Piet
> > >>>Hein
> > >>> > (via Tom White)
> > >>> >
> > >>>
> > >>
> > >>
> > >>
> > >> --
> > >> Best regards,
> > >>
> > >>    - Andy
> > >>
> > >> Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > >> (via Tom White)
> > >
> > >--
> > >CONFIDENTIALITY NOTICE
> > >NOTICE: This message is intended for the use of the individual or entity
> > >to
> > >which it is addressed and may contain information that is confidential,
> > >privileged and exempt from disclosure under applicable law. If the
> reader
> > >of this message is not the intended recipient, you are hereby notified
> > >that
> > >any printing, copying, dissemination, distribution, disclosure or
> > >forwarding of this communication is strictly prohibited. If you have
> > >received this communication in error, please contact the sender
> > >immediately
> > >and delete it from your system. Thank You.
> >
> >
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
> >
>

Re: [VOTE] Merge branch HBASE-10070 to trunk

Posted by Jimmy Xiang <jx...@cloudera.com>.

First of all, I have not looked into the patches recently. I remember there
are some changes to the public interface. I was wondering if it is backward
compatible. Enis mentioned that it's rolling upgradable. Just want to
confirm if it is backward compatible. For existing applications, do they
need to recompile?

Second, there should be some conflict with HBASE-11059 I am working on now,
which is almost finished. How should we resolve this issue? ZK-less region
assignment doesn't know region replica yet.

Thanks,
Jimmy




On Fri, Jun 6, 2014 at 6:12 PM, Jeffrey Zhong <jz...@hortonworks.com>
wrote:

>
> +1. It brings goodies such as region replica, cross region server
> scan&get, anti-affinity of regions, and pave the way for sync region
> replication.
>
> Thanks,
> -Jeffrey
>
> On 6/6/14 2:42 PM, "Devaraj Das" <dd...@hortonworks.com> wrote:
>
> >+1
> >
> >On Fri, Jun 6, 2014 at 2:13 PM, Andrew Purtell <ap...@apache.org>
> >wrote:
> >> +1, thanks Enis
> >>
> >>
> >> On Fri, Jun 6, 2014 at 1:46 PM, Enis Söztutar <en...@gmail.com>
> >>wrote:
> >>
> >>> Sorry, I was mostly out for HadoopSummit.
> >>>
> >>> Yes, the git flow would be very similar to what you propose:
> >>>
> >>>      $ git checkout HBASE-10070
> >>>      $ git rebase --ignore-date master
> >>>      (fixups, git add, git rebase --continue, etc, etc, etc)
> >>>      $ git checkout master
> >>>
> >>>      $ git push origin HBASE-10070 HBASE-10070-rebase-date
> >>>(optionally)
> >>>      $ git merge HBASE-10070
> >>>
> >>> We can either go --ignore-date or not depending on what we want. If
> >>>needed
> >>> I am fine with pushing the rebased master branch for review to main
> >>>repo
> >>> before the merge to another branch. If not, I can just rebase the
> >>>branch
> >>> locally and merge + push to main repo.
> >>>
> >>> Creating final patches and attaching them to jira might be cumbersome.
> >>>If
> >>> we do the rebased-branch on repo, we might not need it. But if we need
> >>>that
> >>> for review, I can do it.
> >>>
> >>> Thanks,
> >>> Enis
> >>>
> >>>
> >>>
> >>> On Wed, Jun 4, 2014 at 10:48 AM, Andrew Purtell <ap...@apache.org>
> >>> wrote:
> >>>
> >>> > I realize this is a vote thread but I need a satisfactory answer to
> >>>the
> >>> > below inquiries before feeling comfortable casting a vote. Or perhaps
> >>> that
> >>> > means we need to cancel this vote and move back to discussion.
> >>> >
> >>> >
> >>> > On Tue, Jun 3, 2014 at 11:17 AM, Andrew Purtell <apurtell@apache.org
> >
> >>> > wrote:
> >>> >
> >>> > > Also after the merge process is completed, do you plan to use git
> >>> > > format-patch to break out the per-JIRA changes into updated
> >>>patches for
> >>> > > those JIRAs representing in effect the final commit?
> >>> > >
> >>> > >
> >>> > > On Tue, Jun 3, 2014 at 11:16 AM, Andrew Purtell
> >>><ap...@apache.org>
> >>> > > wrote:
> >>> > >
> >>> > >>
> >>> > >> On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <en...@apache.org>
> >>> wrote:
> >>> > >>
> >>> > >> This VOTE is for merging back the remaining changes in branch to
> >>> trunk.
> >>> > If
> >>> > >>> passes, we will rebase the branch on top of current trunk, in
> >>>which
> >>> we
> >>> > >>> will
> >>> > >>> keep the commit-per-issue log history. After that we will do a
> >>>git
> >>> > merge
> >>> > >>> for the branch keeping the history clean and not squashing the
> >>> > commits. I
> >>> > >>> expect rebasing to be straightforward, however with some manual
> >>> > conflict
> >>> > >>> resolution. After the merge we'll keep running the tests to make
> >>>sure
> >>> > >>> everything is ok.
> >>> > >>>
> >>> > >>
> >>> > >> Just to clarify that would look something like this:
> >>> > >>
> >>> > >>      $ git checkout HBASE-10070
> >>> > >>      $ git rebase --ignore-date master
> >>> > >>      (fixups, git add, git rebase --continue, etc, etc, etc)
> >>> > >>      $ git checkout master
> >>> > >>      $ git merge HBASE-10070
> >>> > >>
> >>> > >> ?
> >>> > >>
> >>> > >> That sounds good to me, the final merge should be a fast forward
> >>> merge.
> >>> > >>
> >>> > >> Use of ' --ignore-date' could be mildly controversial. It's not
> >>> strictly
> >>> > >> necessary because the commits for 10070 will appear grouped in
> >>> history,
> >>> > but
> >>> > >> then dates on commits will be discontiguous in that section of
> >>> history.
> >>> > I
> >>> > >> suggest using that option so the order of commits and dates sort
> >>>the
> >>> > same
> >>> > >> on master.
> >>> > >>
> >>> > >>
> >>> > >> On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <en...@apache.org>
> >>> wrote:
> >>> > >>
> >>> > >>> Hi,
> >>> > >>>
> >>> > >>> Last week we started some discussion[4] for merging branch
> >>> > hbase-10070[1]
> >>> > >>> into trunk. It seems like the consensus there is to do the merge
> >>> sooner
> >>> > >>> rather than later.
> >>> > >>>
> >>> > >>>
> >>> > >>> We had branched hbase-10070 in Feb out of trunk[5]. The branch
> >>> contains
> >>> > >>> 55
> >>> > >>> jiras committed[2]. Out of these 55, 15 has already been
> >>>committed to
> >>> > >>> trunk
> >>> > >>> and backported to hbase-10070 branch[3].
> >>> > >>>
> >>> > >>> This VOTE is for merging back the remaining changes in branch to
> >>> trunk.
> >>> > >>> If
> >>> > >>> passes, we will rebase the branch on top of current trunk, in
> >>>which
> >>> we
> >>> > >>> will
> >>> > >>> keep the commit-per-issue log history. After that we will do a
> >>>git
> >>> > merge
> >>> > >>> for the branch keeping the history clean and not squashing the
> >>> > commits. I
> >>> > >>> expect rebasing to be straightforward, however with some manual
> >>> > conflict
> >>> > >>> resolution. After the merge we'll keep running the tests to make
> >>>sure
> >>> > >>> everything is ok.
> >>> > >>>
> >>> > >>> An overview of the changes, and the status of the work can be
> >>>found
> >>> > under
> >>> > >>> [4], [6] and [7].In summary, with the code in branch, you can
> >>>create
> >>> > >>> tables
> >>> > >>> with region replicas, do gets / multi gets and scans using
> >>>TIMELINE
> >>> > >>> consistency with high availability. Region replicas periodically
> >>>scan
> >>> > the
> >>> > >>> files of the primary and pick up flushed / committed files. The
> >>>RPC
> >>> > >>> paths /
> >>> > >>> assignment, balancing etc are pretty stable. However some more
> >>> > >>> performance
> >>> > >>> analysis and tuning is needed. Phase 2 work is being worked on
> >>>under
> >>> > >>> HBASE-11183, and we have some working prototype for
> >>>async-replicating
> >>> > and
> >>> > >>> region splits. However, we believe even without those features,
> >>>this
> >>> > work
> >>> > >>> is useable (especially for read-only/bulk load tables) , and can
> >>>be
> >>> > >>> released as an experimental feature in 1.0.
> >>> > >>>
> >>> > >>> Please indicate your choice:
> >>> > >>>
> >>> > >>> [ ] +1 on yes, merge branch hbase-10070 to trunk.
> >>> > >>> [ ] 0 on don't care
> >>> > >>> [ ] -1 don't merge, because ...
> >>> > >>>
> >>> > >>> I'll keep the vote running for 7 days, and close it Mon 9th of
> >>>June,
> >>> > PDT.
> >>> > >>>
> >>> > >>> Here is my official +1.
> >>> > >>>
> >>> > >>> Thanks,
> >>> > >>> Enis
> >>> > >>>
> >>> > >>> [1]
> >>> > >>>
> >>> > >>>
> >>> >
> >>>
> >>>
> https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=log;h=refs/heads/h
> >>>base-10070
> >>> > >>> [2]
> >>> > >>>
> >>> > >>>
> >>> >
> >>>
> >>>
> https://issues.apache.org/jira/browse/HBASE-11214?jql=fixVersion%20%3D%2
> >>>0hbase-10070%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolv
> >>>ed
> >>> > >>> [3]
> >>> > >>>
> >>> > >>>
> >>> >
> >>>
> >>>
> https://issues.apache.org/jira/browse/HBASE-10792?jql=fixVersion%20%3D%2
> >>>0hbase-10070%20and%20fixversion%20%3D%200.99.0%20AND%20project%20%3D%20H
> >>>BASE%20AND%20status%20%3D%20resolved
> >>> > >>> [4]
> >>>https://www.mail-archive.com/dev@hbase.apache.org/msg25795.html
> >>> > >>> [5]
> >>> > >>>
> >>> > >>>
> >>> >
> >>>
> >>>
> https://github.com/apache/hbase/commit/e22c7efeac02efde3451a0c9ff9bdcd27
> >>>25576d0
> >>> > >>> [6]
> >>> > >>>
> >>> > >>>
> >>> >
> >>>
> >>>
> http://www.slideshare.net/enissoz/hbase-high-availability-for-reads-with
> >>>-time
> >>> > >>> [7] https://issues.apache.org/jira/browse/HBASE-10070
> >>> > >>>
> >>> > >>
> >>> > >>
> >>> > >>
> >>> > >> --
> >>> > >> Best regards,
> >>> > >>
> >>> > >>    - Andy
> >>> > >>
> >>> > >> Problems worthy of attack prove their worth by hitting back. -
> >>>Piet
> >>> Hein
> >>> > >> (via Tom White)
> >>> > >>
> >>> > >
> >>> > >
> >>> > >
> >>> > > --
> >>> > > Best regards,
> >>> > >
> >>> > >    - Andy
> >>> > >
> >>> > > Problems worthy of attack prove their worth by hitting back. - Piet
> >>> Hein
> >>> > > (via Tom White)
> >>> > >
> >>> >
> >>> >
> >>> >
> >>> > --
> >>> > Best regards,
> >>> >
> >>> >    - Andy
> >>> >
> >>> > Problems worthy of attack prove their worth by hitting back. - Piet
> >>>Hein
> >>> > (via Tom White)
> >>> >
> >>>
> >>
> >>
> >>
> >> --
> >> Best regards,
> >>
> >>    - Andy
> >>
> >> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> >> (via Tom White)
> >
> >--
> >CONFIDENTIALITY NOTICE
> >NOTICE: This message is intended for the use of the individual or entity
> >to
> >which it is addressed and may contain information that is confidential,
> >privileged and exempt from disclosure under applicable law. If the reader
> >of this message is not the intended recipient, you are hereby notified
> >that
> >any printing, copying, dissemination, distribution, disclosure or
> >forwarding of this communication is strictly prohibited. If you have
> >received this communication in error, please contact the sender
> >immediately
> >and delete it from your system. Thank You.
>
>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

Re: [VOTE] Merge branch HBASE-10070 to trunk

Posted by Jeffrey Zhong <jz...@hortonworks.com>.

+1. It brings goodies such as region replica, cross region server
scan&get, anti-affinity of regions, and pave the way for sync region
replication. 

Thanks,
-Jeffrey 

On 6/6/14 2:42 PM, "Devaraj Das" <dd...@hortonworks.com> wrote:

>+1
>
>On Fri, Jun 6, 2014 at 2:13 PM, Andrew Purtell <ap...@apache.org>
>wrote:
>> +1, thanks Enis
>>
>>
>> On Fri, Jun 6, 2014 at 1:46 PM, Enis Söztutar <en...@gmail.com>
>>wrote:
>>
>>> Sorry, I was mostly out for HadoopSummit.
>>>
>>> Yes, the git flow would be very similar to what you propose:
>>>
>>>      $ git checkout HBASE-10070
>>>      $ git rebase --ignore-date master
>>>      (fixups, git add, git rebase --continue, etc, etc, etc)
>>>      $ git checkout master
>>>
>>>      $ git push origin HBASE-10070 HBASE-10070-rebase-date
>>>(optionally)
>>>      $ git merge HBASE-10070
>>>
>>> We can either go --ignore-date or not depending on what we want. If
>>>needed
>>> I am fine with pushing the rebased master branch for review to main
>>>repo
>>> before the merge to another branch. If not, I can just rebase the
>>>branch
>>> locally and merge + push to main repo.
>>>
>>> Creating final patches and attaching them to jira might be cumbersome.
>>>If
>>> we do the rebased-branch on repo, we might not need it. But if we need
>>>that
>>> for review, I can do it.
>>>
>>> Thanks,
>>> Enis
>>>
>>>
>>>
>>> On Wed, Jun 4, 2014 at 10:48 AM, Andrew Purtell <ap...@apache.org>
>>> wrote:
>>>
>>> > I realize this is a vote thread but I need a satisfactory answer to
>>>the
>>> > below inquiries before feeling comfortable casting a vote. Or perhaps
>>> that
>>> > means we need to cancel this vote and move back to discussion.
>>> >
>>> >
>>> > On Tue, Jun 3, 2014 at 11:17 AM, Andrew Purtell <ap...@apache.org>
>>> > wrote:
>>> >
>>> > > Also after the merge process is completed, do you plan to use git
>>> > > format-patch to break out the per-JIRA changes into updated
>>>patches for
>>> > > those JIRAs representing in effect the final commit?
>>> > >
>>> > >
>>> > > On Tue, Jun 3, 2014 at 11:16 AM, Andrew Purtell
>>><ap...@apache.org>
>>> > > wrote:
>>> > >
>>> > >>
>>> > >> On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <en...@apache.org>
>>> wrote:
>>> > >>
>>> > >> This VOTE is for merging back the remaining changes in branch to
>>> trunk.
>>> > If
>>> > >>> passes, we will rebase the branch on top of current trunk, in
>>>which
>>> we
>>> > >>> will
>>> > >>> keep the commit-per-issue log history. After that we will do a
>>>git
>>> > merge
>>> > >>> for the branch keeping the history clean and not squashing the
>>> > commits. I
>>> > >>> expect rebasing to be straightforward, however with some manual
>>> > conflict
>>> > >>> resolution. After the merge we'll keep running the tests to make
>>>sure
>>> > >>> everything is ok.
>>> > >>>
>>> > >>
>>> > >> Just to clarify that would look something like this:
>>> > >>
>>> > >>      $ git checkout HBASE-10070
>>> > >>      $ git rebase --ignore-date master
>>> > >>      (fixups, git add, git rebase --continue, etc, etc, etc)
>>> > >>      $ git checkout master
>>> > >>      $ git merge HBASE-10070
>>> > >>
>>> > >> ?
>>> > >>
>>> > >> That sounds good to me, the final merge should be a fast forward
>>> merge.
>>> > >>
>>> > >> Use of ' --ignore-date' could be mildly controversial. It's not
>>> strictly
>>> > >> necessary because the commits for 10070 will appear grouped in
>>> history,
>>> > but
>>> > >> then dates on commits will be discontiguous in that section of
>>> history.
>>> > I
>>> > >> suggest using that option so the order of commits and dates sort
>>>the
>>> > same
>>> > >> on master.
>>> > >>
>>> > >>
>>> > >> On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <en...@apache.org>
>>> wrote:
>>> > >>
>>> > >>> Hi,
>>> > >>>
>>> > >>> Last week we started some discussion[4] for merging branch
>>> > hbase-10070[1]
>>> > >>> into trunk. It seems like the consensus there is to do the merge
>>> sooner
>>> > >>> rather than later.
>>> > >>>
>>> > >>>
>>> > >>> We had branched hbase-10070 in Feb out of trunk[5]. The branch
>>> contains
>>> > >>> 55
>>> > >>> jiras committed[2]. Out of these 55, 15 has already been
>>>committed to
>>> > >>> trunk
>>> > >>> and backported to hbase-10070 branch[3].
>>> > >>>
>>> > >>> This VOTE is for merging back the remaining changes in branch to
>>> trunk.
>>> > >>> If
>>> > >>> passes, we will rebase the branch on top of current trunk, in
>>>which
>>> we
>>> > >>> will
>>> > >>> keep the commit-per-issue log history. After that we will do a
>>>git
>>> > merge
>>> > >>> for the branch keeping the history clean and not squashing the
>>> > commits. I
>>> > >>> expect rebasing to be straightforward, however with some manual
>>> > conflict
>>> > >>> resolution. After the merge we'll keep running the tests to make
>>>sure
>>> > >>> everything is ok.
>>> > >>>
>>> > >>> An overview of the changes, and the status of the work can be
>>>found
>>> > under
>>> > >>> [4], [6] and [7].In summary, with the code in branch, you can
>>>create
>>> > >>> tables
>>> > >>> with region replicas, do gets / multi gets and scans using
>>>TIMELINE
>>> > >>> consistency with high availability. Region replicas periodically
>>>scan
>>> > the
>>> > >>> files of the primary and pick up flushed / committed files. The
>>>RPC
>>> > >>> paths /
>>> > >>> assignment, balancing etc are pretty stable. However some more
>>> > >>> performance
>>> > >>> analysis and tuning is needed. Phase 2 work is being worked on
>>>under
>>> > >>> HBASE-11183, and we have some working prototype for
>>>async-replicating
>>> > and
>>> > >>> region splits. However, we believe even without those features,
>>>this
>>> > work
>>> > >>> is useable (especially for read-only/bulk load tables) , and can
>>>be
>>> > >>> released as an experimental feature in 1.0.
>>> > >>>
>>> > >>> Please indicate your choice:
>>> > >>>
>>> > >>> [ ] +1 on yes, merge branch hbase-10070 to trunk.
>>> > >>> [ ] 0 on don't care
>>> > >>> [ ] -1 don't merge, because ...
>>> > >>>
>>> > >>> I'll keep the vote running for 7 days, and close it Mon 9th of
>>>June,
>>> > PDT.
>>> > >>>
>>> > >>> Here is my official +1.
>>> > >>>
>>> > >>> Thanks,
>>> > >>> Enis
>>> > >>>
>>> > >>> [1]
>>> > >>>
>>> > >>>
>>> >
>>> 
>>>https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=log;h=refs/heads/h
>>>base-10070
>>> > >>> [2]
>>> > >>>
>>> > >>>
>>> >
>>> 
>>>https://issues.apache.org/jira/browse/HBASE-11214?jql=fixVersion%20%3D%2
>>>0hbase-10070%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolv
>>>ed
>>> > >>> [3]
>>> > >>>
>>> > >>>
>>> >
>>> 
>>>https://issues.apache.org/jira/browse/HBASE-10792?jql=fixVersion%20%3D%2
>>>0hbase-10070%20and%20fixversion%20%3D%200.99.0%20AND%20project%20%3D%20H
>>>BASE%20AND%20status%20%3D%20resolved
>>> > >>> [4] 
>>>https://www.mail-archive.com/dev@hbase.apache.org/msg25795.html
>>> > >>> [5]
>>> > >>>
>>> > >>>
>>> >
>>> 
>>>https://github.com/apache/hbase/commit/e22c7efeac02efde3451a0c9ff9bdcd27
>>>25576d0
>>> > >>> [6]
>>> > >>>
>>> > >>>
>>> >
>>> 
>>>http://www.slideshare.net/enissoz/hbase-high-availability-for-reads-with
>>>-time
>>> > >>> [7] https://issues.apache.org/jira/browse/HBASE-10070
>>> > >>>
>>> > >>
>>> > >>
>>> > >>
>>> > >> --
>>> > >> Best regards,
>>> > >>
>>> > >>    - Andy
>>> > >>
>>> > >> Problems worthy of attack prove their worth by hitting back. -
>>>Piet
>>> Hein
>>> > >> (via Tom White)
>>> > >>
>>> > >
>>> > >
>>> > >
>>> > > --
>>> > > Best regards,
>>> > >
>>> > >    - Andy
>>> > >
>>> > > Problems worthy of attack prove their worth by hitting back. - Piet
>>> Hein
>>> > > (via Tom White)
>>> > >
>>> >
>>> >
>>> >
>>> > --
>>> > Best regards,
>>> >
>>> >    - Andy
>>> >
>>> > Problems worthy of attack prove their worth by hitting back. - Piet
>>>Hein
>>> > (via Tom White)
>>> >
>>>
>>
>>
>>
>> --
>> Best regards,
>>
>>    - Andy
>>
>> Problems worthy of attack prove their worth by hitting back. - Piet Hein
>> (via Tom White)
>
>-- 
>CONFIDENTIALITY NOTICE
>NOTICE: This message is intended for the use of the individual or entity
>to 
>which it is addressed and may contain information that is confidential,
>privileged and exempt from disclosure under applicable law. If the reader
>of this message is not the intended recipient, you are hereby notified
>that 
>any printing, copying, dissemination, distribution, disclosure or
>forwarding of this communication is strictly prohibited. If you have
>received this communication in error, please contact the sender
>immediately 
>and delete it from your system. Thank You.



-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: [VOTE] Merge branch HBASE-10070 to trunk

Posted by Devaraj Das <dd...@hortonworks.com>.

+1

On Fri, Jun 6, 2014 at 2:13 PM, Andrew Purtell <ap...@apache.org> wrote:
> +1, thanks Enis
>
>
> On Fri, Jun 6, 2014 at 1:46 PM, Enis Söztutar <en...@gmail.com> wrote:
>
>> Sorry, I was mostly out for HadoopSummit.
>>
>> Yes, the git flow would be very similar to what you propose:
>>
>>      $ git checkout HBASE-10070
>>      $ git rebase --ignore-date master
>>      (fixups, git add, git rebase --continue, etc, etc, etc)
>>      $ git checkout master
>>
>>      $ git push origin HBASE-10070 HBASE-10070-rebase-date   (optionally)
>>      $ git merge HBASE-10070
>>
>> We can either go --ignore-date or not depending on what we want. If needed
>> I am fine with pushing the rebased master branch for review to main repo
>> before the merge to another branch. If not, I can just rebase the branch
>> locally and merge + push to main repo.
>>
>> Creating final patches and attaching them to jira might be cumbersome. If
>> we do the rebased-branch on repo, we might not need it. But if we need that
>> for review, I can do it.
>>
>> Thanks,
>> Enis
>>
>>
>>
>> On Wed, Jun 4, 2014 at 10:48 AM, Andrew Purtell <ap...@apache.org>
>> wrote:
>>
>> > I realize this is a vote thread but I need a satisfactory answer to the
>> > below inquiries before feeling comfortable casting a vote. Or perhaps
>> that
>> > means we need to cancel this vote and move back to discussion.
>> >
>> >
>> > On Tue, Jun 3, 2014 at 11:17 AM, Andrew Purtell <ap...@apache.org>
>> > wrote:
>> >
>> > > Also after the merge process is completed, do you plan to use git
>> > > format-patch to break out the per-JIRA changes into updated patches for
>> > > those JIRAs representing in effect the final commit?
>> > >
>> > >
>> > > On Tue, Jun 3, 2014 at 11:16 AM, Andrew Purtell <ap...@apache.org>
>> > > wrote:
>> > >
>> > >>
>> > >> On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <en...@apache.org>
>> wrote:
>> > >>
>> > >> This VOTE is for merging back the remaining changes in branch to
>> trunk.
>> > If
>> > >>> passes, we will rebase the branch on top of current trunk, in which
>> we
>> > >>> will
>> > >>> keep the commit-per-issue log history. After that we will do a git
>> > merge
>> > >>> for the branch keeping the history clean and not squashing the
>> > commits. I
>> > >>> expect rebasing to be straightforward, however with some manual
>> > conflict
>> > >>> resolution. After the merge we'll keep running the tests to make sure
>> > >>> everything is ok.
>> > >>>
>> > >>
>> > >> Just to clarify that would look something like this:
>> > >>
>> > >>      $ git checkout HBASE-10070
>> > >>      $ git rebase --ignore-date master
>> > >>      (fixups, git add, git rebase --continue, etc, etc, etc)
>> > >>      $ git checkout master
>> > >>      $ git merge HBASE-10070
>> > >>
>> > >> ?
>> > >>
>> > >> That sounds good to me, the final merge should be a fast forward
>> merge.
>> > >>
>> > >> Use of ' --ignore-date' could be mildly controversial. It's not
>> strictly
>> > >> necessary because the commits for 10070 will appear grouped in
>> history,
>> > but
>> > >> then dates on commits will be discontiguous in that section of
>> history.
>> > I
>> > >> suggest using that option so the order of commits and dates sort the
>> > same
>> > >> on master.
>> > >>
>> > >>
>> > >> On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <en...@apache.org>
>> wrote:
>> > >>
>> > >>> Hi,
>> > >>>
>> > >>> Last week we started some discussion[4] for merging branch
>> > hbase-10070[1]
>> > >>> into trunk. It seems like the consensus there is to do the merge
>> sooner
>> > >>> rather than later.
>> > >>>
>> > >>>
>> > >>> We had branched hbase-10070 in Feb out of trunk[5]. The branch
>> contains
>> > >>> 55
>> > >>> jiras committed[2]. Out of these 55, 15 has already been committed to
>> > >>> trunk
>> > >>> and backported to hbase-10070 branch[3].
>> > >>>
>> > >>> This VOTE is for merging back the remaining changes in branch to
>> trunk.
>> > >>> If
>> > >>> passes, we will rebase the branch on top of current trunk, in which
>> we
>> > >>> will
>> > >>> keep the commit-per-issue log history. After that we will do a git
>> > merge
>> > >>> for the branch keeping the history clean and not squashing the
>> > commits. I
>> > >>> expect rebasing to be straightforward, however with some manual
>> > conflict
>> > >>> resolution. After the merge we'll keep running the tests to make sure
>> > >>> everything is ok.
>> > >>>
>> > >>> An overview of the changes, and the status of the work can be found
>> > under
>> > >>> [4], [6] and [7].In summary, with the code in branch, you can create
>> > >>> tables
>> > >>> with region replicas, do gets / multi gets and scans using TIMELINE
>> > >>> consistency with high availability. Region replicas periodically scan
>> > the
>> > >>> files of the primary and pick up flushed / committed files. The RPC
>> > >>> paths /
>> > >>> assignment, balancing etc are pretty stable. However some more
>> > >>> performance
>> > >>> analysis and tuning is needed. Phase 2 work is being worked on under
>> > >>> HBASE-11183, and we have some working prototype for async-replicating
>> > and
>> > >>> region splits. However, we believe even without those features, this
>> > work
>> > >>> is useable (especially for read-only/bulk load tables) , and can be
>> > >>> released as an experimental feature in 1.0.
>> > >>>
>> > >>> Please indicate your choice:
>> > >>>
>> > >>> [ ] +1 on yes, merge branch hbase-10070 to trunk.
>> > >>> [ ] 0 on don't care
>> > >>> [ ] -1 don't merge, because ...
>> > >>>
>> > >>> I'll keep the vote running for 7 days, and close it Mon 9th of June,
>> > PDT.
>> > >>>
>> > >>> Here is my official +1.
>> > >>>
>> > >>> Thanks,
>> > >>> Enis
>> > >>>
>> > >>> [1]
>> > >>>
>> > >>>
>> >
>> https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=log;h=refs/heads/hbase-10070
>> > >>> [2]
>> > >>>
>> > >>>
>> >
>> https://issues.apache.org/jira/browse/HBASE-11214?jql=fixVersion%20%3D%20hbase-10070%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolved
>> > >>> [3]
>> > >>>
>> > >>>
>> >
>> https://issues.apache.org/jira/browse/HBASE-10792?jql=fixVersion%20%3D%20hbase-10070%20and%20fixversion%20%3D%200.99.0%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolved
>> > >>> [4] https://www.mail-archive.com/dev@hbase.apache.org/msg25795.html
>> > >>> [5]
>> > >>>
>> > >>>
>> >
>> https://github.com/apache/hbase/commit/e22c7efeac02efde3451a0c9ff9bdcd2725576d0
>> > >>> [6]
>> > >>>
>> > >>>
>> >
>> http://www.slideshare.net/enissoz/hbase-high-availability-for-reads-with-time
>> > >>> [7] https://issues.apache.org/jira/browse/HBASE-10070
>> > >>>
>> > >>
>> > >>
>> > >>
>> > >> --
>> > >> Best regards,
>> > >>
>> > >>    - Andy
>> > >>
>> > >> Problems worthy of attack prove their worth by hitting back. - Piet
>> Hein
>> > >> (via Tom White)
>> > >>
>> > >
>> > >
>> > >
>> > > --
>> > > Best regards,
>> > >
>> > >    - Andy
>> > >
>> > > Problems worthy of attack prove their worth by hitting back. - Piet
>> Hein
>> > > (via Tom White)
>> > >
>> >
>> >
>> >
>> > --
>> > Best regards,
>> >
>> >    - Andy
>> >
>> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
>> > (via Tom White)
>> >
>>
>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: [VOTE] Merge branch HBASE-10070 to trunk

Posted by Andrew Purtell <ap...@apache.org>.

+1, thanks Enis


On Fri, Jun 6, 2014 at 1:46 PM, Enis Söztutar <en...@gmail.com> wrote:

> Sorry, I was mostly out for HadoopSummit.
>
> Yes, the git flow would be very similar to what you propose:
>
>      $ git checkout HBASE-10070
>      $ git rebase --ignore-date master
>      (fixups, git add, git rebase --continue, etc, etc, etc)
>      $ git checkout master
>
>      $ git push origin HBASE-10070 HBASE-10070-rebase-date   (optionally)
>      $ git merge HBASE-10070
>
> We can either go --ignore-date or not depending on what we want. If needed
> I am fine with pushing the rebased master branch for review to main repo
> before the merge to another branch. If not, I can just rebase the branch
> locally and merge + push to main repo.
>
> Creating final patches and attaching them to jira might be cumbersome. If
> we do the rebased-branch on repo, we might not need it. But if we need that
> for review, I can do it.
>
> Thanks,
> Enis
>
>
>
> On Wed, Jun 4, 2014 at 10:48 AM, Andrew Purtell <ap...@apache.org>
> wrote:
>
> > I realize this is a vote thread but I need a satisfactory answer to the
> > below inquiries before feeling comfortable casting a vote. Or perhaps
> that
> > means we need to cancel this vote and move back to discussion.
> >
> >
> > On Tue, Jun 3, 2014 at 11:17 AM, Andrew Purtell <ap...@apache.org>
> > wrote:
> >
> > > Also after the merge process is completed, do you plan to use git
> > > format-patch to break out the per-JIRA changes into updated patches for
> > > those JIRAs representing in effect the final commit?
> > >
> > >
> > > On Tue, Jun 3, 2014 at 11:16 AM, Andrew Purtell <ap...@apache.org>
> > > wrote:
> > >
> > >>  
> > >> On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <en...@apache.org>
> wrote:
> > >>
> > >> This VOTE is for merging back the remaining changes in branch to
> trunk.
> > If
> > >>> passes, we will rebase the branch on top of current trunk, in which
> we
> > >>> will
> > >>> keep the commit-per-issue log history. After that we will do a git
> > merge
> > >>> for the branch keeping the history clean and not squashing the
> > commits. I
> > >>> expect rebasing to be straightforward, however with some manual
> > conflict
> > >>> resolution. After the merge we'll keep running the tests to make sure
> > >>> everything is ok.
> > >>>
> > >>
> > >> Just to clarify that would look something like this:
> > >>
> > >>      $ git checkout HBASE-10070
> > >>      $ git rebase --ignore-date master
> > >>      (fixups, git add, git rebase --continue, etc, etc, etc)
> > >>      $ git checkout master
> > >>      $ git merge HBASE-10070
> > >>
> > >> ?
> > >>
> > >> That sounds good to me, the final merge should be a fast forward
> merge.
> > >>
> > >> Use of ' --ignore-date' could be mildly controversial. It's not
> strictly
> > >> necessary because the commits for 10070 will appear grouped in
> history,
> > but
> > >> then dates on commits will be discontiguous in that section of
> history.
> > I
> > >> suggest using that option so the order of commits and dates sort the
> > same
> > >> on master.
> > >>
> > >>
> > >> On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <en...@apache.org>
> wrote:
> > >>
> > >>> Hi,
> > >>>
> > >>> Last week we started some discussion[4] for merging branch
> > hbase-10070[1]
> > >>> into trunk. It seems like the consensus there is to do the merge
> sooner
> > >>> rather than later.
> > >>>
> > >>>
> > >>> We had branched hbase-10070 in Feb out of trunk[5]. The branch
> contains
> > >>> 55
> > >>> jiras committed[2]. Out of these 55, 15 has already been committed to
> > >>> trunk
> > >>> and backported to hbase-10070 branch[3].
> > >>>
> > >>> This VOTE is for merging back the remaining changes in branch to
> trunk.
> > >>> If
> > >>> passes, we will rebase the branch on top of current trunk, in which
> we
> > >>> will
> > >>> keep the commit-per-issue log history. After that we will do a git
> > merge
> > >>> for the branch keeping the history clean and not squashing the
> > commits. I
> > >>> expect rebasing to be straightforward, however with some manual
> > conflict
> > >>> resolution. After the merge we'll keep running the tests to make sure
> > >>> everything is ok.
> > >>>
> > >>> An overview of the changes, and the status of the work can be found
> > under
> > >>> [4], [6] and [7].In summary, with the code in branch, you can create
> > >>> tables
> > >>> with region replicas, do gets / multi gets and scans using TIMELINE
> > >>> consistency with high availability. Region replicas periodically scan
> > the
> > >>> files of the primary and pick up flushed / committed files. The RPC
> > >>> paths /
> > >>> assignment, balancing etc are pretty stable. However some more
> > >>> performance
> > >>> analysis and tuning is needed. Phase 2 work is being worked on under
> > >>> HBASE-11183, and we have some working prototype for async-replicating
> > and
> > >>> region splits. However, we believe even without those features, this
> > work
> > >>> is useable (especially for read-only/bulk load tables) , and can be
> > >>> released as an experimental feature in 1.0.
> > >>>
> > >>> Please indicate your choice:
> > >>>
> > >>> [ ] +1 on yes, merge branch hbase-10070 to trunk.
> > >>> [ ] 0 on don't care
> > >>> [ ] -1 don't merge, because ...
> > >>>
> > >>> I'll keep the vote running for 7 days, and close it Mon 9th of June,
> > PDT.
> > >>>
> > >>> Here is my official +1.
> > >>>
> > >>> Thanks,
> > >>> Enis
> > >>>
> > >>> [1]
> > >>>
> > >>>
> >
> https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=log;h=refs/heads/hbase-10070
> > >>> [2]
> > >>>
> > >>>
> >
> https://issues.apache.org/jira/browse/HBASE-11214?jql=fixVersion%20%3D%20hbase-10070%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolved
> > >>> [3]
> > >>>
> > >>>
> >
> https://issues.apache.org/jira/browse/HBASE-10792?jql=fixVersion%20%3D%20hbase-10070%20and%20fixversion%20%3D%200.99.0%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolved
> > >>> [4] https://www.mail-archive.com/dev@hbase.apache.org/msg25795.html
> > >>> [5]
> > >>>
> > >>>
> >
> https://github.com/apache/hbase/commit/e22c7efeac02efde3451a0c9ff9bdcd2725576d0
> > >>> [6]
> > >>>
> > >>>
> >
> http://www.slideshare.net/enissoz/hbase-high-availability-for-reads-with-time
> > >>> [7] https://issues.apache.org/jira/browse/HBASE-10070
> > >>>
> > >>
> > >>
> > >>
> > >> --
> > >> Best regards,
> > >>
> > >>    - Andy
> > >>
> > >> Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > >> (via Tom White)
> > >>
> > >
> > >
> > >
> > > --
> > > Best regards,
> > >
> > >    - Andy
> > >
> > > Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > > (via Tom White)
> > >
> >
> >
> >
> > --
> > Best regards,
> >
> >    - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
> >
>



-- 
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Re: [VOTE] Merge branch HBASE-10070 to trunk

Posted by Enis Söztutar <en...@gmail.com>.

Sorry, I was mostly out for HadoopSummit.

Yes, the git flow would be very similar to what you propose:

     $ git checkout HBASE-10070
     $ git rebase --ignore-date master
     (fixups, git add, git rebase --continue, etc, etc, etc)
     $ git checkout master

     $ git push origin HBASE-10070 HBASE-10070-rebase-date   (optionally)
     $ git merge HBASE-10070

We can either go --ignore-date or not depending on what we want. If needed
I am fine with pushing the rebased master branch for review to main repo
before the merge to another branch. If not, I can just rebase the branch
locally and merge + push to main repo.

Creating final patches and attaching them to jira might be cumbersome. If
we do the rebased-branch on repo, we might not need it. But if we need that
for review, I can do it.

Thanks,
Enis



On Wed, Jun 4, 2014 at 10:48 AM, Andrew Purtell <ap...@apache.org> wrote:

> I realize this is a vote thread but I need a satisfactory answer to the
> below inquiries before feeling comfortable casting a vote. Or perhaps that
> means we need to cancel this vote and move back to discussion.
>
>
> On Tue, Jun 3, 2014 at 11:17 AM, Andrew Purtell <ap...@apache.org>
> wrote:
>
> > Also after the merge process is completed, do you plan to use git
> > format-patch to break out the per-JIRA changes into updated patches for
> > those JIRAs representing in effect the final commit?
> >
> >
> > On Tue, Jun 3, 2014 at 11:16 AM, Andrew Purtell <ap...@apache.org>
> > wrote:
> >
> >>  
> >> On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <en...@apache.org> wrote:
> >>
> >> This VOTE is for merging back the remaining changes in branch to trunk.
> If
> >>> passes, we will rebase the branch on top of current trunk, in which we
> >>> will
> >>> keep the commit-per-issue log history. After that we will do a git
> merge
> >>> for the branch keeping the history clean and not squashing the
> commits. I
> >>> expect rebasing to be straightforward, however with some manual
> conflict
> >>> resolution. After the merge we'll keep running the tests to make sure
> >>> everything is ok.
> >>>
> >>
> >> Just to clarify that would look something like this:
> >>
> >>      $ git checkout HBASE-10070
> >>      $ git rebase --ignore-date master
> >>      (fixups, git add, git rebase --continue, etc, etc, etc)
> >>      $ git checkout master
> >>      $ git merge HBASE-10070
> >>
> >> ?
> >>
> >> That sounds good to me, the final merge should be a fast forward merge.
> >>
> >> Use of ' --ignore-date' could be mildly controversial. It's not strictly
> >> necessary because the commits for 10070 will appear grouped in history,
> but
> >> then dates on commits will be discontiguous in that section of history.
> I
> >> suggest using that option so the order of commits and dates sort the
> same
> >> on master.
> >>
> >>
> >> On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <en...@apache.org> wrote:
> >>
> >>> Hi,
> >>>
> >>> Last week we started some discussion[4] for merging branch
> hbase-10070[1]
> >>> into trunk. It seems like the consensus there is to do the merge sooner
> >>> rather than later.
> >>>
> >>>
> >>> We had branched hbase-10070 in Feb out of trunk[5]. The branch contains
> >>> 55
> >>> jiras committed[2]. Out of these 55, 15 has already been committed to
> >>> trunk
> >>> and backported to hbase-10070 branch[3].
> >>>
> >>> This VOTE is for merging back the remaining changes in branch to trunk.
> >>> If
> >>> passes, we will rebase the branch on top of current trunk, in which we
> >>> will
> >>> keep the commit-per-issue log history. After that we will do a git
> merge
> >>> for the branch keeping the history clean and not squashing the
> commits. I
> >>> expect rebasing to be straightforward, however with some manual
> conflict
> >>> resolution. After the merge we'll keep running the tests to make sure
> >>> everything is ok.
> >>>
> >>> An overview of the changes, and the status of the work can be found
> under
> >>> [4], [6] and [7].In summary, with the code in branch, you can create
> >>> tables
> >>> with region replicas, do gets / multi gets and scans using TIMELINE
> >>> consistency with high availability. Region replicas periodically scan
> the
> >>> files of the primary and pick up flushed / committed files. The RPC
> >>> paths /
> >>> assignment, balancing etc are pretty stable. However some more
> >>> performance
> >>> analysis and tuning is needed. Phase 2 work is being worked on under
> >>> HBASE-11183, and we have some working prototype for async-replicating
> and
> >>> region splits. However, we believe even without those features, this
> work
> >>> is useable (especially for read-only/bulk load tables) , and can be
> >>> released as an experimental feature in 1.0.
> >>>
> >>> Please indicate your choice:
> >>>
> >>> [ ] +1 on yes, merge branch hbase-10070 to trunk.
> >>> [ ] 0 on don't care
> >>> [ ] -1 don't merge, because ...
> >>>
> >>> I'll keep the vote running for 7 days, and close it Mon 9th of June,
> PDT.
> >>>
> >>> Here is my official +1.
> >>>
> >>> Thanks,
> >>> Enis
> >>>
> >>> [1]
> >>>
> >>>
> https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=log;h=refs/heads/hbase-10070
> >>> [2]
> >>>
> >>>
> https://issues.apache.org/jira/browse/HBASE-11214?jql=fixVersion%20%3D%20hbase-10070%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolved
> >>> [3]
> >>>
> >>>
> https://issues.apache.org/jira/browse/HBASE-10792?jql=fixVersion%20%3D%20hbase-10070%20and%20fixversion%20%3D%200.99.0%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolved
> >>> [4] https://www.mail-archive.com/dev@hbase.apache.org/msg25795.html
> >>> [5]
> >>>
> >>>
> https://github.com/apache/hbase/commit/e22c7efeac02efde3451a0c9ff9bdcd2725576d0
> >>> [6]
> >>>
> >>>
> http://www.slideshare.net/enissoz/hbase-high-availability-for-reads-with-time
> >>> [7] https://issues.apache.org/jira/browse/HBASE-10070
> >>>
> >>
> >>
> >>
> >> --
> >> Best regards,
> >>
> >>    - Andy
> >>
> >> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> >> (via Tom White)
> >>
> >
> >
> >
> > --
> > Best regards,
> >
> >    - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
> >
>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>

Re: [VOTE] Merge branch HBASE-10070 to trunk

Posted by Andrew Purtell <ap...@apache.org>.

I realize this is a vote thread but I need a satisfactory answer to the
below inquiries before feeling comfortable casting a vote. Or perhaps that
means we need to cancel this vote and move back to discussion.


On Tue, Jun 3, 2014 at 11:17 AM, Andrew Purtell <ap...@apache.org> wrote:

> Also after the merge process is completed, do you plan to use git
> format-patch to break out the per-JIRA changes into updated patches for
> those JIRAs representing in effect the final commit?
>
>
> On Tue, Jun 3, 2014 at 11:16 AM, Andrew Purtell <ap...@apache.org>
> wrote:
>
>>  
>> On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <en...@apache.org> wrote:
>>
>> This VOTE is for merging back the remaining changes in branch to trunk. If
>>> passes, we will rebase the branch on top of current trunk, in which we
>>> will
>>> keep the commit-per-issue log history. After that we will do a git merge
>>> for the branch keeping the history clean and not squashing the commits. I
>>> expect rebasing to be straightforward, however with some manual conflict
>>> resolution. After the merge we'll keep running the tests to make sure
>>> everything is ok.
>>>
>>
>> Just to clarify that would look something like this:
>>
>>      $ git checkout HBASE-10070
>>      $ git rebase --ignore-date master
>>      (fixups, git add, git rebase --continue, etc, etc, etc)
>>      $ git checkout master
>>      $ git merge HBASE-10070
>>
>> ?
>>
>> That sounds good to me, the final merge should be a fast forward merge.
>>
>> Use of ' --ignore-date' could be mildly controversial. It's not strictly
>> necessary because the commits for 10070 will appear grouped in history, but
>> then dates on commits will be discontiguous in that section of history. I
>> suggest using that option so the order of commits and dates sort the same
>> on master.
>>
>>
>> On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <en...@apache.org> wrote:
>>
>>> Hi,
>>>
>>> Last week we started some discussion[4] for merging branch hbase-10070[1]
>>> into trunk. It seems like the consensus there is to do the merge sooner
>>> rather than later.
>>>
>>>
>>> We had branched hbase-10070 in Feb out of trunk[5]. The branch contains
>>> 55
>>> jiras committed[2]. Out of these 55, 15 has already been committed to
>>> trunk
>>> and backported to hbase-10070 branch[3].
>>>
>>> This VOTE is for merging back the remaining changes in branch to trunk.
>>> If
>>> passes, we will rebase the branch on top of current trunk, in which we
>>> will
>>> keep the commit-per-issue log history. After that we will do a git merge
>>> for the branch keeping the history clean and not squashing the commits. I
>>> expect rebasing to be straightforward, however with some manual conflict
>>> resolution. After the merge we'll keep running the tests to make sure
>>> everything is ok.
>>>
>>> An overview of the changes, and the status of the work can be found under
>>> [4], [6] and [7].In summary, with the code in branch, you can create
>>> tables
>>> with region replicas, do gets / multi gets and scans using TIMELINE
>>> consistency with high availability. Region replicas periodically scan the
>>> files of the primary and pick up flushed / committed files. The RPC
>>> paths /
>>> assignment, balancing etc are pretty stable. However some more
>>> performance
>>> analysis and tuning is needed. Phase 2 work is being worked on under
>>> HBASE-11183, and we have some working prototype for async-replicating and
>>> region splits. However, we believe even without those features, this work
>>> is useable (especially for read-only/bulk load tables) , and can be
>>> released as an experimental feature in 1.0.
>>>
>>> Please indicate your choice:
>>>
>>> [ ] +1 on yes, merge branch hbase-10070 to trunk.
>>> [ ] 0 on don't care
>>> [ ] -1 don't merge, because ...
>>>
>>> I'll keep the vote running for 7 days, and close it Mon 9th of June, PDT.
>>>
>>> Here is my official +1.
>>>
>>> Thanks,
>>> Enis
>>>
>>> [1]
>>>
>>> https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=log;h=refs/heads/hbase-10070
>>> [2]
>>>
>>> https://issues.apache.org/jira/browse/HBASE-11214?jql=fixVersion%20%3D%20hbase-10070%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolved
>>> [3]
>>>
>>> https://issues.apache.org/jira/browse/HBASE-10792?jql=fixVersion%20%3D%20hbase-10070%20and%20fixversion%20%3D%200.99.0%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolved
>>> [4] https://www.mail-archive.com/dev@hbase.apache.org/msg25795.html
>>> [5]
>>>
>>> https://github.com/apache/hbase/commit/e22c7efeac02efde3451a0c9ff9bdcd2725576d0
>>> [6]
>>>
>>> http://www.slideshare.net/enissoz/hbase-high-availability-for-reads-with-time
>>> [7] https://issues.apache.org/jira/browse/HBASE-10070
>>>
>>
>>
>>
>> --
>> Best regards,
>>
>>    - Andy
>>
>> Problems worthy of attack prove their worth by hitting back. - Piet Hein
>> (via Tom White)
>>
>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>



-- 
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Re: [VOTE] Merge branch HBASE-10070 to trunk

Posted by Andrew Purtell <ap...@apache.org>.

Also after the merge process is completed, do you plan to use git
format-patch to break out the per-JIRA changes into updated patches for
those JIRAs representing in effect the final commit?


On Tue, Jun 3, 2014 at 11:16 AM, Andrew Purtell <ap...@apache.org> wrote:

>  
> On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <en...@apache.org> wrote:
>
> This VOTE is for merging back the remaining changes in branch to trunk. If
>> passes, we will rebase the branch on top of current trunk, in which we
>> will
>> keep the commit-per-issue log history. After that we will do a git merge
>> for the branch keeping the history clean and not squashing the commits. I
>> expect rebasing to be straightforward, however with some manual conflict
>> resolution. After the merge we'll keep running the tests to make sure
>> everything is ok.
>>
>
> Just to clarify that would look something like this:
>
>      $ git checkout HBASE-10070
>      $ git rebase --ignore-date master
>      (fixups, git add, git rebase --continue, etc, etc, etc)
>      $ git checkout master
>      $ git merge HBASE-10070
>
> ?
>
> That sounds good to me, the final merge should be a fast forward merge.
>
> Use of ' --ignore-date' could be mildly controversial. It's not strictly
> necessary because the commits for 10070 will appear grouped in history, but
> then dates on commits will be discontiguous in that section of history. I
> suggest using that option so the order of commits and dates sort the same
> on master.
>
>
> On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <en...@apache.org> wrote:
>
>> Hi,
>>
>> Last week we started some discussion[4] for merging branch hbase-10070[1]
>> into trunk. It seems like the consensus there is to do the merge sooner
>> rather than later.
>>
>>
>> We had branched hbase-10070 in Feb out of trunk[5]. The branch contains 55
>> jiras committed[2]. Out of these 55, 15 has already been committed to
>> trunk
>> and backported to hbase-10070 branch[3].
>>
>> This VOTE is for merging back the remaining changes in branch to trunk. If
>> passes, we will rebase the branch on top of current trunk, in which we
>> will
>> keep the commit-per-issue log history. After that we will do a git merge
>> for the branch keeping the history clean and not squashing the commits. I
>> expect rebasing to be straightforward, however with some manual conflict
>> resolution. After the merge we'll keep running the tests to make sure
>> everything is ok.
>>
>> An overview of the changes, and the status of the work can be found under
>> [4], [6] and [7].In summary, with the code in branch, you can create
>> tables
>> with region replicas, do gets / multi gets and scans using TIMELINE
>> consistency with high availability. Region replicas periodically scan the
>> files of the primary and pick up flushed / committed files. The RPC paths
>> /
>> assignment, balancing etc are pretty stable. However some more performance
>> analysis and tuning is needed. Phase 2 work is being worked on under
>> HBASE-11183, and we have some working prototype for async-replicating and
>> region splits. However, we believe even without those features, this work
>> is useable (especially for read-only/bulk load tables) , and can be
>> released as an experimental feature in 1.0.
>>
>> Please indicate your choice:
>>
>> [ ] +1 on yes, merge branch hbase-10070 to trunk.
>> [ ] 0 on don't care
>> [ ] -1 don't merge, because ...
>>
>> I'll keep the vote running for 7 days, and close it Mon 9th of June, PDT.
>>
>> Here is my official +1.
>>
>> Thanks,
>> Enis
>>
>> [1]
>>
>> https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=log;h=refs/heads/hbase-10070
>> [2]
>>
>> https://issues.apache.org/jira/browse/HBASE-11214?jql=fixVersion%20%3D%20hbase-10070%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolved
>> [3]
>>
>> https://issues.apache.org/jira/browse/HBASE-10792?jql=fixVersion%20%3D%20hbase-10070%20and%20fixversion%20%3D%200.99.0%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolved
>> [4] https://www.mail-archive.com/dev@hbase.apache.org/msg25795.html
>> [5]
>>
>> https://github.com/apache/hbase/commit/e22c7efeac02efde3451a0c9ff9bdcd2725576d0
>> [6]
>>
>> http://www.slideshare.net/enissoz/hbase-high-availability-for-reads-with-time
>> [7] https://issues.apache.org/jira/browse/HBASE-10070
>>
>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>



-- 
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Re: [VOTE] Merge branch HBASE-10070 to trunk

Posted by Jimmy Xiang <jx...@cloudera.com>.

zk-less assignment will be off by default.

co-location of meta and master is on by default now. We can turn it off if
needed.

Thanks,
Jimmy



On Thu, Jun 12, 2014 at 2:12 PM, Mikhail Antonov <ol...@gmail.com>
wrote:

> So how would the defaults and dependencies look amongst these 3, once
> committed to 0.99?
>
>  - co-location of meta and master
>  - zk-less assignment
> - timeline-consistent replicas?
>
> As I understand -
>  -  All 3 are off by default
>  -  zk-less assignments and region replicas to not be turned on at the same
> time
>  -  zk-less assignments require meta co-location
>  -  region replicas don't care about meta co-location?
>
> -Mikhail
>
>
> 2014-06-12 14:07 GMT-07:00 Jonathan Hsieh <jo...@cloudera.com>:
>
> > sgtm.
> >
> >
> > On Mon, Jun 9, 2014 at 10:46 AM, Enis Söztutar <en...@gmail.com>
> wrote:
> >
> > > Thanks Jon.
> > >
> > > I think we'll try the rebase approach first. I'll start the effort
> today
> > > and see how far along I can get with that. Rebasing each patch might
> be a
> > > bit more work actually. If this turns out painful, we'll revert to the
> > > merge approach similar to what you describe.
> > >
> > > Enis
> > >
> > >
> > > On Sat, Jun 7, 2014 at 7:41 AM, Jonathan Hsieh <jo...@cloudera.com>
> wrote:
> > >
> > > > On Fri, Jun 6, 2014 at 5:05 PM, Enis Söztutar <en...@gmail.com>
> > > wrote:
> > > >
> > > > > My preference would be to do the rebase-then-merge style (last
> style
> > in
> > > > > your comment). For each patch, I am hoping for all the changes
> > between
> > > > > committed version and rebased version to be mechanical. I like
> > having a
> > > > > linear history with explicit commit-to-patch mapping.
> > > > >
> > > > > If the changes are of a mechanical nature and that each patch
> > compiles
> > > > and
> > > > passes unit tests along the way, then rebase-then-merge is perfectly
> > fine
> > > > by me.
> > > >
> > > > Even though snapshots were fairly orthogonal to the rest of the
> > codebase,
> > > > when we did merges we had problems maintaining the compiles and
> passes
> > > with
> > > > every commit invariants when we tried rebase-then-merge approach.
> >  After
> > > > each rebase we would end up doing a bunch of work and still end up
> > > failing
> > > > unit tests.   In that case we (jesse, matteo, myself) went to the
> merge
> > > > approach.   We actually merged into the snapshot branch a few times
> > > fixing
> > > > things due to changes in trunk that broke parts of the snapshots
> > branch.
> > > > Here's a more accurate picture of what the commit history looked like
> > in
> > > > git (we dev'ed in git and in end had to recommit this all in svn).
> > > >
> > > > m  (essentially empty merge patch).
> > > > |\
> > > > t |  (trunk patch with no impact)
> > > > | s6* (patch to fix newly introduced problem)
> > > > |/|
> > > > t |
> > > > t*|   (trunk patch that broke snapshots again)
> > > > | s5* (branch bug fixes due to merge)
> > > > | s4* (mechanical fixups due to the merge)
> > > > |/|
> > > > t |
> > > > t |
> > > > t |
> > > > | s3
> > > > | s2
> > > > | s1
> > > > |/
> > > > t
> > > > t
> > > >
> > > > This approach was captured in github and had the added benefit of
> > > > preserving exact history, and having known good points preserving the
> > > > compile/unit test invariants on trunk.  (unfortunately, some of this
> > > > history was lost when we ported the git history over to svn, but that
> > > isn't
> > > > a problem any more).    If you want to see the whole thing, look at
> my
> > > git
> > > > repo:
> > > >
> > > > git remote add jmhsieh git@github.com:jmhsieh/hbase.git
> > > > git log --oneline --graph --color jmhsieh/hbase-7290
> > > >
> > > >
> > > > Can we do history-preserving merge with the first style? I can do the
> > > > > rebases and upload interdiff if that is big so that we can compare
> > on a
> > > > > per-patch basis.
> > > > >
> > > > >
> > > > It is possible.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > > Enis
> > > > >
> > > > >
> > > > > On Fri, Jun 6, 2014 at 4:36 PM, Jonathan Hsieh <jo...@cloudera.com>
> > > wrote:
> > > > >
> > > > > > When we merged snapshots branch in we did this:
> > > > > >
> > > > > > t = trunk commit
> > > > > > s = snapshot branch commit
> > > > > > m = merge point.
> > > > > >
> > > > > > During work:
> > > > > > t
> > > > > > t
> > > > > > t
> > > > > > |  s3
> > > > > > |  s2
> > > > > > |  s1
> > > > > > |/
> > > > > > t
> > > > > > t
> > > > > >
> > > > > > During after merge:
> > > > > > m  (essentially empty merge patch).
> > > > > > t \
> > > > > > t |
> > > > > > t |
> > > > > > | s4* (fixups due to the merge)
> > > > > > | s3
> > > > > > | s2
> > > > > > | s1
> > > > > > |/
> > > > > > t
> > > > > > t
> > > > > >
> > > > > > Does your proposal mean you are going to do this instead?
> > > > > >
> > > > > > s3* (modified due to rebase)
> > > > > > s2* (modified due to rebase)
> > > > > > s1
> > > > > > t
> > > > > > t
> > > > > > t
> > > > > > | s (now dead hbase-10070 branch)
> > > > > > | s
> > > > > > | s
> > > > > > | s
> > > > > > |/
> > > > > > t
> > > > > > t
> > > > > >
> > > > > > If it is the then we should probably have a review of the
> modified
> > > > > patches.
> > > > > >  If it the same as the snapshot merge, then we just need a review
> > of
> > > > the
> > > > > > merge point delta (as well as a full review).
> > > > > >
> > > > > > Personally I prefer the merge for these large feature branches --
> > it
> > > > > > guarantees that each commit is compilable, and reflects what you
> > guys
> > > > > have
> > > > > > been testing for a while.  If you go with the last approach you
> > might
> > > > > have
> > > > > > stuff broken, and in the mainline commit path.
> > > > > >
> > > > > > Jon.
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Tue, Jun 3, 2014 at 11:16 AM, Andrew Purtell <
> > apurtell@apache.org
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > >  
> > > > > > > On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <enis@apache.org
> >
> > > > wrote:
> > > > > > >
> > > > > > > > This VOTE is for merging back the remaining changes in branch
> > to
> > > > > trunk.
> > > > > > > If
> > > > > > > > passes, we will rebase the branch on top of current trunk, in
> > > which
> > > > > we
> > > > > > > will
> > > > > > > > keep the commit-per-issue log history. After that we will do
> a
> > > git
> > > > > > merge
> > > > > > > > for the branch keeping the history clean and not squashing
> the
> > > > > > commits. I
> > > > > > > > expect rebasing to be straightforward, however with some
> manual
> > > > > > conflict
> > > > > > > > resolution. After the merge we'll keep running the tests to
> > make
> > > > sure
> > > > > > > > everything is ok.
> > > > > > > >
> > > > > > >
> > > > > > > Just to clarify that would look something like this:
> > > > > > >
> > > > > > >      $ git checkout HBASE-10070
> > > > > > >      $ git rebase --ignore-date master
> > > > > > >      (fixups, git add, git rebase --continue, etc, etc, etc)
> > > > > > >      $ git checkout master
> > > > > > >      $ git merge HBASE-10070
> > > > > > >
> > > > > > > ?
> > > > > > >
> > > > > > > That sounds good to me, the final merge should be a fast
> forward
> > > > merge.
> > > > > > >
> > > > > > > Use of ' --ignore-date' could be mildly controversial. It's not
> > > > > strictly
> > > > > > > necessary because the commits for 10070 will appear grouped in
> > > > history,
> > > > > > but
> > > > > > > then dates on commits will be discontiguous in that section of
> > > > > history. I
> > > > > > > suggest using that option so the order of commits and dates
> sort
> > > the
> > > > > same
> > > > > > > on master.
> > > > > > >
> > > > > > >
> > > > > > > On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <enis@apache.org
> >
> > > > wrote:
> > > > > > >
> > > > > > > > Hi,
> > > > > > > >
> > > > > > > > Last week we started some discussion[4] for merging branch
> > > > > > hbase-10070[1]
> > > > > > > > into trunk. It seems like the consensus there is to do the
> > merge
> > > > > sooner
> > > > > > > > rather than later.
> > > > > > > >
> > > > > > > >
> > > > > > > > We had branched hbase-10070 in Feb out of trunk[5]. The
> branch
> > > > > contains
> > > > > > > 55
> > > > > > > > jiras committed[2]. Out of these 55, 15 has already been
> > > committed
> > > > to
> > > > > > > trunk
> > > > > > > > and backported to hbase-10070 branch[3].
> > > > > > > >
> > > > > > > > This VOTE is for merging back the remaining changes in branch
> > to
> > > > > trunk.
> > > > > > > If
> > > > > > > > passes, we will rebase the branch on top of current trunk, in
> > > which
> > > > > we
> > > > > > > will
> > > > > > > > keep the commit-per-issue log history. After that we will do
> a
> > > git
> > > > > > merge
> > > > > > > > for the branch keeping the history clean and not squashing
> the
> > > > > > commits. I
> > > > > > > > expect rebasing to be straightforward, however with some
> manual
> > > > > > conflict
> > > > > > > > resolution. After the merge we'll keep running the tests to
> > make
> > > > sure
> > > > > > > > everything is ok.
> > > > > > > >
> > > > > > > > An overview of the changes, and the status of the work can be
> > > found
> > > > > > under
> > > > > > > > [4], [6] and [7].In summary, with the code in branch, you can
> > > > create
> > > > > > > tables
> > > > > > > > with region replicas, do gets / multi gets and scans using
> > > TIMELINE
> > > > > > > > consistency with high availability. Region replicas
> > periodically
> > > > scan
> > > > > > the
> > > > > > > > files of the primary and pick up flushed / committed files.
> The
> > > RPC
> > > > > > > paths /
> > > > > > > > assignment, balancing etc are pretty stable. However some
> more
> > > > > > > performance
> > > > > > > > analysis and tuning is needed. Phase 2 work is being worked
> on
> > > > under
> > > > > > > > HBASE-11183, and we have some working prototype for
> > > > async-replicating
> > > > > > and
> > > > > > > > region splits. However, we believe even without those
> features,
> > > > this
> > > > > > work
> > > > > > > > is useable (especially for read-only/bulk load tables) , and
> > can
> > > be
> > > > > > > > released as an experimental feature in 1.0.
> > > > > > > >
> > > > > > > > Please indicate your choice:
> > > > > > > >
> > > > > > > > [ ] +1 on yes, merge branch hbase-10070 to trunk.
> > > > > > > > [ ] 0 on don't care
> > > > > > > > [ ] -1 don't merge, because ...
> > > > > > > >
> > > > > > > > I'll keep the vote running for 7 days, and close it Mon 9th
> of
> > > > June,
> > > > > > PDT.
> > > > > > > >
> > > > > > > > Here is my official +1.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Enis
> > > > > > > >
> > > > > > > > [1]
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=log;h=refs/heads/hbase-10070
> > > > > > > > [2]
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/browse/HBASE-11214?jql=fixVersion%20%3D%20hbase-10070%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolved
> > > > > > > > [3]
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/browse/HBASE-10792?jql=fixVersion%20%3D%20hbase-10070%20and%20fixversion%20%3D%200.99.0%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolved
> > > > > > > > [4]
> > > > https://www.mail-archive.com/dev@hbase.apache.org/msg25795.html
> > > > > > > > [5]
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hbase/commit/e22c7efeac02efde3451a0c9ff9bdcd2725576d0
> > > > > > > > [6]
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://www.slideshare.net/enissoz/hbase-high-availability-for-reads-with-time
> > > > > > > > [7] https://issues.apache.org/jira/browse/HBASE-10070
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Best regards,
> > > > > > >
> > > > > > >    - Andy
> > > > > > >
> > > > > > > Problems worthy of attack prove their worth by hitting back. -
> > Piet
> > > > > Hein
> > > > > > > (via Tom White)
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > // Jonathan Hsieh (shay)
> > > > > > // HBase Tech Lead, Software Engineer, Cloudera
> > > > > > // jon@cloudera.com // @jmhsieh
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > // Jonathan Hsieh (shay)
> > > > // HBase Tech Lead, Software Engineer, Cloudera
> > > > // jon@cloudera.com // @jmhsieh
> > > >
> > >
> >
> >
> >
> > --
> > // Jonathan Hsieh (shay)
> > // HBase Tech Lead, Software Engineer, Cloudera
> > // jon@cloudera.com // @jmhsieh
> >
>
>
>
> --
> Thanks,
> Michael Antonov
>

Re: [VOTE] Merge branch HBASE-10070 to trunk

Posted by Mikhail Antonov <ol...@gmail.com>.

I'd think we do. May be server shall simply abort if both are on?

-Sent from mobile, please excuse typos
Mikhail

> On Jun 12, 2014, at 14:33, Ted Yu <yu...@gmail.com> wrote:
> 
> bq. zk-less assignments and region replicas to not be turned on at the same
> time
> 
> Interesting. Do we need to add a configuration check for the above ?
> 
> 
> On Thu, Jun 12, 2014 at 2:12 PM, Mikhail Antonov <ol...@gmail.com>
> wrote:
> 
>> So how would the defaults and dependencies look amongst these 3, once
>> committed to 0.99?
>> 
>> - co-location of meta and master
>> - zk-less assignment
>> - timeline-consistent replicas?
>> 
>> As I understand -
>> -  All 3 are off by default
>> -  zk-less assignments and region replicas to not be turned on at the same
>> time
>> -  zk-less assignments require meta co-location
>> -  region replicas don't care about meta co-location?
>> 
>> -Mikhail
>> 
>> 
>> 2014-06-12 14:07 GMT-07:00 Jonathan Hsieh <jo...@cloudera.com>:
>> 
>>> sgtm.
>>> 
>>> 
>>>> On Mon, Jun 9, 2014 at 10:46 AM, Enis Söztutar <en...@gmail.com>
>>> wrote:
>>> 
>>>> Thanks Jon.
>>>> 
>>>> I think we'll try the rebase approach first. I'll start the effort
>> today
>>>> and see how far along I can get with that. Rebasing each patch might
>> be a
>>>> bit more work actually. If this turns out painful, we'll revert to the
>>>> merge approach similar to what you describe.
>>>> 
>>>> Enis
>>>> 
>>>> 
>>>> On Sat, Jun 7, 2014 at 7:41 AM, Jonathan Hsieh <jo...@cloudera.com>
>> wrote:
>>>> 
>>>>>> On Fri, Jun 6, 2014 at 5:05 PM, Enis Söztutar <en...@gmail.com>
>>>>> wrote:
>>>>> 
>>>>>> My preference would be to do the rebase-then-merge style (last
>> style
>>> in
>>>>>> your comment). For each patch, I am hoping for all the changes
>>> between
>>>>>> committed version and rebased version to be mechanical. I like
>>> having a
>>>>>> linear history with explicit commit-to-patch mapping.
>>>>>> 
>>>>>> If the changes are of a mechanical nature and that each patch
>>> compiles
>>>>> and
>>>>> passes unit tests along the way, then rebase-then-merge is perfectly
>>> fine
>>>>> by me.
>>>>> 
>>>>> Even though snapshots were fairly orthogonal to the rest of the
>>> codebase,
>>>>> when we did merges we had problems maintaining the compiles and
>> passes
>>>> with
>>>>> every commit invariants when we tried rebase-then-merge approach.
>>> After
>>>>> each rebase we would end up doing a bunch of work and still end up
>>>> failing
>>>>> unit tests.   In that case we (jesse, matteo, myself) went to the
>> merge
>>>>> approach.   We actually merged into the snapshot branch a few times
>>>> fixing
>>>>> things due to changes in trunk that broke parts of the snapshots
>>> branch.
>>>>> Here's a more accurate picture of what the commit history looked like
>>> in
>>>>> git (we dev'ed in git and in end had to recommit this all in svn).
>>>>> 
>>>>> m  (essentially empty merge patch).
>>>>> |\
>>>>> t |  (trunk patch with no impact)
>>>>> | s6* (patch to fix newly introduced problem)
>>>>> |/|
>>>>> t |
>>>>> t*|   (trunk patch that broke snapshots again)
>>>>> | s5* (branch bug fixes due to merge)
>>>>> | s4* (mechanical fixups due to the merge)
>>>>> |/|
>>>>> t |
>>>>> t |
>>>>> t |
>>>>> | s3
>>>>> | s2
>>>>> | s1
>>>>> |/
>>>>> t
>>>>> t
>>>>> 
>>>>> This approach was captured in github and had the added benefit of
>>>>> preserving exact history, and having known good points preserving the
>>>>> compile/unit test invariants on trunk.  (unfortunately, some of this
>>>>> history was lost when we ported the git history over to svn, but that
>>>> isn't
>>>>> a problem any more).    If you want to see the whole thing, look at
>> my
>>>> git
>>>>> repo:
>>>>> 
>>>>> git remote add jmhsieh git@github.com:jmhsieh/hbase.git
>>>>> git log --oneline --graph --color jmhsieh/hbase-7290
>>>>> 
>>>>> 
>>>>> Can we do history-preserving merge with the first style? I can do the
>>>>>> rebases and upload interdiff if that is big so that we can compare
>>> on a
>>>>>> per-patch basis.
>>>>> It is possible.
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>>> Enis
>>>>>> 
>>>>>> 
>>>>>> On Fri, Jun 6, 2014 at 4:36 PM, Jonathan Hsieh <jo...@cloudera.com>
>>>> wrote:
>>>>>> 
>>>>>>> When we merged snapshots branch in we did this:
>>>>>>> 
>>>>>>> t = trunk commit
>>>>>>> s = snapshot branch commit
>>>>>>> m = merge point.
>>>>>>> 
>>>>>>> During work:
>>>>>>> t
>>>>>>> t
>>>>>>> t
>>>>>>> |  s3
>>>>>>> |  s2
>>>>>>> |  s1
>>>>>>> |/
>>>>>>> t
>>>>>>> t
>>>>>>> 
>>>>>>> During after merge:
>>>>>>> m  (essentially empty merge patch).
>>>>>>> t \
>>>>>>> t |
>>>>>>> t |
>>>>>>> | s4* (fixups due to the merge)
>>>>>>> | s3
>>>>>>> | s2
>>>>>>> | s1
>>>>>>> |/
>>>>>>> t
>>>>>>> t
>>>>>>> 
>>>>>>> Does your proposal mean you are going to do this instead?
>>>>>>> 
>>>>>>> s3* (modified due to rebase)
>>>>>>> s2* (modified due to rebase)
>>>>>>> s1
>>>>>>> t
>>>>>>> t
>>>>>>> t
>>>>>>> | s (now dead hbase-10070 branch)
>>>>>>> | s
>>>>>>> | s
>>>>>>> | s
>>>>>>> |/
>>>>>>> t
>>>>>>> t
>>>>>>> 
>>>>>>> If it is the then we should probably have a review of the
>> modified
>>>>>> patches.
>>>>>>> If it the same as the snapshot merge, then we just need a review
>>> of
>>>>> the
>>>>>>> merge point delta (as well as a full review).
>>>>>>> 
>>>>>>> Personally I prefer the merge for these large feature branches --
>>> it
>>>>>>> guarantees that each commit is compilable, and reflects what you
>>> guys
>>>>>> have
>>>>>>> been testing for a while.  If you go with the last approach you
>>> might
>>>>>> have
>>>>>>> stuff broken, and in the mainline commit path.
>>>>>>> 
>>>>>>> Jon.
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> On Tue, Jun 3, 2014 at 11:16 AM, Andrew Purtell <
>>> apurtell@apache.org
>>>>> 
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> 
>>>>>>>> On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <enis@apache.org
>>> 
>>>>> wrote:
>>>>>>>> 
>>>>>>>>> This VOTE is for merging back the remaining changes in branch
>>> to
>>>>>> trunk.
>>>>>>>> If
>>>>>>>>> passes, we will rebase the branch on top of current trunk, in
>>>> which
>>>>>> we
>>>>>>>> will
>>>>>>>>> keep the commit-per-issue log history. After that we will do
>> a
>>>> git
>>>>>>> merge
>>>>>>>>> for the branch keeping the history clean and not squashing
>> the
>>>>>>> commits. I
>>>>>>>>> expect rebasing to be straightforward, however with some
>> manual
>>>>>>> conflict
>>>>>>>>> resolution. After the merge we'll keep running the tests to
>>> make
>>>>> sure
>>>>>>>>> everything is ok.
>>>>>>>> 
>>>>>>>> Just to clarify that would look something like this:
>>>>>>>> 
>>>>>>>>     $ git checkout HBASE-10070
>>>>>>>>     $ git rebase --ignore-date master
>>>>>>>>     (fixups, git add, git rebase --continue, etc, etc, etc)
>>>>>>>>     $ git checkout master
>>>>>>>>     $ git merge HBASE-10070
>>>>>>>> 
>>>>>>>> ?
>>>>>>>> 
>>>>>>>> That sounds good to me, the final merge should be a fast
>> forward
>>>>> merge.
>>>>>>>> 
>>>>>>>> Use of ' --ignore-date' could be mildly controversial. It's not
>>>>>> strictly
>>>>>>>> necessary because the commits for 10070 will appear grouped in
>>>>> history,
>>>>>>> but
>>>>>>>> then dates on commits will be discontiguous in that section of
>>>>>> history. I
>>>>>>>> suggest using that option so the order of commits and dates
>> sort
>>>> the
>>>>>> same
>>>>>>>> on master.
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <enis@apache.org
>>> 
>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Hi,
>>>>>>>>> 
>>>>>>>>> Last week we started some discussion[4] for merging branch
>>>>>>> hbase-10070[1]
>>>>>>>>> into trunk. It seems like the consensus there is to do the
>>> merge
>>>>>> sooner
>>>>>>>>> rather than later.
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> We had branched hbase-10070 in Feb out of trunk[5]. The
>> branch
>>>>>> contains
>>>>>>>> 55
>>>>>>>>> jiras committed[2]. Out of these 55, 15 has already been
>>>> committed
>>>>> to
>>>>>>>> trunk
>>>>>>>>> and backported to hbase-10070 branch[3].
>>>>>>>>> 
>>>>>>>>> This VOTE is for merging back the remaining changes in branch
>>> to
>>>>>> trunk.
>>>>>>>> If
>>>>>>>>> passes, we will rebase the branch on top of current trunk, in
>>>> which
>>>>>> we
>>>>>>>> will
>>>>>>>>> keep the commit-per-issue log history. After that we will do
>> a
>>>> git
>>>>>>> merge
>>>>>>>>> for the branch keeping the history clean and not squashing
>> the
>>>>>>> commits. I
>>>>>>>>> expect rebasing to be straightforward, however with some
>> manual
>>>>>>> conflict
>>>>>>>>> resolution. After the merge we'll keep running the tests to
>>> make
>>>>> sure
>>>>>>>>> everything is ok.
>>>>>>>>> 
>>>>>>>>> An overview of the changes, and the status of the work can be
>>>> found
>>>>>>> under
>>>>>>>>> [4], [6] and [7].In summary, with the code in branch, you can
>>>>> create
>>>>>>>> tables
>>>>>>>>> with region replicas, do gets / multi gets and scans using
>>>> TIMELINE
>>>>>>>>> consistency with high availability. Region replicas
>>> periodically
>>>>> scan
>>>>>>> the
>>>>>>>>> files of the primary and pick up flushed / committed files.
>> The
>>>> RPC
>>>>>>>> paths /
>>>>>>>>> assignment, balancing etc are pretty stable. However some
>> more
>>>>>>>> performance
>>>>>>>>> analysis and tuning is needed. Phase 2 work is being worked
>> on
>>>>> under
>>>>>>>>> HBASE-11183, and we have some working prototype for
>>>>> async-replicating
>>>>>>> and
>>>>>>>>> region splits. However, we believe even without those
>> features,
>>>>> this
>>>>>>> work
>>>>>>>>> is useable (especially for read-only/bulk load tables) , and
>>> can
>>>> be
>>>>>>>>> released as an experimental feature in 1.0.
>>>>>>>>> 
>>>>>>>>> Please indicate your choice:
>>>>>>>>> 
>>>>>>>>> [ ] +1 on yes, merge branch hbase-10070 to trunk.
>>>>>>>>> [ ] 0 on don't care
>>>>>>>>> [ ] -1 don't merge, because ...
>>>>>>>>> 
>>>>>>>>> I'll keep the vote running for 7 days, and close it Mon 9th
>> of
>>>>> June,
>>>>>>> PDT.
>>>>>>>>> 
>>>>>>>>> Here is my official +1.
>>>>>>>>> 
>>>>>>>>> Thanks,
>>>>>>>>> Enis
>>>>>>>>> 
>>>>>>>>> [1]
>> https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=log;h=refs/heads/hbase-10070
>>>>>>>>> [2]
>> https://issues.apache.org/jira/browse/HBASE-11214?jql=fixVersion%20%3D%20hbase-10070%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolved
>>>>>>>>> [3]
>> https://issues.apache.org/jira/browse/HBASE-10792?jql=fixVersion%20%3D%20hbase-10070%20and%20fixversion%20%3D%200.99.0%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolved
>>>>>>>>> [4]
>>>>> https://www.mail-archive.com/dev@hbase.apache.org/msg25795.html
>>>>>>>>> [5]
>> https://github.com/apache/hbase/commit/e22c7efeac02efde3451a0c9ff9bdcd2725576d0
>>>>>>>>> [6]
>> http://www.slideshare.net/enissoz/hbase-high-availability-for-reads-with-time
>>>>>>>>> [7] https://issues.apache.org/jira/browse/HBASE-10070
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> --
>>>>>>>> Best regards,
>>>>>>>> 
>>>>>>>>   - Andy
>>>>>>>> 
>>>>>>>> Problems worthy of attack prove their worth by hitting back. -
>>> Piet
>>>>>> Hein
>>>>>>>> (via Tom White)
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> --
>>>>>>> // Jonathan Hsieh (shay)
>>>>>>> // HBase Tech Lead, Software Engineer, Cloudera
>>>>>>> // jon@cloudera.com // @jmhsieh
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> // Jonathan Hsieh (shay)
>>>>> // HBase Tech Lead, Software Engineer, Cloudera
>>>>> // jon@cloudera.com // @jmhsieh
>>> 
>>> 
>>> 
>>> --
>>> // Jonathan Hsieh (shay)
>>> // HBase Tech Lead, Software Engineer, Cloudera
>>> // jon@cloudera.com // @jmhsieh
>> 
>> 
>> 
>> --
>> Thanks,
>> Michael Antonov
>>

Re: [VOTE] Merge branch HBASE-10070 to trunk

Posted by Ted Yu <yu...@gmail.com>.

bq. zk-less assignments and region replicas to not be turned on at the same
time

Interesting. Do we need to add a configuration check for the above ?


On Thu, Jun 12, 2014 at 2:12 PM, Mikhail Antonov <ol...@gmail.com>
wrote:

> So how would the defaults and dependencies look amongst these 3, once
> committed to 0.99?
>
>  - co-location of meta and master
>  - zk-less assignment
> - timeline-consistent replicas?
>
> As I understand -
>  -  All 3 are off by default
>  -  zk-less assignments and region replicas to not be turned on at the same
> time
>  -  zk-less assignments require meta co-location
>  -  region replicas don't care about meta co-location?
>
> -Mikhail
>
>
> 2014-06-12 14:07 GMT-07:00 Jonathan Hsieh <jo...@cloudera.com>:
>
> > sgtm.
> >
> >
> > On Mon, Jun 9, 2014 at 10:46 AM, Enis Söztutar <en...@gmail.com>
> wrote:
> >
> > > Thanks Jon.
> > >
> > > I think we'll try the rebase approach first. I'll start the effort
> today
> > > and see how far along I can get with that. Rebasing each patch might
> be a
> > > bit more work actually. If this turns out painful, we'll revert to the
> > > merge approach similar to what you describe.
> > >
> > > Enis
> > >
> > >
> > > On Sat, Jun 7, 2014 at 7:41 AM, Jonathan Hsieh <jo...@cloudera.com>
> wrote:
> > >
> > > > On Fri, Jun 6, 2014 at 5:05 PM, Enis Söztutar <en...@gmail.com>
> > > wrote:
> > > >
> > > > > My preference would be to do the rebase-then-merge style (last
> style
> > in
> > > > > your comment). For each patch, I am hoping for all the changes
> > between
> > > > > committed version and rebased version to be mechanical. I like
> > having a
> > > > > linear history with explicit commit-to-patch mapping.
> > > > >
> > > > > If the changes are of a mechanical nature and that each patch
> > compiles
> > > > and
> > > > passes unit tests along the way, then rebase-then-merge is perfectly
> > fine
> > > > by me.
> > > >
> > > > Even though snapshots were fairly orthogonal to the rest of the
> > codebase,
> > > > when we did merges we had problems maintaining the compiles and
> passes
> > > with
> > > > every commit invariants when we tried rebase-then-merge approach.
> >  After
> > > > each rebase we would end up doing a bunch of work and still end up
> > > failing
> > > > unit tests.   In that case we (jesse, matteo, myself) went to the
> merge
> > > > approach.   We actually merged into the snapshot branch a few times
> > > fixing
> > > > things due to changes in trunk that broke parts of the snapshots
> > branch.
> > > > Here's a more accurate picture of what the commit history looked like
> > in
> > > > git (we dev'ed in git and in end had to recommit this all in svn).
> > > >
> > > > m  (essentially empty merge patch).
> > > > |\
> > > > t |  (trunk patch with no impact)
> > > > | s6* (patch to fix newly introduced problem)
> > > > |/|
> > > > t |
> > > > t*|   (trunk patch that broke snapshots again)
> > > > | s5* (branch bug fixes due to merge)
> > > > | s4* (mechanical fixups due to the merge)
> > > > |/|
> > > > t |
> > > > t |
> > > > t |
> > > > | s3
> > > > | s2
> > > > | s1
> > > > |/
> > > > t
> > > > t
> > > >
> > > > This approach was captured in github and had the added benefit of
> > > > preserving exact history, and having known good points preserving the
> > > > compile/unit test invariants on trunk.  (unfortunately, some of this
> > > > history was lost when we ported the git history over to svn, but that
> > > isn't
> > > > a problem any more).    If you want to see the whole thing, look at
> my
> > > git
> > > > repo:
> > > >
> > > > git remote add jmhsieh git@github.com:jmhsieh/hbase.git
> > > > git log --oneline --graph --color jmhsieh/hbase-7290
> > > >
> > > >
> > > > Can we do history-preserving merge with the first style? I can do the
> > > > > rebases and upload interdiff if that is big so that we can compare
> > on a
> > > > > per-patch basis.
> > > > >
> > > > >
> > > > It is possible.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > > Enis
> > > > >
> > > > >
> > > > > On Fri, Jun 6, 2014 at 4:36 PM, Jonathan Hsieh <jo...@cloudera.com>
> > > wrote:
> > > > >
> > > > > > When we merged snapshots branch in we did this:
> > > > > >
> > > > > > t = trunk commit
> > > > > > s = snapshot branch commit
> > > > > > m = merge point.
> > > > > >
> > > > > > During work:
> > > > > > t
> > > > > > t
> > > > > > t
> > > > > > |  s3
> > > > > > |  s2
> > > > > > |  s1
> > > > > > |/
> > > > > > t
> > > > > > t
> > > > > >
> > > > > > During after merge:
> > > > > > m  (essentially empty merge patch).
> > > > > > t \
> > > > > > t |
> > > > > > t |
> > > > > > | s4* (fixups due to the merge)
> > > > > > | s3
> > > > > > | s2
> > > > > > | s1
> > > > > > |/
> > > > > > t
> > > > > > t
> > > > > >
> > > > > > Does your proposal mean you are going to do this instead?
> > > > > >
> > > > > > s3* (modified due to rebase)
> > > > > > s2* (modified due to rebase)
> > > > > > s1
> > > > > > t
> > > > > > t
> > > > > > t
> > > > > > | s (now dead hbase-10070 branch)
> > > > > > | s
> > > > > > | s
> > > > > > | s
> > > > > > |/
> > > > > > t
> > > > > > t
> > > > > >
> > > > > > If it is the then we should probably have a review of the
> modified
> > > > > patches.
> > > > > >  If it the same as the snapshot merge, then we just need a review
> > of
> > > > the
> > > > > > merge point delta (as well as a full review).
> > > > > >
> > > > > > Personally I prefer the merge for these large feature branches --
> > it
> > > > > > guarantees that each commit is compilable, and reflects what you
> > guys
> > > > > have
> > > > > > been testing for a while.  If you go with the last approach you
> > might
> > > > > have
> > > > > > stuff broken, and in the mainline commit path.
> > > > > >
> > > > > > Jon.
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Tue, Jun 3, 2014 at 11:16 AM, Andrew Purtell <
> > apurtell@apache.org
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > >  
> > > > > > > On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <enis@apache.org
> >
> > > > wrote:
> > > > > > >
> > > > > > > > This VOTE is for merging back the remaining changes in branch
> > to
> > > > > trunk.
> > > > > > > If
> > > > > > > > passes, we will rebase the branch on top of current trunk, in
> > > which
> > > > > we
> > > > > > > will
> > > > > > > > keep the commit-per-issue log history. After that we will do
> a
> > > git
> > > > > > merge
> > > > > > > > for the branch keeping the history clean and not squashing
> the
> > > > > > commits. I
> > > > > > > > expect rebasing to be straightforward, however with some
> manual
> > > > > > conflict
> > > > > > > > resolution. After the merge we'll keep running the tests to
> > make
> > > > sure
> > > > > > > > everything is ok.
> > > > > > > >
> > > > > > >
> > > > > > > Just to clarify that would look something like this:
> > > > > > >
> > > > > > >      $ git checkout HBASE-10070
> > > > > > >      $ git rebase --ignore-date master
> > > > > > >      (fixups, git add, git rebase --continue, etc, etc, etc)
> > > > > > >      $ git checkout master
> > > > > > >      $ git merge HBASE-10070
> > > > > > >
> > > > > > > ?
> > > > > > >
> > > > > > > That sounds good to me, the final merge should be a fast
> forward
> > > > merge.
> > > > > > >
> > > > > > > Use of ' --ignore-date' could be mildly controversial. It's not
> > > > > strictly
> > > > > > > necessary because the commits for 10070 will appear grouped in
> > > > history,
> > > > > > but
> > > > > > > then dates on commits will be discontiguous in that section of
> > > > > history. I
> > > > > > > suggest using that option so the order of commits and dates
> sort
> > > the
> > > > > same
> > > > > > > on master.
> > > > > > >
> > > > > > >
> > > > > > > On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <enis@apache.org
> >
> > > > wrote:
> > > > > > >
> > > > > > > > Hi,
> > > > > > > >
> > > > > > > > Last week we started some discussion[4] for merging branch
> > > > > > hbase-10070[1]
> > > > > > > > into trunk. It seems like the consensus there is to do the
> > merge
> > > > > sooner
> > > > > > > > rather than later.
> > > > > > > >
> > > > > > > >
> > > > > > > > We had branched hbase-10070 in Feb out of trunk[5]. The
> branch
> > > > > contains
> > > > > > > 55
> > > > > > > > jiras committed[2]. Out of these 55, 15 has already been
> > > committed
> > > > to
> > > > > > > trunk
> > > > > > > > and backported to hbase-10070 branch[3].
> > > > > > > >
> > > > > > > > This VOTE is for merging back the remaining changes in branch
> > to
> > > > > trunk.
> > > > > > > If
> > > > > > > > passes, we will rebase the branch on top of current trunk, in
> > > which
> > > > > we
> > > > > > > will
> > > > > > > > keep the commit-per-issue log history. After that we will do
> a
> > > git
> > > > > > merge
> > > > > > > > for the branch keeping the history clean and not squashing
> the
> > > > > > commits. I
> > > > > > > > expect rebasing to be straightforward, however with some
> manual
> > > > > > conflict
> > > > > > > > resolution. After the merge we'll keep running the tests to
> > make
> > > > sure
> > > > > > > > everything is ok.
> > > > > > > >
> > > > > > > > An overview of the changes, and the status of the work can be
> > > found
> > > > > > under
> > > > > > > > [4], [6] and [7].In summary, with the code in branch, you can
> > > > create
> > > > > > > tables
> > > > > > > > with region replicas, do gets / multi gets and scans using
> > > TIMELINE
> > > > > > > > consistency with high availability. Region replicas
> > periodically
> > > > scan
> > > > > > the
> > > > > > > > files of the primary and pick up flushed / committed files.
> The
> > > RPC
> > > > > > > paths /
> > > > > > > > assignment, balancing etc are pretty stable. However some
> more
> > > > > > > performance
> > > > > > > > analysis and tuning is needed. Phase 2 work is being worked
> on
> > > > under
> > > > > > > > HBASE-11183, and we have some working prototype for
> > > > async-replicating
> > > > > > and
> > > > > > > > region splits. However, we believe even without those
> features,
> > > > this
> > > > > > work
> > > > > > > > is useable (especially for read-only/bulk load tables) , and
> > can
> > > be
> > > > > > > > released as an experimental feature in 1.0.
> > > > > > > >
> > > > > > > > Please indicate your choice:
> > > > > > > >
> > > > > > > > [ ] +1 on yes, merge branch hbase-10070 to trunk.
> > > > > > > > [ ] 0 on don't care
> > > > > > > > [ ] -1 don't merge, because ...
> > > > > > > >
> > > > > > > > I'll keep the vote running for 7 days, and close it Mon 9th
> of
> > > > June,
> > > > > > PDT.
> > > > > > > >
> > > > > > > > Here is my official +1.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Enis
> > > > > > > >
> > > > > > > > [1]
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=log;h=refs/heads/hbase-10070
> > > > > > > > [2]
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/browse/HBASE-11214?jql=fixVersion%20%3D%20hbase-10070%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolved
> > > > > > > > [3]
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/browse/HBASE-10792?jql=fixVersion%20%3D%20hbase-10070%20and%20fixversion%20%3D%200.99.0%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolved
> > > > > > > > [4]
> > > > https://www.mail-archive.com/dev@hbase.apache.org/msg25795.html
> > > > > > > > [5]
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hbase/commit/e22c7efeac02efde3451a0c9ff9bdcd2725576d0
> > > > > > > > [6]
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://www.slideshare.net/enissoz/hbase-high-availability-for-reads-with-time
> > > > > > > > [7] https://issues.apache.org/jira/browse/HBASE-10070
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Best regards,
> > > > > > >
> > > > > > >    - Andy
> > > > > > >
> > > > > > > Problems worthy of attack prove their worth by hitting back. -
> > Piet
> > > > > Hein
> > > > > > > (via Tom White)
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > // Jonathan Hsieh (shay)
> > > > > > // HBase Tech Lead, Software Engineer, Cloudera
> > > > > > // jon@cloudera.com // @jmhsieh
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > // Jonathan Hsieh (shay)
> > > > // HBase Tech Lead, Software Engineer, Cloudera
> > > > // jon@cloudera.com // @jmhsieh
> > > >
> > >
> >
> >
> >
> > --
> > // Jonathan Hsieh (shay)
> > // HBase Tech Lead, Software Engineer, Cloudera
> > // jon@cloudera.com // @jmhsieh
> >
>
>
>
> --
> Thanks,
> Michael Antonov
>

Re: [VOTE] Merge branch HBASE-10070 to trunk

Posted by Mikhail Antonov <ol...@gmail.com>.

So how would the defaults and dependencies look amongst these 3, once
committed to 0.99?

 - co-location of meta and master
 - zk-less assignment
- timeline-consistent replicas?

As I understand -
 -  All 3 are off by default
 -  zk-less assignments and region replicas to not be turned on at the same
time
 -  zk-less assignments require meta co-location
 -  region replicas don't care about meta co-location?

-Mikhail


2014-06-12 14:07 GMT-07:00 Jonathan Hsieh <jo...@cloudera.com>:

> sgtm.
>
>
> On Mon, Jun 9, 2014 at 10:46 AM, Enis Söztutar <en...@gmail.com> wrote:
>
> > Thanks Jon.
> >
> > I think we'll try the rebase approach first. I'll start the effort today
> > and see how far along I can get with that. Rebasing each patch might be a
> > bit more work actually. If this turns out painful, we'll revert to the
> > merge approach similar to what you describe.
> >
> > Enis
> >
> >
> > On Sat, Jun 7, 2014 at 7:41 AM, Jonathan Hsieh <jo...@cloudera.com> wrote:
> >
> > > On Fri, Jun 6, 2014 at 5:05 PM, Enis Söztutar <en...@gmail.com>
> > wrote:
> > >
> > > > My preference would be to do the rebase-then-merge style (last style
> in
> > > > your comment). For each patch, I am hoping for all the changes
> between
> > > > committed version and rebased version to be mechanical. I like
> having a
> > > > linear history with explicit commit-to-patch mapping.
> > > >
> > > > If the changes are of a mechanical nature and that each patch
> compiles
> > > and
> > > passes unit tests along the way, then rebase-then-merge is perfectly
> fine
> > > by me.
> > >
> > > Even though snapshots were fairly orthogonal to the rest of the
> codebase,
> > > when we did merges we had problems maintaining the compiles and passes
> > with
> > > every commit invariants when we tried rebase-then-merge approach.
>  After
> > > each rebase we would end up doing a bunch of work and still end up
> > failing
> > > unit tests.   In that case we (jesse, matteo, myself) went to the merge
> > > approach.   We actually merged into the snapshot branch a few times
> > fixing
> > > things due to changes in trunk that broke parts of the snapshots
> branch.
> > > Here's a more accurate picture of what the commit history looked like
> in
> > > git (we dev'ed in git and in end had to recommit this all in svn).
> > >
> > > m  (essentially empty merge patch).
> > > |\
> > > t |  (trunk patch with no impact)
> > > | s6* (patch to fix newly introduced problem)
> > > |/|
> > > t |
> > > t*|   (trunk patch that broke snapshots again)
> > > | s5* (branch bug fixes due to merge)
> > > | s4* (mechanical fixups due to the merge)
> > > |/|
> > > t |
> > > t |
> > > t |
> > > | s3
> > > | s2
> > > | s1
> > > |/
> > > t
> > > t
> > >
> > > This approach was captured in github and had the added benefit of
> > > preserving exact history, and having known good points preserving the
> > > compile/unit test invariants on trunk.  (unfortunately, some of this
> > > history was lost when we ported the git history over to svn, but that
> > isn't
> > > a problem any more).    If you want to see the whole thing, look at my
> > git
> > > repo:
> > >
> > > git remote add jmhsieh git@github.com:jmhsieh/hbase.git
> > > git log --oneline --graph --color jmhsieh/hbase-7290
> > >
> > >
> > > Can we do history-preserving merge with the first style? I can do the
> > > > rebases and upload interdiff if that is big so that we can compare
> on a
> > > > per-patch basis.
> > > >
> > > >
> > > It is possible.
> > >
> > >
> > >
> > >
> > >
> > > > Enis
> > > >
> > > >
> > > > On Fri, Jun 6, 2014 at 4:36 PM, Jonathan Hsieh <jo...@cloudera.com>
> > wrote:
> > > >
> > > > > When we merged snapshots branch in we did this:
> > > > >
> > > > > t = trunk commit
> > > > > s = snapshot branch commit
> > > > > m = merge point.
> > > > >
> > > > > During work:
> > > > > t
> > > > > t
> > > > > t
> > > > > |  s3
> > > > > |  s2
> > > > > |  s1
> > > > > |/
> > > > > t
> > > > > t
> > > > >
> > > > > During after merge:
> > > > > m  (essentially empty merge patch).
> > > > > t \
> > > > > t |
> > > > > t |
> > > > > | s4* (fixups due to the merge)
> > > > > | s3
> > > > > | s2
> > > > > | s1
> > > > > |/
> > > > > t
> > > > > t
> > > > >
> > > > > Does your proposal mean you are going to do this instead?
> > > > >
> > > > > s3* (modified due to rebase)
> > > > > s2* (modified due to rebase)
> > > > > s1
> > > > > t
> > > > > t
> > > > > t
> > > > > | s (now dead hbase-10070 branch)
> > > > > | s
> > > > > | s
> > > > > | s
> > > > > |/
> > > > > t
> > > > > t
> > > > >
> > > > > If it is the then we should probably have a review of the modified
> > > > patches.
> > > > >  If it the same as the snapshot merge, then we just need a review
> of
> > > the
> > > > > merge point delta (as well as a full review).
> > > > >
> > > > > Personally I prefer the merge for these large feature branches --
> it
> > > > > guarantees that each commit is compilable, and reflects what you
> guys
> > > > have
> > > > > been testing for a while.  If you go with the last approach you
> might
> > > > have
> > > > > stuff broken, and in the mainline commit path.
> > > > >
> > > > > Jon.
> > > > >
> > > > >
> > > > >
> > > > > On Tue, Jun 3, 2014 at 11:16 AM, Andrew Purtell <
> apurtell@apache.org
> > >
> > > > > wrote:
> > > > >
> > > > > >  
> > > > > > On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <en...@apache.org>
> > > wrote:
> > > > > >
> > > > > > > This VOTE is for merging back the remaining changes in branch
> to
> > > > trunk.
> > > > > > If
> > > > > > > passes, we will rebase the branch on top of current trunk, in
> > which
> > > > we
> > > > > > will
> > > > > > > keep the commit-per-issue log history. After that we will do a
> > git
> > > > > merge
> > > > > > > for the branch keeping the history clean and not squashing the
> > > > > commits. I
> > > > > > > expect rebasing to be straightforward, however with some manual
> > > > > conflict
> > > > > > > resolution. After the merge we'll keep running the tests to
> make
> > > sure
> > > > > > > everything is ok.
> > > > > > >
> > > > > >
> > > > > > Just to clarify that would look something like this:
> > > > > >
> > > > > >      $ git checkout HBASE-10070
> > > > > >      $ git rebase --ignore-date master
> > > > > >      (fixups, git add, git rebase --continue, etc, etc, etc)
> > > > > >      $ git checkout master
> > > > > >      $ git merge HBASE-10070
> > > > > >
> > > > > > ?
> > > > > >
> > > > > > That sounds good to me, the final merge should be a fast forward
> > > merge.
> > > > > >
> > > > > > Use of ' --ignore-date' could be mildly controversial. It's not
> > > > strictly
> > > > > > necessary because the commits for 10070 will appear grouped in
> > > history,
> > > > > but
> > > > > > then dates on commits will be discontiguous in that section of
> > > > history. I
> > > > > > suggest using that option so the order of commits and dates sort
> > the
> > > > same
> > > > > > on master.
> > > > > >
> > > > > >
> > > > > > On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <en...@apache.org>
> > > wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > Last week we started some discussion[4] for merging branch
> > > > > hbase-10070[1]
> > > > > > > into trunk. It seems like the consensus there is to do the
> merge
> > > > sooner
> > > > > > > rather than later.
> > > > > > >
> > > > > > >
> > > > > > > We had branched hbase-10070 in Feb out of trunk[5]. The branch
> > > > contains
> > > > > > 55
> > > > > > > jiras committed[2]. Out of these 55, 15 has already been
> > committed
> > > to
> > > > > > trunk
> > > > > > > and backported to hbase-10070 branch[3].
> > > > > > >
> > > > > > > This VOTE is for merging back the remaining changes in branch
> to
> > > > trunk.
> > > > > > If
> > > > > > > passes, we will rebase the branch on top of current trunk, in
> > which
> > > > we
> > > > > > will
> > > > > > > keep the commit-per-issue log history. After that we will do a
> > git
> > > > > merge
> > > > > > > for the branch keeping the history clean and not squashing the
> > > > > commits. I
> > > > > > > expect rebasing to be straightforward, however with some manual
> > > > > conflict
> > > > > > > resolution. After the merge we'll keep running the tests to
> make
> > > sure
> > > > > > > everything is ok.
> > > > > > >
> > > > > > > An overview of the changes, and the status of the work can be
> > found
> > > > > under
> > > > > > > [4], [6] and [7].In summary, with the code in branch, you can
> > > create
> > > > > > tables
> > > > > > > with region replicas, do gets / multi gets and scans using
> > TIMELINE
> > > > > > > consistency with high availability. Region replicas
> periodically
> > > scan
> > > > > the
> > > > > > > files of the primary and pick up flushed / committed files. The
> > RPC
> > > > > > paths /
> > > > > > > assignment, balancing etc are pretty stable. However some more
> > > > > > performance
> > > > > > > analysis and tuning is needed. Phase 2 work is being worked on
> > > under
> > > > > > > HBASE-11183, and we have some working prototype for
> > > async-replicating
> > > > > and
> > > > > > > region splits. However, we believe even without those features,
> > > this
> > > > > work
> > > > > > > is useable (especially for read-only/bulk load tables) , and
> can
> > be
> > > > > > > released as an experimental feature in 1.0.
> > > > > > >
> > > > > > > Please indicate your choice:
> > > > > > >
> > > > > > > [ ] +1 on yes, merge branch hbase-10070 to trunk.
> > > > > > > [ ] 0 on don't care
> > > > > > > [ ] -1 don't merge, because ...
> > > > > > >
> > > > > > > I'll keep the vote running for 7 days, and close it Mon 9th of
> > > June,
> > > > > PDT.
> > > > > > >
> > > > > > > Here is my official +1.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Enis
> > > > > > >
> > > > > > > [1]
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=log;h=refs/heads/hbase-10070
> > > > > > > [2]
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/browse/HBASE-11214?jql=fixVersion%20%3D%20hbase-10070%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolved
> > > > > > > [3]
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/browse/HBASE-10792?jql=fixVersion%20%3D%20hbase-10070%20and%20fixversion%20%3D%200.99.0%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolved
> > > > > > > [4]
> > > https://www.mail-archive.com/dev@hbase.apache.org/msg25795.html
> > > > > > > [5]
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hbase/commit/e22c7efeac02efde3451a0c9ff9bdcd2725576d0
> > > > > > > [6]
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://www.slideshare.net/enissoz/hbase-high-availability-for-reads-with-time
> > > > > > > [7] https://issues.apache.org/jira/browse/HBASE-10070
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Best regards,
> > > > > >
> > > > > >    - Andy
> > > > > >
> > > > > > Problems worthy of attack prove their worth by hitting back. -
> Piet
> > > > Hein
> > > > > > (via Tom White)
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > // Jonathan Hsieh (shay)
> > > > > // HBase Tech Lead, Software Engineer, Cloudera
> > > > > // jon@cloudera.com // @jmhsieh
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > // Jonathan Hsieh (shay)
> > > // HBase Tech Lead, Software Engineer, Cloudera
> > > // jon@cloudera.com // @jmhsieh
> > >
> >
>
>
>
> --
> // Jonathan Hsieh (shay)
> // HBase Tech Lead, Software Engineer, Cloudera
> // jon@cloudera.com // @jmhsieh
>



-- 
Thanks,
Michael Antonov

Re: [VOTE] Merge branch HBASE-10070 to trunk

Posted by Jonathan Hsieh <jo...@cloudera.com>.

sgtm.


On Mon, Jun 9, 2014 at 10:46 AM, Enis Söztutar <en...@gmail.com> wrote:

> Thanks Jon.
>
> I think we'll try the rebase approach first. I'll start the effort today
> and see how far along I can get with that. Rebasing each patch might be a
> bit more work actually. If this turns out painful, we'll revert to the
> merge approach similar to what you describe.
>
> Enis
>
>
> On Sat, Jun 7, 2014 at 7:41 AM, Jonathan Hsieh <jo...@cloudera.com> wrote:
>
> > On Fri, Jun 6, 2014 at 5:05 PM, Enis Söztutar <en...@gmail.com>
> wrote:
> >
> > > My preference would be to do the rebase-then-merge style (last style in
> > > your comment). For each patch, I am hoping for all the changes between
> > > committed version and rebased version to be mechanical. I like having a
> > > linear history with explicit commit-to-patch mapping.
> > >
> > > If the changes are of a mechanical nature and that each patch compiles
> > and
> > passes unit tests along the way, then rebase-then-merge is perfectly fine
> > by me.
> >
> > Even though snapshots were fairly orthogonal to the rest of the codebase,
> > when we did merges we had problems maintaining the compiles and passes
> with
> > every commit invariants when we tried rebase-then-merge approach.  After
> > each rebase we would end up doing a bunch of work and still end up
> failing
> > unit tests.   In that case we (jesse, matteo, myself) went to the merge
> > approach.   We actually merged into the snapshot branch a few times
> fixing
> > things due to changes in trunk that broke parts of the snapshots branch.
> > Here's a more accurate picture of what the commit history looked like in
> > git (we dev'ed in git and in end had to recommit this all in svn).
> >
> > m  (essentially empty merge patch).
> > |\
> > t |  (trunk patch with no impact)
> > | s6* (patch to fix newly introduced problem)
> > |/|
> > t |
> > t*|   (trunk patch that broke snapshots again)
> > | s5* (branch bug fixes due to merge)
> > | s4* (mechanical fixups due to the merge)
> > |/|
> > t |
> > t |
> > t |
> > | s3
> > | s2
> > | s1
> > |/
> > t
> > t
> >
> > This approach was captured in github and had the added benefit of
> > preserving exact history, and having known good points preserving the
> > compile/unit test invariants on trunk.  (unfortunately, some of this
> > history was lost when we ported the git history over to svn, but that
> isn't
> > a problem any more).    If you want to see the whole thing, look at my
> git
> > repo:
> >
> > git remote add jmhsieh git@github.com:jmhsieh/hbase.git
> > git log --oneline --graph --color jmhsieh/hbase-7290
> >
> >
> > Can we do history-preserving merge with the first style? I can do the
> > > rebases and upload interdiff if that is big so that we can compare on a
> > > per-patch basis.
> > >
> > >
> > It is possible.
> >
> >
> >
> >
> >
> > > Enis
> > >
> > >
> > > On Fri, Jun 6, 2014 at 4:36 PM, Jonathan Hsieh <jo...@cloudera.com>
> wrote:
> > >
> > > > When we merged snapshots branch in we did this:
> > > >
> > > > t = trunk commit
> > > > s = snapshot branch commit
> > > > m = merge point.
> > > >
> > > > During work:
> > > > t
> > > > t
> > > > t
> > > > |  s3
> > > > |  s2
> > > > |  s1
> > > > |/
> > > > t
> > > > t
> > > >
> > > > During after merge:
> > > > m  (essentially empty merge patch).
> > > > t \
> > > > t |
> > > > t |
> > > > | s4* (fixups due to the merge)
> > > > | s3
> > > > | s2
> > > > | s1
> > > > |/
> > > > t
> > > > t
> > > >
> > > > Does your proposal mean you are going to do this instead?
> > > >
> > > > s3* (modified due to rebase)
> > > > s2* (modified due to rebase)
> > > > s1
> > > > t
> > > > t
> > > > t
> > > > | s (now dead hbase-10070 branch)
> > > > | s
> > > > | s
> > > > | s
> > > > |/
> > > > t
> > > > t
> > > >
> > > > If it is the then we should probably have a review of the modified
> > > patches.
> > > >  If it the same as the snapshot merge, then we just need a review of
> > the
> > > > merge point delta (as well as a full review).
> > > >
> > > > Personally I prefer the merge for these large feature branches -- it
> > > > guarantees that each commit is compilable, and reflects what you guys
> > > have
> > > > been testing for a while.  If you go with the last approach you might
> > > have
> > > > stuff broken, and in the mainline commit path.
> > > >
> > > > Jon.
> > > >
> > > >
> > > >
> > > > On Tue, Jun 3, 2014 at 11:16 AM, Andrew Purtell <apurtell@apache.org
> >
> > > > wrote:
> > > >
> > > > >  
> > > > > On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <en...@apache.org>
> > wrote:
> > > > >
> > > > > > This VOTE is for merging back the remaining changes in branch to
> > > trunk.
> > > > > If
> > > > > > passes, we will rebase the branch on top of current trunk, in
> which
> > > we
> > > > > will
> > > > > > keep the commit-per-issue log history. After that we will do a
> git
> > > > merge
> > > > > > for the branch keeping the history clean and not squashing the
> > > > commits. I
> > > > > > expect rebasing to be straightforward, however with some manual
> > > > conflict
> > > > > > resolution. After the merge we'll keep running the tests to make
> > sure
> > > > > > everything is ok.
> > > > > >
> > > > >
> > > > > Just to clarify that would look something like this:
> > > > >
> > > > >      $ git checkout HBASE-10070
> > > > >      $ git rebase --ignore-date master
> > > > >      (fixups, git add, git rebase --continue, etc, etc, etc)
> > > > >      $ git checkout master
> > > > >      $ git merge HBASE-10070
> > > > >
> > > > > ?
> > > > >
> > > > > That sounds good to me, the final merge should be a fast forward
> > merge.
> > > > >
> > > > > Use of ' --ignore-date' could be mildly controversial. It's not
> > > strictly
> > > > > necessary because the commits for 10070 will appear grouped in
> > history,
> > > > but
> > > > > then dates on commits will be discontiguous in that section of
> > > history. I
> > > > > suggest using that option so the order of commits and dates sort
> the
> > > same
> > > > > on master.
> > > > >
> > > > >
> > > > > On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <en...@apache.org>
> > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > Last week we started some discussion[4] for merging branch
> > > > hbase-10070[1]
> > > > > > into trunk. It seems like the consensus there is to do the merge
> > > sooner
> > > > > > rather than later.
> > > > > >
> > > > > >
> > > > > > We had branched hbase-10070 in Feb out of trunk[5]. The branch
> > > contains
> > > > > 55
> > > > > > jiras committed[2]. Out of these 55, 15 has already been
> committed
> > to
> > > > > trunk
> > > > > > and backported to hbase-10070 branch[3].
> > > > > >
> > > > > > This VOTE is for merging back the remaining changes in branch to
> > > trunk.
> > > > > If
> > > > > > passes, we will rebase the branch on top of current trunk, in
> which
> > > we
> > > > > will
> > > > > > keep the commit-per-issue log history. After that we will do a
> git
> > > > merge
> > > > > > for the branch keeping the history clean and not squashing the
> > > > commits. I
> > > > > > expect rebasing to be straightforward, however with some manual
> > > > conflict
> > > > > > resolution. After the merge we'll keep running the tests to make
> > sure
> > > > > > everything is ok.
> > > > > >
> > > > > > An overview of the changes, and the status of the work can be
> found
> > > > under
> > > > > > [4], [6] and [7].In summary, with the code in branch, you can
> > create
> > > > > tables
> > > > > > with region replicas, do gets / multi gets and scans using
> TIMELINE
> > > > > > consistency with high availability. Region replicas periodically
> > scan
> > > > the
> > > > > > files of the primary and pick up flushed / committed files. The
> RPC
> > > > > paths /
> > > > > > assignment, balancing etc are pretty stable. However some more
> > > > > performance
> > > > > > analysis and tuning is needed. Phase 2 work is being worked on
> > under
> > > > > > HBASE-11183, and we have some working prototype for
> > async-replicating
> > > > and
> > > > > > region splits. However, we believe even without those features,
> > this
> > > > work
> > > > > > is useable (especially for read-only/bulk load tables) , and can
> be
> > > > > > released as an experimental feature in 1.0.
> > > > > >
> > > > > > Please indicate your choice:
> > > > > >
> > > > > > [ ] +1 on yes, merge branch hbase-10070 to trunk.
> > > > > > [ ] 0 on don't care
> > > > > > [ ] -1 don't merge, because ...
> > > > > >
> > > > > > I'll keep the vote running for 7 days, and close it Mon 9th of
> > June,
> > > > PDT.
> > > > > >
> > > > > > Here is my official +1.
> > > > > >
> > > > > > Thanks,
> > > > > > Enis
> > > > > >
> > > > > > [1]
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=log;h=refs/heads/hbase-10070
> > > > > > [2]
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/browse/HBASE-11214?jql=fixVersion%20%3D%20hbase-10070%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolved
> > > > > > [3]
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/browse/HBASE-10792?jql=fixVersion%20%3D%20hbase-10070%20and%20fixversion%20%3D%200.99.0%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolved
> > > > > > [4]
> > https://www.mail-archive.com/dev@hbase.apache.org/msg25795.html
> > > > > > [5]
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hbase/commit/e22c7efeac02efde3451a0c9ff9bdcd2725576d0
> > > > > > [6]
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://www.slideshare.net/enissoz/hbase-high-availability-for-reads-with-time
> > > > > > [7] https://issues.apache.org/jira/browse/HBASE-10070
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Best regards,
> > > > >
> > > > >    - Andy
> > > > >
> > > > > Problems worthy of attack prove their worth by hitting back. - Piet
> > > Hein
> > > > > (via Tom White)
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > // Jonathan Hsieh (shay)
> > > > // HBase Tech Lead, Software Engineer, Cloudera
> > > > // jon@cloudera.com // @jmhsieh
> > > >
> > >
> >
> >
> >
> > --
> > // Jonathan Hsieh (shay)
> > // HBase Tech Lead, Software Engineer, Cloudera
> > // jon@cloudera.com // @jmhsieh
> >
>



-- 
// Jonathan Hsieh (shay)
// HBase Tech Lead, Software Engineer, Cloudera
// jon@cloudera.com // @jmhsieh

Re: [VOTE] Merge branch HBASE-10070 to trunk

Posted by Nicolas Liochon <nk...@gmail.com>.

+1 for the integration, whatever the solution in git.


On Mon, Jun 9, 2014 at 7:46 PM, Enis Söztutar <en...@gmail.com> wrote:

> Thanks Jon.
>
> I think we'll try the rebase approach first. I'll start the effort today
> and see how far along I can get with that. Rebasing each patch might be a
> bit more work actually. If this turns out painful, we'll revert to the
> merge approach similar to what you describe.
>
> Enis
>
>
> On Sat, Jun 7, 2014 at 7:41 AM, Jonathan Hsieh <jo...@cloudera.com> wrote:
>
> > On Fri, Jun 6, 2014 at 5:05 PM, Enis Söztutar <en...@gmail.com>
> wrote:
> >
> > > My preference would be to do the rebase-then-merge style (last style in
> > > your comment). For each patch, I am hoping for all the changes between
> > > committed version and rebased version to be mechanical. I like having a
> > > linear history with explicit commit-to-patch mapping.
> > >
> > > If the changes are of a mechanical nature and that each patch compiles
> > and
> > passes unit tests along the way, then rebase-then-merge is perfectly fine
> > by me.
> >
> > Even though snapshots were fairly orthogonal to the rest of the codebase,
> > when we did merges we had problems maintaining the compiles and passes
> with
> > every commit invariants when we tried rebase-then-merge approach.  After
> > each rebase we would end up doing a bunch of work and still end up
> failing
> > unit tests.   In that case we (jesse, matteo, myself) went to the merge
> > approach.   We actually merged into the snapshot branch a few times
> fixing
> > things due to changes in trunk that broke parts of the snapshots branch.
> > Here's a more accurate picture of what the commit history looked like in
> > git (we dev'ed in git and in end had to recommit this all in svn).
> >
> > m  (essentially empty merge patch).
> > |\
> > t |  (trunk patch with no impact)
> > | s6* (patch to fix newly introduced problem)
> > |/|
> > t |
> > t*|   (trunk patch that broke snapshots again)
> > | s5* (branch bug fixes due to merge)
> > | s4* (mechanical fixups due to the merge)
> > |/|
> > t |
> > t |
> > t |
> > | s3
> > | s2
> > | s1
> > |/
> > t
> > t
> >
> > This approach was captured in github and had the added benefit of
> > preserving exact history, and having known good points preserving the
> > compile/unit test invariants on trunk.  (unfortunately, some of this
> > history was lost when we ported the git history over to svn, but that
> isn't
> > a problem any more).    If you want to see the whole thing, look at my
> git
> > repo:
> >
> > git remote add jmhsieh git@github.com:jmhsieh/hbase.git
> > git log --oneline --graph --color jmhsieh/hbase-7290
> >
> >
> > Can we do history-preserving merge with the first style? I can do the
> > > rebases and upload interdiff if that is big so that we can compare on a
> > > per-patch basis.
> > >
> > >
> > It is possible.
> >
> >
> >
> >
> >
> > > Enis
> > >
> > >
> > > On Fri, Jun 6, 2014 at 4:36 PM, Jonathan Hsieh <jo...@cloudera.com>
> wrote:
> > >
> > > > When we merged snapshots branch in we did this:
> > > >
> > > > t = trunk commit
> > > > s = snapshot branch commit
> > > > m = merge point.
> > > >
> > > > During work:
> > > > t
> > > > t
> > > > t
> > > > |  s3
> > > > |  s2
> > > > |  s1
> > > > |/
> > > > t
> > > > t
> > > >
> > > > During after merge:
> > > > m  (essentially empty merge patch).
> > > > t \
> > > > t |
> > > > t |
> > > > | s4* (fixups due to the merge)
> > > > | s3
> > > > | s2
> > > > | s1
> > > > |/
> > > > t
> > > > t
> > > >
> > > > Does your proposal mean you are going to do this instead?
> > > >
> > > > s3* (modified due to rebase)
> > > > s2* (modified due to rebase)
> > > > s1
> > > > t
> > > > t
> > > > t
> > > > | s (now dead hbase-10070 branch)
> > > > | s
> > > > | s
> > > > | s
> > > > |/
> > > > t
> > > > t
> > > >
> > > > If it is the then we should probably have a review of the modified
> > > patches.
> > > >  If it the same as the snapshot merge, then we just need a review of
> > the
> > > > merge point delta (as well as a full review).
> > > >
> > > > Personally I prefer the merge for these large feature branches -- it
> > > > guarantees that each commit is compilable, and reflects what you guys
> > > have
> > > > been testing for a while.  If you go with the last approach you might
> > > have
> > > > stuff broken, and in the mainline commit path.
> > > >
> > > > Jon.
> > > >
> > > >
> > > >
> > > > On Tue, Jun 3, 2014 at 11:16 AM, Andrew Purtell <apurtell@apache.org
> >
> > > > wrote:
> > > >
> > > > >  
> > > > > On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <en...@apache.org>
> > wrote:
> > > > >
> > > > > > This VOTE is for merging back the remaining changes in branch to
> > > trunk.
> > > > > If
> > > > > > passes, we will rebase the branch on top of current trunk, in
> which
> > > we
> > > > > will
> > > > > > keep the commit-per-issue log history. After that we will do a
> git
> > > > merge
> > > > > > for the branch keeping the history clean and not squashing the
> > > > commits. I
> > > > > > expect rebasing to be straightforward, however with some manual
> > > > conflict
> > > > > > resolution. After the merge we'll keep running the tests to make
> > sure
> > > > > > everything is ok.
> > > > > >
> > > > >
> > > > > Just to clarify that would look something like this:
> > > > >
> > > > >      $ git checkout HBASE-10070
> > > > >      $ git rebase --ignore-date master
> > > > >      (fixups, git add, git rebase --continue, etc, etc, etc)
> > > > >      $ git checkout master
> > > > >      $ git merge HBASE-10070
> > > > >
> > > > > ?
> > > > >
> > > > > That sounds good to me, the final merge should be a fast forward
> > merge.
> > > > >
> > > > > Use of ' --ignore-date' could be mildly controversial. It's not
> > > strictly
> > > > > necessary because the commits for 10070 will appear grouped in
> > history,
> > > > but
> > > > > then dates on commits will be discontiguous in that section of
> > > history. I
> > > > > suggest using that option so the order of commits and dates sort
> the
> > > same
> > > > > on master.
> > > > >
> > > > >
> > > > > On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <en...@apache.org>
> > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > Last week we started some discussion[4] for merging branch
> > > > hbase-10070[1]
> > > > > > into trunk. It seems like the consensus there is to do the merge
> > > sooner
> > > > > > rather than later.
> > > > > >
> > > > > >
> > > > > > We had branched hbase-10070 in Feb out of trunk[5]. The branch
> > > contains
> > > > > 55
> > > > > > jiras committed[2]. Out of these 55, 15 has already been
> committed
> > to
> > > > > trunk
> > > > > > and backported to hbase-10070 branch[3].
> > > > > >
> > > > > > This VOTE is for merging back the remaining changes in branch to
> > > trunk.
> > > > > If
> > > > > > passes, we will rebase the branch on top of current trunk, in
> which
> > > we
> > > > > will
> > > > > > keep the commit-per-issue log history. After that we will do a
> git
> > > > merge
> > > > > > for the branch keeping the history clean and not squashing the
> > > > commits. I
> > > > > > expect rebasing to be straightforward, however with some manual
> > > > conflict
> > > > > > resolution. After the merge we'll keep running the tests to make
> > sure
> > > > > > everything is ok.
> > > > > >
> > > > > > An overview of the changes, and the status of the work can be
> found
> > > > under
> > > > > > [4], [6] and [7].In summary, with the code in branch, you can
> > create
> > > > > tables
> > > > > > with region replicas, do gets / multi gets and scans using
> TIMELINE
> > > > > > consistency with high availability. Region replicas periodically
> > scan
> > > > the
> > > > > > files of the primary and pick up flushed / committed files. The
> RPC
> > > > > paths /
> > > > > > assignment, balancing etc are pretty stable. However some more
> > > > > performance
> > > > > > analysis and tuning is needed. Phase 2 work is being worked on
> > under
> > > > > > HBASE-11183, and we have some working prototype for
> > async-replicating
> > > > and
> > > > > > region splits. However, we believe even without those features,
> > this
> > > > work
> > > > > > is useable (especially for read-only/bulk load tables) , and can
> be
> > > > > > released as an experimental feature in 1.0.
> > > > > >
> > > > > > Please indicate your choice:
> > > > > >
> > > > > > [ ] +1 on yes, merge branch hbase-10070 to trunk.
> > > > > > [ ] 0 on don't care
> > > > > > [ ] -1 don't merge, because ...
> > > > > >
> > > > > > I'll keep the vote running for 7 days, and close it Mon 9th of
> > June,
> > > > PDT.
> > > > > >
> > > > > > Here is my official +1.
> > > > > >
> > > > > > Thanks,
> > > > > > Enis
> > > > > >
> > > > > > [1]
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=log;h=refs/heads/hbase-10070
> > > > > > [2]
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/browse/HBASE-11214?jql=fixVersion%20%3D%20hbase-10070%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolved
> > > > > > [3]
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/browse/HBASE-10792?jql=fixVersion%20%3D%20hbase-10070%20and%20fixversion%20%3D%200.99.0%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolved
> > > > > > [4]
> > https://www.mail-archive.com/dev@hbase.apache.org/msg25795.html
> > > > > > [5]
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hbase/commit/e22c7efeac02efde3451a0c9ff9bdcd2725576d0
> > > > > > [6]
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://www.slideshare.net/enissoz/hbase-high-availability-for-reads-with-time
> > > > > > [7] https://issues.apache.org/jira/browse/HBASE-10070
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Best regards,
> > > > >
> > > > >    - Andy
> > > > >
> > > > > Problems worthy of attack prove their worth by hitting back. - Piet
> > > Hein
> > > > > (via Tom White)
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > // Jonathan Hsieh (shay)
> > > > // HBase Tech Lead, Software Engineer, Cloudera
> > > > // jon@cloudera.com // @jmhsieh
> > > >
> > >
> >
> >
> >
> > --
> > // Jonathan Hsieh (shay)
> > // HBase Tech Lead, Software Engineer, Cloudera
> > // jon@cloudera.com // @jmhsieh
> >
>

Re: [VOTE] Merge branch HBASE-10070 to trunk

Posted by Enis Söztutar <en...@gmail.com>.

Thanks Jon.

I think we'll try the rebase approach first. I'll start the effort today
and see how far along I can get with that. Rebasing each patch might be a
bit more work actually. If this turns out painful, we'll revert to the
merge approach similar to what you describe.

Enis


On Sat, Jun 7, 2014 at 7:41 AM, Jonathan Hsieh <jo...@cloudera.com> wrote:

> On Fri, Jun 6, 2014 at 5:05 PM, Enis Söztutar <en...@gmail.com> wrote:
>
> > My preference would be to do the rebase-then-merge style (last style in
> > your comment). For each patch, I am hoping for all the changes between
> > committed version and rebased version to be mechanical. I like having a
> > linear history with explicit commit-to-patch mapping.
> >
> > If the changes are of a mechanical nature and that each patch compiles
> and
> passes unit tests along the way, then rebase-then-merge is perfectly fine
> by me.
>
> Even though snapshots were fairly orthogonal to the rest of the codebase,
> when we did merges we had problems maintaining the compiles and passes with
> every commit invariants when we tried rebase-then-merge approach.  After
> each rebase we would end up doing a bunch of work and still end up failing
> unit tests.   In that case we (jesse, matteo, myself) went to the merge
> approach.   We actually merged into the snapshot branch a few times fixing
> things due to changes in trunk that broke parts of the snapshots branch.
> Here's a more accurate picture of what the commit history looked like in
> git (we dev'ed in git and in end had to recommit this all in svn).
>
> m  (essentially empty merge patch).
> |\
> t |  (trunk patch with no impact)
> | s6* (patch to fix newly introduced problem)
> |/|
> t |
> t*|   (trunk patch that broke snapshots again)
> | s5* (branch bug fixes due to merge)
> | s4* (mechanical fixups due to the merge)
> |/|
> t |
> t |
> t |
> | s3
> | s2
> | s1
> |/
> t
> t
>
> This approach was captured in github and had the added benefit of
> preserving exact history, and having known good points preserving the
> compile/unit test invariants on trunk.  (unfortunately, some of this
> history was lost when we ported the git history over to svn, but that isn't
> a problem any more).    If you want to see the whole thing, look at my git
> repo:
>
> git remote add jmhsieh git@github.com:jmhsieh/hbase.git
> git log --oneline --graph --color jmhsieh/hbase-7290
>
>
> Can we do history-preserving merge with the first style? I can do the
> > rebases and upload interdiff if that is big so that we can compare on a
> > per-patch basis.
> >
> >
> It is possible.
>
>
>
>
>
> > Enis
> >
> >
> > On Fri, Jun 6, 2014 at 4:36 PM, Jonathan Hsieh <jo...@cloudera.com> wrote:
> >
> > > When we merged snapshots branch in we did this:
> > >
> > > t = trunk commit
> > > s = snapshot branch commit
> > > m = merge point.
> > >
> > > During work:
> > > t
> > > t
> > > t
> > > |  s3
> > > |  s2
> > > |  s1
> > > |/
> > > t
> > > t
> > >
> > > During after merge:
> > > m  (essentially empty merge patch).
> > > t \
> > > t |
> > > t |
> > > | s4* (fixups due to the merge)
> > > | s3
> > > | s2
> > > | s1
> > > |/
> > > t
> > > t
> > >
> > > Does your proposal mean you are going to do this instead?
> > >
> > > s3* (modified due to rebase)
> > > s2* (modified due to rebase)
> > > s1
> > > t
> > > t
> > > t
> > > | s (now dead hbase-10070 branch)
> > > | s
> > > | s
> > > | s
> > > |/
> > > t
> > > t
> > >
> > > If it is the then we should probably have a review of the modified
> > patches.
> > >  If it the same as the snapshot merge, then we just need a review of
> the
> > > merge point delta (as well as a full review).
> > >
> > > Personally I prefer the merge for these large feature branches -- it
> > > guarantees that each commit is compilable, and reflects what you guys
> > have
> > > been testing for a while.  If you go with the last approach you might
> > have
> > > stuff broken, and in the mainline commit path.
> > >
> > > Jon.
> > >
> > >
> > >
> > > On Tue, Jun 3, 2014 at 11:16 AM, Andrew Purtell <ap...@apache.org>
> > > wrote:
> > >
> > > >  
> > > > On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <en...@apache.org>
> wrote:
> > > >
> > > > > This VOTE is for merging back the remaining changes in branch to
> > trunk.
> > > > If
> > > > > passes, we will rebase the branch on top of current trunk, in which
> > we
> > > > will
> > > > > keep the commit-per-issue log history. After that we will do a git
> > > merge
> > > > > for the branch keeping the history clean and not squashing the
> > > commits. I
> > > > > expect rebasing to be straightforward, however with some manual
> > > conflict
> > > > > resolution. After the merge we'll keep running the tests to make
> sure
> > > > > everything is ok.
> > > > >
> > > >
> > > > Just to clarify that would look something like this:
> > > >
> > > >      $ git checkout HBASE-10070
> > > >      $ git rebase --ignore-date master
> > > >      (fixups, git add, git rebase --continue, etc, etc, etc)
> > > >      $ git checkout master
> > > >      $ git merge HBASE-10070
> > > >
> > > > ?
> > > >
> > > > That sounds good to me, the final merge should be a fast forward
> merge.
> > > >
> > > > Use of ' --ignore-date' could be mildly controversial. It's not
> > strictly
> > > > necessary because the commits for 10070 will appear grouped in
> history,
> > > but
> > > > then dates on commits will be discontiguous in that section of
> > history. I
> > > > suggest using that option so the order of commits and dates sort the
> > same
> > > > on master.
> > > >
> > > >
> > > > On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <en...@apache.org>
> wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > Last week we started some discussion[4] for merging branch
> > > hbase-10070[1]
> > > > > into trunk. It seems like the consensus there is to do the merge
> > sooner
> > > > > rather than later.
> > > > >
> > > > >
> > > > > We had branched hbase-10070 in Feb out of trunk[5]. The branch
> > contains
> > > > 55
> > > > > jiras committed[2]. Out of these 55, 15 has already been committed
> to
> > > > trunk
> > > > > and backported to hbase-10070 branch[3].
> > > > >
> > > > > This VOTE is for merging back the remaining changes in branch to
> > trunk.
> > > > If
> > > > > passes, we will rebase the branch on top of current trunk, in which
> > we
> > > > will
> > > > > keep the commit-per-issue log history. After that we will do a git
> > > merge
> > > > > for the branch keeping the history clean and not squashing the
> > > commits. I
> > > > > expect rebasing to be straightforward, however with some manual
> > > conflict
> > > > > resolution. After the merge we'll keep running the tests to make
> sure
> > > > > everything is ok.
> > > > >
> > > > > An overview of the changes, and the status of the work can be found
> > > under
> > > > > [4], [6] and [7].In summary, with the code in branch, you can
> create
> > > > tables
> > > > > with region replicas, do gets / multi gets and scans using TIMELINE
> > > > > consistency with high availability. Region replicas periodically
> scan
> > > the
> > > > > files of the primary and pick up flushed / committed files. The RPC
> > > > paths /
> > > > > assignment, balancing etc are pretty stable. However some more
> > > > performance
> > > > > analysis and tuning is needed. Phase 2 work is being worked on
> under
> > > > > HBASE-11183, and we have some working prototype for
> async-replicating
> > > and
> > > > > region splits. However, we believe even without those features,
> this
> > > work
> > > > > is useable (especially for read-only/bulk load tables) , and can be
> > > > > released as an experimental feature in 1.0.
> > > > >
> > > > > Please indicate your choice:
> > > > >
> > > > > [ ] +1 on yes, merge branch hbase-10070 to trunk.
> > > > > [ ] 0 on don't care
> > > > > [ ] -1 don't merge, because ...
> > > > >
> > > > > I'll keep the vote running for 7 days, and close it Mon 9th of
> June,
> > > PDT.
> > > > >
> > > > > Here is my official +1.
> > > > >
> > > > > Thanks,
> > > > > Enis
> > > > >
> > > > > [1]
> > > > >
> > > > >
> > > >
> > >
> >
> https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=log;h=refs/heads/hbase-10070
> > > > > [2]
> > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/browse/HBASE-11214?jql=fixVersion%20%3D%20hbase-10070%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolved
> > > > > [3]
> > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/browse/HBASE-10792?jql=fixVersion%20%3D%20hbase-10070%20and%20fixversion%20%3D%200.99.0%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolved
> > > > > [4]
> https://www.mail-archive.com/dev@hbase.apache.org/msg25795.html
> > > > > [5]
> > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hbase/commit/e22c7efeac02efde3451a0c9ff9bdcd2725576d0
> > > > > [6]
> > > > >
> > > > >
> > > >
> > >
> >
> http://www.slideshare.net/enissoz/hbase-high-availability-for-reads-with-time
> > > > > [7] https://issues.apache.org/jira/browse/HBASE-10070
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > >
> > > >    - Andy
> > > >
> > > > Problems worthy of attack prove their worth by hitting back. - Piet
> > Hein
> > > > (via Tom White)
> > > >
> > >
> > >
> > >
> > > --
> > > // Jonathan Hsieh (shay)
> > > // HBase Tech Lead, Software Engineer, Cloudera
> > > // jon@cloudera.com // @jmhsieh
> > >
> >
>
>
>
> --
> // Jonathan Hsieh (shay)
> // HBase Tech Lead, Software Engineer, Cloudera
> // jon@cloudera.com // @jmhsieh
>

Re: [VOTE] Merge branch HBASE-10070 to trunk

Posted by Jonathan Hsieh <jo...@cloudera.com>.

On Fri, Jun 6, 2014 at 5:05 PM, Enis Söztutar <en...@gmail.com> wrote:

> My preference would be to do the rebase-then-merge style (last style in
> your comment). For each patch, I am hoping for all the changes between
> committed version and rebased version to be mechanical. I like having a
> linear history with explicit commit-to-patch mapping.
>
> If the changes are of a mechanical nature and that each patch compiles and
passes unit tests along the way, then rebase-then-merge is perfectly fine
by me.

Even though snapshots were fairly orthogonal to the rest of the codebase,
when we did merges we had problems maintaining the compiles and passes with
every commit invariants when we tried rebase-then-merge approach.  After
each rebase we would end up doing a bunch of work and still end up failing
unit tests.   In that case we (jesse, matteo, myself) went to the merge
approach.   We actually merged into the snapshot branch a few times fixing
things due to changes in trunk that broke parts of the snapshots branch.
Here's a more accurate picture of what the commit history looked like in
git (we dev'ed in git and in end had to recommit this all in svn).

m  (essentially empty merge patch).
|\
t |  (trunk patch with no impact)
| s6* (patch to fix newly introduced problem)
|/|
t |
t*|   (trunk patch that broke snapshots again)
| s5* (branch bug fixes due to merge)
| s4* (mechanical fixups due to the merge)
|/|
t |
t |
t |
| s3
| s2
| s1
|/
t
t

This approach was captured in github and had the added benefit of
preserving exact history, and having known good points preserving the
compile/unit test invariants on trunk.  (unfortunately, some of this
history was lost when we ported the git history over to svn, but that isn't
a problem any more).    If you want to see the whole thing, look at my git
repo:

git remote add jmhsieh git@github.com:jmhsieh/hbase.git
git log --oneline --graph --color jmhsieh/hbase-7290


Can we do history-preserving merge with the first style? I can do the
> rebases and upload interdiff if that is big so that we can compare on a
> per-patch basis.
>
>
It is possible.





> Enis
>
>
> On Fri, Jun 6, 2014 at 4:36 PM, Jonathan Hsieh <jo...@cloudera.com> wrote:
>
> > When we merged snapshots branch in we did this:
> >
> > t = trunk commit
> > s = snapshot branch commit
> > m = merge point.
> >
> > During work:
> > t
> > t
> > t
> > |  s3
> > |  s2
> > |  s1
> > |/
> > t
> > t
> >
> > During after merge:
> > m  (essentially empty merge patch).
> > t \
> > t |
> > t |
> > | s4* (fixups due to the merge)
> > | s3
> > | s2
> > | s1
> > |/
> > t
> > t
> >
> > Does your proposal mean you are going to do this instead?
> >
> > s3* (modified due to rebase)
> > s2* (modified due to rebase)
> > s1
> > t
> > t
> > t
> > | s (now dead hbase-10070 branch)
> > | s
> > | s
> > | s
> > |/
> > t
> > t
> >
> > If it is the then we should probably have a review of the modified
> patches.
> >  If it the same as the snapshot merge, then we just need a review of the
> > merge point delta (as well as a full review).
> >
> > Personally I prefer the merge for these large feature branches -- it
> > guarantees that each commit is compilable, and reflects what you guys
> have
> > been testing for a while.  If you go with the last approach you might
> have
> > stuff broken, and in the mainline commit path.
> >
> > Jon.
> >
> >
> >
> > On Tue, Jun 3, 2014 at 11:16 AM, Andrew Purtell <ap...@apache.org>
> > wrote:
> >
> > >  
> > > On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <en...@apache.org> wrote:
> > >
> > > > This VOTE is for merging back the remaining changes in branch to
> trunk.
> > > If
> > > > passes, we will rebase the branch on top of current trunk, in which
> we
> > > will
> > > > keep the commit-per-issue log history. After that we will do a git
> > merge
> > > > for the branch keeping the history clean and not squashing the
> > commits. I
> > > > expect rebasing to be straightforward, however with some manual
> > conflict
> > > > resolution. After the merge we'll keep running the tests to make sure
> > > > everything is ok.
> > > >
> > >
> > > Just to clarify that would look something like this:
> > >
> > >      $ git checkout HBASE-10070
> > >      $ git rebase --ignore-date master
> > >      (fixups, git add, git rebase --continue, etc, etc, etc)
> > >      $ git checkout master
> > >      $ git merge HBASE-10070
> > >
> > > ?
> > >
> > > That sounds good to me, the final merge should be a fast forward merge.
> > >
> > > Use of ' --ignore-date' could be mildly controversial. It's not
> strictly
> > > necessary because the commits for 10070 will appear grouped in history,
> > but
> > > then dates on commits will be discontiguous in that section of
> history. I
> > > suggest using that option so the order of commits and dates sort the
> same
> > > on master.
> > >
> > >
> > > On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <en...@apache.org> wrote:
> > >
> > > > Hi,
> > > >
> > > > Last week we started some discussion[4] for merging branch
> > hbase-10070[1]
> > > > into trunk. It seems like the consensus there is to do the merge
> sooner
> > > > rather than later.
> > > >
> > > >
> > > > We had branched hbase-10070 in Feb out of trunk[5]. The branch
> contains
> > > 55
> > > > jiras committed[2]. Out of these 55, 15 has already been committed to
> > > trunk
> > > > and backported to hbase-10070 branch[3].
> > > >
> > > > This VOTE is for merging back the remaining changes in branch to
> trunk.
> > > If
> > > > passes, we will rebase the branch on top of current trunk, in which
> we
> > > will
> > > > keep the commit-per-issue log history. After that we will do a git
> > merge
> > > > for the branch keeping the history clean and not squashing the
> > commits. I
> > > > expect rebasing to be straightforward, however with some manual
> > conflict
> > > > resolution. After the merge we'll keep running the tests to make sure
> > > > everything is ok.
> > > >
> > > > An overview of the changes, and the status of the work can be found
> > under
> > > > [4], [6] and [7].In summary, with the code in branch, you can create
> > > tables
> > > > with region replicas, do gets / multi gets and scans using TIMELINE
> > > > consistency with high availability. Region replicas periodically scan
> > the
> > > > files of the primary and pick up flushed / committed files. The RPC
> > > paths /
> > > > assignment, balancing etc are pretty stable. However some more
> > > performance
> > > > analysis and tuning is needed. Phase 2 work is being worked on under
> > > > HBASE-11183, and we have some working prototype for async-replicating
> > and
> > > > region splits. However, we believe even without those features, this
> > work
> > > > is useable (especially for read-only/bulk load tables) , and can be
> > > > released as an experimental feature in 1.0.
> > > >
> > > > Please indicate your choice:
> > > >
> > > > [ ] +1 on yes, merge branch hbase-10070 to trunk.
> > > > [ ] 0 on don't care
> > > > [ ] -1 don't merge, because ...
> > > >
> > > > I'll keep the vote running for 7 days, and close it Mon 9th of June,
> > PDT.
> > > >
> > > > Here is my official +1.
> > > >
> > > > Thanks,
> > > > Enis
> > > >
> > > > [1]
> > > >
> > > >
> > >
> >
> https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=log;h=refs/heads/hbase-10070
> > > > [2]
> > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/browse/HBASE-11214?jql=fixVersion%20%3D%20hbase-10070%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolved
> > > > [3]
> > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/browse/HBASE-10792?jql=fixVersion%20%3D%20hbase-10070%20and%20fixversion%20%3D%200.99.0%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolved
> > > > [4] https://www.mail-archive.com/dev@hbase.apache.org/msg25795.html
> > > > [5]
> > > >
> > > >
> > >
> >
> https://github.com/apache/hbase/commit/e22c7efeac02efde3451a0c9ff9bdcd2725576d0
> > > > [6]
> > > >
> > > >
> > >
> >
> http://www.slideshare.net/enissoz/hbase-high-availability-for-reads-with-time
> > > > [7] https://issues.apache.org/jira/browse/HBASE-10070
> > > >
> > >
> > >
> > >
> > > --
> > > Best regards,
> > >
> > >    - Andy
> > >
> > > Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > > (via Tom White)
> > >
> >
> >
> >
> > --
> > // Jonathan Hsieh (shay)
> > // HBase Tech Lead, Software Engineer, Cloudera
> > // jon@cloudera.com // @jmhsieh
> >
>



-- 
// Jonathan Hsieh (shay)
// HBase Tech Lead, Software Engineer, Cloudera
// jon@cloudera.com // @jmhsieh

Re: [VOTE] Merge branch HBASE-10070 to trunk

Posted by Enis Söztutar <en...@gmail.com>.

My preference would be to do the rebase-then-merge style (last style in
your comment). For each patch, I am hoping for all the changes between
committed version and rebased version to be mechanical. I like having a
linear history with explicit commit-to-patch mapping.

Can we do history-preserving merge with the first style? I can do the
rebases and upload interdiff if that is big so that we can compare on a
per-patch basis.

Enis


On Fri, Jun 6, 2014 at 4:36 PM, Jonathan Hsieh <jo...@cloudera.com> wrote:

> When we merged snapshots branch in we did this:
>
> t = trunk commit
> s = snapshot branch commit
> m = merge point.
>
> During work:
> t
> t
> t
> |  s3
> |  s2
> |  s1
> |/
> t
> t
>
> During after merge:
> m  (essentially empty merge patch).
> t \
> t |
> t |
> | s4* (fixups due to the merge)
> | s3
> | s2
> | s1
> |/
> t
> t
>
> Does your proposal mean you are going to do this instead?
>
> s3* (modified due to rebase)
> s2* (modified due to rebase)
> s1
> t
> t
> t
> | s (now dead hbase-10070 branch)
> | s
> | s
> | s
> |/
> t
> t
>
> If it is the then we should probably have a review of the modified patches.
>  If it the same as the snapshot merge, then we just need a review of the
> merge point delta (as well as a full review).
>
> Personally I prefer the merge for these large feature branches -- it
> guarantees that each commit is compilable, and reflects what you guys have
> been testing for a while.  If you go with the last approach you might have
> stuff broken, and in the mainline commit path.
>
> Jon.
>
>
>
> On Tue, Jun 3, 2014 at 11:16 AM, Andrew Purtell <ap...@apache.org>
> wrote:
>
> >  
> > On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <en...@apache.org> wrote:
> >
> > > This VOTE is for merging back the remaining changes in branch to trunk.
> > If
> > > passes, we will rebase the branch on top of current trunk, in which we
> > will
> > > keep the commit-per-issue log history. After that we will do a git
> merge
> > > for the branch keeping the history clean and not squashing the
> commits. I
> > > expect rebasing to be straightforward, however with some manual
> conflict
> > > resolution. After the merge we'll keep running the tests to make sure
> > > everything is ok.
> > >
> >
> > Just to clarify that would look something like this:
> >
> >      $ git checkout HBASE-10070
> >      $ git rebase --ignore-date master
> >      (fixups, git add, git rebase --continue, etc, etc, etc)
> >      $ git checkout master
> >      $ git merge HBASE-10070
> >
> > ?
> >
> > That sounds good to me, the final merge should be a fast forward merge.
> >
> > Use of ' --ignore-date' could be mildly controversial. It's not strictly
> > necessary because the commits for 10070 will appear grouped in history,
> but
> > then dates on commits will be discontiguous in that section of history. I
> > suggest using that option so the order of commits and dates sort the same
> > on master.
> >
> >
> > On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <en...@apache.org> wrote:
> >
> > > Hi,
> > >
> > > Last week we started some discussion[4] for merging branch
> hbase-10070[1]
> > > into trunk. It seems like the consensus there is to do the merge sooner
> > > rather than later.
> > >
> > >
> > > We had branched hbase-10070 in Feb out of trunk[5]. The branch contains
> > 55
> > > jiras committed[2]. Out of these 55, 15 has already been committed to
> > trunk
> > > and backported to hbase-10070 branch[3].
> > >
> > > This VOTE is for merging back the remaining changes in branch to trunk.
> > If
> > > passes, we will rebase the branch on top of current trunk, in which we
> > will
> > > keep the commit-per-issue log history. After that we will do a git
> merge
> > > for the branch keeping the history clean and not squashing the
> commits. I
> > > expect rebasing to be straightforward, however with some manual
> conflict
> > > resolution. After the merge we'll keep running the tests to make sure
> > > everything is ok.
> > >
> > > An overview of the changes, and the status of the work can be found
> under
> > > [4], [6] and [7].In summary, with the code in branch, you can create
> > tables
> > > with region replicas, do gets / multi gets and scans using TIMELINE
> > > consistency with high availability. Region replicas periodically scan
> the
> > > files of the primary and pick up flushed / committed files. The RPC
> > paths /
> > > assignment, balancing etc are pretty stable. However some more
> > performance
> > > analysis and tuning is needed. Phase 2 work is being worked on under
> > > HBASE-11183, and we have some working prototype for async-replicating
> and
> > > region splits. However, we believe even without those features, this
> work
> > > is useable (especially for read-only/bulk load tables) , and can be
> > > released as an experimental feature in 1.0.
> > >
> > > Please indicate your choice:
> > >
> > > [ ] +1 on yes, merge branch hbase-10070 to trunk.
> > > [ ] 0 on don't care
> > > [ ] -1 don't merge, because ...
> > >
> > > I'll keep the vote running for 7 days, and close it Mon 9th of June,
> PDT.
> > >
> > > Here is my official +1.
> > >
> > > Thanks,
> > > Enis
> > >
> > > [1]
> > >
> > >
> >
> https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=log;h=refs/heads/hbase-10070
> > > [2]
> > >
> > >
> >
> https://issues.apache.org/jira/browse/HBASE-11214?jql=fixVersion%20%3D%20hbase-10070%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolved
> > > [3]
> > >
> > >
> >
> https://issues.apache.org/jira/browse/HBASE-10792?jql=fixVersion%20%3D%20hbase-10070%20and%20fixversion%20%3D%200.99.0%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolved
> > > [4] https://www.mail-archive.com/dev@hbase.apache.org/msg25795.html
> > > [5]
> > >
> > >
> >
> https://github.com/apache/hbase/commit/e22c7efeac02efde3451a0c9ff9bdcd2725576d0
> > > [6]
> > >
> > >
> >
> http://www.slideshare.net/enissoz/hbase-high-availability-for-reads-with-time
> > > [7] https://issues.apache.org/jira/browse/HBASE-10070
> > >
> >
> >
> >
> > --
> > Best regards,
> >
> >    - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
> >
>
>
>
> --
> // Jonathan Hsieh (shay)
> // HBase Tech Lead, Software Engineer, Cloudera
> // jon@cloudera.com // @jmhsieh
>

Re: [VOTE] Merge branch HBASE-10070 to trunk

Posted by Andrew Purtell <ap...@apache.org>.

On Fri, Jun 6, 2014 at 5:07 PM, Enis Söztutar <en...@gmail.com> wrote:

> > This is why I asked for the final patches to be broken out with git
> > format-patch and attached to the JIRA, so we know what was finally
> > committed.
> >
>
> Fair enough. I thought that the canonical version of the patch is always in
> repo anyway, but I'll upload the patches.
>

They are, but I've also been contacted by people wondering about
differences between patches on JIRAs and what was committed, so it would be
good to sync both since format-patch is pretty convenient. Your call
though. 

-- 
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Re: [VOTE] Merge branch HBASE-10070 to trunk

Posted by Enis Söztutar <en...@gmail.com>.

On Fri, Jun 6, 2014 at 4:44 PM, Andrew Purtell <ap...@apache.org> wrote:

> On Fri, Jun 6, 2014 at 4:36 PM, Jonathan Hsieh <jo...@cloudera.com> wrote:
>
> > Personally I prefer the merge for these large feature branches -- it
> > guarantees that each commit is compilable, and reflects what you guys
> have
> > been testing for a while.  If you go with the last approach you might
> have
> > stuff broken, and in the mainline commit path.
> >
>
> This is why I asked for the final patches to be broken out with git
> format-patch and attached to the JIRA, so we know what was finally
> committed.
>

Fair enough. I thought that the canonical version of the patch is always in
repo anyway, but I'll upload the patches.


>
> To allay these concerns it would be great, if rebasing the 10070 patches
> consecutively on top of trunk, to check that the result passes the typical
> release candidate checks and only push after it checks out.
>

Agreed. I'll be running the UT + some tests before the push.


>
> My opinion is this need not be a requirement, even with a history
> preserving merge into trunk we can end up in a broken state if fixups are
> committed in the merge commit. We would have the additional complication of
> a delta committed in the merge commit not directly associated with a JIRA.
>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>

Re: [VOTE] Merge branch HBASE-10070 to trunk

Posted by Andrew Purtell <ap...@apache.org>.

On Fri, Jun 6, 2014 at 4:36 PM, Jonathan Hsieh <jo...@cloudera.com> wrote:

> Personally I prefer the merge for these large feature branches -- it
> guarantees that each commit is compilable, and reflects what you guys have
> been testing for a while.  If you go with the last approach you might have
> stuff broken, and in the mainline commit path.
>

This is why I asked for the final patches to be broken out with git
format-patch and attached to the JIRA, so we know what was finally
committed.

To allay these concerns it would be great, if rebasing the 10070 patches
consecutively on top of trunk, to check that the result passes the typical
release candidate checks and only push after it checks out.

My opinion is this need not be a requirement, even with a history
preserving merge into trunk we can end up in a broken state if fixups are
committed in the merge commit. We would have the additional complication of
a delta committed in the merge commit not directly associated with a JIRA.

-- 
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Re: [VOTE] Merge branch HBASE-10070 to trunk

Posted by Jonathan Hsieh <jo...@cloudera.com>.

When we merged snapshots branch in we did this:

t = trunk commit
s = snapshot branch commit
m = merge point.

During work:
t
t
t
|  s3
|  s2
|  s1
|/
t
t

During after merge:
m  (essentially empty merge patch).
t \
t |
t |
| s4* (fixups due to the merge)
| s3
| s2
| s1
|/
t
t

Does your proposal mean you are going to do this instead?

s3* (modified due to rebase)
s2* (modified due to rebase)
s1
t
t
t
| s (now dead hbase-10070 branch)
| s
| s
| s
|/
t
t

If it is the then we should probably have a review of the modified patches.
 If it the same as the snapshot merge, then we just need a review of the
merge point delta (as well as a full review).

Personally I prefer the merge for these large feature branches -- it
guarantees that each commit is compilable, and reflects what you guys have
been testing for a while.  If you go with the last approach you might have
stuff broken, and in the mainline commit path.

Jon.



On Tue, Jun 3, 2014 at 11:16 AM, Andrew Purtell <ap...@apache.org> wrote:

>  
> On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <en...@apache.org> wrote:
>
> > This VOTE is for merging back the remaining changes in branch to trunk.
> If
> > passes, we will rebase the branch on top of current trunk, in which we
> will
> > keep the commit-per-issue log history. After that we will do a git merge
> > for the branch keeping the history clean and not squashing the commits. I
> > expect rebasing to be straightforward, however with some manual conflict
> > resolution. After the merge we'll keep running the tests to make sure
> > everything is ok.
> >
>
> Just to clarify that would look something like this:
>
>      $ git checkout HBASE-10070
>      $ git rebase --ignore-date master
>      (fixups, git add, git rebase --continue, etc, etc, etc)
>      $ git checkout master
>      $ git merge HBASE-10070
>
> ?
>
> That sounds good to me, the final merge should be a fast forward merge.
>
> Use of ' --ignore-date' could be mildly controversial. It's not strictly
> necessary because the commits for 10070 will appear grouped in history, but
> then dates on commits will be discontiguous in that section of history. I
> suggest using that option so the order of commits and dates sort the same
> on master.
>
>
> On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <en...@apache.org> wrote:
>
> > Hi,
> >
> > Last week we started some discussion[4] for merging branch hbase-10070[1]
> > into trunk. It seems like the consensus there is to do the merge sooner
> > rather than later.
> >
> >
> > We had branched hbase-10070 in Feb out of trunk[5]. The branch contains
> 55
> > jiras committed[2]. Out of these 55, 15 has already been committed to
> trunk
> > and backported to hbase-10070 branch[3].
> >
> > This VOTE is for merging back the remaining changes in branch to trunk.
> If
> > passes, we will rebase the branch on top of current trunk, in which we
> will
> > keep the commit-per-issue log history. After that we will do a git merge
> > for the branch keeping the history clean and not squashing the commits. I
> > expect rebasing to be straightforward, however with some manual conflict
> > resolution. After the merge we'll keep running the tests to make sure
> > everything is ok.
> >
> > An overview of the changes, and the status of the work can be found under
> > [4], [6] and [7].In summary, with the code in branch, you can create
> tables
> > with region replicas, do gets / multi gets and scans using TIMELINE
> > consistency with high availability. Region replicas periodically scan the
> > files of the primary and pick up flushed / committed files. The RPC
> paths /
> > assignment, balancing etc are pretty stable. However some more
> performance
> > analysis and tuning is needed. Phase 2 work is being worked on under
> > HBASE-11183, and we have some working prototype for async-replicating and
> > region splits. However, we believe even without those features, this work
> > is useable (especially for read-only/bulk load tables) , and can be
> > released as an experimental feature in 1.0.
> >
> > Please indicate your choice:
> >
> > [ ] +1 on yes, merge branch hbase-10070 to trunk.
> > [ ] 0 on don't care
> > [ ] -1 don't merge, because ...
> >
> > I'll keep the vote running for 7 days, and close it Mon 9th of June, PDT.
> >
> > Here is my official +1.
> >
> > Thanks,
> > Enis
> >
> > [1]
> >
> >
> https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=log;h=refs/heads/hbase-10070
> > [2]
> >
> >
> https://issues.apache.org/jira/browse/HBASE-11214?jql=fixVersion%20%3D%20hbase-10070%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolved
> > [3]
> >
> >
> https://issues.apache.org/jira/browse/HBASE-10792?jql=fixVersion%20%3D%20hbase-10070%20and%20fixversion%20%3D%200.99.0%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolved
> > [4] https://www.mail-archive.com/dev@hbase.apache.org/msg25795.html
> > [5]
> >
> >
> https://github.com/apache/hbase/commit/e22c7efeac02efde3451a0c9ff9bdcd2725576d0
> > [6]
> >
> >
> http://www.slideshare.net/enissoz/hbase-high-availability-for-reads-with-time
> > [7] https://issues.apache.org/jira/browse/HBASE-10070
> >
>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>



-- 
// Jonathan Hsieh (shay)
// HBase Tech Lead, Software Engineer, Cloudera
// jon@cloudera.com // @jmhsieh

Re: [VOTE] Merge branch HBASE-10070 to trunk

Posted by Andrew Purtell <ap...@apache.org>.

 
On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <en...@apache.org> wrote:

> This VOTE is for merging back the remaining changes in branch to trunk. If
> passes, we will rebase the branch on top of current trunk, in which we will
> keep the commit-per-issue log history. After that we will do a git merge
> for the branch keeping the history clean and not squashing the commits. I
> expect rebasing to be straightforward, however with some manual conflict
> resolution. After the merge we'll keep running the tests to make sure
> everything is ok.
>

Just to clarify that would look something like this:

     $ git checkout HBASE-10070
     $ git rebase --ignore-date master
     (fixups, git add, git rebase --continue, etc, etc, etc)
     $ git checkout master
     $ git merge HBASE-10070

?

That sounds good to me, the final merge should be a fast forward merge.

Use of ' --ignore-date' could be mildly controversial. It's not strictly
necessary because the commits for 10070 will appear grouped in history, but
then dates on commits will be discontiguous in that section of history. I
suggest using that option so the order of commits and dates sort the same
on master.


On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <en...@apache.org> wrote:

> Hi,
>
> Last week we started some discussion[4] for merging branch hbase-10070[1]
> into trunk. It seems like the consensus there is to do the merge sooner
> rather than later.
>
>
> We had branched hbase-10070 in Feb out of trunk[5]. The branch contains 55
> jiras committed[2]. Out of these 55, 15 has already been committed to trunk
> and backported to hbase-10070 branch[3].
>
> This VOTE is for merging back the remaining changes in branch to trunk. If
> passes, we will rebase the branch on top of current trunk, in which we will
> keep the commit-per-issue log history. After that we will do a git merge
> for the branch keeping the history clean and not squashing the commits. I
> expect rebasing to be straightforward, however with some manual conflict
> resolution. After the merge we'll keep running the tests to make sure
> everything is ok.
>
> An overview of the changes, and the status of the work can be found under
> [4], [6] and [7].In summary, with the code in branch, you can create tables
> with region replicas, do gets / multi gets and scans using TIMELINE
> consistency with high availability. Region replicas periodically scan the
> files of the primary and pick up flushed / committed files. The RPC paths /
> assignment, balancing etc are pretty stable. However some more performance
> analysis and tuning is needed. Phase 2 work is being worked on under
> HBASE-11183, and we have some working prototype for async-replicating and
> region splits. However, we believe even without those features, this work
> is useable (especially for read-only/bulk load tables) , and can be
> released as an experimental feature in 1.0.
>
> Please indicate your choice:
>
> [ ] +1 on yes, merge branch hbase-10070 to trunk.
> [ ] 0 on don't care
> [ ] -1 don't merge, because ...
>
> I'll keep the vote running for 7 days, and close it Mon 9th of June, PDT.
>
> Here is my official +1.
>
> Thanks,
> Enis
>
> [1]
>
> https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=log;h=refs/heads/hbase-10070
> [2]
>
> https://issues.apache.org/jira/browse/HBASE-11214?jql=fixVersion%20%3D%20hbase-10070%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolved
> [3]
>
> https://issues.apache.org/jira/browse/HBASE-10792?jql=fixVersion%20%3D%20hbase-10070%20and%20fixversion%20%3D%200.99.0%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolved
> [4] https://www.mail-archive.com/dev@hbase.apache.org/msg25795.html
> [5]
>
> https://github.com/apache/hbase/commit/e22c7efeac02efde3451a0c9ff9bdcd2725576d0
> [6]
>
> http://www.slideshare.net/enissoz/hbase-high-availability-for-reads-with-time
> [7] https://issues.apache.org/jira/browse/HBASE-10070
>



-- 
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)