Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Ted Yu <yu...@gmail.com>]

Reviving this thread.

The following has taken place:

mapreduce dependency has been moved to client side - no mapreduce job
launched from master or region server.
document patch (HBASE-16574) has been integrated.
Updated mega patch has been attached to HBASE-14123: this covers the
refactor in #1 above and the protobuf 3 merge.

If community has more feedback on the merge proposal, I would love to hear
it.

Thanks

On Thu, Sep 22, 2016 at 10:31 AM, Sean Busbey <bu...@cloudera.com> wrote:

> I'd like to see the docs proposed on HBASE-16574 integrated into our
> project's documentation prior to merge.
>
> On Thu, Sep 22, 2016 at 9:02 AM, Ted Yu <yu...@gmail.com> wrote:
> > This feature can be marked experimental due to some limitations such as
> > security.
> >
> > Your previous round of comments have been addressed.
> > Command line tool has gone through:
> >
> > HBASE-16620 Fix backup command-line tool usability issues
> > HBASE-16655 hbase backup describe with incorrect backup id results in NPE
> >
> > The updated doc has been attached to HBASE-16574.
> >
> > Cheers
> >
> > On Thu, Sep 22, 2016 at 8:53 AM, Stack <st...@duboce.net> wrote:
> >
> >> On Wed, Sep 21, 2016 at 7:43 AM, Ted Yu <yu...@gmail.com> wrote:
> >>
> >> > Are there more (review) comments ?
> >> >
> >> >
> >> Are outstanding comments addressed?
> >>
> >> I don't see answer to my 'is this experimental/will it be marked
> >> experimental' question.
> >>
> >> I ran into some issues trying to use the feature and suggested that a
> >> feature likes this needs polish else it'll just rot, unused. Has polish
> >> been applied? All ready for another 'user' test? Suggest that you update
> >> here going forward for the benefit of those trying to follow along and
> who
> >> are not watching JIRA change fly-by.
> >>
> >> It looks like doc got a revision -- I have to check -- to take on
> >> suggestion made above but again, suggest, that this thread gets updated.
> >>
> >> Thanks,
> >> St.Ack
> >>
> >>
> >>
> >> > Thanks
> >> >
> >> > On Tue, Sep 20, 2016 at 10:02 AM, Devaraj Das <dd...@hortonworks.com>
> >> > wrote:
> >> >
> >> > > Just reviving this thread. Thanks Sean, Stack, Dima, and others for
> the
> >> > > thorough reviews and testing. Thanks Ted and Vlad for taking care of
> >> the
> >> > > feedback. Are we all good to do the merge now? Rather do sooner than
> >> > later.
> >> > > ________________________________________
> >> > > From: saint.ack@gmail.com <sa...@gmail.com> on behalf of Stack
> <
> >> > > stack@duboce.net>
> >> > > Sent: Monday, September 12, 2016 1:18 PM
> >> > > To: HBase Dev List
> >> > > Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912
> >> > >
> >> > > On Mon, Sep 12, 2016 at 12:19 PM, Ted Yu <yu...@gmail.com>
> wrote:
> >> > >
> >> > > > Mega patch (rev 18) is on HBASE-14123.
> >> > > >
> >> > > > Please comment on HBASE-14123 on how you want to review.
> >> > > >
> >> > >
> >> > >
> >> > > Yeah. That was my lost tab. Last rb was 6 months ago. Suggest
> updating
> >> > it.
> >> > > RB is pretty good for review. Patch is only 1.5M so should be fine.
> >> > >
> >> > > St.Ack
> >> > >
> >> > >
> >> > > >
> >> > > > Thanks
> >> > > >
> >> > > > On Mon, Sep 12, 2016 at 12:15 PM, Stack <st...@duboce.net> wrote:
> >> > > >
> >> > > > > On review of the 'patch', do I just compare the branch to
> master or
> >> > is
> >> > > > > there a megapatch posted somewhere (I think I saw one but it
> seemed
> >> > > stale
> >> > > > > and then I 'lost' the tab). Sorry for dumb question.
> >> > > > > St.Ack
> >> > > > >
> >> > > > > On Mon, Sep 12, 2016 at 12:01 PM, Stack <st...@duboce.net>
> wrote:
> >> > > > >
> >> > > > > > Late to the game. A few comments after rereading this thread
> as a
> >> > > > 'user'.
> >> > > > > >
> >> > > > > > + Before merge, a user-facing feature like this should work
> (If
> >> > this
> >> > > is
> >> > > > > "higher-bar
> >> > > > > > for new features", bring it on -- smile).
> >> > > > > > + As a user, I tried the branch with tools after reviewing the
> >> > > > > just-posted
> >> > > > > > doc. I had an 'interesting' experience (left comments up on
> >> > issue). I
> >> > > > > think
> >> > > > > > the tooling/doc. important to get right. If it breaks easily
> or
> >> is
> >> > > > > > inconsistent (or lacks 'polish'), operators will judge the
> whole
> >> > > > > > backup/restore tooling chain as not trustworthy and abandon
> it.
> >> > Lets
> >> > > > not
> >> > > > > > have this happen to this feature.
> >> > > > > > + Matteo's suggestion (with a helpful starter list) that there
> >> > needs
> >> > > to
> >> > > > > be
> >> > > > > > explicit qualification on what is actually being delivered --
> >> > > > including a
> >> > > > > > listing of limitations (some look serious such as data bleed
> from
> >> > > other
> >> > > > > > regions in WALs, but maybe I don't care for my use case...) --
> >> > needs
> >> > > to
> >> > > > > > accompany the merge. Lets fold them into the user doc. in the
> >> > > technical
> >> > > > > > overview area as suggested so user expectations are properly
> >> > managed
> >> > > > > > (otherwise, they expect the world and will just give up when
> we
> >> > fall
> >> > > > > > short). Vladimir did a list of what is in each of the phases
> >> above
> >> > > > which
> >> > > > > > would serve as a good start.
> >> > > > > > + Is this feature 'experimental' (Matteo asks above). I'd
> prefer
> >> it
> >> > > is
> >> > > > > > not. If it is, it should be labelled all over that it is so. I
> >> see
> >> > > > > current
> >> > > > > > state called out as a '... technical preview feature'. Does
> this
> >> > mean
> >> > > > > > not-for-users?
> >> > > > > >
> >> > > > > > St.Ack
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > > On Mon, Sep 12, 2016 at 8:03 AM, Ted Yu <yu...@gmail.com>
> >> > wrote:
> >> > > > > >
> >> > > > > >> Sean:
> >> > > > > >> Do you have more comments ?
> >> > > > > >>
> >> > > > > >> Cheers
> >> > > > > >>
> >> > > > > >> On Fri, Sep 9, 2016 at 1:42 PM, Vladimir Rodionov <
> >> > > > > vladrodionov@gmail.com
> >> > > > > >> >
> >> > > > > >> wrote:
> >> > > > > >>
> >> > > > > >> > Sean,
> >> > > > > >> >
> >> > > > > >> > Backup/Restore can fail due to various reasons: network
> outage
> >> > > > > (cluster
> >> > > > > >> > wide), various time-outs in HBase and HDFS layer, M/R
> failure
> >> > due
> >> > > to
> >> > > > > >> "HDFS
> >> > > > > >> > exceeded quota", user error (manual deletion of data) and
> so
> >> on
> >> > so
> >> > > > on.
> >> > > > > >> That
> >> > > > > >> > is impossible to enumerate all possible types of failures
> in a
> >> > > > > >> distributed
> >> > > > > >> > system - that is not our goal/task.
> >> > > > > >> >
> >> > > > > >> > We focus completely on backup system table consistency in a
> >> > > presence
> >> > > > > of
> >> > > > > >> any
> >> > > > > >> > type of failure. That is what I call "tolerance to
> failures".
> >> > > > > >> >
> >> > > > > >> > On a failure:
> >> > > > > >> >
> >> > > > > >> > BACKUP. All backup system information (prior to backup)
> will
> >> be
> >> > > > > restored
> >> > > > > >> > and all temporary data, related to a failed session, in
> HDFS
> >> > will
> >> > > be
> >> > > > > >> > deleted
> >> > > > > >> > RESTORE. We do not care about system data, because restore
> >> does
> >> > > not
> >> > > > > >> change
> >> > > > > >> > it. Temporary data in HDFS will be cleaned up and table
> will
> >> be
> >> > > in a
> >> > > > > >> state
> >> > > > > >> > back to where it was before operation started.
> >> > > > > >> >
> >> > > > > >> > This is what user should expect in case of a failure.
> >> > > > > >> >
> >> > > > > >> > -Vlad
> >> > > > > >> >
> >> > > > > >> >
> >> > > > > >> > -Vlad
> >> > > > > >> >
> >> > > > > >> > On Fri, Sep 9, 2016 at 12:56 PM, Sean Busbey <
> >> busbey@apache.org
> >> > >
> >> > > > > wrote:
> >> > > > > >> >
> >> > > > > >> > > Failing in a consistent way, with docs that explain the
> >> > various
> >> > > > > >> > > expected failures would be sufficient.
> >> > > > > >> > >
> >> > > > > >> > > On Fri, Sep 9, 2016 at 12:16 PM, Vladimir Rodionov
> >> > > > > >> > > <vl...@gmail.com> wrote:
> >> > > > > >> > > > Do not worry Sean, doc is coming today as a preview and
> >> our
> >> > > > writer
> >> > > > > >> > Frank
> >> > > > > >> > > > will be working on a putting  it into Apache repo.
> >> Timeline
> >> > > > > depends
> >> > > > > >> on
> >> > > > > >> > > > Franks schedule but I hope we will get it rather sooner
> >> than
> >> > > > > later.
> >> > > > > >> > > >
> >> > > > > >> > > > As for failure testing, we are focusing only on a
> >> consistent
> >> > > > state
> >> > > > > >> of
> >> > > > > >> > > > backup system data in a presence of any type of
> failures,
> >> We
> >> > > are
> >> > > > > not
> >> > > > > >> > > going
> >> > > > > >> > > > to implement  anything more "fancy", than that. We
> allow
> >> > both:
> >> > > > > >> backup
> >> > > > > >> > and
> >> > > > > >> > > > restore to fail. What we do not allow is to have system
> >> data
> >> > > > > >> corrupted.
> >> > > > > >> > > > Will it suffice for you? Do you have any other
> concerns,
> >> you
> >> > > > want
> >> > > > > >> us to
> >> > > > > >> > > > address?
> >> > > > > >> > > >
> >> > > > > >> > > > -Vlad
> >> > > > > >> > > >
> >> > > > > >> > > >
> >> > > > > >> > > > On Fri, Sep 9, 2016 at 10:56 AM, Sean Busbey <
> >> > > busbey@apache.org
> >> > > > >
> >> > > > > >> > wrote:
> >> > > > > >> > > >
> >> > > > > >> > > >> "docs will come to Apache soon" does not address my
> >> concern
> >> > > > > around
> >> > > > > >> > docs
> >> > > > > >> > > at
> >> > > > > >> > > >> all, unless said docs have already made it into the
> >> project
> >> > > > > repo. I
> >> > > > > >> > > don't
> >> > > > > >> > > >> want third party resources for using a major and
> >> important
> >> > > > > feature
> >> > > > > >> of
> >> > > > > >> > > the
> >> > > > > >> > > >> project, I want us to provide end users with what they
> >> need
> >> > > to
> >> > > > > get
> >> > > > > >> the
> >> > > > > >> > > job
> >> > > > > >> > > >> done.
> >> > > > > >> > > >>
> >> > > > > >> > > >> I see some calls for patience on the failure testing,
> but
> >> > the
> >> > > > > >> appeal
> >> > > > > >> > to
> >> > > > > >> > > us
> >> > > > > >> > > >> having done a bad job of requiring proper tests of
> >> previous
> >> > > > > >> features
> >> > > > > >> > > just
> >> > > > > >> > > >> makes me more concerned about not getting them here. I
> >> > don't
> >> > > > want
> >> > > > > >> to
> >> > > > > >> > set
> >> > > > > >> > > >> yet another bad example that will then be pointed to
> in
> >> the
> >> > > > > future.
> >> > > > > >> > > >>
> >> > > > > >> > > >> On Sep 8, 2016 10:50, "Ted Yu" <yu...@gmail.com>
> >> > wrote:
> >> > > > > >> > > >>
> >> > > > > >> > > >> > Is there any concern which is not addressed ?
> >> > > > > >> > > >> >
> >> > > > > >> > > >> > Do we need another Vote thread ?
> >> > > > > >> > > >> >
> >> > > > > >> > > >> > Thanks
> >> > > > > >> > > >> >
> >> > > > > >> > > >> > On Thu, Sep 8, 2016 at 9:21 AM, Andrew Purtell <
> >> > > > > >> apurtell@apache.org
> >> > > > > >> > >
> >> > > > > >> > > >> > wrote:
> >> > > > > >> > > >> >
> >> > > > > >> > > >> > > Vlad,
> >> > > > > >> > > >> > >
> >> > > > > >> > > >> > > I apologize for using the term 'half-baked' in a
> way
> >> > that
> >> > > > > could
> >> > > > > >> > > seem a
> >> > > > > >> > > >> > > description of HBASE-7912. I meant that as a
> general
> >> > > > > >> hypothetical.
> >> > > > > >> > > >> > >
> >> > > > > >> > > >> > > On Wed, Sep 7, 2016 at 9:36 AM, Vladimir Rodionov
> <
> >> > > > > >> > > >> > vladrodionov@gmail.com>
> >> > > > > >> > > >> > > wrote:
> >> > > > > >> > > >> > >
> >> > > > > >> > > >> > > > >> I'm not sure that "There is already lots of
> >> > > half-baked
> >> > > > > >> code
> >> > > > > >> > in
> >> > > > > >> > > the
> >> > > > > >> > > >> > > > branch,
> >> > > > > >> > > >> > > > so what's the harm in adding more?"
> >> > > > > >> > > >> > > >
> >> > > > > >> > > >> > > > I meant - not production - ready yet. This is
> 2.0
> >> > > > > development
> >> > > > > >> > > branch
> >> > > > > >> > > >> > and,
> >> > > > > >> > > >> > > > hence many features are in works,
> >> > > > > >> > > >> > > > not being tested well etc. I do not consider
> backup
> >> > as
> >> > > > half
> >> > > > > >> > baked
> >> > > > > >> > > >> > > feature -
> >> > > > > >> > > >> > > > it has passed our internal QA and has very good
> >> doc,
> >> > > > which
> >> > > > > we
> >> > > > > >> > will
> >> > > > > >> > > >> > > provide
> >> > > > > >> > > >> > > > to Apache shortly.
> >> > > > > >> > > >> > > >
> >> > > > > >> > > >> > > > -Vlad
> >> > > > > >> > > >> > > >
> >> > > > > >> > > >> > > > On Wed, Sep 7, 2016 at 9:13 AM, Andrew Purtell <
> >> > > > > >> > > apurtell@apache.org>
> >> > > > > >> > > >> > > > wrote:
> >> > > > > >> > > >> > > >
> >> > > > > >> > > >> > > > > We shouldn't admit half baked changes that
> won't
> >> be
> >> > > > > >> finished.
> >> > > > > >> > > >> However
> >> > > > > >> > > >> > > in
> >> > > > > >> > > >> > > > > this case the crew working on this feature are
> >> long
> >> > > > > timers
> >> > > > > >> and
> >> > > > > >> > > less
> >> > > > > >> > > >> > > > likely
> >> > > > > >> > > >> > > > > than just about anyone to leave something in a
> >> half
> >> > > > baked
> >> > > > > >> > > state. Of
> >> > > > > >> > > >> > > > course
> >> > > > > >> > > >> > > > > there is no guarantee how anything will turn
> out,
> >> > > but I
> >> > > > > am
> >> > > > > >> > > willing
> >> > > > > >> > > >> to
> >> > > > > >> > > >> > > > take
> >> > > > > >> > > >> > > > > a little on faith if they feel their best path
> >> > > forward
> >> > > > > now
> >> > > > > >> is
> >> > > > > >> > to
> >> > > > > >> > > >> > merge
> >> > > > > >> > > >> > > to
> >> > > > > >> > > >> > > > > trunk. I only wish I had bandwidth to have
> done
> >> > some
> >> > > > real
> >> > > > > >> > > kicking
> >> > > > > >> > > >> of
> >> > > > > >> > > >> > > the
> >> > > > > >> > > >> > > > > tires by now. Maybe this week.
> >> > > > > >> > > >> > > > >
> >> > > > > >> > > >> > > > > (Yes, I'm using some of that time for this
> email
> >> > :-)
> >> > > > but
> >> > > > > I
> >> > > > > >> > type
> >> > > > > >> > > >> > fast.)
> >> > > > > >> > > >> > > > >
> >> > > > > >> > > >> > > > > That said, I would like to agitate for making
> 2.0
> >> > > more
> >> > > > > real
> >> > > > > >> > and
> >> > > > > >> > > >> spend
> >> > > > > >> > > >> > > > some
> >> > > > > >> > > >> > > > > time on it now that I'm winding down with
> 0.98. I
> >> > > think
> >> > > > > >> that
> >> > > > > >> > > means
> >> > > > > >> > > >> > > > > branching for 2.0 real soon now and even
> evicting
> >> > > > things
> >> > > > > >> from
> >> > > > > >> > > 2.0
> >> > > > > >> > > >> > > branch
> >> > > > > >> > > >> > > > > that aren't finished or stable, leaving them
> only
> >> > > once
> >> > > > > >> again
> >> > > > > >> > in
> >> > > > > >> > > the
> >> > > > > >> > > >> > > > master
> >> > > > > >> > > >> > > > > branch. Or, maybe just evicting them. Let's
> take
> >> it
> >> > > > case
> >> > > > > by
> >> > > > > >> > > case.
> >> > > > > >> > > >> > > > >
> >> > > > > >> > > >> > > > > I think this feature can come in relatively
> >> safely.
> >> > > As
> >> > > > > >> added
> >> > > > > >> > > >> > insurance,
> >> > > > > >> > > >> > > > > let's admit the possibility it could be
> reverted
> >> on
> >> > > the
> >> > > > > 2.0
> >> > > > > >> > > branch
> >> > > > > >> > > >> if
> >> > > > > >> > > >> > > > folks
> >> > > > > >> > > >> > > > > working on stabilizing 2.0 decide to evict it
> >> > because
> >> > > > it
> >> > > > > is
> >> > > > > >> > > >> > unfinished
> >> > > > > >> > > >> > > or
> >> > > > > >> > > >> > > > > unstable, because that certainly can happen. I
> >> > would
> >> > > > > >> expect if
> >> > > > > >> > > talk
> >> > > > > >> > > >> > > like
> >> > > > > >> > > >> > > > > that starts, we'd get help finishing or
> >> stabilizing
> >> > > > > what's
> >> > > > > >> > under
> >> > > > > >> > > >> > > > discussion
> >> > > > > >> > > >> > > > > for revert. Or, we'd have a revert. Either way
> >> the
> >> > > > > outcome
> >> > > > > >> is
> >> > > > > >> > > >> > > acceptable.
> >> > > > > >> > > >> > > > >
> >> > > > > >> > > >> > > > >
> >> > > > > >> > > >> > > > > On Wed, Sep 7, 2016 at 8:56 AM, Dima Spivak <
> >> > > > > >> > > dimaspivak@apache.org
> >> > > > > >> > > >> >
> >> > > > > >> > > >> > > > wrote:
> >> > > > > >> > > >> > > > >
> >> > > > > >> > > >> > > > > > I'm not sure that "There is already lots of
> >> > > > half-baked
> >> > > > > >> code
> >> > > > > >> > in
> >> > > > > >> > > >> the
> >> > > > > >> > > >> > > > > branch,
> >> > > > > >> > > >> > > > > > so what's the harm in adding more?" is a
> good
> >> > code
> >> > > > > commit
> >> > > > > >> > > >> > philosophy
> >> > > > > >> > > >> > > > for
> >> > > > > >> > > >> > > > > a
> >> > > > > >> > > >> > > > > > fault-tolerant distributed data store. ;)
> >> > > > > >> > > >> > > > > >
> >> > > > > >> > > >> > > > > > More seriously, a lack of test coverage for
> >> > > existing
> >> > > > > >> > features
> >> > > > > >> > > >> > > shouldn't
> >> > > > > >> > > >> > > > > be
> >> > > > > >> > > >> > > > > > used as justification for introducing new
> >> > features
> >> > > > with
> >> > > > > >> the
> >> > > > > >> > > same
> >> > > > > >> > > >> > > > > > shortcomings. Ultimately, it's the end user
> who
> >> > > will
> >> > > > > feel
> >> > > > > >> > the
> >> > > > > >> > > >> pain,
> >> > > > > >> > > >> > > so
> >> > > > > >> > > >> > > > > > shouldn't we do everything we can to
> mitigate
> >> > that?
> >> > > > > >> > > >> > > > > >
> >> > > > > >> > > >> > > > > > -Dima
> >> > > > > >> > > >> > > > > >
> >> > > > > >> > > >> > > > > > On Wed, Sep 7, 2016 at 8:46 AM, Vladimir
> >> > Rodionov <
> >> > > > > >> > > >> > > > > vladrodionov@gmail.com>
> >> > > > > >> > > >> > > > > > wrote:
> >> > > > > >> > > >> > > > > >
> >> > > > > >> > > >> > > > > > > Sean,
> >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > >> > > > > > > * have docs
> >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > >> > > > > > > Agree. We have a doc and backup is the
> most
> >> > > > > documented
> >> > > > > >> > > feature
> >> > > > > >> > > >> > :),
> >> > > > > >> > > >> > > we
> >> > > > > >> > > >> > > > > > will
> >> > > > > >> > > >> > > > > > > release it shortly to Apache.
> >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > >> > > > > > > * have sunny-day correctness tests
> >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > >> > > > > > > Feature has  close to 60 test cases, which
> >> run
> >> > > for
> >> > > > > >> approx
> >> > > > > >> > 30
> >> > > > > >> > > >> min.
> >> > > > > >> > > >> > > We
> >> > > > > >> > > >> > > > > can
> >> > > > > >> > > >> > > > > > > add more, if community do not mind :)
> >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > >> > > > > > > * have correctness-in-face-of-failure
> tests
> >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > >> > > > > > > Any examples of these tests in existing
> >> > features?
> >> > > > In
> >> > > > > >> > works,
> >> > > > > >> > > we
> >> > > > > >> > > >> > > have a
> >> > > > > >> > > >> > > > > > clear
> >> > > > > >> > > >> > > > > > > understanding of what should be done by
> the
> >> > time
> >> > > of
> >> > > > > 2.0
> >> > > > > >> > > >> release.
> >> > > > > >> > > >> > > > > > > That is very close goal for us, to verify
> IT
> >> > > monkey
> >> > > > > for
> >> > > > > >> > > >> existing
> >> > > > > >> > > >> > > > code.
> >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > >> > > > > > > * don't rely on things outside of HBase
> for
> >> > > normal
> >> > > > > >> > operation
> >> > > > > >> > > >> > (okay
> >> > > > > >> > > >> > > > for
> >> > > > > >> > > >> > > > > > > advanced operation)
> >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > >> > > > > > > We do not.
> >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > >> > > > > > > Enormous time has been spent already on
> the
> >> > > > > development
> >> > > > > >> > and
> >> > > > > >> > > >> > testing
> >> > > > > >> > > >> > > > the
> >> > > > > >> > > >> > > > > > > feature, it has passed our internal tests
> and
> >> > > many
> >> > > > > >> rounds
> >> > > > > >> > of
> >> > > > > >> > > >> code
> >> > > > > >> > > >> > > > > reviews
> >> > > > > >> > > >> > > > > > > by HBase committers. We do not mind if
> >> someone
> >> > > from
> >> > > > > >> HBase
> >> > > > > >> > > >> > community
> >> > > > > >> > > >> > > > > > > (outside of HW) will review the code, but
> it
> >> > will
> >> > > > > >> probably
> >> > > > > >> > > >> takes
> >> > > > > >> > > >> > > > > forever
> >> > > > > >> > > >> > > > > > to
> >> > > > > >> > > >> > > > > > > wait for volunteer?, the feature is quite
> >> large
> >> > > > (1MB+
> >> > > > > >> > > >> cumulative
> >> > > > > >> > > >> > > > patch)
> >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > >> > > > > > > 2.0 branch is full of half baked features,
> >> most
> >> > > of
> >> > > > > them
> >> > > > > >> > are
> >> > > > > >> > > in
> >> > > > > >> > > >> > > active
> >> > > > > >> > > >> > > > > > > development, therefore I am not following
> you
> >> > > here,
> >> > > > > >> Sean?
> >> > > > > >> > > Why
> >> > > > > >> > > >> > > > > HBASE-7912
> >> > > > > >> > > >> > > > > > is
> >> > > > > >> > > >> > > > > > > not good enough yet to be integrated into
> 2.0
> >> > > > branch?
> >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > >> > > > > > > -Vlad
> >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > >> > > > > > > On Wed, Sep 7, 2016 at 8:23 AM, Sean
> Busbey <
> >> > > > > >> > > busbey@apache.org
> >> > > > > >> > > >> >
> >> > > > > >> > > >> > > > wrote:
> >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > >> > > > > > > > On Tue, Sep 6, 2016 at 10:36 PM, Josh
> >> Elser <
> >> > > > > >> > > >> > > josh.elser@gmail.com>
> >> > > > > >> > > >> > > > > > > wrote:
> >> > > > > >> > > >> > > > > > > > > So, the answer to Sean's original
> >> question
> >> > is
> >> > > > "as
> >> > > > > >> > > robust as
> >> > > > > >> > > >> > > > > snapshots
> >> > > > > >> > > >> > > > > > > > > presently are"? (independence of
> >> > > backup/restore
> >> > > > > >> > failure
> >> > > > > >> > > >> > > tolerance
> >> > > > > >> > > >> > > > > > from
> >> > > > > >> > > >> > > > > > > > > snapshot failure tolerance)
> >> > > > > >> > > >> > > > > > > > >
> >> > > > > >> > > >> > > > > > > > > Is this just a question WRT context of
> >> the
> >> > > > > change,
> >> > > > > >> or
> >> > > > > >> > > is it
> >> > > > > >> > > >> > > means
> >> > > > > >> > > >> > > > > > for a
> >> > > > > >> > > >> > > > > > > > veto
> >> > > > > >> > > >> > > > > > > > > from you, Sean? Just trying to make
> sure
> >> > I'm
> >> > > > > >> following
> >> > > > > >> > > >> along
> >> > > > > >> > > >> > > > > > > adequately.
> >> > > > > >> > > >> > > > > > > > >
> >> > > > > >> > > >> > > > > > > > >
> >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > >> > > > > > > > I'd say ATM I'm -0, bordering on -1 but
> not
> >> > for
> >> > > > > >> reasons
> >> > > > > >> > I
> >> > > > > >> > > can
> >> > > > > >> > > >> > > > > > articulate
> >> > > > > >> > > >> > > > > > > > well.
> >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > >> > > > > > > > Here's an attempt.
> >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > >> > > > > > > > We've been trying to move, as a
> community,
> >> > > > towards
> >> > > > > >> > > minimizing
> >> > > > > >> > > >> > > risk
> >> > > > > >> > > >> > > > to
> >> > > > > >> > > >> > > > > > > > downstream folks by getting "complete
> >> enough
> >> > > for
> >> > > > > use"
> >> > > > > >> > > gates
> >> > > > > >> > > >> in
> >> > > > > >> > > >> > > > place
> >> > > > > >> > > >> > > > > > > > before we introduce new features. This
> was
> >> > > > spurred
> >> > > > > >> by a
> >> > > > > >> > > some
> >> > > > > >> > > >> > > > features
> >> > > > > >> > > >> > > > > > > > getting in half-baked and never making
> it
> >> to
> >> > > "can
> >> > > > > >> really
> >> > > > > >> > > use"
> >> > > > > >> > > >> > > > status
> >> > > > > >> > > >> > > > > > > > (I'm thinking of distributed log replay
> and
> >> > the
> >> > > > > >> zk-less
> >> > > > > >> > > >> > > assignment
> >> > > > > >> > > >> > > > > > > > stuff, I don't recall if there was
> more).
> >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > >> > > > > > > > The gates, generally, included things
> like:
> >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > >> > > > > > > > * have docs
> >> > > > > >> > > >> > > > > > > > * have sunny-day correctness tests
> >> > > > > >> > > >> > > > > > > > * have correctness-in-face-of-failure
> tests
> >> > > > > >> > > >> > > > > > > > * don't rely on things outside of HBase
> for
> >> > > > normal
> >> > > > > >> > > operation
> >> > > > > >> > > >> > > (okay
> >> > > > > >> > > >> > > > > for
> >> > > > > >> > > >> > > > > > > > advanced operation)
> >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > >> > > > > > > > As an example, we kept the MOB work off
> in
> >> a
> >> > > > branch
> >> > > > > >> and
> >> > > > > >> > > out
> >> > > > > >> > > >> of
> >> > > > > >> > > >> > > > master
> >> > > > > >> > > >> > > > > > > > until it could pass these criteria. The
> big
> >> > > > > exemption
> >> > > > > >> > > we've
> >> > > > > >> > > >> had
> >> > > > > >> > > >> > > to
> >> > > > > >> > > >> > > > > > > > this was the hbase-spark integration,
> where
> >> > we
> >> > > > all
> >> > > > > >> > agreed
> >> > > > > >> > > it
> >> > > > > >> > > >> > > could
> >> > > > > >> > > >> > > > > > > > land in master because it was very well
> >> > > isolated
> >> > > > > (the
> >> > > > > >> > > slide
> >> > > > > >> > > >> > away
> >> > > > > >> > > >> > > > from
> >> > > > > >> > > >> > > > > > > > including docs as a first-class part of
> >> > > building
> >> > > > up
> >> > > > > >> that
> >> > > > > >> > > >> > > > integration
> >> > > > > >> > > >> > > > > > > > has led me to doubt the wisdom of this
> >> > > decision).
> >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > >> > > > > > > > We've also been treating inclusion in a
> >> > > "probably
> >> > > > > >> will
> >> > > > > >> > be
> >> > > > > >> > > >> > > released
> >> > > > > >> > > >> > > > to
> >> > > > > >> > > >> > > > > > > > downstream" branches as a higher bar,
> >> > requiring
> >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > >> > > > > > > > * don't moderately impact performance
> when
> >> > the
> >> > > > > >> feature
> >> > > > > >> > > isn't
> >> > > > > >> > > >> in
> >> > > > > >> > > >> > > use
> >> > > > > >> > > >> > > > > > > > * don't severely impact performance when
> >> the
> >> > > > > feature
> >> > > > > >> is
> >> > > > > >> > in
> >> > > > > >> > > >> use
> >> > > > > >> > > >> > > > > > > > * either default-to-on or show enough
> >> demand
> >> > to
> >> > > > > >> believe
> >> > > > > >> > a
> >> > > > > >> > > >> > > > non-trivial
> >> > > > > >> > > >> > > > > > > > number of folks will turn the feature on
> >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > >> > > > > > > > The above has kept MOB and hbase-spark
> >> > > > integration
> >> > > > > >> out
> >> > > > > >> > of
> >> > > > > >> > > >> > > branch-1,
> >> > > > > >> > > >> > > > > > > > presumably while they've "gotten more
> >> stable"
> >> > > in
> >> > > > > >> master
> >> > > > > >> > > from
> >> > > > > >> > > >> > the
> >> > > > > >> > > >> > > > odd
> >> > > > > >> > > >> > > > > > > > vendor inclusion.
> >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > >> > > > > > > > Are we going to have a 2.0 release
> before
> >> the
> >> > > end
> >> > > > > of
> >> > > > > >> the
> >> > > > > >> > > >> year?
> >> > > > > >> > > >> > > > We're
> >> > > > > >> > > >> > > > > > > > coming up on 1.5 years since the
> release of
> >> > > > version
> >> > > > > >> 1.0;
> >> > > > > >> > > >> seems
> >> > > > > >> > > >> > > like
> >> > > > > >> > > >> > > > > > > > it's about time, though I haven't seen
> any
> >> > > > concrete
> >> > > > > >> > plans
> >> > > > > >> > > >> this
> >> > > > > >> > > >> > > > year.
> >> > > > > >> > > >> > > > > > > > Presuming we are going to have one by
> the
> >> end
> >> > > of
> >> > > > > the
> >> > > > > >> > > year, it
> >> > > > > >> > > >> > > > seems a
> >> > > > > >> > > >> > > > > > > > bit close to still be adding in
> "features
> >> > that
> >> > > > need
> >> > > > > >> > > maturing"
> >> > > > > >> > > >> > on
> >> > > > > >> > > >> > > > the
> >> > > > > >> > > >> > > > > > > > branch.
> >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > >> > > > > > > > The lack of a concrete plan for 2.0
> keeps
> >> me
> >> > > from
> >> > > > > >> > > considering
> >> > > > > >> > > >> > > these
> >> > > > > >> > > >> > > > > > > > things blocker at the moment. But I know
> >> > first
> >> > > > hand
> >> > > > > >> how
> >> > > > > >> > > much
> >> > > > > >> > > >> > > > trouble
> >> > > > > >> > > >> > > > > > > > folks have had with other features that
> >> have
> >> > > gone
> >> > > > > >> into
> >> > > > > >> > > >> > downstream
> >> > > > > >> > > >> > > > > > > > facing releases without robustness
> checks
> >> > (i.e.
> >> > > > > >> > > replication),
> >> > > > > >> > > >> > and
> >> > > > > >> > > >> > > > I'm
> >> > > > > >> > > >> > > > > > > > concerned about what we're setting up if
> >> 2.0
> >> > > goes
> >> > > > > out
> >> > > > > >> > with
> >> > > > > >> > > >> this
> >> > > > > >> > > >> > > > > > > > feature in its current state.
> >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > >> > > > > >
> >> > > > > >> > > >> > > > >
> >> > > > > >> > > >> > > > >
> >> > > > > >> > > >> > > > >
> >> > > > > >> > > >> > > > > --
> >> > > > > >> > > >> > > > > Best regards,
> >> > > > > >> > > >> > > > >
> >> > > > > >> > > >> > > > >    - Andy
> >> > > > > >> > > >> > > > >
> >> > > > > >> > > >> > > > > Problems worthy of attack prove their worth by
> >> > > hitting
> >> > > > > >> back. -
> >> > > > > >> > > Piet
> >> > > > > >> > > >> > > Hein
> >> > > > > >> > > >> > > > > (via Tom White)
> >> > > > > >> > > >> > > > >
> >> > > > > >> > > >> > > >
> >> > > > > >> > > >> > >
> >> > > > > >> > > >> > >
> >> > > > > >> > > >> > >
> >> > > > > >> > > >> > > --
> >> > > > > >> > > >> > > Best regards,
> >> > > > > >> > > >> > >
> >> > > > > >> > > >> > >    - Andy
> >> > > > > >> > > >> > >
> >> > > > > >> > > >> > > Problems worthy of attack prove their worth by
> >> hitting
> >> > > > back.
> >> > > > > -
> >> > > > > >> > Piet
> >> > > > > >> > > >> Hein
> >> > > > > >> > > >> > > (via Tom White)
> >> > > > > >> > > >> > >
> >> > > > > >> > > >> >
> >> > > > > >> > > >>
> >> > > > > >> > >
> >> > > > > >> >
> >> > > > > >>
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
>
>
>
> --
> busbey
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Enis Söztutar <en...@apache.org>]

> A vote thread can be started if there is no further comment.
Agreed.

Devs, please raise any remaining concerns, if any, now. Otherwise, let's
start the vote in 2 days.

Enis

On Mon, Oct 10, 2016 at 2:46 PM, Ted Yu <yu...@gmail.com> wrote:

> bq. Does the branch ready for merge
>
> Yes - see the mega patch.
>
> We can discuss the actual procedure of merging if the community gives green
> light to merging.
>
> A vote thread can be started if there is no further comment.
>
> On Mon, Oct 10, 2016 at 2:42 PM, Enis Söztutar <en...@apache.org> wrote:
>
> > Great work.
> >
> > So what are the next steps? It seems from the discussion that all
> > outstanding requests have been addressed. Does the branch ready for merge
> > or, you have to rebase the branch as well?
> >
> > Time to start a merge vote?
> > Enis
> >
> > On Mon, Oct 10, 2016 at 1:44 PM, Vladimir Rodionov <
> vladrodionov@gmail.com
> > >
> > wrote:
> >
> > > >> mapreduce dependency has been moved to client side - no mapreduce
> job
> > >
> > > 1. We have no code in the client module anymore, due to dependency on
> > > internal server API (HFile and WAL access).
> > > 2. Backup/ restore are client - driven operations, but all the code
> > resides
> > > in the server module
> > > 3. No MR in Master, no procedure - driven execution.
> > > 4. Old good MR from command-line.
> > > 5. Security was simplified and now only super-user is allowed to run
> > > backup/restores.
> > > 6. HBase Backup API was gone due to 1. Now only command-line access to
> > > backup tools.
> > >
> > > These consequences of refactoring has been discussed in HBASE-16727.
> > >
> > > -Vlad
> > >
> > >
> > >
> > >
> > > On Mon, Oct 10, 2016 at 1:31 PM, Ted Yu <yu...@gmail.com> wrote:
> > >
> > > > Reviving this thread.
> > > >
> > > > The following has taken place:
> > > >
> > > > mapreduce dependency has been moved to client side - no mapreduce job
> > > > launched from master or region server.
> > > > document patch (HBASE-16574) has been integrated.
> > > > Updated mega patch has been attached to HBASE-14123: this covers the
> > > > refactor in #1 above and the protobuf 3 merge.
> > > >
> > > > If community has more feedback on the merge proposal, I would love to
> > > hear
> > > > it.
> > > >
> > > > Thanks
> > > >
> > > > On Thu, Sep 22, 2016 at 10:31 AM, Sean Busbey <bu...@cloudera.com>
> > > wrote:
> > > >
> > > > > I'd like to see the docs proposed on HBASE-16574 integrated into
> our
> > > > > project's documentation prior to merge.
> > > > >
> > > > > On Thu, Sep 22, 2016 at 9:02 AM, Ted Yu <yu...@gmail.com>
> wrote:
> > > > > > This feature can be marked experimental due to some limitations
> > such
> > > as
> > > > > > security.
> > > > > >
> > > > > > Your previous round of comments have been addressed.
> > > > > > Command line tool has gone through:
> > > > > >
> > > > > > HBASE-16620 Fix backup command-line tool usability issues
> > > > > > HBASE-16655 hbase backup describe with incorrect backup id
> results
> > in
> > > > NPE
> > > > > >
> > > > > > The updated doc has been attached to HBASE-16574.
> > > > > >
> > > > > > Cheers
> > > > > >
> > > > > > On Thu, Sep 22, 2016 at 8:53 AM, Stack <st...@duboce.net> wrote:
> > > > > >
> > > > > >> On Wed, Sep 21, 2016 at 7:43 AM, Ted Yu <yu...@gmail.com>
> > > wrote:
> > > > > >>
> > > > > >> > Are there more (review) comments ?
> > > > > >> >
> > > > > >> >
> > > > > >> Are outstanding comments addressed?
> > > > > >>
> > > > > >> I don't see answer to my 'is this experimental/will it be marked
> > > > > >> experimental' question.
> > > > > >>
> > > > > >> I ran into some issues trying to use the feature and suggested
> > that
> > > a
> > > > > >> feature likes this needs polish else it'll just rot, unused. Has
> > > > polish
> > > > > >> been applied? All ready for another 'user' test? Suggest that
> you
> > > > update
> > > > > >> here going forward for the benefit of those trying to follow
> along
> > > and
> > > > > who
> > > > > >> are not watching JIRA change fly-by.
> > > > > >>
> > > > > >> It looks like doc got a revision -- I have to check -- to take
> on
> > > > > >> suggestion made above but again, suggest, that this thread gets
> > > > updated.
> > > > > >>
> > > > > >> Thanks,
> > > > > >> St.Ack
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> > Thanks
> > > > > >> >
> > > > > >> > On Tue, Sep 20, 2016 at 10:02 AM, Devaraj Das <
> > > ddas@hortonworks.com
> > > > >
> > > > > >> > wrote:
> > > > > >> >
> > > > > >> > > Just reviving this thread. Thanks Sean, Stack, Dima, and
> > others
> > > > for
> > > > > the
> > > > > >> > > thorough reviews and testing. Thanks Ted and Vlad for taking
> > > care
> > > > of
> > > > > >> the
> > > > > >> > > feedback. Are we all good to do the merge now? Rather do
> > sooner
> > > > than
> > > > > >> > later.
> > > > > >> > > ________________________________________
> > > > > >> > > From: saint.ack@gmail.com <sa...@gmail.com> on behalf
> of
> > > > Stack
> > > > > <
> > > > > >> > > stack@duboce.net>
> > > > > >> > > Sent: Monday, September 12, 2016 1:18 PM
> > > > > >> > > To: HBase Dev List
> > > > > >> > > Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch
> > > > HBASE-7912
> > > > > >> > >
> > > > > >> > > On Mon, Sep 12, 2016 at 12:19 PM, Ted Yu <
> yuzhihong@gmail.com
> > >
> > > > > wrote:
> > > > > >> > >
> > > > > >> > > > Mega patch (rev 18) is on HBASE-14123.
> > > > > >> > > >
> > > > > >> > > > Please comment on HBASE-14123 on how you want to review.
> > > > > >> > > >
> > > > > >> > >
> > > > > >> > >
> > > > > >> > > Yeah. That was my lost tab. Last rb was 6 months ago.
> Suggest
> > > > > updating
> > > > > >> > it.
> > > > > >> > > RB is pretty good for review. Patch is only 1.5M so should
> be
> > > > fine.
> > > > > >> > >
> > > > > >> > > St.Ack
> > > > > >> > >
> > > > > >> > >
> > > > > >> > > >
> > > > > >> > > > Thanks
> > > > > >> > > >
> > > > > >> > > > On Mon, Sep 12, 2016 at 12:15 PM, Stack <stack@duboce.net
> >
> > > > wrote:
> > > > > >> > > >
> > > > > >> > > > > On review of the 'patch', do I just compare the branch
> to
> > > > > master or
> > > > > >> > is
> > > > > >> > > > > there a megapatch posted somewhere (I think I saw one
> but
> > it
> > > > > seemed
> > > > > >> > > stale
> > > > > >> > > > > and then I 'lost' the tab). Sorry for dumb question.
> > > > > >> > > > > St.Ack
> > > > > >> > > > >
> > > > > >> > > > > On Mon, Sep 12, 2016 at 12:01 PM, Stack <
> stack@duboce.net
> > >
> > > > > wrote:
> > > > > >> > > > >
> > > > > >> > > > > > Late to the game. A few comments after rereading this
> > > thread
> > > > > as a
> > > > > >> > > > 'user'.
> > > > > >> > > > > >
> > > > > >> > > > > > + Before merge, a user-facing feature like this should
> > > work
> > > > > (If
> > > > > >> > this
> > > > > >> > > is
> > > > > >> > > > > "higher-bar
> > > > > >> > > > > > for new features", bring it on -- smile).
> > > > > >> > > > > > + As a user, I tried the branch with tools after
> > reviewing
> > > > the
> > > > > >> > > > > just-posted
> > > > > >> > > > > > doc. I had an 'interesting' experience (left comments
> up
> > > on
> > > > > >> > issue). I
> > > > > >> > > > > think
> > > > > >> > > > > > the tooling/doc. important to get right. If it breaks
> > > easily
> > > > > or
> > > > > >> is
> > > > > >> > > > > > inconsistent (or lacks 'polish'), operators will judge
> > the
> > > > > whole
> > > > > >> > > > > > backup/restore tooling chain as not trustworthy and
> > > abandon
> > > > > it.
> > > > > >> > Lets
> > > > > >> > > > not
> > > > > >> > > > > > have this happen to this feature.
> > > > > >> > > > > > + Matteo's suggestion (with a helpful starter list)
> that
> > > > there
> > > > > >> > needs
> > > > > >> > > to
> > > > > >> > > > > be
> > > > > >> > > > > > explicit qualification on what is actually being
> > delivered
> > > > --
> > > > > >> > > > including a
> > > > > >> > > > > > listing of limitations (some look serious such as data
> > > bleed
> > > > > from
> > > > > >> > > other
> > > > > >> > > > > > regions in WALs, but maybe I don't care for my use
> > > case...)
> > > > --
> > > > > >> > needs
> > > > > >> > > to
> > > > > >> > > > > > accompany the merge. Lets fold them into the user doc.
> > in
> > > > the
> > > > > >> > > technical
> > > > > >> > > > > > overview area as suggested so user expectations are
> > > properly
> > > > > >> > managed
> > > > > >> > > > > > (otherwise, they expect the world and will just give
> up
> > > when
> > > > > we
> > > > > >> > fall
> > > > > >> > > > > > short). Vladimir did a list of what is in each of the
> > > phases
> > > > > >> above
> > > > > >> > > > which
> > > > > >> > > > > > would serve as a good start.
> > > > > >> > > > > > + Is this feature 'experimental' (Matteo asks above).
> > I'd
> > > > > prefer
> > > > > >> it
> > > > > >> > > is
> > > > > >> > > > > > not. If it is, it should be labelled all over that it
> is
> > > > so. I
> > > > > >> see
> > > > > >> > > > > current
> > > > > >> > > > > > state called out as a '... technical preview feature'.
> > > Does
> > > > > this
> > > > > >> > mean
> > > > > >> > > > > > not-for-users?
> > > > > >> > > > > >
> > > > > >> > > > > > St.Ack
> > > > > >> > > > > >
> > > > > >> > > > > >
> > > > > >> > > > > >
> > > > > >> > > > > >
> > > > > >> > > > > >
> > > > > >> > > > > >
> > > > > >> > > > > >
> > > > > >> > > > > >
> > > > > >> > > > > >
> > > > > >> > > > > >
> > > > > >> > > > > >
> > > > > >> > > > > > On Mon, Sep 12, 2016 at 8:03 AM, Ted Yu <
> > > > yuzhihong@gmail.com>
> > > > > >> > wrote:
> > > > > >> > > > > >
> > > > > >> > > > > >> Sean:
> > > > > >> > > > > >> Do you have more comments ?
> > > > > >> > > > > >>
> > > > > >> > > > > >> Cheers
> > > > > >> > > > > >>
> > > > > >> > > > > >> On Fri, Sep 9, 2016 at 1:42 PM, Vladimir Rodionov <
> > > > > >> > > > > vladrodionov@gmail.com
> > > > > >> > > > > >> >
> > > > > >> > > > > >> wrote:
> > > > > >> > > > > >>
> > > > > >> > > > > >> > Sean,
> > > > > >> > > > > >> >
> > > > > >> > > > > >> > Backup/Restore can fail due to various reasons:
> > network
> > > > > outage
> > > > > >> > > > > (cluster
> > > > > >> > > > > >> > wide), various time-outs in HBase and HDFS layer,
> M/R
> > > > > failure
> > > > > >> > due
> > > > > >> > > to
> > > > > >> > > > > >> "HDFS
> > > > > >> > > > > >> > exceeded quota", user error (manual deletion of
> data)
> > > and
> > > > > so
> > > > > >> on
> > > > > >> > so
> > > > > >> > > > on.
> > > > > >> > > > > >> That
> > > > > >> > > > > >> > is impossible to enumerate all possible types of
> > > failures
> > > > > in a
> > > > > >> > > > > >> distributed
> > > > > >> > > > > >> > system - that is not our goal/task.
> > > > > >> > > > > >> >
> > > > > >> > > > > >> > We focus completely on backup system table
> > consistency
> > > > in a
> > > > > >> > > presence
> > > > > >> > > > > of
> > > > > >> > > > > >> any
> > > > > >> > > > > >> > type of failure. That is what I call "tolerance to
> > > > > failures".
> > > > > >> > > > > >> >
> > > > > >> > > > > >> > On a failure:
> > > > > >> > > > > >> >
> > > > > >> > > > > >> > BACKUP. All backup system information (prior to
> > backup)
> > > > > will
> > > > > >> be
> > > > > >> > > > > restored
> > > > > >> > > > > >> > and all temporary data, related to a failed
> session,
> > in
> > > > > HDFS
> > > > > >> > will
> > > > > >> > > be
> > > > > >> > > > > >> > deleted
> > > > > >> > > > > >> > RESTORE. We do not care about system data, because
> > > > restore
> > > > > >> does
> > > > > >> > > not
> > > > > >> > > > > >> change
> > > > > >> > > > > >> > it. Temporary data in HDFS will be cleaned up and
> > table
> > > > > will
> > > > > >> be
> > > > > >> > > in a
> > > > > >> > > > > >> state
> > > > > >> > > > > >> > back to where it was before operation started.
> > > > > >> > > > > >> >
> > > > > >> > > > > >> > This is what user should expect in case of a
> failure.
> > > > > >> > > > > >> >
> > > > > >> > > > > >> > -Vlad
> > > > > >> > > > > >> >
> > > > > >> > > > > >> >
> > > > > >> > > > > >> > -Vlad
> > > > > >> > > > > >> >
> > > > > >> > > > > >> > On Fri, Sep 9, 2016 at 12:56 PM, Sean Busbey <
> > > > > >> busbey@apache.org
> > > > > >> > >
> > > > > >> > > > > wrote:
> > > > > >> > > > > >> >
> > > > > >> > > > > >> > > Failing in a consistent way, with docs that
> explain
> > > the
> > > > > >> > various
> > > > > >> > > > > >> > > expected failures would be sufficient.
> > > > > >> > > > > >> > >
> > > > > >> > > > > >> > > On Fri, Sep 9, 2016 at 12:16 PM, Vladimir
> Rodionov
> > > > > >> > > > > >> > > <vl...@gmail.com> wrote:
> > > > > >> > > > > >> > > > Do not worry Sean, doc is coming today as a
> > preview
> > > > and
> > > > > >> our
> > > > > >> > > > writer
> > > > > >> > > > > >> > Frank
> > > > > >> > > > > >> > > > will be working on a putting  it into Apache
> > repo.
> > > > > >> Timeline
> > > > > >> > > > > depends
> > > > > >> > > > > >> on
> > > > > >> > > > > >> > > > Franks schedule but I hope we will get it
> rather
> > > > sooner
> > > > > >> than
> > > > > >> > > > > later.
> > > > > >> > > > > >> > > >
> > > > > >> > > > > >> > > > As for failure testing, we are focusing only
> on a
> > > > > >> consistent
> > > > > >> > > > state
> > > > > >> > > > > >> of
> > > > > >> > > > > >> > > > backup system data in a presence of any type of
> > > > > failures,
> > > > > >> We
> > > > > >> > > are
> > > > > >> > > > > not
> > > > > >> > > > > >> > > going
> > > > > >> > > > > >> > > > to implement  anything more "fancy", than that.
> > We
> > > > > allow
> > > > > >> > both:
> > > > > >> > > > > >> backup
> > > > > >> > > > > >> > and
> > > > > >> > > > > >> > > > restore to fail. What we do not allow is to
> have
> > > > system
> > > > > >> data
> > > > > >> > > > > >> corrupted.
> > > > > >> > > > > >> > > > Will it suffice for you? Do you have any other
> > > > > concerns,
> > > > > >> you
> > > > > >> > > > want
> > > > > >> > > > > >> us to
> > > > > >> > > > > >> > > > address?
> > > > > >> > > > > >> > > >
> > > > > >> > > > > >> > > > -Vlad
> > > > > >> > > > > >> > > >
> > > > > >> > > > > >> > > >
> > > > > >> > > > > >> > > > On Fri, Sep 9, 2016 at 10:56 AM, Sean Busbey <
> > > > > >> > > busbey@apache.org
> > > > > >> > > > >
> > > > > >> > > > > >> > wrote:
> > > > > >> > > > > >> > > >
> > > > > >> > > > > >> > > >> "docs will come to Apache soon" does not
> address
> > > my
> > > > > >> concern
> > > > > >> > > > > around
> > > > > >> > > > > >> > docs
> > > > > >> > > > > >> > > at
> > > > > >> > > > > >> > > >> all, unless said docs have already made it
> into
> > > the
> > > > > >> project
> > > > > >> > > > > repo. I
> > > > > >> > > > > >> > > don't
> > > > > >> > > > > >> > > >> want third party resources for using a major
> and
> > > > > >> important
> > > > > >> > > > > feature
> > > > > >> > > > > >> of
> > > > > >> > > > > >> > > the
> > > > > >> > > > > >> > > >> project, I want us to provide end users with
> > what
> > > > they
> > > > > >> need
> > > > > >> > > to
> > > > > >> > > > > get
> > > > > >> > > > > >> the
> > > > > >> > > > > >> > > job
> > > > > >> > > > > >> > > >> done.
> > > > > >> > > > > >> > > >>
> > > > > >> > > > > >> > > >> I see some calls for patience on the failure
> > > > testing,
> > > > > but
> > > > > >> > the
> > > > > >> > > > > >> appeal
> > > > > >> > > > > >> > to
> > > > > >> > > > > >> > > us
> > > > > >> > > > > >> > > >> having done a bad job of requiring proper
> tests
> > of
> > > > > >> previous
> > > > > >> > > > > >> features
> > > > > >> > > > > >> > > just
> > > > > >> > > > > >> > > >> makes me more concerned about not getting them
> > > > here. I
> > > > > >> > don't
> > > > > >> > > > want
> > > > > >> > > > > >> to
> > > > > >> > > > > >> > set
> > > > > >> > > > > >> > > >> yet another bad example that will then be
> > pointed
> > > to
> > > > > in
> > > > > >> the
> > > > > >> > > > > future.
> > > > > >> > > > > >> > > >>
> > > > > >> > > > > >> > > >> On Sep 8, 2016 10:50, "Ted Yu" <
> > > yuzhihong@gmail.com
> > > > >
> > > > > >> > wrote:
> > > > > >> > > > > >> > > >>
> > > > > >> > > > > >> > > >> > Is there any concern which is not addressed
> ?
> > > > > >> > > > > >> > > >> >
> > > > > >> > > > > >> > > >> > Do we need another Vote thread ?
> > > > > >> > > > > >> > > >> >
> > > > > >> > > > > >> > > >> > Thanks
> > > > > >> > > > > >> > > >> >
> > > > > >> > > > > >> > > >> > On Thu, Sep 8, 2016 at 9:21 AM, Andrew
> > Purtell <
> > > > > >> > > > > >> apurtell@apache.org
> > > > > >> > > > > >> > >
> > > > > >> > > > > >> > > >> > wrote:
> > > > > >> > > > > >> > > >> >
> > > > > >> > > > > >> > > >> > > Vlad,
> > > > > >> > > > > >> > > >> > >
> > > > > >> > > > > >> > > >> > > I apologize for using the term
> 'half-baked'
> > > in a
> > > > > way
> > > > > >> > that
> > > > > >> > > > > could
> > > > > >> > > > > >> > > seem a
> > > > > >> > > > > >> > > >> > > description of HBASE-7912. I meant that
> as a
> > > > > general
> > > > > >> > > > > >> hypothetical.
> > > > > >> > > > > >> > > >> > >
> > > > > >> > > > > >> > > >> > > On Wed, Sep 7, 2016 at 9:36 AM, Vladimir
> > > > Rodionov
> > > > > <
> > > > > >> > > > > >> > > >> > vladrodionov@gmail.com>
> > > > > >> > > > > >> > > >> > > wrote:
> > > > > >> > > > > >> > > >> > >
> > > > > >> > > > > >> > > >> > > > >> I'm not sure that "There is already
> > lots
> > > of
> > > > > >> > > half-baked
> > > > > >> > > > > >> code
> > > > > >> > > > > >> > in
> > > > > >> > > > > >> > > the
> > > > > >> > > > > >> > > >> > > > branch,
> > > > > >> > > > > >> > > >> > > > so what's the harm in adding more?"
> > > > > >> > > > > >> > > >> > > >
> > > > > >> > > > > >> > > >> > > > I meant - not production - ready yet.
> This
> > > is
> > > > > 2.0
> > > > > >> > > > > development
> > > > > >> > > > > >> > > branch
> > > > > >> > > > > >> > > >> > and,
> > > > > >> > > > > >> > > >> > > > hence many features are in works,
> > > > > >> > > > > >> > > >> > > > not being tested well etc. I do not
> > consider
> > > > > backup
> > > > > >> > as
> > > > > >> > > > half
> > > > > >> > > > > >> > baked
> > > > > >> > > > > >> > > >> > > feature -
> > > > > >> > > > > >> > > >> > > > it has passed our internal QA and has
> very
> > > > good
> > > > > >> doc,
> > > > > >> > > > which
> > > > > >> > > > > we
> > > > > >> > > > > >> > will
> > > > > >> > > > > >> > > >> > > provide
> > > > > >> > > > > >> > > >> > > > to Apache shortly.
> > > > > >> > > > > >> > > >> > > >
> > > > > >> > > > > >> > > >> > > > -Vlad
> > > > > >> > > > > >> > > >> > > >
> > > > > >> > > > > >> > > >> > > > On Wed, Sep 7, 2016 at 9:13 AM, Andrew
> > > > Purtell <
> > > > > >> > > > > >> > > apurtell@apache.org>
> > > > > >> > > > > >> > > >> > > > wrote:
> > > > > >> > > > > >> > > >> > > >
> > > > > >> > > > > >> > > >> > > > > We shouldn't admit half baked changes
> > that
> > > > > won't
> > > > > >> be
> > > > > >> > > > > >> finished.
> > > > > >> > > > > >> > > >> However
> > > > > >> > > > > >> > > >> > > in
> > > > > >> > > > > >> > > >> > > > > this case the crew working on this
> > feature
> > > > are
> > > > > >> long
> > > > > >> > > > > timers
> > > > > >> > > > > >> and
> > > > > >> > > > > >> > > less
> > > > > >> > > > > >> > > >> > > > likely
> > > > > >> > > > > >> > > >> > > > > than just about anyone to leave
> > something
> > > > in a
> > > > > >> half
> > > > > >> > > > baked
> > > > > >> > > > > >> > > state. Of
> > > > > >> > > > > >> > > >> > > > course
> > > > > >> > > > > >> > > >> > > > > there is no guarantee how anything
> will
> > > turn
> > > > > out,
> > > > > >> > > but I
> > > > > >> > > > > am
> > > > > >> > > > > >> > > willing
> > > > > >> > > > > >> > > >> to
> > > > > >> > > > > >> > > >> > > > take
> > > > > >> > > > > >> > > >> > > > > a little on faith if they feel their
> > best
> > > > path
> > > > > >> > > forward
> > > > > >> > > > > now
> > > > > >> > > > > >> is
> > > > > >> > > > > >> > to
> > > > > >> > > > > >> > > >> > merge
> > > > > >> > > > > >> > > >> > > to
> > > > > >> > > > > >> > > >> > > > > trunk. I only wish I had bandwidth to
> > have
> > > > > done
> > > > > >> > some
> > > > > >> > > > real
> > > > > >> > > > > >> > > kicking
> > > > > >> > > > > >> > > >> of
> > > > > >> > > > > >> > > >> > > the
> > > > > >> > > > > >> > > >> > > > > tires by now. Maybe this week.
> > > > > >> > > > > >> > > >> > > > >
> > > > > >> > > > > >> > > >> > > > > (Yes, I'm using some of that time for
> > this
> > > > > email
> > > > > >> > :-)
> > > > > >> > > > but
> > > > > >> > > > > I
> > > > > >> > > > > >> > type
> > > > > >> > > > > >> > > >> > fast.)
> > > > > >> > > > > >> > > >> > > > >
> > > > > >> > > > > >> > > >> > > > > That said, I would like to agitate for
> > > > making
> > > > > 2.0
> > > > > >> > > more
> > > > > >> > > > > real
> > > > > >> > > > > >> > and
> > > > > >> > > > > >> > > >> spend
> > > > > >> > > > > >> > > >> > > > some
> > > > > >> > > > > >> > > >> > > > > time on it now that I'm winding down
> > with
> > > > > 0.98. I
> > > > > >> > > think
> > > > > >> > > > > >> that
> > > > > >> > > > > >> > > means
> > > > > >> > > > > >> > > >> > > > > branching for 2.0 real soon now and
> even
> > > > > evicting
> > > > > >> > > > things
> > > > > >> > > > > >> from
> > > > > >> > > > > >> > > 2.0
> > > > > >> > > > > >> > > >> > > branch
> > > > > >> > > > > >> > > >> > > > > that aren't finished or stable,
> leaving
> > > them
> > > > > only
> > > > > >> > > once
> > > > > >> > > > > >> again
> > > > > >> > > > > >> > in
> > > > > >> > > > > >> > > the
> > > > > >> > > > > >> > > >> > > > master
> > > > > >> > > > > >> > > >> > > > > branch. Or, maybe just evicting them.
> > > Let's
> > > > > take
> > > > > >> it
> > > > > >> > > > case
> > > > > >> > > > > by
> > > > > >> > > > > >> > > case.
> > > > > >> > > > > >> > > >> > > > >
> > > > > >> > > > > >> > > >> > > > > I think this feature can come in
> > > relatively
> > > > > >> safely.
> > > > > >> > > As
> > > > > >> > > > > >> added
> > > > > >> > > > > >> > > >> > insurance,
> > > > > >> > > > > >> > > >> > > > > let's admit the possibility it could
> be
> > > > > reverted
> > > > > >> on
> > > > > >> > > the
> > > > > >> > > > > 2.0
> > > > > >> > > > > >> > > branch
> > > > > >> > > > > >> > > >> if
> > > > > >> > > > > >> > > >> > > > folks
> > > > > >> > > > > >> > > >> > > > > working on stabilizing 2.0 decide to
> > evict
> > > > it
> > > > > >> > because
> > > > > >> > > > it
> > > > > >> > > > > is
> > > > > >> > > > > >> > > >> > unfinished
> > > > > >> > > > > >> > > >> > > or
> > > > > >> > > > > >> > > >> > > > > unstable, because that certainly can
> > > > happen. I
> > > > > >> > would
> > > > > >> > > > > >> expect if
> > > > > >> > > > > >> > > talk
> > > > > >> > > > > >> > > >> > > like
> > > > > >> > > > > >> > > >> > > > > that starts, we'd get help finishing
> or
> > > > > >> stabilizing
> > > > > >> > > > > what's
> > > > > >> > > > > >> > under
> > > > > >> > > > > >> > > >> > > > discussion
> > > > > >> > > > > >> > > >> > > > > for revert. Or, we'd have a revert.
> > Either
> > > > way
> > > > > >> the
> > > > > >> > > > > outcome
> > > > > >> > > > > >> is
> > > > > >> > > > > >> > > >> > > acceptable.
> > > > > >> > > > > >> > > >> > > > >
> > > > > >> > > > > >> > > >> > > > >
> > > > > >> > > > > >> > > >> > > > > On Wed, Sep 7, 2016 at 8:56 AM, Dima
> > > Spivak
> > > > <
> > > > > >> > > > > >> > > dimaspivak@apache.org
> > > > > >> > > > > >> > > >> >
> > > > > >> > > > > >> > > >> > > > wrote:
> > > > > >> > > > > >> > > >> > > > >
> > > > > >> > > > > >> > > >> > > > > > I'm not sure that "There is already
> > lots
> > > > of
> > > > > >> > > > half-baked
> > > > > >> > > > > >> code
> > > > > >> > > > > >> > in
> > > > > >> > > > > >> > > >> the
> > > > > >> > > > > >> > > >> > > > > branch,
> > > > > >> > > > > >> > > >> > > > > > so what's the harm in adding more?"
> > is a
> > > > > good
> > > > > >> > code
> > > > > >> > > > > commit
> > > > > >> > > > > >> > > >> > philosophy
> > > > > >> > > > > >> > > >> > > > for
> > > > > >> > > > > >> > > >> > > > > a
> > > > > >> > > > > >> > > >> > > > > > fault-tolerant distributed data
> store.
> > > ;)
> > > > > >> > > > > >> > > >> > > > > >
> > > > > >> > > > > >> > > >> > > > > > More seriously, a lack of test
> > coverage
> > > > for
> > > > > >> > > existing
> > > > > >> > > > > >> > features
> > > > > >> > > > > >> > > >> > > shouldn't
> > > > > >> > > > > >> > > >> > > > > be
> > > > > >> > > > > >> > > >> > > > > > used as justification for
> introducing
> > > new
> > > > > >> > features
> > > > > >> > > > with
> > > > > >> > > > > >> the
> > > > > >> > > > > >> > > same
> > > > > >> > > > > >> > > >> > > > > > shortcomings. Ultimately, it's the
> end
> > > > user
> > > > > who
> > > > > >> > > will
> > > > > >> > > > > feel
> > > > > >> > > > > >> > the
> > > > > >> > > > > >> > > >> pain,
> > > > > >> > > > > >> > > >> > > so
> > > > > >> > > > > >> > > >> > > > > > shouldn't we do everything we can to
> > > > > mitigate
> > > > > >> > that?
> > > > > >> > > > > >> > > >> > > > > >
> > > > > >> > > > > >> > > >> > > > > > -Dima
> > > > > >> > > > > >> > > >> > > > > >
> > > > > >> > > > > >> > > >> > > > > > On Wed, Sep 7, 2016 at 8:46 AM,
> > Vladimir
> > > > > >> > Rodionov <
> > > > > >> > > > > >> > > >> > > > > vladrodionov@gmail.com>
> > > > > >> > > > > >> > > >> > > > > > wrote:
> > > > > >> > > > > >> > > >> > > > > >
> > > > > >> > > > > >> > > >> > > > > > > Sean,
> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > * have docs
> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > Agree. We have a doc and backup is
> > the
> > > > > most
> > > > > >> > > > > documented
> > > > > >> > > > > >> > > feature
> > > > > >> > > > > >> > > >> > :),
> > > > > >> > > > > >> > > >> > > we
> > > > > >> > > > > >> > > >> > > > > > will
> > > > > >> > > > > >> > > >> > > > > > > release it shortly to Apache.
> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > * have sunny-day correctness tests
> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > Feature has  close to 60 test
> cases,
> > > > which
> > > > > >> run
> > > > > >> > > for
> > > > > >> > > > > >> approx
> > > > > >> > > > > >> > 30
> > > > > >> > > > > >> > > >> min.
> > > > > >> > > > > >> > > >> > > We
> > > > > >> > > > > >> > > >> > > > > can
> > > > > >> > > > > >> > > >> > > > > > > add more, if community do not mind
> > :)
> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > * have
> > correctness-in-face-of-failure
> > > > > tests
> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > Any examples of these tests in
> > > existing
> > > > > >> > features?
> > > > > >> > > > In
> > > > > >> > > > > >> > works,
> > > > > >> > > > > >> > > we
> > > > > >> > > > > >> > > >> > > have a
> > > > > >> > > > > >> > > >> > > > > > clear
> > > > > >> > > > > >> > > >> > > > > > > understanding of what should be
> done
> > > by
> > > > > the
> > > > > >> > time
> > > > > >> > > of
> > > > > >> > > > > 2.0
> > > > > >> > > > > >> > > >> release.
> > > > > >> > > > > >> > > >> > > > > > > That is very close goal for us, to
> > > > verify
> > > > > IT
> > > > > >> > > monkey
> > > > > >> > > > > for
> > > > > >> > > > > >> > > >> existing
> > > > > >> > > > > >> > > >> > > > code.
> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > * don't rely on things outside of
> > > HBase
> > > > > for
> > > > > >> > > normal
> > > > > >> > > > > >> > operation
> > > > > >> > > > > >> > > >> > (okay
> > > > > >> > > > > >> > > >> > > > for
> > > > > >> > > > > >> > > >> > > > > > > advanced operation)
> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > We do not.
> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > Enormous time has been spent
> already
> > > on
> > > > > the
> > > > > >> > > > > development
> > > > > >> > > > > >> > and
> > > > > >> > > > > >> > > >> > testing
> > > > > >> > > > > >> > > >> > > > the
> > > > > >> > > > > >> > > >> > > > > > > feature, it has passed our
> internal
> > > > tests
> > > > > and
> > > > > >> > > many
> > > > > >> > > > > >> rounds
> > > > > >> > > > > >> > of
> > > > > >> > > > > >> > > >> code
> > > > > >> > > > > >> > > >> > > > > reviews
> > > > > >> > > > > >> > > >> > > > > > > by HBase committers. We do not
> mind
> > if
> > > > > >> someone
> > > > > >> > > from
> > > > > >> > > > > >> HBase
> > > > > >> > > > > >> > > >> > community
> > > > > >> > > > > >> > > >> > > > > > > (outside of HW) will review the
> > code,
> > > > but
> > > > > it
> > > > > >> > will
> > > > > >> > > > > >> probably
> > > > > >> > > > > >> > > >> takes
> > > > > >> > > > > >> > > >> > > > > forever
> > > > > >> > > > > >> > > >> > > > > > to
> > > > > >> > > > > >> > > >> > > > > > > wait for volunteer?, the feature
> is
> > > > quite
> > > > > >> large
> > > > > >> > > > (1MB+
> > > > > >> > > > > >> > > >> cumulative
> > > > > >> > > > > >> > > >> > > > patch)
> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > 2.0 branch is full of half baked
> > > > features,
> > > > > >> most
> > > > > >> > > of
> > > > > >> > > > > them
> > > > > >> > > > > >> > are
> > > > > >> > > > > >> > > in
> > > > > >> > > > > >> > > >> > > active
> > > > > >> > > > > >> > > >> > > > > > > development, therefore I am not
> > > > following
> > > > > you
> > > > > >> > > here,
> > > > > >> > > > > >> Sean?
> > > > > >> > > > > >> > > Why
> > > > > >> > > > > >> > > >> > > > > HBASE-7912
> > > > > >> > > > > >> > > >> > > > > > is
> > > > > >> > > > > >> > > >> > > > > > > not good enough yet to be
> integrated
> > > > into
> > > > > 2.0
> > > > > >> > > > branch?
> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > -Vlad
> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > On Wed, Sep 7, 2016 at 8:23 AM,
> Sean
> > > > > Busbey <
> > > > > >> > > > > >> > > busbey@apache.org
> > > > > >> > > > > >> > > >> >
> > > > > >> > > > > >> > > >> > > > wrote:
> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > > On Tue, Sep 6, 2016 at 10:36 PM,
> > > Josh
> > > > > >> Elser <
> > > > > >> > > > > >> > > >> > > josh.elser@gmail.com>
> > > > > >> > > > > >> > > >> > > > > > > wrote:
> > > > > >> > > > > >> > > >> > > > > > > > > So, the answer to Sean's
> > original
> > > > > >> question
> > > > > >> > is
> > > > > >> > > > "as
> > > > > >> > > > > >> > > robust as
> > > > > >> > > > > >> > > >> > > > > snapshots
> > > > > >> > > > > >> > > >> > > > > > > > > presently are"? (independence
> of
> > > > > >> > > backup/restore
> > > > > >> > > > > >> > failure
> > > > > >> > > > > >> > > >> > > tolerance
> > > > > >> > > > > >> > > >> > > > > > from
> > > > > >> > > > > >> > > >> > > > > > > > > snapshot failure tolerance)
> > > > > >> > > > > >> > > >> > > > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > > > Is this just a question WRT
> > > context
> > > > of
> > > > > >> the
> > > > > >> > > > > change,
> > > > > >> > > > > >> or
> > > > > >> > > > > >> > > is it
> > > > > >> > > > > >> > > >> > > means
> > > > > >> > > > > >> > > >> > > > > > for a
> > > > > >> > > > > >> > > >> > > > > > > > veto
> > > > > >> > > > > >> > > >> > > > > > > > > from you, Sean? Just trying to
> > > make
> > > > > sure
> > > > > >> > I'm
> > > > > >> > > > > >> following
> > > > > >> > > > > >> > > >> along
> > > > > >> > > > > >> > > >> > > > > > > adequately.
> > > > > >> > > > > >> > > >> > > > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > > I'd say ATM I'm -0, bordering on
> > -1
> > > > but
> > > > > not
> > > > > >> > for
> > > > > >> > > > > >> reasons
> > > > > >> > > > > >> > I
> > > > > >> > > > > >> > > can
> > > > > >> > > > > >> > > >> > > > > > articulate
> > > > > >> > > > > >> > > >> > > > > > > > well.
> > > > > >> > > > > >> > > >> > > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > > Here's an attempt.
> > > > > >> > > > > >> > > >> > > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > > We've been trying to move, as a
> > > > > community,
> > > > > >> > > > towards
> > > > > >> > > > > >> > > minimizing
> > > > > >> > > > > >> > > >> > > risk
> > > > > >> > > > > >> > > >> > > > to
> > > > > >> > > > > >> > > >> > > > > > > > downstream folks by getting
> > > "complete
> > > > > >> enough
> > > > > >> > > for
> > > > > >> > > > > use"
> > > > > >> > > > > >> > > gates
> > > > > >> > > > > >> > > >> in
> > > > > >> > > > > >> > > >> > > > place
> > > > > >> > > > > >> > > >> > > > > > > > before we introduce new
> features.
> > > This
> > > > > was
> > > > > >> > > > spurred
> > > > > >> > > > > >> by a
> > > > > >> > > > > >> > > some
> > > > > >> > > > > >> > > >> > > > features
> > > > > >> > > > > >> > > >> > > > > > > > getting in half-baked and never
> > > making
> > > > > it
> > > > > >> to
> > > > > >> > > "can
> > > > > >> > > > > >> really
> > > > > >> > > > > >> > > use"
> > > > > >> > > > > >> > > >> > > > status
> > > > > >> > > > > >> > > >> > > > > > > > (I'm thinking of distributed log
> > > > replay
> > > > > and
> > > > > >> > the
> > > > > >> > > > > >> zk-less
> > > > > >> > > > > >> > > >> > > assignment
> > > > > >> > > > > >> > > >> > > > > > > > stuff, I don't recall if there
> was
> > > > > more).
> > > > > >> > > > > >> > > >> > > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > > The gates, generally, included
> > > things
> > > > > like:
> > > > > >> > > > > >> > > >> > > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > > * have docs
> > > > > >> > > > > >> > > >> > > > > > > > * have sunny-day correctness
> tests
> > > > > >> > > > > >> > > >> > > > > > > > * have
> > > correctness-in-face-of-failure
> > > > > tests
> > > > > >> > > > > >> > > >> > > > > > > > * don't rely on things outside
> of
> > > > HBase
> > > > > for
> > > > > >> > > > normal
> > > > > >> > > > > >> > > operation
> > > > > >> > > > > >> > > >> > > (okay
> > > > > >> > > > > >> > > >> > > > > for
> > > > > >> > > > > >> > > >> > > > > > > > advanced operation)
> > > > > >> > > > > >> > > >> > > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > > As an example, we kept the MOB
> > work
> > > > off
> > > > > in
> > > > > >> a
> > > > > >> > > > branch
> > > > > >> > > > > >> and
> > > > > >> > > > > >> > > out
> > > > > >> > > > > >> > > >> of
> > > > > >> > > > > >> > > >> > > > master
> > > > > >> > > > > >> > > >> > > > > > > > until it could pass these
> > criteria.
> > > > The
> > > > > big
> > > > > >> > > > > exemption
> > > > > >> > > > > >> > > we've
> > > > > >> > > > > >> > > >> had
> > > > > >> > > > > >> > > >> > > to
> > > > > >> > > > > >> > > >> > > > > > > > this was the hbase-spark
> > > integration,
> > > > > where
> > > > > >> > we
> > > > > >> > > > all
> > > > > >> > > > > >> > agreed
> > > > > >> > > > > >> > > it
> > > > > >> > > > > >> > > >> > > could
> > > > > >> > > > > >> > > >> > > > > > > > land in master because it was
> very
> > > > well
> > > > > >> > > isolated
> > > > > >> > > > > (the
> > > > > >> > > > > >> > > slide
> > > > > >> > > > > >> > > >> > away
> > > > > >> > > > > >> > > >> > > > from
> > > > > >> > > > > >> > > >> > > > > > > > including docs as a first-class
> > part
> > > > of
> > > > > >> > > building
> > > > > >> > > > up
> > > > > >> > > > > >> that
> > > > > >> > > > > >> > > >> > > > integration
> > > > > >> > > > > >> > > >> > > > > > > > has led me to doubt the wisdom
> of
> > > this
> > > > > >> > > decision).
> > > > > >> > > > > >> > > >> > > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > > We've also been treating
> inclusion
> > > in
> > > > a
> > > > > >> > > "probably
> > > > > >> > > > > >> will
> > > > > >> > > > > >> > be
> > > > > >> > > > > >> > > >> > > released
> > > > > >> > > > > >> > > >> > > > to
> > > > > >> > > > > >> > > >> > > > > > > > downstream" branches as a higher
> > > bar,
> > > > > >> > requiring
> > > > > >> > > > > >> > > >> > > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > > * don't moderately impact
> > > performance
> > > > > when
> > > > > >> > the
> > > > > >> > > > > >> feature
> > > > > >> > > > > >> > > isn't
> > > > > >> > > > > >> > > >> in
> > > > > >> > > > > >> > > >> > > use
> > > > > >> > > > > >> > > >> > > > > > > > * don't severely impact
> > performance
> > > > when
> > > > > >> the
> > > > > >> > > > > feature
> > > > > >> > > > > >> is
> > > > > >> > > > > >> > in
> > > > > >> > > > > >> > > >> use
> > > > > >> > > > > >> > > >> > > > > > > > * either default-to-on or show
> > > enough
> > > > > >> demand
> > > > > >> > to
> > > > > >> > > > > >> believe
> > > > > >> > > > > >> > a
> > > > > >> > > > > >> > > >> > > > non-trivial
> > > > > >> > > > > >> > > >> > > > > > > > number of folks will turn the
> > > feature
> > > > on
> > > > > >> > > > > >> > > >> > > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > > The above has kept MOB and
> > > hbase-spark
> > > > > >> > > > integration
> > > > > >> > > > > >> out
> > > > > >> > > > > >> > of
> > > > > >> > > > > >> > > >> > > branch-1,
> > > > > >> > > > > >> > > >> > > > > > > > presumably while they've "gotten
> > > more
> > > > > >> stable"
> > > > > >> > > in
> > > > > >> > > > > >> master
> > > > > >> > > > > >> > > from
> > > > > >> > > > > >> > > >> > the
> > > > > >> > > > > >> > > >> > > > odd
> > > > > >> > > > > >> > > >> > > > > > > > vendor inclusion.
> > > > > >> > > > > >> > > >> > > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > > Are we going to have a 2.0
> release
> > > > > before
> > > > > >> the
> > > > > >> > > end
> > > > > >> > > > > of
> > > > > >> > > > > >> the
> > > > > >> > > > > >> > > >> year?
> > > > > >> > > > > >> > > >> > > > We're
> > > > > >> > > > > >> > > >> > > > > > > > coming up on 1.5 years since the
> > > > > release of
> > > > > >> > > > version
> > > > > >> > > > > >> 1.0;
> > > > > >> > > > > >> > > >> seems
> > > > > >> > > > > >> > > >> > > like
> > > > > >> > > > > >> > > >> > > > > > > > it's about time, though I
> haven't
> > > seen
> > > > > any
> > > > > >> > > > concrete
> > > > > >> > > > > >> > plans
> > > > > >> > > > > >> > > >> this
> > > > > >> > > > > >> > > >> > > > year.
> > > > > >> > > > > >> > > >> > > > > > > > Presuming we are going to have
> one
> > > by
> > > > > the
> > > > > >> end
> > > > > >> > > of
> > > > > >> > > > > the
> > > > > >> > > > > >> > > year, it
> > > > > >> > > > > >> > > >> > > > seems a
> > > > > >> > > > > >> > > >> > > > > > > > bit close to still be adding in
> > > > > "features
> > > > > >> > that
> > > > > >> > > > need
> > > > > >> > > > > >> > > maturing"
> > > > > >> > > > > >> > > >> > on
> > > > > >> > > > > >> > > >> > > > the
> > > > > >> > > > > >> > > >> > > > > > > > branch.
> > > > > >> > > > > >> > > >> > > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > > The lack of a concrete plan for
> > 2.0
> > > > > keeps
> > > > > >> me
> > > > > >> > > from
> > > > > >> > > > > >> > > considering
> > > > > >> > > > > >> > > >> > > these
> > > > > >> > > > > >> > > >> > > > > > > > things blocker at the moment.
> But
> > I
> > > > know
> > > > > >> > first
> > > > > >> > > > hand
> > > > > >> > > > > >> how
> > > > > >> > > > > >> > > much
> > > > > >> > > > > >> > > >> > > > trouble
> > > > > >> > > > > >> > > >> > > > > > > > folks have had with other
> features
> > > > that
> > > > > >> have
> > > > > >> > > gone
> > > > > >> > > > > >> into
> > > > > >> > > > > >> > > >> > downstream
> > > > > >> > > > > >> > > >> > > > > > > > facing releases without
> robustness
> > > > > checks
> > > > > >> > (i.e.
> > > > > >> > > > > >> > > replication),
> > > > > >> > > > > >> > > >> > and
> > > > > >> > > > > >> > > >> > > > I'm
> > > > > >> > > > > >> > > >> > > > > > > > concerned about what we're
> setting
> > > up
> > > > if
> > > > > >> 2.0
> > > > > >> > > goes
> > > > > >> > > > > out
> > > > > >> > > > > >> > with
> > > > > >> > > > > >> > > >> this
> > > > > >> > > > > >> > > >> > > > > > > > feature in its current state.
> > > > > >> > > > > >> > > >> > > > > > > >
> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > >> > > > > >
> > > > > >> > > > > >> > > >> > > > >
> > > > > >> > > > > >> > > >> > > > >
> > > > > >> > > > > >> > > >> > > > >
> > > > > >> > > > > >> > > >> > > > > --
> > > > > >> > > > > >> > > >> > > > > Best regards,
> > > > > >> > > > > >> > > >> > > > >
> > > > > >> > > > > >> > > >> > > > >    - Andy
> > > > > >> > > > > >> > > >> > > > >
> > > > > >> > > > > >> > > >> > > > > Problems worthy of attack prove their
> > > worth
> > > > by
> > > > > >> > > hitting
> > > > > >> > > > > >> back. -
> > > > > >> > > > > >> > > Piet
> > > > > >> > > > > >> > > >> > > Hein
> > > > > >> > > > > >> > > >> > > > > (via Tom White)
> > > > > >> > > > > >> > > >> > > > >
> > > > > >> > > > > >> > > >> > > >
> > > > > >> > > > > >> > > >> > >
> > > > > >> > > > > >> > > >> > >
> > > > > >> > > > > >> > > >> > >
> > > > > >> > > > > >> > > >> > > --
> > > > > >> > > > > >> > > >> > > Best regards,
> > > > > >> > > > > >> > > >> > >
> > > > > >> > > > > >> > > >> > >    - Andy
> > > > > >> > > > > >> > > >> > >
> > > > > >> > > > > >> > > >> > > Problems worthy of attack prove their
> worth
> > by
> > > > > >> hitting
> > > > > >> > > > back.
> > > > > >> > > > > -
> > > > > >> > > > > >> > Piet
> > > > > >> > > > > >> > > >> Hein
> > > > > >> > > > > >> > > >> > > (via Tom White)
> > > > > >> > > > > >> > > >> > >
> > > > > >> > > > > >> > > >> >
> > > > > >> > > > > >> > > >>
> > > > > >> > > > > >> > >
> > > > > >> > > > > >> >
> > > > > >> > > > > >>
> > > > > >> > > > > >
> > > > > >> > > > > >
> > > > > >> > > > >
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > busbey
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Ted Yu <yu...@gmail.com>]

bq. Does the branch ready for merge

Yes - see the mega patch.

We can discuss the actual procedure of merging if the community gives green
light to merging.

A vote thread can be started if there is no further comment.

On Mon, Oct 10, 2016 at 2:42 PM, Enis Söztutar <en...@apache.org> wrote:

> Great work.
>
> So what are the next steps? It seems from the discussion that all
> outstanding requests have been addressed. Does the branch ready for merge
> or, you have to rebase the branch as well?
>
> Time to start a merge vote?
> Enis
>
> On Mon, Oct 10, 2016 at 1:44 PM, Vladimir Rodionov <vladrodionov@gmail.com
> >
> wrote:
>
> > >> mapreduce dependency has been moved to client side - no mapreduce job
> >
> > 1. We have no code in the client module anymore, due to dependency on
> > internal server API (HFile and WAL access).
> > 2. Backup/ restore are client - driven operations, but all the code
> resides
> > in the server module
> > 3. No MR in Master, no procedure - driven execution.
> > 4. Old good MR from command-line.
> > 5. Security was simplified and now only super-user is allowed to run
> > backup/restores.
> > 6. HBase Backup API was gone due to 1. Now only command-line access to
> > backup tools.
> >
> > These consequences of refactoring has been discussed in HBASE-16727.
> >
> > -Vlad
> >
> >
> >
> >
> > On Mon, Oct 10, 2016 at 1:31 PM, Ted Yu <yu...@gmail.com> wrote:
> >
> > > Reviving this thread.
> > >
> > > The following has taken place:
> > >
> > > mapreduce dependency has been moved to client side - no mapreduce job
> > > launched from master or region server.
> > > document patch (HBASE-16574) has been integrated.
> > > Updated mega patch has been attached to HBASE-14123: this covers the
> > > refactor in #1 above and the protobuf 3 merge.
> > >
> > > If community has more feedback on the merge proposal, I would love to
> > hear
> > > it.
> > >
> > > Thanks
> > >
> > > On Thu, Sep 22, 2016 at 10:31 AM, Sean Busbey <bu...@cloudera.com>
> > wrote:
> > >
> > > > I'd like to see the docs proposed on HBASE-16574 integrated into our
> > > > project's documentation prior to merge.
> > > >
> > > > On Thu, Sep 22, 2016 at 9:02 AM, Ted Yu <yu...@gmail.com> wrote:
> > > > > This feature can be marked experimental due to some limitations
> such
> > as
> > > > > security.
> > > > >
> > > > > Your previous round of comments have been addressed.
> > > > > Command line tool has gone through:
> > > > >
> > > > > HBASE-16620 Fix backup command-line tool usability issues
> > > > > HBASE-16655 hbase backup describe with incorrect backup id results
> in
> > > NPE
> > > > >
> > > > > The updated doc has been attached to HBASE-16574.
> > > > >
> > > > > Cheers
> > > > >
> > > > > On Thu, Sep 22, 2016 at 8:53 AM, Stack <st...@duboce.net> wrote:
> > > > >
> > > > >> On Wed, Sep 21, 2016 at 7:43 AM, Ted Yu <yu...@gmail.com>
> > wrote:
> > > > >>
> > > > >> > Are there more (review) comments ?
> > > > >> >
> > > > >> >
> > > > >> Are outstanding comments addressed?
> > > > >>
> > > > >> I don't see answer to my 'is this experimental/will it be marked
> > > > >> experimental' question.
> > > > >>
> > > > >> I ran into some issues trying to use the feature and suggested
> that
> > a
> > > > >> feature likes this needs polish else it'll just rot, unused. Has
> > > polish
> > > > >> been applied? All ready for another 'user' test? Suggest that you
> > > update
> > > > >> here going forward for the benefit of those trying to follow along
> > and
> > > > who
> > > > >> are not watching JIRA change fly-by.
> > > > >>
> > > > >> It looks like doc got a revision -- I have to check -- to take on
> > > > >> suggestion made above but again, suggest, that this thread gets
> > > updated.
> > > > >>
> > > > >> Thanks,
> > > > >> St.Ack
> > > > >>
> > > > >>
> > > > >>
> > > > >> > Thanks
> > > > >> >
> > > > >> > On Tue, Sep 20, 2016 at 10:02 AM, Devaraj Das <
> > ddas@hortonworks.com
> > > >
> > > > >> > wrote:
> > > > >> >
> > > > >> > > Just reviving this thread. Thanks Sean, Stack, Dima, and
> others
> > > for
> > > > the
> > > > >> > > thorough reviews and testing. Thanks Ted and Vlad for taking
> > care
> > > of
> > > > >> the
> > > > >> > > feedback. Are we all good to do the merge now? Rather do
> sooner
> > > than
> > > > >> > later.
> > > > >> > > ________________________________________
> > > > >> > > From: saint.ack@gmail.com <sa...@gmail.com> on behalf of
> > > Stack
> > > > <
> > > > >> > > stack@duboce.net>
> > > > >> > > Sent: Monday, September 12, 2016 1:18 PM
> > > > >> > > To: HBase Dev List
> > > > >> > > Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch
> > > HBASE-7912
> > > > >> > >
> > > > >> > > On Mon, Sep 12, 2016 at 12:19 PM, Ted Yu <yuzhihong@gmail.com
> >
> > > > wrote:
> > > > >> > >
> > > > >> > > > Mega patch (rev 18) is on HBASE-14123.
> > > > >> > > >
> > > > >> > > > Please comment on HBASE-14123 on how you want to review.
> > > > >> > > >
> > > > >> > >
> > > > >> > >
> > > > >> > > Yeah. That was my lost tab. Last rb was 6 months ago. Suggest
> > > > updating
> > > > >> > it.
> > > > >> > > RB is pretty good for review. Patch is only 1.5M so should be
> > > fine.
> > > > >> > >
> > > > >> > > St.Ack
> > > > >> > >
> > > > >> > >
> > > > >> > > >
> > > > >> > > > Thanks
> > > > >> > > >
> > > > >> > > > On Mon, Sep 12, 2016 at 12:15 PM, Stack <st...@duboce.net>
> > > wrote:
> > > > >> > > >
> > > > >> > > > > On review of the 'patch', do I just compare the branch to
> > > > master or
> > > > >> > is
> > > > >> > > > > there a megapatch posted somewhere (I think I saw one but
> it
> > > > seemed
> > > > >> > > stale
> > > > >> > > > > and then I 'lost' the tab). Sorry for dumb question.
> > > > >> > > > > St.Ack
> > > > >> > > > >
> > > > >> > > > > On Mon, Sep 12, 2016 at 12:01 PM, Stack <stack@duboce.net
> >
> > > > wrote:
> > > > >> > > > >
> > > > >> > > > > > Late to the game. A few comments after rereading this
> > thread
> > > > as a
> > > > >> > > > 'user'.
> > > > >> > > > > >
> > > > >> > > > > > + Before merge, a user-facing feature like this should
> > work
> > > > (If
> > > > >> > this
> > > > >> > > is
> > > > >> > > > > "higher-bar
> > > > >> > > > > > for new features", bring it on -- smile).
> > > > >> > > > > > + As a user, I tried the branch with tools after
> reviewing
> > > the
> > > > >> > > > > just-posted
> > > > >> > > > > > doc. I had an 'interesting' experience (left comments up
> > on
> > > > >> > issue). I
> > > > >> > > > > think
> > > > >> > > > > > the tooling/doc. important to get right. If it breaks
> > easily
> > > > or
> > > > >> is
> > > > >> > > > > > inconsistent (or lacks 'polish'), operators will judge
> the
> > > > whole
> > > > >> > > > > > backup/restore tooling chain as not trustworthy and
> > abandon
> > > > it.
> > > > >> > Lets
> > > > >> > > > not
> > > > >> > > > > > have this happen to this feature.
> > > > >> > > > > > + Matteo's suggestion (with a helpful starter list) that
> > > there
> > > > >> > needs
> > > > >> > > to
> > > > >> > > > > be
> > > > >> > > > > > explicit qualification on what is actually being
> delivered
> > > --
> > > > >> > > > including a
> > > > >> > > > > > listing of limitations (some look serious such as data
> > bleed
> > > > from
> > > > >> > > other
> > > > >> > > > > > regions in WALs, but maybe I don't care for my use
> > case...)
> > > --
> > > > >> > needs
> > > > >> > > to
> > > > >> > > > > > accompany the merge. Lets fold them into the user doc.
> in
> > > the
> > > > >> > > technical
> > > > >> > > > > > overview area as suggested so user expectations are
> > properly
> > > > >> > managed
> > > > >> > > > > > (otherwise, they expect the world and will just give up
> > when
> > > > we
> > > > >> > fall
> > > > >> > > > > > short). Vladimir did a list of what is in each of the
> > phases
> > > > >> above
> > > > >> > > > which
> > > > >> > > > > > would serve as a good start.
> > > > >> > > > > > + Is this feature 'experimental' (Matteo asks above).
> I'd
> > > > prefer
> > > > >> it
> > > > >> > > is
> > > > >> > > > > > not. If it is, it should be labelled all over that it is
> > > so. I
> > > > >> see
> > > > >> > > > > current
> > > > >> > > > > > state called out as a '... technical preview feature'.
> > Does
> > > > this
> > > > >> > mean
> > > > >> > > > > > not-for-users?
> > > > >> > > > > >
> > > > >> > > > > > St.Ack
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > > > On Mon, Sep 12, 2016 at 8:03 AM, Ted Yu <
> > > yuzhihong@gmail.com>
> > > > >> > wrote:
> > > > >> > > > > >
> > > > >> > > > > >> Sean:
> > > > >> > > > > >> Do you have more comments ?
> > > > >> > > > > >>
> > > > >> > > > > >> Cheers
> > > > >> > > > > >>
> > > > >> > > > > >> On Fri, Sep 9, 2016 at 1:42 PM, Vladimir Rodionov <
> > > > >> > > > > vladrodionov@gmail.com
> > > > >> > > > > >> >
> > > > >> > > > > >> wrote:
> > > > >> > > > > >>
> > > > >> > > > > >> > Sean,
> > > > >> > > > > >> >
> > > > >> > > > > >> > Backup/Restore can fail due to various reasons:
> network
> > > > outage
> > > > >> > > > > (cluster
> > > > >> > > > > >> > wide), various time-outs in HBase and HDFS layer, M/R
> > > > failure
> > > > >> > due
> > > > >> > > to
> > > > >> > > > > >> "HDFS
> > > > >> > > > > >> > exceeded quota", user error (manual deletion of data)
> > and
> > > > so
> > > > >> on
> > > > >> > so
> > > > >> > > > on.
> > > > >> > > > > >> That
> > > > >> > > > > >> > is impossible to enumerate all possible types of
> > failures
> > > > in a
> > > > >> > > > > >> distributed
> > > > >> > > > > >> > system - that is not our goal/task.
> > > > >> > > > > >> >
> > > > >> > > > > >> > We focus completely on backup system table
> consistency
> > > in a
> > > > >> > > presence
> > > > >> > > > > of
> > > > >> > > > > >> any
> > > > >> > > > > >> > type of failure. That is what I call "tolerance to
> > > > failures".
> > > > >> > > > > >> >
> > > > >> > > > > >> > On a failure:
> > > > >> > > > > >> >
> > > > >> > > > > >> > BACKUP. All backup system information (prior to
> backup)
> > > > will
> > > > >> be
> > > > >> > > > > restored
> > > > >> > > > > >> > and all temporary data, related to a failed session,
> in
> > > > HDFS
> > > > >> > will
> > > > >> > > be
> > > > >> > > > > >> > deleted
> > > > >> > > > > >> > RESTORE. We do not care about system data, because
> > > restore
> > > > >> does
> > > > >> > > not
> > > > >> > > > > >> change
> > > > >> > > > > >> > it. Temporary data in HDFS will be cleaned up and
> table
> > > > will
> > > > >> be
> > > > >> > > in a
> > > > >> > > > > >> state
> > > > >> > > > > >> > back to where it was before operation started.
> > > > >> > > > > >> >
> > > > >> > > > > >> > This is what user should expect in case of a failure.
> > > > >> > > > > >> >
> > > > >> > > > > >> > -Vlad
> > > > >> > > > > >> >
> > > > >> > > > > >> >
> > > > >> > > > > >> > -Vlad
> > > > >> > > > > >> >
> > > > >> > > > > >> > On Fri, Sep 9, 2016 at 12:56 PM, Sean Busbey <
> > > > >> busbey@apache.org
> > > > >> > >
> > > > >> > > > > wrote:
> > > > >> > > > > >> >
> > > > >> > > > > >> > > Failing in a consistent way, with docs that explain
> > the
> > > > >> > various
> > > > >> > > > > >> > > expected failures would be sufficient.
> > > > >> > > > > >> > >
> > > > >> > > > > >> > > On Fri, Sep 9, 2016 at 12:16 PM, Vladimir Rodionov
> > > > >> > > > > >> > > <vl...@gmail.com> wrote:
> > > > >> > > > > >> > > > Do not worry Sean, doc is coming today as a
> preview
> > > and
> > > > >> our
> > > > >> > > > writer
> > > > >> > > > > >> > Frank
> > > > >> > > > > >> > > > will be working on a putting  it into Apache
> repo.
> > > > >> Timeline
> > > > >> > > > > depends
> > > > >> > > > > >> on
> > > > >> > > > > >> > > > Franks schedule but I hope we will get it rather
> > > sooner
> > > > >> than
> > > > >> > > > > later.
> > > > >> > > > > >> > > >
> > > > >> > > > > >> > > > As for failure testing, we are focusing only on a
> > > > >> consistent
> > > > >> > > > state
> > > > >> > > > > >> of
> > > > >> > > > > >> > > > backup system data in a presence of any type of
> > > > failures,
> > > > >> We
> > > > >> > > are
> > > > >> > > > > not
> > > > >> > > > > >> > > going
> > > > >> > > > > >> > > > to implement  anything more "fancy", than that.
> We
> > > > allow
> > > > >> > both:
> > > > >> > > > > >> backup
> > > > >> > > > > >> > and
> > > > >> > > > > >> > > > restore to fail. What we do not allow is to have
> > > system
> > > > >> data
> > > > >> > > > > >> corrupted.
> > > > >> > > > > >> > > > Will it suffice for you? Do you have any other
> > > > concerns,
> > > > >> you
> > > > >> > > > want
> > > > >> > > > > >> us to
> > > > >> > > > > >> > > > address?
> > > > >> > > > > >> > > >
> > > > >> > > > > >> > > > -Vlad
> > > > >> > > > > >> > > >
> > > > >> > > > > >> > > >
> > > > >> > > > > >> > > > On Fri, Sep 9, 2016 at 10:56 AM, Sean Busbey <
> > > > >> > > busbey@apache.org
> > > > >> > > > >
> > > > >> > > > > >> > wrote:
> > > > >> > > > > >> > > >
> > > > >> > > > > >> > > >> "docs will come to Apache soon" does not address
> > my
> > > > >> concern
> > > > >> > > > > around
> > > > >> > > > > >> > docs
> > > > >> > > > > >> > > at
> > > > >> > > > > >> > > >> all, unless said docs have already made it into
> > the
> > > > >> project
> > > > >> > > > > repo. I
> > > > >> > > > > >> > > don't
> > > > >> > > > > >> > > >> want third party resources for using a major and
> > > > >> important
> > > > >> > > > > feature
> > > > >> > > > > >> of
> > > > >> > > > > >> > > the
> > > > >> > > > > >> > > >> project, I want us to provide end users with
> what
> > > they
> > > > >> need
> > > > >> > > to
> > > > >> > > > > get
> > > > >> > > > > >> the
> > > > >> > > > > >> > > job
> > > > >> > > > > >> > > >> done.
> > > > >> > > > > >> > > >>
> > > > >> > > > > >> > > >> I see some calls for patience on the failure
> > > testing,
> > > > but
> > > > >> > the
> > > > >> > > > > >> appeal
> > > > >> > > > > >> > to
> > > > >> > > > > >> > > us
> > > > >> > > > > >> > > >> having done a bad job of requiring proper tests
> of
> > > > >> previous
> > > > >> > > > > >> features
> > > > >> > > > > >> > > just
> > > > >> > > > > >> > > >> makes me more concerned about not getting them
> > > here. I
> > > > >> > don't
> > > > >> > > > want
> > > > >> > > > > >> to
> > > > >> > > > > >> > set
> > > > >> > > > > >> > > >> yet another bad example that will then be
> pointed
> > to
> > > > in
> > > > >> the
> > > > >> > > > > future.
> > > > >> > > > > >> > > >>
> > > > >> > > > > >> > > >> On Sep 8, 2016 10:50, "Ted Yu" <
> > yuzhihong@gmail.com
> > > >
> > > > >> > wrote:
> > > > >> > > > > >> > > >>
> > > > >> > > > > >> > > >> > Is there any concern which is not addressed ?
> > > > >> > > > > >> > > >> >
> > > > >> > > > > >> > > >> > Do we need another Vote thread ?
> > > > >> > > > > >> > > >> >
> > > > >> > > > > >> > > >> > Thanks
> > > > >> > > > > >> > > >> >
> > > > >> > > > > >> > > >> > On Thu, Sep 8, 2016 at 9:21 AM, Andrew
> Purtell <
> > > > >> > > > > >> apurtell@apache.org
> > > > >> > > > > >> > >
> > > > >> > > > > >> > > >> > wrote:
> > > > >> > > > > >> > > >> >
> > > > >> > > > > >> > > >> > > Vlad,
> > > > >> > > > > >> > > >> > >
> > > > >> > > > > >> > > >> > > I apologize for using the term 'half-baked'
> > in a
> > > > way
> > > > >> > that
> > > > >> > > > > could
> > > > >> > > > > >> > > seem a
> > > > >> > > > > >> > > >> > > description of HBASE-7912. I meant that as a
> > > > general
> > > > >> > > > > >> hypothetical.
> > > > >> > > > > >> > > >> > >
> > > > >> > > > > >> > > >> > > On Wed, Sep 7, 2016 at 9:36 AM, Vladimir
> > > Rodionov
> > > > <
> > > > >> > > > > >> > > >> > vladrodionov@gmail.com>
> > > > >> > > > > >> > > >> > > wrote:
> > > > >> > > > > >> > > >> > >
> > > > >> > > > > >> > > >> > > > >> I'm not sure that "There is already
> lots
> > of
> > > > >> > > half-baked
> > > > >> > > > > >> code
> > > > >> > > > > >> > in
> > > > >> > > > > >> > > the
> > > > >> > > > > >> > > >> > > > branch,
> > > > >> > > > > >> > > >> > > > so what's the harm in adding more?"
> > > > >> > > > > >> > > >> > > >
> > > > >> > > > > >> > > >> > > > I meant - not production - ready yet. This
> > is
> > > > 2.0
> > > > >> > > > > development
> > > > >> > > > > >> > > branch
> > > > >> > > > > >> > > >> > and,
> > > > >> > > > > >> > > >> > > > hence many features are in works,
> > > > >> > > > > >> > > >> > > > not being tested well etc. I do not
> consider
> > > > backup
> > > > >> > as
> > > > >> > > > half
> > > > >> > > > > >> > baked
> > > > >> > > > > >> > > >> > > feature -
> > > > >> > > > > >> > > >> > > > it has passed our internal QA and has very
> > > good
> > > > >> doc,
> > > > >> > > > which
> > > > >> > > > > we
> > > > >> > > > > >> > will
> > > > >> > > > > >> > > >> > > provide
> > > > >> > > > > >> > > >> > > > to Apache shortly.
> > > > >> > > > > >> > > >> > > >
> > > > >> > > > > >> > > >> > > > -Vlad
> > > > >> > > > > >> > > >> > > >
> > > > >> > > > > >> > > >> > > > On Wed, Sep 7, 2016 at 9:13 AM, Andrew
> > > Purtell <
> > > > >> > > > > >> > > apurtell@apache.org>
> > > > >> > > > > >> > > >> > > > wrote:
> > > > >> > > > > >> > > >> > > >
> > > > >> > > > > >> > > >> > > > > We shouldn't admit half baked changes
> that
> > > > won't
> > > > >> be
> > > > >> > > > > >> finished.
> > > > >> > > > > >> > > >> However
> > > > >> > > > > >> > > >> > > in
> > > > >> > > > > >> > > >> > > > > this case the crew working on this
> feature
> > > are
> > > > >> long
> > > > >> > > > > timers
> > > > >> > > > > >> and
> > > > >> > > > > >> > > less
> > > > >> > > > > >> > > >> > > > likely
> > > > >> > > > > >> > > >> > > > > than just about anyone to leave
> something
> > > in a
> > > > >> half
> > > > >> > > > baked
> > > > >> > > > > >> > > state. Of
> > > > >> > > > > >> > > >> > > > course
> > > > >> > > > > >> > > >> > > > > there is no guarantee how anything will
> > turn
> > > > out,
> > > > >> > > but I
> > > > >> > > > > am
> > > > >> > > > > >> > > willing
> > > > >> > > > > >> > > >> to
> > > > >> > > > > >> > > >> > > > take
> > > > >> > > > > >> > > >> > > > > a little on faith if they feel their
> best
> > > path
> > > > >> > > forward
> > > > >> > > > > now
> > > > >> > > > > >> is
> > > > >> > > > > >> > to
> > > > >> > > > > >> > > >> > merge
> > > > >> > > > > >> > > >> > > to
> > > > >> > > > > >> > > >> > > > > trunk. I only wish I had bandwidth to
> have
> > > > done
> > > > >> > some
> > > > >> > > > real
> > > > >> > > > > >> > > kicking
> > > > >> > > > > >> > > >> of
> > > > >> > > > > >> > > >> > > the
> > > > >> > > > > >> > > >> > > > > tires by now. Maybe this week.
> > > > >> > > > > >> > > >> > > > >
> > > > >> > > > > >> > > >> > > > > (Yes, I'm using some of that time for
> this
> > > > email
> > > > >> > :-)
> > > > >> > > > but
> > > > >> > > > > I
> > > > >> > > > > >> > type
> > > > >> > > > > >> > > >> > fast.)
> > > > >> > > > > >> > > >> > > > >
> > > > >> > > > > >> > > >> > > > > That said, I would like to agitate for
> > > making
> > > > 2.0
> > > > >> > > more
> > > > >> > > > > real
> > > > >> > > > > >> > and
> > > > >> > > > > >> > > >> spend
> > > > >> > > > > >> > > >> > > > some
> > > > >> > > > > >> > > >> > > > > time on it now that I'm winding down
> with
> > > > 0.98. I
> > > > >> > > think
> > > > >> > > > > >> that
> > > > >> > > > > >> > > means
> > > > >> > > > > >> > > >> > > > > branching for 2.0 real soon now and even
> > > > evicting
> > > > >> > > > things
> > > > >> > > > > >> from
> > > > >> > > > > >> > > 2.0
> > > > >> > > > > >> > > >> > > branch
> > > > >> > > > > >> > > >> > > > > that aren't finished or stable, leaving
> > them
> > > > only
> > > > >> > > once
> > > > >> > > > > >> again
> > > > >> > > > > >> > in
> > > > >> > > > > >> > > the
> > > > >> > > > > >> > > >> > > > master
> > > > >> > > > > >> > > >> > > > > branch. Or, maybe just evicting them.
> > Let's
> > > > take
> > > > >> it
> > > > >> > > > case
> > > > >> > > > > by
> > > > >> > > > > >> > > case.
> > > > >> > > > > >> > > >> > > > >
> > > > >> > > > > >> > > >> > > > > I think this feature can come in
> > relatively
> > > > >> safely.
> > > > >> > > As
> > > > >> > > > > >> added
> > > > >> > > > > >> > > >> > insurance,
> > > > >> > > > > >> > > >> > > > > let's admit the possibility it could be
> > > > reverted
> > > > >> on
> > > > >> > > the
> > > > >> > > > > 2.0
> > > > >> > > > > >> > > branch
> > > > >> > > > > >> > > >> if
> > > > >> > > > > >> > > >> > > > folks
> > > > >> > > > > >> > > >> > > > > working on stabilizing 2.0 decide to
> evict
> > > it
> > > > >> > because
> > > > >> > > > it
> > > > >> > > > > is
> > > > >> > > > > >> > > >> > unfinished
> > > > >> > > > > >> > > >> > > or
> > > > >> > > > > >> > > >> > > > > unstable, because that certainly can
> > > happen. I
> > > > >> > would
> > > > >> > > > > >> expect if
> > > > >> > > > > >> > > talk
> > > > >> > > > > >> > > >> > > like
> > > > >> > > > > >> > > >> > > > > that starts, we'd get help finishing or
> > > > >> stabilizing
> > > > >> > > > > what's
> > > > >> > > > > >> > under
> > > > >> > > > > >> > > >> > > > discussion
> > > > >> > > > > >> > > >> > > > > for revert. Or, we'd have a revert.
> Either
> > > way
> > > > >> the
> > > > >> > > > > outcome
> > > > >> > > > > >> is
> > > > >> > > > > >> > > >> > > acceptable.
> > > > >> > > > > >> > > >> > > > >
> > > > >> > > > > >> > > >> > > > >
> > > > >> > > > > >> > > >> > > > > On Wed, Sep 7, 2016 at 8:56 AM, Dima
> > Spivak
> > > <
> > > > >> > > > > >> > > dimaspivak@apache.org
> > > > >> > > > > >> > > >> >
> > > > >> > > > > >> > > >> > > > wrote:
> > > > >> > > > > >> > > >> > > > >
> > > > >> > > > > >> > > >> > > > > > I'm not sure that "There is already
> lots
> > > of
> > > > >> > > > half-baked
> > > > >> > > > > >> code
> > > > >> > > > > >> > in
> > > > >> > > > > >> > > >> the
> > > > >> > > > > >> > > >> > > > > branch,
> > > > >> > > > > >> > > >> > > > > > so what's the harm in adding more?"
> is a
> > > > good
> > > > >> > code
> > > > >> > > > > commit
> > > > >> > > > > >> > > >> > philosophy
> > > > >> > > > > >> > > >> > > > for
> > > > >> > > > > >> > > >> > > > > a
> > > > >> > > > > >> > > >> > > > > > fault-tolerant distributed data store.
> > ;)
> > > > >> > > > > >> > > >> > > > > >
> > > > >> > > > > >> > > >> > > > > > More seriously, a lack of test
> coverage
> > > for
> > > > >> > > existing
> > > > >> > > > > >> > features
> > > > >> > > > > >> > > >> > > shouldn't
> > > > >> > > > > >> > > >> > > > > be
> > > > >> > > > > >> > > >> > > > > > used as justification for introducing
> > new
> > > > >> > features
> > > > >> > > > with
> > > > >> > > > > >> the
> > > > >> > > > > >> > > same
> > > > >> > > > > >> > > >> > > > > > shortcomings. Ultimately, it's the end
> > > user
> > > > who
> > > > >> > > will
> > > > >> > > > > feel
> > > > >> > > > > >> > the
> > > > >> > > > > >> > > >> pain,
> > > > >> > > > > >> > > >> > > so
> > > > >> > > > > >> > > >> > > > > > shouldn't we do everything we can to
> > > > mitigate
> > > > >> > that?
> > > > >> > > > > >> > > >> > > > > >
> > > > >> > > > > >> > > >> > > > > > -Dima
> > > > >> > > > > >> > > >> > > > > >
> > > > >> > > > > >> > > >> > > > > > On Wed, Sep 7, 2016 at 8:46 AM,
> Vladimir
> > > > >> > Rodionov <
> > > > >> > > > > >> > > >> > > > > vladrodionov@gmail.com>
> > > > >> > > > > >> > > >> > > > > > wrote:
> > > > >> > > > > >> > > >> > > > > >
> > > > >> > > > > >> > > >> > > > > > > Sean,
> > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > >> > > > > > > * have docs
> > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > >> > > > > > > Agree. We have a doc and backup is
> the
> > > > most
> > > > >> > > > > documented
> > > > >> > > > > >> > > feature
> > > > >> > > > > >> > > >> > :),
> > > > >> > > > > >> > > >> > > we
> > > > >> > > > > >> > > >> > > > > > will
> > > > >> > > > > >> > > >> > > > > > > release it shortly to Apache.
> > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > >> > > > > > > * have sunny-day correctness tests
> > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > >> > > > > > > Feature has  close to 60 test cases,
> > > which
> > > > >> run
> > > > >> > > for
> > > > >> > > > > >> approx
> > > > >> > > > > >> > 30
> > > > >> > > > > >> > > >> min.
> > > > >> > > > > >> > > >> > > We
> > > > >> > > > > >> > > >> > > > > can
> > > > >> > > > > >> > > >> > > > > > > add more, if community do not mind
> :)
> > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > >> > > > > > > * have
> correctness-in-face-of-failure
> > > > tests
> > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > >> > > > > > > Any examples of these tests in
> > existing
> > > > >> > features?
> > > > >> > > > In
> > > > >> > > > > >> > works,
> > > > >> > > > > >> > > we
> > > > >> > > > > >> > > >> > > have a
> > > > >> > > > > >> > > >> > > > > > clear
> > > > >> > > > > >> > > >> > > > > > > understanding of what should be done
> > by
> > > > the
> > > > >> > time
> > > > >> > > of
> > > > >> > > > > 2.0
> > > > >> > > > > >> > > >> release.
> > > > >> > > > > >> > > >> > > > > > > That is very close goal for us, to
> > > verify
> > > > IT
> > > > >> > > monkey
> > > > >> > > > > for
> > > > >> > > > > >> > > >> existing
> > > > >> > > > > >> > > >> > > > code.
> > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > >> > > > > > > * don't rely on things outside of
> > HBase
> > > > for
> > > > >> > > normal
> > > > >> > > > > >> > operation
> > > > >> > > > > >> > > >> > (okay
> > > > >> > > > > >> > > >> > > > for
> > > > >> > > > > >> > > >> > > > > > > advanced operation)
> > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > >> > > > > > > We do not.
> > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > >> > > > > > > Enormous time has been spent already
> > on
> > > > the
> > > > >> > > > > development
> > > > >> > > > > >> > and
> > > > >> > > > > >> > > >> > testing
> > > > >> > > > > >> > > >> > > > the
> > > > >> > > > > >> > > >> > > > > > > feature, it has passed our internal
> > > tests
> > > > and
> > > > >> > > many
> > > > >> > > > > >> rounds
> > > > >> > > > > >> > of
> > > > >> > > > > >> > > >> code
> > > > >> > > > > >> > > >> > > > > reviews
> > > > >> > > > > >> > > >> > > > > > > by HBase committers. We do not mind
> if
> > > > >> someone
> > > > >> > > from
> > > > >> > > > > >> HBase
> > > > >> > > > > >> > > >> > community
> > > > >> > > > > >> > > >> > > > > > > (outside of HW) will review the
> code,
> > > but
> > > > it
> > > > >> > will
> > > > >> > > > > >> probably
> > > > >> > > > > >> > > >> takes
> > > > >> > > > > >> > > >> > > > > forever
> > > > >> > > > > >> > > >> > > > > > to
> > > > >> > > > > >> > > >> > > > > > > wait for volunteer?, the feature is
> > > quite
> > > > >> large
> > > > >> > > > (1MB+
> > > > >> > > > > >> > > >> cumulative
> > > > >> > > > > >> > > >> > > > patch)
> > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > >> > > > > > > 2.0 branch is full of half baked
> > > features,
> > > > >> most
> > > > >> > > of
> > > > >> > > > > them
> > > > >> > > > > >> > are
> > > > >> > > > > >> > > in
> > > > >> > > > > >> > > >> > > active
> > > > >> > > > > >> > > >> > > > > > > development, therefore I am not
> > > following
> > > > you
> > > > >> > > here,
> > > > >> > > > > >> Sean?
> > > > >> > > > > >> > > Why
> > > > >> > > > > >> > > >> > > > > HBASE-7912
> > > > >> > > > > >> > > >> > > > > > is
> > > > >> > > > > >> > > >> > > > > > > not good enough yet to be integrated
> > > into
> > > > 2.0
> > > > >> > > > branch?
> > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > >> > > > > > > -Vlad
> > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > >> > > > > > > On Wed, Sep 7, 2016 at 8:23 AM, Sean
> > > > Busbey <
> > > > >> > > > > >> > > busbey@apache.org
> > > > >> > > > > >> > > >> >
> > > > >> > > > > >> > > >> > > > wrote:
> > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > >> > > > > > > > On Tue, Sep 6, 2016 at 10:36 PM,
> > Josh
> > > > >> Elser <
> > > > >> > > > > >> > > >> > > josh.elser@gmail.com>
> > > > >> > > > > >> > > >> > > > > > > wrote:
> > > > >> > > > > >> > > >> > > > > > > > > So, the answer to Sean's
> original
> > > > >> question
> > > > >> > is
> > > > >> > > > "as
> > > > >> > > > > >> > > robust as
> > > > >> > > > > >> > > >> > > > > snapshots
> > > > >> > > > > >> > > >> > > > > > > > > presently are"? (independence of
> > > > >> > > backup/restore
> > > > >> > > > > >> > failure
> > > > >> > > > > >> > > >> > > tolerance
> > > > >> > > > > >> > > >> > > > > > from
> > > > >> > > > > >> > > >> > > > > > > > > snapshot failure tolerance)
> > > > >> > > > > >> > > >> > > > > > > > >
> > > > >> > > > > >> > > >> > > > > > > > > Is this just a question WRT
> > context
> > > of
> > > > >> the
> > > > >> > > > > change,
> > > > >> > > > > >> or
> > > > >> > > > > >> > > is it
> > > > >> > > > > >> > > >> > > means
> > > > >> > > > > >> > > >> > > > > > for a
> > > > >> > > > > >> > > >> > > > > > > > veto
> > > > >> > > > > >> > > >> > > > > > > > > from you, Sean? Just trying to
> > make
> > > > sure
> > > > >> > I'm
> > > > >> > > > > >> following
> > > > >> > > > > >> > > >> along
> > > > >> > > > > >> > > >> > > > > > > adequately.
> > > > >> > > > > >> > > >> > > > > > > > >
> > > > >> > > > > >> > > >> > > > > > > > >
> > > > >> > > > > >> > > >> > > > > > > >
> > > > >> > > > > >> > > >> > > > > > > > I'd say ATM I'm -0, bordering on
> -1
> > > but
> > > > not
> > > > >> > for
> > > > >> > > > > >> reasons
> > > > >> > > > > >> > I
> > > > >> > > > > >> > > can
> > > > >> > > > > >> > > >> > > > > > articulate
> > > > >> > > > > >> > > >> > > > > > > > well.
> > > > >> > > > > >> > > >> > > > > > > >
> > > > >> > > > > >> > > >> > > > > > > > Here's an attempt.
> > > > >> > > > > >> > > >> > > > > > > >
> > > > >> > > > > >> > > >> > > > > > > > We've been trying to move, as a
> > > > community,
> > > > >> > > > towards
> > > > >> > > > > >> > > minimizing
> > > > >> > > > > >> > > >> > > risk
> > > > >> > > > > >> > > >> > > > to
> > > > >> > > > > >> > > >> > > > > > > > downstream folks by getting
> > "complete
> > > > >> enough
> > > > >> > > for
> > > > >> > > > > use"
> > > > >> > > > > >> > > gates
> > > > >> > > > > >> > > >> in
> > > > >> > > > > >> > > >> > > > place
> > > > >> > > > > >> > > >> > > > > > > > before we introduce new features.
> > This
> > > > was
> > > > >> > > > spurred
> > > > >> > > > > >> by a
> > > > >> > > > > >> > > some
> > > > >> > > > > >> > > >> > > > features
> > > > >> > > > > >> > > >> > > > > > > > getting in half-baked and never
> > making
> > > > it
> > > > >> to
> > > > >> > > "can
> > > > >> > > > > >> really
> > > > >> > > > > >> > > use"
> > > > >> > > > > >> > > >> > > > status
> > > > >> > > > > >> > > >> > > > > > > > (I'm thinking of distributed log
> > > replay
> > > > and
> > > > >> > the
> > > > >> > > > > >> zk-less
> > > > >> > > > > >> > > >> > > assignment
> > > > >> > > > > >> > > >> > > > > > > > stuff, I don't recall if there was
> > > > more).
> > > > >> > > > > >> > > >> > > > > > > >
> > > > >> > > > > >> > > >> > > > > > > > The gates, generally, included
> > things
> > > > like:
> > > > >> > > > > >> > > >> > > > > > > >
> > > > >> > > > > >> > > >> > > > > > > > * have docs
> > > > >> > > > > >> > > >> > > > > > > > * have sunny-day correctness tests
> > > > >> > > > > >> > > >> > > > > > > > * have
> > correctness-in-face-of-failure
> > > > tests
> > > > >> > > > > >> > > >> > > > > > > > * don't rely on things outside of
> > > HBase
> > > > for
> > > > >> > > > normal
> > > > >> > > > > >> > > operation
> > > > >> > > > > >> > > >> > > (okay
> > > > >> > > > > >> > > >> > > > > for
> > > > >> > > > > >> > > >> > > > > > > > advanced operation)
> > > > >> > > > > >> > > >> > > > > > > >
> > > > >> > > > > >> > > >> > > > > > > > As an example, we kept the MOB
> work
> > > off
> > > > in
> > > > >> a
> > > > >> > > > branch
> > > > >> > > > > >> and
> > > > >> > > > > >> > > out
> > > > >> > > > > >> > > >> of
> > > > >> > > > > >> > > >> > > > master
> > > > >> > > > > >> > > >> > > > > > > > until it could pass these
> criteria.
> > > The
> > > > big
> > > > >> > > > > exemption
> > > > >> > > > > >> > > we've
> > > > >> > > > > >> > > >> had
> > > > >> > > > > >> > > >> > > to
> > > > >> > > > > >> > > >> > > > > > > > this was the hbase-spark
> > integration,
> > > > where
> > > > >> > we
> > > > >> > > > all
> > > > >> > > > > >> > agreed
> > > > >> > > > > >> > > it
> > > > >> > > > > >> > > >> > > could
> > > > >> > > > > >> > > >> > > > > > > > land in master because it was very
> > > well
> > > > >> > > isolated
> > > > >> > > > > (the
> > > > >> > > > > >> > > slide
> > > > >> > > > > >> > > >> > away
> > > > >> > > > > >> > > >> > > > from
> > > > >> > > > > >> > > >> > > > > > > > including docs as a first-class
> part
> > > of
> > > > >> > > building
> > > > >> > > > up
> > > > >> > > > > >> that
> > > > >> > > > > >> > > >> > > > integration
> > > > >> > > > > >> > > >> > > > > > > > has led me to doubt the wisdom of
> > this
> > > > >> > > decision).
> > > > >> > > > > >> > > >> > > > > > > >
> > > > >> > > > > >> > > >> > > > > > > > We've also been treating inclusion
> > in
> > > a
> > > > >> > > "probably
> > > > >> > > > > >> will
> > > > >> > > > > >> > be
> > > > >> > > > > >> > > >> > > released
> > > > >> > > > > >> > > >> > > > to
> > > > >> > > > > >> > > >> > > > > > > > downstream" branches as a higher
> > bar,
> > > > >> > requiring
> > > > >> > > > > >> > > >> > > > > > > >
> > > > >> > > > > >> > > >> > > > > > > > * don't moderately impact
> > performance
> > > > when
> > > > >> > the
> > > > >> > > > > >> feature
> > > > >> > > > > >> > > isn't
> > > > >> > > > > >> > > >> in
> > > > >> > > > > >> > > >> > > use
> > > > >> > > > > >> > > >> > > > > > > > * don't severely impact
> performance
> > > when
> > > > >> the
> > > > >> > > > > feature
> > > > >> > > > > >> is
> > > > >> > > > > >> > in
> > > > >> > > > > >> > > >> use
> > > > >> > > > > >> > > >> > > > > > > > * either default-to-on or show
> > enough
> > > > >> demand
> > > > >> > to
> > > > >> > > > > >> believe
> > > > >> > > > > >> > a
> > > > >> > > > > >> > > >> > > > non-trivial
> > > > >> > > > > >> > > >> > > > > > > > number of folks will turn the
> > feature
> > > on
> > > > >> > > > > >> > > >> > > > > > > >
> > > > >> > > > > >> > > >> > > > > > > > The above has kept MOB and
> > hbase-spark
> > > > >> > > > integration
> > > > >> > > > > >> out
> > > > >> > > > > >> > of
> > > > >> > > > > >> > > >> > > branch-1,
> > > > >> > > > > >> > > >> > > > > > > > presumably while they've "gotten
> > more
> > > > >> stable"
> > > > >> > > in
> > > > >> > > > > >> master
> > > > >> > > > > >> > > from
> > > > >> > > > > >> > > >> > the
> > > > >> > > > > >> > > >> > > > odd
> > > > >> > > > > >> > > >> > > > > > > > vendor inclusion.
> > > > >> > > > > >> > > >> > > > > > > >
> > > > >> > > > > >> > > >> > > > > > > > Are we going to have a 2.0 release
> > > > before
> > > > >> the
> > > > >> > > end
> > > > >> > > > > of
> > > > >> > > > > >> the
> > > > >> > > > > >> > > >> year?
> > > > >> > > > > >> > > >> > > > We're
> > > > >> > > > > >> > > >> > > > > > > > coming up on 1.5 years since the
> > > > release of
> > > > >> > > > version
> > > > >> > > > > >> 1.0;
> > > > >> > > > > >> > > >> seems
> > > > >> > > > > >> > > >> > > like
> > > > >> > > > > >> > > >> > > > > > > > it's about time, though I haven't
> > seen
> > > > any
> > > > >> > > > concrete
> > > > >> > > > > >> > plans
> > > > >> > > > > >> > > >> this
> > > > >> > > > > >> > > >> > > > year.
> > > > >> > > > > >> > > >> > > > > > > > Presuming we are going to have one
> > by
> > > > the
> > > > >> end
> > > > >> > > of
> > > > >> > > > > the
> > > > >> > > > > >> > > year, it
> > > > >> > > > > >> > > >> > > > seems a
> > > > >> > > > > >> > > >> > > > > > > > bit close to still be adding in
> > > > "features
> > > > >> > that
> > > > >> > > > need
> > > > >> > > > > >> > > maturing"
> > > > >> > > > > >> > > >> > on
> > > > >> > > > > >> > > >> > > > the
> > > > >> > > > > >> > > >> > > > > > > > branch.
> > > > >> > > > > >> > > >> > > > > > > >
> > > > >> > > > > >> > > >> > > > > > > > The lack of a concrete plan for
> 2.0
> > > > keeps
> > > > >> me
> > > > >> > > from
> > > > >> > > > > >> > > considering
> > > > >> > > > > >> > > >> > > these
> > > > >> > > > > >> > > >> > > > > > > > things blocker at the moment. But
> I
> > > know
> > > > >> > first
> > > > >> > > > hand
> > > > >> > > > > >> how
> > > > >> > > > > >> > > much
> > > > >> > > > > >> > > >> > > > trouble
> > > > >> > > > > >> > > >> > > > > > > > folks have had with other features
> > > that
> > > > >> have
> > > > >> > > gone
> > > > >> > > > > >> into
> > > > >> > > > > >> > > >> > downstream
> > > > >> > > > > >> > > >> > > > > > > > facing releases without robustness
> > > > checks
> > > > >> > (i.e.
> > > > >> > > > > >> > > replication),
> > > > >> > > > > >> > > >> > and
> > > > >> > > > > >> > > >> > > > I'm
> > > > >> > > > > >> > > >> > > > > > > > concerned about what we're setting
> > up
> > > if
> > > > >> 2.0
> > > > >> > > goes
> > > > >> > > > > out
> > > > >> > > > > >> > with
> > > > >> > > > > >> > > >> this
> > > > >> > > > > >> > > >> > > > > > > > feature in its current state.
> > > > >> > > > > >> > > >> > > > > > > >
> > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > >> > > > > >
> > > > >> > > > > >> > > >> > > > >
> > > > >> > > > > >> > > >> > > > >
> > > > >> > > > > >> > > >> > > > >
> > > > >> > > > > >> > > >> > > > > --
> > > > >> > > > > >> > > >> > > > > Best regards,
> > > > >> > > > > >> > > >> > > > >
> > > > >> > > > > >> > > >> > > > >    - Andy
> > > > >> > > > > >> > > >> > > > >
> > > > >> > > > > >> > > >> > > > > Problems worthy of attack prove their
> > worth
> > > by
> > > > >> > > hitting
> > > > >> > > > > >> back. -
> > > > >> > > > > >> > > Piet
> > > > >> > > > > >> > > >> > > Hein
> > > > >> > > > > >> > > >> > > > > (via Tom White)
> > > > >> > > > > >> > > >> > > > >
> > > > >> > > > > >> > > >> > > >
> > > > >> > > > > >> > > >> > >
> > > > >> > > > > >> > > >> > >
> > > > >> > > > > >> > > >> > >
> > > > >> > > > > >> > > >> > > --
> > > > >> > > > > >> > > >> > > Best regards,
> > > > >> > > > > >> > > >> > >
> > > > >> > > > > >> > > >> > >    - Andy
> > > > >> > > > > >> > > >> > >
> > > > >> > > > > >> > > >> > > Problems worthy of attack prove their worth
> by
> > > > >> hitting
> > > > >> > > > back.
> > > > >> > > > > -
> > > > >> > > > > >> > Piet
> > > > >> > > > > >> > > >> Hein
> > > > >> > > > > >> > > >> > > (via Tom White)
> > > > >> > > > > >> > > >> > >
> > > > >> > > > > >> > > >> >
> > > > >> > > > > >> > > >>
> > > > >> > > > > >> > >
> > > > >> > > > > >> >
> > > > >> > > > > >>
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > busbey
> > > >
> > >
> >
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Enis Söztutar <en...@apache.org>]

Great work.

So what are the next steps? It seems from the discussion that all
outstanding requests have been addressed. Does the branch ready for merge
or, you have to rebase the branch as well?

Time to start a merge vote?
Enis

On Mon, Oct 10, 2016 at 1:44 PM, Vladimir Rodionov <vl...@gmail.com>
wrote:

> >> mapreduce dependency has been moved to client side - no mapreduce job
>
> 1. We have no code in the client module anymore, due to dependency on
> internal server API (HFile and WAL access).
> 2. Backup/ restore are client - driven operations, but all the code resides
> in the server module
> 3. No MR in Master, no procedure - driven execution.
> 4. Old good MR from command-line.
> 5. Security was simplified and now only super-user is allowed to run
> backup/restores.
> 6. HBase Backup API was gone due to 1. Now only command-line access to
> backup tools.
>
> These consequences of refactoring has been discussed in HBASE-16727.
>
> -Vlad
>
>
>
>
> On Mon, Oct 10, 2016 at 1:31 PM, Ted Yu <yu...@gmail.com> wrote:
>
> > Reviving this thread.
> >
> > The following has taken place:
> >
> > mapreduce dependency has been moved to client side - no mapreduce job
> > launched from master or region server.
> > document patch (HBASE-16574) has been integrated.
> > Updated mega patch has been attached to HBASE-14123: this covers the
> > refactor in #1 above and the protobuf 3 merge.
> >
> > If community has more feedback on the merge proposal, I would love to
> hear
> > it.
> >
> > Thanks
> >
> > On Thu, Sep 22, 2016 at 10:31 AM, Sean Busbey <bu...@cloudera.com>
> wrote:
> >
> > > I'd like to see the docs proposed on HBASE-16574 integrated into our
> > > project's documentation prior to merge.
> > >
> > > On Thu, Sep 22, 2016 at 9:02 AM, Ted Yu <yu...@gmail.com> wrote:
> > > > This feature can be marked experimental due to some limitations such
> as
> > > > security.
> > > >
> > > > Your previous round of comments have been addressed.
> > > > Command line tool has gone through:
> > > >
> > > > HBASE-16620 Fix backup command-line tool usability issues
> > > > HBASE-16655 hbase backup describe with incorrect backup id results in
> > NPE
> > > >
> > > > The updated doc has been attached to HBASE-16574.
> > > >
> > > > Cheers
> > > >
> > > > On Thu, Sep 22, 2016 at 8:53 AM, Stack <st...@duboce.net> wrote:
> > > >
> > > >> On Wed, Sep 21, 2016 at 7:43 AM, Ted Yu <yu...@gmail.com>
> wrote:
> > > >>
> > > >> > Are there more (review) comments ?
> > > >> >
> > > >> >
> > > >> Are outstanding comments addressed?
> > > >>
> > > >> I don't see answer to my 'is this experimental/will it be marked
> > > >> experimental' question.
> > > >>
> > > >> I ran into some issues trying to use the feature and suggested that
> a
> > > >> feature likes this needs polish else it'll just rot, unused. Has
> > polish
> > > >> been applied? All ready for another 'user' test? Suggest that you
> > update
> > > >> here going forward for the benefit of those trying to follow along
> and
> > > who
> > > >> are not watching JIRA change fly-by.
> > > >>
> > > >> It looks like doc got a revision -- I have to check -- to take on
> > > >> suggestion made above but again, suggest, that this thread gets
> > updated.
> > > >>
> > > >> Thanks,
> > > >> St.Ack
> > > >>
> > > >>
> > > >>
> > > >> > Thanks
> > > >> >
> > > >> > On Tue, Sep 20, 2016 at 10:02 AM, Devaraj Das <
> ddas@hortonworks.com
> > >
> > > >> > wrote:
> > > >> >
> > > >> > > Just reviving this thread. Thanks Sean, Stack, Dima, and others
> > for
> > > the
> > > >> > > thorough reviews and testing. Thanks Ted and Vlad for taking
> care
> > of
> > > >> the
> > > >> > > feedback. Are we all good to do the merge now? Rather do sooner
> > than
> > > >> > later.
> > > >> > > ________________________________________
> > > >> > > From: saint.ack@gmail.com <sa...@gmail.com> on behalf of
> > Stack
> > > <
> > > >> > > stack@duboce.net>
> > > >> > > Sent: Monday, September 12, 2016 1:18 PM
> > > >> > > To: HBase Dev List
> > > >> > > Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch
> > HBASE-7912
> > > >> > >
> > > >> > > On Mon, Sep 12, 2016 at 12:19 PM, Ted Yu <yu...@gmail.com>
> > > wrote:
> > > >> > >
> > > >> > > > Mega patch (rev 18) is on HBASE-14123.
> > > >> > > >
> > > >> > > > Please comment on HBASE-14123 on how you want to review.
> > > >> > > >
> > > >> > >
> > > >> > >
> > > >> > > Yeah. That was my lost tab. Last rb was 6 months ago. Suggest
> > > updating
> > > >> > it.
> > > >> > > RB is pretty good for review. Patch is only 1.5M so should be
> > fine.
> > > >> > >
> > > >> > > St.Ack
> > > >> > >
> > > >> > >
> > > >> > > >
> > > >> > > > Thanks
> > > >> > > >
> > > >> > > > On Mon, Sep 12, 2016 at 12:15 PM, Stack <st...@duboce.net>
> > wrote:
> > > >> > > >
> > > >> > > > > On review of the 'patch', do I just compare the branch to
> > > master or
> > > >> > is
> > > >> > > > > there a megapatch posted somewhere (I think I saw one but it
> > > seemed
> > > >> > > stale
> > > >> > > > > and then I 'lost' the tab). Sorry for dumb question.
> > > >> > > > > St.Ack
> > > >> > > > >
> > > >> > > > > On Mon, Sep 12, 2016 at 12:01 PM, Stack <st...@duboce.net>
> > > wrote:
> > > >> > > > >
> > > >> > > > > > Late to the game. A few comments after rereading this
> thread
> > > as a
> > > >> > > > 'user'.
> > > >> > > > > >
> > > >> > > > > > + Before merge, a user-facing feature like this should
> work
> > > (If
> > > >> > this
> > > >> > > is
> > > >> > > > > "higher-bar
> > > >> > > > > > for new features", bring it on -- smile).
> > > >> > > > > > + As a user, I tried the branch with tools after reviewing
> > the
> > > >> > > > > just-posted
> > > >> > > > > > doc. I had an 'interesting' experience (left comments up
> on
> > > >> > issue). I
> > > >> > > > > think
> > > >> > > > > > the tooling/doc. important to get right. If it breaks
> easily
> > > or
> > > >> is
> > > >> > > > > > inconsistent (or lacks 'polish'), operators will judge the
> > > whole
> > > >> > > > > > backup/restore tooling chain as not trustworthy and
> abandon
> > > it.
> > > >> > Lets
> > > >> > > > not
> > > >> > > > > > have this happen to this feature.
> > > >> > > > > > + Matteo's suggestion (with a helpful starter list) that
> > there
> > > >> > needs
> > > >> > > to
> > > >> > > > > be
> > > >> > > > > > explicit qualification on what is actually being delivered
> > --
> > > >> > > > including a
> > > >> > > > > > listing of limitations (some look serious such as data
> bleed
> > > from
> > > >> > > other
> > > >> > > > > > regions in WALs, but maybe I don't care for my use
> case...)
> > --
> > > >> > needs
> > > >> > > to
> > > >> > > > > > accompany the merge. Lets fold them into the user doc. in
> > the
> > > >> > > technical
> > > >> > > > > > overview area as suggested so user expectations are
> properly
> > > >> > managed
> > > >> > > > > > (otherwise, they expect the world and will just give up
> when
> > > we
> > > >> > fall
> > > >> > > > > > short). Vladimir did a list of what is in each of the
> phases
> > > >> above
> > > >> > > > which
> > > >> > > > > > would serve as a good start.
> > > >> > > > > > + Is this feature 'experimental' (Matteo asks above). I'd
> > > prefer
> > > >> it
> > > >> > > is
> > > >> > > > > > not. If it is, it should be labelled all over that it is
> > so. I
> > > >> see
> > > >> > > > > current
> > > >> > > > > > state called out as a '... technical preview feature'.
> Does
> > > this
> > > >> > mean
> > > >> > > > > > not-for-users?
> > > >> > > > > >
> > > >> > > > > > St.Ack
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > > On Mon, Sep 12, 2016 at 8:03 AM, Ted Yu <
> > yuzhihong@gmail.com>
> > > >> > wrote:
> > > >> > > > > >
> > > >> > > > > >> Sean:
> > > >> > > > > >> Do you have more comments ?
> > > >> > > > > >>
> > > >> > > > > >> Cheers
> > > >> > > > > >>
> > > >> > > > > >> On Fri, Sep 9, 2016 at 1:42 PM, Vladimir Rodionov <
> > > >> > > > > vladrodionov@gmail.com
> > > >> > > > > >> >
> > > >> > > > > >> wrote:
> > > >> > > > > >>
> > > >> > > > > >> > Sean,
> > > >> > > > > >> >
> > > >> > > > > >> > Backup/Restore can fail due to various reasons: network
> > > outage
> > > >> > > > > (cluster
> > > >> > > > > >> > wide), various time-outs in HBase and HDFS layer, M/R
> > > failure
> > > >> > due
> > > >> > > to
> > > >> > > > > >> "HDFS
> > > >> > > > > >> > exceeded quota", user error (manual deletion of data)
> and
> > > so
> > > >> on
> > > >> > so
> > > >> > > > on.
> > > >> > > > > >> That
> > > >> > > > > >> > is impossible to enumerate all possible types of
> failures
> > > in a
> > > >> > > > > >> distributed
> > > >> > > > > >> > system - that is not our goal/task.
> > > >> > > > > >> >
> > > >> > > > > >> > We focus completely on backup system table consistency
> > in a
> > > >> > > presence
> > > >> > > > > of
> > > >> > > > > >> any
> > > >> > > > > >> > type of failure. That is what I call "tolerance to
> > > failures".
> > > >> > > > > >> >
> > > >> > > > > >> > On a failure:
> > > >> > > > > >> >
> > > >> > > > > >> > BACKUP. All backup system information (prior to backup)
> > > will
> > > >> be
> > > >> > > > > restored
> > > >> > > > > >> > and all temporary data, related to a failed session, in
> > > HDFS
> > > >> > will
> > > >> > > be
> > > >> > > > > >> > deleted
> > > >> > > > > >> > RESTORE. We do not care about system data, because
> > restore
> > > >> does
> > > >> > > not
> > > >> > > > > >> change
> > > >> > > > > >> > it. Temporary data in HDFS will be cleaned up and table
> > > will
> > > >> be
> > > >> > > in a
> > > >> > > > > >> state
> > > >> > > > > >> > back to where it was before operation started.
> > > >> > > > > >> >
> > > >> > > > > >> > This is what user should expect in case of a failure.
> > > >> > > > > >> >
> > > >> > > > > >> > -Vlad
> > > >> > > > > >> >
> > > >> > > > > >> >
> > > >> > > > > >> > -Vlad
> > > >> > > > > >> >
> > > >> > > > > >> > On Fri, Sep 9, 2016 at 12:56 PM, Sean Busbey <
> > > >> busbey@apache.org
> > > >> > >
> > > >> > > > > wrote:
> > > >> > > > > >> >
> > > >> > > > > >> > > Failing in a consistent way, with docs that explain
> the
> > > >> > various
> > > >> > > > > >> > > expected failures would be sufficient.
> > > >> > > > > >> > >
> > > >> > > > > >> > > On Fri, Sep 9, 2016 at 12:16 PM, Vladimir Rodionov
> > > >> > > > > >> > > <vl...@gmail.com> wrote:
> > > >> > > > > >> > > > Do not worry Sean, doc is coming today as a preview
> > and
> > > >> our
> > > >> > > > writer
> > > >> > > > > >> > Frank
> > > >> > > > > >> > > > will be working on a putting  it into Apache repo.
> > > >> Timeline
> > > >> > > > > depends
> > > >> > > > > >> on
> > > >> > > > > >> > > > Franks schedule but I hope we will get it rather
> > sooner
> > > >> than
> > > >> > > > > later.
> > > >> > > > > >> > > >
> > > >> > > > > >> > > > As for failure testing, we are focusing only on a
> > > >> consistent
> > > >> > > > state
> > > >> > > > > >> of
> > > >> > > > > >> > > > backup system data in a presence of any type of
> > > failures,
> > > >> We
> > > >> > > are
> > > >> > > > > not
> > > >> > > > > >> > > going
> > > >> > > > > >> > > > to implement  anything more "fancy", than that. We
> > > allow
> > > >> > both:
> > > >> > > > > >> backup
> > > >> > > > > >> > and
> > > >> > > > > >> > > > restore to fail. What we do not allow is to have
> > system
> > > >> data
> > > >> > > > > >> corrupted.
> > > >> > > > > >> > > > Will it suffice for you? Do you have any other
> > > concerns,
> > > >> you
> > > >> > > > want
> > > >> > > > > >> us to
> > > >> > > > > >> > > > address?
> > > >> > > > > >> > > >
> > > >> > > > > >> > > > -Vlad
> > > >> > > > > >> > > >
> > > >> > > > > >> > > >
> > > >> > > > > >> > > > On Fri, Sep 9, 2016 at 10:56 AM, Sean Busbey <
> > > >> > > busbey@apache.org
> > > >> > > > >
> > > >> > > > > >> > wrote:
> > > >> > > > > >> > > >
> > > >> > > > > >> > > >> "docs will come to Apache soon" does not address
> my
> > > >> concern
> > > >> > > > > around
> > > >> > > > > >> > docs
> > > >> > > > > >> > > at
> > > >> > > > > >> > > >> all, unless said docs have already made it into
> the
> > > >> project
> > > >> > > > > repo. I
> > > >> > > > > >> > > don't
> > > >> > > > > >> > > >> want third party resources for using a major and
> > > >> important
> > > >> > > > > feature
> > > >> > > > > >> of
> > > >> > > > > >> > > the
> > > >> > > > > >> > > >> project, I want us to provide end users with what
> > they
> > > >> need
> > > >> > > to
> > > >> > > > > get
> > > >> > > > > >> the
> > > >> > > > > >> > > job
> > > >> > > > > >> > > >> done.
> > > >> > > > > >> > > >>
> > > >> > > > > >> > > >> I see some calls for patience on the failure
> > testing,
> > > but
> > > >> > the
> > > >> > > > > >> appeal
> > > >> > > > > >> > to
> > > >> > > > > >> > > us
> > > >> > > > > >> > > >> having done a bad job of requiring proper tests of
> > > >> previous
> > > >> > > > > >> features
> > > >> > > > > >> > > just
> > > >> > > > > >> > > >> makes me more concerned about not getting them
> > here. I
> > > >> > don't
> > > >> > > > want
> > > >> > > > > >> to
> > > >> > > > > >> > set
> > > >> > > > > >> > > >> yet another bad example that will then be pointed
> to
> > > in
> > > >> the
> > > >> > > > > future.
> > > >> > > > > >> > > >>
> > > >> > > > > >> > > >> On Sep 8, 2016 10:50, "Ted Yu" <
> yuzhihong@gmail.com
> > >
> > > >> > wrote:
> > > >> > > > > >> > > >>
> > > >> > > > > >> > > >> > Is there any concern which is not addressed ?
> > > >> > > > > >> > > >> >
> > > >> > > > > >> > > >> > Do we need another Vote thread ?
> > > >> > > > > >> > > >> >
> > > >> > > > > >> > > >> > Thanks
> > > >> > > > > >> > > >> >
> > > >> > > > > >> > > >> > On Thu, Sep 8, 2016 at 9:21 AM, Andrew Purtell <
> > > >> > > > > >> apurtell@apache.org
> > > >> > > > > >> > >
> > > >> > > > > >> > > >> > wrote:
> > > >> > > > > >> > > >> >
> > > >> > > > > >> > > >> > > Vlad,
> > > >> > > > > >> > > >> > >
> > > >> > > > > >> > > >> > > I apologize for using the term 'half-baked'
> in a
> > > way
> > > >> > that
> > > >> > > > > could
> > > >> > > > > >> > > seem a
> > > >> > > > > >> > > >> > > description of HBASE-7912. I meant that as a
> > > general
> > > >> > > > > >> hypothetical.
> > > >> > > > > >> > > >> > >
> > > >> > > > > >> > > >> > > On Wed, Sep 7, 2016 at 9:36 AM, Vladimir
> > Rodionov
> > > <
> > > >> > > > > >> > > >> > vladrodionov@gmail.com>
> > > >> > > > > >> > > >> > > wrote:
> > > >> > > > > >> > > >> > >
> > > >> > > > > >> > > >> > > > >> I'm not sure that "There is already lots
> of
> > > >> > > half-baked
> > > >> > > > > >> code
> > > >> > > > > >> > in
> > > >> > > > > >> > > the
> > > >> > > > > >> > > >> > > > branch,
> > > >> > > > > >> > > >> > > > so what's the harm in adding more?"
> > > >> > > > > >> > > >> > > >
> > > >> > > > > >> > > >> > > > I meant - not production - ready yet. This
> is
> > > 2.0
> > > >> > > > > development
> > > >> > > > > >> > > branch
> > > >> > > > > >> > > >> > and,
> > > >> > > > > >> > > >> > > > hence many features are in works,
> > > >> > > > > >> > > >> > > > not being tested well etc. I do not consider
> > > backup
> > > >> > as
> > > >> > > > half
> > > >> > > > > >> > baked
> > > >> > > > > >> > > >> > > feature -
> > > >> > > > > >> > > >> > > > it has passed our internal QA and has very
> > good
> > > >> doc,
> > > >> > > > which
> > > >> > > > > we
> > > >> > > > > >> > will
> > > >> > > > > >> > > >> > > provide
> > > >> > > > > >> > > >> > > > to Apache shortly.
> > > >> > > > > >> > > >> > > >
> > > >> > > > > >> > > >> > > > -Vlad
> > > >> > > > > >> > > >> > > >
> > > >> > > > > >> > > >> > > > On Wed, Sep 7, 2016 at 9:13 AM, Andrew
> > Purtell <
> > > >> > > > > >> > > apurtell@apache.org>
> > > >> > > > > >> > > >> > > > wrote:
> > > >> > > > > >> > > >> > > >
> > > >> > > > > >> > > >> > > > > We shouldn't admit half baked changes that
> > > won't
> > > >> be
> > > >> > > > > >> finished.
> > > >> > > > > >> > > >> However
> > > >> > > > > >> > > >> > > in
> > > >> > > > > >> > > >> > > > > this case the crew working on this feature
> > are
> > > >> long
> > > >> > > > > timers
> > > >> > > > > >> and
> > > >> > > > > >> > > less
> > > >> > > > > >> > > >> > > > likely
> > > >> > > > > >> > > >> > > > > than just about anyone to leave something
> > in a
> > > >> half
> > > >> > > > baked
> > > >> > > > > >> > > state. Of
> > > >> > > > > >> > > >> > > > course
> > > >> > > > > >> > > >> > > > > there is no guarantee how anything will
> turn
> > > out,
> > > >> > > but I
> > > >> > > > > am
> > > >> > > > > >> > > willing
> > > >> > > > > >> > > >> to
> > > >> > > > > >> > > >> > > > take
> > > >> > > > > >> > > >> > > > > a little on faith if they feel their best
> > path
> > > >> > > forward
> > > >> > > > > now
> > > >> > > > > >> is
> > > >> > > > > >> > to
> > > >> > > > > >> > > >> > merge
> > > >> > > > > >> > > >> > > to
> > > >> > > > > >> > > >> > > > > trunk. I only wish I had bandwidth to have
> > > done
> > > >> > some
> > > >> > > > real
> > > >> > > > > >> > > kicking
> > > >> > > > > >> > > >> of
> > > >> > > > > >> > > >> > > the
> > > >> > > > > >> > > >> > > > > tires by now. Maybe this week.
> > > >> > > > > >> > > >> > > > >
> > > >> > > > > >> > > >> > > > > (Yes, I'm using some of that time for this
> > > email
> > > >> > :-)
> > > >> > > > but
> > > >> > > > > I
> > > >> > > > > >> > type
> > > >> > > > > >> > > >> > fast.)
> > > >> > > > > >> > > >> > > > >
> > > >> > > > > >> > > >> > > > > That said, I would like to agitate for
> > making
> > > 2.0
> > > >> > > more
> > > >> > > > > real
> > > >> > > > > >> > and
> > > >> > > > > >> > > >> spend
> > > >> > > > > >> > > >> > > > some
> > > >> > > > > >> > > >> > > > > time on it now that I'm winding down with
> > > 0.98. I
> > > >> > > think
> > > >> > > > > >> that
> > > >> > > > > >> > > means
> > > >> > > > > >> > > >> > > > > branching for 2.0 real soon now and even
> > > evicting
> > > >> > > > things
> > > >> > > > > >> from
> > > >> > > > > >> > > 2.0
> > > >> > > > > >> > > >> > > branch
> > > >> > > > > >> > > >> > > > > that aren't finished or stable, leaving
> them
> > > only
> > > >> > > once
> > > >> > > > > >> again
> > > >> > > > > >> > in
> > > >> > > > > >> > > the
> > > >> > > > > >> > > >> > > > master
> > > >> > > > > >> > > >> > > > > branch. Or, maybe just evicting them.
> Let's
> > > take
> > > >> it
> > > >> > > > case
> > > >> > > > > by
> > > >> > > > > >> > > case.
> > > >> > > > > >> > > >> > > > >
> > > >> > > > > >> > > >> > > > > I think this feature can come in
> relatively
> > > >> safely.
> > > >> > > As
> > > >> > > > > >> added
> > > >> > > > > >> > > >> > insurance,
> > > >> > > > > >> > > >> > > > > let's admit the possibility it could be
> > > reverted
> > > >> on
> > > >> > > the
> > > >> > > > > 2.0
> > > >> > > > > >> > > branch
> > > >> > > > > >> > > >> if
> > > >> > > > > >> > > >> > > > folks
> > > >> > > > > >> > > >> > > > > working on stabilizing 2.0 decide to evict
> > it
> > > >> > because
> > > >> > > > it
> > > >> > > > > is
> > > >> > > > > >> > > >> > unfinished
> > > >> > > > > >> > > >> > > or
> > > >> > > > > >> > > >> > > > > unstable, because that certainly can
> > happen. I
> > > >> > would
> > > >> > > > > >> expect if
> > > >> > > > > >> > > talk
> > > >> > > > > >> > > >> > > like
> > > >> > > > > >> > > >> > > > > that starts, we'd get help finishing or
> > > >> stabilizing
> > > >> > > > > what's
> > > >> > > > > >> > under
> > > >> > > > > >> > > >> > > > discussion
> > > >> > > > > >> > > >> > > > > for revert. Or, we'd have a revert. Either
> > way
> > > >> the
> > > >> > > > > outcome
> > > >> > > > > >> is
> > > >> > > > > >> > > >> > > acceptable.
> > > >> > > > > >> > > >> > > > >
> > > >> > > > > >> > > >> > > > >
> > > >> > > > > >> > > >> > > > > On Wed, Sep 7, 2016 at 8:56 AM, Dima
> Spivak
> > <
> > > >> > > > > >> > > dimaspivak@apache.org
> > > >> > > > > >> > > >> >
> > > >> > > > > >> > > >> > > > wrote:
> > > >> > > > > >> > > >> > > > >
> > > >> > > > > >> > > >> > > > > > I'm not sure that "There is already lots
> > of
> > > >> > > > half-baked
> > > >> > > > > >> code
> > > >> > > > > >> > in
> > > >> > > > > >> > > >> the
> > > >> > > > > >> > > >> > > > > branch,
> > > >> > > > > >> > > >> > > > > > so what's the harm in adding more?" is a
> > > good
> > > >> > code
> > > >> > > > > commit
> > > >> > > > > >> > > >> > philosophy
> > > >> > > > > >> > > >> > > > for
> > > >> > > > > >> > > >> > > > > a
> > > >> > > > > >> > > >> > > > > > fault-tolerant distributed data store.
> ;)
> > > >> > > > > >> > > >> > > > > >
> > > >> > > > > >> > > >> > > > > > More seriously, a lack of test coverage
> > for
> > > >> > > existing
> > > >> > > > > >> > features
> > > >> > > > > >> > > >> > > shouldn't
> > > >> > > > > >> > > >> > > > > be
> > > >> > > > > >> > > >> > > > > > used as justification for introducing
> new
> > > >> > features
> > > >> > > > with
> > > >> > > > > >> the
> > > >> > > > > >> > > same
> > > >> > > > > >> > > >> > > > > > shortcomings. Ultimately, it's the end
> > user
> > > who
> > > >> > > will
> > > >> > > > > feel
> > > >> > > > > >> > the
> > > >> > > > > >> > > >> pain,
> > > >> > > > > >> > > >> > > so
> > > >> > > > > >> > > >> > > > > > shouldn't we do everything we can to
> > > mitigate
> > > >> > that?
> > > >> > > > > >> > > >> > > > > >
> > > >> > > > > >> > > >> > > > > > -Dima
> > > >> > > > > >> > > >> > > > > >
> > > >> > > > > >> > > >> > > > > > On Wed, Sep 7, 2016 at 8:46 AM, Vladimir
> > > >> > Rodionov <
> > > >> > > > > >> > > >> > > > > vladrodionov@gmail.com>
> > > >> > > > > >> > > >> > > > > > wrote:
> > > >> > > > > >> > > >> > > > > >
> > > >> > > > > >> > > >> > > > > > > Sean,
> > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > >> > > > > > > * have docs
> > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > >> > > > > > > Agree. We have a doc and backup is the
> > > most
> > > >> > > > > documented
> > > >> > > > > >> > > feature
> > > >> > > > > >> > > >> > :),
> > > >> > > > > >> > > >> > > we
> > > >> > > > > >> > > >> > > > > > will
> > > >> > > > > >> > > >> > > > > > > release it shortly to Apache.
> > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > >> > > > > > > * have sunny-day correctness tests
> > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > >> > > > > > > Feature has  close to 60 test cases,
> > which
> > > >> run
> > > >> > > for
> > > >> > > > > >> approx
> > > >> > > > > >> > 30
> > > >> > > > > >> > > >> min.
> > > >> > > > > >> > > >> > > We
> > > >> > > > > >> > > >> > > > > can
> > > >> > > > > >> > > >> > > > > > > add more, if community do not mind :)
> > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > >> > > > > > > * have correctness-in-face-of-failure
> > > tests
> > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > >> > > > > > > Any examples of these tests in
> existing
> > > >> > features?
> > > >> > > > In
> > > >> > > > > >> > works,
> > > >> > > > > >> > > we
> > > >> > > > > >> > > >> > > have a
> > > >> > > > > >> > > >> > > > > > clear
> > > >> > > > > >> > > >> > > > > > > understanding of what should be done
> by
> > > the
> > > >> > time
> > > >> > > of
> > > >> > > > > 2.0
> > > >> > > > > >> > > >> release.
> > > >> > > > > >> > > >> > > > > > > That is very close goal for us, to
> > verify
> > > IT
> > > >> > > monkey
> > > >> > > > > for
> > > >> > > > > >> > > >> existing
> > > >> > > > > >> > > >> > > > code.
> > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > >> > > > > > > * don't rely on things outside of
> HBase
> > > for
> > > >> > > normal
> > > >> > > > > >> > operation
> > > >> > > > > >> > > >> > (okay
> > > >> > > > > >> > > >> > > > for
> > > >> > > > > >> > > >> > > > > > > advanced operation)
> > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > >> > > > > > > We do not.
> > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > >> > > > > > > Enormous time has been spent already
> on
> > > the
> > > >> > > > > development
> > > >> > > > > >> > and
> > > >> > > > > >> > > >> > testing
> > > >> > > > > >> > > >> > > > the
> > > >> > > > > >> > > >> > > > > > > feature, it has passed our internal
> > tests
> > > and
> > > >> > > many
> > > >> > > > > >> rounds
> > > >> > > > > >> > of
> > > >> > > > > >> > > >> code
> > > >> > > > > >> > > >> > > > > reviews
> > > >> > > > > >> > > >> > > > > > > by HBase committers. We do not mind if
> > > >> someone
> > > >> > > from
> > > >> > > > > >> HBase
> > > >> > > > > >> > > >> > community
> > > >> > > > > >> > > >> > > > > > > (outside of HW) will review the code,
> > but
> > > it
> > > >> > will
> > > >> > > > > >> probably
> > > >> > > > > >> > > >> takes
> > > >> > > > > >> > > >> > > > > forever
> > > >> > > > > >> > > >> > > > > > to
> > > >> > > > > >> > > >> > > > > > > wait for volunteer?, the feature is
> > quite
> > > >> large
> > > >> > > > (1MB+
> > > >> > > > > >> > > >> cumulative
> > > >> > > > > >> > > >> > > > patch)
> > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > >> > > > > > > 2.0 branch is full of half baked
> > features,
> > > >> most
> > > >> > > of
> > > >> > > > > them
> > > >> > > > > >> > are
> > > >> > > > > >> > > in
> > > >> > > > > >> > > >> > > active
> > > >> > > > > >> > > >> > > > > > > development, therefore I am not
> > following
> > > you
> > > >> > > here,
> > > >> > > > > >> Sean?
> > > >> > > > > >> > > Why
> > > >> > > > > >> > > >> > > > > HBASE-7912
> > > >> > > > > >> > > >> > > > > > is
> > > >> > > > > >> > > >> > > > > > > not good enough yet to be integrated
> > into
> > > 2.0
> > > >> > > > branch?
> > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > >> > > > > > > -Vlad
> > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > >> > > > > > > On Wed, Sep 7, 2016 at 8:23 AM, Sean
> > > Busbey <
> > > >> > > > > >> > > busbey@apache.org
> > > >> > > > > >> > > >> >
> > > >> > > > > >> > > >> > > > wrote:
> > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > >> > > > > > > > On Tue, Sep 6, 2016 at 10:36 PM,
> Josh
> > > >> Elser <
> > > >> > > > > >> > > >> > > josh.elser@gmail.com>
> > > >> > > > > >> > > >> > > > > > > wrote:
> > > >> > > > > >> > > >> > > > > > > > > So, the answer to Sean's original
> > > >> question
> > > >> > is
> > > >> > > > "as
> > > >> > > > > >> > > robust as
> > > >> > > > > >> > > >> > > > > snapshots
> > > >> > > > > >> > > >> > > > > > > > > presently are"? (independence of
> > > >> > > backup/restore
> > > >> > > > > >> > failure
> > > >> > > > > >> > > >> > > tolerance
> > > >> > > > > >> > > >> > > > > > from
> > > >> > > > > >> > > >> > > > > > > > > snapshot failure tolerance)
> > > >> > > > > >> > > >> > > > > > > > >
> > > >> > > > > >> > > >> > > > > > > > > Is this just a question WRT
> context
> > of
> > > >> the
> > > >> > > > > change,
> > > >> > > > > >> or
> > > >> > > > > >> > > is it
> > > >> > > > > >> > > >> > > means
> > > >> > > > > >> > > >> > > > > > for a
> > > >> > > > > >> > > >> > > > > > > > veto
> > > >> > > > > >> > > >> > > > > > > > > from you, Sean? Just trying to
> make
> > > sure
> > > >> > I'm
> > > >> > > > > >> following
> > > >> > > > > >> > > >> along
> > > >> > > > > >> > > >> > > > > > > adequately.
> > > >> > > > > >> > > >> > > > > > > > >
> > > >> > > > > >> > > >> > > > > > > > >
> > > >> > > > > >> > > >> > > > > > > >
> > > >> > > > > >> > > >> > > > > > > > I'd say ATM I'm -0, bordering on -1
> > but
> > > not
> > > >> > for
> > > >> > > > > >> reasons
> > > >> > > > > >> > I
> > > >> > > > > >> > > can
> > > >> > > > > >> > > >> > > > > > articulate
> > > >> > > > > >> > > >> > > > > > > > well.
> > > >> > > > > >> > > >> > > > > > > >
> > > >> > > > > >> > > >> > > > > > > > Here's an attempt.
> > > >> > > > > >> > > >> > > > > > > >
> > > >> > > > > >> > > >> > > > > > > > We've been trying to move, as a
> > > community,
> > > >> > > > towards
> > > >> > > > > >> > > minimizing
> > > >> > > > > >> > > >> > > risk
> > > >> > > > > >> > > >> > > > to
> > > >> > > > > >> > > >> > > > > > > > downstream folks by getting
> "complete
> > > >> enough
> > > >> > > for
> > > >> > > > > use"
> > > >> > > > > >> > > gates
> > > >> > > > > >> > > >> in
> > > >> > > > > >> > > >> > > > place
> > > >> > > > > >> > > >> > > > > > > > before we introduce new features.
> This
> > > was
> > > >> > > > spurred
> > > >> > > > > >> by a
> > > >> > > > > >> > > some
> > > >> > > > > >> > > >> > > > features
> > > >> > > > > >> > > >> > > > > > > > getting in half-baked and never
> making
> > > it
> > > >> to
> > > >> > > "can
> > > >> > > > > >> really
> > > >> > > > > >> > > use"
> > > >> > > > > >> > > >> > > > status
> > > >> > > > > >> > > >> > > > > > > > (I'm thinking of distributed log
> > replay
> > > and
> > > >> > the
> > > >> > > > > >> zk-less
> > > >> > > > > >> > > >> > > assignment
> > > >> > > > > >> > > >> > > > > > > > stuff, I don't recall if there was
> > > more).
> > > >> > > > > >> > > >> > > > > > > >
> > > >> > > > > >> > > >> > > > > > > > The gates, generally, included
> things
> > > like:
> > > >> > > > > >> > > >> > > > > > > >
> > > >> > > > > >> > > >> > > > > > > > * have docs
> > > >> > > > > >> > > >> > > > > > > > * have sunny-day correctness tests
> > > >> > > > > >> > > >> > > > > > > > * have
> correctness-in-face-of-failure
> > > tests
> > > >> > > > > >> > > >> > > > > > > > * don't rely on things outside of
> > HBase
> > > for
> > > >> > > > normal
> > > >> > > > > >> > > operation
> > > >> > > > > >> > > >> > > (okay
> > > >> > > > > >> > > >> > > > > for
> > > >> > > > > >> > > >> > > > > > > > advanced operation)
> > > >> > > > > >> > > >> > > > > > > >
> > > >> > > > > >> > > >> > > > > > > > As an example, we kept the MOB work
> > off
> > > in
> > > >> a
> > > >> > > > branch
> > > >> > > > > >> and
> > > >> > > > > >> > > out
> > > >> > > > > >> > > >> of
> > > >> > > > > >> > > >> > > > master
> > > >> > > > > >> > > >> > > > > > > > until it could pass these criteria.
> > The
> > > big
> > > >> > > > > exemption
> > > >> > > > > >> > > we've
> > > >> > > > > >> > > >> had
> > > >> > > > > >> > > >> > > to
> > > >> > > > > >> > > >> > > > > > > > this was the hbase-spark
> integration,
> > > where
> > > >> > we
> > > >> > > > all
> > > >> > > > > >> > agreed
> > > >> > > > > >> > > it
> > > >> > > > > >> > > >> > > could
> > > >> > > > > >> > > >> > > > > > > > land in master because it was very
> > well
> > > >> > > isolated
> > > >> > > > > (the
> > > >> > > > > >> > > slide
> > > >> > > > > >> > > >> > away
> > > >> > > > > >> > > >> > > > from
> > > >> > > > > >> > > >> > > > > > > > including docs as a first-class part
> > of
> > > >> > > building
> > > >> > > > up
> > > >> > > > > >> that
> > > >> > > > > >> > > >> > > > integration
> > > >> > > > > >> > > >> > > > > > > > has led me to doubt the wisdom of
> this
> > > >> > > decision).
> > > >> > > > > >> > > >> > > > > > > >
> > > >> > > > > >> > > >> > > > > > > > We've also been treating inclusion
> in
> > a
> > > >> > > "probably
> > > >> > > > > >> will
> > > >> > > > > >> > be
> > > >> > > > > >> > > >> > > released
> > > >> > > > > >> > > >> > > > to
> > > >> > > > > >> > > >> > > > > > > > downstream" branches as a higher
> bar,
> > > >> > requiring
> > > >> > > > > >> > > >> > > > > > > >
> > > >> > > > > >> > > >> > > > > > > > * don't moderately impact
> performance
> > > when
> > > >> > the
> > > >> > > > > >> feature
> > > >> > > > > >> > > isn't
> > > >> > > > > >> > > >> in
> > > >> > > > > >> > > >> > > use
> > > >> > > > > >> > > >> > > > > > > > * don't severely impact performance
> > when
> > > >> the
> > > >> > > > > feature
> > > >> > > > > >> is
> > > >> > > > > >> > in
> > > >> > > > > >> > > >> use
> > > >> > > > > >> > > >> > > > > > > > * either default-to-on or show
> enough
> > > >> demand
> > > >> > to
> > > >> > > > > >> believe
> > > >> > > > > >> > a
> > > >> > > > > >> > > >> > > > non-trivial
> > > >> > > > > >> > > >> > > > > > > > number of folks will turn the
> feature
> > on
> > > >> > > > > >> > > >> > > > > > > >
> > > >> > > > > >> > > >> > > > > > > > The above has kept MOB and
> hbase-spark
> > > >> > > > integration
> > > >> > > > > >> out
> > > >> > > > > >> > of
> > > >> > > > > >> > > >> > > branch-1,
> > > >> > > > > >> > > >> > > > > > > > presumably while they've "gotten
> more
> > > >> stable"
> > > >> > > in
> > > >> > > > > >> master
> > > >> > > > > >> > > from
> > > >> > > > > >> > > >> > the
> > > >> > > > > >> > > >> > > > odd
> > > >> > > > > >> > > >> > > > > > > > vendor inclusion.
> > > >> > > > > >> > > >> > > > > > > >
> > > >> > > > > >> > > >> > > > > > > > Are we going to have a 2.0 release
> > > before
> > > >> the
> > > >> > > end
> > > >> > > > > of
> > > >> > > > > >> the
> > > >> > > > > >> > > >> year?
> > > >> > > > > >> > > >> > > > We're
> > > >> > > > > >> > > >> > > > > > > > coming up on 1.5 years since the
> > > release of
> > > >> > > > version
> > > >> > > > > >> 1.0;
> > > >> > > > > >> > > >> seems
> > > >> > > > > >> > > >> > > like
> > > >> > > > > >> > > >> > > > > > > > it's about time, though I haven't
> seen
> > > any
> > > >> > > > concrete
> > > >> > > > > >> > plans
> > > >> > > > > >> > > >> this
> > > >> > > > > >> > > >> > > > year.
> > > >> > > > > >> > > >> > > > > > > > Presuming we are going to have one
> by
> > > the
> > > >> end
> > > >> > > of
> > > >> > > > > the
> > > >> > > > > >> > > year, it
> > > >> > > > > >> > > >> > > > seems a
> > > >> > > > > >> > > >> > > > > > > > bit close to still be adding in
> > > "features
> > > >> > that
> > > >> > > > need
> > > >> > > > > >> > > maturing"
> > > >> > > > > >> > > >> > on
> > > >> > > > > >> > > >> > > > the
> > > >> > > > > >> > > >> > > > > > > > branch.
> > > >> > > > > >> > > >> > > > > > > >
> > > >> > > > > >> > > >> > > > > > > > The lack of a concrete plan for 2.0
> > > keeps
> > > >> me
> > > >> > > from
> > > >> > > > > >> > > considering
> > > >> > > > > >> > > >> > > these
> > > >> > > > > >> > > >> > > > > > > > things blocker at the moment. But I
> > know
> > > >> > first
> > > >> > > > hand
> > > >> > > > > >> how
> > > >> > > > > >> > > much
> > > >> > > > > >> > > >> > > > trouble
> > > >> > > > > >> > > >> > > > > > > > folks have had with other features
> > that
> > > >> have
> > > >> > > gone
> > > >> > > > > >> into
> > > >> > > > > >> > > >> > downstream
> > > >> > > > > >> > > >> > > > > > > > facing releases without robustness
> > > checks
> > > >> > (i.e.
> > > >> > > > > >> > > replication),
> > > >> > > > > >> > > >> > and
> > > >> > > > > >> > > >> > > > I'm
> > > >> > > > > >> > > >> > > > > > > > concerned about what we're setting
> up
> > if
> > > >> 2.0
> > > >> > > goes
> > > >> > > > > out
> > > >> > > > > >> > with
> > > >> > > > > >> > > >> this
> > > >> > > > > >> > > >> > > > > > > > feature in its current state.
> > > >> > > > > >> > > >> > > > > > > >
> > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > >> > > > > >
> > > >> > > > > >> > > >> > > > >
> > > >> > > > > >> > > >> > > > >
> > > >> > > > > >> > > >> > > > >
> > > >> > > > > >> > > >> > > > > --
> > > >> > > > > >> > > >> > > > > Best regards,
> > > >> > > > > >> > > >> > > > >
> > > >> > > > > >> > > >> > > > >    - Andy
> > > >> > > > > >> > > >> > > > >
> > > >> > > > > >> > > >> > > > > Problems worthy of attack prove their
> worth
> > by
> > > >> > > hitting
> > > >> > > > > >> back. -
> > > >> > > > > >> > > Piet
> > > >> > > > > >> > > >> > > Hein
> > > >> > > > > >> > > >> > > > > (via Tom White)
> > > >> > > > > >> > > >> > > > >
> > > >> > > > > >> > > >> > > >
> > > >> > > > > >> > > >> > >
> > > >> > > > > >> > > >> > >
> > > >> > > > > >> > > >> > >
> > > >> > > > > >> > > >> > > --
> > > >> > > > > >> > > >> > > Best regards,
> > > >> > > > > >> > > >> > >
> > > >> > > > > >> > > >> > >    - Andy
> > > >> > > > > >> > > >> > >
> > > >> > > > > >> > > >> > > Problems worthy of attack prove their worth by
> > > >> hitting
> > > >> > > > back.
> > > >> > > > > -
> > > >> > > > > >> > Piet
> > > >> > > > > >> > > >> Hein
> > > >> > > > > >> > > >> > > (via Tom White)
> > > >> > > > > >> > > >> > >
> > > >> > > > > >> > > >> >
> > > >> > > > > >> > > >>
> > > >> > > > > >> > >
> > > >> > > > > >> >
> > > >> > > > > >>
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> > >
> > >
> > > --
> > > busbey
> > >
> >
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Ted Yu <yu...@gmail.com>]

Here is my environment:

Java HotSpot(TM) 64-Bit Server VM warning: ignoring option
MaxPermSize=512M; support was removed in 8.0
Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5;
2015-11-10T08:41:47-08:00)
Maven home: /Users/tyu/apache-maven-3.3.9
Java version: 1.8.0_91, vendor: Oracle Corporation
Java home:
/Library/Java/JavaVirtualMachines/jdk1.8.0_91.jdk/Contents/Home/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.11.3", arch: "x86_64", family: "mac"

I just tried the same mvn command you gave:

[INFO] Apache HBase - Exemplar for hbase-client archetype . SUCCESS [
 2.078 s]
[INFO] Apache HBase - Exemplar for hbase-shaded-client archetype SUCCESS [
 2.082 s]
[INFO] Apache HBase - Archetype builder ................... SUCCESS [
12.513 s]
[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 02:28 min
[INFO] Finished at: 2016-10-12T12:57:19-07:00
[INFO] Final Memory: 205M/983M


Is this the top commit you see in your local workspace?

commit a072f6f49a26a7259ff2aaef6cb56d85eb592482

On Wed, Oct 12, 2016 at 12:31 PM, Stack <st...@duboce.net> wrote:

> More info:
>
> stack@ve0524:~/hbase.git$ git checkout origin/HBASE-7912 -b 7912v2
> Branch 7912v2 set up to track remote branch HBASE-7912 from origin.
> Switched to a new branch '7912v2'
> stack@ve0524:~/hbase.git$ java -version
> java version "1.8.0_101"
> Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
> Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode)
> stack@ve0524:~/hbase.git$ mvn clean install -DskipTests &> /tmp/out.txt
>
> ...
>
> St.Ack
>
>
> On Wed, Oct 12, 2016 at 12:29 PM, Stack <st...@duboce.net> wrote:
>
> > Interesting. When I try it fails w/ below:
> >
> > [INFO] 26 warnings
> > 322 [INFO] -------------------------------------------------------------
> > 323 [INFO] -------------------------------------------------------------
> > 324 [ERROR] COMPILATION ERROR :
> > 325 [INFO] -------------------------------------------------------------
> > 326 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.java:[48,8]
> > org.apache.hadoop.hbase.io.encoding.rowindexV2.RowIndexCodecV2 is not
> > abstract and does not override abstract method createSeeker(org.ap
> >  ache.hadoop.hbase.CellComparator,org.apache.hadoop.hbase.io.encoding.
> HFileBlockDecodingContext)
> > in org.apache.hadoop.hbase.io.encoding.DataBlockEncoder
> > 327 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.java:[143,3]
> > method does not override or implement a method from a supertype
> > 328 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.java:[147,29]
> > incompatible types: java.nio.ByteBuffer cannot be converted to
> > org.apache.hadoop.hbase.nio.ByteBuff
> > 329 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.java:[148,33]
> > cannot find symbol
> > 330   symbol:   method getKeyDeepCopy()
> > 331   location: variable seeker of type org.apache.hadoop.hbase.io.
> > encoding.DataBlockEncoder.EncodedSeeker
> > 332 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.java:[153,3]
> > method does not override or implement a method from a supertype
> > 333 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> > apache/hadoop/hbase/io/encoding/rowindexV1/RowIndexCodecV1.java:[45,8]
> > org.apache.hadoop.hbase.io.encoding.rowindexV1.RowIndexCodecV1 is not
> > abstract and does not override abstract method createSeeker(org.ap
> >  ache.hadoop.hbase.CellComparator,org.apache.hadoop.hbase.io.encoding.
> HFileBlockDecodingContext)
> > in org.apache.hadoop.hbase.io.encoding.DataBlockEncoder
> > 334 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> > apache/hadoop/hbase/io/encoding/rowindexV1/RowIndexCodecV1.java:[145,3]
> > method does not override or implement a method from a supertype
> > 335 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> > apache/hadoop/hbase/io/encoding/rowindexV1/RowIndexCodecV1.java:[158,3]
> > method does not override or implement a method from a supertype
> > 336 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.java:[46,8]
> > org.apache.hadoop.hbase.io.encoding.rowindexV2.RowIndexSeekerV2 is not
> > abstract and does not override abstract method compareKey(org.ap
> >  ache.hadoop.hbase.CellComparator,org.apache.hadoop.hbase.Cell) in
> > org.apache.hadoop.hbase.io.encoding.DataBlockEncoder.EncodedSeeker
> > 337 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.java:[79,3]
> > method does not override or implement a method from a supertype
> > 338 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.java:[117,3]
> > method does not override or implement a method from a supertype
> > 339 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.java:[190,3]
> > method does not override or implement a method from a supertype
> > 340 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.java:[214,3]
> > method does not override or implement a method from a supertype
> > 341 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.java:[349,3]
> > method does not override or implement a method from a supertype
> > 342 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.java:[355,3]
> > method does not override or implement a method from a supertype
> > 343 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> > apache/hadoop/hbase/io/encoding/rowindexV2/
> RowIndexSeekerV2.java:[421,36]
> > no suitable method found for uncompressTags(java.nio.
> > ByteBuffer,byte[],int,int)
> > 344     method org.apache.hadoop.hbase.io.TagCompressionContext.
> > uncompressTags(java.io.InputStream,byte[],int,int) is not applicable
> > 345       (argument mismatch; java.nio.ByteBuffer cannot be converted to
> > java.io.InputStream)
> > 346     method org.apache.hadoop.hbase.io.TagCompressionContext.
> > uncompressTags(org.apache.hadoop.hbase.nio.ByteBuff,byte[],int,int) is
> > not applicable
> > 347       (argument mismatch; java.nio.ByteBuffer cannot be converted to
> > org.apache.hadoop.hbase.nio.ByteBuff)
> >
> > ....
> >
> > St.Ack
> >
> > On Tue, Oct 11, 2016 at 5:12 PM, Apekshit Sharma <ap...@cloudera.com>
> > wrote:
> >
> >> @stack, it compiled for me.
> >>
> >> Also tried few commands, and have to say, it's well designed from user
> >> commands perspective.
> >>
> >> On Tue, Oct 11, 2016 at 5:08 PM, Stack <st...@duboce.net> wrote:
> >>
> >> > On Tue, Oct 11, 2016 at 1:50 PM, Vladimir Rodionov <
> >> vladrodionov@gmail.com
> >> > >
> >> > wrote:
> >> >
> >> > > Michael,
> >> > >
> >> > > Its in HBASE-7912
> >> > >
> >> > > This is tip of git log:
> >> > >
> >> > > commit a072f6f49a26a7259ff2aaef6cb56d85eb592482
> >> > > Author: Frank Welsch <fw...@jps.net>
> >> > > Date:   Fri Sep 23 18:00:42 2016 -0400
> >> > >
> >> > >     HBASE-16574 Book updates for backup and restore
> >> > >
> >> > > commit b14e2ab1c24e65ff88dd4c579acf83cb4ed0605e
> >> > > Author: tedyu <yu...@gmail.com>
> >> > > Date:   Wed Oct 5 16:29:40 2016 -0700
> >> > >
> >> > >     HBASE-16727 Backup refactoring: remove MR dependencies from
> >> HMaster
> >> > > (Vladimir Rodionov)
> >> > >
> >> > >
> >> > Thanks. I have that. I tried it and it doesn't compile for me. Does it
> >> > compile for you?
> >> > Thanks,
> >> > M
> >> >
> >> >
> >> >
> >> > > -Vlad
> >> > >
> >> > > On Tue, Oct 11, 2016 at 1:44 PM, Stack <st...@duboce.net> wrote:
> >> > >
> >> > > > Which branch do I check out to try it? HBASE-7912 is not it. I
> don't
> >> > see
> >> > > an
> >> > > > HBASE-16727...
> >> > > > Thanks,
> >> > > > M
> >> > > >
> >> > > >
> >> > > > On Tue, Oct 11, 2016 at 11:41 AM, Vladimir Rodionov <
> >> > > > vladrodionov@gmail.com>
> >> > > > wrote:
> >> > > >
> >> > > > > The last patch is on review board:
> >> > > > > https://reviews.apache.org/r/52748
> >> > > > >
> >> > > > > On Mon, Oct 10, 2016 at 3:52 PM, Vladimir Rodionov <
> >> > > > vladrodionov@gmail.com
> >> > > > > >
> >> > > > > wrote:
> >> > > > >
> >> > > > > > >> How hard to put in an hbase-backup module? hbase-server is
> >> fat
> >> > > > enough
> >> > > > > > >> already. Could be done as a follow-up.
> >> > > > > >
> >> > > > > > https://issues.apache.org/jira/browse/HBASE-16727?
> >> > > > > > focusedCommentId=15531237&page=com.atlassian.jira.
> >> > > > > > plugin.system.issuetabpanels:comment-tabpanel#comment-
> 15531237
> >> > > > > >
> >> > > > > > Can we do merge first? Then we can discuss separate module.
> >> > > > > >
> >> > > > > > -Vlad
> >> > > > > >
> >> > > > > > On Mon, Oct 10, 2016 at 3:44 PM, Ted Yu <yu...@gmail.com>
> >> > wrote:
> >> > > > > >
> >> > > > > >> Looks like the first quote was cut off.
> >> > > > > >> The original sentence was:
> >> > > > > >>
> >> > > > > >> bq. no mapreduce job launched from master or region server.
> >> > > > > >>
> >> > > > > >> mapreduce job is launched from the node where command line
> >> tool is
> >> > > > run.
> >> > > > > >>
> >> > > > > >> On Mon, Oct 10, 2016 at 3:38 PM, Stack <st...@duboce.net>
> >> wrote:
> >> > > > > >>
> >> > > > > >> > bq. launched from master or region server.
> >> > > > > >> >
> >> > > > > >> > What does this mean please? Has to be run from Master or
> >> > > > RegionServer?
> >> > > > > >> Can
> >> > > > > >> > it be run from another node altogether?
> >> > > > > >> >
> >> > > > > >> > On Mon, Oct 10, 2016 at 1:44 PM, Vladimir Rodionov <
> >> > > > > >> vladrodionov@gmail.com
> >> > > > > >> > >
> >> > > > > >> > wrote:
> >> > > > > >> >
> >> > > > > >> > > >> mapreduce dependency has been moved to client side -
> no
> >> > > > mapreduce
> >> > > > > >> job
> >> > > > > >> > >
> >> > > > > >> > > 1. We have no code in the client module anymore, due to
> >> > > dependency
> >> > > > > on
> >> > > > > >> > > internal server API (HFile and WAL access).
> >> > > > > >> > > 2. Backup/ restore are client - driven operations, but
> all
> >> the
> >> > > > code
> >> > > > > >> > resides
> >> > > > > >> > > in the server module
> >> > > > > >> > >
> >> > > > > >> >
> >> > > > > >> > How hard to put in an hbase-backup module? hbase-server is
> >> fat
> >> > > > enough
> >> > > > > >> > already. Could be done as a follow-up.
> >> > > > > >> >
> >> > > > > >> > Thanks,
> >> > > > > >> > St.Ack
> >> > > > > >> >
> >> > > > > >> >
> >> > > > > >> >
> >> > > > > >> > > 3. No MR in Master, no procedure - driven execution.
> >> > > > > >> > > 4. Old good MR from command-line.
> >> > > > > >> > > 5. Security was simplified and now only super-user is
> >> allowed
> >> > to
> >> > > > run
> >> > > > > >> > > backup/restores.
> >> > > > > >> > > 6. HBase Backup API was gone due to 1. Now only
> >> command-line
> >> > > > access
> >> > > > > to
> >> > > > > >> > > backup tools.
> >> > > > > >> > >
> >> > > > > >> > > These consequences of refactoring has been discussed in
> >> > > > HBASE-16727.
> >> > > > > >> > >
> >> > > > > >> > > -Vlad
> >> > > > > >> > >
> >> > > > > >> > >
> >> > > > > >> > >
> >> > > > > >> > >
> >> > > > > >> > > On Mon, Oct 10, 2016 at 1:31 PM, Ted Yu <
> >> yuzhihong@gmail.com>
> >> > > > > wrote:
> >> > > > > >> > >
> >> > > > > >> > > > Reviving this thread.
> >> > > > > >> > > >
> >> > > > > >> > > > The following has taken place:
> >> > > > > >> > > >
> >> > > > > >> > > > mapreduce dependency has been moved to client side - no
> >> > > > mapreduce
> >> > > > > >> job
> >> > > > > >> > > > launched from master or region server.
> >> > > > > >> > > > document patch (HBASE-16574) has been integrated.
> >> > > > > >> > > > Updated mega patch has been attached to HBASE-14123:
> this
> >> > > covers
> >> > > > > the
> >> > > > > >> > > > refactor in #1 above and the protobuf 3 merge.
> >> > > > > >> > > >
> >> > > > > >> > > > If community has more feedback on the merge proposal, I
> >> > would
> >> > > > love
> >> > > > > >> to
> >> > > > > >> > > hear
> >> > > > > >> > > > it.
> >> > > > > >> > > >
> >> > > > > >> > > > Thanks
> >> > > > > >> > > >
> >> > > > > >> > > > On Thu, Sep 22, 2016 at 10:31 AM, Sean Busbey <
> >> > > > > busbey@cloudera.com>
> >> > > > > >> > > wrote:
> >> > > > > >> > > >
> >> > > > > >> > > > > I'd like to see the docs proposed on HBASE-16574
> >> > integrated
> >> > > > into
> >> > > > > >> our
> >> > > > > >> > > > > project's documentation prior to merge.
> >> > > > > >> > > > >
> >> > > > > >> > > > > On Thu, Sep 22, 2016 at 9:02 AM, Ted Yu <
> >> > > yuzhihong@gmail.com>
> >> > > > > >> wrote:
> >> > > > > >> > > > > > This feature can be marked experimental due to some
> >> > > > > limitations
> >> > > > > >> > such
> >> > > > > >> > > as
> >> > > > > >> > > > > > security.
> >> > > > > >> > > > > >
> >> > > > > >> > > > > > Your previous round of comments have been
> addressed.
> >> > > > > >> > > > > > Command line tool has gone through:
> >> > > > > >> > > > > >
> >> > > > > >> > > > > > HBASE-16620 Fix backup command-line tool usability
> >> > issues
> >> > > > > >> > > > > > HBASE-16655 hbase backup describe with incorrect
> >> backup
> >> > id
> >> > > > > >> results
> >> > > > > >> > in
> >> > > > > >> > > > NPE
> >> > > > > >> > > > > >
> >> > > > > >> > > > > > The updated doc has been attached to HBASE-16574.
> >> > > > > >> > > > > >
> >> > > > > >> > > > > > Cheers
> >> > > > > >> > > > > >
> >> > > > > >> > > > > > On Thu, Sep 22, 2016 at 8:53 AM, Stack <
> >> > stack@duboce.net>
> >> > > > > >> wrote:
> >> > > > > >> > > > > >
> >> > > > > >> > > > > >> On Wed, Sep 21, 2016 at 7:43 AM, Ted Yu <
> >> > > > yuzhihong@gmail.com
> >> > > > > >
> >> > > > > >> > > wrote:
> >> > > > > >> > > > > >>
> >> > > > > >> > > > > >> > Are there more (review) comments ?
> >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> Are outstanding comments addressed?
> >> > > > > >> > > > > >>
> >> > > > > >> > > > > >> I don't see answer to my 'is this
> experimental/will
> >> it
> >> > be
> >> > > > > >> marked
> >> > > > > >> > > > > >> experimental' question.
> >> > > > > >> > > > > >>
> >> > > > > >> > > > > >> I ran into some issues trying to use the feature
> and
> >> > > > > suggested
> >> > > > > >> > that
> >> > > > > >> > > a
> >> > > > > >> > > > > >> feature likes this needs polish else it'll just
> rot,
> >> > > > unused.
> >> > > > > >> Has
> >> > > > > >> > > > polish
> >> > > > > >> > > > > >> been applied? All ready for another 'user' test?
> >> > Suggest
> >> > > > that
> >> > > > > >> you
> >> > > > > >> > > > update
> >> > > > > >> > > > > >> here going forward for the benefit of those trying
> >> to
> >> > > > follow
> >> > > > > >> along
> >> > > > > >> > > and
> >> > > > > >> > > > > who
> >> > > > > >> > > > > >> are not watching JIRA change fly-by.
> >> > > > > >> > > > > >>
> >> > > > > >> > > > > >> It looks like doc got a revision -- I have to
> check
> >> --
> >> > to
> >> > > > > take
> >> > > > > >> on
> >> > > > > >> > > > > >> suggestion made above but again, suggest, that
> this
> >> > > thread
> >> > > > > gets
> >> > > > > >> > > > updated.
> >> > > > > >> > > > > >>
> >> > > > > >> > > > > >> Thanks,
> >> > > > > >> > > > > >> St.Ack
> >> > > > > >> > > > > >>
> >> > > > > >> > > > > >>
> >> > > > > >> > > > > >>
> >> > > > > >> > > > > >> > Thanks
> >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > On Tue, Sep 20, 2016 at 10:02 AM, Devaraj Das <
> >> > > > > >> > > ddas@hortonworks.com
> >> > > > > >> > > > >
> >> > > > > >> > > > > >> > wrote:
> >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > > Just reviving this thread. Thanks Sean, Stack,
> >> > Dima,
> >> > > > and
> >> > > > > >> > others
> >> > > > > >> > > > for
> >> > > > > >> > > > > the
> >> > > > > >> > > > > >> > > thorough reviews and testing. Thanks Ted and
> >> Vlad
> >> > for
> >> > > > > >> taking
> >> > > > > >> > > care
> >> > > > > >> > > > of
> >> > > > > >> > > > > >> the
> >> > > > > >> > > > > >> > > feedback. Are we all good to do the merge now?
> >> > Rather
> >> > > > do
> >> > > > > >> > sooner
> >> > > > > >> > > > than
> >> > > > > >> > > > > >> > later.
> >> > > > > >> > > > > >> > > ________________________________________
> >> > > > > >> > > > > >> > > From: saint.ack@gmail.com <
> saint.ack@gmail.com>
> >> on
> >> > > > > behalf
> >> > > > > >> of
> >> > > > > >> > > > Stack
> >> > > > > >> > > > > <
> >> > > > > >> > > > > >> > > stack@duboce.net>
> >> > > > > >> > > > > >> > > Sent: Monday, September 12, 2016 1:18 PM
> >> > > > > >> > > > > >> > > To: HBase Dev List
> >> > > > > >> > > > > >> > > Subject: Re: [DISCUSSION] Merge Backup /
> >> Restore -
> >> > > > Branch
> >> > > > > >> > > > HBASE-7912
> >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> > > On Mon, Sep 12, 2016 at 12:19 PM, Ted Yu <
> >> > > > > >> yuzhihong@gmail.com
> >> > > > > >> > >
> >> > > > > >> > > > > wrote:
> >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> > > > Mega patch (rev 18) is on HBASE-14123.
> >> > > > > >> > > > > >> > > >
> >> > > > > >> > > > > >> > > > Please comment on HBASE-14123 on how you
> want
> >> to
> >> > > > > review.
> >> > > > > >> > > > > >> > > >
> >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> > > Yeah. That was my lost tab. Last rb was 6
> months
> >> > ago.
> >> > > > > >> Suggest
> >> > > > > >> > > > > updating
> >> > > > > >> > > > > >> > it.
> >> > > > > >> > > > > >> > > RB is pretty good for review. Patch is only
> >> 1.5M so
> >> > > > > should
> >> > > > > >> be
> >> > > > > >> > > > fine.
> >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> > > St.Ack
> >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> > > >
> >> > > > > >> > > > > >> > > > Thanks
> >> > > > > >> > > > > >> > > >
> >> > > > > >> > > > > >> > > > On Mon, Sep 12, 2016 at 12:15 PM, Stack <
> >> > > > > >> stack@duboce.net>
> >> > > > > >> > > > wrote:
> >> > > > > >> > > > > >> > > >
> >> > > > > >> > > > > >> > > > > On review of the 'patch', do I just
> compare
> >> the
> >> > > > > branch
> >> > > > > >> to
> >> > > > > >> > > > > master or
> >> > > > > >> > > > > >> > is
> >> > > > > >> > > > > >> > > > > there a megapatch posted somewhere (I
> think
> >> I
> >> > saw
> >> > > > one
> >> > > > > >> but
> >> > > > > >> > it
> >> > > > > >> > > > > seemed
> >> > > > > >> > > > > >> > > stale
> >> > > > > >> > > > > >> > > > > and then I 'lost' the tab). Sorry for dumb
> >> > > > question.
> >> > > > > >> > > > > >> > > > > St.Ack
> >> > > > > >> > > > > >> > > > >
> >> > > > > >> > > > > >> > > > > On Mon, Sep 12, 2016 at 12:01 PM, Stack <
> >> > > > > >> stack@duboce.net
> >> > > > > >> > >
> >> > > > > >> > > > > wrote:
> >> > > > > >> > > > > >> > > > >
> >> > > > > >> > > > > >> > > > > > Late to the game. A few comments after
> >> > > rereading
> >> > > > > this
> >> > > > > >> > > thread
> >> > > > > >> > > > > as a
> >> > > > > >> > > > > >> > > > 'user'.
> >> > > > > >> > > > > >> > > > > >
> >> > > > > >> > > > > >> > > > > > + Before merge, a user-facing feature
> like
> >> > this
> >> > > > > >> should
> >> > > > > >> > > work
> >> > > > > >> > > > > (If
> >> > > > > >> > > > > >> > this
> >> > > > > >> > > > > >> > > is
> >> > > > > >> > > > > >> > > > > "higher-bar
> >> > > > > >> > > > > >> > > > > > for new features", bring it on --
> smile).
> >> > > > > >> > > > > >> > > > > > + As a user, I tried the branch with
> tools
> >> > > after
> >> > > > > >> > reviewing
> >> > > > > >> > > > the
> >> > > > > >> > > > > >> > > > > just-posted
> >> > > > > >> > > > > >> > > > > > doc. I had an 'interesting' experience
> >> (left
> >> > > > > >> comments up
> >> > > > > >> > > on
> >> > > > > >> > > > > >> > issue). I
> >> > > > > >> > > > > >> > > > > think
> >> > > > > >> > > > > >> > > > > > the tooling/doc. important to get right.
> >> If
> >> > it
> >> > > > > breaks
> >> > > > > >> > > easily
> >> > > > > >> > > > > or
> >> > > > > >> > > > > >> is
> >> > > > > >> > > > > >> > > > > > inconsistent (or lacks 'polish'),
> >> operators
> >> > > will
> >> > > > > >> judge
> >> > > > > >> > the
> >> > > > > >> > > > > whole
> >> > > > > >> > > > > >> > > > > > backup/restore tooling chain as not
> >> > trustworthy
> >> > > > and
> >> > > > > >> > > abandon
> >> > > > > >> > > > > it.
> >> > > > > >> > > > > >> > Lets
> >> > > > > >> > > > > >> > > > not
> >> > > > > >> > > > > >> > > > > > have this happen to this feature.
> >> > > > > >> > > > > >> > > > > > + Matteo's suggestion (with a helpful
> >> starter
> >> > > > list)
> >> > > > > >> that
> >> > > > > >> > > > there
> >> > > > > >> > > > > >> > needs
> >> > > > > >> > > > > >> > > to
> >> > > > > >> > > > > >> > > > > be
> >> > > > > >> > > > > >> > > > > > explicit qualification on what is
> actually
> >> > > being
> >> > > > > >> > delivered
> >> > > > > >> > > > --
> >> > > > > >> > > > > >> > > > including a
> >> > > > > >> > > > > >> > > > > > listing of limitations (some look
> serious
> >> > such
> >> > > as
> >> > > > > >> data
> >> > > > > >> > > bleed
> >> > > > > >> > > > > from
> >> > > > > >> > > > > >> > > other
> >> > > > > >> > > > > >> > > > > > regions in WALs, but maybe I don't care
> >> for
> >> > my
> >> > > > use
> >> > > > > >> > > case...)
> >> > > > > >> > > > --
> >> > > > > >> > > > > >> > needs
> >> > > > > >> > > > > >> > > to
> >> > > > > >> > > > > >> > > > > > accompany the merge. Lets fold them into
> >> the
> >> > > user
> >> > > > > >> doc.
> >> > > > > >> > in
> >> > > > > >> > > > the
> >> > > > > >> > > > > >> > > technical
> >> > > > > >> > > > > >> > > > > > overview area as suggested so user
> >> > expectations
> >> > > > are
> >> > > > > >> > > properly
> >> > > > > >> > > > > >> > managed
> >> > > > > >> > > > > >> > > > > > (otherwise, they expect the world and
> will
> >> > just
> >> > > > > give
> >> > > > > >> up
> >> > > > > >> > > when
> >> > > > > >> > > > > we
> >> > > > > >> > > > > >> > fall
> >> > > > > >> > > > > >> > > > > > short). Vladimir did a list of what is
> in
> >> > each
> >> > > of
> >> > > > > the
> >> > > > > >> > > phases
> >> > > > > >> > > > > >> above
> >> > > > > >> > > > > >> > > > which
> >> > > > > >> > > > > >> > > > > > would serve as a good start.
> >> > > > > >> > > > > >> > > > > > + Is this feature 'experimental' (Matteo
> >> asks
> >> > > > > above).
> >> > > > > >> > I'd
> >> > > > > >> > > > > prefer
> >> > > > > >> > > > > >> it
> >> > > > > >> > > > > >> > > is
> >> > > > > >> > > > > >> > > > > > not. If it is, it should be labelled all
> >> over
> >> > > > that
> >> > > > > >> it is
> >> > > > > >> > > > so. I
> >> > > > > >> > > > > >> see
> >> > > > > >> > > > > >> > > > > current
> >> > > > > >> > > > > >> > > > > > state called out as a '... technical
> >> preview
> >> > > > > >> feature'.
> >> > > > > >> > > Does
> >> > > > > >> > > > > this
> >> > > > > >> > > > > >> > mean
> >> > > > > >> > > > > >> > > > > > not-for-users?
> >> > > > > >> > > > > >> > > > > >
> >> > > > > >> > > > > >> > > > > > St.Ack
> >> > > > > >> > > > > >> > > > > >
> >> > > > > >> > > > > >> > > > > >
> >> > > > > >> > > > > >> > > > > >
> >> > > > > >> > > > > >> > > > > >
> >> > > > > >> > > > > >> > > > > >
> >> > > > > >> > > > > >> > > > > >
> >> > > > > >> > > > > >> > > > > >
> >> > > > > >> > > > > >> > > > > >
> >> > > > > >> > > > > >> > > > > >
> >> > > > > >> > > > > >> > > > > >
> >> > > > > >> > > > > >> > > > > >
> >> > > > > >> > > > > >> > > > > > On Mon, Sep 12, 2016 at 8:03 AM, Ted Yu
> <
> >> > > > > >> > > > yuzhihong@gmail.com>
> >> > > > > >> > > > > >> > wrote:
> >> > > > > >> > > > > >> > > > > >
> >> > > > > >> > > > > >> > > > > >> Sean:
> >> > > > > >> > > > > >> > > > > >> Do you have more comments ?
> >> > > > > >> > > > > >> > > > > >>
> >> > > > > >> > > > > >> > > > > >> Cheers
> >> > > > > >> > > > > >> > > > > >>
> >> > > > > >> > > > > >> > > > > >> On Fri, Sep 9, 2016 at 1:42 PM,
> Vladimir
> >> > > > Rodionov
> >> > > > > <
> >> > > > > >> > > > > >> > > > > vladrodionov@gmail.com
> >> > > > > >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > > > > >> wrote:
> >> > > > > >> > > > > >> > > > > >>
> >> > > > > >> > > > > >> > > > > >> > Sean,
> >> > > > > >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > > > > >> > Backup/Restore can fail due to
> various
> >> > > > reasons:
> >> > > > > >> > network
> >> > > > > >> > > > > outage
> >> > > > > >> > > > > >> > > > > (cluster
> >> > > > > >> > > > > >> > > > > >> > wide), various time-outs in HBase and
> >> HDFS
> >> > > > > layer,
> >> > > > > >> M/R
> >> > > > > >> > > > > failure
> >> > > > > >> > > > > >> > due
> >> > > > > >> > > > > >> > > to
> >> > > > > >> > > > > >> > > > > >> "HDFS
> >> > > > > >> > > > > >> > > > > >> > exceeded quota", user error (manual
> >> > deletion
> >> > > > of
> >> > > > > >> data)
> >> > > > > >> > > and
> >> > > > > >> > > > > so
> >> > > > > >> > > > > >> on
> >> > > > > >> > > > > >> > so
> >> > > > > >> > > > > >> > > > on.
> >> > > > > >> > > > > >> > > > > >> That
> >> > > > > >> > > > > >> > > > > >> > is impossible to enumerate all
> possible
> >> > > types
> >> > > > of
> >> > > > > >> > > failures
> >> > > > > >> > > > > in a
> >> > > > > >> > > > > >> > > > > >> distributed
> >> > > > > >> > > > > >> > > > > >> > system - that is not our goal/task.
> >> > > > > >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > > > > >> > We focus completely on backup system
> >> table
> >> > > > > >> > consistency
> >> > > > > >> > > > in a
> >> > > > > >> > > > > >> > > presence
> >> > > > > >> > > > > >> > > > > of
> >> > > > > >> > > > > >> > > > > >> any
> >> > > > > >> > > > > >> > > > > >> > type of failure. That is what I call
> >> > > > "tolerance
> >> > > > > to
> >> > > > > >> > > > > failures".
> >> > > > > >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > > > > >> > On a failure:
> >> > > > > >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > > > > >> > BACKUP. All backup system information
> >> > (prior
> >> > > > to
> >> > > > > >> > backup)
> >> > > > > >> > > > > will
> >> > > > > >> > > > > >> be
> >> > > > > >> > > > > >> > > > > restored
> >> > > > > >> > > > > >> > > > > >> > and all temporary data, related to a
> >> > failed
> >> > > > > >> session,
> >> > > > > >> > in
> >> > > > > >> > > > > HDFS
> >> > > > > >> > > > > >> > will
> >> > > > > >> > > > > >> > > be
> >> > > > > >> > > > > >> > > > > >> > deleted
> >> > > > > >> > > > > >> > > > > >> > RESTORE. We do not care about system
> >> data,
> >> > > > > because
> >> > > > > >> > > > restore
> >> > > > > >> > > > > >> does
> >> > > > > >> > > > > >> > > not
> >> > > > > >> > > > > >> > > > > >> change
> >> > > > > >> > > > > >> > > > > >> > it. Temporary data in HDFS will be
> >> cleaned
> >> > > up
> >> > > > > and
> >> > > > > >> > table
> >> > > > > >> > > > > will
> >> > > > > >> > > > > >> be
> >> > > > > >> > > > > >> > > in a
> >> > > > > >> > > > > >> > > > > >> state
> >> > > > > >> > > > > >> > > > > >> > back to where it was before operation
> >> > > started.
> >> > > > > >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > > > > >> > This is what user should expect in
> case
> >> > of a
> >> > > > > >> failure.
> >> > > > > >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > > > > >> > -Vlad
> >> > > > > >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > > > > >> > -Vlad
> >> > > > > >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > > > > >> > On Fri, Sep 9, 2016 at 12:56 PM, Sean
> >> > > Busbey <
> >> > > > > >> > > > > >> busbey@apache.org
> >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> > > > > wrote:
> >> > > > > >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > > > > >> > > Failing in a consistent way, with
> >> docs
> >> > > that
> >> > > > > >> explain
> >> > > > > >> > > the
> >> > > > > >> > > > > >> > various
> >> > > > > >> > > > > >> > > > > >> > > expected failures would be
> >> sufficient.
> >> > > > > >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> > > > > >> > > On Fri, Sep 9, 2016 at 12:16 PM,
> >> > Vladimir
> >> > > > > >> Rodionov
> >> > > > > >> > > > > >> > > > > >> > > <vl...@gmail.com> wrote:
> >> > > > > >> > > > > >> > > > > >> > > > Do not worry Sean, doc is coming
> >> today
> >> > > as
> >> > > > a
> >> > > > > >> > preview
> >> > > > > >> > > > and
> >> > > > > >> > > > > >> our
> >> > > > > >> > > > > >> > > > writer
> >> > > > > >> > > > > >> > > > > >> > Frank
> >> > > > > >> > > > > >> > > > > >> > > > will be working on a putting  it
> >> into
> >> > > > Apache
> >> > > > > >> > repo.
> >> > > > > >> > > > > >> Timeline
> >> > > > > >> > > > > >> > > > > depends
> >> > > > > >> > > > > >> > > > > >> on
> >> > > > > >> > > > > >> > > > > >> > > > Franks schedule but I hope we
> will
> >> get
> >> > > it
> >> > > > > >> rather
> >> > > > > >> > > > sooner
> >> > > > > >> > > > > >> than
> >> > > > > >> > > > > >> > > > > later.
> >> > > > > >> > > > > >> > > > > >> > > >
> >> > > > > >> > > > > >> > > > > >> > > > As for failure testing, we are
> >> > focusing
> >> > > > only
> >> > > > > >> on a
> >> > > > > >> > > > > >> consistent
> >> > > > > >> > > > > >> > > > state
> >> > > > > >> > > > > >> > > > > >> of
> >> > > > > >> > > > > >> > > > > >> > > > backup system data in a presence
> of
> >> > any
> >> > > > type
> >> > > > > >> of
> >> > > > > >> > > > > failures,
> >> > > > > >> > > > > >> We
> >> > > > > >> > > > > >> > > are
> >> > > > > >> > > > > >> > > > > not
> >> > > > > >> > > > > >> > > > > >> > > going
> >> > > > > >> > > > > >> > > > > >> > > > to implement  anything more
> >> "fancy",
> >> > > than
> >> > > > > >> that.
> >> > > > > >> > We
> >> > > > > >> > > > > allow
> >> > > > > >> > > > > >> > both:
> >> > > > > >> > > > > >> > > > > >> backup
> >> > > > > >> > > > > >> > > > > >> > and
> >> > > > > >> > > > > >> > > > > >> > > > restore to fail. What we do not
> >> allow
> >> > is
> >> > > > to
> >> > > > > >> have
> >> > > > > >> > > > system
> >> > > > > >> > > > > >> data
> >> > > > > >> > > > > >> > > > > >> corrupted.
> >> > > > > >> > > > > >> > > > > >> > > > Will it suffice for you? Do you
> >> have
> >> > any
> >> > > > > other
> >> > > > > >> > > > > concerns,
> >> > > > > >> > > > > >> you
> >> > > > > >> > > > > >> > > > want
> >> > > > > >> > > > > >> > > > > >> us to
> >> > > > > >> > > > > >> > > > > >> > > > address?
> >> > > > > >> > > > > >> > > > > >> > > >
> >> > > > > >> > > > > >> > > > > >> > > > -Vlad
> >> > > > > >> > > > > >> > > > > >> > > >
> >> > > > > >> > > > > >> > > > > >> > > >
> >> > > > > >> > > > > >> > > > > >> > > > On Fri, Sep 9, 2016 at 10:56 AM,
> >> Sean
> >> > > > > Busbey <
> >> > > > > >> > > > > >> > > busbey@apache.org
> >> > > > > >> > > > > >> > > > >
> >> > > > > >> > > > > >> > > > > >> > wrote:
> >> > > > > >> > > > > >> > > > > >> > > >
> >> > > > > >> > > > > >> > > > > >> > > >> "docs will come to Apache soon"
> >> does
> >> > > not
> >> > > > > >> address
> >> > > > > >> > > my
> >> > > > > >> > > > > >> concern
> >> > > > > >> > > > > >> > > > > around
> >> > > > > >> > > > > >> > > > > >> > docs
> >> > > > > >> > > > > >> > > > > >> > > at
> >> > > > > >> > > > > >> > > > > >> > > >> all, unless said docs have
> already
> >> > made
> >> > > > it
> >> > > > > >> into
> >> > > > > >> > > the
> >> > > > > >> > > > > >> project
> >> > > > > >> > > > > >> > > > > repo. I
> >> > > > > >> > > > > >> > > > > >> > > don't
> >> > > > > >> > > > > >> > > > > >> > > >> want third party resources for
> >> using
> >> > a
> >> > > > > major
> >> > > > > >> and
> >> > > > > >> > > > > >> important
> >> > > > > >> > > > > >> > > > > feature
> >> > > > > >> > > > > >> > > > > >> of
> >> > > > > >> > > > > >> > > > > >> > > the
> >> > > > > >> > > > > >> > > > > >> > > >> project, I want us to provide
> end
> >> > users
> >> > > > > with
> >> > > > > >> > what
> >> > > > > >> > > > they
> >> > > > > >> > > > > >> need
> >> > > > > >> > > > > >> > > to
> >> > > > > >> > > > > >> > > > > get
> >> > > > > >> > > > > >> > > > > >> the
> >> > > > > >> > > > > >> > > > > >> > > job
> >> > > > > >> > > > > >> > > > > >> > > >> done.
> >> > > > > >> > > > > >> > > > > >> > > >>
> >> > > > > >> > > > > >> > > > > >> > > >> I see some calls for patience on
> >> the
> >> > > > > failure
> >> > > > > >> > > > testing,
> >> > > > > >> > > > > but
> >> > > > > >> > > > > >> > the
> >> > > > > >> > > > > >> > > > > >> appeal
> >> > > > > >> > > > > >> > > > > >> > to
> >> > > > > >> > > > > >> > > > > >> > > us
> >> > > > > >> > > > > >> > > > > >> > > >> having done a bad job of
> requiring
> >> > > proper
> >> > > > > >> tests
> >> > > > > >> > of
> >> > > > > >> > > > > >> previous
> >> > > > > >> > > > > >> > > > > >> features
> >> > > > > >> > > > > >> > > > > >> > > just
> >> > > > > >> > > > > >> > > > > >> > > >> makes me more concerned about
> not
> >> > > getting
> >> > > > > >> them
> >> > > > > >> > > > here. I
> >> > > > > >> > > > > >> > don't
> >> > > > > >> > > > > >> > > > want
> >> > > > > >> > > > > >> > > > > >> to
> >> > > > > >> > > > > >> > > > > >> > set
> >> > > > > >> > > > > >> > > > > >> > > >> yet another bad example that
> will
> >> > then
> >> > > be
> >> > > > > >> > pointed
> >> > > > > >> > > to
> >> > > > > >> > > > > in
> >> > > > > >> > > > > >> the
> >> > > > > >> > > > > >> > > > > future.
> >> > > > > >> > > > > >> > > > > >> > > >>
> >> > > > > >> > > > > >> > > > > >> > > >> On Sep 8, 2016 10:50, "Ted Yu" <
> >> > > > > >> > > yuzhihong@gmail.com
> >> > > > > >> > > > >
> >> > > > > >> > > > > >> > wrote:
> >> > > > > >> > > > > >> > > > > >> > > >>
> >> > > > > >> > > > > >> > > > > >> > > >> > Is there any concern which is
> >> not
> >> > > > > >> addressed ?
> >> > > > > >> > > > > >> > > > > >> > > >> >
> >> > > > > >> > > > > >> > > > > >> > > >> > Do we need another Vote
> thread ?
> >> > > > > >> > > > > >> > > > > >> > > >> >
> >> > > > > >> > > > > >> > > > > >> > > >> > Thanks
> >> > > > > >> > > > > >> > > > > >> > > >> >
> >> > > > > >> > > > > >> > > > > >> > > >> > On Thu, Sep 8, 2016 at 9:21
> AM,
> >> > > Andrew
> >> > > > > >> > Purtell <
> >> > > > > >> > > > > >> > > > > >> apurtell@apache.org
> >> > > > > >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> > > > > >> > > >> > wrote:
> >> > > > > >> > > > > >> > > > > >> > > >> >
> >> > > > > >> > > > > >> > > > > >> > > >> > > Vlad,
> >> > > > > >> > > > > >> > > > > >> > > >> > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > I apologize for using the
> term
> >> > > > > >> 'half-baked'
> >> > > > > >> > > in a
> >> > > > > >> > > > > way
> >> > > > > >> > > > > >> > that
> >> > > > > >> > > > > >> > > > > could
> >> > > > > >> > > > > >> > > > > >> > > seem a
> >> > > > > >> > > > > >> > > > > >> > > >> > > description of HBASE-7912. I
> >> > meant
> >> > > > that
> >> > > > > >> as a
> >> > > > > >> > > > > general
> >> > > > > >> > > > > >> > > > > >> hypothetical.
> >> > > > > >> > > > > >> > > > > >> > > >> > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > On Wed, Sep 7, 2016 at 9:36
> >> AM,
> >> > > > > Vladimir
> >> > > > > >> > > > Rodionov
> >> > > > > >> > > > > <
> >> > > > > >> > > > > >> > > > > >> > > >> > vladrodionov@gmail.com>
> >> > > > > >> > > > > >> > > > > >> > > >> > > wrote:
> >> > > > > >> > > > > >> > > > > >> > > >> > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > >> I'm not sure that
> "There
> >> is
> >> > > > > already
> >> > > > > >> > lots
> >> > > > > >> > > of
> >> > > > > >> > > > > >> > > half-baked
> >> > > > > >> > > > > >> > > > > >> code
> >> > > > > >> > > > > >> > > > > >> > in
> >> > > > > >> > > > > >> > > > > >> > > the
> >> > > > > >> > > > > >> > > > > >> > > >> > > > branch,
> >> > > > > >> > > > > >> > > > > >> > > >> > > > so what's the harm in
> adding
> >> > > more?"
> >> > > > > >> > > > > >> > > > > >> > > >> > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > I meant - not production -
> >> > ready
> >> > > > yet.
> >> > > > > >> This
> >> > > > > >> > > is
> >> > > > > >> > > > > 2.0
> >> > > > > >> > > > > >> > > > > development
> >> > > > > >> > > > > >> > > > > >> > > branch
> >> > > > > >> > > > > >> > > > > >> > > >> > and,
> >> > > > > >> > > > > >> > > > > >> > > >> > > > hence many features are in
> >> > works,
> >> > > > > >> > > > > >> > > > > >> > > >> > > > not being tested well etc.
> >> I do
> >> > > not
> >> > > > > >> > consider
> >> > > > > >> > > > > backup
> >> > > > > >> > > > > >> > as
> >> > > > > >> > > > > >> > > > half
> >> > > > > >> > > > > >> > > > > >> > baked
> >> > > > > >> > > > > >> > > > > >> > > >> > > feature -
> >> > > > > >> > > > > >> > > > > >> > > >> > > > it has passed our internal
> >> QA
> >> > and
> >> > > > has
> >> > > > > >> very
> >> > > > > >> > > > good
> >> > > > > >> > > > > >> doc,
> >> > > > > >> > > > > >> > > > which
> >> > > > > >> > > > > >> > > > > we
> >> > > > > >> > > > > >> > > > > >> > will
> >> > > > > >> > > > > >> > > > > >> > > >> > > provide
> >> > > > > >> > > > > >> > > > > >> > > >> > > > to Apache shortly.
> >> > > > > >> > > > > >> > > > > >> > > >> > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > -Vlad
> >> > > > > >> > > > > >> > > > > >> > > >> > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > On Wed, Sep 7, 2016 at
> 9:13
> >> AM,
> >> > > > > Andrew
> >> > > > > >> > > > Purtell <
> >> > > > > >> > > > > >> > > > > >> > > apurtell@apache.org>
> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
> >> > > > > >> > > > > >> > > > > >> > > >> > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > We shouldn't admit half
> >> baked
> >> > > > > changes
> >> > > > > >> > that
> >> > > > > >> > > > > won't
> >> > > > > >> > > > > >> be
> >> > > > > >> > > > > >> > > > > >> finished.
> >> > > > > >> > > > > >> > > > > >> > > >> However
> >> > > > > >> > > > > >> > > > > >> > > >> > > in
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > this case the crew
> >> working on
> >> > > > this
> >> > > > > >> > feature
> >> > > > > >> > > > are
> >> > > > > >> > > > > >> long
> >> > > > > >> > > > > >> > > > > timers
> >> > > > > >> > > > > >> > > > > >> and
> >> > > > > >> > > > > >> > > > > >> > > less
> >> > > > > >> > > > > >> > > > > >> > > >> > > > likely
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > than just about anyone
> to
> >> > leave
> >> > > > > >> > something
> >> > > > > >> > > > in a
> >> > > > > >> > > > > >> half
> >> > > > > >> > > > > >> > > > baked
> >> > > > > >> > > > > >> > > > > >> > > state. Of
> >> > > > > >> > > > > >> > > > > >> > > >> > > > course
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > there is no guarantee
> how
> >> > > > anything
> >> > > > > >> will
> >> > > > > >> > > turn
> >> > > > > >> > > > > out,
> >> > > > > >> > > > > >> > > but I
> >> > > > > >> > > > > >> > > > > am
> >> > > > > >> > > > > >> > > > > >> > > willing
> >> > > > > >> > > > > >> > > > > >> > > >> to
> >> > > > > >> > > > > >> > > > > >> > > >> > > > take
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > a little on faith if
> they
> >> > feel
> >> > > > > their
> >> > > > > >> > best
> >> > > > > >> > > > path
> >> > > > > >> > > > > >> > > forward
> >> > > > > >> > > > > >> > > > > now
> >> > > > > >> > > > > >> > > > > >> is
> >> > > > > >> > > > > >> > > > > >> > to
> >> > > > > >> > > > > >> > > > > >> > > >> > merge
> >> > > > > >> > > > > >> > > > > >> > > >> > > to
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > trunk. I only wish I had
> >> > > > bandwidth
> >> > > > > to
> >> > > > > >> > have
> >> > > > > >> > > > > done
> >> > > > > >> > > > > >> > some
> >> > > > > >> > > > > >> > > > real
> >> > > > > >> > > > > >> > > > > >> > > kicking
> >> > > > > >> > > > > >> > > > > >> > > >> of
> >> > > > > >> > > > > >> > > > > >> > > >> > > the
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > tires by now. Maybe this
> >> > week.
> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > (Yes, I'm using some of
> >> that
> >> > > time
> >> > > > > for
> >> > > > > >> > this
> >> > > > > >> > > > > email
> >> > > > > >> > > > > >> > :-)
> >> > > > > >> > > > > >> > > > but
> >> > > > > >> > > > > >> > > > > I
> >> > > > > >> > > > > >> > > > > >> > type
> >> > > > > >> > > > > >> > > > > >> > > >> > fast.)
> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > That said, I would like
> to
> >> > > > agitate
> >> > > > > >> for
> >> > > > > >> > > > making
> >> > > > > >> > > > > 2.0
> >> > > > > >> > > > > >> > > more
> >> > > > > >> > > > > >> > > > > real
> >> > > > > >> > > > > >> > > > > >> > and
> >> > > > > >> > > > > >> > > > > >> > > >> spend
> >> > > > > >> > > > > >> > > > > >> > > >> > > > some
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > time on it now that I'm
> >> > winding
> >> > > > > down
> >> > > > > >> > with
> >> > > > > >> > > > > 0.98. I
> >> > > > > >> > > > > >> > > think
> >> > > > > >> > > > > >> > > > > >> that
> >> > > > > >> > > > > >> > > > > >> > > means
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branching for 2.0 real
> >> soon
> >> > now
> >> > > > and
> >> > > > > >> even
> >> > > > > >> > > > > evicting
> >> > > > > >> > > > > >> > > > things
> >> > > > > >> > > > > >> > > > > >> from
> >> > > > > >> > > > > >> > > > > >> > > 2.0
> >> > > > > >> > > > > >> > > > > >> > > >> > > branch
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > that aren't finished or
> >> > stable,
> >> > > > > >> leaving
> >> > > > > >> > > them
> >> > > > > >> > > > > only
> >> > > > > >> > > > > >> > > once
> >> > > > > >> > > > > >> > > > > >> again
> >> > > > > >> > > > > >> > > > > >> > in
> >> > > > > >> > > > > >> > > > > >> > > the
> >> > > > > >> > > > > >> > > > > >> > > >> > > > master
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branch. Or, maybe just
> >> > evicting
> >> > > > > them.
> >> > > > > >> > > Let's
> >> > > > > >> > > > > take
> >> > > > > >> > > > > >> it
> >> > > > > >> > > > > >> > > > case
> >> > > > > >> > > > > >> > > > > by
> >> > > > > >> > > > > >> > > > > >> > > case.
> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > I think this feature can
> >> come
> >> > > in
> >> > > > > >> > > relatively
> >> > > > > >> > > > > >> safely.
> >> > > > > >> > > > > >> > > As
> >> > > > > >> > > > > >> > > > > >> added
> >> > > > > >> > > > > >> > > > > >> > > >> > insurance,
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > let's admit the
> >> possibility
> >> > it
> >> > > > > could
> >> > > > > >> be
> >> > > > > >> > > > > reverted
> >> > > > > >> > > > > >> on
> >> > > > > >> > > > > >> > > the
> >> > > > > >> > > > > >> > > > > 2.0
> >> > > > > >> > > > > >> > > > > >> > > branch
> >> > > > > >> > > > > >> > > > > >> > > >> if
> >> > > > > >> > > > > >> > > > > >> > > >> > > > folks
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > working on stabilizing
> 2.0
> >> > > decide
> >> > > > > to
> >> > > > > >> > evict
> >> > > > > >> > > > it
> >> > > > > >> > > > > >> > because
> >> > > > > >> > > > > >> > > > it
> >> > > > > >> > > > > >> > > > > is
> >> > > > > >> > > > > >> > > > > >> > > >> > unfinished
> >> > > > > >> > > > > >> > > > > >> > > >> > > or
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > unstable, because that
> >> > > certainly
> >> > > > > can
> >> > > > > >> > > > happen. I
> >> > > > > >> > > > > >> > would
> >> > > > > >> > > > > >> > > > > >> expect if
> >> > > > > >> > > > > >> > > > > >> > > talk
> >> > > > > >> > > > > >> > > > > >> > > >> > > like
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > that starts, we'd get
> help
> >> > > > > finishing
> >> > > > > >> or
> >> > > > > >> > > > > >> stabilizing
> >> > > > > >> > > > > >> > > > > what's
> >> > > > > >> > > > > >> > > > > >> > under
> >> > > > > >> > > > > >> > > > > >> > > >> > > > discussion
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > for revert. Or, we'd
> have
> >> a
> >> > > > revert.
> >> > > > > >> > Either
> >> > > > > >> > > > way
> >> > > > > >> > > > > >> the
> >> > > > > >> > > > > >> > > > > outcome
> >> > > > > >> > > > > >> > > > > >> is
> >> > > > > >> > > > > >> > > > > >> > > >> > > acceptable.
> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > On Wed, Sep 7, 2016 at
> >> 8:56
> >> > AM,
> >> > > > > Dima
> >> > > > > >> > > Spivak
> >> > > > > >> > > > <
> >> > > > > >> > > > > >> > > > > >> > > dimaspivak@apache.org
> >> > > > > >> > > > > >> > > > > >> > > >> >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > I'm not sure that
> >> "There is
> >> > > > > already
> >> > > > > >> > lots
> >> > > > > >> > > > of
> >> > > > > >> > > > > >> > > > half-baked
> >> > > > > >> > > > > >> > > > > >> code
> >> > > > > >> > > > > >> > > > > >> > in
> >> > > > > >> > > > > >> > > > > >> > > >> the
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branch,
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > so what's the harm in
> >> > adding
> >> > > > > more?"
> >> > > > > >> > is a
> >> > > > > >> > > > > good
> >> > > > > >> > > > > >> > code
> >> > > > > >> > > > > >> > > > > commit
> >> > > > > >> > > > > >> > > > > >> > > >> > philosophy
> >> > > > > >> > > > > >> > > > > >> > > >> > > > for
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > a
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > fault-tolerant
> >> distributed
> >> > > data
> >> > > > > >> store.
> >> > > > > >> > > ;)
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > More seriously, a lack
> >> of
> >> > > test
> >> > > > > >> > coverage
> >> > > > > >> > > > for
> >> > > > > >> > > > > >> > > existing
> >> > > > > >> > > > > >> > > > > >> > features
> >> > > > > >> > > > > >> > > > > >> > > >> > > shouldn't
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > be
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > used as justification
> >> for
> >> > > > > >> introducing
> >> > > > > >> > > new
> >> > > > > >> > > > > >> > features
> >> > > > > >> > > > > >> > > > with
> >> > > > > >> > > > > >> > > > > >> the
> >> > > > > >> > > > > >> > > > > >> > > same
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > shortcomings.
> >> Ultimately,
> >> > > it's
> >> > > > > the
> >> > > > > >> end
> >> > > > > >> > > > user
> >> > > > > >> > > > > who
> >> > > > > >> > > > > >> > > will
> >> > > > > >> > > > > >> > > > > feel
> >> > > > > >> > > > > >> > > > > >> > the
> >> > > > > >> > > > > >> > > > > >> > > >> pain,
> >> > > > > >> > > > > >> > > > > >> > > >> > > so
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > shouldn't we do
> >> everything
> >> > we
> >> > > > can
> >> > > > > >> to
> >> > > > > >> > > > > mitigate
> >> > > > > >> > > > > >> > that?
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > -Dima
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > On Wed, Sep 7, 2016 at
> >> 8:46
> >> > > AM,
> >> > > > > >> > Vladimir
> >> > > > > >> > > > > >> > Rodionov <
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > vladrodionov@gmail.com>
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > wrote:
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Sean,
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have docs
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Agree. We have a doc
> >> and
> >> > > > backup
> >> > > > > >> is
> >> > > > > >> > the
> >> > > > > >> > > > > most
> >> > > > > >> > > > > >> > > > > documented
> >> > > > > >> > > > > >> > > > > >> > > feature
> >> > > > > >> > > > > >> > > > > >> > > >> > :),
> >> > > > > >> > > > > >> > > > > >> > > >> > > we
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > will
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > release it shortly
> to
> >> > > Apache.
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have sunny-day
> >> > > correctness
> >> > > > > >> tests
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Feature has  close
> to
> >> 60
> >> > > test
> >> > > > > >> cases,
> >> > > > > >> > > > which
> >> > > > > >> > > > > >> run
> >> > > > > >> > > > > >> > > for
> >> > > > > >> > > > > >> > > > > >> approx
> >> > > > > >> > > > > >> > > > > >> > 30
> >> > > > > >> > > > > >> > > > > >> > > >> min.
> >> > > > > >> > > > > >> > > > > >> > > >> > > We
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > can
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > add more, if
> >> community do
> >> > > not
> >> > > > > >> mind
> >> > > > > >> > :)
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have
> >> > > > > >> > correctness-in-face-of-failure
> >> > > > > >> > > > > tests
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Any examples of
> these
> >> > tests
> >> > > > in
> >> > > > > >> > > existing
> >> > > > > >> > > > > >> > features?
> >> > > > > >> > > > > >> > > > In
> >> > > > > >> > > > > >> > > > > >> > works,
> >> > > > > >> > > > > >> > > > > >> > > we
> >> > > > > >> > > > > >> > > > > >> > > >> > > have a
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > clear
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > understanding of
> what
> >> > > should
> >> > > > be
> >> > > > > >> done
> >> > > > > >> > > by
> >> > > > > >> > > > > the
> >> > > > > >> > > > > >> > time
> >> > > > > >> > > > > >> > > of
> >> > > > > >> > > > > >> > > > > 2.0
> >> > > > > >> > > > > >> > > > > >> > > >> release.
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > That is very close
> >> goal
> >> > for
> >> > > > us,
> >> > > > > >> to
> >> > > > > >> > > > verify
> >> > > > > >> > > > > IT
> >> > > > > >> > > > > >> > > monkey
> >> > > > > >> > > > > >> > > > > for
> >> > > > > >> > > > > >> > > > > >> > > >> existing
> >> > > > > >> > > > > >> > > > > >> > > >> > > > code.
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * don't rely on
> things
> >> > > > outside
> >> > > > > of
> >> > > > > >> > > HBase
> >> > > > > >> > > > > for
> >> > > > > >> > > > > >> > > normal
> >> > > > > >> > > > > >> > > > > >> > operation
> >> > > > > >> > > > > >> > > > > >> > > >> > (okay
> >> > > > > >> > > > > >> > > > > >> > > >> > > > for
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > advanced operation)
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > We do not.
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Enormous time has
> been
> >> > > spent
> >> > > > > >> already
> >> > > > > >> > > on
> >> > > > > >> > > > > the
> >> > > > > >> > > > > >> > > > > development
> >> > > > > >> > > > > >> > > > > >> > and
> >> > > > > >> > > > > >> > > > > >> > > >> > testing
> >> > > > > >> > > > > >> > > > > >> > > >> > > > the
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > feature, it has
> passed
> >> > our
> >> > > > > >> internal
> >> > > > > >> > > > tests
> >> > > > > >> > > > > and
> >> > > > > >> > > > > >> > > many
> >> > > > > >> > > > > >> > > > > >> rounds
> >> > > > > >> > > > > >> > > > > >> > of
> >> > > > > >> > > > > >> > > > > >> > > >> code
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > reviews
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > by HBase committers.
> >> We
> >> > do
> >> > > > not
> >> > > > > >> mind
> >> > > > > >> > if
> >> > > > > >> > > > > >> someone
> >> > > > > >> > > > > >> > > from
> >> > > > > >> > > > > >> > > > > >> HBase
> >> > > > > >> > > > > >> > > > > >> > > >> > community
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > (outside of HW) will
> >> > review
> >> > > > the
> >> > > > > >> > code,
> >> > > > > >> > > > but
> >> > > > > >> > > > > it
> >> > > > > >> > > > > >> > will
> >> > > > > >> > > > > >> > > > > >> probably
> >> > > > > >> > > > > >> > > > > >> > > >> takes
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > forever
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > to
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > wait for volunteer?,
> >> the
> >> > > > > feature
> >> > > > > >> is
> >> > > > > >> > > > quite
> >> > > > > >> > > > > >> large
> >> > > > > >> > > > > >> > > > (1MB+
> >> > > > > >> > > > > >> > > > > >> > > >> cumulative
> >> > > > > >> > > > > >> > > > > >> > > >> > > > patch)
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > 2.0 branch is full
> of
> >> > half
> >> > > > > baked
> >> > > > > >> > > > features,
> >> > > > > >> > > > > >> most
> >> > > > > >> > > > > >> > > of
> >> > > > > >> > > > > >> > > > > them
> >> > > > > >> > > > > >> > > > > >> > are
> >> > > > > >> > > > > >> > > > > >> > > in
> >> > > > > >> > > > > >> > > > > >> > > >> > > active
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > development,
> >> therefore I
> >> > am
> >> > > > not
> >> > > > > >> > > > following
> >> > > > > >> > > > > you
> >> > > > > >> > > > > >> > > here,
> >> > > > > >> > > > > >> > > > > >> Sean?
> >> > > > > >> > > > > >> > > > > >> > > Why
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > HBASE-7912
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > is
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > not good enough yet
> >> to be
> >> > > > > >> integrated
> >> > > > > >> > > > into
> >> > > > > >> > > > > 2.0
> >> > > > > >> > > > > >> > > > branch?
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > -Vlad
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > On Wed, Sep 7, 2016
> at
> >> > 8:23
> >> > > > AM,
> >> > > > > >> Sean
> >> > > > > >> > > > > Busbey <
> >> > > > > >> > > > > >> > > > > >> > > busbey@apache.org
> >> > > > > >> > > > > >> > > > > >> > > >> >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > On Tue, Sep 6,
> 2016
> >> at
> >> > > > 10:36
> >> > > > > >> PM,
> >> > > > > >> > > Josh
> >> > > > > >> > > > > >> Elser <
> >> > > > > >> > > > > >> > > > > >> > > >> > > josh.elser@gmail.com>
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > wrote:
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > So, the answer
> to
> >> > > Sean's
> >> > > > > >> > original
> >> > > > > >> > > > > >> question
> >> > > > > >> > > > > >> > is
> >> > > > > >> > > > > >> > > > "as
> >> > > > > >> > > > > >> > > > > >> > > robust as
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > snapshots
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > presently are"?
> >> > > > > >> (independence of
> >> > > > > >> > > > > >> > > backup/restore
> >> > > > > >> > > > > >> > > > > >> > failure
> >> > > > > >> > > > > >> > > > > >> > > >> > > tolerance
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > from
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > snapshot failure
> >> > > > tolerance)
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > Is this just a
> >> > question
> >> > > > WRT
> >> > > > > >> > > context
> >> > > > > >> > > > of
> >> > > > > >> > > > > >> the
> >> > > > > >> > > > > >> > > > > change,
> >> > > > > >> > > > > >> > > > > >> or
> >> > > > > >> > > > > >> > > > > >> > > is it
> >> > > > > >> > > > > >> > > > > >> > > >> > > means
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > for a
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > veto
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > from you, Sean?
> >> Just
> >> > > > trying
> >> > > > > >> to
> >> > > > > >> > > make
> >> > > > > >> > > > > sure
> >> > > > > >> > > > > >> > I'm
> >> > > > > >> > > > > >> > > > > >> following
> >> > > > > >> > > > > >> > > > > >> > > >> along
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > adequately.
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > I'd say ATM I'm
> -0,
> >> > > > bordering
> >> > > > > >> on
> >> > > > > >> > -1
> >> > > > > >> > > > but
> >> > > > > >> > > > > not
> >> > > > > >> > > > > >> > for
> >> > > > > >> > > > > >> > > > > >> reasons
> >> > > > > >> > > > > >> > > > > >> > I
> >> > > > > >> > > > > >> > > > > >> > > can
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > articulate
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > well.
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Here's an attempt.
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > We've been trying
> to
> >> > > move,
> >> > > > > as a
> >> > > > > >> > > > > community,
> >> > > > > >> > > > > >> > > > towards
> >> > > > > >> > > > > >> > > > > >> > > minimizing
> >> > > > > >> > > > > >> > > > > >> > > >> > > risk
> >> > > > > >> > > > > >> > > > > >> > > >> > > > to
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > downstream folks
> by
> >> > > getting
> >> > > > > >> > > "complete
> >> > > > > >> > > > > >> enough
> >> > > > > >> > > > > >> > > for
> >> > > > > >> > > > > >> > > > > use"
> >> > > > > >> > > > > >> > > > > >> > > gates
> >> > > > > >> > > > > >> > > > > >> > > >> in
> >> > > > > >> > > > > >> > > > > >> > > >> > > > place
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > before we
> introduce
> >> new
> >> > > > > >> features.
> >> > > > > >> > > This
> >> > > > > >> > > > > was
> >> > > > > >> > > > > >> > > > spurred
> >> > > > > >> > > > > >> > > > > >> by a
> >> > > > > >> > > > > >> > > > > >> > > some
> >> > > > > >> > > > > >> > > > > >> > > >> > > > features
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > getting in
> >> half-baked
> >> > and
> >> > > > > never
> >> > > > > >> > > making
> >> > > > > >> > > > > it
> >> > > > > >> > > > > >> to
> >> > > > > >> > > > > >> > > "can
> >> > > > > >> > > > > >> > > > > >> really
> >> > > > > >> > > > > >> > > > > >> > > use"
> >> > > > > >> > > > > >> > > > > >> > > >> > > > status
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > (I'm thinking of
> >> > > > distributed
> >> > > > > >> log
> >> > > > > >> > > > replay
> >> > > > > >> > > > > and
> >> > > > > >> > > > > >> > the
> >> > > > > >> > > > > >> > > > > >> zk-less
> >> > > > > >> > > > > >> > > > > >> > > >> > > assignment
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > stuff, I don't
> >> recall
> >> > if
> >> > > > > there
> >> > > > > >> was
> >> > > > > >> > > > > more).
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The gates,
> >> generally,
> >> > > > > included
> >> > > > > >> > > things
> >> > > > > >> > > > > like:
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have docs
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have sunny-day
> >> > > > correctness
> >> > > > > >> tests
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have
> >> > > > > >> > > correctness-in-face-of-failure
> >> > > > > >> > > > > tests
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't rely on
> >> things
> >> > > > > outside
> >> > > > > >> of
> >> > > > > >> > > > HBase
> >> > > > > >> > > > > for
> >> > > > > >> > > > > >> > > > normal
> >> > > > > >> > > > > >> > > > > >> > > operation
> >> > > > > >> > > > > >> > > > > >> > > >> > > (okay
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > for
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > advanced
> operation)
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > As an example, we
> >> kept
> >> > > the
> >> > > > > MOB
> >> > > > > >> > work
> >> > > > > >> > > > off
> >> > > > > >> > > > > in
> >> > > > > >> > > > > >> a
> >> > > > > >> > > > > >> > > > branch
> >> > > > > >> > > > > >> > > > > >> and
> >> > > > > >> > > > > >> > > > > >> > > out
> >> > > > > >> > > > > >> > > > > >> > > >> of
> >> > > > > >> > > > > >> > > > > >> > > >> > > > master
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > until it could
> pass
> >> > these
> >> > > > > >> > criteria.
> >> > > > > >> > > > The
> >> > > > > >> > > > > big
> >> > > > > >> > > > > >> > > > > exemption
> >> > > > > >> > > > > >> > > > > >> > > we've
> >> > > > > >> > > > > >> > > > > >> > > >> had
> >> > > > > >> > > > > >> > > > > >> > > >> > > to
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > this was the
> >> > hbase-spark
> >> > > > > >> > > integration,
> >> > > > > >> > > > > where
> >> > > > > >> > > > > >> > we
> >> > > > > >> > > > > >> > > > all
> >> > > > > >> > > > > >> > > > > >> > agreed
> >> > > > > >> > > > > >> > > > > >> > > it
> >> > > > > >> > > > > >> > > > > >> > > >> > > could
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > land in master
> >> because
> >> > it
> >> > > > was
> >> > > > > >> very
> >> > > > > >> > > > well
> >> > > > > >> > > > > >> > > isolated
> >> > > > > >> > > > > >> > > > > (the
> >> > > > > >> > > > > >> > > > > >> > > slide
> >> > > > > >> > > > > >> > > > > >> > > >> > away
> >> > > > > >> > > > > >> > > > > >> > > >> > > > from
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > including docs as
> a
> >> > > > > first-class
> >> > > > > >> > part
> >> > > > > >> > > > of
> >> > > > > >> > > > > >> > > building
> >> > > > > >> > > > > >> > > > up
> >> > > > > >> > > > > >> > > > > >> that
> >> > > > > >> > > > > >> > > > > >> > > >> > > > integration
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > has led me to
> doubt
> >> the
> >> > > > > wisdom
> >> > > > > >> of
> >> > > > > >> > > this
> >> > > > > >> > > > > >> > > decision).
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > We've also been
> >> > treating
> >> > > > > >> inclusion
> >> > > > > >> > > in
> >> > > > > >> > > > a
> >> > > > > >> > > > > >> > > "probably
> >> > > > > >> > > > > >> > > > > >> will
> >> > > > > >> > > > > >> > > > > >> > be
> >> > > > > >> > > > > >> > > > > >> > > >> > > released
> >> > > > > >> > > > > >> > > > > >> > > >> > > > to
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > downstream"
> branches
> >> > as a
> >> > > > > >> higher
> >> > > > > >> > > bar,
> >> > > > > >> > > > > >> > requiring
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't moderately
> >> > impact
> >> > > > > >> > > performance
> >> > > > > >> > > > > when
> >> > > > > >> > > > > >> > the
> >> > > > > >> > > > > >> > > > > >> feature
> >> > > > > >> > > > > >> > > > > >> > > isn't
> >> > > > > >> > > > > >> > > > > >> > > >> in
> >> > > > > >> > > > > >> > > > > >> > > >> > > use
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't severely
> >> impact
> >> > > > > >> > performance
> >> > > > > >> > > > when
> >> > > > > >> > > > > >> the
> >> > > > > >> > > > > >> > > > > feature
> >> > > > > >> > > > > >> > > > > >> is
> >> > > > > >> > > > > >> > > > > >> > in
> >> > > > > >> > > > > >> > > > > >> > > >> use
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * either
> >> default-to-on
> >> > or
> >> > > > > show
> >> > > > > >> > > enough
> >> > > > > >> > > > > >> demand
> >> > > > > >> > > > > >> > to
> >> > > > > >> > > > > >> > > > > >> believe
> >> > > > > >> > > > > >> > > > > >> > a
> >> > > > > >> > > > > >> > > > > >> > > >> > > > non-trivial
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > number of folks
> will
> >> > turn
> >> > > > the
> >> > > > > >> > > feature
> >> > > > > >> > > > on
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The above has kept
> >> MOB
> >> > > and
> >> > > > > >> > > hbase-spark
> >> > > > > >> > > > > >> > > > integration
> >> > > > > >> > > > > >> > > > > >> out
> >> > > > > >> > > > > >> > > > > >> > of
> >> > > > > >> > > > > >> > > > > >> > > >> > > branch-1,
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > presumably while
> >> > they've
> >> > > > > >> "gotten
> >> > > > > >> > > more
> >> > > > > >> > > > > >> stable"
> >> > > > > >> > > > > >> > > in
> >> > > > > >> > > > > >> > > > > >> master
> >> > > > > >> > > > > >> > > > > >> > > from
> >> > > > > >> > > > > >> > > > > >> > > >> > the
> >> > > > > >> > > > > >> > > > > >> > > >> > > > odd
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > vendor inclusion.
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Are we going to
> >> have a
> >> > > 2.0
> >> > > > > >> release
> >> > > > > >> > > > > before
> >> > > > > >> > > > > >> the
> >> > > > > >> > > > > >> > > end
> >> > > > > >> > > > > >> > > > > of
> >> > > > > >> > > > > >> > > > > >> the
> >> > > > > >> > > > > >> > > > > >> > > >> year?
> >> > > > > >> > > > > >> > > > > >> > > >> > > > We're
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > coming up on 1.5
> >> years
> >> > > > since
> >> > > > > >> the
> >> > > > > >> > > > > release of
> >> > > > > >> > > > > >> > > > version
> >> > > > > >> > > > > >> > > > > >> 1.0;
> >> > > > > >> > > > > >> > > > > >> > > >> seems
> >> > > > > >> > > > > >> > > > > >> > > >> > > like
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > it's about time,
> >> > though I
> >> > > > > >> haven't
> >> > > > > >> > > seen
> >> > > > > >> > > > > any
> >> > > > > >> > > > > >> > > > concrete
> >> > > > > >> > > > > >> > > > > >> > plans
> >> > > > > >> > > > > >> > > > > >> > > >> this
> >> > > > > >> > > > > >> > > > > >> > > >> > > > year.
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Presuming we are
> >> going
> >> > to
> >> > > > > have
> >> > > > > >> one
> >> > > > > >> > > by
> >> > > > > >> > > > > the
> >> > > > > >> > > > > >> end
> >> > > > > >> > > > > >> > > of
> >> > > > > >> > > > > >> > > > > the
> >> > > > > >> > > > > >> > > > > >> > > year, it
> >> > > > > >> > > > > >> > > > > >> > > >> > > > seems a
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > bit close to still
> >> be
> >> > > > adding
> >> > > > > in
> >> > > > > >> > > > > "features
> >> > > > > >> > > > > >> > that
> >> > > > > >> > > > > >> > > > need
> >> > > > > >> > > > > >> > > > > >> > > maturing"
> >> > > > > >> > > > > >> > > > > >> > > >> > on
> >> > > > > >> > > > > >> > > > > >> > > >> > > > the
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > branch.
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The lack of a
> >> concrete
> >> > > plan
> >> > > > > for
> >> > > > > >> > 2.0
> >> > > > > >> > > > > keeps
> >> > > > > >> > > > > >> me
> >> > > > > >> > > > > >> > > from
> >> > > > > >> > > > > >> > > > > >> > > considering
> >> > > > > >> > > > > >> > > > > >> > > >> > > these
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > things blocker at
> >> the
> >> > > > moment.
> >> > > > > >> But
> >> > > > > >> > I
> >> > > > > >> > > > know
> >> > > > > >> > > > > >> > first
> >> > > > > >> > > > > >> > > > hand
> >> > > > > >> > > > > >> > > > > >> how
> >> > > > > >> > > > > >> > > > > >> > > much
> >> > > > > >> > > > > >> > > > > >> > > >> > > > trouble
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > folks have had
> with
> >> > other
> >> > > > > >> features
> >> > > > > >> > > > that
> >> > > > > >> > > > > >> have
> >> > > > > >> > > > > >> > > gone
> >> > > > > >> > > > > >> > > > > >> into
> >> > > > > >> > > > > >> > > > > >> > > >> > downstream
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > facing releases
> >> without
> >> > > > > >> robustness
> >> > > > > >> > > > > checks
> >> > > > > >> > > > > >> > (i.e.
> >> > > > > >> > > > > >> > > > > >> > > replication),
> >> > > > > >> > > > > >> > > > > >> > > >> > and
> >> > > > > >> > > > > >> > > > > >> > > >> > > > I'm
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > concerned about
> what
> >> > > we're
> >> > > > > >> setting
> >> > > > > >> > > up
> >> > > > > >> > > > if
> >> > > > > >> > > > > >> 2.0
> >> > > > > >> > > > > >> > > goes
> >> > > > > >> > > > > >> > > > > out
> >> > > > > >> > > > > >> > > > > >> > with
> >> > > > > >> > > > > >> > > > > >> > > >> this
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > feature in its
> >> current
> >> > > > state.
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > --
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > Best regards,
> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > >    - Andy
> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > Problems worthy of
> attack
> >> > prove
> >> > > > > their
> >> > > > > >> > > worth
> >> > > > > >> > > > by
> >> > > > > >> > > > > >> > > hitting
> >> > > > > >> > > > > >> > > > > >> back. -
> >> > > > > >> > > > > >> > > > > >> > > Piet
> >> > > > > >> > > > > >> > > > > >> > > >> > > Hein
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > (via Tom White)
> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > >
> >> > > > > >> > > > > >> > > > > >> > > >> > >
> >> > > > > >> > > > > >> > > > > >> > > >> > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > --
> >> > > > > >> > > > > >> > > > > >> > > >> > > Best regards,
> >> > > > > >> > > > > >> > > > > >> > > >> > >
> >> > > > > >> > > > > >> > > > > >> > > >> > >    - Andy
> >> > > > > >> > > > > >> > > > > >> > > >> > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > Problems worthy of attack
> >> prove
> >> > > their
> >> > > > > >> worth
> >> > > > > >> > by
> >> > > > > >> > > > > >> hitting
> >> > > > > >> > > > > >> > > > back.
> >> > > > > >> > > > > >> > > > > -
> >> > > > > >> > > > > >> > > > > >> > Piet
> >> > > > > >> > > > > >> > > > > >> > > >> Hein
> >> > > > > >> > > > > >> > > > > >> > > >> > > (via Tom White)
> >> > > > > >> > > > > >> > > > > >> > > >> > >
> >> > > > > >> > > > > >> > > > > >> > > >> >
> >> > > > > >> > > > > >> > > > > >> > > >>
> >> > > > > >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > > > > >>
> >> > > > > >> > > > > >> > > > > >
> >> > > > > >> > > > > >> > > > > >
> >> > > > > >> > > > > >> > > > >
> >> > > > > >> > > > > >> > > >
> >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >>
> >> > > > > >> > > > >
> >> > > > > >> > > > >
> >> > > > > >> > > > >
> >> > > > > >> > > > > --
> >> > > > > >> > > > > busbey
> >> > > > > >> > > > >
> >> > > > > >> > > >
> >> > > > > >> > >
> >> > > > > >> >
> >> > > > > >>
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >>
> >>
> >> --
> >>
> >> -- Appy
> >>
> >
> >
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Devaraj Das <dd...@hortonworks.com>]

Ditto. Worked for me:

$ java -version
java version "1.8.0_102"
Java(TM) SE Runtime Environment (build 1.8.0_102-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.102-b14, mixed mode)


________________________________________
From: Vladimir Rodionov <vl...@gmail.com>
Sent: Wednesday, October 12, 2016 1:10 PM
To: dev@hbase.apache.org
Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

checked out HBASE-7912

ran:

mvn clean install -DskipTests

successfully.

-Vlad

On Wed, Oct 12, 2016 at 1:01 PM, Vladimir Rodionov <vl...@gmail.com>
wrote:

> I usually use:
>
> mvn clean install -DskipTests
>
> and it works.
>
> On Wed, Oct 12, 2016 at 1:01 PM, Vladimir Rodionov <vladrodionov@gmail.com
> > wrote:
>
>> Michael,
>>
>> you can try master + latest patch on HBASE-14123 (v29). No need to use
>> HBASE-7912 branch. I will double check HBASE-7912.
>>
>> -Vlad
>>
>> On Wed, Oct 12, 2016 at 12:31 PM, Stack <st...@duboce.net> wrote:
>>
>>> More info:
>>>
>>> stack@ve0524:~/hbase.git$ git checkout origin/HBASE-7912 -b 7912v2
>>> Branch 7912v2 set up to track remote branch HBASE-7912 from origin.
>>> Switched to a new branch '7912v2'
>>> stack@ve0524:~/hbase.git$ java -version
>>> java version "1.8.0_101"
>>> Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
>>> Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode)
>>> stack@ve0524:~/hbase.git$ mvn clean install -DskipTests &> /tmp/out.txt
>>>
>>> ...
>>>
>>> St.Ack
>>>
>>>
>>> On Wed, Oct 12, 2016 at 12:29 PM, Stack <st...@duboce.net> wrote:
>>>
>>> > Interesting. When I try it fails w/ below:
>>> >
>>> > [INFO] 26 warnings
>>> > 322 [INFO] ------------------------------
>>> -------------------------------
>>> > 323 [INFO] ------------------------------
>>> -------------------------------
>>> > 324 [ERROR] COMPILATION ERROR :
>>> > 325 [INFO] ------------------------------
>>> -------------------------------
>>> > 326 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.java:[48,8]
>>> > org.apache.hadoop.hbase.io.encoding.rowindexV2.RowIndexCodecV2 is not
>>> > abstract and does not override abstract method createSeeker(org.ap
>>> >  ache.hadoop.hbase.CellComparator,org.apache.hadoop.hbase.io.
>>> encoding.HFileBlockDecodingContext)
>>> > in org.apache.hadoop.hbase.io.encoding.DataBlockEncoder
>>> > 327 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.j
>>> ava:[143,3]
>>> > method does not override or implement a method from a supertype
>>> > 328 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.j
>>> ava:[147,29]
>>> > incompatible types: java.nio.ByteBuffer cannot be converted to
>>> > org.apache.hadoop.hbase.nio.ByteBuff
>>> > 329 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.j
>>> ava:[148,33]
>>> > cannot find symbol
>>> > 330   symbol:   method getKeyDeepCopy()
>>> > 331   location: variable seeker of type org.apache.hadoop.hbase.io.
>>> > encoding.DataBlockEncoder.EncodedSeeker
>>> > 332 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.j
>>> ava:[153,3]
>>> > method does not override or implement a method from a supertype
>>> > 333 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> > apache/hadoop/hbase/io/encoding/rowindexV1/RowIndexCodecV1.java:[45,8]
>>> > org.apache.hadoop.hbase.io.encoding.rowindexV1.RowIndexCodecV1 is not
>>> > abstract and does not override abstract method createSeeker(org.ap
>>> >  ache.hadoop.hbase.CellComparator,org.apache.hadoop.hbase.io.
>>> encoding.HFileBlockDecodingContext)
>>> > in org.apache.hadoop.hbase.io.encoding.DataBlockEncoder
>>> > 334 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> > apache/hadoop/hbase/io/encoding/rowindexV1/RowIndexCodecV1.j
>>> ava:[145,3]
>>> > method does not override or implement a method from a supertype
>>> > 335 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> > apache/hadoop/hbase/io/encoding/rowindexV1/RowIndexCodecV1.j
>>> ava:[158,3]
>>> > method does not override or implement a method from a supertype
>>> > 336 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>>> java:[46,8]
>>> > org.apache.hadoop.hbase.io.encoding.rowindexV2.RowIndexSeekerV2 is not
>>> > abstract and does not override abstract method compareKey(org.ap
>>> >  ache.hadoop.hbase.CellComparator,org.apache.hadoop.hbase.Cell) in
>>> > org.apache.hadoop.hbase.io.encoding.DataBlockEncoder.EncodedSeeker
>>> > 337 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>>> java:[79,3]
>>> > method does not override or implement a method from a supertype
>>> > 338 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>>> java:[117,3]
>>> > method does not override or implement a method from a supertype
>>> > 339 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>>> java:[190,3]
>>> > method does not override or implement a method from a supertype
>>> > 340 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>>> java:[214,3]
>>> > method does not override or implement a method from a supertype
>>> > 341 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>>> java:[349,3]
>>> > method does not override or implement a method from a supertype
>>> > 342 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>>> java:[355,3]
>>> > method does not override or implement a method from a supertype
>>> > 343 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>>> java:[421,36]
>>> > no suitable method found for uncompressTags(java.nio.
>>> > ByteBuffer,byte[],int,int)
>>> > 344     method org.apache.hadoop.hbase.io.TagCompressionContext.
>>> > uncompressTags(java.io.InputStream,byte[],int,int) is not applicable
>>> > 345       (argument mismatch; java.nio.ByteBuffer cannot be converted
>>> to
>>> > java.io.InputStream)
>>> > 346     method org.apache.hadoop.hbase.io.TagCompressionContext.
>>> > uncompressTags(org.apache.hadoop.hbase.nio.ByteBuff,byte[],int,int) is
>>> > not applicable
>>> > 347       (argument mismatch; java.nio.ByteBuffer cannot be converted
>>> to
>>> > org.apache.hadoop.hbase.nio.ByteBuff)
>>> >
>>> > ....
>>> >
>>> > St.Ack
>>> >
>>> > On Tue, Oct 11, 2016 at 5:12 PM, Apekshit Sharma <ap...@cloudera.com>
>>> > wrote:
>>> >
>>> >> @stack, it compiled for me.
>>> >>
>>> >> Also tried few commands, and have to say, it's well designed from user
>>> >> commands perspective.
>>> >>
>>> >> On Tue, Oct 11, 2016 at 5:08 PM, Stack <st...@duboce.net> wrote:
>>> >>
>>> >> > On Tue, Oct 11, 2016 at 1:50 PM, Vladimir Rodionov <
>>> >> vladrodionov@gmail.com
>>> >> > >
>>> >> > wrote:
>>> >> >
>>> >> > > Michael,
>>> >> > >
>>> >> > > Its in HBASE-7912
>>> >> > >
>>> >> > > This is tip of git log:
>>> >> > >
>>> >> > > commit a072f6f49a26a7259ff2aaef6cb56d85eb592482
>>> >> > > Author: Frank Welsch <fw...@jps.net>
>>> >> > > Date:   Fri Sep 23 18:00:42 2016 -0400
>>> >> > >
>>> >> > >     HBASE-16574 Book updates for backup and restore
>>> >> > >
>>> >> > > commit b14e2ab1c24e65ff88dd4c579acf83cb4ed0605e
>>> >> > > Author: tedyu <yu...@gmail.com>
>>> >> > > Date:   Wed Oct 5 16:29:40 2016 -0700
>>> >> > >
>>> >> > >     HBASE-16727 Backup refactoring: remove MR dependencies from
>>> >> HMaster
>>> >> > > (Vladimir Rodionov)
>>> >> > >
>>> >> > >
>>> >> > Thanks. I have that. I tried it and it doesn't compile for me. Does
>>> it
>>> >> > compile for you?
>>> >> > Thanks,
>>> >> > M
>>> >> >
>>> >> >
>>> >> >
>>> >> > > -Vlad
>>> >> > >
>>> >> > > On Tue, Oct 11, 2016 at 1:44 PM, Stack <st...@duboce.net> wrote:
>>> >> > >
>>> >> > > > Which branch do I check out to try it? HBASE-7912 is not it. I
>>> don't
>>> >> > see
>>> >> > > an
>>> >> > > > HBASE-16727...
>>> >> > > > Thanks,
>>> >> > > > M
>>> >> > > >
>>> >> > > >
>>> >> > > > On Tue, Oct 11, 2016 at 11:41 AM, Vladimir Rodionov <
>>> >> > > > vladrodionov@gmail.com>
>>> >> > > > wrote:
>>> >> > > >
>>> >> > > > > The last patch is on review board:
>>> >> > > > > https://reviews.apache.org/r/52748
>>> >> > > > >
>>> >> > > > > On Mon, Oct 10, 2016 at 3:52 PM, Vladimir Rodionov <
>>> >> > > > vladrodionov@gmail.com
>>> >> > > > > >
>>> >> > > > > wrote:
>>> >> > > > >
>>> >> > > > > > >> How hard to put in an hbase-backup module? hbase-server
>>> is
>>> >> fat
>>> >> > > > enough
>>> >> > > > > > >> already. Could be done as a follow-up.
>>> >> > > > > >
>>> >> > > > > > https://issues.apache.org/jira/browse/HBASE-16727?
>>> >> > > > > > focusedCommentId=15531237&page=com.atlassian.jira.
>>> >> > > > > > plugin.system.issuetabpanels:c
>>> omment-tabpanel#comment-15531237
>>> >> > > > > >
>>> >> > > > > > Can we do merge first? Then we can discuss separate module.
>>> >> > > > > >
>>> >> > > > > > -Vlad
>>> >> > > > > >
>>> >> > > > > > On Mon, Oct 10, 2016 at 3:44 PM, Ted Yu <
>>> yuzhihong@gmail.com>
>>> >> > wrote:
>>> >> > > > > >
>>> >> > > > > >> Looks like the first quote was cut off.
>>> >> > > > > >> The original sentence was:
>>> >> > > > > >>
>>> >> > > > > >> bq. no mapreduce job launched from master or region server.
>>> >> > > > > >>
>>> >> > > > > >> mapreduce job is launched from the node where command line
>>> >> tool is
>>> >> > > > run.
>>> >> > > > > >>
>>> >> > > > > >> On Mon, Oct 10, 2016 at 3:38 PM, Stack <st...@duboce.net>
>>> >> wrote:
>>> >> > > > > >>
>>> >> > > > > >> > bq. launched from master or region server.
>>> >> > > > > >> >
>>> >> > > > > >> > What does this mean please? Has to be run from Master or
>>> >> > > > RegionServer?
>>> >> > > > > >> Can
>>> >> > > > > >> > it be run from another node altogether?
>>> >> > > > > >> >
>>> >> > > > > >> > On Mon, Oct 10, 2016 at 1:44 PM, Vladimir Rodionov <
>>> >> > > > > >> vladrodionov@gmail.com
>>> >> > > > > >> > >
>>> >> > > > > >> > wrote:
>>> >> > > > > >> >
>>> >> > > > > >> > > >> mapreduce dependency has been moved to client side
>>> - no
>>> >> > > > mapreduce
>>> >> > > > > >> job
>>> >> > > > > >> > >
>>> >> > > > > >> > > 1. We have no code in the client module anymore, due to
>>> >> > > dependency
>>> >> > > > > on
>>> >> > > > > >> > > internal server API (HFile and WAL access).
>>> >> > > > > >> > > 2. Backup/ restore are client - driven operations, but
>>> all
>>> >> the
>>> >> > > > code
>>> >> > > > > >> > resides
>>> >> > > > > >> > > in the server module
>>> >> > > > > >> > >
>>> >> > > > > >> >
>>> >> > > > > >> > How hard to put in an hbase-backup module? hbase-server
>>> is
>>> >> fat
>>> >> > > > enough
>>> >> > > > > >> > already. Could be done as a follow-up.
>>> >> > > > > >> >
>>> >> > > > > >> > Thanks,
>>> >> > > > > >> > St.Ack
>>> >> > > > > >> >
>>> >> > > > > >> >
>>> >> > > > > >> >
>>> >> > > > > >> > > 3. No MR in Master, no procedure - driven execution.
>>> >> > > > > >> > > 4. Old good MR from command-line.
>>> >> > > > > >> > > 5. Security was simplified and now only super-user is
>>> >> allowed
>>> >> > to
>>> >> > > > run
>>> >> > > > > >> > > backup/restores.
>>> >> > > > > >> > > 6. HBase Backup API was gone due to 1. Now only
>>> >> command-line
>>> >> > > > access
>>> >> > > > > to
>>> >> > > > > >> > > backup tools.
>>> >> > > > > >> > >
>>> >> > > > > >> > > These consequences of refactoring has been discussed in
>>> >> > > > HBASE-16727.
>>> >> > > > > >> > >
>>> >> > > > > >> > > -Vlad
>>> >> > > > > >> > >
>>> >> > > > > >> > >
>>> >> > > > > >> > >
>>> >> > > > > >> > >
>>> >> > > > > >> > > On Mon, Oct 10, 2016 at 1:31 PM, Ted Yu <
>>> >> yuzhihong@gmail.com>
>>> >> > > > > wrote:
>>> >> > > > > >> > >
>>> >> > > > > >> > > > Reviving this thread.
>>> >> > > > > >> > > >
>>> >> > > > > >> > > > The following has taken place:
>>> >> > > > > >> > > >
>>> >> > > > > >> > > > mapreduce dependency has been moved to client side -
>>> no
>>> >> > > > mapreduce
>>> >> > > > > >> job
>>> >> > > > > >> > > > launched from master or region server.
>>> >> > > > > >> > > > document patch (HBASE-16574) has been integrated.
>>> >> > > > > >> > > > Updated mega patch has been attached to HBASE-14123:
>>> this
>>> >> > > covers
>>> >> > > > > the
>>> >> > > > > >> > > > refactor in #1 above and the protobuf 3 merge.
>>> >> > > > > >> > > >
>>> >> > > > > >> > > > If community has more feedback on the merge
>>> proposal, I
>>> >> > would
>>> >> > > > love
>>> >> > > > > >> to
>>> >> > > > > >> > > hear
>>> >> > > > > >> > > > it.
>>> >> > > > > >> > > >
>>> >> > > > > >> > > > Thanks
>>> >> > > > > >> > > >
>>> >> > > > > >> > > > On Thu, Sep 22, 2016 at 10:31 AM, Sean Busbey <
>>> >> > > > > busbey@cloudera.com>
>>> >> > > > > >> > > wrote:
>>> >> > > > > >> > > >
>>> >> > > > > >> > > > > I'd like to see the docs proposed on HBASE-16574
>>> >> > integrated
>>> >> > > > into
>>> >> > > > > >> our
>>> >> > > > > >> > > > > project's documentation prior to merge.
>>> >> > > > > >> > > > >
>>> >> > > > > >> > > > > On Thu, Sep 22, 2016 at 9:02 AM, Ted Yu <
>>> >> > > yuzhihong@gmail.com>
>>> >> > > > > >> wrote:
>>> >> > > > > >> > > > > > This feature can be marked experimental due to
>>> some
>>> >> > > > > limitations
>>> >> > > > > >> > such
>>> >> > > > > >> > > as
>>> >> > > > > >> > > > > > security.
>>> >> > > > > >> > > > > >
>>> >> > > > > >> > > > > > Your previous round of comments have been
>>> addressed.
>>> >> > > > > >> > > > > > Command line tool has gone through:
>>> >> > > > > >> > > > > >
>>> >> > > > > >> > > > > > HBASE-16620 Fix backup command-line tool
>>> usability
>>> >> > issues
>>> >> > > > > >> > > > > > HBASE-16655 hbase backup describe with incorrect
>>> >> backup
>>> >> > id
>>> >> > > > > >> results
>>> >> > > > > >> > in
>>> >> > > > > >> > > > NPE
>>> >> > > > > >> > > > > >
>>> >> > > > > >> > > > > > The updated doc has been attached to HBASE-16574.
>>> >> > > > > >> > > > > >
>>> >> > > > > >> > > > > > Cheers
>>> >> > > > > >> > > > > >
>>> >> > > > > >> > > > > > On Thu, Sep 22, 2016 at 8:53 AM, Stack <
>>> >> > stack@duboce.net>
>>> >> > > > > >> wrote:
>>> >> > > > > >> > > > > >
>>> >> > > > > >> > > > > >> On Wed, Sep 21, 2016 at 7:43 AM, Ted Yu <
>>> >> > > > yuzhihong@gmail.com
>>> >> > > > > >
>>> >> > > > > >> > > wrote:
>>> >> > > > > >> > > > > >>
>>> >> > > > > >> > > > > >> > Are there more (review) comments ?
>>> >> > > > > >> > > > > >> >
>>> >> > > > > >> > > > > >> >
>>> >> > > > > >> > > > > >> Are outstanding comments addressed?
>>> >> > > > > >> > > > > >>
>>> >> > > > > >> > > > > >> I don't see answer to my 'is this
>>> experimental/will
>>> >> it
>>> >> > be
>>> >> > > > > >> marked
>>> >> > > > > >> > > > > >> experimental' question.
>>> >> > > > > >> > > > > >>
>>> >> > > > > >> > > > > >> I ran into some issues trying to use the
>>> feature and
>>> >> > > > > suggested
>>> >> > > > > >> > that
>>> >> > > > > >> > > a
>>> >> > > > > >> > > > > >> feature likes this needs polish else it'll just
>>> rot,
>>> >> > > > unused.
>>> >> > > > > >> Has
>>> >> > > > > >> > > > polish
>>> >> > > > > >> > > > > >> been applied? All ready for another 'user' test?
>>> >> > Suggest
>>> >> > > > that
>>> >> > > > > >> you
>>> >> > > > > >> > > > update
>>> >> > > > > >> > > > > >> here going forward for the benefit of those
>>> trying
>>> >> to
>>> >> > > > follow
>>> >> > > > > >> along
>>> >> > > > > >> > > and
>>> >> > > > > >> > > > > who
>>> >> > > > > >> > > > > >> are not watching JIRA change fly-by.
>>> >> > > > > >> > > > > >>
>>> >> > > > > >> > > > > >> It looks like doc got a revision -- I have to
>>> check
>>> >> --
>>> >> > to
>>> >> > > > > take
>>> >> > > > > >> on
>>> >> > > > > >> > > > > >> suggestion made above but again, suggest, that
>>> this
>>> >> > > thread
>>> >> > > > > gets
>>> >> > > > > >> > > > updated.
>>> >> > > > > >> > > > > >>
>>> >> > > > > >> > > > > >> Thanks,
>>> >> > > > > >> > > > > >> St.Ack
>>> >> > > > > >> > > > > >>
>>> >> > > > > >> > > > > >>
>>> >> > > > > >> > > > > >>
>>> >> > > > > >> > > > > >> > Thanks
>>> >> > > > > >> > > > > >> >
>>> >> > > > > >> > > > > >> > On Tue, Sep 20, 2016 at 10:02 AM, Devaraj Das
>>> <
>>> >> > > > > >> > > ddas@hortonworks.com
>>> >> > > > > >> > > > >
>>> >> > > > > >> > > > > >> > wrote:
>>> >> > > > > >> > > > > >> >
>>> >> > > > > >> > > > > >> > > Just reviving this thread. Thanks Sean,
>>> Stack,
>>> >> > Dima,
>>> >> > > > and
>>> >> > > > > >> > others
>>> >> > > > > >> > > > for
>>> >> > > > > >> > > > > the
>>> >> > > > > >> > > > > >> > > thorough reviews and testing. Thanks Ted and
>>> >> Vlad
>>> >> > for
>>> >> > > > > >> taking
>>> >> > > > > >> > > care
>>> >> > > > > >> > > > of
>>> >> > > > > >> > > > > >> the
>>> >> > > > > >> > > > > >> > > feedback. Are we all good to do the merge
>>> now?
>>> >> > Rather
>>> >> > > > do
>>> >> > > > > >> > sooner
>>> >> > > > > >> > > > than
>>> >> > > > > >> > > > > >> > later.
>>> >> > > > > >> > > > > >> > > ________________________________________
>>> >> > > > > >> > > > > >> > > From: saint.ack@gmail.com <
>>> saint.ack@gmail.com>
>>> >> on
>>> >> > > > > behalf
>>> >> > > > > >> of
>>> >> > > > > >> > > > Stack
>>> >> > > > > >> > > > > <
>>> >> > > > > >> > > > > >> > > stack@duboce.net>
>>> >> > > > > >> > > > > >> > > Sent: Monday, September 12, 2016 1:18 PM
>>> >> > > > > >> > > > > >> > > To: HBase Dev List
>>> >> > > > > >> > > > > >> > > Subject: Re: [DISCUSSION] Merge Backup /
>>> >> Restore -
>>> >> > > > Branch
>>> >> > > > > >> > > > HBASE-7912
>>> >> > > > > >> > > > > >> > >
>>> >> > > > > >> > > > > >> > > On Mon, Sep 12, 2016 at 12:19 PM, Ted Yu <
>>> >> > > > > >> yuzhihong@gmail.com
>>> >> > > > > >> > >
>>> >> > > > > >> > > > > wrote:
>>> >> > > > > >> > > > > >> > >
>>> >> > > > > >> > > > > >> > > > Mega patch (rev 18) is on HBASE-14123.
>>> >> > > > > >> > > > > >> > > >
>>> >> > > > > >> > > > > >> > > > Please comment on HBASE-14123 on how you
>>> want
>>> >> to
>>> >> > > > > review.
>>> >> > > > > >> > > > > >> > > >
>>> >> > > > > >> > > > > >> > >
>>> >> > > > > >> > > > > >> > >
>>> >> > > > > >> > > > > >> > > Yeah. That was my lost tab. Last rb was 6
>>> months
>>> >> > ago.
>>> >> > > > > >> Suggest
>>> >> > > > > >> > > > > updating
>>> >> > > > > >> > > > > >> > it.
>>> >> > > > > >> > > > > >> > > RB is pretty good for review. Patch is only
>>> >> 1.5M so
>>> >> > > > > should
>>> >> > > > > >> be
>>> >> > > > > >> > > > fine.
>>> >> > > > > >> > > > > >> > >
>>> >> > > > > >> > > > > >> > > St.Ack
>>> >> > > > > >> > > > > >> > >
>>> >> > > > > >> > > > > >> > >
>>> >> > > > > >> > > > > >> > > >
>>> >> > > > > >> > > > > >> > > > Thanks
>>> >> > > > > >> > > > > >> > > >
>>> >> > > > > >> > > > > >> > > > On Mon, Sep 12, 2016 at 12:15 PM, Stack <
>>> >> > > > > >> stack@duboce.net>
>>> >> > > > > >> > > > wrote:
>>> >> > > > > >> > > > > >> > > >
>>> >> > > > > >> > > > > >> > > > > On review of the 'patch', do I just
>>> compare
>>> >> the
>>> >> > > > > branch
>>> >> > > > > >> to
>>> >> > > > > >> > > > > master or
>>> >> > > > > >> > > > > >> > is
>>> >> > > > > >> > > > > >> > > > > there a megapatch posted somewhere (I
>>> think
>>> >> I
>>> >> > saw
>>> >> > > > one
>>> >> > > > > >> but
>>> >> > > > > >> > it
>>> >> > > > > >> > > > > seemed
>>> >> > > > > >> > > > > >> > > stale
>>> >> > > > > >> > > > > >> > > > > and then I 'lost' the tab). Sorry for
>>> dumb
>>> >> > > > question.
>>> >> > > > > >> > > > > >> > > > > St.Ack
>>> >> > > > > >> > > > > >> > > > >
>>> >> > > > > >> > > > > >> > > > > On Mon, Sep 12, 2016 at 12:01 PM, Stack
>>> <
>>> >> > > > > >> stack@duboce.net
>>> >> > > > > >> > >
>>> >> > > > > >> > > > > wrote:
>>> >> > > > > >> > > > > >> > > > >
>>> >> > > > > >> > > > > >> > > > > > Late to the game. A few comments after
>>> >> > > rereading
>>> >> > > > > this
>>> >> > > > > >> > > thread
>>> >> > > > > >> > > > > as a
>>> >> > > > > >> > > > > >> > > > 'user'.
>>> >> > > > > >> > > > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > > > + Before merge, a user-facing feature
>>> like
>>> >> > this
>>> >> > > > > >> should
>>> >> > > > > >> > > work
>>> >> > > > > >> > > > > (If
>>> >> > > > > >> > > > > >> > this
>>> >> > > > > >> > > > > >> > > is
>>> >> > > > > >> > > > > >> > > > > "higher-bar
>>> >> > > > > >> > > > > >> > > > > > for new features", bring it on --
>>> smile).
>>> >> > > > > >> > > > > >> > > > > > + As a user, I tried the branch with
>>> tools
>>> >> > > after
>>> >> > > > > >> > reviewing
>>> >> > > > > >> > > > the
>>> >> > > > > >> > > > > >> > > > > just-posted
>>> >> > > > > >> > > > > >> > > > > > doc. I had an 'interesting' experience
>>> >> (left
>>> >> > > > > >> comments up
>>> >> > > > > >> > > on
>>> >> > > > > >> > > > > >> > issue). I
>>> >> > > > > >> > > > > >> > > > > think
>>> >> > > > > >> > > > > >> > > > > > the tooling/doc. important to get
>>> right.
>>> >> If
>>> >> > it
>>> >> > > > > breaks
>>> >> > > > > >> > > easily
>>> >> > > > > >> > > > > or
>>> >> > > > > >> > > > > >> is
>>> >> > > > > >> > > > > >> > > > > > inconsistent (or lacks 'polish'),
>>> >> operators
>>> >> > > will
>>> >> > > > > >> judge
>>> >> > > > > >> > the
>>> >> > > > > >> > > > > whole
>>> >> > > > > >> > > > > >> > > > > > backup/restore tooling chain as not
>>> >> > trustworthy
>>> >> > > > and
>>> >> > > > > >> > > abandon
>>> >> > > > > >> > > > > it.
>>> >> > > > > >> > > > > >> > Lets
>>> >> > > > > >> > > > > >> > > > not
>>> >> > > > > >> > > > > >> > > > > > have this happen to this feature.
>>> >> > > > > >> > > > > >> > > > > > + Matteo's suggestion (with a helpful
>>> >> starter
>>> >> > > > list)
>>> >> > > > > >> that
>>> >> > > > > >> > > > there
>>> >> > > > > >> > > > > >> > needs
>>> >> > > > > >> > > > > >> > > to
>>> >> > > > > >> > > > > >> > > > > be
>>> >> > > > > >> > > > > >> > > > > > explicit qualification on what is
>>> actually
>>> >> > > being
>>> >> > > > > >> > delivered
>>> >> > > > > >> > > > --
>>> >> > > > > >> > > > > >> > > > including a
>>> >> > > > > >> > > > > >> > > > > > listing of limitations (some look
>>> serious
>>> >> > such
>>> >> > > as
>>> >> > > > > >> data
>>> >> > > > > >> > > bleed
>>> >> > > > > >> > > > > from
>>> >> > > > > >> > > > > >> > > other
>>> >> > > > > >> > > > > >> > > > > > regions in WALs, but maybe I don't
>>> care
>>> >> for
>>> >> > my
>>> >> > > > use
>>> >> > > > > >> > > case...)
>>> >> > > > > >> > > > --
>>> >> > > > > >> > > > > >> > needs
>>> >> > > > > >> > > > > >> > > to
>>> >> > > > > >> > > > > >> > > > > > accompany the merge. Lets fold them
>>> into
>>> >> the
>>> >> > > user
>>> >> > > > > >> doc.
>>> >> > > > > >> > in
>>> >> > > > > >> > > > the
>>> >> > > > > >> > > > > >> > > technical
>>> >> > > > > >> > > > > >> > > > > > overview area as suggested so user
>>> >> > expectations
>>> >> > > > are
>>> >> > > > > >> > > properly
>>> >> > > > > >> > > > > >> > managed
>>> >> > > > > >> > > > > >> > > > > > (otherwise, they expect the world and
>>> will
>>> >> > just
>>> >> > > > > give
>>> >> > > > > >> up
>>> >> > > > > >> > > when
>>> >> > > > > >> > > > > we
>>> >> > > > > >> > > > > >> > fall
>>> >> > > > > >> > > > > >> > > > > > short). Vladimir did a list of what
>>> is in
>>> >> > each
>>> >> > > of
>>> >> > > > > the
>>> >> > > > > >> > > phases
>>> >> > > > > >> > > > > >> above
>>> >> > > > > >> > > > > >> > > > which
>>> >> > > > > >> > > > > >> > > > > > would serve as a good start.
>>> >> > > > > >> > > > > >> > > > > > + Is this feature 'experimental'
>>> (Matteo
>>> >> asks
>>> >> > > > > above).
>>> >> > > > > >> > I'd
>>> >> > > > > >> > > > > prefer
>>> >> > > > > >> > > > > >> it
>>> >> > > > > >> > > > > >> > > is
>>> >> > > > > >> > > > > >> > > > > > not. If it is, it should be labelled
>>> all
>>> >> over
>>> >> > > > that
>>> >> > > > > >> it is
>>> >> > > > > >> > > > so. I
>>> >> > > > > >> > > > > >> see
>>> >> > > > > >> > > > > >> > > > > current
>>> >> > > > > >> > > > > >> > > > > > state called out as a '... technical
>>> >> preview
>>> >> > > > > >> feature'.
>>> >> > > > > >> > > Does
>>> >> > > > > >> > > > > this
>>> >> > > > > >> > > > > >> > mean
>>> >> > > > > >> > > > > >> > > > > > not-for-users?
>>> >> > > > > >> > > > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > > > St.Ack
>>> >> > > > > >> > > > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > > > On Mon, Sep 12, 2016 at 8:03 AM, Ted
>>> Yu <
>>> >> > > > > >> > > > yuzhihong@gmail.com>
>>> >> > > > > >> > > > > >> > wrote:
>>> >> > > > > >> > > > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > > >> Sean:
>>> >> > > > > >> > > > > >> > > > > >> Do you have more comments ?
>>> >> > > > > >> > > > > >> > > > > >>
>>> >> > > > > >> > > > > >> > > > > >> Cheers
>>> >> > > > > >> > > > > >> > > > > >>
>>> >> > > > > >> > > > > >> > > > > >> On Fri, Sep 9, 2016 at 1:42 PM,
>>> Vladimir
>>> >> > > > Rodionov
>>> >> > > > > <
>>> >> > > > > >> > > > > >> > > > > vladrodionov@gmail.com
>>> >> > > > > >> > > > > >> > > > > >> >
>>> >> > > > > >> > > > > >> > > > > >> wrote:
>>> >> > > > > >> > > > > >> > > > > >>
>>> >> > > > > >> > > > > >> > > > > >> > Sean,
>>> >> > > > > >> > > > > >> > > > > >> >
>>> >> > > > > >> > > > > >> > > > > >> > Backup/Restore can fail due to
>>> various
>>> >> > > > reasons:
>>> >> > > > > >> > network
>>> >> > > > > >> > > > > outage
>>> >> > > > > >> > > > > >> > > > > (cluster
>>> >> > > > > >> > > > > >> > > > > >> > wide), various time-outs in HBase
>>> and
>>> >> HDFS
>>> >> > > > > layer,
>>> >> > > > > >> M/R
>>> >> > > > > >> > > > > failure
>>> >> > > > > >> > > > > >> > due
>>> >> > > > > >> > > > > >> > > to
>>> >> > > > > >> > > > > >> > > > > >> "HDFS
>>> >> > > > > >> > > > > >> > > > > >> > exceeded quota", user error (manual
>>> >> > deletion
>>> >> > > > of
>>> >> > > > > >> data)
>>> >> > > > > >> > > and
>>> >> > > > > >> > > > > so
>>> >> > > > > >> > > > > >> on
>>> >> > > > > >> > > > > >> > so
>>> >> > > > > >> > > > > >> > > > on.
>>> >> > > > > >> > > > > >> > > > > >> That
>>> >> > > > > >> > > > > >> > > > > >> > is impossible to enumerate all
>>> possible
>>> >> > > types
>>> >> > > > of
>>> >> > > > > >> > > failures
>>> >> > > > > >> > > > > in a
>>> >> > > > > >> > > > > >> > > > > >> distributed
>>> >> > > > > >> > > > > >> > > > > >> > system - that is not our goal/task.
>>> >> > > > > >> > > > > >> > > > > >> >
>>> >> > > > > >> > > > > >> > > > > >> > We focus completely on backup
>>> system
>>> >> table
>>> >> > > > > >> > consistency
>>> >> > > > > >> > > > in a
>>> >> > > > > >> > > > > >> > > presence
>>> >> > > > > >> > > > > >> > > > > of
>>> >> > > > > >> > > > > >> > > > > >> any
>>> >> > > > > >> > > > > >> > > > > >> > type of failure. That is what I
>>> call
>>> >> > > > "tolerance
>>> >> > > > > to
>>> >> > > > > >> > > > > failures".
>>> >> > > > > >> > > > > >> > > > > >> >
>>> >> > > > > >> > > > > >> > > > > >> > On a failure:
>>> >> > > > > >> > > > > >> > > > > >> >
>>> >> > > > > >> > > > > >> > > > > >> > BACKUP. All backup system
>>> information
>>> >> > (prior
>>> >> > > > to
>>> >> > > > > >> > backup)
>>> >> > > > > >> > > > > will
>>> >> > > > > >> > > > > >> be
>>> >> > > > > >> > > > > >> > > > > restored
>>> >> > > > > >> > > > > >> > > > > >> > and all temporary data, related to
>>> a
>>> >> > failed
>>> >> > > > > >> session,
>>> >> > > > > >> > in
>>> >> > > > > >> > > > > HDFS
>>> >> > > > > >> > > > > >> > will
>>> >> > > > > >> > > > > >> > > be
>>> >> > > > > >> > > > > >> > > > > >> > deleted
>>> >> > > > > >> > > > > >> > > > > >> > RESTORE. We do not care about
>>> system
>>> >> data,
>>> >> > > > > because
>>> >> > > > > >> > > > restore
>>> >> > > > > >> > > > > >> does
>>> >> > > > > >> > > > > >> > > not
>>> >> > > > > >> > > > > >> > > > > >> change
>>> >> > > > > >> > > > > >> > > > > >> > it. Temporary data in HDFS will be
>>> >> cleaned
>>> >> > > up
>>> >> > > > > and
>>> >> > > > > >> > table
>>> >> > > > > >> > > > > will
>>> >> > > > > >> > > > > >> be
>>> >> > > > > >> > > > > >> > > in a
>>> >> > > > > >> > > > > >> > > > > >> state
>>> >> > > > > >> > > > > >> > > > > >> > back to where it was before
>>> operation
>>> >> > > started.
>>> >> > > > > >> > > > > >> > > > > >> >
>>> >> > > > > >> > > > > >> > > > > >> > This is what user should expect in
>>> case
>>> >> > of a
>>> >> > > > > >> failure.
>>> >> > > > > >> > > > > >> > > > > >> >
>>> >> > > > > >> > > > > >> > > > > >> > -Vlad
>>> >> > > > > >> > > > > >> > > > > >> >
>>> >> > > > > >> > > > > >> > > > > >> >
>>> >> > > > > >> > > > > >> > > > > >> > -Vlad
>>> >> > > > > >> > > > > >> > > > > >> >
>>> >> > > > > >> > > > > >> > > > > >> > On Fri, Sep 9, 2016 at 12:56 PM,
>>> Sean
>>> >> > > Busbey <
>>> >> > > > > >> > > > > >> busbey@apache.org
>>> >> > > > > >> > > > > >> > >
>>> >> > > > > >> > > > > >> > > > > wrote:
>>> >> > > > > >> > > > > >> > > > > >> >
>>> >> > > > > >> > > > > >> > > > > >> > > Failing in a consistent way, with
>>> >> docs
>>> >> > > that
>>> >> > > > > >> explain
>>> >> > > > > >> > > the
>>> >> > > > > >> > > > > >> > various
>>> >> > > > > >> > > > > >> > > > > >> > > expected failures would be
>>> >> sufficient.
>>> >> > > > > >> > > > > >> > > > > >> > >
>>> >> > > > > >> > > > > >> > > > > >> > > On Fri, Sep 9, 2016 at 12:16 PM,
>>> >> > Vladimir
>>> >> > > > > >> Rodionov
>>> >> > > > > >> > > > > >> > > > > >> > > <vl...@gmail.com> wrote:
>>> >> > > > > >> > > > > >> > > > > >> > > > Do not worry Sean, doc is
>>> coming
>>> >> today
>>> >> > > as
>>> >> > > > a
>>> >> > > > > >> > preview
>>> >> > > > > >> > > > and
>>> >> > > > > >> > > > > >> our
>>> >> > > > > >> > > > > >> > > > writer
>>> >> > > > > >> > > > > >> > > > > >> > Frank
>>> >> > > > > >> > > > > >> > > > > >> > > > will be working on a putting
>>> it
>>> >> into
>>> >> > > > Apache
>>> >> > > > > >> > repo.
>>> >> > > > > >> > > > > >> Timeline
>>> >> > > > > >> > > > > >> > > > > depends
>>> >> > > > > >> > > > > >> > > > > >> on
>>> >> > > > > >> > > > > >> > > > > >> > > > Franks schedule but I hope we
>>> will
>>> >> get
>>> >> > > it
>>> >> > > > > >> rather
>>> >> > > > > >> > > > sooner
>>> >> > > > > >> > > > > >> than
>>> >> > > > > >> > > > > >> > > > > later.
>>> >> > > > > >> > > > > >> > > > > >> > > >
>>> >> > > > > >> > > > > >> > > > > >> > > > As for failure testing, we are
>>> >> > focusing
>>> >> > > > only
>>> >> > > > > >> on a
>>> >> > > > > >> > > > > >> consistent
>>> >> > > > > >> > > > > >> > > > state
>>> >> > > > > >> > > > > >> > > > > >> of
>>> >> > > > > >> > > > > >> > > > > >> > > > backup system data in a
>>> presence of
>>> >> > any
>>> >> > > > type
>>> >> > > > > >> of
>>> >> > > > > >> > > > > failures,
>>> >> > > > > >> > > > > >> We
>>> >> > > > > >> > > > > >> > > are
>>> >> > > > > >> > > > > >> > > > > not
>>> >> > > > > >> > > > > >> > > > > >> > > going
>>> >> > > > > >> > > > > >> > > > > >> > > > to implement  anything more
>>> >> "fancy",
>>> >> > > than
>>> >> > > > > >> that.
>>> >> > > > > >> > We
>>> >> > > > > >> > > > > allow
>>> >> > > > > >> > > > > >> > both:
>>> >> > > > > >> > > > > >> > > > > >> backup
>>> >> > > > > >> > > > > >> > > > > >> > and
>>> >> > > > > >> > > > > >> > > > > >> > > > restore to fail. What we do not
>>> >> allow
>>> >> > is
>>> >> > > > to
>>> >> > > > > >> have
>>> >> > > > > >> > > > system
>>> >> > > > > >> > > > > >> data
>>> >> > > > > >> > > > > >> > > > > >> corrupted.
>>> >> > > > > >> > > > > >> > > > > >> > > > Will it suffice for you? Do you
>>> >> have
>>> >> > any
>>> >> > > > > other
>>> >> > > > > >> > > > > concerns,
>>> >> > > > > >> > > > > >> you
>>> >> > > > > >> > > > > >> > > > want
>>> >> > > > > >> > > > > >> > > > > >> us to
>>> >> > > > > >> > > > > >> > > > > >> > > > address?
>>> >> > > > > >> > > > > >> > > > > >> > > >
>>> >> > > > > >> > > > > >> > > > > >> > > > -Vlad
>>> >> > > > > >> > > > > >> > > > > >> > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >
>>> >> > > > > >> > > > > >> > > > > >> > > > On Fri, Sep 9, 2016 at 10:56
>>> AM,
>>> >> Sean
>>> >> > > > > Busbey <
>>> >> > > > > >> > > > > >> > > busbey@apache.org
>>> >> > > > > >> > > > > >> > > > >
>>> >> > > > > >> > > > > >> > > > > >> > wrote:
>>> >> > > > > >> > > > > >> > > > > >> > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> "docs will come to Apache
>>> soon"
>>> >> does
>>> >> > > not
>>> >> > > > > >> address
>>> >> > > > > >> > > my
>>> >> > > > > >> > > > > >> concern
>>> >> > > > > >> > > > > >> > > > > around
>>> >> > > > > >> > > > > >> > > > > >> > docs
>>> >> > > > > >> > > > > >> > > > > >> > > at
>>> >> > > > > >> > > > > >> > > > > >> > > >> all, unless said docs have
>>> already
>>> >> > made
>>> >> > > > it
>>> >> > > > > >> into
>>> >> > > > > >> > > the
>>> >> > > > > >> > > > > >> project
>>> >> > > > > >> > > > > >> > > > > repo. I
>>> >> > > > > >> > > > > >> > > > > >> > > don't
>>> >> > > > > >> > > > > >> > > > > >> > > >> want third party resources for
>>> >> using
>>> >> > a
>>> >> > > > > major
>>> >> > > > > >> and
>>> >> > > > > >> > > > > >> important
>>> >> > > > > >> > > > > >> > > > > feature
>>> >> > > > > >> > > > > >> > > > > >> of
>>> >> > > > > >> > > > > >> > > > > >> > > the
>>> >> > > > > >> > > > > >> > > > > >> > > >> project, I want us to provide
>>> end
>>> >> > users
>>> >> > > > > with
>>> >> > > > > >> > what
>>> >> > > > > >> > > > they
>>> >> > > > > >> > > > > >> need
>>> >> > > > > >> > > > > >> > > to
>>> >> > > > > >> > > > > >> > > > > get
>>> >> > > > > >> > > > > >> > > > > >> the
>>> >> > > > > >> > > > > >> > > > > >> > > job
>>> >> > > > > >> > > > > >> > > > > >> > > >> done.
>>> >> > > > > >> > > > > >> > > > > >> > > >>
>>> >> > > > > >> > > > > >> > > > > >> > > >> I see some calls for patience
>>> on
>>> >> the
>>> >> > > > > failure
>>> >> > > > > >> > > > testing,
>>> >> > > > > >> > > > > but
>>> >> > > > > >> > > > > >> > the
>>> >> > > > > >> > > > > >> > > > > >> appeal
>>> >> > > > > >> > > > > >> > > > > >> > to
>>> >> > > > > >> > > > > >> > > > > >> > > us
>>> >> > > > > >> > > > > >> > > > > >> > > >> having done a bad job of
>>> requiring
>>> >> > > proper
>>> >> > > > > >> tests
>>> >> > > > > >> > of
>>> >> > > > > >> > > > > >> previous
>>> >> > > > > >> > > > > >> > > > > >> features
>>> >> > > > > >> > > > > >> > > > > >> > > just
>>> >> > > > > >> > > > > >> > > > > >> > > >> makes me more concerned about
>>> not
>>> >> > > getting
>>> >> > > > > >> them
>>> >> > > > > >> > > > here. I
>>> >> > > > > >> > > > > >> > don't
>>> >> > > > > >> > > > > >> > > > want
>>> >> > > > > >> > > > > >> > > > > >> to
>>> >> > > > > >> > > > > >> > > > > >> > set
>>> >> > > > > >> > > > > >> > > > > >> > > >> yet another bad example that
>>> will
>>> >> > then
>>> >> > > be
>>> >> > > > > >> > pointed
>>> >> > > > > >> > > to
>>> >> > > > > >> > > > > in
>>> >> > > > > >> > > > > >> the
>>> >> > > > > >> > > > > >> > > > > future.
>>> >> > > > > >> > > > > >> > > > > >> > > >>
>>> >> > > > > >> > > > > >> > > > > >> > > >> On Sep 8, 2016 10:50, "Ted
>>> Yu" <
>>> >> > > > > >> > > yuzhihong@gmail.com
>>> >> > > > > >> > > > >
>>> >> > > > > >> > > > > >> > wrote:
>>> >> > > > > >> > > > > >> > > > > >> > > >>
>>> >> > > > > >> > > > > >> > > > > >> > > >> > Is there any concern which
>>> is
>>> >> not
>>> >> > > > > >> addressed ?
>>> >> > > > > >> > > > > >> > > > > >> > > >> >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > Do we need another Vote
>>> thread ?
>>> >> > > > > >> > > > > >> > > > > >> > > >> >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > Thanks
>>> >> > > > > >> > > > > >> > > > > >> > > >> >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > On Thu, Sep 8, 2016 at 9:21
>>> AM,
>>> >> > > Andrew
>>> >> > > > > >> > Purtell <
>>> >> > > > > >> > > > > >> > > > > >> apurtell@apache.org
>>> >> > > > > >> > > > > >> > > > > >> > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > wrote:
>>> >> > > > > >> > > > > >> > > > > >> > > >> >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > Vlad,
>>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > I apologize for using the
>>> term
>>> >> > > > > >> 'half-baked'
>>> >> > > > > >> > > in a
>>> >> > > > > >> > > > > way
>>> >> > > > > >> > > > > >> > that
>>> >> > > > > >> > > > > >> > > > > could
>>> >> > > > > >> > > > > >> > > > > >> > > seem a
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > description of
>>> HBASE-7912. I
>>> >> > meant
>>> >> > > > that
>>> >> > > > > >> as a
>>> >> > > > > >> > > > > general
>>> >> > > > > >> > > > > >> > > > > >> hypothetical.
>>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > On Wed, Sep 7, 2016 at
>>> 9:36
>>> >> AM,
>>> >> > > > > Vladimir
>>> >> > > > > >> > > > Rodionov
>>> >> > > > > >> > > > > <
>>> >> > > > > >> > > > > >> > > > > >> > > >> > vladrodionov@gmail.com>
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > wrote:
>>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >> I'm not sure that
>>> "There
>>> >> is
>>> >> > > > > already
>>> >> > > > > >> > lots
>>> >> > > > > >> > > of
>>> >> > > > > >> > > > > >> > > half-baked
>>> >> > > > > >> > > > > >> > > > > >> code
>>> >> > > > > >> > > > > >> > > > > >> > in
>>> >> > > > > >> > > > > >> > > > > >> > > the
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > branch,
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > so what's the harm in
>>> adding
>>> >> > > more?"
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > I meant - not
>>> production -
>>> >> > ready
>>> >> > > > yet.
>>> >> > > > > >> This
>>> >> > > > > >> > > is
>>> >> > > > > >> > > > > 2.0
>>> >> > > > > >> > > > > >> > > > > development
>>> >> > > > > >> > > > > >> > > > > >> > > branch
>>> >> > > > > >> > > > > >> > > > > >> > > >> > and,
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > hence many features are
>>> in
>>> >> > works,
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > not being tested well
>>> etc.
>>> >> I do
>>> >> > > not
>>> >> > > > > >> > consider
>>> >> > > > > >> > > > > backup
>>> >> > > > > >> > > > > >> > as
>>> >> > > > > >> > > > > >> > > > half
>>> >> > > > > >> > > > > >> > > > > >> > baked
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > feature -
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > it has passed our
>>> internal
>>> >> QA
>>> >> > and
>>> >> > > > has
>>> >> > > > > >> very
>>> >> > > > > >> > > > good
>>> >> > > > > >> > > > > >> doc,
>>> >> > > > > >> > > > > >> > > > which
>>> >> > > > > >> > > > > >> > > > > we
>>> >> > > > > >> > > > > >> > > > > >> > will
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > provide
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > to Apache shortly.
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > -Vlad
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > On Wed, Sep 7, 2016 at
>>> 9:13
>>> >> AM,
>>> >> > > > > Andrew
>>> >> > > > > >> > > > Purtell <
>>> >> > > > > >> > > > > >> > > > > >> > > apurtell@apache.org>
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > We shouldn't admit
>>> half
>>> >> baked
>>> >> > > > > changes
>>> >> > > > > >> > that
>>> >> > > > > >> > > > > won't
>>> >> > > > > >> > > > > >> be
>>> >> > > > > >> > > > > >> > > > > >> finished.
>>> >> > > > > >> > > > > >> > > > > >> > > >> However
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > in
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > this case the crew
>>> >> working on
>>> >> > > > this
>>> >> > > > > >> > feature
>>> >> > > > > >> > > > are
>>> >> > > > > >> > > > > >> long
>>> >> > > > > >> > > > > >> > > > > timers
>>> >> > > > > >> > > > > >> > > > > >> and
>>> >> > > > > >> > > > > >> > > > > >> > > less
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > likely
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > than just about
>>> anyone to
>>> >> > leave
>>> >> > > > > >> > something
>>> >> > > > > >> > > > in a
>>> >> > > > > >> > > > > >> half
>>> >> > > > > >> > > > > >> > > > baked
>>> >> > > > > >> > > > > >> > > > > >> > > state. Of
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > course
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > there is no guarantee
>>> how
>>> >> > > > anything
>>> >> > > > > >> will
>>> >> > > > > >> > > turn
>>> >> > > > > >> > > > > out,
>>> >> > > > > >> > > > > >> > > but I
>>> >> > > > > >> > > > > >> > > > > am
>>> >> > > > > >> > > > > >> > > > > >> > > willing
>>> >> > > > > >> > > > > >> > > > > >> > > >> to
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > take
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > a little on faith if
>>> they
>>> >> > feel
>>> >> > > > > their
>>> >> > > > > >> > best
>>> >> > > > > >> > > > path
>>> >> > > > > >> > > > > >> > > forward
>>> >> > > > > >> > > > > >> > > > > now
>>> >> > > > > >> > > > > >> > > > > >> is
>>> >> > > > > >> > > > > >> > > > > >> > to
>>> >> > > > > >> > > > > >> > > > > >> > > >> > merge
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > to
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > trunk. I only wish I
>>> had
>>> >> > > > bandwidth
>>> >> > > > > to
>>> >> > > > > >> > have
>>> >> > > > > >> > > > > done
>>> >> > > > > >> > > > > >> > some
>>> >> > > > > >> > > > > >> > > > real
>>> >> > > > > >> > > > > >> > > > > >> > > kicking
>>> >> > > > > >> > > > > >> > > > > >> > > >> of
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > the
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > tires by now. Maybe
>>> this
>>> >> > week.
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > (Yes, I'm using some
>>> of
>>> >> that
>>> >> > > time
>>> >> > > > > for
>>> >> > > > > >> > this
>>> >> > > > > >> > > > > email
>>> >> > > > > >> > > > > >> > :-)
>>> >> > > > > >> > > > > >> > > > but
>>> >> > > > > >> > > > > >> > > > > I
>>> >> > > > > >> > > > > >> > > > > >> > type
>>> >> > > > > >> > > > > >> > > > > >> > > >> > fast.)
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > That said, I would
>>> like to
>>> >> > > > agitate
>>> >> > > > > >> for
>>> >> > > > > >> > > > making
>>> >> > > > > >> > > > > 2.0
>>> >> > > > > >> > > > > >> > > more
>>> >> > > > > >> > > > > >> > > > > real
>>> >> > > > > >> > > > > >> > > > > >> > and
>>> >> > > > > >> > > > > >> > > > > >> > > >> spend
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > some
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > time on it now that
>>> I'm
>>> >> > winding
>>> >> > > > > down
>>> >> > > > > >> > with
>>> >> > > > > >> > > > > 0.98. I
>>> >> > > > > >> > > > > >> > > think
>>> >> > > > > >> > > > > >> > > > > >> that
>>> >> > > > > >> > > > > >> > > > > >> > > means
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branching for 2.0 real
>>> >> soon
>>> >> > now
>>> >> > > > and
>>> >> > > > > >> even
>>> >> > > > > >> > > > > evicting
>>> >> > > > > >> > > > > >> > > > things
>>> >> > > > > >> > > > > >> > > > > >> from
>>> >> > > > > >> > > > > >> > > > > >> > > 2.0
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > branch
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > that aren't finished
>>> or
>>> >> > stable,
>>> >> > > > > >> leaving
>>> >> > > > > >> > > them
>>> >> > > > > >> > > > > only
>>> >> > > > > >> > > > > >> > > once
>>> >> > > > > >> > > > > >> > > > > >> again
>>> >> > > > > >> > > > > >> > > > > >> > in
>>> >> > > > > >> > > > > >> > > > > >> > > the
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > master
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branch. Or, maybe just
>>> >> > evicting
>>> >> > > > > them.
>>> >> > > > > >> > > Let's
>>> >> > > > > >> > > > > take
>>> >> > > > > >> > > > > >> it
>>> >> > > > > >> > > > > >> > > > case
>>> >> > > > > >> > > > > >> > > > > by
>>> >> > > > > >> > > > > >> > > > > >> > > case.
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > I think this feature
>>> can
>>> >> come
>>> >> > > in
>>> >> > > > > >> > > relatively
>>> >> > > > > >> > > > > >> safely.
>>> >> > > > > >> > > > > >> > > As
>>> >> > > > > >> > > > > >> > > > > >> added
>>> >> > > > > >> > > > > >> > > > > >> > > >> > insurance,
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > let's admit the
>>> >> possibility
>>> >> > it
>>> >> > > > > could
>>> >> > > > > >> be
>>> >> > > > > >> > > > > reverted
>>> >> > > > > >> > > > > >> on
>>> >> > > > > >> > > > > >> > > the
>>> >> > > > > >> > > > > >> > > > > 2.0
>>> >> > > > > >> > > > > >> > > > > >> > > branch
>>> >> > > > > >> > > > > >> > > > > >> > > >> if
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > folks
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > working on
>>> stabilizing 2.0
>>> >> > > decide
>>> >> > > > > to
>>> >> > > > > >> > evict
>>> >> > > > > >> > > > it
>>> >> > > > > >> > > > > >> > because
>>> >> > > > > >> > > > > >> > > > it
>>> >> > > > > >> > > > > >> > > > > is
>>> >> > > > > >> > > > > >> > > > > >> > > >> > unfinished
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > or
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > unstable, because that
>>> >> > > certainly
>>> >> > > > > can
>>> >> > > > > >> > > > happen. I
>>> >> > > > > >> > > > > >> > would
>>> >> > > > > >> > > > > >> > > > > >> expect if
>>> >> > > > > >> > > > > >> > > > > >> > > talk
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > like
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > that starts, we'd get
>>> help
>>> >> > > > > finishing
>>> >> > > > > >> or
>>> >> > > > > >> > > > > >> stabilizing
>>> >> > > > > >> > > > > >> > > > > what's
>>> >> > > > > >> > > > > >> > > > > >> > under
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > discussion
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > for revert. Or, we'd
>>> have
>>> >> a
>>> >> > > > revert.
>>> >> > > > > >> > Either
>>> >> > > > > >> > > > way
>>> >> > > > > >> > > > > >> the
>>> >> > > > > >> > > > > >> > > > > outcome
>>> >> > > > > >> > > > > >> > > > > >> is
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > acceptable.
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > On Wed, Sep 7, 2016 at
>>> >> 8:56
>>> >> > AM,
>>> >> > > > > Dima
>>> >> > > > > >> > > Spivak
>>> >> > > > > >> > > > <
>>> >> > > > > >> > > > > >> > > > > >> > > dimaspivak@apache.org
>>> >> > > > > >> > > > > >> > > > > >> > > >> >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > I'm not sure that
>>> >> "There is
>>> >> > > > > already
>>> >> > > > > >> > lots
>>> >> > > > > >> > > > of
>>> >> > > > > >> > > > > >> > > > half-baked
>>> >> > > > > >> > > > > >> > > > > >> code
>>> >> > > > > >> > > > > >> > > > > >> > in
>>> >> > > > > >> > > > > >> > > > > >> > > >> the
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branch,
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > so what's the harm
>>> in
>>> >> > adding
>>> >> > > > > more?"
>>> >> > > > > >> > is a
>>> >> > > > > >> > > > > good
>>> >> > > > > >> > > > > >> > code
>>> >> > > > > >> > > > > >> > > > > commit
>>> >> > > > > >> > > > > >> > > > > >> > > >> > philosophy
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > for
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > a
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > fault-tolerant
>>> >> distributed
>>> >> > > data
>>> >> > > > > >> store.
>>> >> > > > > >> > > ;)
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > More seriously, a
>>> lack
>>> >> of
>>> >> > > test
>>> >> > > > > >> > coverage
>>> >> > > > > >> > > > for
>>> >> > > > > >> > > > > >> > > existing
>>> >> > > > > >> > > > > >> > > > > >> > features
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > shouldn't
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > be
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > used as
>>> justification
>>> >> for
>>> >> > > > > >> introducing
>>> >> > > > > >> > > new
>>> >> > > > > >> > > > > >> > features
>>> >> > > > > >> > > > > >> > > > with
>>> >> > > > > >> > > > > >> > > > > >> the
>>> >> > > > > >> > > > > >> > > > > >> > > same
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > shortcomings.
>>> >> Ultimately,
>>> >> > > it's
>>> >> > > > > the
>>> >> > > > > >> end
>>> >> > > > > >> > > > user
>>> >> > > > > >> > > > > who
>>> >> > > > > >> > > > > >> > > will
>>> >> > > > > >> > > > > >> > > > > feel
>>> >> > > > > >> > > > > >> > > > > >> > the
>>> >> > > > > >> > > > > >> > > > > >> > > >> pain,
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > so
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > shouldn't we do
>>> >> everything
>>> >> > we
>>> >> > > > can
>>> >> > > > > >> to
>>> >> > > > > >> > > > > mitigate
>>> >> > > > > >> > > > > >> > that?
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > -Dima
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > On Wed, Sep 7, 2016
>>> at
>>> >> 8:46
>>> >> > > AM,
>>> >> > > > > >> > Vladimir
>>> >> > > > > >> > > > > >> > Rodionov <
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>> vladrodionov@gmail.com>
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > wrote:
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Sean,
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have docs
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Agree. We have a
>>> doc
>>> >> and
>>> >> > > > backup
>>> >> > > > > >> is
>>> >> > > > > >> > the
>>> >> > > > > >> > > > > most
>>> >> > > > > >> > > > > >> > > > > documented
>>> >> > > > > >> > > > > >> > > > > >> > > feature
>>> >> > > > > >> > > > > >> > > > > >> > > >> > :),
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > we
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > will
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > release it
>>> shortly to
>>> >> > > Apache.
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have sunny-day
>>> >> > > correctness
>>> >> > > > > >> tests
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Feature has
>>> close to
>>> >> 60
>>> >> > > test
>>> >> > > > > >> cases,
>>> >> > > > > >> > > > which
>>> >> > > > > >> > > > > >> run
>>> >> > > > > >> > > > > >> > > for
>>> >> > > > > >> > > > > >> > > > > >> approx
>>> >> > > > > >> > > > > >> > > > > >> > 30
>>> >> > > > > >> > > > > >> > > > > >> > > >> min.
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > We
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > can
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > add more, if
>>> >> community do
>>> >> > > not
>>> >> > > > > >> mind
>>> >> > > > > >> > :)
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have
>>> >> > > > > >> > correctness-in-face-of-failure
>>> >> > > > > >> > > > > tests
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Any examples of
>>> these
>>> >> > tests
>>> >> > > > in
>>> >> > > > > >> > > existing
>>> >> > > > > >> > > > > >> > features?
>>> >> > > > > >> > > > > >> > > > In
>>> >> > > > > >> > > > > >> > > > > >> > works,
>>> >> > > > > >> > > > > >> > > > > >> > > we
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > have a
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > clear
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > understanding of
>>> what
>>> >> > > should
>>> >> > > > be
>>> >> > > > > >> done
>>> >> > > > > >> > > by
>>> >> > > > > >> > > > > the
>>> >> > > > > >> > > > > >> > time
>>> >> > > > > >> > > > > >> > > of
>>> >> > > > > >> > > > > >> > > > > 2.0
>>> >> > > > > >> > > > > >> > > > > >> > > >> release.
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > That is very close
>>> >> goal
>>> >> > for
>>> >> > > > us,
>>> >> > > > > >> to
>>> >> > > > > >> > > > verify
>>> >> > > > > >> > > > > IT
>>> >> > > > > >> > > > > >> > > monkey
>>> >> > > > > >> > > > > >> > > > > for
>>> >> > > > > >> > > > > >> > > > > >> > > >> existing
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > code.
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * don't rely on
>>> things
>>> >> > > > outside
>>> >> > > > > of
>>> >> > > > > >> > > HBase
>>> >> > > > > >> > > > > for
>>> >> > > > > >> > > > > >> > > normal
>>> >> > > > > >> > > > > >> > > > > >> > operation
>>> >> > > > > >> > > > > >> > > > > >> > > >> > (okay
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > for
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > advanced
>>> operation)
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > We do not.
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Enormous time has
>>> been
>>> >> > > spent
>>> >> > > > > >> already
>>> >> > > > > >> > > on
>>> >> > > > > >> > > > > the
>>> >> > > > > >> > > > > >> > > > > development
>>> >> > > > > >> > > > > >> > > > > >> > and
>>> >> > > > > >> > > > > >> > > > > >> > > >> > testing
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > the
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > feature, it has
>>> passed
>>> >> > our
>>> >> > > > > >> internal
>>> >> > > > > >> > > > tests
>>> >> > > > > >> > > > > and
>>> >> > > > > >> > > > > >> > > many
>>> >> > > > > >> > > > > >> > > > > >> rounds
>>> >> > > > > >> > > > > >> > > > > >> > of
>>> >> > > > > >> > > > > >> > > > > >> > > >> code
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > reviews
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > by HBase
>>> committers.
>>> >> We
>>> >> > do
>>> >> > > > not
>>> >> > > > > >> mind
>>> >> > > > > >> > if
>>> >> > > > > >> > > > > >> someone
>>> >> > > > > >> > > > > >> > > from
>>> >> > > > > >> > > > > >> > > > > >> HBase
>>> >> > > > > >> > > > > >> > > > > >> > > >> > community
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > (outside of HW)
>>> will
>>> >> > review
>>> >> > > > the
>>> >> > > > > >> > code,
>>> >> > > > > >> > > > but
>>> >> > > > > >> > > > > it
>>> >> > > > > >> > > > > >> > will
>>> >> > > > > >> > > > > >> > > > > >> probably
>>> >> > > > > >> > > > > >> > > > > >> > > >> takes
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > forever
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > to
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > wait for
>>> volunteer?,
>>> >> the
>>> >> > > > > feature
>>> >> > > > > >> is
>>> >> > > > > >> > > > quite
>>> >> > > > > >> > > > > >> large
>>> >> > > > > >> > > > > >> > > > (1MB+
>>> >> > > > > >> > > > > >> > > > > >> > > >> cumulative
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > patch)
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > 2.0 branch is
>>> full of
>>> >> > half
>>> >> > > > > baked
>>> >> > > > > >> > > > features,
>>> >> > > > > >> > > > > >> most
>>> >> > > > > >> > > > > >> > > of
>>> >> > > > > >> > > > > >> > > > > them
>>> >> > > > > >> > > > > >> > > > > >> > are
>>> >> > > > > >> > > > > >> > > > > >> > > in
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > active
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > development,
>>> >> therefore I
>>> >> > am
>>> >> > > > not
>>> >> > > > > >> > > > following
>>> >> > > > > >> > > > > you
>>> >> > > > > >> > > > > >> > > here,
>>> >> > > > > >> > > > > >> > > > > >> Sean?
>>> >> > > > > >> > > > > >> > > > > >> > > Why
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > HBASE-7912
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > is
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > not good enough
>>> yet
>>> >> to be
>>> >> > > > > >> integrated
>>> >> > > > > >> > > > into
>>> >> > > > > >> > > > > 2.0
>>> >> > > > > >> > > > > >> > > > branch?
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > -Vlad
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > On Wed, Sep 7,
>>> 2016 at
>>> >> > 8:23
>>> >> > > > AM,
>>> >> > > > > >> Sean
>>> >> > > > > >> > > > > Busbey <
>>> >> > > > > >> > > > > >> > > > > >> > > busbey@apache.org
>>> >> > > > > >> > > > > >> > > > > >> > > >> >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > On Tue, Sep 6,
>>> 2016
>>> >> at
>>> >> > > > 10:36
>>> >> > > > > >> PM,
>>> >> > > > > >> > > Josh
>>> >> > > > > >> > > > > >> Elser <
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > josh.elser@gmail.com>
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > wrote:
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > So, the
>>> answer to
>>> >> > > Sean's
>>> >> > > > > >> > original
>>> >> > > > > >> > > > > >> question
>>> >> > > > > >> > > > > >> > is
>>> >> > > > > >> > > > > >> > > > "as
>>> >> > > > > >> > > > > >> > > > > >> > > robust as
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > snapshots
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > presently
>>> are"?
>>> >> > > > > >> (independence of
>>> >> > > > > >> > > > > >> > > backup/restore
>>> >> > > > > >> > > > > >> > > > > >> > failure
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > tolerance
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > from
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > snapshot
>>> failure
>>> >> > > > tolerance)
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > Is this just a
>>> >> > question
>>> >> > > > WRT
>>> >> > > > > >> > > context
>>> >> > > > > >> > > > of
>>> >> > > > > >> > > > > >> the
>>> >> > > > > >> > > > > >> > > > > change,
>>> >> > > > > >> > > > > >> > > > > >> or
>>> >> > > > > >> > > > > >> > > > > >> > > is it
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > means
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > for a
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > veto
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > from you,
>>> Sean?
>>> >> Just
>>> >> > > > trying
>>> >> > > > > >> to
>>> >> > > > > >> > > make
>>> >> > > > > >> > > > > sure
>>> >> > > > > >> > > > > >> > I'm
>>> >> > > > > >> > > > > >> > > > > >> following
>>> >> > > > > >> > > > > >> > > > > >> > > >> along
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > adequately.
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > I'd say ATM I'm
>>> -0,
>>> >> > > > bordering
>>> >> > > > > >> on
>>> >> > > > > >> > -1
>>> >> > > > > >> > > > but
>>> >> > > > > >> > > > > not
>>> >> > > > > >> > > > > >> > for
>>> >> > > > > >> > > > > >> > > > > >> reasons
>>> >> > > > > >> > > > > >> > > > > >> > I
>>> >> > > > > >> > > > > >> > > > > >> > > can
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > articulate
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > well.
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Here's an
>>> attempt.
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > We've been
>>> trying to
>>> >> > > move,
>>> >> > > > > as a
>>> >> > > > > >> > > > > community,
>>> >> > > > > >> > > > > >> > > > towards
>>> >> > > > > >> > > > > >> > > > > >> > > minimizing
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > risk
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > to
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > downstream
>>> folks by
>>> >> > > getting
>>> >> > > > > >> > > "complete
>>> >> > > > > >> > > > > >> enough
>>> >> > > > > >> > > > > >> > > for
>>> >> > > > > >> > > > > >> > > > > use"
>>> >> > > > > >> > > > > >> > > > > >> > > gates
>>> >> > > > > >> > > > > >> > > > > >> > > >> in
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > place
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > before we
>>> introduce
>>> >> new
>>> >> > > > > >> features.
>>> >> > > > > >> > > This
>>> >> > > > > >> > > > > was
>>> >> > > > > >> > > > > >> > > > spurred
>>> >> > > > > >> > > > > >> > > > > >> by a
>>> >> > > > > >> > > > > >> > > > > >> > > some
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > features
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > getting in
>>> >> half-baked
>>> >> > and
>>> >> > > > > never
>>> >> > > > > >> > > making
>>> >> > > > > >> > > > > it
>>> >> > > > > >> > > > > >> to
>>> >> > > > > >> > > > > >> > > "can
>>> >> > > > > >> > > > > >> > > > > >> really
>>> >> > > > > >> > > > > >> > > > > >> > > use"
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > status
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > (I'm thinking of
>>> >> > > > distributed
>>> >> > > > > >> log
>>> >> > > > > >> > > > replay
>>> >> > > > > >> > > > > and
>>> >> > > > > >> > > > > >> > the
>>> >> > > > > >> > > > > >> > > > > >> zk-less
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > assignment
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > stuff, I don't
>>> >> recall
>>> >> > if
>>> >> > > > > there
>>> >> > > > > >> was
>>> >> > > > > >> > > > > more).
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The gates,
>>> >> generally,
>>> >> > > > > included
>>> >> > > > > >> > > things
>>> >> > > > > >> > > > > like:
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have docs
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have sunny-day
>>> >> > > > correctness
>>> >> > > > > >> tests
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have
>>> >> > > > > >> > > correctness-in-face-of-failure
>>> >> > > > > >> > > > > tests
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't rely on
>>> >> things
>>> >> > > > > outside
>>> >> > > > > >> of
>>> >> > > > > >> > > > HBase
>>> >> > > > > >> > > > > for
>>> >> > > > > >> > > > > >> > > > normal
>>> >> > > > > >> > > > > >> > > > > >> > > operation
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > (okay
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > for
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > advanced
>>> operation)
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > As an example,
>>> we
>>> >> kept
>>> >> > > the
>>> >> > > > > MOB
>>> >> > > > > >> > work
>>> >> > > > > >> > > > off
>>> >> > > > > >> > > > > in
>>> >> > > > > >> > > > > >> a
>>> >> > > > > >> > > > > >> > > > branch
>>> >> > > > > >> > > > > >> > > > > >> and
>>> >> > > > > >> > > > > >> > > > > >> > > out
>>> >> > > > > >> > > > > >> > > > > >> > > >> of
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > master
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > until it could
>>> pass
>>> >> > these
>>> >> > > > > >> > criteria.
>>> >> > > > > >> > > > The
>>> >> > > > > >> > > > > big
>>> >> > > > > >> > > > > >> > > > > exemption
>>> >> > > > > >> > > > > >> > > > > >> > > we've
>>> >> > > > > >> > > > > >> > > > > >> > > >> had
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > to
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > this was the
>>> >> > hbase-spark
>>> >> > > > > >> > > integration,
>>> >> > > > > >> > > > > where
>>> >> > > > > >> > > > > >> > we
>>> >> > > > > >> > > > > >> > > > all
>>> >> > > > > >> > > > > >> > > > > >> > agreed
>>> >> > > > > >> > > > > >> > > > > >> > > it
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > could
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > land in master
>>> >> because
>>> >> > it
>>> >> > > > was
>>> >> > > > > >> very
>>> >> > > > > >> > > > well
>>> >> > > > > >> > > > > >> > > isolated
>>> >> > > > > >> > > > > >> > > > > (the
>>> >> > > > > >> > > > > >> > > > > >> > > slide
>>> >> > > > > >> > > > > >> > > > > >> > > >> > away
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > from
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > including docs
>>> as a
>>> >> > > > > first-class
>>> >> > > > > >> > part
>>> >> > > > > >> > > > of
>>> >> > > > > >> > > > > >> > > building
>>> >> > > > > >> > > > > >> > > > up
>>> >> > > > > >> > > > > >> > > > > >> that
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > integration
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > has led me to
>>> doubt
>>> >> the
>>> >> > > > > wisdom
>>> >> > > > > >> of
>>> >> > > > > >> > > this
>>> >> > > > > >> > > > > >> > > decision).
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > We've also been
>>> >> > treating
>>> >> > > > > >> inclusion
>>> >> > > > > >> > > in
>>> >> > > > > >> > > > a
>>> >> > > > > >> > > > > >> > > "probably
>>> >> > > > > >> > > > > >> > > > > >> will
>>> >> > > > > >> > > > > >> > > > > >> > be
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > released
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > to
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > downstream"
>>> branches
>>> >> > as a
>>> >> > > > > >> higher
>>> >> > > > > >> > > bar,
>>> >> > > > > >> > > > > >> > requiring
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't
>>> moderately
>>> >> > impact
>>> >> > > > > >> > > performance
>>> >> > > > > >> > > > > when
>>> >> > > > > >> > > > > >> > the
>>> >> > > > > >> > > > > >> > > > > >> feature
>>> >> > > > > >> > > > > >> > > > > >> > > isn't
>>> >> > > > > >> > > > > >> > > > > >> > > >> in
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > use
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't severely
>>> >> impact
>>> >> > > > > >> > performance
>>> >> > > > > >> > > > when
>>> >> > > > > >> > > > > >> the
>>> >> > > > > >> > > > > >> > > > > feature
>>> >> > > > > >> > > > > >> > > > > >> is
>>> >> > > > > >> > > > > >> > > > > >> > in
>>> >> > > > > >> > > > > >> > > > > >> > > >> use
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * either
>>> >> default-to-on
>>> >> > or
>>> >> > > > > show
>>> >> > > > > >> > > enough
>>> >> > > > > >> > > > > >> demand
>>> >> > > > > >> > > > > >> > to
>>> >> > > > > >> > > > > >> > > > > >> believe
>>> >> > > > > >> > > > > >> > > > > >> > a
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > non-trivial
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > number of folks
>>> will
>>> >> > turn
>>> >> > > > the
>>> >> > > > > >> > > feature
>>> >> > > > > >> > > > on
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The above has
>>> kept
>>> >> MOB
>>> >> > > and
>>> >> > > > > >> > > hbase-spark
>>> >> > > > > >> > > > > >> > > > integration
>>> >> > > > > >> > > > > >> > > > > >> out
>>> >> > > > > >> > > > > >> > > > > >> > of
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > branch-1,
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > presumably while
>>> >> > they've
>>> >> > > > > >> "gotten
>>> >> > > > > >> > > more
>>> >> > > > > >> > > > > >> stable"
>>> >> > > > > >> > > > > >> > > in
>>> >> > > > > >> > > > > >> > > > > >> master
>>> >> > > > > >> > > > > >> > > > > >> > > from
>>> >> > > > > >> > > > > >> > > > > >> > > >> > the
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > odd
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > vendor
>>> inclusion.
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Are we going to
>>> >> have a
>>> >> > > 2.0
>>> >> > > > > >> release
>>> >> > > > > >> > > > > before
>>> >> > > > > >> > > > > >> the
>>> >> > > > > >> > > > > >> > > end
>>> >> > > > > >> > > > > >> > > > > of
>>> >> > > > > >> > > > > >> > > > > >> the
>>> >> > > > > >> > > > > >> > > > > >> > > >> year?
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > We're
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > coming up on 1.5
>>> >> years
>>> >> > > > since
>>> >> > > > > >> the
>>> >> > > > > >> > > > > release of
>>> >> > > > > >> > > > > >> > > > version
>>> >> > > > > >> > > > > >> > > > > >> 1.0;
>>> >> > > > > >> > > > > >> > > > > >> > > >> seems
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > like
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > it's about time,
>>> >> > though I
>>> >> > > > > >> haven't
>>> >> > > > > >> > > seen
>>> >> > > > > >> > > > > any
>>> >> > > > > >> > > > > >> > > > concrete
>>> >> > > > > >> > > > > >> > > > > >> > plans
>>> >> > > > > >> > > > > >> > > > > >> > > >> this
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > year.
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Presuming we are
>>> >> going
>>> >> > to
>>> >> > > > > have
>>> >> > > > > >> one
>>> >> > > > > >> > > by
>>> >> > > > > >> > > > > the
>>> >> > > > > >> > > > > >> end
>>> >> > > > > >> > > > > >> > > of
>>> >> > > > > >> > > > > >> > > > > the
>>> >> > > > > >> > > > > >> > > > > >> > > year, it
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > seems a
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > bit close to
>>> still
>>> >> be
>>> >> > > > adding
>>> >> > > > > in
>>> >> > > > > >> > > > > "features
>>> >> > > > > >> > > > > >> > that
>>> >> > > > > >> > > > > >> > > > need
>>> >> > > > > >> > > > > >> > > > > >> > > maturing"
>>> >> > > > > >> > > > > >> > > > > >> > > >> > on
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > the
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > branch.
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The lack of a
>>> >> concrete
>>> >> > > plan
>>> >> > > > > for
>>> >> > > > > >> > 2.0
>>> >> > > > > >> > > > > keeps
>>> >> > > > > >> > > > > >> me
>>> >> > > > > >> > > > > >> > > from
>>> >> > > > > >> > > > > >> > > > > >> > > considering
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > these
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > things blocker
>>> at
>>> >> the
>>> >> > > > moment.
>>> >> > > > > >> But
>>> >> > > > > >> > I
>>> >> > > > > >> > > > know
>>> >> > > > > >> > > > > >> > first
>>> >> > > > > >> > > > > >> > > > hand
>>> >> > > > > >> > > > > >> > > > > >> how
>>> >> > > > > >> > > > > >> > > > > >> > > much
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > trouble
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > folks have had
>>> with
>>> >> > other
>>> >> > > > > >> features
>>> >> > > > > >> > > > that
>>> >> > > > > >> > > > > >> have
>>> >> > > > > >> > > > > >> > > gone
>>> >> > > > > >> > > > > >> > > > > >> into
>>> >> > > > > >> > > > > >> > > > > >> > > >> > downstream
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > facing releases
>>> >> without
>>> >> > > > > >> robustness
>>> >> > > > > >> > > > > checks
>>> >> > > > > >> > > > > >> > (i.e.
>>> >> > > > > >> > > > > >> > > > > >> > > replication),
>>> >> > > > > >> > > > > >> > > > > >> > > >> > and
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > I'm
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > concerned about
>>> what
>>> >> > > we're
>>> >> > > > > >> setting
>>> >> > > > > >> > > up
>>> >> > > > > >> > > > if
>>> >> > > > > >> > > > > >> 2.0
>>> >> > > > > >> > > > > >> > > goes
>>> >> > > > > >> > > > > >> > > > > out
>>> >> > > > > >> > > > > >> > > > > >> > with
>>> >> > > > > >> > > > > >> > > > > >> > > >> this
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > feature in its
>>> >> current
>>> >> > > > state.
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > --
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > Best regards,
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >    - Andy
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > Problems worthy of
>>> attack
>>> >> > prove
>>> >> > > > > their
>>> >> > > > > >> > > worth
>>> >> > > > > >> > > > by
>>> >> > > > > >> > > > > >> > > hitting
>>> >> > > > > >> > > > > >> > > > > >> back. -
>>> >> > > > > >> > > > > >> > > > > >> > > Piet
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > Hein
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > (via Tom White)
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > --
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > Best regards,
>>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > >    - Andy
>>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > Problems worthy of attack
>>> >> prove
>>> >> > > their
>>> >> > > > > >> worth
>>> >> > > > > >> > by
>>> >> > > > > >> > > > > >> hitting
>>> >> > > > > >> > > > > >> > > > back.
>>> >> > > > > >> > > > > >> > > > > -
>>> >> > > > > >> > > > > >> > > > > >> > Piet
>>> >> > > > > >> > > > > >> > > > > >> > > >> Hein
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > (via Tom White)
>>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> >
>>> >> > > > > >> > > > > >> > > > > >> > > >>
>>> >> > > > > >> > > > > >> > > > > >> > >
>>> >> > > > > >> > > > > >> > > > > >> >
>>> >> > > > > >> > > > > >> > > > > >>
>>> >> > > > > >> > > > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > >
>>> >> > > > > >> > > > > >> > > >
>>> >> > > > > >> > > > > >> > >
>>> >> > > > > >> > > > > >> >
>>> >> > > > > >> > > > > >>
>>> >> > > > > >> > > > >
>>> >> > > > > >> > > > >
>>> >> > > > > >> > > > >
>>> >> > > > > >> > > > > --
>>> >> > > > > >> > > > > busbey
>>> >> > > > > >> > > > >
>>> >> > > > > >> > > >
>>> >> > > > > >> > >
>>> >> > > > > >> >
>>> >> > > > > >>
>>> >> > > > > >
>>> >> > > > > >
>>> >> > > > >
>>> >> > > >
>>> >> > >
>>> >> >
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >>
>>> >> -- Appy
>>> >>
>>> >
>>> >
>>>
>>
>>
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Stack <st...@duboce.net>]

Nice writeup Matteo. I'm trying to get to where you are at but am not there
yet. Almost. Agree that it should be a blocker that all gets hoisted out of
core into a backup module and that the limitations are spelled out clearly
in doc.

I have spent a good bit of time reviewing and testing this feature. I would
like my review and concerns addressed and I'd like it to be clear how;
either explicit follow-on issues, pointers to where in the patch or doc my
remarks have been catered to, etc. Until then, I am against commit.

St.Ack


On Fri, Nov 18, 2016 at 3:48 PM, Matteo Bertozzi <th...@gmail.com>
wrote:

> I did one last pass to the mega patch. I don't see anything major that
> should block the merge.
>
> - most of the code is isolated in the backup package
> - all the backup code is client side
> - there are few changes to the server side, mainly for cleaners, wal
> rolling and similar (which is ok)
> - there is a good number of tests, and an integration test
>
> the code seems to have still some left overs from the old implementation,
> and some stuff needs a cleanup. but I don't think this should be used as an
> argument to block the merge. I think the guys will keep working on this and
> they may also get help of others once the patch is in master.
>
> I still have my concerns about the current limitations, but these are
> things already planned for phase 3, so some of this stuff may even be in
> the final 2.0.
> but as long as we have a "current limitations" section in the user guide
> mentioning important stuff like the ones below, I'm ok with it.
>  - if you write to the table with Durability.SKIP_WALS your data will not
> be in the incremental-backup
>  - if you bulkload files that data will not be in the incremental backup
> (HBASE-14417)
>  - the incremental backup will not only contains the data of the table you
> specified but also the regions from other tables that are on the same set
> of RSs (HBASE-14141) ...maybe a note about security around this topic
>  - the incremental backup will not contains just the "latest row" between
> backup A and B, but it will also contains all the updates occurred in
> between. but the restore does not allow you to restore up to a certain
> point in time, the restore will always be up to the "latest backup point".
>  - you should limit the number of "incremental" up to N (or maybe SIZE), to
> avoid replay time becoming the bottleneck. (HBASE-14135)
>
> I'll be ok even with the above not being in the final 2.0,
> but i'd like to see as blocker for the final 2.0 (not the merge)
>  - the backup code moved in an hbase-backup module
>  - and some more work around tools, especially to try to unify and make
> simple the backup experience (simple example: in some case there is a
> backup_id argument in others a backupId argument. or things like.. restore
> is not clear if given an incremental id it will do the full restore from
> full up to that point or if i need to apply manually everything).
>
> in conclusion, I think we can open a merge vote. I'll be +1 on it, and I
> think we should try to reject -1 with just a "code cleanup" motivation,
> since there will still be work going on on the code after the merge.
>
> Matteo
>
>
> On Sun, Nov 6, 2016 at 10:54 PM, Devaraj Das <dd...@hortonworks.com> wrote:
>
> > Stack and others, anything else on the patch? Merge to master now?
> >
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Vladimir Rodionov <vl...@gmail.com>]

Thanks, Matteo

It is a good write up.

HBASE-14417 (bulk load support) is pretty close to code complete now. There
is already work in progress on
HBASE-14141 (Filtering WAL on backup). I would say they will be complete in
a 2-3 weeks.

and I agree that "code clean up before merge" requests should be ignored.
This is work in progress and we are doing clean up all the time.

-Vlad

On Fri, Nov 18, 2016 at 3:53 PM, Ted Yu <yu...@gmail.com> wrote:

> Thanks, Matteo.
>
> bq. restore is not clear if given an incremental id it will do the full
> restore from full up to that point or if i need to apply manually
> everything
>
> The restore takes into consideration of the dependent backup(s).
> So there is no need to apply preceding backup(s) manually.
>
> On Fri, Nov 18, 2016 at 3:48 PM, Matteo Bertozzi <th...@gmail.com>
> wrote:
>
> > I did one last pass to the mega patch. I don't see anything major that
> > should block the merge.
> >
> > - most of the code is isolated in the backup package
> > - all the backup code is client side
> > - there are few changes to the server side, mainly for cleaners, wal
> > rolling and similar (which is ok)
> > - there is a good number of tests, and an integration test
> >
> > the code seems to have still some left overs from the old implementation,
> > and some stuff needs a cleanup. but I don't think this should be used as
> an
> > argument to block the merge. I think the guys will keep working on this
> and
> > they may also get help of others once the patch is in master.
> >
> > I still have my concerns about the current limitations, but these are
> > things already planned for phase 3, so some of this stuff may even be in
> > the final 2.0.
> > but as long as we have a "current limitations" section in the user guide
> > mentioning important stuff like the ones below, I'm ok with it.
> >  - if you write to the table with Durability.SKIP_WALS your data will not
> > be in the incremental-backup
> >  - if you bulkload files that data will not be in the incremental backup
> > (HBASE-14417)
> >  - the incremental backup will not only contains the data of the table
> you
> > specified but also the regions from other tables that are on the same set
> > of RSs (HBASE-14141) ...maybe a note about security around this topic
> >  - the incremental backup will not contains just the "latest row" between
> > backup A and B, but it will also contains all the updates occurred in
> > between. but the restore does not allow you to restore up to a certain
> > point in time, the restore will always be up to the "latest backup
> point".
> >  - you should limit the number of "incremental" up to N (or maybe SIZE),
> to
> > avoid replay time becoming the bottleneck. (HBASE-14135)
> >
> > I'll be ok even with the above not being in the final 2.0,
> > but i'd like to see as blocker for the final 2.0 (not the merge)
> >  - the backup code moved in an hbase-backup module
> >  - and some more work around tools, especially to try to unify and make
> > simple the backup experience (simple example: in some case there is a
> > backup_id argument in others a backupId argument. or things like..
> restore
> > is not clear if given an incremental id it will do the full restore from
> > full up to that point or if i need to apply manually everything).
> >
> > in conclusion, I think we can open a merge vote. I'll be +1 on it, and I
> > think we should try to reject -1 with just a "code cleanup" motivation,
> > since there will still be work going on on the code after the merge.
> >
> > Matteo
> >
> >
> > On Sun, Nov 6, 2016 at 10:54 PM, Devaraj Das <dd...@hortonworks.com>
> wrote:
> >
> > > Stack and others, anything else on the patch? Merge to master now?
> > >
> >
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Stack <st...@duboce.net>]

On Sat, Sep 9, 2017 at 5:58 PM, Vladimir Rodionov <vl...@gmail.com>
wrote:

> That was I thought. Thanks. Can you tell me that you are not considering AM
> v2 as an unfinished and untested feature? The question to Stack as well.
>
>
It is tested. Every HBase2 cluster deploy exercises AMv2. At scale, I have
been able to run with Chaos Monkeys well in excess of what I was able to do
in old AMv1.

Regards 'unfinished', plainly it is doing the job of AMv1. There are unit
tests to reenable before beta, tests that mostly no longer work in their
current form because they made assumption about assign internals,
assumptions that no longer hold in the new regime.

AMv2 addresses head-on the source of most issues folks encounter running
hbase; i.e. mangled assign. Among other improvements -- performance,
resilience -- AMv2 leaves a trail and transitions can be reasoned through.
When bugs, they are fixable and tests can be written standalone, something
that was not possible with AMv1.

AMv2 is core to hbase2. On the other hand, backup/restore is not. I have a
hard time assessing what is there and getting straight answers when I ask
about it. You have some time before beta to turn this state of affairs
around.

Thanks,
St.Ack




> On Sat, Sep 9, 2017 at 5:53 PM, Andrew Purtell <an...@gmail.com>
> wrote:
>
> > No all I have to do is pay attention to words you have written yourself
> in
> > emails and on JIRA. Don't argue with us not to believe our lying eyes,
> > consider finishing the work. I'll be happy to try it out when you
> indicate
> > it can work if anything happens to fail on the cluster at the time. Until
> > then there are a lot of other things need doing first.
> >
> >
> > On Sep 9, 2017, at 5:25 PM, Vladimir Rodionov <vl...@gmail.com>
> > wrote:
> >
> > >>> but the impression we have is it is unfinished and untested.
> > > To make a conclusion that "feature is not finished and tested"  you
> have
> > > had to test it at least.
> > > Andrew, If you have discovered issues, why wouldn't you open bug JIRAs?
> > >
> > > -Vlad
> > >
> > > On Sat, Sep 9, 2017 at 4:40 PM, Andrew Purtell <
> andrew.purtell@gmail.com
> > >
> > > wrote:
> > >
> > >> For what it's worth, I think AMv2 is the main reason to have a 2.0 in
> > the
> > >> first place, so I would both agree it needs a lot more testing and
> yet I
> > >> would want us to have a 2.0 release as the vehicle for getting that to
> > >> happen. For other features without testing from a number of parties or
> > at
> > >> scale the value proposition is less clear and it's fine by me for the
> > RM to
> > >> set them aside for future releases.
> > >>
> > >> Also, I can relay that there is some interest where I work in
> utilizing
> > >> HBASE-7912 but the impression we have is it is unfinished and
> untested.
> > So
> > >> for now we are ignoring it and continuing with home grown solutions.
> > Part
> > >> of the problem is fault tolerance was left to the last phase(s) and
> yet
> > it
> > >> is an essential property for adoption for serious work. The best way
> to
> > >> resolve this IMHO is for the developers of this feature to complete
> > those
> > >> unfinished JIRAs, especially concerning resilience to failures.
> > >>
> > >>
> > >>> On Sep 9, 2017, at 4:11 PM, Vladimir Rodionov <
> vladrodionov@gmail.com>
> > >> wrote:
> > >>>
> > >>> Hmm, the next on your list (of kicked out from branch v2) should be
> AM
> > >> v2 I
> > >>> presume?
> > >>>
> > >>> -Vlad
> > >>>
> > >>>> On Sat, Sep 9, 2017 at 4:04 PM, stack <sa...@gmail.com> wrote:
> > >>>>
> > >>>> In spite of repeated requests for eng summary of state of this
> feature
> > >> --
> > >>>> summary of what is in 2.0, what is not, what the capabilities are,
> how
> > >> well
> > >>>> it has been tested and at what scale -- all I get, when the requests
> > are
> > >>>> not ignored, are pointers to lists of ill-describing jiras and some
> > >> pending
> > >>>> user facing doc update.
> > >>>>
> > >>>> For other features, mob or region server groups, I know that they
> have
> > >> been
> > >>>> running at scale in production for as much as a year and more. I
> have
> > >> some
> > >>>> confidence these items basically work.  For backup/restore I have no
> > >> such
> > >>>> sense even after spending time in review and trying to use the
> > feature.
> > >>>>
> > >>>> As release manager, I have say over what makes it into a release.
> > >> Unless
> > >>>> the work is done to convince me that backup/restore is more than a
> > lump
> > >> of
> > >>>> code and a few unit tests that can pass on some fellows laptop, I am
> > >> going
> > >>>> to kick it out of branch-2.  Let the feature harden more in master
> > >> branch
> > >>>> before it ships in a release.
> > >>>>
> > >>>> S
> > >>>>
> > >>>> On Sep 8, 2017 10:59 PM, "Vladimir Rodionov" <
> vladrodionov@gmail.com>
> > >>>> wrote:
> > >>>>
> > >>>>>>> Have I grasped the state of things correctly, Vlad?
> > >>>>>
> > >>>>> Josh, the only thing which is still pending is doc update. All
> other
> > >>>>> features are good to have but not a blockers for 2.0 release.
> > >>>>>
> > >>>>> -Vlad
> > >>>>>
> > >>>>> On Fri, Sep 8, 2017 at 10:42 PM, Vladimir Rodionov <
> > >>>> vladrodionov@gmail.com
> > >>>>>>
> > >>>>> wrote:
> > >>>>>
> > >>>>>>>> What testing and at what
> > >>>>>>>> scale has testing been done?
> > >>>>>>
> > >>>>>> Do we have have that for other features?
> > >>>>>>
> > >>>>>>
> > >>>>>> On Fri, Sep 8, 2017 at 10:41 PM, Vladimir Rodionov <
> > >>>>> vladrodionov@gmail.com
> > >>>>>>> wrote:
> > >>>>>>
> > >>>>>>>>> It asks: "How do I figure what of backup/restore feature is
> going
> > >>>> to
> > >>>>>>> be in
> > >>>>>>>>> hbase-2.0.0?
> > >>>>>>>
> > >>>>>>> Hmm, wait for doc update.
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>> On Fri, Sep 8, 2017 at 2:39 PM, Stack <st...@duboce.net> wrote:
> > >>>>>>>>
> > >>>>>>>> HBASE-14414 is a JIRA with a list of random seeming issues w/
> > >>>>>>>> non-descript
> > >>>>>>>> summaries: "Add nonce support to TableBackupProcedure, BackupID
> > must
> > >>>>>>>> include backup set name, ...". The last comment in that issue is
> > >> from
> > >>>>>>>> July.
> > >>>>>>>> It asks: "How do I figure what of backup/restore feature is
> going
> > to
> > >>>> be
> > >>>>>>>> in
> > >>>>>>>> hbase-2.0.0? Thanks Vladimir Rodionov
> > >>>>>>>> <https://issues.apache.org/jira/secure/ViewProfile.jspa?
> > >>>> name=vrodionov
> > >>>>>>>>> ."
> > >>>>>>>> to which there is no answer.  Doc update is TODO.
> > >>>>>>>>
> > >>>>>>>> Where is the summary of the capability in hbase-2? What testing
> > and
> > >>>> at
> > >>>>>>>> what
> > >>>>>>>> scale has testing been done? Is this 'stable or experimental'?
> If
> > I
> > >>>>> can't
> > >>>>>>>> get basic info on this feature though I ask repeatedly, what
> hope
> > >>>> does
> > >>>>>>>> the
> > >>>>>>>> poor old operator have?
> > >>>>>>>>
> > >>>>>>>> St.Ack
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On Fri, Sep 8, 2017 at 1:59 PM, Vladimir Rodionov <
> > >>>>>>>> vladrodionov@gmail.com>
> > >>>>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>>> HBASE-14414
> > >>>>>>>>>
> > >>>>>>>>>> On Fri, Sep 8, 2017 at 1:14 PM, Stack <st...@duboce.net>
> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>> Where do I go to get the current status of this feature?
> Looking
> > >>>> in
> > >>>>>>>> JIRA
> > >>>>>>>>> I
> > >>>>>>>>>> see loads of issues open against backup including some against
> > >>>>>>>>> hbase-2.0.0
> > >>>>>>>>>> and no progress being made that I can discern.
> > >>>>>>>>>>
> > >>>>>>>>>> Thanks,
> > >>>>>>>>>> S
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>> On Wed, Nov 23, 2016 at 8:52 AM, Stack <st...@duboce.net>
> > wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Tue, Nov 22, 2016 at 6:48 PM, Stack <st...@duboce.net>
> > >>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>>> On Tue, Nov 22, 2016 at 3:17 PM, Vladimir Rodionov <
> > >>>>>>>>>>>> vladrodionov@gmail.com> wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>>>> and/or he answered most of the review feedback
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> No, questions are still open, but I do not see any blockers
> > >>>> and
> > >>>>>>>> we
> > >>>>>>>>> have
> > >>>>>>>>>>>>> HBASE-16940 to address these questions.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>> Agree. No blockers but stuff that should be dealt with (No
> one
> > >>>>>>>> will
> > >>>>>>>>> pay
> > >>>>>>>>>>>> me any attention once merge goes in -- smile).
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>> Let me clarify the above. I want review addressed before
> merge
> > >>>>>>>> happens.
> > >>>>>>>>>>> Sorry if any confusion.
> > >>>>>>>>>>> St.Ack
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>> St.Ack
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> On Tue, Nov 22, 2016 at 3:04 PM, Devaraj Das <
> > >>>>>>>> ddas@hortonworks.com>
> > >>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Hi Stack, hats off to you for spending so much time on
> > >>>> this!
> > >>>>>>>>> Thanks!
> > >>>>>>>>>>>>> From
> > >>>>>>>>>>>>>> my understanding, Vlad has raised follow-up jiras for the
> > >>>>>>>> issues
> > >>>>>>>>> you
> > >>>>>>>>>>>>>> raised, and/or he answered most of the review feedback.
> So,
> > >>>>> do
> > >>>>>>>> you
> > >>>>>>>>>>>>> think we
> > >>>>>>>>>>>>>> could do a merge vote now?
> > >>>>>>>>>>>>>> Devaraj.
> > >>>>>>>>>>>>>> ________________________________________
> > >>>>>>>>>>>>>> From: Vladimir Rodionov <vl...@gmail.com>
> > >>>>>>>>>>>>>> Sent: Monday, November 21, 2016 8:34 PM
> > >>>>>>>>>>>>>> To: dev@hbase.apache.org
> > >>>>>>>>>>>>>> Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch
> > >>>>>>>>> HBASE-7912
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> I have spent a good bit of time reviewing and testing
> > >>>> this
> > >>>>>>>>>> feature.
> > >>>>>>>>>>>>> I
> > >>>>>>>>>>>>>> would
> > >>>>>>>>>>>>>>>> like my review and concerns addressed and I'd like it to
> > >>>>> be
> > >>>>>>>>> clear
> > >>>>>>>>>>>>> how;
> > >>>>>>>>>>>>>>>> either explicit follow-on issues, pointers to where in
> > >>>> the
> > >>>>>>>> patch
> > >>>>>>>>>> or
> > >>>>>>>>>>>>> doc
> > >>>>>>>>>>>>>> my
> > >>>>>>>>>>>>>>>> remarks have been catered to, etc. Until then, I am
> > >>>>> against
> > >>>>>>>>>> commit.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Stack, mega patch review comments will be addressed in the
> > >>>>>>>>> dedicated
> > >>>>>>>>>>>>> JIRA:
> > >>>>>>>>>>>>>> HBASE-16940
> > >>>>>>>>>>>>>> I have open several other JIRAs to address your other
> > >>>>> comments
> > >>>>>>>> (not
> > >>>>>>>>>> on
> > >>>>>>>>>>>>>> review board).
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Details are here (end of the thread):
> > >>>>>>>>>>>>>> https://issues.apache.org/jira/browse/HBASE-14123
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Let me know what else should we do to move merge forward.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> -Vlad
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> On Fri, Nov 18, 2016 at 4:54 PM, Stack <st...@duboce.net>
> > >>>>>>>> wrote:
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> On Fri, Nov 18, 2016 at 3:53 PM, Ted Yu <
> > >>>>> yuzhihong@gmail.com
> > >>>>>>>>>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Thanks, Matteo.
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> bq. restore is not clear if given an incremental id it
> > >>>>>>>> will do
> > >>>>>>>>>> the
> > >>>>>>>>>>>>> full
> > >>>>>>>>>>>>>>>> restore from full up to that point or if i need to
> > >>>> apply
> > >>>>>>>>> manually
> > >>>>>>>>>>>>>>>> everything
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> The restore takes into consideration of the dependent
> > >>>>>>>>> backup(s).
> > >>>>>>>>>>>>>>>> So there is no need to apply preceding backup(s)
> > >>>>> manually.
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> I ask this question on the issue. It is not clear from
> > >>>> the
> > >>>>>>>> usage
> > >>>>>>>>> or
> > >>>>>>>>>>>>> doc
> > >>>>>>>>>>>>>> how
> > >>>>>>>>>>>>>>> to run a restore from incremental. Can you fix in doc and
> > >>>>>>>> usage
> > >>>>>>>>> how
> > >>>>>>>>>>>>> so I
> > >>>>>>>>>>>>>>> can be clear and try it. Currently I am stuck verifying a
> > >>>>>>>> round
> > >>>>>>>>>> trip
> > >>>>>>>>>>>>>> backup
> > >>>>>>>>>>>>>>> restore made of incrementals.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>> S
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> On Fri, Nov 18, 2016 at 3:48 PM, Matteo Bertozzi <
> > >>>>>>>>>>>>>>> theo.bertozzi@gmail.com>
> > >>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> I did one last pass to the mega patch. I don't see
> > >>>>>>>> anything
> > >>>>>>>>>> major
> > >>>>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>>> should block the merge.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> - most of the code is isolated in the backup package
> > >>>>>>>>>>>>>>>>> - all the backup code is client side
> > >>>>>>>>>>>>>>>>> - there are few changes to the server side, mainly
> > >>>> for
> > >>>>>>>>>> cleaners,
> > >>>>>>>>>>>>> wal
> > >>>>>>>>>>>>>>>>> rolling and similar (which is ok)
> > >>>>>>>>>>>>>>>>> - there is a good number of tests, and an integration
> > >>>>>>>> test
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> the code seems to have still some left overs from the
> > >>>>> old
> > >>>>>>>>>>>>>>> implementation,
> > >>>>>>>>>>>>>>>>> and some stuff needs a cleanup. but I don't think
> > >>>> this
> > >>>>>>>> should
> > >>>>>>>>>> be
> > >>>>>>>>>>>>> used
> > >>>>>>>>>>>>>>> as
> > >>>>>>>>>>>>>>>> an
> > >>>>>>>>>>>>>>>>> argument to block the merge. I think the guys will
> > >>>> keep
> > >>>>>>>>> working
> > >>>>>>>>>>>>> on
> > >>>>>>>>>>>>>> this
> > >>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>> they may also get help of others once the patch is in
> > >>>>>>>> master.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> I still have my concerns about the current
> > >>>> limitations,
> > >>>>>>>> but
> > >>>>>>>>>>>>> these are
> > >>>>>>>>>>>>>>>>> things already planned for phase 3, so some of this
> > >>>>>>>> stuff may
> > >>>>>>>>>>>>> even be
> > >>>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>> the final 2.0.
> > >>>>>>>>>>>>>>>>> but as long as we have a "current limitations"
> > >>>> section
> > >>>>>>>> in the
> > >>>>>>>>>>>>> user
> > >>>>>>>>>>>>>>> guide
> > >>>>>>>>>>>>>>>>> mentioning important stuff like the ones below, I'm
> > >>>> ok
> > >>>>>>>> with
> > >>>>>>>>> it.
> > >>>>>>>>>>>>>>>>> - if you write to the table with
> > >>>> Durability.SKIP_WALS
> > >>>>>>>> your
> > >>>>>>>>>> data
> > >>>>>>>>>>>>> will
> > >>>>>>>>>>>>>>> not
> > >>>>>>>>>>>>>>>>> be in the incremental-backup
> > >>>>>>>>>>>>>>>>> - if you bulkload files that data will not be in the
> > >>>>>>>>>> incremental
> > >>>>>>>>>>>>>>> backup
> > >>>>>>>>>>>>>>>>> (HBASE-14417)
> > >>>>>>>>>>>>>>>>> - the incremental backup will not only contains the
> > >>>>>>>> data of
> > >>>>>>>>>> the
> > >>>>>>>>>>>>>> table
> > >>>>>>>>>>>>>>>> you
> > >>>>>>>>>>>>>>>>> specified but also the regions from other tables that
> > >>>>>>>> are on
> > >>>>>>>>>> the
> > >>>>>>>>>>>>> same
> > >>>>>>>>>>>>>>> set
> > >>>>>>>>>>>>>>>>> of RSs (HBASE-14141) ...maybe a note about security
> > >>>>>>>> around
> > >>>>>>>>> this
> > >>>>>>>>>>>>> topic
> > >>>>>>>>>>>>>>>>> - the incremental backup will not contains just the
> > >>>>>>>> "latest
> > >>>>>>>>>> row"
> > >>>>>>>>>>>>>>> between
> > >>>>>>>>>>>>>>>>> backup A and B, but it will also contains all the
> > >>>>> updates
> > >>>>>>>>>>>>> occurred in
> > >>>>>>>>>>>>>>>>> between. but the restore does not allow you to
> > >>>> restore
> > >>>>>>>> up to
> > >>>>>>>>> a
> > >>>>>>>>>>>>>> certain
> > >>>>>>>>>>>>>>>>> point in time, the restore will always be up to the
> > >>>>>>>> "latest
> > >>>>>>>>>>>>> backup
> > >>>>>>>>>>>>>>>> point".
> > >>>>>>>>>>>>>>>>> - you should limit the number of "incremental" up
> > >>>> to N
> > >>>>>>>> (or
> > >>>>>>>>>> maybe
> > >>>>>>>>>>>>>>> SIZE),
> > >>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>> avoid replay time becoming the bottleneck.
> > >>>>> (HBASE-14135)
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> I'll be ok even with the above not being in the final
> > >>>>>>>> 2.0,
> > >>>>>>>>>>>>>>>>> but i'd like to see as blocker for the final 2.0 (not
> > >>>>> the
> > >>>>>>>>>> merge)
> > >>>>>>>>>>>>>>>>> - the backup code moved in an hbase-backup module
> > >>>>>>>>>>>>>>>>> - and some more work around tools, especially to try
> > >>>>> to
> > >>>>>>>>> unify
> > >>>>>>>>>>>>> and
> > >>>>>>>>>>>>>> make
> > >>>>>>>>>>>>>>>>> simple the backup experience (simple example: in some
> > >>>>>>>> case
> > >>>>>>>>>> there
> > >>>>>>>>>>>>> is a
> > >>>>>>>>>>>>>>>>> backup_id argument in others a backupId argument. or
> > >>>>>>>> things
> > >>>>>>>>>>>>> like..
> > >>>>>>>>>>>>>>>> restore
> > >>>>>>>>>>>>>>>>> is not clear if given an incremental id it will do
> > >>>> the
> > >>>>>>>> full
> > >>>>>>>>>>>>> restore
> > >>>>>>>>>>>>>>> from
> > >>>>>>>>>>>>>>>>> full up to that point or if i need to apply manually
> > >>>>>>>>>> everything).
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> in conclusion, I think we can open a merge vote. I'll
> > >>>>> be
> > >>>>>>>> +1
> > >>>>>>>>> on
> > >>>>>>>>>>>>> it,
> > >>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>> I
> > >>>>>>>>>>>>>>>>> think we should try to reject -1 with just a "code
> > >>>>>>>> cleanup"
> > >>>>>>>>>>>>>> motivation,
> > >>>>>>>>>>>>>>>>> since there will still be work going on on the code
> > >>>>>>>> after the
> > >>>>>>>>>>>>> merge.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Matteo
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> On Sun, Nov 6, 2016 at 10:54 PM, Devaraj Das <
> > >>>>>>>>>>>>> ddas@hortonworks.com>
> > >>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Stack and others, anything else on the patch? Merge
> > >>>>> to
> > >>>>>>>>> master
> > >>>>>>>>>>>>> now?
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>
> > >>
> >
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Andrew Purtell <an...@gmail.com>]

To your question, I think it is misplaced (at least to me). Go back and read what I wrote you just today. 

As for the backup feature, once the work for HBASE-15227 is committed and it is resolved, I think we'd be interested in trying it out, and until then I am not personally interested in its  inclusion in a release, but that is just me. 


> On Sep 9, 2017, at 5:58 PM, Vladimir Rodionov <vl...@gmail.com> wrote:
> 
> That was I thought. Thanks. Can you tell me that you are not considering AM
> v2 as an unfinished and untested feature? The question to Stack as well.
> 
> On Sat, Sep 9, 2017 at 5:53 PM, Andrew Purtell <an...@gmail.com>
> wrote:
> 
>> No all I have to do is pay attention to words you have written yourself in
>> emails and on JIRA. Don't argue with us not to believe our lying eyes,
>> consider finishing the work. I'll be happy to try it out when you indicate
>> it can work if anything happens to fail on the cluster at the time. Until
>> then there are a lot of other things need doing first.
>> 
>> 
>> On Sep 9, 2017, at 5:25 PM, Vladimir Rodionov <vl...@gmail.com>
>> wrote:
>> 
>>>>> but the impression we have is it is unfinished and untested.
>>> To make a conclusion that "feature is not finished and tested"  you have
>>> had to test it at least.
>>> Andrew, If you have discovered issues, why wouldn't you open bug JIRAs?
>>> 
>>> -Vlad
>>> 
>>> On Sat, Sep 9, 2017 at 4:40 PM, Andrew Purtell <andrew.purtell@gmail.com
>>> 
>>> wrote:
>>> 
>>>> For what it's worth, I think AMv2 is the main reason to have a 2.0 in
>> the
>>>> first place, so I would both agree it needs a lot more testing and yet I
>>>> would want us to have a 2.0 release as the vehicle for getting that to
>>>> happen. For other features without testing from a number of parties or
>> at
>>>> scale the value proposition is less clear and it's fine by me for the
>> RM to
>>>> set them aside for future releases.
>>>> 
>>>> Also, I can relay that there is some interest where I work in utilizing
>>>> HBASE-7912 but the impression we have is it is unfinished and untested.
>> So
>>>> for now we are ignoring it and continuing with home grown solutions.
>> Part
>>>> of the problem is fault tolerance was left to the last phase(s) and yet
>> it
>>>> is an essential property for adoption for serious work. The best way to
>>>> resolve this IMHO is for the developers of this feature to complete
>> those
>>>> unfinished JIRAs, especially concerning resilience to failures.
>>>> 
>>>> 
>>>>> On Sep 9, 2017, at 4:11 PM, Vladimir Rodionov <vl...@gmail.com>
>>>> wrote:
>>>>> 
>>>>> Hmm, the next on your list (of kicked out from branch v2) should be AM
>>>> v2 I
>>>>> presume?
>>>>> 
>>>>> -Vlad
>>>>> 
>>>>>> On Sat, Sep 9, 2017 at 4:04 PM, stack <sa...@gmail.com> wrote:
>>>>>> 
>>>>>> In spite of repeated requests for eng summary of state of this feature
>>>> --
>>>>>> summary of what is in 2.0, what is not, what the capabilities are, how
>>>> well
>>>>>> it has been tested and at what scale -- all I get, when the requests
>> are
>>>>>> not ignored, are pointers to lists of ill-describing jiras and some
>>>> pending
>>>>>> user facing doc update.
>>>>>> 
>>>>>> For other features, mob or region server groups, I know that they have
>>>> been
>>>>>> running at scale in production for as much as a year and more. I have
>>>> some
>>>>>> confidence these items basically work.  For backup/restore I have no
>>>> such
>>>>>> sense even after spending time in review and trying to use the
>> feature.
>>>>>> 
>>>>>> As release manager, I have say over what makes it into a release.
>>>> Unless
>>>>>> the work is done to convince me that backup/restore is more than a
>> lump
>>>> of
>>>>>> code and a few unit tests that can pass on some fellows laptop, I am
>>>> going
>>>>>> to kick it out of branch-2.  Let the feature harden more in master
>>>> branch
>>>>>> before it ships in a release.
>>>>>> 
>>>>>> S
>>>>>> 
>>>>>> On Sep 8, 2017 10:59 PM, "Vladimir Rodionov" <vl...@gmail.com>
>>>>>> wrote:
>>>>>> 
>>>>>>>>> Have I grasped the state of things correctly, Vlad?
>>>>>>> 
>>>>>>> Josh, the only thing which is still pending is doc update. All other
>>>>>>> features are good to have but not a blockers for 2.0 release.
>>>>>>> 
>>>>>>> -Vlad
>>>>>>> 
>>>>>>> On Fri, Sep 8, 2017 at 10:42 PM, Vladimir Rodionov <
>>>>>> vladrodionov@gmail.com
>>>>>>>> 
>>>>>>> wrote:
>>>>>>> 
>>>>>>>>>> What testing and at what
>>>>>>>>>> scale has testing been done?
>>>>>>>> 
>>>>>>>> Do we have have that for other features?
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On Fri, Sep 8, 2017 at 10:41 PM, Vladimir Rodionov <
>>>>>>> vladrodionov@gmail.com
>>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>>>> It asks: "How do I figure what of backup/restore feature is going
>>>>>> to
>>>>>>>>> be in
>>>>>>>>>>> hbase-2.0.0?
>>>>>>>>> 
>>>>>>>>> Hmm, wait for doc update.
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>> On Fri, Sep 8, 2017 at 2:39 PM, Stack <st...@duboce.net> wrote:
>>>>>>>>>> 
>>>>>>>>>> HBASE-14414 is a JIRA with a list of random seeming issues w/
>>>>>>>>>> non-descript
>>>>>>>>>> summaries: "Add nonce support to TableBackupProcedure, BackupID
>> must
>>>>>>>>>> include backup set name, ...". The last comment in that issue is
>>>> from
>>>>>>>>>> July.
>>>>>>>>>> It asks: "How do I figure what of backup/restore feature is going
>> to
>>>>>> be
>>>>>>>>>> in
>>>>>>>>>> hbase-2.0.0? Thanks Vladimir Rodionov
>>>>>>>>>> <https://issues.apache.org/jira/secure/ViewProfile.jspa?
>>>>>> name=vrodionov
>>>>>>>>>>> ."
>>>>>>>>>> to which there is no answer.  Doc update is TODO.
>>>>>>>>>> 
>>>>>>>>>> Where is the summary of the capability in hbase-2? What testing
>> and
>>>>>> at
>>>>>>>>>> what
>>>>>>>>>> scale has testing been done? Is this 'stable or experimental'? If
>> I
>>>>>>> can't
>>>>>>>>>> get basic info on this feature though I ask repeatedly, what hope
>>>>>> does
>>>>>>>>>> the
>>>>>>>>>> poor old operator have?
>>>>>>>>>> 
>>>>>>>>>> St.Ack
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> On Fri, Sep 8, 2017 at 1:59 PM, Vladimir Rodionov <
>>>>>>>>>> vladrodionov@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>>> HBASE-14414
>>>>>>>>>>> 
>>>>>>>>>>>> On Fri, Sep 8, 2017 at 1:14 PM, Stack <st...@duboce.net> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>> Where do I go to get the current status of this feature? Looking
>>>>>> in
>>>>>>>>>> JIRA
>>>>>>>>>>> I
>>>>>>>>>>>> see loads of issues open against backup including some against
>>>>>>>>>>> hbase-2.0.0
>>>>>>>>>>>> and no progress being made that I can discern.
>>>>>>>>>>>> 
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> S
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>>> On Wed, Nov 23, 2016 at 8:52 AM, Stack <st...@duboce.net>
>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Tue, Nov 22, 2016 at 6:48 PM, Stack <st...@duboce.net>
>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Tue, Nov 22, 2016 at 3:17 PM, Vladimir Rodionov <
>>>>>>>>>>>>>> vladrodionov@gmail.com> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> and/or he answered most of the review feedback
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> No, questions are still open, but I do not see any blockers
>>>>>> and
>>>>>>>>>> we
>>>>>>>>>>> have
>>>>>>>>>>>>>>> HBASE-16940 to address these questions.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Agree. No blockers but stuff that should be dealt with (No one
>>>>>>>>>> will
>>>>>>>>>>> pay
>>>>>>>>>>>>>> me any attention once merge goes in -- smile).
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> Let me clarify the above. I want review addressed before merge
>>>>>>>>>> happens.
>>>>>>>>>>>>> Sorry if any confusion.
>>>>>>>>>>>>> St.Ack
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> St.Ack
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On Tue, Nov 22, 2016 at 3:04 PM, Devaraj Das <
>>>>>>>>>> ddas@hortonworks.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Hi Stack, hats off to you for spending so much time on
>>>>>> this!
>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>> From
>>>>>>>>>>>>>>>> my understanding, Vlad has raised follow-up jiras for the
>>>>>>>>>> issues
>>>>>>>>>>> you
>>>>>>>>>>>>>>>> raised, and/or he answered most of the review feedback. So,
>>>>>>> do
>>>>>>>>>> you
>>>>>>>>>>>>>>> think we
>>>>>>>>>>>>>>>> could do a merge vote now?
>>>>>>>>>>>>>>>> Devaraj.
>>>>>>>>>>>>>>>> ________________________________________
>>>>>>>>>>>>>>>> From: Vladimir Rodionov <vl...@gmail.com>
>>>>>>>>>>>>>>>> Sent: Monday, November 21, 2016 8:34 PM
>>>>>>>>>>>>>>>> To: dev@hbase.apache.org
>>>>>>>>>>>>>>>> Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch
>>>>>>>>>>> HBASE-7912
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> I have spent a good bit of time reviewing and testing
>>>>>> this
>>>>>>>>>>>> feature.
>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>> like my review and concerns addressed and I'd like it to
>>>>>>> be
>>>>>>>>>>> clear
>>>>>>>>>>>>>>> how;
>>>>>>>>>>>>>>>>>> either explicit follow-on issues, pointers to where in
>>>>>> the
>>>>>>>>>> patch
>>>>>>>>>>>> or
>>>>>>>>>>>>>>> doc
>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>> remarks have been catered to, etc. Until then, I am
>>>>>>> against
>>>>>>>>>>>> commit.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Stack, mega patch review comments will be addressed in the
>>>>>>>>>>> dedicated
>>>>>>>>>>>>>>> JIRA:
>>>>>>>>>>>>>>>> HBASE-16940
>>>>>>>>>>>>>>>> I have open several other JIRAs to address your other
>>>>>>> comments
>>>>>>>>>> (not
>>>>>>>>>>>> on
>>>>>>>>>>>>>>>> review board).
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Details are here (end of the thread):
>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/HBASE-14123
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Let me know what else should we do to move merge forward.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> -Vlad
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On Fri, Nov 18, 2016 at 4:54 PM, Stack <st...@duboce.net>
>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> On Fri, Nov 18, 2016 at 3:53 PM, Ted Yu <
>>>>>>> yuzhihong@gmail.com
>>>>>>>>>>> 
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Thanks, Matteo.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> bq. restore is not clear if given an incremental id it
>>>>>>>>>> will do
>>>>>>>>>>>> the
>>>>>>>>>>>>>>> full
>>>>>>>>>>>>>>>>>> restore from full up to that point or if i need to
>>>>>> apply
>>>>>>>>>>> manually
>>>>>>>>>>>>>>>>>> everything
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> The restore takes into consideration of the dependent
>>>>>>>>>>> backup(s).
>>>>>>>>>>>>>>>>>> So there is no need to apply preceding backup(s)
>>>>>>> manually.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> I ask this question on the issue. It is not clear from
>>>>>> the
>>>>>>>>>> usage
>>>>>>>>>>> or
>>>>>>>>>>>>>>> doc
>>>>>>>>>>>>>>>> how
>>>>>>>>>>>>>>>>> to run a restore from incremental. Can you fix in doc and
>>>>>>>>>> usage
>>>>>>>>>>> how
>>>>>>>>>>>>>>> so I
>>>>>>>>>>>>>>>>> can be clear and try it. Currently I am stuck verifying a
>>>>>>>>>> round
>>>>>>>>>>>> trip
>>>>>>>>>>>>>>>> backup
>>>>>>>>>>>>>>>>> restore made of incrementals.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>> S
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> On Fri, Nov 18, 2016 at 3:48 PM, Matteo Bertozzi <
>>>>>>>>>>>>>>>>> theo.bertozzi@gmail.com>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> I did one last pass to the mega patch. I don't see
>>>>>>>>>> anything
>>>>>>>>>>>> major
>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>> should block the merge.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> - most of the code is isolated in the backup package
>>>>>>>>>>>>>>>>>>> - all the backup code is client side
>>>>>>>>>>>>>>>>>>> - there are few changes to the server side, mainly
>>>>>> for
>>>>>>>>>>>> cleaners,
>>>>>>>>>>>>>>> wal
>>>>>>>>>>>>>>>>>>> rolling and similar (which is ok)
>>>>>>>>>>>>>>>>>>> - there is a good number of tests, and an integration
>>>>>>>>>> test
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> the code seems to have still some left overs from the
>>>>>>> old
>>>>>>>>>>>>>>>>> implementation,
>>>>>>>>>>>>>>>>>>> and some stuff needs a cleanup. but I don't think
>>>>>> this
>>>>>>>>>> should
>>>>>>>>>>>> be
>>>>>>>>>>>>>>> used
>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>> argument to block the merge. I think the guys will
>>>>>> keep
>>>>>>>>>>> working
>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>> they may also get help of others once the patch is in
>>>>>>>>>> master.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> I still have my concerns about the current
>>>>>> limitations,
>>>>>>>>>> but
>>>>>>>>>>>>>>> these are
>>>>>>>>>>>>>>>>>>> things already planned for phase 3, so some of this
>>>>>>>>>> stuff may
>>>>>>>>>>>>>>> even be
>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>> the final 2.0.
>>>>>>>>>>>>>>>>>>> but as long as we have a "current limitations"
>>>>>> section
>>>>>>>>>> in the
>>>>>>>>>>>>>>> user
>>>>>>>>>>>>>>>>> guide
>>>>>>>>>>>>>>>>>>> mentioning important stuff like the ones below, I'm
>>>>>> ok
>>>>>>>>>> with
>>>>>>>>>>> it.
>>>>>>>>>>>>>>>>>>> - if you write to the table with
>>>>>> Durability.SKIP_WALS
>>>>>>>>>> your
>>>>>>>>>>>> data
>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>> be in the incremental-backup
>>>>>>>>>>>>>>>>>>> - if you bulkload files that data will not be in the
>>>>>>>>>>>> incremental
>>>>>>>>>>>>>>>>> backup
>>>>>>>>>>>>>>>>>>> (HBASE-14417)
>>>>>>>>>>>>>>>>>>> - the incremental backup will not only contains the
>>>>>>>>>> data of
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> table
>>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>> specified but also the regions from other tables that
>>>>>>>>>> are on
>>>>>>>>>>>> the
>>>>>>>>>>>>>>> same
>>>>>>>>>>>>>>>>> set
>>>>>>>>>>>>>>>>>>> of RSs (HBASE-14141) ...maybe a note about security
>>>>>>>>>> around
>>>>>>>>>>> this
>>>>>>>>>>>>>>> topic
>>>>>>>>>>>>>>>>>>> - the incremental backup will not contains just the
>>>>>>>>>> "latest
>>>>>>>>>>>> row"
>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>> backup A and B, but it will also contains all the
>>>>>>> updates
>>>>>>>>>>>>>>> occurred in
>>>>>>>>>>>>>>>>>>> between. but the restore does not allow you to
>>>>>> restore
>>>>>>>>>> up to
>>>>>>>>>>> a
>>>>>>>>>>>>>>>> certain
>>>>>>>>>>>>>>>>>>> point in time, the restore will always be up to the
>>>>>>>>>> "latest
>>>>>>>>>>>>>>> backup
>>>>>>>>>>>>>>>>>> point".
>>>>>>>>>>>>>>>>>>> - you should limit the number of "incremental" up
>>>>>> to N
>>>>>>>>>> (or
>>>>>>>>>>>> maybe
>>>>>>>>>>>>>>>>> SIZE),
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> avoid replay time becoming the bottleneck.
>>>>>>> (HBASE-14135)
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> I'll be ok even with the above not being in the final
>>>>>>>>>> 2.0,
>>>>>>>>>>>>>>>>>>> but i'd like to see as blocker for the final 2.0 (not
>>>>>>> the
>>>>>>>>>>>> merge)
>>>>>>>>>>>>>>>>>>> - the backup code moved in an hbase-backup module
>>>>>>>>>>>>>>>>>>> - and some more work around tools, especially to try
>>>>>>> to
>>>>>>>>>>> unify
>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>> simple the backup experience (simple example: in some
>>>>>>>>>> case
>>>>>>>>>>>> there
>>>>>>>>>>>>>>> is a
>>>>>>>>>>>>>>>>>>> backup_id argument in others a backupId argument. or
>>>>>>>>>> things
>>>>>>>>>>>>>>> like..
>>>>>>>>>>>>>>>>>> restore
>>>>>>>>>>>>>>>>>>> is not clear if given an incremental id it will do
>>>>>> the
>>>>>>>>>> full
>>>>>>>>>>>>>>> restore
>>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>> full up to that point or if i need to apply manually
>>>>>>>>>>>> everything).
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> in conclusion, I think we can open a merge vote. I'll
>>>>>>> be
>>>>>>>>>> +1
>>>>>>>>>>> on
>>>>>>>>>>>>>>> it,
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>> think we should try to reject -1 with just a "code
>>>>>>>>>> cleanup"
>>>>>>>>>>>>>>>> motivation,
>>>>>>>>>>>>>>>>>>> since there will still be work going on on the code
>>>>>>>>>> after the
>>>>>>>>>>>>>>> merge.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Matteo
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> On Sun, Nov 6, 2016 at 10:54 PM, Devaraj Das <
>>>>>>>>>>>>>>> ddas@hortonworks.com>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Stack and others, anything else on the patch? Merge
>>>>>>> to
>>>>>>>>>>> master
>>>>>>>>>>>>>>> now?
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>> 
>> 

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Vladimir Rodionov <vl...@gmail.com>]

That was I thought. Thanks. Can you tell me that you are not considering AM
v2 as an unfinished and untested feature? The question to Stack as well.

On Sat, Sep 9, 2017 at 5:53 PM, Andrew Purtell <an...@gmail.com>
wrote:

> No all I have to do is pay attention to words you have written yourself in
> emails and on JIRA. Don't argue with us not to believe our lying eyes,
> consider finishing the work. I'll be happy to try it out when you indicate
> it can work if anything happens to fail on the cluster at the time. Until
> then there are a lot of other things need doing first.
>
>
> On Sep 9, 2017, at 5:25 PM, Vladimir Rodionov <vl...@gmail.com>
> wrote:
>
> >>> but the impression we have is it is unfinished and untested.
> > To make a conclusion that "feature is not finished and tested"  you have
> > had to test it at least.
> > Andrew, If you have discovered issues, why wouldn't you open bug JIRAs?
> >
> > -Vlad
> >
> > On Sat, Sep 9, 2017 at 4:40 PM, Andrew Purtell <andrew.purtell@gmail.com
> >
> > wrote:
> >
> >> For what it's worth, I think AMv2 is the main reason to have a 2.0 in
> the
> >> first place, so I would both agree it needs a lot more testing and yet I
> >> would want us to have a 2.0 release as the vehicle for getting that to
> >> happen. For other features without testing from a number of parties or
> at
> >> scale the value proposition is less clear and it's fine by me for the
> RM to
> >> set them aside for future releases.
> >>
> >> Also, I can relay that there is some interest where I work in utilizing
> >> HBASE-7912 but the impression we have is it is unfinished and untested.
> So
> >> for now we are ignoring it and continuing with home grown solutions.
> Part
> >> of the problem is fault tolerance was left to the last phase(s) and yet
> it
> >> is an essential property for adoption for serious work. The best way to
> >> resolve this IMHO is for the developers of this feature to complete
> those
> >> unfinished JIRAs, especially concerning resilience to failures.
> >>
> >>
> >>> On Sep 9, 2017, at 4:11 PM, Vladimir Rodionov <vl...@gmail.com>
> >> wrote:
> >>>
> >>> Hmm, the next on your list (of kicked out from branch v2) should be AM
> >> v2 I
> >>> presume?
> >>>
> >>> -Vlad
> >>>
> >>>> On Sat, Sep 9, 2017 at 4:04 PM, stack <sa...@gmail.com> wrote:
> >>>>
> >>>> In spite of repeated requests for eng summary of state of this feature
> >> --
> >>>> summary of what is in 2.0, what is not, what the capabilities are, how
> >> well
> >>>> it has been tested and at what scale -- all I get, when the requests
> are
> >>>> not ignored, are pointers to lists of ill-describing jiras and some
> >> pending
> >>>> user facing doc update.
> >>>>
> >>>> For other features, mob or region server groups, I know that they have
> >> been
> >>>> running at scale in production for as much as a year and more. I have
> >> some
> >>>> confidence these items basically work.  For backup/restore I have no
> >> such
> >>>> sense even after spending time in review and trying to use the
> feature.
> >>>>
> >>>> As release manager, I have say over what makes it into a release.
> >> Unless
> >>>> the work is done to convince me that backup/restore is more than a
> lump
> >> of
> >>>> code and a few unit tests that can pass on some fellows laptop, I am
> >> going
> >>>> to kick it out of branch-2.  Let the feature harden more in master
> >> branch
> >>>> before it ships in a release.
> >>>>
> >>>> S
> >>>>
> >>>> On Sep 8, 2017 10:59 PM, "Vladimir Rodionov" <vl...@gmail.com>
> >>>> wrote:
> >>>>
> >>>>>>> Have I grasped the state of things correctly, Vlad?
> >>>>>
> >>>>> Josh, the only thing which is still pending is doc update. All other
> >>>>> features are good to have but not a blockers for 2.0 release.
> >>>>>
> >>>>> -Vlad
> >>>>>
> >>>>> On Fri, Sep 8, 2017 at 10:42 PM, Vladimir Rodionov <
> >>>> vladrodionov@gmail.com
> >>>>>>
> >>>>> wrote:
> >>>>>
> >>>>>>>> What testing and at what
> >>>>>>>> scale has testing been done?
> >>>>>>
> >>>>>> Do we have have that for other features?
> >>>>>>
> >>>>>>
> >>>>>> On Fri, Sep 8, 2017 at 10:41 PM, Vladimir Rodionov <
> >>>>> vladrodionov@gmail.com
> >>>>>>> wrote:
> >>>>>>
> >>>>>>>>> It asks: "How do I figure what of backup/restore feature is going
> >>>> to
> >>>>>>> be in
> >>>>>>>>> hbase-2.0.0?
> >>>>>>>
> >>>>>>> Hmm, wait for doc update.
> >>>>>>>
> >>>>>>>
> >>>>>>>> On Fri, Sep 8, 2017 at 2:39 PM, Stack <st...@duboce.net> wrote:
> >>>>>>>>
> >>>>>>>> HBASE-14414 is a JIRA with a list of random seeming issues w/
> >>>>>>>> non-descript
> >>>>>>>> summaries: "Add nonce support to TableBackupProcedure, BackupID
> must
> >>>>>>>> include backup set name, ...". The last comment in that issue is
> >> from
> >>>>>>>> July.
> >>>>>>>> It asks: "How do I figure what of backup/restore feature is going
> to
> >>>> be
> >>>>>>>> in
> >>>>>>>> hbase-2.0.0? Thanks Vladimir Rodionov
> >>>>>>>> <https://issues.apache.org/jira/secure/ViewProfile.jspa?
> >>>> name=vrodionov
> >>>>>>>>> ."
> >>>>>>>> to which there is no answer.  Doc update is TODO.
> >>>>>>>>
> >>>>>>>> Where is the summary of the capability in hbase-2? What testing
> and
> >>>> at
> >>>>>>>> what
> >>>>>>>> scale has testing been done? Is this 'stable or experimental'? If
> I
> >>>>> can't
> >>>>>>>> get basic info on this feature though I ask repeatedly, what hope
> >>>> does
> >>>>>>>> the
> >>>>>>>> poor old operator have?
> >>>>>>>>
> >>>>>>>> St.Ack
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Fri, Sep 8, 2017 at 1:59 PM, Vladimir Rodionov <
> >>>>>>>> vladrodionov@gmail.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> HBASE-14414
> >>>>>>>>>
> >>>>>>>>>> On Fri, Sep 8, 2017 at 1:14 PM, Stack <st...@duboce.net> wrote:
> >>>>>>>>>>
> >>>>>>>>>> Where do I go to get the current status of this feature? Looking
> >>>> in
> >>>>>>>> JIRA
> >>>>>>>>> I
> >>>>>>>>>> see loads of issues open against backup including some against
> >>>>>>>>> hbase-2.0.0
> >>>>>>>>>> and no progress being made that I can discern.
> >>>>>>>>>>
> >>>>>>>>>> Thanks,
> >>>>>>>>>> S
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> On Wed, Nov 23, 2016 at 8:52 AM, Stack <st...@duboce.net>
> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> On Tue, Nov 22, 2016 at 6:48 PM, Stack <st...@duboce.net>
> >>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> On Tue, Nov 22, 2016 at 3:17 PM, Vladimir Rodionov <
> >>>>>>>>>>>> vladrodionov@gmail.com> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>>>> and/or he answered most of the review feedback
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> No, questions are still open, but I do not see any blockers
> >>>> and
> >>>>>>>> we
> >>>>>>>>> have
> >>>>>>>>>>>>> HBASE-16940 to address these questions.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>> Agree. No blockers but stuff that should be dealt with (No one
> >>>>>>>> will
> >>>>>>>>> pay
> >>>>>>>>>>>> me any attention once merge goes in -- smile).
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>> Let me clarify the above. I want review addressed before merge
> >>>>>>>> happens.
> >>>>>>>>>>> Sorry if any confusion.
> >>>>>>>>>>> St.Ack
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>> St.Ack
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>> On Tue, Nov 22, 2016 at 3:04 PM, Devaraj Das <
> >>>>>>>> ddas@hortonworks.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hi Stack, hats off to you for spending so much time on
> >>>> this!
> >>>>>>>>> Thanks!
> >>>>>>>>>>>>> From
> >>>>>>>>>>>>>> my understanding, Vlad has raised follow-up jiras for the
> >>>>>>>> issues
> >>>>>>>>> you
> >>>>>>>>>>>>>> raised, and/or he answered most of the review feedback. So,
> >>>>> do
> >>>>>>>> you
> >>>>>>>>>>>>> think we
> >>>>>>>>>>>>>> could do a merge vote now?
> >>>>>>>>>>>>>> Devaraj.
> >>>>>>>>>>>>>> ________________________________________
> >>>>>>>>>>>>>> From: Vladimir Rodionov <vl...@gmail.com>
> >>>>>>>>>>>>>> Sent: Monday, November 21, 2016 8:34 PM
> >>>>>>>>>>>>>> To: dev@hbase.apache.org
> >>>>>>>>>>>>>> Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch
> >>>>>>>>> HBASE-7912
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I have spent a good bit of time reviewing and testing
> >>>> this
> >>>>>>>>>> feature.
> >>>>>>>>>>>>> I
> >>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>> like my review and concerns addressed and I'd like it to
> >>>>> be
> >>>>>>>>> clear
> >>>>>>>>>>>>> how;
> >>>>>>>>>>>>>>>> either explicit follow-on issues, pointers to where in
> >>>> the
> >>>>>>>> patch
> >>>>>>>>>> or
> >>>>>>>>>>>>> doc
> >>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>> remarks have been catered to, etc. Until then, I am
> >>>>> against
> >>>>>>>>>> commit.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Stack, mega patch review comments will be addressed in the
> >>>>>>>>> dedicated
> >>>>>>>>>>>>> JIRA:
> >>>>>>>>>>>>>> HBASE-16940
> >>>>>>>>>>>>>> I have open several other JIRAs to address your other
> >>>>> comments
> >>>>>>>> (not
> >>>>>>>>>> on
> >>>>>>>>>>>>>> review board).
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Details are here (end of the thread):
> >>>>>>>>>>>>>> https://issues.apache.org/jira/browse/HBASE-14123
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Let me know what else should we do to move merge forward.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> -Vlad
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Fri, Nov 18, 2016 at 4:54 PM, Stack <st...@duboce.net>
> >>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Fri, Nov 18, 2016 at 3:53 PM, Ted Yu <
> >>>>> yuzhihong@gmail.com
> >>>>>>>>>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thanks, Matteo.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> bq. restore is not clear if given an incremental id it
> >>>>>>>> will do
> >>>>>>>>>> the
> >>>>>>>>>>>>> full
> >>>>>>>>>>>>>>>> restore from full up to that point or if i need to
> >>>> apply
> >>>>>>>>> manually
> >>>>>>>>>>>>>>>> everything
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> The restore takes into consideration of the dependent
> >>>>>>>>> backup(s).
> >>>>>>>>>>>>>>>> So there is no need to apply preceding backup(s)
> >>>>> manually.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I ask this question on the issue. It is not clear from
> >>>> the
> >>>>>>>> usage
> >>>>>>>>> or
> >>>>>>>>>>>>> doc
> >>>>>>>>>>>>>> how
> >>>>>>>>>>>>>>> to run a restore from incremental. Can you fix in doc and
> >>>>>>>> usage
> >>>>>>>>> how
> >>>>>>>>>>>>> so I
> >>>>>>>>>>>>>>> can be clear and try it. Currently I am stuck verifying a
> >>>>>>>> round
> >>>>>>>>>> trip
> >>>>>>>>>>>>>> backup
> >>>>>>>>>>>>>>> restore made of incrementals.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>> S
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Fri, Nov 18, 2016 at 3:48 PM, Matteo Bertozzi <
> >>>>>>>>>>>>>>> theo.bertozzi@gmail.com>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I did one last pass to the mega patch. I don't see
> >>>>>>>> anything
> >>>>>>>>>> major
> >>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>> should block the merge.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> - most of the code is isolated in the backup package
> >>>>>>>>>>>>>>>>> - all the backup code is client side
> >>>>>>>>>>>>>>>>> - there are few changes to the server side, mainly
> >>>> for
> >>>>>>>>>> cleaners,
> >>>>>>>>>>>>> wal
> >>>>>>>>>>>>>>>>> rolling and similar (which is ok)
> >>>>>>>>>>>>>>>>> - there is a good number of tests, and an integration
> >>>>>>>> test
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> the code seems to have still some left overs from the
> >>>>> old
> >>>>>>>>>>>>>>> implementation,
> >>>>>>>>>>>>>>>>> and some stuff needs a cleanup. but I don't think
> >>>> this
> >>>>>>>> should
> >>>>>>>>>> be
> >>>>>>>>>>>>> used
> >>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>> argument to block the merge. I think the guys will
> >>>> keep
> >>>>>>>>> working
> >>>>>>>>>>>>> on
> >>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>> they may also get help of others once the patch is in
> >>>>>>>> master.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I still have my concerns about the current
> >>>> limitations,
> >>>>>>>> but
> >>>>>>>>>>>>> these are
> >>>>>>>>>>>>>>>>> things already planned for phase 3, so some of this
> >>>>>>>> stuff may
> >>>>>>>>>>>>> even be
> >>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>> the final 2.0.
> >>>>>>>>>>>>>>>>> but as long as we have a "current limitations"
> >>>> section
> >>>>>>>> in the
> >>>>>>>>>>>>> user
> >>>>>>>>>>>>>>> guide
> >>>>>>>>>>>>>>>>> mentioning important stuff like the ones below, I'm
> >>>> ok
> >>>>>>>> with
> >>>>>>>>> it.
> >>>>>>>>>>>>>>>>> - if you write to the table with
> >>>> Durability.SKIP_WALS
> >>>>>>>> your
> >>>>>>>>>> data
> >>>>>>>>>>>>> will
> >>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>> be in the incremental-backup
> >>>>>>>>>>>>>>>>> - if you bulkload files that data will not be in the
> >>>>>>>>>> incremental
> >>>>>>>>>>>>>>> backup
> >>>>>>>>>>>>>>>>> (HBASE-14417)
> >>>>>>>>>>>>>>>>> - the incremental backup will not only contains the
> >>>>>>>> data of
> >>>>>>>>>> the
> >>>>>>>>>>>>>> table
> >>>>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>> specified but also the regions from other tables that
> >>>>>>>> are on
> >>>>>>>>>> the
> >>>>>>>>>>>>> same
> >>>>>>>>>>>>>>> set
> >>>>>>>>>>>>>>>>> of RSs (HBASE-14141) ...maybe a note about security
> >>>>>>>> around
> >>>>>>>>> this
> >>>>>>>>>>>>> topic
> >>>>>>>>>>>>>>>>> - the incremental backup will not contains just the
> >>>>>>>> "latest
> >>>>>>>>>> row"
> >>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>> backup A and B, but it will also contains all the
> >>>>> updates
> >>>>>>>>>>>>> occurred in
> >>>>>>>>>>>>>>>>> between. but the restore does not allow you to
> >>>> restore
> >>>>>>>> up to
> >>>>>>>>> a
> >>>>>>>>>>>>>> certain
> >>>>>>>>>>>>>>>>> point in time, the restore will always be up to the
> >>>>>>>> "latest
> >>>>>>>>>>>>> backup
> >>>>>>>>>>>>>>>> point".
> >>>>>>>>>>>>>>>>> - you should limit the number of "incremental" up
> >>>> to N
> >>>>>>>> (or
> >>>>>>>>>> maybe
> >>>>>>>>>>>>>>> SIZE),
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>> avoid replay time becoming the bottleneck.
> >>>>> (HBASE-14135)
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I'll be ok even with the above not being in the final
> >>>>>>>> 2.0,
> >>>>>>>>>>>>>>>>> but i'd like to see as blocker for the final 2.0 (not
> >>>>> the
> >>>>>>>>>> merge)
> >>>>>>>>>>>>>>>>> - the backup code moved in an hbase-backup module
> >>>>>>>>>>>>>>>>> - and some more work around tools, especially to try
> >>>>> to
> >>>>>>>>> unify
> >>>>>>>>>>>>> and
> >>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>> simple the backup experience (simple example: in some
> >>>>>>>> case
> >>>>>>>>>> there
> >>>>>>>>>>>>> is a
> >>>>>>>>>>>>>>>>> backup_id argument in others a backupId argument. or
> >>>>>>>> things
> >>>>>>>>>>>>> like..
> >>>>>>>>>>>>>>>> restore
> >>>>>>>>>>>>>>>>> is not clear if given an incremental id it will do
> >>>> the
> >>>>>>>> full
> >>>>>>>>>>>>> restore
> >>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>> full up to that point or if i need to apply manually
> >>>>>>>>>> everything).
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> in conclusion, I think we can open a merge vote. I'll
> >>>>> be
> >>>>>>>> +1
> >>>>>>>>> on
> >>>>>>>>>>>>> it,
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>> think we should try to reject -1 with just a "code
> >>>>>>>> cleanup"
> >>>>>>>>>>>>>> motivation,
> >>>>>>>>>>>>>>>>> since there will still be work going on on the code
> >>>>>>>> after the
> >>>>>>>>>>>>> merge.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Matteo
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Sun, Nov 6, 2016 at 10:54 PM, Devaraj Das <
> >>>>>>>>>>>>> ddas@hortonworks.com>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Stack and others, anything else on the patch? Merge
> >>>>> to
> >>>>>>>>> master
> >>>>>>>>>>>>> now?
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Andrew Purtell <an...@gmail.com>]

No all I have to do is pay attention to words you have written yourself in emails and on JIRA. Don't argue with us not to believe our lying eyes, consider finishing the work. I'll be happy to try it out when you indicate it can work if anything happens to fail on the cluster at the time. Until then there are a lot of other things need doing first. 


On Sep 9, 2017, at 5:25 PM, Vladimir Rodionov <vl...@gmail.com> wrote:

>>> but the impression we have is it is unfinished and untested.
> To make a conclusion that "feature is not finished and tested"  you have
> had to test it at least.
> Andrew, If you have discovered issues, why wouldn't you open bug JIRAs?
> 
> -Vlad
> 
> On Sat, Sep 9, 2017 at 4:40 PM, Andrew Purtell <an...@gmail.com>
> wrote:
> 
>> For what it's worth, I think AMv2 is the main reason to have a 2.0 in the
>> first place, so I would both agree it needs a lot more testing and yet I
>> would want us to have a 2.0 release as the vehicle for getting that to
>> happen. For other features without testing from a number of parties or at
>> scale the value proposition is less clear and it's fine by me for the RM to
>> set them aside for future releases.
>> 
>> Also, I can relay that there is some interest where I work in utilizing
>> HBASE-7912 but the impression we have is it is unfinished and untested. So
>> for now we are ignoring it and continuing with home grown solutions. Part
>> of the problem is fault tolerance was left to the last phase(s) and yet it
>> is an essential property for adoption for serious work. The best way to
>> resolve this IMHO is for the developers of this feature to complete those
>> unfinished JIRAs, especially concerning resilience to failures.
>> 
>> 
>>> On Sep 9, 2017, at 4:11 PM, Vladimir Rodionov <vl...@gmail.com>
>> wrote:
>>> 
>>> Hmm, the next on your list (of kicked out from branch v2) should be AM
>> v2 I
>>> presume?
>>> 
>>> -Vlad
>>> 
>>>> On Sat, Sep 9, 2017 at 4:04 PM, stack <sa...@gmail.com> wrote:
>>>> 
>>>> In spite of repeated requests for eng summary of state of this feature
>> --
>>>> summary of what is in 2.0, what is not, what the capabilities are, how
>> well
>>>> it has been tested and at what scale -- all I get, when the requests are
>>>> not ignored, are pointers to lists of ill-describing jiras and some
>> pending
>>>> user facing doc update.
>>>> 
>>>> For other features, mob or region server groups, I know that they have
>> been
>>>> running at scale in production for as much as a year and more. I have
>> some
>>>> confidence these items basically work.  For backup/restore I have no
>> such
>>>> sense even after spending time in review and trying to use the feature.
>>>> 
>>>> As release manager, I have say over what makes it into a release.
>> Unless
>>>> the work is done to convince me that backup/restore is more than a lump
>> of
>>>> code and a few unit tests that can pass on some fellows laptop, I am
>> going
>>>> to kick it out of branch-2.  Let the feature harden more in master
>> branch
>>>> before it ships in a release.
>>>> 
>>>> S
>>>> 
>>>> On Sep 8, 2017 10:59 PM, "Vladimir Rodionov" <vl...@gmail.com>
>>>> wrote:
>>>> 
>>>>>>> Have I grasped the state of things correctly, Vlad?
>>>>> 
>>>>> Josh, the only thing which is still pending is doc update. All other
>>>>> features are good to have but not a blockers for 2.0 release.
>>>>> 
>>>>> -Vlad
>>>>> 
>>>>> On Fri, Sep 8, 2017 at 10:42 PM, Vladimir Rodionov <
>>>> vladrodionov@gmail.com
>>>>>> 
>>>>> wrote:
>>>>> 
>>>>>>>> What testing and at what
>>>>>>>> scale has testing been done?
>>>>>> 
>>>>>> Do we have have that for other features?
>>>>>> 
>>>>>> 
>>>>>> On Fri, Sep 8, 2017 at 10:41 PM, Vladimir Rodionov <
>>>>> vladrodionov@gmail.com
>>>>>>> wrote:
>>>>>> 
>>>>>>>>> It asks: "How do I figure what of backup/restore feature is going
>>>> to
>>>>>>> be in
>>>>>>>>> hbase-2.0.0?
>>>>>>> 
>>>>>>> Hmm, wait for doc update.
>>>>>>> 
>>>>>>> 
>>>>>>>> On Fri, Sep 8, 2017 at 2:39 PM, Stack <st...@duboce.net> wrote:
>>>>>>>> 
>>>>>>>> HBASE-14414 is a JIRA with a list of random seeming issues w/
>>>>>>>> non-descript
>>>>>>>> summaries: "Add nonce support to TableBackupProcedure, BackupID must
>>>>>>>> include backup set name, ...". The last comment in that issue is
>> from
>>>>>>>> July.
>>>>>>>> It asks: "How do I figure what of backup/restore feature is going to
>>>> be
>>>>>>>> in
>>>>>>>> hbase-2.0.0? Thanks Vladimir Rodionov
>>>>>>>> <https://issues.apache.org/jira/secure/ViewProfile.jspa?
>>>> name=vrodionov
>>>>>>>>> ."
>>>>>>>> to which there is no answer.  Doc update is TODO.
>>>>>>>> 
>>>>>>>> Where is the summary of the capability in hbase-2? What testing and
>>>> at
>>>>>>>> what
>>>>>>>> scale has testing been done? Is this 'stable or experimental'? If I
>>>>> can't
>>>>>>>> get basic info on this feature though I ask repeatedly, what hope
>>>> does
>>>>>>>> the
>>>>>>>> poor old operator have?
>>>>>>>> 
>>>>>>>> St.Ack
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On Fri, Sep 8, 2017 at 1:59 PM, Vladimir Rodionov <
>>>>>>>> vladrodionov@gmail.com>
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> HBASE-14414
>>>>>>>>> 
>>>>>>>>>> On Fri, Sep 8, 2017 at 1:14 PM, Stack <st...@duboce.net> wrote:
>>>>>>>>>> 
>>>>>>>>>> Where do I go to get the current status of this feature? Looking
>>>> in
>>>>>>>> JIRA
>>>>>>>>> I
>>>>>>>>>> see loads of issues open against backup including some against
>>>>>>>>> hbase-2.0.0
>>>>>>>>>> and no progress being made that I can discern.
>>>>>>>>>> 
>>>>>>>>>> Thanks,
>>>>>>>>>> S
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>> On Wed, Nov 23, 2016 at 8:52 AM, Stack <st...@duboce.net> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> On Tue, Nov 22, 2016 at 6:48 PM, Stack <st...@duboce.net>
>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> On Tue, Nov 22, 2016 at 3:17 PM, Vladimir Rodionov <
>>>>>>>>>>>> vladrodionov@gmail.com> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>>>> and/or he answered most of the review feedback
>>>>>>>>>>>>> 
>>>>>>>>>>>>> No, questions are still open, but I do not see any blockers
>>>> and
>>>>>>>> we
>>>>>>>>> have
>>>>>>>>>>>>> HBASE-16940 to address these questions.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>> Agree. No blockers but stuff that should be dealt with (No one
>>>>>>>> will
>>>>>>>>> pay
>>>>>>>>>>>> me any attention once merge goes in -- smile).
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>> Let me clarify the above. I want review addressed before merge
>>>>>>>> happens.
>>>>>>>>>>> Sorry if any confusion.
>>>>>>>>>>> St.Ack
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>> St.Ack
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>>> On Tue, Nov 22, 2016 at 3:04 PM, Devaraj Das <
>>>>>>>> ddas@hortonworks.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Hi Stack, hats off to you for spending so much time on
>>>> this!
>>>>>>>>> Thanks!
>>>>>>>>>>>>> From
>>>>>>>>>>>>>> my understanding, Vlad has raised follow-up jiras for the
>>>>>>>> issues
>>>>>>>>> you
>>>>>>>>>>>>>> raised, and/or he answered most of the review feedback. So,
>>>>> do
>>>>>>>> you
>>>>>>>>>>>>> think we
>>>>>>>>>>>>>> could do a merge vote now?
>>>>>>>>>>>>>> Devaraj.
>>>>>>>>>>>>>> ________________________________________
>>>>>>>>>>>>>> From: Vladimir Rodionov <vl...@gmail.com>
>>>>>>>>>>>>>> Sent: Monday, November 21, 2016 8:34 PM
>>>>>>>>>>>>>> To: dev@hbase.apache.org
>>>>>>>>>>>>>> Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch
>>>>>>>>> HBASE-7912
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> I have spent a good bit of time reviewing and testing
>>>> this
>>>>>>>>>> feature.
>>>>>>>>>>>>> I
>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>> like my review and concerns addressed and I'd like it to
>>>>> be
>>>>>>>>> clear
>>>>>>>>>>>>> how;
>>>>>>>>>>>>>>>> either explicit follow-on issues, pointers to where in
>>>> the
>>>>>>>> patch
>>>>>>>>>> or
>>>>>>>>>>>>> doc
>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>> remarks have been catered to, etc. Until then, I am
>>>>> against
>>>>>>>>>> commit.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Stack, mega patch review comments will be addressed in the
>>>>>>>>> dedicated
>>>>>>>>>>>>> JIRA:
>>>>>>>>>>>>>> HBASE-16940
>>>>>>>>>>>>>> I have open several other JIRAs to address your other
>>>>> comments
>>>>>>>> (not
>>>>>>>>>> on
>>>>>>>>>>>>>> review board).
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Details are here (end of the thread):
>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/HBASE-14123
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Let me know what else should we do to move merge forward.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> -Vlad
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Fri, Nov 18, 2016 at 4:54 PM, Stack <st...@duboce.net>
>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On Fri, Nov 18, 2016 at 3:53 PM, Ted Yu <
>>>>> yuzhihong@gmail.com
>>>>>>>>> 
>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Thanks, Matteo.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> bq. restore is not clear if given an incremental id it
>>>>>>>> will do
>>>>>>>>>> the
>>>>>>>>>>>>> full
>>>>>>>>>>>>>>>> restore from full up to that point or if i need to
>>>> apply
>>>>>>>>> manually
>>>>>>>>>>>>>>>> everything
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> The restore takes into consideration of the dependent
>>>>>>>>> backup(s).
>>>>>>>>>>>>>>>> So there is no need to apply preceding backup(s)
>>>>> manually.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> I ask this question on the issue. It is not clear from
>>>> the
>>>>>>>> usage
>>>>>>>>> or
>>>>>>>>>>>>> doc
>>>>>>>>>>>>>> how
>>>>>>>>>>>>>>> to run a restore from incremental. Can you fix in doc and
>>>>>>>> usage
>>>>>>>>> how
>>>>>>>>>>>>> so I
>>>>>>>>>>>>>>> can be clear and try it. Currently I am stuck verifying a
>>>>>>>> round
>>>>>>>>>> trip
>>>>>>>>>>>>>> backup
>>>>>>>>>>>>>>> restore made of incrementals.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> S
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On Fri, Nov 18, 2016 at 3:48 PM, Matteo Bertozzi <
>>>>>>>>>>>>>>> theo.bertozzi@gmail.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> I did one last pass to the mega patch. I don't see
>>>>>>>> anything
>>>>>>>>>> major
>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>> should block the merge.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> - most of the code is isolated in the backup package
>>>>>>>>>>>>>>>>> - all the backup code is client side
>>>>>>>>>>>>>>>>> - there are few changes to the server side, mainly
>>>> for
>>>>>>>>>> cleaners,
>>>>>>>>>>>>> wal
>>>>>>>>>>>>>>>>> rolling and similar (which is ok)
>>>>>>>>>>>>>>>>> - there is a good number of tests, and an integration
>>>>>>>> test
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> the code seems to have still some left overs from the
>>>>> old
>>>>>>>>>>>>>>> implementation,
>>>>>>>>>>>>>>>>> and some stuff needs a cleanup. but I don't think
>>>> this
>>>>>>>> should
>>>>>>>>>> be
>>>>>>>>>>>>> used
>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>> argument to block the merge. I think the guys will
>>>> keep
>>>>>>>>> working
>>>>>>>>>>>>> on
>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> they may also get help of others once the patch is in
>>>>>>>> master.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> I still have my concerns about the current
>>>> limitations,
>>>>>>>> but
>>>>>>>>>>>>> these are
>>>>>>>>>>>>>>>>> things already planned for phase 3, so some of this
>>>>>>>> stuff may
>>>>>>>>>>>>> even be
>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>> the final 2.0.
>>>>>>>>>>>>>>>>> but as long as we have a "current limitations"
>>>> section
>>>>>>>> in the
>>>>>>>>>>>>> user
>>>>>>>>>>>>>>> guide
>>>>>>>>>>>>>>>>> mentioning important stuff like the ones below, I'm
>>>> ok
>>>>>>>> with
>>>>>>>>> it.
>>>>>>>>>>>>>>>>> - if you write to the table with
>>>> Durability.SKIP_WALS
>>>>>>>> your
>>>>>>>>>> data
>>>>>>>>>>>>> will
>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>> be in the incremental-backup
>>>>>>>>>>>>>>>>> - if you bulkload files that data will not be in the
>>>>>>>>>> incremental
>>>>>>>>>>>>>>> backup
>>>>>>>>>>>>>>>>> (HBASE-14417)
>>>>>>>>>>>>>>>>> - the incremental backup will not only contains the
>>>>>>>> data of
>>>>>>>>>> the
>>>>>>>>>>>>>> table
>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>> specified but also the regions from other tables that
>>>>>>>> are on
>>>>>>>>>> the
>>>>>>>>>>>>> same
>>>>>>>>>>>>>>> set
>>>>>>>>>>>>>>>>> of RSs (HBASE-14141) ...maybe a note about security
>>>>>>>> around
>>>>>>>>> this
>>>>>>>>>>>>> topic
>>>>>>>>>>>>>>>>> - the incremental backup will not contains just the
>>>>>>>> "latest
>>>>>>>>>> row"
>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>> backup A and B, but it will also contains all the
>>>>> updates
>>>>>>>>>>>>> occurred in
>>>>>>>>>>>>>>>>> between. but the restore does not allow you to
>>>> restore
>>>>>>>> up to
>>>>>>>>> a
>>>>>>>>>>>>>> certain
>>>>>>>>>>>>>>>>> point in time, the restore will always be up to the
>>>>>>>> "latest
>>>>>>>>>>>>> backup
>>>>>>>>>>>>>>>> point".
>>>>>>>>>>>>>>>>> - you should limit the number of "incremental" up
>>>> to N
>>>>>>>> (or
>>>>>>>>>> maybe
>>>>>>>>>>>>>>> SIZE),
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> avoid replay time becoming the bottleneck.
>>>>> (HBASE-14135)
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> I'll be ok even with the above not being in the final
>>>>>>>> 2.0,
>>>>>>>>>>>>>>>>> but i'd like to see as blocker for the final 2.0 (not
>>>>> the
>>>>>>>>>> merge)
>>>>>>>>>>>>>>>>> - the backup code moved in an hbase-backup module
>>>>>>>>>>>>>>>>> - and some more work around tools, especially to try
>>>>> to
>>>>>>>>> unify
>>>>>>>>>>>>> and
>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>> simple the backup experience (simple example: in some
>>>>>>>> case
>>>>>>>>>> there
>>>>>>>>>>>>> is a
>>>>>>>>>>>>>>>>> backup_id argument in others a backupId argument. or
>>>>>>>> things
>>>>>>>>>>>>> like..
>>>>>>>>>>>>>>>> restore
>>>>>>>>>>>>>>>>> is not clear if given an incremental id it will do
>>>> the
>>>>>>>> full
>>>>>>>>>>>>> restore
>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>> full up to that point or if i need to apply manually
>>>>>>>>>> everything).
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> in conclusion, I think we can open a merge vote. I'll
>>>>> be
>>>>>>>> +1
>>>>>>>>> on
>>>>>>>>>>>>> it,
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>> think we should try to reject -1 with just a "code
>>>>>>>> cleanup"
>>>>>>>>>>>>>> motivation,
>>>>>>>>>>>>>>>>> since there will still be work going on on the code
>>>>>>>> after the
>>>>>>>>>>>>> merge.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Matteo
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> On Sun, Nov 6, 2016 at 10:54 PM, Devaraj Das <
>>>>>>>>>>>>> ddas@hortonworks.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Stack and others, anything else on the patch? Merge
>>>>> to
>>>>>>>>> master
>>>>>>>>>>>>> now?
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>> 

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Vladimir Rodionov <vl...@gmail.com>]

>> but the impression we have is it is unfinished and untested.
To make a conclusion that "feature is not finished and tested"  you have
had to test it at least.
Andrew, If you have discovered issues, why wouldn't you open bug JIRAs?

-Vlad

On Sat, Sep 9, 2017 at 4:40 PM, Andrew Purtell <an...@gmail.com>
wrote:

> For what it's worth, I think AMv2 is the main reason to have a 2.0 in the
> first place, so I would both agree it needs a lot more testing and yet I
> would want us to have a 2.0 release as the vehicle for getting that to
> happen. For other features without testing from a number of parties or at
> scale the value proposition is less clear and it's fine by me for the RM to
> set them aside for future releases.
>
> Also, I can relay that there is some interest where I work in utilizing
> HBASE-7912 but the impression we have is it is unfinished and untested. So
> for now we are ignoring it and continuing with home grown solutions. Part
> of the problem is fault tolerance was left to the last phase(s) and yet it
> is an essential property for adoption for serious work. The best way to
> resolve this IMHO is for the developers of this feature to complete those
> unfinished JIRAs, especially concerning resilience to failures.
>
>
> > On Sep 9, 2017, at 4:11 PM, Vladimir Rodionov <vl...@gmail.com>
> wrote:
> >
> > Hmm, the next on your list (of kicked out from branch v2) should be AM
> v2 I
> > presume?
> >
> > -Vlad
> >
> >> On Sat, Sep 9, 2017 at 4:04 PM, stack <sa...@gmail.com> wrote:
> >>
> >> In spite of repeated requests for eng summary of state of this feature
> --
> >> summary of what is in 2.0, what is not, what the capabilities are, how
> well
> >> it has been tested and at what scale -- all I get, when the requests are
> >> not ignored, are pointers to lists of ill-describing jiras and some
> pending
> >> user facing doc update.
> >>
> >> For other features, mob or region server groups, I know that they have
> been
> >> running at scale in production for as much as a year and more. I have
> some
> >> confidence these items basically work.  For backup/restore I have no
> such
> >> sense even after spending time in review and trying to use the feature.
> >>
> >> As release manager, I have say over what makes it into a release.
> Unless
> >> the work is done to convince me that backup/restore is more than a lump
> of
> >> code and a few unit tests that can pass on some fellows laptop, I am
> going
> >> to kick it out of branch-2.  Let the feature harden more in master
> branch
> >> before it ships in a release.
> >>
> >> S
> >>
> >> On Sep 8, 2017 10:59 PM, "Vladimir Rodionov" <vl...@gmail.com>
> >> wrote:
> >>
> >>>>> Have I grasped the state of things correctly, Vlad?
> >>>
> >>> Josh, the only thing which is still pending is doc update. All other
> >>> features are good to have but not a blockers for 2.0 release.
> >>>
> >>> -Vlad
> >>>
> >>> On Fri, Sep 8, 2017 at 10:42 PM, Vladimir Rodionov <
> >> vladrodionov@gmail.com
> >>>>
> >>> wrote:
> >>>
> >>>>>> What testing and at what
> >>>>>> scale has testing been done?
> >>>>
> >>>> Do we have have that for other features?
> >>>>
> >>>>
> >>>> On Fri, Sep 8, 2017 at 10:41 PM, Vladimir Rodionov <
> >>> vladrodionov@gmail.com
> >>>>> wrote:
> >>>>
> >>>>>>> It asks: "How do I figure what of backup/restore feature is going
> >> to
> >>>>> be in
> >>>>>>> hbase-2.0.0?
> >>>>>
> >>>>> Hmm, wait for doc update.
> >>>>>
> >>>>>
> >>>>>> On Fri, Sep 8, 2017 at 2:39 PM, Stack <st...@duboce.net> wrote:
> >>>>>>
> >>>>>> HBASE-14414 is a JIRA with a list of random seeming issues w/
> >>>>>> non-descript
> >>>>>> summaries: "Add nonce support to TableBackupProcedure, BackupID must
> >>>>>> include backup set name, ...". The last comment in that issue is
> from
> >>>>>> July.
> >>>>>> It asks: "How do I figure what of backup/restore feature is going to
> >> be
> >>>>>> in
> >>>>>> hbase-2.0.0? Thanks Vladimir Rodionov
> >>>>>> <https://issues.apache.org/jira/secure/ViewProfile.jspa?
> >> name=vrodionov
> >>>>>>> ."
> >>>>>> to which there is no answer.  Doc update is TODO.
> >>>>>>
> >>>>>> Where is the summary of the capability in hbase-2? What testing and
> >> at
> >>>>>> what
> >>>>>> scale has testing been done? Is this 'stable or experimental'? If I
> >>> can't
> >>>>>> get basic info on this feature though I ask repeatedly, what hope
> >> does
> >>>>>> the
> >>>>>> poor old operator have?
> >>>>>>
> >>>>>> St.Ack
> >>>>>>
> >>>>>>
> >>>>>> On Fri, Sep 8, 2017 at 1:59 PM, Vladimir Rodionov <
> >>>>>> vladrodionov@gmail.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> HBASE-14414
> >>>>>>>
> >>>>>>>> On Fri, Sep 8, 2017 at 1:14 PM, Stack <st...@duboce.net> wrote:
> >>>>>>>>
> >>>>>>>> Where do I go to get the current status of this feature? Looking
> >> in
> >>>>>> JIRA
> >>>>>>> I
> >>>>>>>> see loads of issues open against backup including some against
> >>>>>>> hbase-2.0.0
> >>>>>>>> and no progress being made that I can discern.
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> S
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> On Wed, Nov 23, 2016 at 8:52 AM, Stack <st...@duboce.net> wrote:
> >>>>>>>>>
> >>>>>>>>> On Tue, Nov 22, 2016 at 6:48 PM, Stack <st...@duboce.net>
> >> wrote:
> >>>>>>>>>
> >>>>>>>>>> On Tue, Nov 22, 2016 at 3:17 PM, Vladimir Rodionov <
> >>>>>>>>>> vladrodionov@gmail.com> wrote:
> >>>>>>>>>>
> >>>>>>>>>>>>> and/or he answered most of the review feedback
> >>>>>>>>>>>
> >>>>>>>>>>> No, questions are still open, but I do not see any blockers
> >> and
> >>>>>> we
> >>>>>>> have
> >>>>>>>>>>> HBASE-16940 to address these questions.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>> Agree. No blockers but stuff that should be dealt with (No one
> >>>>>> will
> >>>>>>> pay
> >>>>>>>>>> me any attention once merge goes in -- smile).
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>> Let me clarify the above. I want review addressed before merge
> >>>>>> happens.
> >>>>>>>>> Sorry if any confusion.
> >>>>>>>>> St.Ack
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>> St.Ack
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> On Tue, Nov 22, 2016 at 3:04 PM, Devaraj Das <
> >>>>>> ddas@hortonworks.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Hi Stack, hats off to you for spending so much time on
> >> this!
> >>>>>>> Thanks!
> >>>>>>>>>>> From
> >>>>>>>>>>>> my understanding, Vlad has raised follow-up jiras for the
> >>>>>> issues
> >>>>>>> you
> >>>>>>>>>>>> raised, and/or he answered most of the review feedback. So,
> >>> do
> >>>>>> you
> >>>>>>>>>>> think we
> >>>>>>>>>>>> could do a merge vote now?
> >>>>>>>>>>>> Devaraj.
> >>>>>>>>>>>> ________________________________________
> >>>>>>>>>>>> From: Vladimir Rodionov <vl...@gmail.com>
> >>>>>>>>>>>> Sent: Monday, November 21, 2016 8:34 PM
> >>>>>>>>>>>> To: dev@hbase.apache.org
> >>>>>>>>>>>> Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch
> >>>>>>> HBASE-7912
> >>>>>>>>>>>>
> >>>>>>>>>>>>>> I have spent a good bit of time reviewing and testing
> >> this
> >>>>>>>> feature.
> >>>>>>>>>>> I
> >>>>>>>>>>>> would
> >>>>>>>>>>>>>> like my review and concerns addressed and I'd like it to
> >>> be
> >>>>>>> clear
> >>>>>>>>>>> how;
> >>>>>>>>>>>>>> either explicit follow-on issues, pointers to where in
> >> the
> >>>>>> patch
> >>>>>>>> or
> >>>>>>>>>>> doc
> >>>>>>>>>>>> my
> >>>>>>>>>>>>>> remarks have been catered to, etc. Until then, I am
> >>> against
> >>>>>>>> commit.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Stack, mega patch review comments will be addressed in the
> >>>>>>> dedicated
> >>>>>>>>>>> JIRA:
> >>>>>>>>>>>> HBASE-16940
> >>>>>>>>>>>> I have open several other JIRAs to address your other
> >>> comments
> >>>>>> (not
> >>>>>>>> on
> >>>>>>>>>>>> review board).
> >>>>>>>>>>>>
> >>>>>>>>>>>> Details are here (end of the thread):
> >>>>>>>>>>>> https://issues.apache.org/jira/browse/HBASE-14123
> >>>>>>>>>>>>
> >>>>>>>>>>>> Let me know what else should we do to move merge forward.
> >>>>>>>>>>>>
> >>>>>>>>>>>> -Vlad
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Fri, Nov 18, 2016 at 4:54 PM, Stack <st...@duboce.net>
> >>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> On Fri, Nov 18, 2016 at 3:53 PM, Ted Yu <
> >>> yuzhihong@gmail.com
> >>>>>>>
> >>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks, Matteo.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> bq. restore is not clear if given an incremental id it
> >>>>>> will do
> >>>>>>>> the
> >>>>>>>>>>> full
> >>>>>>>>>>>>>> restore from full up to that point or if i need to
> >> apply
> >>>>>>> manually
> >>>>>>>>>>>>>> everything
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> The restore takes into consideration of the dependent
> >>>>>>> backup(s).
> >>>>>>>>>>>>>> So there is no need to apply preceding backup(s)
> >>> manually.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> I ask this question on the issue. It is not clear from
> >> the
> >>>>>> usage
> >>>>>>> or
> >>>>>>>>>>> doc
> >>>>>>>>>>>> how
> >>>>>>>>>>>>> to run a restore from incremental. Can you fix in doc and
> >>>>>> usage
> >>>>>>> how
> >>>>>>>>>>> so I
> >>>>>>>>>>>>> can be clear and try it. Currently I am stuck verifying a
> >>>>>> round
> >>>>>>>> trip
> >>>>>>>>>>>> backup
> >>>>>>>>>>>>> restore made of incrementals.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>> S
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Fri, Nov 18, 2016 at 3:48 PM, Matteo Bertozzi <
> >>>>>>>>>>>>> theo.bertozzi@gmail.com>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I did one last pass to the mega patch. I don't see
> >>>>>> anything
> >>>>>>>> major
> >>>>>>>>>>>> that
> >>>>>>>>>>>>>>> should block the merge.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> - most of the code is isolated in the backup package
> >>>>>>>>>>>>>>> - all the backup code is client side
> >>>>>>>>>>>>>>> - there are few changes to the server side, mainly
> >> for
> >>>>>>>> cleaners,
> >>>>>>>>>>> wal
> >>>>>>>>>>>>>>> rolling and similar (which is ok)
> >>>>>>>>>>>>>>> - there is a good number of tests, and an integration
> >>>>>> test
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> the code seems to have still some left overs from the
> >>> old
> >>>>>>>>>>>>> implementation,
> >>>>>>>>>>>>>>> and some stuff needs a cleanup. but I don't think
> >> this
> >>>>>> should
> >>>>>>>> be
> >>>>>>>>>>> used
> >>>>>>>>>>>>> as
> >>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>> argument to block the merge. I think the guys will
> >> keep
> >>>>>>> working
> >>>>>>>>>>> on
> >>>>>>>>>>>> this
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>> they may also get help of others once the patch is in
> >>>>>> master.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I still have my concerns about the current
> >> limitations,
> >>>>>> but
> >>>>>>>>>>> these are
> >>>>>>>>>>>>>>> things already planned for phase 3, so some of this
> >>>>>> stuff may
> >>>>>>>>>>> even be
> >>>>>>>>>>>>> in
> >>>>>>>>>>>>>>> the final 2.0.
> >>>>>>>>>>>>>>> but as long as we have a "current limitations"
> >> section
> >>>>>> in the
> >>>>>>>>>>> user
> >>>>>>>>>>>>> guide
> >>>>>>>>>>>>>>> mentioning important stuff like the ones below, I'm
> >> ok
> >>>>>> with
> >>>>>>> it.
> >>>>>>>>>>>>>>> - if you write to the table with
> >> Durability.SKIP_WALS
> >>>>>> your
> >>>>>>>> data
> >>>>>>>>>>> will
> >>>>>>>>>>>>> not
> >>>>>>>>>>>>>>> be in the incremental-backup
> >>>>>>>>>>>>>>> - if you bulkload files that data will not be in the
> >>>>>>>> incremental
> >>>>>>>>>>>>> backup
> >>>>>>>>>>>>>>> (HBASE-14417)
> >>>>>>>>>>>>>>> - the incremental backup will not only contains the
> >>>>>> data of
> >>>>>>>> the
> >>>>>>>>>>>> table
> >>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>> specified but also the regions from other tables that
> >>>>>> are on
> >>>>>>>> the
> >>>>>>>>>>> same
> >>>>>>>>>>>>> set
> >>>>>>>>>>>>>>> of RSs (HBASE-14141) ...maybe a note about security
> >>>>>> around
> >>>>>>> this
> >>>>>>>>>>> topic
> >>>>>>>>>>>>>>> - the incremental backup will not contains just the
> >>>>>> "latest
> >>>>>>>> row"
> >>>>>>>>>>>>> between
> >>>>>>>>>>>>>>> backup A and B, but it will also contains all the
> >>> updates
> >>>>>>>>>>> occurred in
> >>>>>>>>>>>>>>> between. but the restore does not allow you to
> >> restore
> >>>>>> up to
> >>>>>>> a
> >>>>>>>>>>>> certain
> >>>>>>>>>>>>>>> point in time, the restore will always be up to the
> >>>>>> "latest
> >>>>>>>>>>> backup
> >>>>>>>>>>>>>> point".
> >>>>>>>>>>>>>>> - you should limit the number of "incremental" up
> >> to N
> >>>>>> (or
> >>>>>>>> maybe
> >>>>>>>>>>>>> SIZE),
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>> avoid replay time becoming the bottleneck.
> >>> (HBASE-14135)
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I'll be ok even with the above not being in the final
> >>>>>> 2.0,
> >>>>>>>>>>>>>>> but i'd like to see as blocker for the final 2.0 (not
> >>> the
> >>>>>>>> merge)
> >>>>>>>>>>>>>>> - the backup code moved in an hbase-backup module
> >>>>>>>>>>>>>>> - and some more work around tools, especially to try
> >>> to
> >>>>>>> unify
> >>>>>>>>>>> and
> >>>>>>>>>>>> make
> >>>>>>>>>>>>>>> simple the backup experience (simple example: in some
> >>>>>> case
> >>>>>>>> there
> >>>>>>>>>>> is a
> >>>>>>>>>>>>>>> backup_id argument in others a backupId argument. or
> >>>>>> things
> >>>>>>>>>>> like..
> >>>>>>>>>>>>>> restore
> >>>>>>>>>>>>>>> is not clear if given an incremental id it will do
> >> the
> >>>>>> full
> >>>>>>>>>>> restore
> >>>>>>>>>>>>> from
> >>>>>>>>>>>>>>> full up to that point or if i need to apply manually
> >>>>>>>> everything).
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> in conclusion, I think we can open a merge vote. I'll
> >>> be
> >>>>>> +1
> >>>>>>> on
> >>>>>>>>>>> it,
> >>>>>>>>>>>> and
> >>>>>>>>>>>>> I
> >>>>>>>>>>>>>>> think we should try to reject -1 with just a "code
> >>>>>> cleanup"
> >>>>>>>>>>>> motivation,
> >>>>>>>>>>>>>>> since there will still be work going on on the code
> >>>>>> after the
> >>>>>>>>>>> merge.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Matteo
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Sun, Nov 6, 2016 at 10:54 PM, Devaraj Das <
> >>>>>>>>>>> ddas@hortonworks.com>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Stack and others, anything else on the patch? Merge
> >>> to
> >>>>>>> master
> >>>>>>>>>>> now?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Andrew Purtell <an...@gmail.com>]

For what it's worth, I think AMv2 is the main reason to have a 2.0 in the first place, so I would both agree it needs a lot more testing and yet I would want us to have a 2.0 release as the vehicle for getting that to happen. For other features without testing from a number of parties or at scale the value proposition is less clear and it's fine by me for the RM to set them aside for future releases. 

Also, I can relay that there is some interest where I work in utilizing HBASE-7912 but the impression we have is it is unfinished and untested. So for now we are ignoring it and continuing with home grown solutions. Part of the problem is fault tolerance was left to the last phase(s) and yet it is an essential property for adoption for serious work. The best way to resolve this IMHO is for the developers of this feature to complete those unfinished JIRAs, especially concerning resilience to failures. 


> On Sep 9, 2017, at 4:11 PM, Vladimir Rodionov <vl...@gmail.com> wrote:
> 
> Hmm, the next on your list (of kicked out from branch v2) should be AM v2 I
> presume?
> 
> -Vlad
> 
>> On Sat, Sep 9, 2017 at 4:04 PM, stack <sa...@gmail.com> wrote:
>> 
>> In spite of repeated requests for eng summary of state of this feature --
>> summary of what is in 2.0, what is not, what the capabilities are, how well
>> it has been tested and at what scale -- all I get, when the requests are
>> not ignored, are pointers to lists of ill-describing jiras and some pending
>> user facing doc update.
>> 
>> For other features, mob or region server groups, I know that they have been
>> running at scale in production for as much as a year and more. I have some
>> confidence these items basically work.  For backup/restore I have no such
>> sense even after spending time in review and trying to use the feature.
>> 
>> As release manager, I have say over what makes it into a release.  Unless
>> the work is done to convince me that backup/restore is more than a lump of
>> code and a few unit tests that can pass on some fellows laptop, I am going
>> to kick it out of branch-2.  Let the feature harden more in master branch
>> before it ships in a release.
>> 
>> S
>> 
>> On Sep 8, 2017 10:59 PM, "Vladimir Rodionov" <vl...@gmail.com>
>> wrote:
>> 
>>>>> Have I grasped the state of things correctly, Vlad?
>>> 
>>> Josh, the only thing which is still pending is doc update. All other
>>> features are good to have but not a blockers for 2.0 release.
>>> 
>>> -Vlad
>>> 
>>> On Fri, Sep 8, 2017 at 10:42 PM, Vladimir Rodionov <
>> vladrodionov@gmail.com
>>>> 
>>> wrote:
>>> 
>>>>>> What testing and at what
>>>>>> scale has testing been done?
>>>> 
>>>> Do we have have that for other features?
>>>> 
>>>> 
>>>> On Fri, Sep 8, 2017 at 10:41 PM, Vladimir Rodionov <
>>> vladrodionov@gmail.com
>>>>> wrote:
>>>> 
>>>>>>> It asks: "How do I figure what of backup/restore feature is going
>> to
>>>>> be in
>>>>>>> hbase-2.0.0?
>>>>> 
>>>>> Hmm, wait for doc update.
>>>>> 
>>>>> 
>>>>>> On Fri, Sep 8, 2017 at 2:39 PM, Stack <st...@duboce.net> wrote:
>>>>>> 
>>>>>> HBASE-14414 is a JIRA with a list of random seeming issues w/
>>>>>> non-descript
>>>>>> summaries: "Add nonce support to TableBackupProcedure, BackupID must
>>>>>> include backup set name, ...". The last comment in that issue is from
>>>>>> July.
>>>>>> It asks: "How do I figure what of backup/restore feature is going to
>> be
>>>>>> in
>>>>>> hbase-2.0.0? Thanks Vladimir Rodionov
>>>>>> <https://issues.apache.org/jira/secure/ViewProfile.jspa?
>> name=vrodionov
>>>>>>> ."
>>>>>> to which there is no answer.  Doc update is TODO.
>>>>>> 
>>>>>> Where is the summary of the capability in hbase-2? What testing and
>> at
>>>>>> what
>>>>>> scale has testing been done? Is this 'stable or experimental'? If I
>>> can't
>>>>>> get basic info on this feature though I ask repeatedly, what hope
>> does
>>>>>> the
>>>>>> poor old operator have?
>>>>>> 
>>>>>> St.Ack
>>>>>> 
>>>>>> 
>>>>>> On Fri, Sep 8, 2017 at 1:59 PM, Vladimir Rodionov <
>>>>>> vladrodionov@gmail.com>
>>>>>> wrote:
>>>>>> 
>>>>>>> HBASE-14414
>>>>>>> 
>>>>>>>> On Fri, Sep 8, 2017 at 1:14 PM, Stack <st...@duboce.net> wrote:
>>>>>>>> 
>>>>>>>> Where do I go to get the current status of this feature? Looking
>> in
>>>>>> JIRA
>>>>>>> I
>>>>>>>> see loads of issues open against backup including some against
>>>>>>> hbase-2.0.0
>>>>>>>> and no progress being made that I can discern.
>>>>>>>> 
>>>>>>>> Thanks,
>>>>>>>> S
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>>> On Wed, Nov 23, 2016 at 8:52 AM, Stack <st...@duboce.net> wrote:
>>>>>>>>> 
>>>>>>>>> On Tue, Nov 22, 2016 at 6:48 PM, Stack <st...@duboce.net>
>> wrote:
>>>>>>>>> 
>>>>>>>>>> On Tue, Nov 22, 2016 at 3:17 PM, Vladimir Rodionov <
>>>>>>>>>> vladrodionov@gmail.com> wrote:
>>>>>>>>>> 
>>>>>>>>>>>>> and/or he answered most of the review feedback
>>>>>>>>>>> 
>>>>>>>>>>> No, questions are still open, but I do not see any blockers
>> and
>>>>>> we
>>>>>>> have
>>>>>>>>>>> HBASE-16940 to address these questions.
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> Agree. No blockers but stuff that should be dealt with (No one
>>>>>> will
>>>>>>> pay
>>>>>>>>>> me any attention once merge goes in -- smile).
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> Let me clarify the above. I want review addressed before merge
>>>>>> happens.
>>>>>>>>> Sorry if any confusion.
>>>>>>>>> St.Ack
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>> St.Ack
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>> On Tue, Nov 22, 2016 at 3:04 PM, Devaraj Das <
>>>>>> ddas@hortonworks.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> Hi Stack, hats off to you for spending so much time on
>> this!
>>>>>>> Thanks!
>>>>>>>>>>> From
>>>>>>>>>>>> my understanding, Vlad has raised follow-up jiras for the
>>>>>> issues
>>>>>>> you
>>>>>>>>>>>> raised, and/or he answered most of the review feedback. So,
>>> do
>>>>>> you
>>>>>>>>>>> think we
>>>>>>>>>>>> could do a merge vote now?
>>>>>>>>>>>> Devaraj.
>>>>>>>>>>>> ________________________________________
>>>>>>>>>>>> From: Vladimir Rodionov <vl...@gmail.com>
>>>>>>>>>>>> Sent: Monday, November 21, 2016 8:34 PM
>>>>>>>>>>>> To: dev@hbase.apache.org
>>>>>>>>>>>> Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch
>>>>>>> HBASE-7912
>>>>>>>>>>>> 
>>>>>>>>>>>>>> I have spent a good bit of time reviewing and testing
>> this
>>>>>>>> feature.
>>>>>>>>>>> I
>>>>>>>>>>>> would
>>>>>>>>>>>>>> like my review and concerns addressed and I'd like it to
>>> be
>>>>>>> clear
>>>>>>>>>>> how;
>>>>>>>>>>>>>> either explicit follow-on issues, pointers to where in
>> the
>>>>>> patch
>>>>>>>> or
>>>>>>>>>>> doc
>>>>>>>>>>>> my
>>>>>>>>>>>>>> remarks have been catered to, etc. Until then, I am
>>> against
>>>>>>>> commit.
>>>>>>>>>>>> 
>>>>>>>>>>>> Stack, mega patch review comments will be addressed in the
>>>>>>> dedicated
>>>>>>>>>>> JIRA:
>>>>>>>>>>>> HBASE-16940
>>>>>>>>>>>> I have open several other JIRAs to address your other
>>> comments
>>>>>> (not
>>>>>>>> on
>>>>>>>>>>>> review board).
>>>>>>>>>>>> 
>>>>>>>>>>>> Details are here (end of the thread):
>>>>>>>>>>>> https://issues.apache.org/jira/browse/HBASE-14123
>>>>>>>>>>>> 
>>>>>>>>>>>> Let me know what else should we do to move merge forward.
>>>>>>>>>>>> 
>>>>>>>>>>>> -Vlad
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> On Fri, Nov 18, 2016 at 4:54 PM, Stack <st...@duboce.net>
>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> On Fri, Nov 18, 2016 at 3:53 PM, Ted Yu <
>>> yuzhihong@gmail.com
>>>>>>> 
>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Thanks, Matteo.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> bq. restore is not clear if given an incremental id it
>>>>>> will do
>>>>>>>> the
>>>>>>>>>>> full
>>>>>>>>>>>>>> restore from full up to that point or if i need to
>> apply
>>>>>>> manually
>>>>>>>>>>>>>> everything
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> The restore takes into consideration of the dependent
>>>>>>> backup(s).
>>>>>>>>>>>>>> So there is no need to apply preceding backup(s)
>>> manually.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> I ask this question on the issue. It is not clear from
>> the
>>>>>> usage
>>>>>>> or
>>>>>>>>>>> doc
>>>>>>>>>>>> how
>>>>>>>>>>>>> to run a restore from incremental. Can you fix in doc and
>>>>>> usage
>>>>>>> how
>>>>>>>>>>> so I
>>>>>>>>>>>>> can be clear and try it. Currently I am stuck verifying a
>>>>>> round
>>>>>>>> trip
>>>>>>>>>>>> backup
>>>>>>>>>>>>> restore made of incrementals.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> S
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Fri, Nov 18, 2016 at 3:48 PM, Matteo Bertozzi <
>>>>>>>>>>>>> theo.bertozzi@gmail.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> I did one last pass to the mega patch. I don't see
>>>>>> anything
>>>>>>>> major
>>>>>>>>>>>> that
>>>>>>>>>>>>>>> should block the merge.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> - most of the code is isolated in the backup package
>>>>>>>>>>>>>>> - all the backup code is client side
>>>>>>>>>>>>>>> - there are few changes to the server side, mainly
>> for
>>>>>>>> cleaners,
>>>>>>>>>>> wal
>>>>>>>>>>>>>>> rolling and similar (which is ok)
>>>>>>>>>>>>>>> - there is a good number of tests, and an integration
>>>>>> test
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> the code seems to have still some left overs from the
>>> old
>>>>>>>>>>>>> implementation,
>>>>>>>>>>>>>>> and some stuff needs a cleanup. but I don't think
>> this
>>>>>> should
>>>>>>>> be
>>>>>>>>>>> used
>>>>>>>>>>>>> as
>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>> argument to block the merge. I think the guys will
>> keep
>>>>>>> working
>>>>>>>>>>> on
>>>>>>>>>>>> this
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>> they may also get help of others once the patch is in
>>>>>> master.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> I still have my concerns about the current
>> limitations,
>>>>>> but
>>>>>>>>>>> these are
>>>>>>>>>>>>>>> things already planned for phase 3, so some of this
>>>>>> stuff may
>>>>>>>>>>> even be
>>>>>>>>>>>>> in
>>>>>>>>>>>>>>> the final 2.0.
>>>>>>>>>>>>>>> but as long as we have a "current limitations"
>> section
>>>>>> in the
>>>>>>>>>>> user
>>>>>>>>>>>>> guide
>>>>>>>>>>>>>>> mentioning important stuff like the ones below, I'm
>> ok
>>>>>> with
>>>>>>> it.
>>>>>>>>>>>>>>> - if you write to the table with
>> Durability.SKIP_WALS
>>>>>> your
>>>>>>>> data
>>>>>>>>>>> will
>>>>>>>>>>>>> not
>>>>>>>>>>>>>>> be in the incremental-backup
>>>>>>>>>>>>>>> - if you bulkload files that data will not be in the
>>>>>>>> incremental
>>>>>>>>>>>>> backup
>>>>>>>>>>>>>>> (HBASE-14417)
>>>>>>>>>>>>>>> - the incremental backup will not only contains the
>>>>>> data of
>>>>>>>> the
>>>>>>>>>>>> table
>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>> specified but also the regions from other tables that
>>>>>> are on
>>>>>>>> the
>>>>>>>>>>> same
>>>>>>>>>>>>> set
>>>>>>>>>>>>>>> of RSs (HBASE-14141) ...maybe a note about security
>>>>>> around
>>>>>>> this
>>>>>>>>>>> topic
>>>>>>>>>>>>>>> - the incremental backup will not contains just the
>>>>>> "latest
>>>>>>>> row"
>>>>>>>>>>>>> between
>>>>>>>>>>>>>>> backup A and B, but it will also contains all the
>>> updates
>>>>>>>>>>> occurred in
>>>>>>>>>>>>>>> between. but the restore does not allow you to
>> restore
>>>>>> up to
>>>>>>> a
>>>>>>>>>>>> certain
>>>>>>>>>>>>>>> point in time, the restore will always be up to the
>>>>>> "latest
>>>>>>>>>>> backup
>>>>>>>>>>>>>> point".
>>>>>>>>>>>>>>> - you should limit the number of "incremental" up
>> to N
>>>>>> (or
>>>>>>>> maybe
>>>>>>>>>>>>> SIZE),
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> avoid replay time becoming the bottleneck.
>>> (HBASE-14135)
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> I'll be ok even with the above not being in the final
>>>>>> 2.0,
>>>>>>>>>>>>>>> but i'd like to see as blocker for the final 2.0 (not
>>> the
>>>>>>>> merge)
>>>>>>>>>>>>>>> - the backup code moved in an hbase-backup module
>>>>>>>>>>>>>>> - and some more work around tools, especially to try
>>> to
>>>>>>> unify
>>>>>>>>>>> and
>>>>>>>>>>>> make
>>>>>>>>>>>>>>> simple the backup experience (simple example: in some
>>>>>> case
>>>>>>>> there
>>>>>>>>>>> is a
>>>>>>>>>>>>>>> backup_id argument in others a backupId argument. or
>>>>>> things
>>>>>>>>>>> like..
>>>>>>>>>>>>>> restore
>>>>>>>>>>>>>>> is not clear if given an incremental id it will do
>> the
>>>>>> full
>>>>>>>>>>> restore
>>>>>>>>>>>>> from
>>>>>>>>>>>>>>> full up to that point or if i need to apply manually
>>>>>>>> everything).
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> in conclusion, I think we can open a merge vote. I'll
>>> be
>>>>>> +1
>>>>>>> on
>>>>>>>>>>> it,
>>>>>>>>>>>> and
>>>>>>>>>>>>> I
>>>>>>>>>>>>>>> think we should try to reject -1 with just a "code
>>>>>> cleanup"
>>>>>>>>>>>> motivation,
>>>>>>>>>>>>>>> since there will still be work going on on the code
>>>>>> after the
>>>>>>>>>>> merge.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Matteo
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On Sun, Nov 6, 2016 at 10:54 PM, Devaraj Das <
>>>>>>>>>>> ddas@hortonworks.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Stack and others, anything else on the patch? Merge
>>> to
>>>>>>> master
>>>>>>>>>>> now?
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>>> 
>>> 
>> 

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Vladimir Rodionov <vl...@gmail.com>]

Hmm, the next on your list (of kicked out from branch v2) should be AM v2 I
presume?

-Vlad

On Sat, Sep 9, 2017 at 4:04 PM, stack <sa...@gmail.com> wrote:

> In spite of repeated requests for eng summary of state of this feature --
> summary of what is in 2.0, what is not, what the capabilities are, how well
> it has been tested and at what scale -- all I get, when the requests are
> not ignored, are pointers to lists of ill-describing jiras and some pending
> user facing doc update.
>
> For other features, mob or region server groups, I know that they have been
> running at scale in production for as much as a year and more. I have some
> confidence these items basically work.  For backup/restore I have no such
> sense even after spending time in review and trying to use the feature.
>
> As release manager, I have say over what makes it into a release.  Unless
> the work is done to convince me that backup/restore is more than a lump of
> code and a few unit tests that can pass on some fellows laptop, I am going
> to kick it out of branch-2.  Let the feature harden more in master branch
> before it ships in a release.
>
> S
>
> On Sep 8, 2017 10:59 PM, "Vladimir Rodionov" <vl...@gmail.com>
> wrote:
>
> > >> Have I grasped the state of things correctly, Vlad?
> >
> > Josh, the only thing which is still pending is doc update. All other
> > features are good to have but not a blockers for 2.0 release.
> >
> > -Vlad
> >
> > On Fri, Sep 8, 2017 at 10:42 PM, Vladimir Rodionov <
> vladrodionov@gmail.com
> > >
> > wrote:
> >
> > > >> What testing and at what
> > > >> scale has testing been done?
> > >
> > > Do we have have that for other features?
> > >
> > >
> > > On Fri, Sep 8, 2017 at 10:41 PM, Vladimir Rodionov <
> > vladrodionov@gmail.com
> > > > wrote:
> > >
> > >> >> It asks: "How do I figure what of backup/restore feature is going
> to
> > >> be in
> > >> >>hbase-2.0.0?
> > >>
> > >> Hmm, wait for doc update.
> > >>
> > >>
> > >> On Fri, Sep 8, 2017 at 2:39 PM, Stack <st...@duboce.net> wrote:
> > >>
> > >>> HBASE-14414 is a JIRA with a list of random seeming issues w/
> > >>> non-descript
> > >>> summaries: "Add nonce support to TableBackupProcedure, BackupID must
> > >>> include backup set name, ...". The last comment in that issue is from
> > >>> July.
> > >>> It asks: "How do I figure what of backup/restore feature is going to
> be
> > >>> in
> > >>> hbase-2.0.0? Thanks Vladimir Rodionov
> > >>> <https://issues.apache.org/jira/secure/ViewProfile.jspa?
> name=vrodionov
> > >>> >."
> > >>> to which there is no answer.  Doc update is TODO.
> > >>>
> > >>> Where is the summary of the capability in hbase-2? What testing and
> at
> > >>> what
> > >>> scale has testing been done? Is this 'stable or experimental'? If I
> > can't
> > >>> get basic info on this feature though I ask repeatedly, what hope
> does
> > >>> the
> > >>> poor old operator have?
> > >>>
> > >>> St.Ack
> > >>>
> > >>>
> > >>> On Fri, Sep 8, 2017 at 1:59 PM, Vladimir Rodionov <
> > >>> vladrodionov@gmail.com>
> > >>> wrote:
> > >>>
> > >>> > HBASE-14414
> > >>> >
> > >>> > On Fri, Sep 8, 2017 at 1:14 PM, Stack <st...@duboce.net> wrote:
> > >>> >
> > >>> > > Where do I go to get the current status of this feature? Looking
> in
> > >>> JIRA
> > >>> > I
> > >>> > > see loads of issues open against backup including some against
> > >>> > hbase-2.0.0
> > >>> > > and no progress being made that I can discern.
> > >>> > >
> > >>> > > Thanks,
> > >>> > > S
> > >>> > >
> > >>> > >
> > >>> > >
> > >>> > > On Wed, Nov 23, 2016 at 8:52 AM, Stack <st...@duboce.net> wrote:
> > >>> > >
> > >>> > > > On Tue, Nov 22, 2016 at 6:48 PM, Stack <st...@duboce.net>
> wrote:
> > >>> > > >
> > >>> > > >> On Tue, Nov 22, 2016 at 3:17 PM, Vladimir Rodionov <
> > >>> > > >> vladrodionov@gmail.com> wrote:
> > >>> > > >>
> > >>> > > >>> >> and/or he answered most of the review feedback
> > >>> > > >>>
> > >>> > > >>> No, questions are still open, but I do not see any blockers
> and
> > >>> we
> > >>> > have
> > >>> > > >>> HBASE-16940 to address these questions.
> > >>> > > >>>
> > >>> > > >>>
> > >>> > > >> Agree. No blockers but stuff that should be dealt with (No one
> > >>> will
> > >>> > pay
> > >>> > > >> me any attention once merge goes in -- smile).
> > >>> > > >>
> > >>> > > >>
> > >>> > > > Let me clarify the above. I want review addressed before merge
> > >>> happens.
> > >>> > > > Sorry if any confusion.
> > >>> > > > St.Ack
> > >>> > > >
> > >>> > > >
> > >>> > > >
> > >>> > > >
> > >>> > > >
> > >>> > > >
> > >>> > > >> St.Ack
> > >>> > > >>
> > >>> > > >>
> > >>> > > >>
> > >>> > > >>> On Tue, Nov 22, 2016 at 3:04 PM, Devaraj Das <
> > >>> ddas@hortonworks.com>
> > >>> > > >>> wrote:
> > >>> > > >>>
> > >>> > > >>> > Hi Stack, hats off to you for spending so much time on
> this!
> > >>> > Thanks!
> > >>> > > >>> From
> > >>> > > >>> > my understanding, Vlad has raised follow-up jiras for the
> > >>> issues
> > >>> > you
> > >>> > > >>> > raised, and/or he answered most of the review feedback. So,
> > do
> > >>> you
> > >>> > > >>> think we
> > >>> > > >>> > could do a merge vote now?
> > >>> > > >>> > Devaraj.
> > >>> > > >>> > ________________________________________
> > >>> > > >>> > From: Vladimir Rodionov <vl...@gmail.com>
> > >>> > > >>> > Sent: Monday, November 21, 2016 8:34 PM
> > >>> > > >>> > To: dev@hbase.apache.org
> > >>> > > >>> > Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch
> > >>> > HBASE-7912
> > >>> > > >>> >
> > >>> > > >>> > >> I have spent a good bit of time reviewing and testing
> this
> > >>> > > feature.
> > >>> > > >>> I
> > >>> > > >>> > would
> > >>> > > >>> > >> like my review and concerns addressed and I'd like it to
> > be
> > >>> > clear
> > >>> > > >>> how;
> > >>> > > >>> > >> either explicit follow-on issues, pointers to where in
> the
> > >>> patch
> > >>> > > or
> > >>> > > >>> doc
> > >>> > > >>> > my
> > >>> > > >>> > >> remarks have been catered to, etc. Until then, I am
> > against
> > >>> > > commit.
> > >>> > > >>> >
> > >>> > > >>> > Stack, mega patch review comments will be addressed in the
> > >>> > dedicated
> > >>> > > >>> JIRA:
> > >>> > > >>> > HBASE-16940
> > >>> > > >>> > I have open several other JIRAs to address your other
> > comments
> > >>> (not
> > >>> > > on
> > >>> > > >>> > review board).
> > >>> > > >>> >
> > >>> > > >>> > Details are here (end of the thread):
> > >>> > > >>> > https://issues.apache.org/jira/browse/HBASE-14123
> > >>> > > >>> >
> > >>> > > >>> > Let me know what else should we do to move merge forward.
> > >>> > > >>> >
> > >>> > > >>> > -Vlad
> > >>> > > >>> >
> > >>> > > >>> >
> > >>> > > >>> > On Fri, Nov 18, 2016 at 4:54 PM, Stack <st...@duboce.net>
> > >>> wrote:
> > >>> > > >>> >
> > >>> > > >>> > > On Fri, Nov 18, 2016 at 3:53 PM, Ted Yu <
> > yuzhihong@gmail.com
> > >>> >
> > >>> > > wrote:
> > >>> > > >>> > >
> > >>> > > >>> > > > Thanks, Matteo.
> > >>> > > >>> > > >
> > >>> > > >>> > > > bq. restore is not clear if given an incremental id it
> > >>> will do
> > >>> > > the
> > >>> > > >>> full
> > >>> > > >>> > > > restore from full up to that point or if i need to
> apply
> > >>> > manually
> > >>> > > >>> > > > everything
> > >>> > > >>> > > >
> > >>> > > >>> > > > The restore takes into consideration of the dependent
> > >>> > backup(s).
> > >>> > > >>> > > > So there is no need to apply preceding backup(s)
> > manually.
> > >>> > > >>> > > >
> > >>> > > >>> > > >
> > >>> > > >>> > > I ask this question on the issue. It is not clear from
> the
> > >>> usage
> > >>> > or
> > >>> > > >>> doc
> > >>> > > >>> > how
> > >>> > > >>> > > to run a restore from incremental. Can you fix in doc and
> > >>> usage
> > >>> > how
> > >>> > > >>> so I
> > >>> > > >>> > > can be clear and try it. Currently I am stuck verifying a
> > >>> round
> > >>> > > trip
> > >>> > > >>> > backup
> > >>> > > >>> > > restore made of incrementals.
> > >>> > > >>> > >
> > >>> > > >>> > > Thanks,
> > >>> > > >>> > > S
> > >>> > > >>> > >
> > >>> > > >>> > >
> > >>> > > >>> > >
> > >>> > > >>> > > > On Fri, Nov 18, 2016 at 3:48 PM, Matteo Bertozzi <
> > >>> > > >>> > > theo.bertozzi@gmail.com>
> > >>> > > >>> > > > wrote:
> > >>> > > >>> > > >
> > >>> > > >>> > > > > I did one last pass to the mega patch. I don't see
> > >>> anything
> > >>> > > major
> > >>> > > >>> > that
> > >>> > > >>> > > > > should block the merge.
> > >>> > > >>> > > > >
> > >>> > > >>> > > > > - most of the code is isolated in the backup package
> > >>> > > >>> > > > > - all the backup code is client side
> > >>> > > >>> > > > > - there are few changes to the server side, mainly
> for
> > >>> > > cleaners,
> > >>> > > >>> wal
> > >>> > > >>> > > > > rolling and similar (which is ok)
> > >>> > > >>> > > > > - there is a good number of tests, and an integration
> > >>> test
> > >>> > > >>> > > > >
> > >>> > > >>> > > > > the code seems to have still some left overs from the
> > old
> > >>> > > >>> > > implementation,
> > >>> > > >>> > > > > and some stuff needs a cleanup. but I don't think
> this
> > >>> should
> > >>> > > be
> > >>> > > >>> used
> > >>> > > >>> > > as
> > >>> > > >>> > > > an
> > >>> > > >>> > > > > argument to block the merge. I think the guys will
> keep
> > >>> > working
> > >>> > > >>> on
> > >>> > > >>> > this
> > >>> > > >>> > > > and
> > >>> > > >>> > > > > they may also get help of others once the patch is in
> > >>> master.
> > >>> > > >>> > > > >
> > >>> > > >>> > > > > I still have my concerns about the current
> limitations,
> > >>> but
> > >>> > > >>> these are
> > >>> > > >>> > > > > things already planned for phase 3, so some of this
> > >>> stuff may
> > >>> > > >>> even be
> > >>> > > >>> > > in
> > >>> > > >>> > > > > the final 2.0.
> > >>> > > >>> > > > > but as long as we have a "current limitations"
> section
> > >>> in the
> > >>> > > >>> user
> > >>> > > >>> > > guide
> > >>> > > >>> > > > > mentioning important stuff like the ones below, I'm
> ok
> > >>> with
> > >>> > it.
> > >>> > > >>> > > > >  - if you write to the table with
> Durability.SKIP_WALS
> > >>> your
> > >>> > > data
> > >>> > > >>> will
> > >>> > > >>> > > not
> > >>> > > >>> > > > > be in the incremental-backup
> > >>> > > >>> > > > >  - if you bulkload files that data will not be in the
> > >>> > > incremental
> > >>> > > >>> > > backup
> > >>> > > >>> > > > > (HBASE-14417)
> > >>> > > >>> > > > >  - the incremental backup will not only contains the
> > >>> data of
> > >>> > > the
> > >>> > > >>> > table
> > >>> > > >>> > > > you
> > >>> > > >>> > > > > specified but also the regions from other tables that
> > >>> are on
> > >>> > > the
> > >>> > > >>> same
> > >>> > > >>> > > set
> > >>> > > >>> > > > > of RSs (HBASE-14141) ...maybe a note about security
> > >>> around
> > >>> > this
> > >>> > > >>> topic
> > >>> > > >>> > > > >  - the incremental backup will not contains just the
> > >>> "latest
> > >>> > > row"
> > >>> > > >>> > > between
> > >>> > > >>> > > > > backup A and B, but it will also contains all the
> > updates
> > >>> > > >>> occurred in
> > >>> > > >>> > > > > between. but the restore does not allow you to
> restore
> > >>> up to
> > >>> > a
> > >>> > > >>> > certain
> > >>> > > >>> > > > > point in time, the restore will always be up to the
> > >>> "latest
> > >>> > > >>> backup
> > >>> > > >>> > > > point".
> > >>> > > >>> > > > >  - you should limit the number of "incremental" up
> to N
> > >>> (or
> > >>> > > maybe
> > >>> > > >>> > > SIZE),
> > >>> > > >>> > > > to
> > >>> > > >>> > > > > avoid replay time becoming the bottleneck.
> > (HBASE-14135)
> > >>> > > >>> > > > >
> > >>> > > >>> > > > > I'll be ok even with the above not being in the final
> > >>> 2.0,
> > >>> > > >>> > > > > but i'd like to see as blocker for the final 2.0 (not
> > the
> > >>> > > merge)
> > >>> > > >>> > > > >  - the backup code moved in an hbase-backup module
> > >>> > > >>> > > > >  - and some more work around tools, especially to try
> > to
> > >>> > unify
> > >>> > > >>> and
> > >>> > > >>> > make
> > >>> > > >>> > > > > simple the backup experience (simple example: in some
> > >>> case
> > >>> > > there
> > >>> > > >>> is a
> > >>> > > >>> > > > > backup_id argument in others a backupId argument. or
> > >>> things
> > >>> > > >>> like..
> > >>> > > >>> > > > restore
> > >>> > > >>> > > > > is not clear if given an incremental id it will do
> the
> > >>> full
> > >>> > > >>> restore
> > >>> > > >>> > > from
> > >>> > > >>> > > > > full up to that point or if i need to apply manually
> > >>> > > everything).
> > >>> > > >>> > > > >
> > >>> > > >>> > > > > in conclusion, I think we can open a merge vote. I'll
> > be
> > >>> +1
> > >>> > on
> > >>> > > >>> it,
> > >>> > > >>> > and
> > >>> > > >>> > > I
> > >>> > > >>> > > > > think we should try to reject -1 with just a "code
> > >>> cleanup"
> > >>> > > >>> > motivation,
> > >>> > > >>> > > > > since there will still be work going on on the code
> > >>> after the
> > >>> > > >>> merge.
> > >>> > > >>> > > > >
> > >>> > > >>> > > > > Matteo
> > >>> > > >>> > > > >
> > >>> > > >>> > > > >
> > >>> > > >>> > > > > On Sun, Nov 6, 2016 at 10:54 PM, Devaraj Das <
> > >>> > > >>> ddas@hortonworks.com>
> > >>> > > >>> > > > wrote:
> > >>> > > >>> > > > >
> > >>> > > >>> > > > > > Stack and others, anything else on the patch? Merge
> > to
> > >>> > master
> > >>> > > >>> now?
> > >>> > > >>> > > > > >
> > >>> > > >>> > > > >
> > >>> > > >>> > > >
> > >>> > > >>> > >
> > >>> > > >>> >
> > >>> > > >>>
> > >>> > > >>
> > >>> > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> > >>
> > >>
> > >
> >
>

Re: [DISCUSS] Plan for Distributed testing of Backup and Restore

[Josh Elser <el...@apache.org>]

On 9/12/17 2:51 PM, Andrew Purtell wrote:
>> making backup working in challenging conditions was not a goal of FT
> design, correct failure handling was a goal.
> 
> Every real-world production environment has challenging conditions.
> 
> That said, making progress in the face of failures is only one aspect of
> FT, and an equally valid one is that failures do not cause data corruption.
> 
> If testing with chaos proves this backup solution will fail if there is any
> failure while backup is in progress, but at least it will successfully
> clean up and not corrupt existing state - that could be ok, for some.
> Possibly, us.

Agreed. There are always differences of opinion around acceptable levels 
of tolerance. Understanding how things fail (avoiding the need for 
manual interaction to correct) is a good initial goal-post as we can 
concisely document that for users. My impression is that this wouldn't 
require a significant amount of work to achieve an acceptable degree of 
stability.

> If testing with chaos proves this backup solution will not suffer
> corruption if there is a failure *and* can still successfully complete if
> there is any failure while backup is in progress - that would obviously
> improve the perceived value proposition.
> 
> It would be fine to test this using hbase-it chaos facilities but with a
> less aggressive policy than slowDeterministic that allows for backups to
> successfully complete once in a while yet also demonstrate that when the
> failures do happen things are properly cleaned up and data corruption does
> not happen.
> 
> 
> 
> 
> On Tue, Sep 12, 2017 at 11:25 AM, Vladimir Rodionov <vl...@gmail.com>
> wrote:
> 
>>>> Vlad: I'm obviously curious to see what you think about this stuff, in
>> addition to what you already had in mind :)
>>
>> Yes, I think that we need a test tool similar to ITBLL. Btw, making backup
>> working in challenging conditions was not a goal of FT design, correct
>> failure handling was a goal.

Based on Ted's mention of ITBackupRestore (thanks btwm Ted!), I think 
that gets into the details a little to much for this thread. Definitely 
need to improve on that test for what we're discussing here, but perhaps 
it's a nice starting point?

>> On Tue, Sep 12, 2017 at 9:53 AM, Josh Elser <el...@apache.org> wrote:
>>
>>> Thanks for the quick feedback!
>>>
>>> On 9/12/17 12:36 PM, Stack wrote:
>>>
>>>> On Tue, Sep 12, 2017 at 9:33 AM, Andrew Purtell <
>> andrew.purtell@gmail.com
>>>>>
>>>> wrote:
>>>>
>>>> I think those are reasonable criteria Josh.
>>>>>
>>>>> What I would like to see is something like "we ran ITBLL (or custom
>>>>> generator with similar correctness validation if you prefer) on a dev
>>>>> cluster (5-10 nodes) for 24 hours with server killing chaos agents
>>>>> active,
>>>>> attempted 1,440 backups (one per minute), of which 1,000 succeeded and
>>>>> 100%
>>>>> if these were successfully restored and validated." This implies your
>>>>> points on automation and no manual intervention. Maybe the number of
>>>>> successful backups under challenging conditions will be lower. Point is
>>>>> they demonstrate we can rely on it even when a cluster is partially
>>>>> unhealthy, which in production is often the normal order of affairs.
>>>>>
>>>>>
>>>>>
>>> I like it. I hadn't thought about stressing quite this aggressively, but
>>> now that I think about it, sounds like a great plan. Having some ballpark
>>> measure to quantify the cost of a "backup-heavy" workload would be cool
>> in
>>> addition to seeing how the system reacts in unexpected manners.
>>>
>>> Sounds good to me.
>>>>
>>>> How will you test the restore aspect? After 1k (or whatever makes sense)
>>>> incremental backups over the life of the chaos, could you restore and
>>>> validate that the table had all expected data in place.
>>>>
>>>
>>> Exactly. My thinking was that, at any point, we should be able to do a
>>> restore and validate. Maybe something like: every Nth ITBLL iteration,
>> make
>>> a new backup point, restore a previous backup point, verify, restore to
>>> newest backup point. The previous backup point should be a full or
>>> incremental point.
>>>
>>> Vlad: I'm obviously curious to see what you think about this stuff, in
>>> addition to what you already had in mind :)
>>>
>>
> 
> 
> 

Re: [DISCUSS] Plan for Distributed testing of Backup and Restore

[Vladimir Rodionov <vl...@gmail.com>]

Yes, we have already some IT, so will need to upgrade it for scale testing.

On Tue, Sep 12, 2017 at 11:28 AM, Ted Yu <yu...@gmail.com> wrote:

> bq. we need a test tool similar to ITBLL
>
> How about making the following such a tool ?
>
> hbase-it/src/test/java/org/apache/hadoop/hbase/
> IntegrationTestBackupRestore.java
>
> On Tue, Sep 12, 2017 at 11:25 AM, Vladimir Rodionov <
> vladrodionov@gmail.com>
> wrote:
>
> > >> Vlad: I'm obviously curious to see what you think about this stuff, in
> > addition to what you already had in mind :)
> >
> > Yes, I think that we need a test tool similar to ITBLL. Btw, making
> backup
> > working in challenging conditions was not a goal of FT design, correct
> > failure handling was a goal.
> >
> > On Tue, Sep 12, 2017 at 9:53 AM, Josh Elser <el...@apache.org> wrote:
> >
> > > Thanks for the quick feedback!
> > >
> > > On 9/12/17 12:36 PM, Stack wrote:
> > >
> > >> On Tue, Sep 12, 2017 at 9:33 AM, Andrew Purtell <
> > andrew.purtell@gmail.com
> > >> >
> > >> wrote:
> > >>
> > >> I think those are reasonable criteria Josh.
> > >>>
> > >>> What I would like to see is something like "we ran ITBLL (or custom
> > >>> generator with similar correctness validation if you prefer) on a dev
> > >>> cluster (5-10 nodes) for 24 hours with server killing chaos agents
> > >>> active,
> > >>> attempted 1,440 backups (one per minute), of which 1,000 succeeded
> and
> > >>> 100%
> > >>> if these were successfully restored and validated." This implies your
> > >>> points on automation and no manual intervention. Maybe the number of
> > >>> successful backups under challenging conditions will be lower. Point
> is
> > >>> they demonstrate we can rely on it even when a cluster is partially
> > >>> unhealthy, which in production is often the normal order of affairs.
> > >>>
> > >>>
> > >>>
> > > I like it. I hadn't thought about stressing quite this aggressively,
> but
> > > now that I think about it, sounds like a great plan. Having some
> ballpark
> > > measure to quantify the cost of a "backup-heavy" workload would be cool
> > in
> > > addition to seeing how the system reacts in unexpected manners.
> > >
> > > Sounds good to me.
> > >>
> > >> How will you test the restore aspect? After 1k (or whatever makes
> sense)
> > >> incremental backups over the life of the chaos, could you restore and
> > >> validate that the table had all expected data in place.
> > >>
> > >
> > > Exactly. My thinking was that, at any point, we should be able to do a
> > > restore and validate. Maybe something like: every Nth ITBLL iteration,
> > make
> > > a new backup point, restore a previous backup point, verify, restore to
> > > newest backup point. The previous backup point should be a full or
> > > incremental point.
> > >
> > > Vlad: I'm obviously curious to see what you think about this stuff, in
> > > addition to what you already had in mind :)
> > >
> >
>

Re: [DISCUSS] Plan for Distributed testing of Backup and Restore

[Ted Yu <yu...@gmail.com>]

bq. we need a test tool similar to ITBLL

How about making the following such a tool ?

hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestBackupRestore.java

On Tue, Sep 12, 2017 at 11:25 AM, Vladimir Rodionov <vl...@gmail.com>
wrote:

> >> Vlad: I'm obviously curious to see what you think about this stuff, in
> addition to what you already had in mind :)
>
> Yes, I think that we need a test tool similar to ITBLL. Btw, making backup
> working in challenging conditions was not a goal of FT design, correct
> failure handling was a goal.
>
> On Tue, Sep 12, 2017 at 9:53 AM, Josh Elser <el...@apache.org> wrote:
>
> > Thanks for the quick feedback!
> >
> > On 9/12/17 12:36 PM, Stack wrote:
> >
> >> On Tue, Sep 12, 2017 at 9:33 AM, Andrew Purtell <
> andrew.purtell@gmail.com
> >> >
> >> wrote:
> >>
> >> I think those are reasonable criteria Josh.
> >>>
> >>> What I would like to see is something like "we ran ITBLL (or custom
> >>> generator with similar correctness validation if you prefer) on a dev
> >>> cluster (5-10 nodes) for 24 hours with server killing chaos agents
> >>> active,
> >>> attempted 1,440 backups (one per minute), of which 1,000 succeeded and
> >>> 100%
> >>> if these were successfully restored and validated." This implies your
> >>> points on automation and no manual intervention. Maybe the number of
> >>> successful backups under challenging conditions will be lower. Point is
> >>> they demonstrate we can rely on it even when a cluster is partially
> >>> unhealthy, which in production is often the normal order of affairs.
> >>>
> >>>
> >>>
> > I like it. I hadn't thought about stressing quite this aggressively, but
> > now that I think about it, sounds like a great plan. Having some ballpark
> > measure to quantify the cost of a "backup-heavy" workload would be cool
> in
> > addition to seeing how the system reacts in unexpected manners.
> >
> > Sounds good to me.
> >>
> >> How will you test the restore aspect? After 1k (or whatever makes sense)
> >> incremental backups over the life of the chaos, could you restore and
> >> validate that the table had all expected data in place.
> >>
> >
> > Exactly. My thinking was that, at any point, we should be able to do a
> > restore and validate. Maybe something like: every Nth ITBLL iteration,
> make
> > a new backup point, restore a previous backup point, verify, restore to
> > newest backup point. The previous backup point should be a full or
> > incremental point.
> >
> > Vlad: I'm obviously curious to see what you think about this stuff, in
> > addition to what you already had in mind :)
> >
>

Re: [DISCUSS] Plan for Distributed testing of Backup and Restore

[Andrew Purtell <ap...@apache.org>]

> making backup working in challenging conditions was not a goal of FT
design, correct failure handling was a goal.

Every real-world production environment has challenging conditions.

That said, making progress in the face of failures is only one aspect of
FT, and an equally valid one is that failures do not cause data corruption.

If testing with chaos proves this backup solution will fail if there is any
failure while backup is in progress, but at least it will successfully
clean up and not corrupt existing state - that could be ok, for some.
Possibly, us.

If testing with chaos proves this backup solution will not suffer
corruption if there is a failure *and* can still successfully complete if
there is any failure while backup is in progress - that would obviously
improve the perceived value proposition.

It would be fine to test this using hbase-it chaos facilities but with a
less aggressive policy than slowDeterministic that allows for backups to
successfully complete once in a while yet also demonstrate that when the
failures do happen things are properly cleaned up and data corruption does
not happen.




On Tue, Sep 12, 2017 at 11:25 AM, Vladimir Rodionov <vl...@gmail.com>
wrote:

> >> Vlad: I'm obviously curious to see what you think about this stuff, in
> addition to what you already had in mind :)
>
> Yes, I think that we need a test tool similar to ITBLL. Btw, making backup
> working in challenging conditions was not a goal of FT design, correct
> failure handling was a goal.
>
> On Tue, Sep 12, 2017 at 9:53 AM, Josh Elser <el...@apache.org> wrote:
>
> > Thanks for the quick feedback!
> >
> > On 9/12/17 12:36 PM, Stack wrote:
> >
> >> On Tue, Sep 12, 2017 at 9:33 AM, Andrew Purtell <
> andrew.purtell@gmail.com
> >> >
> >> wrote:
> >>
> >> I think those are reasonable criteria Josh.
> >>>
> >>> What I would like to see is something like "we ran ITBLL (or custom
> >>> generator with similar correctness validation if you prefer) on a dev
> >>> cluster (5-10 nodes) for 24 hours with server killing chaos agents
> >>> active,
> >>> attempted 1,440 backups (one per minute), of which 1,000 succeeded and
> >>> 100%
> >>> if these were successfully restored and validated." This implies your
> >>> points on automation and no manual intervention. Maybe the number of
> >>> successful backups under challenging conditions will be lower. Point is
> >>> they demonstrate we can rely on it even when a cluster is partially
> >>> unhealthy, which in production is often the normal order of affairs.
> >>>
> >>>
> >>>
> > I like it. I hadn't thought about stressing quite this aggressively, but
> > now that I think about it, sounds like a great plan. Having some ballpark
> > measure to quantify the cost of a "backup-heavy" workload would be cool
> in
> > addition to seeing how the system reacts in unexpected manners.
> >
> > Sounds good to me.
> >>
> >> How will you test the restore aspect? After 1k (or whatever makes sense)
> >> incremental backups over the life of the chaos, could you restore and
> >> validate that the table had all expected data in place.
> >>
> >
> > Exactly. My thinking was that, at any point, we should be able to do a
> > restore and validate. Maybe something like: every Nth ITBLL iteration,
> make
> > a new backup point, restore a previous backup point, verify, restore to
> > newest backup point. The previous backup point should be a full or
> > incremental point.
> >
> > Vlad: I'm obviously curious to see what you think about this stuff, in
> > addition to what you already had in mind :)
> >
>



-- 
Best regards,
Andrew

Words like orphans lost among the crosstalk, meaning torn from truth's
decrepit hands
   - A23, Crosstalk

Re: [DISCUSS] Plan for Distributed testing of Backup and Restore

[Vladimir Rodionov <vl...@gmail.com>]

>> Vlad: I'm obviously curious to see what you think about this stuff, in
addition to what you already had in mind :)

Yes, I think that we need a test tool similar to ITBLL. Btw, making backup
working in challenging conditions was not a goal of FT design, correct
failure handling was a goal.

On Tue, Sep 12, 2017 at 9:53 AM, Josh Elser <el...@apache.org> wrote:

> Thanks for the quick feedback!
>
> On 9/12/17 12:36 PM, Stack wrote:
>
>> On Tue, Sep 12, 2017 at 9:33 AM, Andrew Purtell <andrew.purtell@gmail.com
>> >
>> wrote:
>>
>> I think those are reasonable criteria Josh.
>>>
>>> What I would like to see is something like "we ran ITBLL (or custom
>>> generator with similar correctness validation if you prefer) on a dev
>>> cluster (5-10 nodes) for 24 hours with server killing chaos agents
>>> active,
>>> attempted 1,440 backups (one per minute), of which 1,000 succeeded and
>>> 100%
>>> if these were successfully restored and validated." This implies your
>>> points on automation and no manual intervention. Maybe the number of
>>> successful backups under challenging conditions will be lower. Point is
>>> they demonstrate we can rely on it even when a cluster is partially
>>> unhealthy, which in production is often the normal order of affairs.
>>>
>>>
>>>
> I like it. I hadn't thought about stressing quite this aggressively, but
> now that I think about it, sounds like a great plan. Having some ballpark
> measure to quantify the cost of a "backup-heavy" workload would be cool in
> addition to seeing how the system reacts in unexpected manners.
>
> Sounds good to me.
>>
>> How will you test the restore aspect? After 1k (or whatever makes sense)
>> incremental backups over the life of the chaos, could you restore and
>> validate that the table had all expected data in place.
>>
>
> Exactly. My thinking was that, at any point, we should be able to do a
> restore and validate. Maybe something like: every Nth ITBLL iteration, make
> a new backup point, restore a previous backup point, verify, restore to
> newest backup point. The previous backup point should be a full or
> incremental point.
>
> Vlad: I'm obviously curious to see what you think about this stuff, in
> addition to what you already had in mind :)
>

Re: [DISCUSS] Plan for Distributed testing of Backup and Restore

[Josh Elser <el...@apache.org>]

Thanks for the quick feedback!

On 9/12/17 12:36 PM, Stack wrote:
> On Tue, Sep 12, 2017 at 9:33 AM, Andrew Purtell <an...@gmail.com>
> wrote:
> 
>> I think those are reasonable criteria Josh.
>>
>> What I would like to see is something like "we ran ITBLL (or custom
>> generator with similar correctness validation if you prefer) on a dev
>> cluster (5-10 nodes) for 24 hours with server killing chaos agents active,
>> attempted 1,440 backups (one per minute), of which 1,000 succeeded and 100%
>> if these were successfully restored and validated." This implies your
>> points on automation and no manual intervention. Maybe the number of
>> successful backups under challenging conditions will be lower. Point is
>> they demonstrate we can rely on it even when a cluster is partially
>> unhealthy, which in production is often the normal order of affairs.
>>
>>

I like it. I hadn't thought about stressing quite this aggressively, but 
now that I think about it, sounds like a great plan. Having some 
ballpark measure to quantify the cost of a "backup-heavy" workload would 
be cool in addition to seeing how the system reacts in unexpected manners.

> Sounds good to me.
> 
> How will you test the restore aspect? After 1k (or whatever makes sense)
> incremental backups over the life of the chaos, could you restore and
> validate that the table had all expected data in place.

Exactly. My thinking was that, at any point, we should be able to do a 
restore and validate. Maybe something like: every Nth ITBLL iteration, 
make a new backup point, restore a previous backup point, verify, 
restore to newest backup point. The previous backup point should be a 
full or incremental point.

Vlad: I'm obviously curious to see what you think about this stuff, in 
addition to what you already had in mind :)

Re: [DISCUSS] Plan for Distributed testing of Backup and Restore

[Stack <st...@duboce.net>]

On Tue, Sep 12, 2017 at 9:33 AM, Andrew Purtell <an...@gmail.com>
wrote:

> I think those are reasonable criteria Josh.
>
> What I would like to see is something like "we ran ITBLL (or custom
> generator with similar correctness validation if you prefer) on a dev
> cluster (5-10 nodes) for 24 hours with server killing chaos agents active,
> attempted 1,440 backups (one per minute), of which 1,000 succeeded and 100%
> if these were successfully restored and validated." This implies your
> points on automation and no manual intervention. Maybe the number of
> successful backups under challenging conditions will be lower. Point is
> they demonstrate we can rely on it even when a cluster is partially
> unhealthy, which in production is often the normal order of affairs.
>
>
Sounds good to me.

How will you test the restore aspect? After 1k (or whatever makes sense)
incremental backups over the life of the chaos, could you restore and
validate that the table had all expected data in place.

Thanks,
St.Ack



>
> > On Sep 12, 2017, at 9:07 AM, Josh Elser <el...@apache.org> wrote:
> >
> >> On 9/11/17 11:52 PM, Stack wrote:
> >> On Mon, Sep 11, 2017 at 11:07 AM, Vladimir Rodionov <
> vladrodionov@gmail.com>
> >> wrote:
> >>> ...
> >>> That is mostly it. Yes, We have not done real testing with real data
> on a
> >>> real cluster yet, except QA  testing on a small OpenStack
> >>> cluster (10 nodes). That is our probably the biggest minus right now. I
> >>> would like to inform community that this week we are going to start
> >>> full scale testing with reasonably sized data sets.
> >>>
> >> ... Completion of HA seems important as is result of the scale testing.
> >
> > I think we should knock out a rough sketch on what effective "scale"
> testing would look like since that is a very subjective phrase. Let me
> start the ball rolling with a few things that come to my mind.
> >
> > (interpreting requirements as per rfc2119)
> >
> > * MUST have >5 RegionServers and >1 Masters in play
> > * MUST have Non-trivial final data sizes (final data size would be >=
> 100's of GB)
> > * MUST have some clear pass/fail determination for correctness of B&R
> > * MUST have some fault-injection
> >
> > * SHOULD be a completely automated test, not require coordination of a
> human to executing commands.
> > * SHOULD be able to acquire operational insight (metrics) while
> performing operations to determine success of testing
> > * SHOULD NOT require manual intervention, e.g. working around known
> issues/limitations
> > * SHOULD reuse the IntegrationTest framework in hbase-it
> >
> > Since we have a concern of correctness, ITBLL sounds like a good
> starting point to avoid having to re-write similar kinds of logic.
> ChaosMonkey is always great for fault-injection.
> >
> > Thoughts?
>

Re: [DISCUSS] Plan for Distributed testing of Backup and Restore

[Andrew Purtell <an...@gmail.com>]

I think those are reasonable criteria Josh. 

What I would like to see is something like "we ran ITBLL (or custom generator with similar correctness validation if you prefer) on a dev cluster (5-10 nodes) for 24 hours with server killing chaos agents active, attempted 1,440 backups (one per minute), of which 1,000 succeeded and 100% if these were successfully restored and validated." This implies your points on automation and no manual intervention. Maybe the number of successful backups under challenging conditions will be lower. Point is they demonstrate we can rely on it even when a cluster is partially unhealthy, which in production is often the normal order of affairs. 


> On Sep 12, 2017, at 9:07 AM, Josh Elser <el...@apache.org> wrote:
> 
>> On 9/11/17 11:52 PM, Stack wrote:
>> On Mon, Sep 11, 2017 at 11:07 AM, Vladimir Rodionov <vl...@gmail.com>
>> wrote:
>>> ...
>>> That is mostly it. Yes, We have not done real testing with real data on a
>>> real cluster yet, except QA  testing on a small OpenStack
>>> cluster (10 nodes). That is our probably the biggest minus right now. I
>>> would like to inform community that this week we are going to start
>>> full scale testing with reasonably sized data sets.
>>> 
>> ... Completion of HA seems important as is result of the scale testing.
> 
> I think we should knock out a rough sketch on what effective "scale" testing would look like since that is a very subjective phrase. Let me start the ball rolling with a few things that come to my mind.
> 
> (interpreting requirements as per rfc2119)
> 
> * MUST have >5 RegionServers and >1 Masters in play
> * MUST have Non-trivial final data sizes (final data size would be >= 100's of GB)
> * MUST have some clear pass/fail determination for correctness of B&R
> * MUST have some fault-injection
> 
> * SHOULD be a completely automated test, not require coordination of a human to executing commands.
> * SHOULD be able to acquire operational insight (metrics) while performing operations to determine success of testing
> * SHOULD NOT require manual intervention, e.g. working around known issues/limitations
> * SHOULD reuse the IntegrationTest framework in hbase-it
> 
> Since we have a concern of correctness, ITBLL sounds like a good starting point to avoid having to re-write similar kinds of logic. ChaosMonkey is always great for fault-injection.
> 
> Thoughts?

[DISCUSS] Plan for Distributed testing of Backup and Restore

[Josh Elser <el...@apache.org>]

On 9/11/17 11:52 PM, Stack wrote:
> On Mon, Sep 11, 2017 at 11:07 AM, Vladimir Rodionov <vl...@gmail.com>
> wrote:
> 
>> ...
>> That is mostly it. Yes, We have not done real testing with real data on a
>> real cluster yet, except QA  testing on a small OpenStack
>> cluster (10 nodes). That is our probably the biggest minus right now. I
>> would like to inform community that this week we are going to start
>> full scale testing with reasonably sized data sets.
>>
> ... Completion of HA seems important as is result of the scale testing.
> 

I think we should knock out a rough sketch on what effective "scale" 
testing would look like since that is a very subjective phrase. Let me 
start the ball rolling with a few things that come to my mind.

(interpreting requirements as per rfc2119)

* MUST have >5 RegionServers and >1 Masters in play
* MUST have Non-trivial final data sizes (final data size would be >= 
100's of GB)
* MUST have some clear pass/fail determination for correctness of B&R
* MUST have some fault-injection

* SHOULD be a completely automated test, not require coordination of a 
human to executing commands.
* SHOULD be able to acquire operational insight (metrics) while 
performing operations to determine success of testing
* SHOULD NOT require manual intervention, e.g. working around known 
issues/limitations
* SHOULD reuse the IntegrationTest framework in hbase-it

Since we have a concern of correctness, ITBLL sounds like a good 
starting point to avoid having to re-write similar kinds of logic. 
ChaosMonkey is always great for fault-injection.

Thoughts?

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Josh Elser <el...@apache.org>]

On 9/11/17 11:52 PM, Stack wrote:
> On Mon, Sep 11, 2017 at 11:07 AM, Vladimir Rodionov <vl...@gmail.com>
> wrote:

<snip/>

>> My plan is to finish HBASE-17825 (further performance optimizations). This
>> will cut down number of MR jobs during incremental backup
>> from 2*N to 2  (N - number of tables). That will probably take 2-3 more
>> days

Unless you have this patch ready to go already, could I convince you to 
temporarily pause work on HBASE-17825 to take up the 3 points you've 
outlined immediately below?

We can treat the optimization as a one-off instead of lumping it in with 
the B&R feature at-large.

>> Then:
>>
>> 1. Address remaining two sub-tasks in HBASE-15227
>> 2. Update Release notes for all relevant B&R JIRAs
>> 3. Work on doc
>>
>> After that we can call it feature full complete. Taking into account the
>> vast amount of efforts
>> spent on this feature (including QA testing) I would say that we are
>> probably quite close to GA right now, but only
>> after real testing is done (I do not anticipate significant issues, except
>> probably correct failure handling).
>>
>> On a feature itself. We provide tools to fully automate backup and restore
>> tasks: create backup (full and incremental), restore
>> from image, delete backups, merge backups, history, history per table,
>> backup set management.
>>
>> Hopefully, my write up addresses at least some of your concerns.
>>
>>
> Thanks for updating us (community) w/ status. Completion of HA seems
> important as is result of the scale testing.
> 
As we're quickly approaching that beta-1 mark, I think it would be in 
our combined best-interest to knock out what we have identified as 
blockers. While these items aren't "significant" (read-as: not feature 
work), they are still risks to the beta "train". We should prioritize 
de-risking ourselves as much as possible.

After we finish that up (or in parallel), we can look at getting some 
scale testing done. Let me spin out another thread to discuss the 
concrete details on that one.

- Josh

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Stack <st...@duboce.net>]

On Mon, Sep 11, 2017 at 11:07 AM, Vladimir Rodionov <vl...@gmail.com>
wrote:

> Stack, Andrew
>
> We have doc blocker and (partially) HBASE-15227: two sub-tasks remain: one
> is unit test (you can't call it blocker)
> and another for FT support during incremental backup with bulk loading. The
> latter one have been probably addressed
> already in other HBASE-15527 subtasks. I have to reassess this.
>
> That is mostly it. Yes, We have not done real testing with real data on a
> real cluster yet, except QA  testing on a small OpenStack
> cluster (10 nodes). That is our probably the biggest minus right now. I
> would like to inform community that this week we are going to start
> full scale testing with reasonably sized data sets.
>
> The recent committed improvements, such as ability to run backup/restore on
> a particular Yarn pool (queue) allows precise control
> of a cluster utilization during operation (not to interfere much with a
> regular cluster operations). Another one -
>  converting WAL on the fly to HFiles - significantly improves storage usage
> on a backup site.
>
> My plan is to finish HBASE-17825 (further performance optimizations). This
> will cut down number of MR jobs during incremental backup
> from 2*N to 2  (N - number of tables). That will probably take 2-3 more
> days
>
> Then:
>
> 1. Address remaining two sub-tasks in HBASE-15227
> 2. Update Release notes for all relevant B&R JIRAs
> 3. Work on doc
>
> After that we can call it feature full complete. Taking into account the
> vast amount of efforts
> spent on this feature (including QA testing) I would say that we are
> probably quite close to GA right now, but only
> after real testing is done (I do not anticipate significant issues, except
> probably correct failure handling).
>
> On a feature itself. We provide tools to fully automate backup and restore
> tasks: create backup (full and incremental), restore
> from image, delete backups, merge backups, history, history per table,
> backup set management.
>
> Hopefully, my write up addresses at least some of your concerns.
>
>
Thanks for updating us (community) w/ status. Completion of HA seems
important as is result of the scale testing.

St.Ack




> -Vlad
>
> On Sun, Sep 10, 2017 at 6:27 AM, Josh Elser <el...@apache.org> wrote:
>
> > On Sat, Sep 9, 2017 at 7:04 PM, stack <sa...@gmail.com> wrote:
> > > In spite of repeated requests for eng summary of state of this feature
> --
> > > summary of what is in 2.0, what is not, what the capabilities are, how
> > well
> > > it has been tested and at what scale -- all I get, when the requests
> are
> > > not ignored, are pointers to lists of ill-describing jiras and some
> > pending
> > > user facing doc update.
> >
> > Yes, this is a problem. We, especially you as RM, shouldn't have
> > outstanding questions as to the quality/state of B&R.
> >
> > > For other features, mob or region server groups, I know that they have
> > been
> > > running at scale in production for as much as a year and more. I have
> > some
> > > confidence these items basically work.  For backup/restore I have no
> such
> > > sense even after spending time in review and trying to use the feature.
> >
> > I can attest to the feature being tested on small clusters. I'm not
> > sure about larger than 10node tests. If this is less a worry and more
> > a veto, let's get some criteria on the kind of testing you're looking
> > for to avoid having to rehash later.
> >
> > Do we have any kind of integration tests in the codebase now that can
> > help increase Stack's confidence?
> >
> > > As release manager, I have say over what makes it into a release.
> Unless
> > > the work is done to convince me that backup/restore is more than a lump
> > of
> > > code and a few unit tests that can pass on some fellows laptop, I am
> > going
> > > to kick it out of branch-2.  Let the feature harden more in master
> branch
> > > before it ships in a release.
> >
> > While it was a few months ago now, I can also attest to this being
> > more than some unit tests (I think I looked at it after I saw you last
> > down in the weeds).
> >
> > I do worry about trying to remove it at this state.
> >
> > * Do you consider the B&R code in the repository implicitly harmful?
> > Is there harm in shipping with docs capturing the concern.
> > * Trying to revert all relevant pieces from branch-2 is non-trivial.
> > * I would feel quite dejected if some feature I spent a year+ working
> > on (*not* making assertions on my perception of quality) was removed
> > from the release line it was expected to land.
> >
> > > S
> > >
> > > On Sep 8, 2017 10:59 PM, "Vladimir Rodionov" <vl...@gmail.com>
> > wrote:
> > >
> > >> >> Have I grasped the state of things correctly, Vlad?
> > >>
> > >> Josh, the only thing which is still pending is doc update. All other
> > >> features are good to have but not a blockers for 2.0 release.
> > >>
> > >> -Vlad
> > >>
> > >> On Fri, Sep 8, 2017 at 10:42 PM, Vladimir Rodionov <
> > vladrodionov@gmail.com
> > >> >
> > >> wrote:
> > >>
> > >> > >> What testing and at what
> > >> > >> scale has testing been done?
> > >> >
> > >> > Do we have have that for other features?
> > >> >
> > >> >
> > >> > On Fri, Sep 8, 2017 at 10:41 PM, Vladimir Rodionov <
> > >> vladrodionov@gmail.com
> > >> > > wrote:
> > >> >
> > >> >> >> It asks: "How do I figure what of backup/restore feature is
> going
> > to
> > >> >> be in
> > >> >> >>hbase-2.0.0?
> > >> >>
> > >> >> Hmm, wait for doc update.
> > >> >>
> > >> >>
> > >> >> On Fri, Sep 8, 2017 at 2:39 PM, Stack <st...@duboce.net> wrote:
> > >> >>
> > >> >>> HBASE-14414 is a JIRA with a list of random seeming issues w/
> > >> >>> non-descript
> > >> >>> summaries: "Add nonce support to TableBackupProcedure, BackupID
> must
> > >> >>> include backup set name, ...". The last comment in that issue is
> > from
> > >> >>> July.
> > >> >>> It asks: "How do I figure what of backup/restore feature is going
> > to be
> > >> >>> in
> > >> >>> hbase-2.0.0? Thanks Vladimir Rodionov
> > >> >>> <https://issues.apache.org/jira/secure/ViewProfile.jspa?
> > name=vrodionov
> > >> >>> >."
> > >> >>> to which there is no answer.  Doc update is TODO.
> > >> >>>
> > >> >>> Where is the summary of the capability in hbase-2? What testing
> and
> > at
> > >> >>> what
> > >> >>> scale has testing been done? Is this 'stable or experimental'? If
> I
> > >> can't
> > >> >>> get basic info on this feature though I ask repeatedly, what hope
> > does
> > >> >>> the
> > >> >>> poor old operator have?
> > >> >>>
> > >> >>> St.Ack
> > >> >>>
> > >> >>>
> > >> >>> On Fri, Sep 8, 2017 at 1:59 PM, Vladimir Rodionov <
> > >> >>> vladrodionov@gmail.com>
> > >> >>> wrote:
> > >> >>>
> > >> >>> > HBASE-14414
> > >> >>> >
> > >> >>> > On Fri, Sep 8, 2017 at 1:14 PM, Stack <st...@duboce.net> wrote:
> > >> >>> >
> > >> >>> > > Where do I go to get the current status of this feature?
> > Looking in
> > >> >>> JIRA
> > >> >>> > I
> > >> >>> > > see loads of issues open against backup including some against
> > >> >>> > hbase-2.0.0
> > >> >>> > > and no progress being made that I can discern.
> > >> >>> > >
> > >> >>> > > Thanks,
> > >> >>> > > S
> > >> >>> > >
> > >> >>> > >
> > >> >>> > >
> > >> >>> > > On Wed, Nov 23, 2016 at 8:52 AM, Stack <st...@duboce.net>
> > wrote:
> > >> >>> > >
> > >> >>> > > > On Tue, Nov 22, 2016 at 6:48 PM, Stack <st...@duboce.net>
> > wrote:
> > >> >>> > > >
> > >> >>> > > >> On Tue, Nov 22, 2016 at 3:17 PM, Vladimir Rodionov <
> > >> >>> > > >> vladrodionov@gmail.com> wrote:
> > >> >>> > > >>
> > >> >>> > > >>> >> and/or he answered most of the review feedback
> > >> >>> > > >>>
> > >> >>> > > >>> No, questions are still open, but I do not see any
> blockers
> > and
> > >> >>> we
> > >> >>> > have
> > >> >>> > > >>> HBASE-16940 to address these questions.
> > >> >>> > > >>>
> > >> >>> > > >>>
> > >> >>> > > >> Agree. No blockers but stuff that should be dealt with (No
> > one
> > >> >>> will
> > >> >>> > pay
> > >> >>> > > >> me any attention once merge goes in -- smile).
> > >> >>> > > >>
> > >> >>> > > >>
> > >> >>> > > > Let me clarify the above. I want review addressed before
> merge
> > >> >>> happens.
> > >> >>> > > > Sorry if any confusion.
> > >> >>> > > > St.Ack
> > >> >>> > > >
> > >> >>> > > >
> > >> >>> > > >
> > >> >>> > > >
> > >> >>> > > >
> > >> >>> > > >
> > >> >>> > > >> St.Ack
> > >> >>> > > >>
> > >> >>> > > >>
> > >> >>> > > >>
> > >> >>> > > >>> On Tue, Nov 22, 2016 at 3:04 PM, Devaraj Das <
> > >> >>> ddas@hortonworks.com>
> > >> >>> > > >>> wrote:
> > >> >>> > > >>>
> > >> >>> > > >>> > Hi Stack, hats off to you for spending so much time on
> > this!
> > >> >>> > Thanks!
> > >> >>> > > >>> From
> > >> >>> > > >>> > my understanding, Vlad has raised follow-up jiras for
> the
> > >> >>> issues
> > >> >>> > you
> > >> >>> > > >>> > raised, and/or he answered most of the review feedback.
> > So,
> > >> do
> > >> >>> you
> > >> >>> > > >>> think we
> > >> >>> > > >>> > could do a merge vote now?
> > >> >>> > > >>> > Devaraj.
> > >> >>> > > >>> > ________________________________________
> > >> >>> > > >>> > From: Vladimir Rodionov <vl...@gmail.com>
> > >> >>> > > >>> > Sent: Monday, November 21, 2016 8:34 PM
> > >> >>> > > >>> > To: dev@hbase.apache.org
> > >> >>> > > >>> > Subject: Re: [DISCUSSION] Merge Backup / Restore -
> Branch
> > >> >>> > HBASE-7912
> > >> >>> > > >>> >
> > >> >>> > > >>> > >> I have spent a good bit of time reviewing and testing
> > this
> > >> >>> > > feature.
> > >> >>> > > >>> I
> > >> >>> > > >>> > would
> > >> >>> > > >>> > >> like my review and concerns addressed and I'd like it
> > to
> > >> be
> > >> >>> > clear
> > >> >>> > > >>> how;
> > >> >>> > > >>> > >> either explicit follow-on issues, pointers to where
> in
> > the
> > >> >>> patch
> > >> >>> > > or
> > >> >>> > > >>> doc
> > >> >>> > > >>> > my
> > >> >>> > > >>> > >> remarks have been catered to, etc. Until then, I am
> > >> against
> > >> >>> > > commit.
> > >> >>> > > >>> >
> > >> >>> > > >>> > Stack, mega patch review comments will be addressed in
> the
> > >> >>> > dedicated
> > >> >>> > > >>> JIRA:
> > >> >>> > > >>> > HBASE-16940
> > >> >>> > > >>> > I have open several other JIRAs to address your other
> > >> comments
> > >> >>> (not
> > >> >>> > > on
> > >> >>> > > >>> > review board).
> > >> >>> > > >>> >
> > >> >>> > > >>> > Details are here (end of the thread):
> > >> >>> > > >>> > https://issues.apache.org/jira/browse/HBASE-14123
> > >> >>> > > >>> >
> > >> >>> > > >>> > Let me know what else should we do to move merge
> forward.
> > >> >>> > > >>> >
> > >> >>> > > >>> > -Vlad
> > >> >>> > > >>> >
> > >> >>> > > >>> >
> > >> >>> > > >>> > On Fri, Nov 18, 2016 at 4:54 PM, Stack <
> stack@duboce.net>
> > >> >>> wrote:
> > >> >>> > > >>> >
> > >> >>> > > >>> > > On Fri, Nov 18, 2016 at 3:53 PM, Ted Yu <
> > >> yuzhihong@gmail.com
> > >> >>> >
> > >> >>> > > wrote:
> > >> >>> > > >>> > >
> > >> >>> > > >>> > > > Thanks, Matteo.
> > >> >>> > > >>> > > >
> > >> >>> > > >>> > > > bq. restore is not clear if given an incremental id
> it
> > >> >>> will do
> > >> >>> > > the
> > >> >>> > > >>> full
> > >> >>> > > >>> > > > restore from full up to that point or if i need to
> > apply
> > >> >>> > manually
> > >> >>> > > >>> > > > everything
> > >> >>> > > >>> > > >
> > >> >>> > > >>> > > > The restore takes into consideration of the
> dependent
> > >> >>> > backup(s).
> > >> >>> > > >>> > > > So there is no need to apply preceding backup(s)
> > >> manually.
> > >> >>> > > >>> > > >
> > >> >>> > > >>> > > >
> > >> >>> > > >>> > > I ask this question on the issue. It is not clear from
> > the
> > >> >>> usage
> > >> >>> > or
> > >> >>> > > >>> doc
> > >> >>> > > >>> > how
> > >> >>> > > >>> > > to run a restore from incremental. Can you fix in doc
> > and
> > >> >>> usage
> > >> >>> > how
> > >> >>> > > >>> so I
> > >> >>> > > >>> > > can be clear and try it. Currently I am stuck
> verifying
> > a
> > >> >>> round
> > >> >>> > > trip
> > >> >>> > > >>> > backup
> > >> >>> > > >>> > > restore made of incrementals.
> > >> >>> > > >>> > >
> > >> >>> > > >>> > > Thanks,
> > >> >>> > > >>> > > S
> > >> >>> > > >>> > >
> > >> >>> > > >>> > >
> > >> >>> > > >>> > >
> > >> >>> > > >>> > > > On Fri, Nov 18, 2016 at 3:48 PM, Matteo Bertozzi <
> > >> >>> > > >>> > > theo.bertozzi@gmail.com>
> > >> >>> > > >>> > > > wrote:
> > >> >>> > > >>> > > >
> > >> >>> > > >>> > > > > I did one last pass to the mega patch. I don't see
> > >> >>> anything
> > >> >>> > > major
> > >> >>> > > >>> > that
> > >> >>> > > >>> > > > > should block the merge.
> > >> >>> > > >>> > > > >
> > >> >>> > > >>> > > > > - most of the code is isolated in the backup
> package
> > >> >>> > > >>> > > > > - all the backup code is client side
> > >> >>> > > >>> > > > > - there are few changes to the server side, mainly
> > for
> > >> >>> > > cleaners,
> > >> >>> > > >>> wal
> > >> >>> > > >>> > > > > rolling and similar (which is ok)
> > >> >>> > > >>> > > > > - there is a good number of tests, and an
> > integration
> > >> >>> test
> > >> >>> > > >>> > > > >
> > >> >>> > > >>> > > > > the code seems to have still some left overs from
> > the
> > >> old
> > >> >>> > > >>> > > implementation,
> > >> >>> > > >>> > > > > and some stuff needs a cleanup. but I don't think
> > this
> > >> >>> should
> > >> >>> > > be
> > >> >>> > > >>> used
> > >> >>> > > >>> > > as
> > >> >>> > > >>> > > > an
> > >> >>> > > >>> > > > > argument to block the merge. I think the guys will
> > keep
> > >> >>> > working
> > >> >>> > > >>> on
> > >> >>> > > >>> > this
> > >> >>> > > >>> > > > and
> > >> >>> > > >>> > > > > they may also get help of others once the patch is
> > in
> > >> >>> master.
> > >> >>> > > >>> > > > >
> > >> >>> > > >>> > > > > I still have my concerns about the current
> > limitations,
> > >> >>> but
> > >> >>> > > >>> these are
> > >> >>> > > >>> > > > > things already planned for phase 3, so some of
> this
> > >> >>> stuff may
> > >> >>> > > >>> even be
> > >> >>> > > >>> > > in
> > >> >>> > > >>> > > > > the final 2.0.
> > >> >>> > > >>> > > > > but as long as we have a "current limitations"
> > section
> > >> >>> in the
> > >> >>> > > >>> user
> > >> >>> > > >>> > > guide
> > >> >>> > > >>> > > > > mentioning important stuff like the ones below,
> I'm
> > ok
> > >> >>> with
> > >> >>> > it.
> > >> >>> > > >>> > > > >  - if you write to the table with
> > Durability.SKIP_WALS
> > >> >>> your
> > >> >>> > > data
> > >> >>> > > >>> will
> > >> >>> > > >>> > > not
> > >> >>> > > >>> > > > > be in the incremental-backup
> > >> >>> > > >>> > > > >  - if you bulkload files that data will not be in
> > the
> > >> >>> > > incremental
> > >> >>> > > >>> > > backup
> > >> >>> > > >>> > > > > (HBASE-14417)
> > >> >>> > > >>> > > > >  - the incremental backup will not only contains
> the
> > >> >>> data of
> > >> >>> > > the
> > >> >>> > > >>> > table
> > >> >>> > > >>> > > > you
> > >> >>> > > >>> > > > > specified but also the regions from other tables
> > that
> > >> >>> are on
> > >> >>> > > the
> > >> >>> > > >>> same
> > >> >>> > > >>> > > set
> > >> >>> > > >>> > > > > of RSs (HBASE-14141) ...maybe a note about
> security
> > >> >>> around
> > >> >>> > this
> > >> >>> > > >>> topic
> > >> >>> > > >>> > > > >  - the incremental backup will not contains just
> the
> > >> >>> "latest
> > >> >>> > > row"
> > >> >>> > > >>> > > between
> > >> >>> > > >>> > > > > backup A and B, but it will also contains all the
> > >> updates
> > >> >>> > > >>> occurred in
> > >> >>> > > >>> > > > > between. but the restore does not allow you to
> > restore
> > >> >>> up to
> > >> >>> > a
> > >> >>> > > >>> > certain
> > >> >>> > > >>> > > > > point in time, the restore will always be up to
> the
> > >> >>> "latest
> > >> >>> > > >>> backup
> > >> >>> > > >>> > > > point".
> > >> >>> > > >>> > > > >  - you should limit the number of "incremental" up
> > to N
> > >> >>> (or
> > >> >>> > > maybe
> > >> >>> > > >>> > > SIZE),
> > >> >>> > > >>> > > > to
> > >> >>> > > >>> > > > > avoid replay time becoming the bottleneck.
> > >> (HBASE-14135)
> > >> >>> > > >>> > > > >
> > >> >>> > > >>> > > > > I'll be ok even with the above not being in the
> > final
> > >> >>> 2.0,
> > >> >>> > > >>> > > > > but i'd like to see as blocker for the final 2.0
> > (not
> > >> the
> > >> >>> > > merge)
> > >> >>> > > >>> > > > >  - the backup code moved in an hbase-backup module
> > >> >>> > > >>> > > > >  - and some more work around tools, especially to
> > try
> > >> to
> > >> >>> > unify
> > >> >>> > > >>> and
> > >> >>> > > >>> > make
> > >> >>> > > >>> > > > > simple the backup experience (simple example: in
> > some
> > >> >>> case
> > >> >>> > > there
> > >> >>> > > >>> is a
> > >> >>> > > >>> > > > > backup_id argument in others a backupId argument.
> or
> > >> >>> things
> > >> >>> > > >>> like..
> > >> >>> > > >>> > > > restore
> > >> >>> > > >>> > > > > is not clear if given an incremental id it will do
> > the
> > >> >>> full
> > >> >>> > > >>> restore
> > >> >>> > > >>> > > from
> > >> >>> > > >>> > > > > full up to that point or if i need to apply
> manually
> > >> >>> > > everything).
> > >> >>> > > >>> > > > >
> > >> >>> > > >>> > > > > in conclusion, I think we can open a merge vote.
> > I'll
> > >> be
> > >> >>> +1
> > >> >>> > on
> > >> >>> > > >>> it,
> > >> >>> > > >>> > and
> > >> >>> > > >>> > > I
> > >> >>> > > >>> > > > > think we should try to reject -1 with just a "code
> > >> >>> cleanup"
> > >> >>> > > >>> > motivation,
> > >> >>> > > >>> > > > > since there will still be work going on on the
> code
> > >> >>> after the
> > >> >>> > > >>> merge.
> > >> >>> > > >>> > > > >
> > >> >>> > > >>> > > > > Matteo
> > >> >>> > > >>> > > > >
> > >> >>> > > >>> > > > >
> > >> >>> > > >>> > > > > On Sun, Nov 6, 2016 at 10:54 PM, Devaraj Das <
> > >> >>> > > >>> ddas@hortonworks.com>
> > >> >>> > > >>> > > > wrote:
> > >> >>> > > >>> > > > >
> > >> >>> > > >>> > > > > > Stack and others, anything else on the patch?
> > Merge
> > >> to
> > >> >>> > master
> > >> >>> > > >>> now?
> > >> >>> > > >>> > > > > >
> > >> >>> > > >>> > > > >
> > >> >>> > > >>> > > >
> > >> >>> > > >>> > >
> > >> >>> > > >>> >
> > >> >>> > > >>>
> > >> >>> > > >>
> > >> >>> > > >>
> > >> >>> > > >
> > >> >>> > >
> > >> >>> >
> > >> >>>
> > >> >>
> > >> >>
> > >> >
> > >>
> >
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Vladimir Rodionov <vl...@gmail.com>]

Stack, Andrew

We have doc blocker and (partially) HBASE-15227: two sub-tasks remain: one
is unit test (you can't call it blocker)
and another for FT support during incremental backup with bulk loading. The
latter one have been probably addressed
already in other HBASE-15527 subtasks. I have to reassess this.

That is mostly it. Yes, We have not done real testing with real data on a
real cluster yet, except QA  testing on a small OpenStack
cluster (10 nodes). That is our probably the biggest minus right now. I
would like to inform community that this week we are going to start
full scale testing with reasonably sized data sets.

The recent committed improvements, such as ability to run backup/restore on
a particular Yarn pool (queue) allows precise control
of a cluster utilization during operation (not to interfere much with a
regular cluster operations). Another one -
 converting WAL on the fly to HFiles - significantly improves storage usage
on a backup site.

My plan is to finish HBASE-17825 (further performance optimizations). This
will cut down number of MR jobs during incremental backup
from 2*N to 2  (N - number of tables). That will probably take 2-3 more days

Then:

1. Address remaining two sub-tasks in HBASE-15227
2. Update Release notes for all relevant B&R JIRAs
3. Work on doc

After that we can call it feature full complete. Taking into account the
vast amount of efforts
spent on this feature (including QA testing) I would say that we are
probably quite close to GA right now, but only
after real testing is done (I do not anticipate significant issues, except
probably correct failure handling).

On a feature itself. We provide tools to fully automate backup and restore
tasks: create backup (full and incremental), restore
from image, delete backups, merge backups, history, history per table,
backup set management.

Hopefully, my write up addresses at least some of your concerns.

-Vlad

On Sun, Sep 10, 2017 at 6:27 AM, Josh Elser <el...@apache.org> wrote:

> On Sat, Sep 9, 2017 at 7:04 PM, stack <sa...@gmail.com> wrote:
> > In spite of repeated requests for eng summary of state of this feature --
> > summary of what is in 2.0, what is not, what the capabilities are, how
> well
> > it has been tested and at what scale -- all I get, when the requests are
> > not ignored, are pointers to lists of ill-describing jiras and some
> pending
> > user facing doc update.
>
> Yes, this is a problem. We, especially you as RM, shouldn't have
> outstanding questions as to the quality/state of B&R.
>
> > For other features, mob or region server groups, I know that they have
> been
> > running at scale in production for as much as a year and more. I have
> some
> > confidence these items basically work.  For backup/restore I have no such
> > sense even after spending time in review and trying to use the feature.
>
> I can attest to the feature being tested on small clusters. I'm not
> sure about larger than 10node tests. If this is less a worry and more
> a veto, let's get some criteria on the kind of testing you're looking
> for to avoid having to rehash later.
>
> Do we have any kind of integration tests in the codebase now that can
> help increase Stack's confidence?
>
> > As release manager, I have say over what makes it into a release.  Unless
> > the work is done to convince me that backup/restore is more than a lump
> of
> > code and a few unit tests that can pass on some fellows laptop, I am
> going
> > to kick it out of branch-2.  Let the feature harden more in master branch
> > before it ships in a release.
>
> While it was a few months ago now, I can also attest to this being
> more than some unit tests (I think I looked at it after I saw you last
> down in the weeds).
>
> I do worry about trying to remove it at this state.
>
> * Do you consider the B&R code in the repository implicitly harmful?
> Is there harm in shipping with docs capturing the concern.
> * Trying to revert all relevant pieces from branch-2 is non-trivial.
> * I would feel quite dejected if some feature I spent a year+ working
> on (*not* making assertions on my perception of quality) was removed
> from the release line it was expected to land.
>
> > S
> >
> > On Sep 8, 2017 10:59 PM, "Vladimir Rodionov" <vl...@gmail.com>
> wrote:
> >
> >> >> Have I grasped the state of things correctly, Vlad?
> >>
> >> Josh, the only thing which is still pending is doc update. All other
> >> features are good to have but not a blockers for 2.0 release.
> >>
> >> -Vlad
> >>
> >> On Fri, Sep 8, 2017 at 10:42 PM, Vladimir Rodionov <
> vladrodionov@gmail.com
> >> >
> >> wrote:
> >>
> >> > >> What testing and at what
> >> > >> scale has testing been done?
> >> >
> >> > Do we have have that for other features?
> >> >
> >> >
> >> > On Fri, Sep 8, 2017 at 10:41 PM, Vladimir Rodionov <
> >> vladrodionov@gmail.com
> >> > > wrote:
> >> >
> >> >> >> It asks: "How do I figure what of backup/restore feature is going
> to
> >> >> be in
> >> >> >>hbase-2.0.0?
> >> >>
> >> >> Hmm, wait for doc update.
> >> >>
> >> >>
> >> >> On Fri, Sep 8, 2017 at 2:39 PM, Stack <st...@duboce.net> wrote:
> >> >>
> >> >>> HBASE-14414 is a JIRA with a list of random seeming issues w/
> >> >>> non-descript
> >> >>> summaries: "Add nonce support to TableBackupProcedure, BackupID must
> >> >>> include backup set name, ...". The last comment in that issue is
> from
> >> >>> July.
> >> >>> It asks: "How do I figure what of backup/restore feature is going
> to be
> >> >>> in
> >> >>> hbase-2.0.0? Thanks Vladimir Rodionov
> >> >>> <https://issues.apache.org/jira/secure/ViewProfile.jspa?
> name=vrodionov
> >> >>> >."
> >> >>> to which there is no answer.  Doc update is TODO.
> >> >>>
> >> >>> Where is the summary of the capability in hbase-2? What testing and
> at
> >> >>> what
> >> >>> scale has testing been done? Is this 'stable or experimental'? If I
> >> can't
> >> >>> get basic info on this feature though I ask repeatedly, what hope
> does
> >> >>> the
> >> >>> poor old operator have?
> >> >>>
> >> >>> St.Ack
> >> >>>
> >> >>>
> >> >>> On Fri, Sep 8, 2017 at 1:59 PM, Vladimir Rodionov <
> >> >>> vladrodionov@gmail.com>
> >> >>> wrote:
> >> >>>
> >> >>> > HBASE-14414
> >> >>> >
> >> >>> > On Fri, Sep 8, 2017 at 1:14 PM, Stack <st...@duboce.net> wrote:
> >> >>> >
> >> >>> > > Where do I go to get the current status of this feature?
> Looking in
> >> >>> JIRA
> >> >>> > I
> >> >>> > > see loads of issues open against backup including some against
> >> >>> > hbase-2.0.0
> >> >>> > > and no progress being made that I can discern.
> >> >>> > >
> >> >>> > > Thanks,
> >> >>> > > S
> >> >>> > >
> >> >>> > >
> >> >>> > >
> >> >>> > > On Wed, Nov 23, 2016 at 8:52 AM, Stack <st...@duboce.net>
> wrote:
> >> >>> > >
> >> >>> > > > On Tue, Nov 22, 2016 at 6:48 PM, Stack <st...@duboce.net>
> wrote:
> >> >>> > > >
> >> >>> > > >> On Tue, Nov 22, 2016 at 3:17 PM, Vladimir Rodionov <
> >> >>> > > >> vladrodionov@gmail.com> wrote:
> >> >>> > > >>
> >> >>> > > >>> >> and/or he answered most of the review feedback
> >> >>> > > >>>
> >> >>> > > >>> No, questions are still open, but I do not see any blockers
> and
> >> >>> we
> >> >>> > have
> >> >>> > > >>> HBASE-16940 to address these questions.
> >> >>> > > >>>
> >> >>> > > >>>
> >> >>> > > >> Agree. No blockers but stuff that should be dealt with (No
> one
> >> >>> will
> >> >>> > pay
> >> >>> > > >> me any attention once merge goes in -- smile).
> >> >>> > > >>
> >> >>> > > >>
> >> >>> > > > Let me clarify the above. I want review addressed before merge
> >> >>> happens.
> >> >>> > > > Sorry if any confusion.
> >> >>> > > > St.Ack
> >> >>> > > >
> >> >>> > > >
> >> >>> > > >
> >> >>> > > >
> >> >>> > > >
> >> >>> > > >
> >> >>> > > >> St.Ack
> >> >>> > > >>
> >> >>> > > >>
> >> >>> > > >>
> >> >>> > > >>> On Tue, Nov 22, 2016 at 3:04 PM, Devaraj Das <
> >> >>> ddas@hortonworks.com>
> >> >>> > > >>> wrote:
> >> >>> > > >>>
> >> >>> > > >>> > Hi Stack, hats off to you for spending so much time on
> this!
> >> >>> > Thanks!
> >> >>> > > >>> From
> >> >>> > > >>> > my understanding, Vlad has raised follow-up jiras for the
> >> >>> issues
> >> >>> > you
> >> >>> > > >>> > raised, and/or he answered most of the review feedback.
> So,
> >> do
> >> >>> you
> >> >>> > > >>> think we
> >> >>> > > >>> > could do a merge vote now?
> >> >>> > > >>> > Devaraj.
> >> >>> > > >>> > ________________________________________
> >> >>> > > >>> > From: Vladimir Rodionov <vl...@gmail.com>
> >> >>> > > >>> > Sent: Monday, November 21, 2016 8:34 PM
> >> >>> > > >>> > To: dev@hbase.apache.org
> >> >>> > > >>> > Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch
> >> >>> > HBASE-7912
> >> >>> > > >>> >
> >> >>> > > >>> > >> I have spent a good bit of time reviewing and testing
> this
> >> >>> > > feature.
> >> >>> > > >>> I
> >> >>> > > >>> > would
> >> >>> > > >>> > >> like my review and concerns addressed and I'd like it
> to
> >> be
> >> >>> > clear
> >> >>> > > >>> how;
> >> >>> > > >>> > >> either explicit follow-on issues, pointers to where in
> the
> >> >>> patch
> >> >>> > > or
> >> >>> > > >>> doc
> >> >>> > > >>> > my
> >> >>> > > >>> > >> remarks have been catered to, etc. Until then, I am
> >> against
> >> >>> > > commit.
> >> >>> > > >>> >
> >> >>> > > >>> > Stack, mega patch review comments will be addressed in the
> >> >>> > dedicated
> >> >>> > > >>> JIRA:
> >> >>> > > >>> > HBASE-16940
> >> >>> > > >>> > I have open several other JIRAs to address your other
> >> comments
> >> >>> (not
> >> >>> > > on
> >> >>> > > >>> > review board).
> >> >>> > > >>> >
> >> >>> > > >>> > Details are here (end of the thread):
> >> >>> > > >>> > https://issues.apache.org/jira/browse/HBASE-14123
> >> >>> > > >>> >
> >> >>> > > >>> > Let me know what else should we do to move merge forward.
> >> >>> > > >>> >
> >> >>> > > >>> > -Vlad
> >> >>> > > >>> >
> >> >>> > > >>> >
> >> >>> > > >>> > On Fri, Nov 18, 2016 at 4:54 PM, Stack <st...@duboce.net>
> >> >>> wrote:
> >> >>> > > >>> >
> >> >>> > > >>> > > On Fri, Nov 18, 2016 at 3:53 PM, Ted Yu <
> >> yuzhihong@gmail.com
> >> >>> >
> >> >>> > > wrote:
> >> >>> > > >>> > >
> >> >>> > > >>> > > > Thanks, Matteo.
> >> >>> > > >>> > > >
> >> >>> > > >>> > > > bq. restore is not clear if given an incremental id it
> >> >>> will do
> >> >>> > > the
> >> >>> > > >>> full
> >> >>> > > >>> > > > restore from full up to that point or if i need to
> apply
> >> >>> > manually
> >> >>> > > >>> > > > everything
> >> >>> > > >>> > > >
> >> >>> > > >>> > > > The restore takes into consideration of the dependent
> >> >>> > backup(s).
> >> >>> > > >>> > > > So there is no need to apply preceding backup(s)
> >> manually.
> >> >>> > > >>> > > >
> >> >>> > > >>> > > >
> >> >>> > > >>> > > I ask this question on the issue. It is not clear from
> the
> >> >>> usage
> >> >>> > or
> >> >>> > > >>> doc
> >> >>> > > >>> > how
> >> >>> > > >>> > > to run a restore from incremental. Can you fix in doc
> and
> >> >>> usage
> >> >>> > how
> >> >>> > > >>> so I
> >> >>> > > >>> > > can be clear and try it. Currently I am stuck verifying
> a
> >> >>> round
> >> >>> > > trip
> >> >>> > > >>> > backup
> >> >>> > > >>> > > restore made of incrementals.
> >> >>> > > >>> > >
> >> >>> > > >>> > > Thanks,
> >> >>> > > >>> > > S
> >> >>> > > >>> > >
> >> >>> > > >>> > >
> >> >>> > > >>> > >
> >> >>> > > >>> > > > On Fri, Nov 18, 2016 at 3:48 PM, Matteo Bertozzi <
> >> >>> > > >>> > > theo.bertozzi@gmail.com>
> >> >>> > > >>> > > > wrote:
> >> >>> > > >>> > > >
> >> >>> > > >>> > > > > I did one last pass to the mega patch. I don't see
> >> >>> anything
> >> >>> > > major
> >> >>> > > >>> > that
> >> >>> > > >>> > > > > should block the merge.
> >> >>> > > >>> > > > >
> >> >>> > > >>> > > > > - most of the code is isolated in the backup package
> >> >>> > > >>> > > > > - all the backup code is client side
> >> >>> > > >>> > > > > - there are few changes to the server side, mainly
> for
> >> >>> > > cleaners,
> >> >>> > > >>> wal
> >> >>> > > >>> > > > > rolling and similar (which is ok)
> >> >>> > > >>> > > > > - there is a good number of tests, and an
> integration
> >> >>> test
> >> >>> > > >>> > > > >
> >> >>> > > >>> > > > > the code seems to have still some left overs from
> the
> >> old
> >> >>> > > >>> > > implementation,
> >> >>> > > >>> > > > > and some stuff needs a cleanup. but I don't think
> this
> >> >>> should
> >> >>> > > be
> >> >>> > > >>> used
> >> >>> > > >>> > > as
> >> >>> > > >>> > > > an
> >> >>> > > >>> > > > > argument to block the merge. I think the guys will
> keep
> >> >>> > working
> >> >>> > > >>> on
> >> >>> > > >>> > this
> >> >>> > > >>> > > > and
> >> >>> > > >>> > > > > they may also get help of others once the patch is
> in
> >> >>> master.
> >> >>> > > >>> > > > >
> >> >>> > > >>> > > > > I still have my concerns about the current
> limitations,
> >> >>> but
> >> >>> > > >>> these are
> >> >>> > > >>> > > > > things already planned for phase 3, so some of this
> >> >>> stuff may
> >> >>> > > >>> even be
> >> >>> > > >>> > > in
> >> >>> > > >>> > > > > the final 2.0.
> >> >>> > > >>> > > > > but as long as we have a "current limitations"
> section
> >> >>> in the
> >> >>> > > >>> user
> >> >>> > > >>> > > guide
> >> >>> > > >>> > > > > mentioning important stuff like the ones below, I'm
> ok
> >> >>> with
> >> >>> > it.
> >> >>> > > >>> > > > >  - if you write to the table with
> Durability.SKIP_WALS
> >> >>> your
> >> >>> > > data
> >> >>> > > >>> will
> >> >>> > > >>> > > not
> >> >>> > > >>> > > > > be in the incremental-backup
> >> >>> > > >>> > > > >  - if you bulkload files that data will not be in
> the
> >> >>> > > incremental
> >> >>> > > >>> > > backup
> >> >>> > > >>> > > > > (HBASE-14417)
> >> >>> > > >>> > > > >  - the incremental backup will not only contains the
> >> >>> data of
> >> >>> > > the
> >> >>> > > >>> > table
> >> >>> > > >>> > > > you
> >> >>> > > >>> > > > > specified but also the regions from other tables
> that
> >> >>> are on
> >> >>> > > the
> >> >>> > > >>> same
> >> >>> > > >>> > > set
> >> >>> > > >>> > > > > of RSs (HBASE-14141) ...maybe a note about security
> >> >>> around
> >> >>> > this
> >> >>> > > >>> topic
> >> >>> > > >>> > > > >  - the incremental backup will not contains just the
> >> >>> "latest
> >> >>> > > row"
> >> >>> > > >>> > > between
> >> >>> > > >>> > > > > backup A and B, but it will also contains all the
> >> updates
> >> >>> > > >>> occurred in
> >> >>> > > >>> > > > > between. but the restore does not allow you to
> restore
> >> >>> up to
> >> >>> > a
> >> >>> > > >>> > certain
> >> >>> > > >>> > > > > point in time, the restore will always be up to the
> >> >>> "latest
> >> >>> > > >>> backup
> >> >>> > > >>> > > > point".
> >> >>> > > >>> > > > >  - you should limit the number of "incremental" up
> to N
> >> >>> (or
> >> >>> > > maybe
> >> >>> > > >>> > > SIZE),
> >> >>> > > >>> > > > to
> >> >>> > > >>> > > > > avoid replay time becoming the bottleneck.
> >> (HBASE-14135)
> >> >>> > > >>> > > > >
> >> >>> > > >>> > > > > I'll be ok even with the above not being in the
> final
> >> >>> 2.0,
> >> >>> > > >>> > > > > but i'd like to see as blocker for the final 2.0
> (not
> >> the
> >> >>> > > merge)
> >> >>> > > >>> > > > >  - the backup code moved in an hbase-backup module
> >> >>> > > >>> > > > >  - and some more work around tools, especially to
> try
> >> to
> >> >>> > unify
> >> >>> > > >>> and
> >> >>> > > >>> > make
> >> >>> > > >>> > > > > simple the backup experience (simple example: in
> some
> >> >>> case
> >> >>> > > there
> >> >>> > > >>> is a
> >> >>> > > >>> > > > > backup_id argument in others a backupId argument. or
> >> >>> things
> >> >>> > > >>> like..
> >> >>> > > >>> > > > restore
> >> >>> > > >>> > > > > is not clear if given an incremental id it will do
> the
> >> >>> full
> >> >>> > > >>> restore
> >> >>> > > >>> > > from
> >> >>> > > >>> > > > > full up to that point or if i need to apply manually
> >> >>> > > everything).
> >> >>> > > >>> > > > >
> >> >>> > > >>> > > > > in conclusion, I think we can open a merge vote.
> I'll
> >> be
> >> >>> +1
> >> >>> > on
> >> >>> > > >>> it,
> >> >>> > > >>> > and
> >> >>> > > >>> > > I
> >> >>> > > >>> > > > > think we should try to reject -1 with just a "code
> >> >>> cleanup"
> >> >>> > > >>> > motivation,
> >> >>> > > >>> > > > > since there will still be work going on on the code
> >> >>> after the
> >> >>> > > >>> merge.
> >> >>> > > >>> > > > >
> >> >>> > > >>> > > > > Matteo
> >> >>> > > >>> > > > >
> >> >>> > > >>> > > > >
> >> >>> > > >>> > > > > On Sun, Nov 6, 2016 at 10:54 PM, Devaraj Das <
> >> >>> > > >>> ddas@hortonworks.com>
> >> >>> > > >>> > > > wrote:
> >> >>> > > >>> > > > >
> >> >>> > > >>> > > > > > Stack and others, anything else on the patch?
> Merge
> >> to
> >> >>> > master
> >> >>> > > >>> now?
> >> >>> > > >>> > > > > >
> >> >>> > > >>> > > > >
> >> >>> > > >>> > > >
> >> >>> > > >>> > >
> >> >>> > > >>> >
> >> >>> > > >>>
> >> >>> > > >>
> >> >>> > > >>
> >> >>> > > >
> >> >>> > >
> >> >>> >
> >> >>>
> >> >>
> >> >>
> >> >
> >>
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Josh Elser <el...@apache.org>]

On Sat, Sep 9, 2017 at 7:04 PM, stack <sa...@gmail.com> wrote:
> In spite of repeated requests for eng summary of state of this feature --
> summary of what is in 2.0, what is not, what the capabilities are, how well
> it has been tested and at what scale -- all I get, when the requests are
> not ignored, are pointers to lists of ill-describing jiras and some pending
> user facing doc update.

Yes, this is a problem. We, especially you as RM, shouldn't have
outstanding questions as to the quality/state of B&R.

> For other features, mob or region server groups, I know that they have been
> running at scale in production for as much as a year and more. I have some
> confidence these items basically work.  For backup/restore I have no such
> sense even after spending time in review and trying to use the feature.

I can attest to the feature being tested on small clusters. I'm not
sure about larger than 10node tests. If this is less a worry and more
a veto, let's get some criteria on the kind of testing you're looking
for to avoid having to rehash later.

Do we have any kind of integration tests in the codebase now that can
help increase Stack's confidence?

> As release manager, I have say over what makes it into a release.  Unless
> the work is done to convince me that backup/restore is more than a lump of
> code and a few unit tests that can pass on some fellows laptop, I am going
> to kick it out of branch-2.  Let the feature harden more in master branch
> before it ships in a release.

While it was a few months ago now, I can also attest to this being
more than some unit tests (I think I looked at it after I saw you last
down in the weeds).

I do worry about trying to remove it at this state.

* Do you consider the B&R code in the repository implicitly harmful?
Is there harm in shipping with docs capturing the concern.
* Trying to revert all relevant pieces from branch-2 is non-trivial.
* I would feel quite dejected if some feature I spent a year+ working
on (*not* making assertions on my perception of quality) was removed
from the release line it was expected to land.

> S
>
> On Sep 8, 2017 10:59 PM, "Vladimir Rodionov" <vl...@gmail.com> wrote:
>
>> >> Have I grasped the state of things correctly, Vlad?
>>
>> Josh, the only thing which is still pending is doc update. All other
>> features are good to have but not a blockers for 2.0 release.
>>
>> -Vlad
>>
>> On Fri, Sep 8, 2017 at 10:42 PM, Vladimir Rodionov <vladrodionov@gmail.com
>> >
>> wrote:
>>
>> > >> What testing and at what
>> > >> scale has testing been done?
>> >
>> > Do we have have that for other features?
>> >
>> >
>> > On Fri, Sep 8, 2017 at 10:41 PM, Vladimir Rodionov <
>> vladrodionov@gmail.com
>> > > wrote:
>> >
>> >> >> It asks: "How do I figure what of backup/restore feature is going to
>> >> be in
>> >> >>hbase-2.0.0?
>> >>
>> >> Hmm, wait for doc update.
>> >>
>> >>
>> >> On Fri, Sep 8, 2017 at 2:39 PM, Stack <st...@duboce.net> wrote:
>> >>
>> >>> HBASE-14414 is a JIRA with a list of random seeming issues w/
>> >>> non-descript
>> >>> summaries: "Add nonce support to TableBackupProcedure, BackupID must
>> >>> include backup set name, ...". The last comment in that issue is from
>> >>> July.
>> >>> It asks: "How do I figure what of backup/restore feature is going to be
>> >>> in
>> >>> hbase-2.0.0? Thanks Vladimir Rodionov
>> >>> <https://issues.apache.org/jira/secure/ViewProfile.jspa?name=vrodionov
>> >>> >."
>> >>> to which there is no answer.  Doc update is TODO.
>> >>>
>> >>> Where is the summary of the capability in hbase-2? What testing and at
>> >>> what
>> >>> scale has testing been done? Is this 'stable or experimental'? If I
>> can't
>> >>> get basic info on this feature though I ask repeatedly, what hope does
>> >>> the
>> >>> poor old operator have?
>> >>>
>> >>> St.Ack
>> >>>
>> >>>
>> >>> On Fri, Sep 8, 2017 at 1:59 PM, Vladimir Rodionov <
>> >>> vladrodionov@gmail.com>
>> >>> wrote:
>> >>>
>> >>> > HBASE-14414
>> >>> >
>> >>> > On Fri, Sep 8, 2017 at 1:14 PM, Stack <st...@duboce.net> wrote:
>> >>> >
>> >>> > > Where do I go to get the current status of this feature? Looking in
>> >>> JIRA
>> >>> > I
>> >>> > > see loads of issues open against backup including some against
>> >>> > hbase-2.0.0
>> >>> > > and no progress being made that I can discern.
>> >>> > >
>> >>> > > Thanks,
>> >>> > > S
>> >>> > >
>> >>> > >
>> >>> > >
>> >>> > > On Wed, Nov 23, 2016 at 8:52 AM, Stack <st...@duboce.net> wrote:
>> >>> > >
>> >>> > > > On Tue, Nov 22, 2016 at 6:48 PM, Stack <st...@duboce.net> wrote:
>> >>> > > >
>> >>> > > >> On Tue, Nov 22, 2016 at 3:17 PM, Vladimir Rodionov <
>> >>> > > >> vladrodionov@gmail.com> wrote:
>> >>> > > >>
>> >>> > > >>> >> and/or he answered most of the review feedback
>> >>> > > >>>
>> >>> > > >>> No, questions are still open, but I do not see any blockers and
>> >>> we
>> >>> > have
>> >>> > > >>> HBASE-16940 to address these questions.
>> >>> > > >>>
>> >>> > > >>>
>> >>> > > >> Agree. No blockers but stuff that should be dealt with (No one
>> >>> will
>> >>> > pay
>> >>> > > >> me any attention once merge goes in -- smile).
>> >>> > > >>
>> >>> > > >>
>> >>> > > > Let me clarify the above. I want review addressed before merge
>> >>> happens.
>> >>> > > > Sorry if any confusion.
>> >>> > > > St.Ack
>> >>> > > >
>> >>> > > >
>> >>> > > >
>> >>> > > >
>> >>> > > >
>> >>> > > >
>> >>> > > >> St.Ack
>> >>> > > >>
>> >>> > > >>
>> >>> > > >>
>> >>> > > >>> On Tue, Nov 22, 2016 at 3:04 PM, Devaraj Das <
>> >>> ddas@hortonworks.com>
>> >>> > > >>> wrote:
>> >>> > > >>>
>> >>> > > >>> > Hi Stack, hats off to you for spending so much time on this!
>> >>> > Thanks!
>> >>> > > >>> From
>> >>> > > >>> > my understanding, Vlad has raised follow-up jiras for the
>> >>> issues
>> >>> > you
>> >>> > > >>> > raised, and/or he answered most of the review feedback. So,
>> do
>> >>> you
>> >>> > > >>> think we
>> >>> > > >>> > could do a merge vote now?
>> >>> > > >>> > Devaraj.
>> >>> > > >>> > ________________________________________
>> >>> > > >>> > From: Vladimir Rodionov <vl...@gmail.com>
>> >>> > > >>> > Sent: Monday, November 21, 2016 8:34 PM
>> >>> > > >>> > To: dev@hbase.apache.org
>> >>> > > >>> > Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch
>> >>> > HBASE-7912
>> >>> > > >>> >
>> >>> > > >>> > >> I have spent a good bit of time reviewing and testing this
>> >>> > > feature.
>> >>> > > >>> I
>> >>> > > >>> > would
>> >>> > > >>> > >> like my review and concerns addressed and I'd like it to
>> be
>> >>> > clear
>> >>> > > >>> how;
>> >>> > > >>> > >> either explicit follow-on issues, pointers to where in the
>> >>> patch
>> >>> > > or
>> >>> > > >>> doc
>> >>> > > >>> > my
>> >>> > > >>> > >> remarks have been catered to, etc. Until then, I am
>> against
>> >>> > > commit.
>> >>> > > >>> >
>> >>> > > >>> > Stack, mega patch review comments will be addressed in the
>> >>> > dedicated
>> >>> > > >>> JIRA:
>> >>> > > >>> > HBASE-16940
>> >>> > > >>> > I have open several other JIRAs to address your other
>> comments
>> >>> (not
>> >>> > > on
>> >>> > > >>> > review board).
>> >>> > > >>> >
>> >>> > > >>> > Details are here (end of the thread):
>> >>> > > >>> > https://issues.apache.org/jira/browse/HBASE-14123
>> >>> > > >>> >
>> >>> > > >>> > Let me know what else should we do to move merge forward.
>> >>> > > >>> >
>> >>> > > >>> > -Vlad
>> >>> > > >>> >
>> >>> > > >>> >
>> >>> > > >>> > On Fri, Nov 18, 2016 at 4:54 PM, Stack <st...@duboce.net>
>> >>> wrote:
>> >>> > > >>> >
>> >>> > > >>> > > On Fri, Nov 18, 2016 at 3:53 PM, Ted Yu <
>> yuzhihong@gmail.com
>> >>> >
>> >>> > > wrote:
>> >>> > > >>> > >
>> >>> > > >>> > > > Thanks, Matteo.
>> >>> > > >>> > > >
>> >>> > > >>> > > > bq. restore is not clear if given an incremental id it
>> >>> will do
>> >>> > > the
>> >>> > > >>> full
>> >>> > > >>> > > > restore from full up to that point or if i need to apply
>> >>> > manually
>> >>> > > >>> > > > everything
>> >>> > > >>> > > >
>> >>> > > >>> > > > The restore takes into consideration of the dependent
>> >>> > backup(s).
>> >>> > > >>> > > > So there is no need to apply preceding backup(s)
>> manually.
>> >>> > > >>> > > >
>> >>> > > >>> > > >
>> >>> > > >>> > > I ask this question on the issue. It is not clear from the
>> >>> usage
>> >>> > or
>> >>> > > >>> doc
>> >>> > > >>> > how
>> >>> > > >>> > > to run a restore from incremental. Can you fix in doc and
>> >>> usage
>> >>> > how
>> >>> > > >>> so I
>> >>> > > >>> > > can be clear and try it. Currently I am stuck verifying a
>> >>> round
>> >>> > > trip
>> >>> > > >>> > backup
>> >>> > > >>> > > restore made of incrementals.
>> >>> > > >>> > >
>> >>> > > >>> > > Thanks,
>> >>> > > >>> > > S
>> >>> > > >>> > >
>> >>> > > >>> > >
>> >>> > > >>> > >
>> >>> > > >>> > > > On Fri, Nov 18, 2016 at 3:48 PM, Matteo Bertozzi <
>> >>> > > >>> > > theo.bertozzi@gmail.com>
>> >>> > > >>> > > > wrote:
>> >>> > > >>> > > >
>> >>> > > >>> > > > > I did one last pass to the mega patch. I don't see
>> >>> anything
>> >>> > > major
>> >>> > > >>> > that
>> >>> > > >>> > > > > should block the merge.
>> >>> > > >>> > > > >
>> >>> > > >>> > > > > - most of the code is isolated in the backup package
>> >>> > > >>> > > > > - all the backup code is client side
>> >>> > > >>> > > > > - there are few changes to the server side, mainly for
>> >>> > > cleaners,
>> >>> > > >>> wal
>> >>> > > >>> > > > > rolling and similar (which is ok)
>> >>> > > >>> > > > > - there is a good number of tests, and an integration
>> >>> test
>> >>> > > >>> > > > >
>> >>> > > >>> > > > > the code seems to have still some left overs from the
>> old
>> >>> > > >>> > > implementation,
>> >>> > > >>> > > > > and some stuff needs a cleanup. but I don't think this
>> >>> should
>> >>> > > be
>> >>> > > >>> used
>> >>> > > >>> > > as
>> >>> > > >>> > > > an
>> >>> > > >>> > > > > argument to block the merge. I think the guys will keep
>> >>> > working
>> >>> > > >>> on
>> >>> > > >>> > this
>> >>> > > >>> > > > and
>> >>> > > >>> > > > > they may also get help of others once the patch is in
>> >>> master.
>> >>> > > >>> > > > >
>> >>> > > >>> > > > > I still have my concerns about the current limitations,
>> >>> but
>> >>> > > >>> these are
>> >>> > > >>> > > > > things already planned for phase 3, so some of this
>> >>> stuff may
>> >>> > > >>> even be
>> >>> > > >>> > > in
>> >>> > > >>> > > > > the final 2.0.
>> >>> > > >>> > > > > but as long as we have a "current limitations" section
>> >>> in the
>> >>> > > >>> user
>> >>> > > >>> > > guide
>> >>> > > >>> > > > > mentioning important stuff like the ones below, I'm ok
>> >>> with
>> >>> > it.
>> >>> > > >>> > > > >  - if you write to the table with Durability.SKIP_WALS
>> >>> your
>> >>> > > data
>> >>> > > >>> will
>> >>> > > >>> > > not
>> >>> > > >>> > > > > be in the incremental-backup
>> >>> > > >>> > > > >  - if you bulkload files that data will not be in the
>> >>> > > incremental
>> >>> > > >>> > > backup
>> >>> > > >>> > > > > (HBASE-14417)
>> >>> > > >>> > > > >  - the incremental backup will not only contains the
>> >>> data of
>> >>> > > the
>> >>> > > >>> > table
>> >>> > > >>> > > > you
>> >>> > > >>> > > > > specified but also the regions from other tables that
>> >>> are on
>> >>> > > the
>> >>> > > >>> same
>> >>> > > >>> > > set
>> >>> > > >>> > > > > of RSs (HBASE-14141) ...maybe a note about security
>> >>> around
>> >>> > this
>> >>> > > >>> topic
>> >>> > > >>> > > > >  - the incremental backup will not contains just the
>> >>> "latest
>> >>> > > row"
>> >>> > > >>> > > between
>> >>> > > >>> > > > > backup A and B, but it will also contains all the
>> updates
>> >>> > > >>> occurred in
>> >>> > > >>> > > > > between. but the restore does not allow you to restore
>> >>> up to
>> >>> > a
>> >>> > > >>> > certain
>> >>> > > >>> > > > > point in time, the restore will always be up to the
>> >>> "latest
>> >>> > > >>> backup
>> >>> > > >>> > > > point".
>> >>> > > >>> > > > >  - you should limit the number of "incremental" up to N
>> >>> (or
>> >>> > > maybe
>> >>> > > >>> > > SIZE),
>> >>> > > >>> > > > to
>> >>> > > >>> > > > > avoid replay time becoming the bottleneck.
>> (HBASE-14135)
>> >>> > > >>> > > > >
>> >>> > > >>> > > > > I'll be ok even with the above not being in the final
>> >>> 2.0,
>> >>> > > >>> > > > > but i'd like to see as blocker for the final 2.0 (not
>> the
>> >>> > > merge)
>> >>> > > >>> > > > >  - the backup code moved in an hbase-backup module
>> >>> > > >>> > > > >  - and some more work around tools, especially to try
>> to
>> >>> > unify
>> >>> > > >>> and
>> >>> > > >>> > make
>> >>> > > >>> > > > > simple the backup experience (simple example: in some
>> >>> case
>> >>> > > there
>> >>> > > >>> is a
>> >>> > > >>> > > > > backup_id argument in others a backupId argument. or
>> >>> things
>> >>> > > >>> like..
>> >>> > > >>> > > > restore
>> >>> > > >>> > > > > is not clear if given an incremental id it will do the
>> >>> full
>> >>> > > >>> restore
>> >>> > > >>> > > from
>> >>> > > >>> > > > > full up to that point or if i need to apply manually
>> >>> > > everything).
>> >>> > > >>> > > > >
>> >>> > > >>> > > > > in conclusion, I think we can open a merge vote. I'll
>> be
>> >>> +1
>> >>> > on
>> >>> > > >>> it,
>> >>> > > >>> > and
>> >>> > > >>> > > I
>> >>> > > >>> > > > > think we should try to reject -1 with just a "code
>> >>> cleanup"
>> >>> > > >>> > motivation,
>> >>> > > >>> > > > > since there will still be work going on on the code
>> >>> after the
>> >>> > > >>> merge.
>> >>> > > >>> > > > >
>> >>> > > >>> > > > > Matteo
>> >>> > > >>> > > > >
>> >>> > > >>> > > > >
>> >>> > > >>> > > > > On Sun, Nov 6, 2016 at 10:54 PM, Devaraj Das <
>> >>> > > >>> ddas@hortonworks.com>
>> >>> > > >>> > > > wrote:
>> >>> > > >>> > > > >
>> >>> > > >>> > > > > > Stack and others, anything else on the patch? Merge
>> to
>> >>> > master
>> >>> > > >>> now?
>> >>> > > >>> > > > > >
>> >>> > > >>> > > > >
>> >>> > > >>> > > >
>> >>> > > >>> > >
>> >>> > > >>> >
>> >>> > > >>>
>> >>> > > >>
>> >>> > > >>
>> >>> > > >
>> >>> > >
>> >>> >
>> >>>
>> >>
>> >>
>> >
>>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[stack <sa...@gmail.com>]

In spite of repeated requests for eng summary of state of this feature --
summary of what is in 2.0, what is not, what the capabilities are, how well
it has been tested and at what scale -- all I get, when the requests are
not ignored, are pointers to lists of ill-describing jiras and some pending
user facing doc update.

For other features, mob or region server groups, I know that they have been
running at scale in production for as much as a year and more. I have some
confidence these items basically work.  For backup/restore I have no such
sense even after spending time in review and trying to use the feature.

As release manager, I have say over what makes it into a release.  Unless
the work is done to convince me that backup/restore is more than a lump of
code and a few unit tests that can pass on some fellows laptop, I am going
to kick it out of branch-2.  Let the feature harden more in master branch
before it ships in a release.

S

On Sep 8, 2017 10:59 PM, "Vladimir Rodionov" <vl...@gmail.com> wrote:

> >> Have I grasped the state of things correctly, Vlad?
>
> Josh, the only thing which is still pending is doc update. All other
> features are good to have but not a blockers for 2.0 release.
>
> -Vlad
>
> On Fri, Sep 8, 2017 at 10:42 PM, Vladimir Rodionov <vladrodionov@gmail.com
> >
> wrote:
>
> > >> What testing and at what
> > >> scale has testing been done?
> >
> > Do we have have that for other features?
> >
> >
> > On Fri, Sep 8, 2017 at 10:41 PM, Vladimir Rodionov <
> vladrodionov@gmail.com
> > > wrote:
> >
> >> >> It asks: "How do I figure what of backup/restore feature is going to
> >> be in
> >> >>hbase-2.0.0?
> >>
> >> Hmm, wait for doc update.
> >>
> >>
> >> On Fri, Sep 8, 2017 at 2:39 PM, Stack <st...@duboce.net> wrote:
> >>
> >>> HBASE-14414 is a JIRA with a list of random seeming issues w/
> >>> non-descript
> >>> summaries: "Add nonce support to TableBackupProcedure, BackupID must
> >>> include backup set name, ...". The last comment in that issue is from
> >>> July.
> >>> It asks: "How do I figure what of backup/restore feature is going to be
> >>> in
> >>> hbase-2.0.0? Thanks Vladimir Rodionov
> >>> <https://issues.apache.org/jira/secure/ViewProfile.jspa?name=vrodionov
> >>> >."
> >>> to which there is no answer.  Doc update is TODO.
> >>>
> >>> Where is the summary of the capability in hbase-2? What testing and at
> >>> what
> >>> scale has testing been done? Is this 'stable or experimental'? If I
> can't
> >>> get basic info on this feature though I ask repeatedly, what hope does
> >>> the
> >>> poor old operator have?
> >>>
> >>> St.Ack
> >>>
> >>>
> >>> On Fri, Sep 8, 2017 at 1:59 PM, Vladimir Rodionov <
> >>> vladrodionov@gmail.com>
> >>> wrote:
> >>>
> >>> > HBASE-14414
> >>> >
> >>> > On Fri, Sep 8, 2017 at 1:14 PM, Stack <st...@duboce.net> wrote:
> >>> >
> >>> > > Where do I go to get the current status of this feature? Looking in
> >>> JIRA
> >>> > I
> >>> > > see loads of issues open against backup including some against
> >>> > hbase-2.0.0
> >>> > > and no progress being made that I can discern.
> >>> > >
> >>> > > Thanks,
> >>> > > S
> >>> > >
> >>> > >
> >>> > >
> >>> > > On Wed, Nov 23, 2016 at 8:52 AM, Stack <st...@duboce.net> wrote:
> >>> > >
> >>> > > > On Tue, Nov 22, 2016 at 6:48 PM, Stack <st...@duboce.net> wrote:
> >>> > > >
> >>> > > >> On Tue, Nov 22, 2016 at 3:17 PM, Vladimir Rodionov <
> >>> > > >> vladrodionov@gmail.com> wrote:
> >>> > > >>
> >>> > > >>> >> and/or he answered most of the review feedback
> >>> > > >>>
> >>> > > >>> No, questions are still open, but I do not see any blockers and
> >>> we
> >>> > have
> >>> > > >>> HBASE-16940 to address these questions.
> >>> > > >>>
> >>> > > >>>
> >>> > > >> Agree. No blockers but stuff that should be dealt with (No one
> >>> will
> >>> > pay
> >>> > > >> me any attention once merge goes in -- smile).
> >>> > > >>
> >>> > > >>
> >>> > > > Let me clarify the above. I want review addressed before merge
> >>> happens.
> >>> > > > Sorry if any confusion.
> >>> > > > St.Ack
> >>> > > >
> >>> > > >
> >>> > > >
> >>> > > >
> >>> > > >
> >>> > > >
> >>> > > >> St.Ack
> >>> > > >>
> >>> > > >>
> >>> > > >>
> >>> > > >>> On Tue, Nov 22, 2016 at 3:04 PM, Devaraj Das <
> >>> ddas@hortonworks.com>
> >>> > > >>> wrote:
> >>> > > >>>
> >>> > > >>> > Hi Stack, hats off to you for spending so much time on this!
> >>> > Thanks!
> >>> > > >>> From
> >>> > > >>> > my understanding, Vlad has raised follow-up jiras for the
> >>> issues
> >>> > you
> >>> > > >>> > raised, and/or he answered most of the review feedback. So,
> do
> >>> you
> >>> > > >>> think we
> >>> > > >>> > could do a merge vote now?
> >>> > > >>> > Devaraj.
> >>> > > >>> > ________________________________________
> >>> > > >>> > From: Vladimir Rodionov <vl...@gmail.com>
> >>> > > >>> > Sent: Monday, November 21, 2016 8:34 PM
> >>> > > >>> > To: dev@hbase.apache.org
> >>> > > >>> > Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch
> >>> > HBASE-7912
> >>> > > >>> >
> >>> > > >>> > >> I have spent a good bit of time reviewing and testing this
> >>> > > feature.
> >>> > > >>> I
> >>> > > >>> > would
> >>> > > >>> > >> like my review and concerns addressed and I'd like it to
> be
> >>> > clear
> >>> > > >>> how;
> >>> > > >>> > >> either explicit follow-on issues, pointers to where in the
> >>> patch
> >>> > > or
> >>> > > >>> doc
> >>> > > >>> > my
> >>> > > >>> > >> remarks have been catered to, etc. Until then, I am
> against
> >>> > > commit.
> >>> > > >>> >
> >>> > > >>> > Stack, mega patch review comments will be addressed in the
> >>> > dedicated
> >>> > > >>> JIRA:
> >>> > > >>> > HBASE-16940
> >>> > > >>> > I have open several other JIRAs to address your other
> comments
> >>> (not
> >>> > > on
> >>> > > >>> > review board).
> >>> > > >>> >
> >>> > > >>> > Details are here (end of the thread):
> >>> > > >>> > https://issues.apache.org/jira/browse/HBASE-14123
> >>> > > >>> >
> >>> > > >>> > Let me know what else should we do to move merge forward.
> >>> > > >>> >
> >>> > > >>> > -Vlad
> >>> > > >>> >
> >>> > > >>> >
> >>> > > >>> > On Fri, Nov 18, 2016 at 4:54 PM, Stack <st...@duboce.net>
> >>> wrote:
> >>> > > >>> >
> >>> > > >>> > > On Fri, Nov 18, 2016 at 3:53 PM, Ted Yu <
> yuzhihong@gmail.com
> >>> >
> >>> > > wrote:
> >>> > > >>> > >
> >>> > > >>> > > > Thanks, Matteo.
> >>> > > >>> > > >
> >>> > > >>> > > > bq. restore is not clear if given an incremental id it
> >>> will do
> >>> > > the
> >>> > > >>> full
> >>> > > >>> > > > restore from full up to that point or if i need to apply
> >>> > manually
> >>> > > >>> > > > everything
> >>> > > >>> > > >
> >>> > > >>> > > > The restore takes into consideration of the dependent
> >>> > backup(s).
> >>> > > >>> > > > So there is no need to apply preceding backup(s)
> manually.
> >>> > > >>> > > >
> >>> > > >>> > > >
> >>> > > >>> > > I ask this question on the issue. It is not clear from the
> >>> usage
> >>> > or
> >>> > > >>> doc
> >>> > > >>> > how
> >>> > > >>> > > to run a restore from incremental. Can you fix in doc and
> >>> usage
> >>> > how
> >>> > > >>> so I
> >>> > > >>> > > can be clear and try it. Currently I am stuck verifying a
> >>> round
> >>> > > trip
> >>> > > >>> > backup
> >>> > > >>> > > restore made of incrementals.
> >>> > > >>> > >
> >>> > > >>> > > Thanks,
> >>> > > >>> > > S
> >>> > > >>> > >
> >>> > > >>> > >
> >>> > > >>> > >
> >>> > > >>> > > > On Fri, Nov 18, 2016 at 3:48 PM, Matteo Bertozzi <
> >>> > > >>> > > theo.bertozzi@gmail.com>
> >>> > > >>> > > > wrote:
> >>> > > >>> > > >
> >>> > > >>> > > > > I did one last pass to the mega patch. I don't see
> >>> anything
> >>> > > major
> >>> > > >>> > that
> >>> > > >>> > > > > should block the merge.
> >>> > > >>> > > > >
> >>> > > >>> > > > > - most of the code is isolated in the backup package
> >>> > > >>> > > > > - all the backup code is client side
> >>> > > >>> > > > > - there are few changes to the server side, mainly for
> >>> > > cleaners,
> >>> > > >>> wal
> >>> > > >>> > > > > rolling and similar (which is ok)
> >>> > > >>> > > > > - there is a good number of tests, and an integration
> >>> test
> >>> > > >>> > > > >
> >>> > > >>> > > > > the code seems to have still some left overs from the
> old
> >>> > > >>> > > implementation,
> >>> > > >>> > > > > and some stuff needs a cleanup. but I don't think this
> >>> should
> >>> > > be
> >>> > > >>> used
> >>> > > >>> > > as
> >>> > > >>> > > > an
> >>> > > >>> > > > > argument to block the merge. I think the guys will keep
> >>> > working
> >>> > > >>> on
> >>> > > >>> > this
> >>> > > >>> > > > and
> >>> > > >>> > > > > they may also get help of others once the patch is in
> >>> master.
> >>> > > >>> > > > >
> >>> > > >>> > > > > I still have my concerns about the current limitations,
> >>> but
> >>> > > >>> these are
> >>> > > >>> > > > > things already planned for phase 3, so some of this
> >>> stuff may
> >>> > > >>> even be
> >>> > > >>> > > in
> >>> > > >>> > > > > the final 2.0.
> >>> > > >>> > > > > but as long as we have a "current limitations" section
> >>> in the
> >>> > > >>> user
> >>> > > >>> > > guide
> >>> > > >>> > > > > mentioning important stuff like the ones below, I'm ok
> >>> with
> >>> > it.
> >>> > > >>> > > > >  - if you write to the table with Durability.SKIP_WALS
> >>> your
> >>> > > data
> >>> > > >>> will
> >>> > > >>> > > not
> >>> > > >>> > > > > be in the incremental-backup
> >>> > > >>> > > > >  - if you bulkload files that data will not be in the
> >>> > > incremental
> >>> > > >>> > > backup
> >>> > > >>> > > > > (HBASE-14417)
> >>> > > >>> > > > >  - the incremental backup will not only contains the
> >>> data of
> >>> > > the
> >>> > > >>> > table
> >>> > > >>> > > > you
> >>> > > >>> > > > > specified but also the regions from other tables that
> >>> are on
> >>> > > the
> >>> > > >>> same
> >>> > > >>> > > set
> >>> > > >>> > > > > of RSs (HBASE-14141) ...maybe a note about security
> >>> around
> >>> > this
> >>> > > >>> topic
> >>> > > >>> > > > >  - the incremental backup will not contains just the
> >>> "latest
> >>> > > row"
> >>> > > >>> > > between
> >>> > > >>> > > > > backup A and B, but it will also contains all the
> updates
> >>> > > >>> occurred in
> >>> > > >>> > > > > between. but the restore does not allow you to restore
> >>> up to
> >>> > a
> >>> > > >>> > certain
> >>> > > >>> > > > > point in time, the restore will always be up to the
> >>> "latest
> >>> > > >>> backup
> >>> > > >>> > > > point".
> >>> > > >>> > > > >  - you should limit the number of "incremental" up to N
> >>> (or
> >>> > > maybe
> >>> > > >>> > > SIZE),
> >>> > > >>> > > > to
> >>> > > >>> > > > > avoid replay time becoming the bottleneck.
> (HBASE-14135)
> >>> > > >>> > > > >
> >>> > > >>> > > > > I'll be ok even with the above not being in the final
> >>> 2.0,
> >>> > > >>> > > > > but i'd like to see as blocker for the final 2.0 (not
> the
> >>> > > merge)
> >>> > > >>> > > > >  - the backup code moved in an hbase-backup module
> >>> > > >>> > > > >  - and some more work around tools, especially to try
> to
> >>> > unify
> >>> > > >>> and
> >>> > > >>> > make
> >>> > > >>> > > > > simple the backup experience (simple example: in some
> >>> case
> >>> > > there
> >>> > > >>> is a
> >>> > > >>> > > > > backup_id argument in others a backupId argument. or
> >>> things
> >>> > > >>> like..
> >>> > > >>> > > > restore
> >>> > > >>> > > > > is not clear if given an incremental id it will do the
> >>> full
> >>> > > >>> restore
> >>> > > >>> > > from
> >>> > > >>> > > > > full up to that point or if i need to apply manually
> >>> > > everything).
> >>> > > >>> > > > >
> >>> > > >>> > > > > in conclusion, I think we can open a merge vote. I'll
> be
> >>> +1
> >>> > on
> >>> > > >>> it,
> >>> > > >>> > and
> >>> > > >>> > > I
> >>> > > >>> > > > > think we should try to reject -1 with just a "code
> >>> cleanup"
> >>> > > >>> > motivation,
> >>> > > >>> > > > > since there will still be work going on on the code
> >>> after the
> >>> > > >>> merge.
> >>> > > >>> > > > >
> >>> > > >>> > > > > Matteo
> >>> > > >>> > > > >
> >>> > > >>> > > > >
> >>> > > >>> > > > > On Sun, Nov 6, 2016 at 10:54 PM, Devaraj Das <
> >>> > > >>> ddas@hortonworks.com>
> >>> > > >>> > > > wrote:
> >>> > > >>> > > > >
> >>> > > >>> > > > > > Stack and others, anything else on the patch? Merge
> to
> >>> > master
> >>> > > >>> now?
> >>> > > >>> > > > > >
> >>> > > >>> > > > >
> >>> > > >>> > > >
> >>> > > >>> > >
> >>> > > >>> >
> >>> > > >>>
> >>> > > >>
> >>> > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> >>
> >>
> >
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Vladimir Rodionov <vl...@gmail.com>]

>> Have I grasped the state of things correctly, Vlad?

Josh, the only thing which is still pending is doc update. All other
features are good to have but not a blockers for 2.0 release.

-Vlad

On Fri, Sep 8, 2017 at 10:42 PM, Vladimir Rodionov <vl...@gmail.com>
wrote:

> >> What testing and at what
> >> scale has testing been done?
>
> Do we have have that for other features?
>
>
> On Fri, Sep 8, 2017 at 10:41 PM, Vladimir Rodionov <vladrodionov@gmail.com
> > wrote:
>
>> >> It asks: "How do I figure what of backup/restore feature is going to
>> be in
>> >>hbase-2.0.0?
>>
>> Hmm, wait for doc update.
>>
>>
>> On Fri, Sep 8, 2017 at 2:39 PM, Stack <st...@duboce.net> wrote:
>>
>>> HBASE-14414 is a JIRA with a list of random seeming issues w/
>>> non-descript
>>> summaries: "Add nonce support to TableBackupProcedure, BackupID must
>>> include backup set name, ...". The last comment in that issue is from
>>> July.
>>> It asks: "How do I figure what of backup/restore feature is going to be
>>> in
>>> hbase-2.0.0? Thanks Vladimir Rodionov
>>> <https://issues.apache.org/jira/secure/ViewProfile.jspa?name=vrodionov
>>> >."
>>> to which there is no answer.  Doc update is TODO.
>>>
>>> Where is the summary of the capability in hbase-2? What testing and at
>>> what
>>> scale has testing been done? Is this 'stable or experimental'? If I can't
>>> get basic info on this feature though I ask repeatedly, what hope does
>>> the
>>> poor old operator have?
>>>
>>> St.Ack
>>>
>>>
>>> On Fri, Sep 8, 2017 at 1:59 PM, Vladimir Rodionov <
>>> vladrodionov@gmail.com>
>>> wrote:
>>>
>>> > HBASE-14414
>>> >
>>> > On Fri, Sep 8, 2017 at 1:14 PM, Stack <st...@duboce.net> wrote:
>>> >
>>> > > Where do I go to get the current status of this feature? Looking in
>>> JIRA
>>> > I
>>> > > see loads of issues open against backup including some against
>>> > hbase-2.0.0
>>> > > and no progress being made that I can discern.
>>> > >
>>> > > Thanks,
>>> > > S
>>> > >
>>> > >
>>> > >
>>> > > On Wed, Nov 23, 2016 at 8:52 AM, Stack <st...@duboce.net> wrote:
>>> > >
>>> > > > On Tue, Nov 22, 2016 at 6:48 PM, Stack <st...@duboce.net> wrote:
>>> > > >
>>> > > >> On Tue, Nov 22, 2016 at 3:17 PM, Vladimir Rodionov <
>>> > > >> vladrodionov@gmail.com> wrote:
>>> > > >>
>>> > > >>> >> and/or he answered most of the review feedback
>>> > > >>>
>>> > > >>> No, questions are still open, but I do not see any blockers and
>>> we
>>> > have
>>> > > >>> HBASE-16940 to address these questions.
>>> > > >>>
>>> > > >>>
>>> > > >> Agree. No blockers but stuff that should be dealt with (No one
>>> will
>>> > pay
>>> > > >> me any attention once merge goes in -- smile).
>>> > > >>
>>> > > >>
>>> > > > Let me clarify the above. I want review addressed before merge
>>> happens.
>>> > > > Sorry if any confusion.
>>> > > > St.Ack
>>> > > >
>>> > > >
>>> > > >
>>> > > >
>>> > > >
>>> > > >
>>> > > >> St.Ack
>>> > > >>
>>> > > >>
>>> > > >>
>>> > > >>> On Tue, Nov 22, 2016 at 3:04 PM, Devaraj Das <
>>> ddas@hortonworks.com>
>>> > > >>> wrote:
>>> > > >>>
>>> > > >>> > Hi Stack, hats off to you for spending so much time on this!
>>> > Thanks!
>>> > > >>> From
>>> > > >>> > my understanding, Vlad has raised follow-up jiras for the
>>> issues
>>> > you
>>> > > >>> > raised, and/or he answered most of the review feedback. So, do
>>> you
>>> > > >>> think we
>>> > > >>> > could do a merge vote now?
>>> > > >>> > Devaraj.
>>> > > >>> > ________________________________________
>>> > > >>> > From: Vladimir Rodionov <vl...@gmail.com>
>>> > > >>> > Sent: Monday, November 21, 2016 8:34 PM
>>> > > >>> > To: dev@hbase.apache.org
>>> > > >>> > Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch
>>> > HBASE-7912
>>> > > >>> >
>>> > > >>> > >> I have spent a good bit of time reviewing and testing this
>>> > > feature.
>>> > > >>> I
>>> > > >>> > would
>>> > > >>> > >> like my review and concerns addressed and I'd like it to be
>>> > clear
>>> > > >>> how;
>>> > > >>> > >> either explicit follow-on issues, pointers to where in the
>>> patch
>>> > > or
>>> > > >>> doc
>>> > > >>> > my
>>> > > >>> > >> remarks have been catered to, etc. Until then, I am against
>>> > > commit.
>>> > > >>> >
>>> > > >>> > Stack, mega patch review comments will be addressed in the
>>> > dedicated
>>> > > >>> JIRA:
>>> > > >>> > HBASE-16940
>>> > > >>> > I have open several other JIRAs to address your other comments
>>> (not
>>> > > on
>>> > > >>> > review board).
>>> > > >>> >
>>> > > >>> > Details are here (end of the thread):
>>> > > >>> > https://issues.apache.org/jira/browse/HBASE-14123
>>> > > >>> >
>>> > > >>> > Let me know what else should we do to move merge forward.
>>> > > >>> >
>>> > > >>> > -Vlad
>>> > > >>> >
>>> > > >>> >
>>> > > >>> > On Fri, Nov 18, 2016 at 4:54 PM, Stack <st...@duboce.net>
>>> wrote:
>>> > > >>> >
>>> > > >>> > > On Fri, Nov 18, 2016 at 3:53 PM, Ted Yu <yuzhihong@gmail.com
>>> >
>>> > > wrote:
>>> > > >>> > >
>>> > > >>> > > > Thanks, Matteo.
>>> > > >>> > > >
>>> > > >>> > > > bq. restore is not clear if given an incremental id it
>>> will do
>>> > > the
>>> > > >>> full
>>> > > >>> > > > restore from full up to that point or if i need to apply
>>> > manually
>>> > > >>> > > > everything
>>> > > >>> > > >
>>> > > >>> > > > The restore takes into consideration of the dependent
>>> > backup(s).
>>> > > >>> > > > So there is no need to apply preceding backup(s) manually.
>>> > > >>> > > >
>>> > > >>> > > >
>>> > > >>> > > I ask this question on the issue. It is not clear from the
>>> usage
>>> > or
>>> > > >>> doc
>>> > > >>> > how
>>> > > >>> > > to run a restore from incremental. Can you fix in doc and
>>> usage
>>> > how
>>> > > >>> so I
>>> > > >>> > > can be clear and try it. Currently I am stuck verifying a
>>> round
>>> > > trip
>>> > > >>> > backup
>>> > > >>> > > restore made of incrementals.
>>> > > >>> > >
>>> > > >>> > > Thanks,
>>> > > >>> > > S
>>> > > >>> > >
>>> > > >>> > >
>>> > > >>> > >
>>> > > >>> > > > On Fri, Nov 18, 2016 at 3:48 PM, Matteo Bertozzi <
>>> > > >>> > > theo.bertozzi@gmail.com>
>>> > > >>> > > > wrote:
>>> > > >>> > > >
>>> > > >>> > > > > I did one last pass to the mega patch. I don't see
>>> anything
>>> > > major
>>> > > >>> > that
>>> > > >>> > > > > should block the merge.
>>> > > >>> > > > >
>>> > > >>> > > > > - most of the code is isolated in the backup package
>>> > > >>> > > > > - all the backup code is client side
>>> > > >>> > > > > - there are few changes to the server side, mainly for
>>> > > cleaners,
>>> > > >>> wal
>>> > > >>> > > > > rolling and similar (which is ok)
>>> > > >>> > > > > - there is a good number of tests, and an integration
>>> test
>>> > > >>> > > > >
>>> > > >>> > > > > the code seems to have still some left overs from the old
>>> > > >>> > > implementation,
>>> > > >>> > > > > and some stuff needs a cleanup. but I don't think this
>>> should
>>> > > be
>>> > > >>> used
>>> > > >>> > > as
>>> > > >>> > > > an
>>> > > >>> > > > > argument to block the merge. I think the guys will keep
>>> > working
>>> > > >>> on
>>> > > >>> > this
>>> > > >>> > > > and
>>> > > >>> > > > > they may also get help of others once the patch is in
>>> master.
>>> > > >>> > > > >
>>> > > >>> > > > > I still have my concerns about the current limitations,
>>> but
>>> > > >>> these are
>>> > > >>> > > > > things already planned for phase 3, so some of this
>>> stuff may
>>> > > >>> even be
>>> > > >>> > > in
>>> > > >>> > > > > the final 2.0.
>>> > > >>> > > > > but as long as we have a "current limitations" section
>>> in the
>>> > > >>> user
>>> > > >>> > > guide
>>> > > >>> > > > > mentioning important stuff like the ones below, I'm ok
>>> with
>>> > it.
>>> > > >>> > > > >  - if you write to the table with Durability.SKIP_WALS
>>> your
>>> > > data
>>> > > >>> will
>>> > > >>> > > not
>>> > > >>> > > > > be in the incremental-backup
>>> > > >>> > > > >  - if you bulkload files that data will not be in the
>>> > > incremental
>>> > > >>> > > backup
>>> > > >>> > > > > (HBASE-14417)
>>> > > >>> > > > >  - the incremental backup will not only contains the
>>> data of
>>> > > the
>>> > > >>> > table
>>> > > >>> > > > you
>>> > > >>> > > > > specified but also the regions from other tables that
>>> are on
>>> > > the
>>> > > >>> same
>>> > > >>> > > set
>>> > > >>> > > > > of RSs (HBASE-14141) ...maybe a note about security
>>> around
>>> > this
>>> > > >>> topic
>>> > > >>> > > > >  - the incremental backup will not contains just the
>>> "latest
>>> > > row"
>>> > > >>> > > between
>>> > > >>> > > > > backup A and B, but it will also contains all the updates
>>> > > >>> occurred in
>>> > > >>> > > > > between. but the restore does not allow you to restore
>>> up to
>>> > a
>>> > > >>> > certain
>>> > > >>> > > > > point in time, the restore will always be up to the
>>> "latest
>>> > > >>> backup
>>> > > >>> > > > point".
>>> > > >>> > > > >  - you should limit the number of "incremental" up to N
>>> (or
>>> > > maybe
>>> > > >>> > > SIZE),
>>> > > >>> > > > to
>>> > > >>> > > > > avoid replay time becoming the bottleneck. (HBASE-14135)
>>> > > >>> > > > >
>>> > > >>> > > > > I'll be ok even with the above not being in the final
>>> 2.0,
>>> > > >>> > > > > but i'd like to see as blocker for the final 2.0 (not the
>>> > > merge)
>>> > > >>> > > > >  - the backup code moved in an hbase-backup module
>>> > > >>> > > > >  - and some more work around tools, especially to try to
>>> > unify
>>> > > >>> and
>>> > > >>> > make
>>> > > >>> > > > > simple the backup experience (simple example: in some
>>> case
>>> > > there
>>> > > >>> is a
>>> > > >>> > > > > backup_id argument in others a backupId argument. or
>>> things
>>> > > >>> like..
>>> > > >>> > > > restore
>>> > > >>> > > > > is not clear if given an incremental id it will do the
>>> full
>>> > > >>> restore
>>> > > >>> > > from
>>> > > >>> > > > > full up to that point or if i need to apply manually
>>> > > everything).
>>> > > >>> > > > >
>>> > > >>> > > > > in conclusion, I think we can open a merge vote. I'll be
>>> +1
>>> > on
>>> > > >>> it,
>>> > > >>> > and
>>> > > >>> > > I
>>> > > >>> > > > > think we should try to reject -1 with just a "code
>>> cleanup"
>>> > > >>> > motivation,
>>> > > >>> > > > > since there will still be work going on on the code
>>> after the
>>> > > >>> merge.
>>> > > >>> > > > >
>>> > > >>> > > > > Matteo
>>> > > >>> > > > >
>>> > > >>> > > > >
>>> > > >>> > > > > On Sun, Nov 6, 2016 at 10:54 PM, Devaraj Das <
>>> > > >>> ddas@hortonworks.com>
>>> > > >>> > > > wrote:
>>> > > >>> > > > >
>>> > > >>> > > > > > Stack and others, anything else on the patch? Merge to
>>> > master
>>> > > >>> now?
>>> > > >>> > > > > >
>>> > > >>> > > > >
>>> > > >>> > > >
>>> > > >>> > >
>>> > > >>> >
>>> > > >>>
>>> > > >>
>>> > > >>
>>> > > >
>>> > >
>>> >
>>>
>>
>>
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Vladimir Rodionov <vl...@gmail.com>]

>> What testing and at what
>> scale has testing been done?

Do we have have that for other features?


On Fri, Sep 8, 2017 at 10:41 PM, Vladimir Rodionov <vl...@gmail.com>
wrote:

> >> It asks: "How do I figure what of backup/restore feature is going to
> be in
> >>hbase-2.0.0?
>
> Hmm, wait for doc update.
>
>
> On Fri, Sep 8, 2017 at 2:39 PM, Stack <st...@duboce.net> wrote:
>
>> HBASE-14414 is a JIRA with a list of random seeming issues w/ non-descript
>> summaries: "Add nonce support to TableBackupProcedure, BackupID must
>> include backup set name, ...". The last comment in that issue is from
>> July.
>> It asks: "How do I figure what of backup/restore feature is going to be in
>> hbase-2.0.0? Thanks Vladimir Rodionov
>> <https://issues.apache.org/jira/secure/ViewProfile.jspa?name=vrodionov>."
>> to which there is no answer.  Doc update is TODO.
>>
>> Where is the summary of the capability in hbase-2? What testing and at
>> what
>> scale has testing been done? Is this 'stable or experimental'? If I can't
>> get basic info on this feature though I ask repeatedly, what hope does the
>> poor old operator have?
>>
>> St.Ack
>>
>>
>> On Fri, Sep 8, 2017 at 1:59 PM, Vladimir Rodionov <vladrodionov@gmail.com
>> >
>> wrote:
>>
>> > HBASE-14414
>> >
>> > On Fri, Sep 8, 2017 at 1:14 PM, Stack <st...@duboce.net> wrote:
>> >
>> > > Where do I go to get the current status of this feature? Looking in
>> JIRA
>> > I
>> > > see loads of issues open against backup including some against
>> > hbase-2.0.0
>> > > and no progress being made that I can discern.
>> > >
>> > > Thanks,
>> > > S
>> > >
>> > >
>> > >
>> > > On Wed, Nov 23, 2016 at 8:52 AM, Stack <st...@duboce.net> wrote:
>> > >
>> > > > On Tue, Nov 22, 2016 at 6:48 PM, Stack <st...@duboce.net> wrote:
>> > > >
>> > > >> On Tue, Nov 22, 2016 at 3:17 PM, Vladimir Rodionov <
>> > > >> vladrodionov@gmail.com> wrote:
>> > > >>
>> > > >>> >> and/or he answered most of the review feedback
>> > > >>>
>> > > >>> No, questions are still open, but I do not see any blockers and we
>> > have
>> > > >>> HBASE-16940 to address these questions.
>> > > >>>
>> > > >>>
>> > > >> Agree. No blockers but stuff that should be dealt with (No one will
>> > pay
>> > > >> me any attention once merge goes in -- smile).
>> > > >>
>> > > >>
>> > > > Let me clarify the above. I want review addressed before merge
>> happens.
>> > > > Sorry if any confusion.
>> > > > St.Ack
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >> St.Ack
>> > > >>
>> > > >>
>> > > >>
>> > > >>> On Tue, Nov 22, 2016 at 3:04 PM, Devaraj Das <
>> ddas@hortonworks.com>
>> > > >>> wrote:
>> > > >>>
>> > > >>> > Hi Stack, hats off to you for spending so much time on this!
>> > Thanks!
>> > > >>> From
>> > > >>> > my understanding, Vlad has raised follow-up jiras for the issues
>> > you
>> > > >>> > raised, and/or he answered most of the review feedback. So, do
>> you
>> > > >>> think we
>> > > >>> > could do a merge vote now?
>> > > >>> > Devaraj.
>> > > >>> > ________________________________________
>> > > >>> > From: Vladimir Rodionov <vl...@gmail.com>
>> > > >>> > Sent: Monday, November 21, 2016 8:34 PM
>> > > >>> > To: dev@hbase.apache.org
>> > > >>> > Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch
>> > HBASE-7912
>> > > >>> >
>> > > >>> > >> I have spent a good bit of time reviewing and testing this
>> > > feature.
>> > > >>> I
>> > > >>> > would
>> > > >>> > >> like my review and concerns addressed and I'd like it to be
>> > clear
>> > > >>> how;
>> > > >>> > >> either explicit follow-on issues, pointers to where in the
>> patch
>> > > or
>> > > >>> doc
>> > > >>> > my
>> > > >>> > >> remarks have been catered to, etc. Until then, I am against
>> > > commit.
>> > > >>> >
>> > > >>> > Stack, mega patch review comments will be addressed in the
>> > dedicated
>> > > >>> JIRA:
>> > > >>> > HBASE-16940
>> > > >>> > I have open several other JIRAs to address your other comments
>> (not
>> > > on
>> > > >>> > review board).
>> > > >>> >
>> > > >>> > Details are here (end of the thread):
>> > > >>> > https://issues.apache.org/jira/browse/HBASE-14123
>> > > >>> >
>> > > >>> > Let me know what else should we do to move merge forward.
>> > > >>> >
>> > > >>> > -Vlad
>> > > >>> >
>> > > >>> >
>> > > >>> > On Fri, Nov 18, 2016 at 4:54 PM, Stack <st...@duboce.net>
>> wrote:
>> > > >>> >
>> > > >>> > > On Fri, Nov 18, 2016 at 3:53 PM, Ted Yu <yu...@gmail.com>
>> > > wrote:
>> > > >>> > >
>> > > >>> > > > Thanks, Matteo.
>> > > >>> > > >
>> > > >>> > > > bq. restore is not clear if given an incremental id it will
>> do
>> > > the
>> > > >>> full
>> > > >>> > > > restore from full up to that point or if i need to apply
>> > manually
>> > > >>> > > > everything
>> > > >>> > > >
>> > > >>> > > > The restore takes into consideration of the dependent
>> > backup(s).
>> > > >>> > > > So there is no need to apply preceding backup(s) manually.
>> > > >>> > > >
>> > > >>> > > >
>> > > >>> > > I ask this question on the issue. It is not clear from the
>> usage
>> > or
>> > > >>> doc
>> > > >>> > how
>> > > >>> > > to run a restore from incremental. Can you fix in doc and
>> usage
>> > how
>> > > >>> so I
>> > > >>> > > can be clear and try it. Currently I am stuck verifying a
>> round
>> > > trip
>> > > >>> > backup
>> > > >>> > > restore made of incrementals.
>> > > >>> > >
>> > > >>> > > Thanks,
>> > > >>> > > S
>> > > >>> > >
>> > > >>> > >
>> > > >>> > >
>> > > >>> > > > On Fri, Nov 18, 2016 at 3:48 PM, Matteo Bertozzi <
>> > > >>> > > theo.bertozzi@gmail.com>
>> > > >>> > > > wrote:
>> > > >>> > > >
>> > > >>> > > > > I did one last pass to the mega patch. I don't see
>> anything
>> > > major
>> > > >>> > that
>> > > >>> > > > > should block the merge.
>> > > >>> > > > >
>> > > >>> > > > > - most of the code is isolated in the backup package
>> > > >>> > > > > - all the backup code is client side
>> > > >>> > > > > - there are few changes to the server side, mainly for
>> > > cleaners,
>> > > >>> wal
>> > > >>> > > > > rolling and similar (which is ok)
>> > > >>> > > > > - there is a good number of tests, and an integration test
>> > > >>> > > > >
>> > > >>> > > > > the code seems to have still some left overs from the old
>> > > >>> > > implementation,
>> > > >>> > > > > and some stuff needs a cleanup. but I don't think this
>> should
>> > > be
>> > > >>> used
>> > > >>> > > as
>> > > >>> > > > an
>> > > >>> > > > > argument to block the merge. I think the guys will keep
>> > working
>> > > >>> on
>> > > >>> > this
>> > > >>> > > > and
>> > > >>> > > > > they may also get help of others once the patch is in
>> master.
>> > > >>> > > > >
>> > > >>> > > > > I still have my concerns about the current limitations,
>> but
>> > > >>> these are
>> > > >>> > > > > things already planned for phase 3, so some of this stuff
>> may
>> > > >>> even be
>> > > >>> > > in
>> > > >>> > > > > the final 2.0.
>> > > >>> > > > > but as long as we have a "current limitations" section in
>> the
>> > > >>> user
>> > > >>> > > guide
>> > > >>> > > > > mentioning important stuff like the ones below, I'm ok
>> with
>> > it.
>> > > >>> > > > >  - if you write to the table with Durability.SKIP_WALS
>> your
>> > > data
>> > > >>> will
>> > > >>> > > not
>> > > >>> > > > > be in the incremental-backup
>> > > >>> > > > >  - if you bulkload files that data will not be in the
>> > > incremental
>> > > >>> > > backup
>> > > >>> > > > > (HBASE-14417)
>> > > >>> > > > >  - the incremental backup will not only contains the data
>> of
>> > > the
>> > > >>> > table
>> > > >>> > > > you
>> > > >>> > > > > specified but also the regions from other tables that are
>> on
>> > > the
>> > > >>> same
>> > > >>> > > set
>> > > >>> > > > > of RSs (HBASE-14141) ...maybe a note about security around
>> > this
>> > > >>> topic
>> > > >>> > > > >  - the incremental backup will not contains just the
>> "latest
>> > > row"
>> > > >>> > > between
>> > > >>> > > > > backup A and B, but it will also contains all the updates
>> > > >>> occurred in
>> > > >>> > > > > between. but the restore does not allow you to restore up
>> to
>> > a
>> > > >>> > certain
>> > > >>> > > > > point in time, the restore will always be up to the
>> "latest
>> > > >>> backup
>> > > >>> > > > point".
>> > > >>> > > > >  - you should limit the number of "incremental" up to N
>> (or
>> > > maybe
>> > > >>> > > SIZE),
>> > > >>> > > > to
>> > > >>> > > > > avoid replay time becoming the bottleneck. (HBASE-14135)
>> > > >>> > > > >
>> > > >>> > > > > I'll be ok even with the above not being in the final 2.0,
>> > > >>> > > > > but i'd like to see as blocker for the final 2.0 (not the
>> > > merge)
>> > > >>> > > > >  - the backup code moved in an hbase-backup module
>> > > >>> > > > >  - and some more work around tools, especially to try to
>> > unify
>> > > >>> and
>> > > >>> > make
>> > > >>> > > > > simple the backup experience (simple example: in some case
>> > > there
>> > > >>> is a
>> > > >>> > > > > backup_id argument in others a backupId argument. or
>> things
>> > > >>> like..
>> > > >>> > > > restore
>> > > >>> > > > > is not clear if given an incremental id it will do the
>> full
>> > > >>> restore
>> > > >>> > > from
>> > > >>> > > > > full up to that point or if i need to apply manually
>> > > everything).
>> > > >>> > > > >
>> > > >>> > > > > in conclusion, I think we can open a merge vote. I'll be
>> +1
>> > on
>> > > >>> it,
>> > > >>> > and
>> > > >>> > > I
>> > > >>> > > > > think we should try to reject -1 with just a "code
>> cleanup"
>> > > >>> > motivation,
>> > > >>> > > > > since there will still be work going on on the code after
>> the
>> > > >>> merge.
>> > > >>> > > > >
>> > > >>> > > > > Matteo
>> > > >>> > > > >
>> > > >>> > > > >
>> > > >>> > > > > On Sun, Nov 6, 2016 at 10:54 PM, Devaraj Das <
>> > > >>> ddas@hortonworks.com>
>> > > >>> > > > wrote:
>> > > >>> > > > >
>> > > >>> > > > > > Stack and others, anything else on the patch? Merge to
>> > master
>> > > >>> now?
>> > > >>> > > > > >
>> > > >>> > > > >
>> > > >>> > > >
>> > > >>> > >
>> > > >>> >
>> > > >>>
>> > > >>
>> > > >>
>> > > >
>> > >
>> >
>>
>
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Vladimir Rodionov <vl...@gmail.com>]

>> It asks: "How do I figure what of backup/restore feature is going to be
in
>>hbase-2.0.0?

Hmm, wait for doc update.


On Fri, Sep 8, 2017 at 2:39 PM, Stack <st...@duboce.net> wrote:

> HBASE-14414 is a JIRA with a list of random seeming issues w/ non-descript
> summaries: "Add nonce support to TableBackupProcedure, BackupID must
> include backup set name, ...". The last comment in that issue is from July.
> It asks: "How do I figure what of backup/restore feature is going to be in
> hbase-2.0.0? Thanks Vladimir Rodionov
> <https://issues.apache.org/jira/secure/ViewProfile.jspa?name=vrodionov>."
> to which there is no answer.  Doc update is TODO.
>
> Where is the summary of the capability in hbase-2? What testing and at what
> scale has testing been done? Is this 'stable or experimental'? If I can't
> get basic info on this feature though I ask repeatedly, what hope does the
> poor old operator have?
>
> St.Ack
>
>
> On Fri, Sep 8, 2017 at 1:59 PM, Vladimir Rodionov <vl...@gmail.com>
> wrote:
>
> > HBASE-14414
> >
> > On Fri, Sep 8, 2017 at 1:14 PM, Stack <st...@duboce.net> wrote:
> >
> > > Where do I go to get the current status of this feature? Looking in
> JIRA
> > I
> > > see loads of issues open against backup including some against
> > hbase-2.0.0
> > > and no progress being made that I can discern.
> > >
> > > Thanks,
> > > S
> > >
> > >
> > >
> > > On Wed, Nov 23, 2016 at 8:52 AM, Stack <st...@duboce.net> wrote:
> > >
> > > > On Tue, Nov 22, 2016 at 6:48 PM, Stack <st...@duboce.net> wrote:
> > > >
> > > >> On Tue, Nov 22, 2016 at 3:17 PM, Vladimir Rodionov <
> > > >> vladrodionov@gmail.com> wrote:
> > > >>
> > > >>> >> and/or he answered most of the review feedback
> > > >>>
> > > >>> No, questions are still open, but I do not see any blockers and we
> > have
> > > >>> HBASE-16940 to address these questions.
> > > >>>
> > > >>>
> > > >> Agree. No blockers but stuff that should be dealt with (No one will
> > pay
> > > >> me any attention once merge goes in -- smile).
> > > >>
> > > >>
> > > > Let me clarify the above. I want review addressed before merge
> happens.
> > > > Sorry if any confusion.
> > > > St.Ack
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >> St.Ack
> > > >>
> > > >>
> > > >>
> > > >>> On Tue, Nov 22, 2016 at 3:04 PM, Devaraj Das <ddas@hortonworks.com
> >
> > > >>> wrote:
> > > >>>
> > > >>> > Hi Stack, hats off to you for spending so much time on this!
> > Thanks!
> > > >>> From
> > > >>> > my understanding, Vlad has raised follow-up jiras for the issues
> > you
> > > >>> > raised, and/or he answered most of the review feedback. So, do
> you
> > > >>> think we
> > > >>> > could do a merge vote now?
> > > >>> > Devaraj.
> > > >>> > ________________________________________
> > > >>> > From: Vladimir Rodionov <vl...@gmail.com>
> > > >>> > Sent: Monday, November 21, 2016 8:34 PM
> > > >>> > To: dev@hbase.apache.org
> > > >>> > Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch
> > HBASE-7912
> > > >>> >
> > > >>> > >> I have spent a good bit of time reviewing and testing this
> > > feature.
> > > >>> I
> > > >>> > would
> > > >>> > >> like my review and concerns addressed and I'd like it to be
> > clear
> > > >>> how;
> > > >>> > >> either explicit follow-on issues, pointers to where in the
> patch
> > > or
> > > >>> doc
> > > >>> > my
> > > >>> > >> remarks have been catered to, etc. Until then, I am against
> > > commit.
> > > >>> >
> > > >>> > Stack, mega patch review comments will be addressed in the
> > dedicated
> > > >>> JIRA:
> > > >>> > HBASE-16940
> > > >>> > I have open several other JIRAs to address your other comments
> (not
> > > on
> > > >>> > review board).
> > > >>> >
> > > >>> > Details are here (end of the thread):
> > > >>> > https://issues.apache.org/jira/browse/HBASE-14123
> > > >>> >
> > > >>> > Let me know what else should we do to move merge forward.
> > > >>> >
> > > >>> > -Vlad
> > > >>> >
> > > >>> >
> > > >>> > On Fri, Nov 18, 2016 at 4:54 PM, Stack <st...@duboce.net> wrote:
> > > >>> >
> > > >>> > > On Fri, Nov 18, 2016 at 3:53 PM, Ted Yu <yu...@gmail.com>
> > > wrote:
> > > >>> > >
> > > >>> > > > Thanks, Matteo.
> > > >>> > > >
> > > >>> > > > bq. restore is not clear if given an incremental id it will
> do
> > > the
> > > >>> full
> > > >>> > > > restore from full up to that point or if i need to apply
> > manually
> > > >>> > > > everything
> > > >>> > > >
> > > >>> > > > The restore takes into consideration of the dependent
> > backup(s).
> > > >>> > > > So there is no need to apply preceding backup(s) manually.
> > > >>> > > >
> > > >>> > > >
> > > >>> > > I ask this question on the issue. It is not clear from the
> usage
> > or
> > > >>> doc
> > > >>> > how
> > > >>> > > to run a restore from incremental. Can you fix in doc and usage
> > how
> > > >>> so I
> > > >>> > > can be clear and try it. Currently I am stuck verifying a round
> > > trip
> > > >>> > backup
> > > >>> > > restore made of incrementals.
> > > >>> > >
> > > >>> > > Thanks,
> > > >>> > > S
> > > >>> > >
> > > >>> > >
> > > >>> > >
> > > >>> > > > On Fri, Nov 18, 2016 at 3:48 PM, Matteo Bertozzi <
> > > >>> > > theo.bertozzi@gmail.com>
> > > >>> > > > wrote:
> > > >>> > > >
> > > >>> > > > > I did one last pass to the mega patch. I don't see anything
> > > major
> > > >>> > that
> > > >>> > > > > should block the merge.
> > > >>> > > > >
> > > >>> > > > > - most of the code is isolated in the backup package
> > > >>> > > > > - all the backup code is client side
> > > >>> > > > > - there are few changes to the server side, mainly for
> > > cleaners,
> > > >>> wal
> > > >>> > > > > rolling and similar (which is ok)
> > > >>> > > > > - there is a good number of tests, and an integration test
> > > >>> > > > >
> > > >>> > > > > the code seems to have still some left overs from the old
> > > >>> > > implementation,
> > > >>> > > > > and some stuff needs a cleanup. but I don't think this
> should
> > > be
> > > >>> used
> > > >>> > > as
> > > >>> > > > an
> > > >>> > > > > argument to block the merge. I think the guys will keep
> > working
> > > >>> on
> > > >>> > this
> > > >>> > > > and
> > > >>> > > > > they may also get help of others once the patch is in
> master.
> > > >>> > > > >
> > > >>> > > > > I still have my concerns about the current limitations, but
> > > >>> these are
> > > >>> > > > > things already planned for phase 3, so some of this stuff
> may
> > > >>> even be
> > > >>> > > in
> > > >>> > > > > the final 2.0.
> > > >>> > > > > but as long as we have a "current limitations" section in
> the
> > > >>> user
> > > >>> > > guide
> > > >>> > > > > mentioning important stuff like the ones below, I'm ok with
> > it.
> > > >>> > > > >  - if you write to the table with Durability.SKIP_WALS your
> > > data
> > > >>> will
> > > >>> > > not
> > > >>> > > > > be in the incremental-backup
> > > >>> > > > >  - if you bulkload files that data will not be in the
> > > incremental
> > > >>> > > backup
> > > >>> > > > > (HBASE-14417)
> > > >>> > > > >  - the incremental backup will not only contains the data
> of
> > > the
> > > >>> > table
> > > >>> > > > you
> > > >>> > > > > specified but also the regions from other tables that are
> on
> > > the
> > > >>> same
> > > >>> > > set
> > > >>> > > > > of RSs (HBASE-14141) ...maybe a note about security around
> > this
> > > >>> topic
> > > >>> > > > >  - the incremental backup will not contains just the
> "latest
> > > row"
> > > >>> > > between
> > > >>> > > > > backup A and B, but it will also contains all the updates
> > > >>> occurred in
> > > >>> > > > > between. but the restore does not allow you to restore up
> to
> > a
> > > >>> > certain
> > > >>> > > > > point in time, the restore will always be up to the "latest
> > > >>> backup
> > > >>> > > > point".
> > > >>> > > > >  - you should limit the number of "incremental" up to N (or
> > > maybe
> > > >>> > > SIZE),
> > > >>> > > > to
> > > >>> > > > > avoid replay time becoming the bottleneck. (HBASE-14135)
> > > >>> > > > >
> > > >>> > > > > I'll be ok even with the above not being in the final 2.0,
> > > >>> > > > > but i'd like to see as blocker for the final 2.0 (not the
> > > merge)
> > > >>> > > > >  - the backup code moved in an hbase-backup module
> > > >>> > > > >  - and some more work around tools, especially to try to
> > unify
> > > >>> and
> > > >>> > make
> > > >>> > > > > simple the backup experience (simple example: in some case
> > > there
> > > >>> is a
> > > >>> > > > > backup_id argument in others a backupId argument. or things
> > > >>> like..
> > > >>> > > > restore
> > > >>> > > > > is not clear if given an incremental id it will do the full
> > > >>> restore
> > > >>> > > from
> > > >>> > > > > full up to that point or if i need to apply manually
> > > everything).
> > > >>> > > > >
> > > >>> > > > > in conclusion, I think we can open a merge vote. I'll be +1
> > on
> > > >>> it,
> > > >>> > and
> > > >>> > > I
> > > >>> > > > > think we should try to reject -1 with just a "code cleanup"
> > > >>> > motivation,
> > > >>> > > > > since there will still be work going on on the code after
> the
> > > >>> merge.
> > > >>> > > > >
> > > >>> > > > > Matteo
> > > >>> > > > >
> > > >>> > > > >
> > > >>> > > > > On Sun, Nov 6, 2016 at 10:54 PM, Devaraj Das <
> > > >>> ddas@hortonworks.com>
> > > >>> > > > wrote:
> > > >>> > > > >
> > > >>> > > > > > Stack and others, anything else on the patch? Merge to
> > master
> > > >>> now?
> > > >>> > > > > >
> > > >>> > > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > > >>
> > > >>
> > > >
> > >
> >
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Josh Elser <el...@apache.org>]

Based on the list of stuff on HBASE-14414 and offline-chats had with 
Vlad myself, I know of the following being needed

https://issues.apache.org/jira/browse/HBASE-15227 - Fault tolerance 
umbrella (no-op on its own)
https://issues.apache.org/jira/browse/HBASE-17852 - I believe Vlad is 
working on this one now
https://issues.apache.org/jira/browse/HBASE-16465 - disable 
splits/merges (marked as required for 15227 -- not sure if that's 
accurate anymore)
https://issues.apache.org/jira/browse/HBASE-17851 -- Just a unit test
https://issues.apache.org/jira/browse/HBASE-17133 - Docs

Echo'ing Stack, we're at the point where we need to draw a line in the 
sand for what will hit 2.0.0 to avoid mucking up testing.

Have I grasped the state of things correctly, Vlad?

On 9/8/17 5:39 PM, Stack wrote:
> HBASE-14414 is a JIRA with a list of random seeming issues w/ non-descript
> summaries: "Add nonce support to TableBackupProcedure, BackupID must
> include backup set name, ...". The last comment in that issue is from July.
> It asks: "How do I figure what of backup/restore feature is going to be in
> hbase-2.0.0? Thanks Vladimir Rodionov
> <https://issues.apache.org/jira/secure/ViewProfile.jspa?name=vrodionov>."
> to which there is no answer.  Doc update is TODO.
> 
> Where is the summary of the capability in hbase-2? What testing and at what
> scale has testing been done? Is this 'stable or experimental'? If I can't
> get basic info on this feature though I ask repeatedly, what hope does the
> poor old operator have?
> 
> St.Ack
> 
> 
> On Fri, Sep 8, 2017 at 1:59 PM, Vladimir Rodionov <vl...@gmail.com>
> wrote:
> 
>> HBASE-14414
>>
>> On Fri, Sep 8, 2017 at 1:14 PM, Stack <st...@duboce.net> wrote:
>>
>>> Where do I go to get the current status of this feature? Looking in JIRA
>> I
>>> see loads of issues open against backup including some against
>> hbase-2.0.0
>>> and no progress being made that I can discern.
>>>
>>> Thanks,
>>> S
>>>
>>>
>>>
>>> On Wed, Nov 23, 2016 at 8:52 AM, Stack <st...@duboce.net> wrote:
>>>
>>>> On Tue, Nov 22, 2016 at 6:48 PM, Stack <st...@duboce.net> wrote:
>>>>
>>>>> On Tue, Nov 22, 2016 at 3:17 PM, Vladimir Rodionov <
>>>>> vladrodionov@gmail.com> wrote:
>>>>>
>>>>>>>> and/or he answered most of the review feedback
>>>>>>
>>>>>> No, questions are still open, but I do not see any blockers and we
>> have
>>>>>> HBASE-16940 to address these questions.
>>>>>>
>>>>>>
>>>>> Agree. No blockers but stuff that should be dealt with (No one will
>> pay
>>>>> me any attention once merge goes in -- smile).
>>>>>
>>>>>
>>>> Let me clarify the above. I want review addressed before merge happens.
>>>> Sorry if any confusion.
>>>> St.Ack
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> St.Ack
>>>>>
>>>>>
>>>>>
>>>>>> On Tue, Nov 22, 2016 at 3:04 PM, Devaraj Das <dd...@hortonworks.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Stack, hats off to you for spending so much time on this!
>> Thanks!
>>>>>> From
>>>>>>> my understanding, Vlad has raised follow-up jiras for the issues
>> you
>>>>>>> raised, and/or he answered most of the review feedback. So, do you
>>>>>> think we
>>>>>>> could do a merge vote now?
>>>>>>> Devaraj.
>>>>>>> ________________________________________
>>>>>>> From: Vladimir Rodionov <vl...@gmail.com>
>>>>>>> Sent: Monday, November 21, 2016 8:34 PM
>>>>>>> To: dev@hbase.apache.org
>>>>>>> Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch
>> HBASE-7912
>>>>>>>
>>>>>>>>> I have spent a good bit of time reviewing and testing this
>>> feature.
>>>>>> I
>>>>>>> would
>>>>>>>>> like my review and concerns addressed and I'd like it to be
>> clear
>>>>>> how;
>>>>>>>>> either explicit follow-on issues, pointers to where in the patch
>>> or
>>>>>> doc
>>>>>>> my
>>>>>>>>> remarks have been catered to, etc. Until then, I am against
>>> commit.
>>>>>>>
>>>>>>> Stack, mega patch review comments will be addressed in the
>> dedicated
>>>>>> JIRA:
>>>>>>> HBASE-16940
>>>>>>> I have open several other JIRAs to address your other comments (not
>>> on
>>>>>>> review board).
>>>>>>>
>>>>>>> Details are here (end of the thread):
>>>>>>> https://issues.apache.org/jira/browse/HBASE-14123
>>>>>>>
>>>>>>> Let me know what else should we do to move merge forward.
>>>>>>>
>>>>>>> -Vlad
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Nov 18, 2016 at 4:54 PM, Stack <st...@duboce.net> wrote:
>>>>>>>
>>>>>>>> On Fri, Nov 18, 2016 at 3:53 PM, Ted Yu <yu...@gmail.com>
>>> wrote:
>>>>>>>>
>>>>>>>>> Thanks, Matteo.
>>>>>>>>>
>>>>>>>>> bq. restore is not clear if given an incremental id it will do
>>> the
>>>>>> full
>>>>>>>>> restore from full up to that point or if i need to apply
>> manually
>>>>>>>>> everything
>>>>>>>>>
>>>>>>>>> The restore takes into consideration of the dependent
>> backup(s).
>>>>>>>>> So there is no need to apply preceding backup(s) manually.
>>>>>>>>>
>>>>>>>>>
>>>>>>>> I ask this question on the issue. It is not clear from the usage
>> or
>>>>>> doc
>>>>>>> how
>>>>>>>> to run a restore from incremental. Can you fix in doc and usage
>> how
>>>>>> so I
>>>>>>>> can be clear and try it. Currently I am stuck verifying a round
>>> trip
>>>>>>> backup
>>>>>>>> restore made of incrementals.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> S
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> On Fri, Nov 18, 2016 at 3:48 PM, Matteo Bertozzi <
>>>>>>>> theo.bertozzi@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> I did one last pass to the mega patch. I don't see anything
>>> major
>>>>>>> that
>>>>>>>>>> should block the merge.
>>>>>>>>>>
>>>>>>>>>> - most of the code is isolated in the backup package
>>>>>>>>>> - all the backup code is client side
>>>>>>>>>> - there are few changes to the server side, mainly for
>>> cleaners,
>>>>>> wal
>>>>>>>>>> rolling and similar (which is ok)
>>>>>>>>>> - there is a good number of tests, and an integration test
>>>>>>>>>>
>>>>>>>>>> the code seems to have still some left overs from the old
>>>>>>>> implementation,
>>>>>>>>>> and some stuff needs a cleanup. but I don't think this should
>>> be
>>>>>> used
>>>>>>>> as
>>>>>>>>> an
>>>>>>>>>> argument to block the merge. I think the guys will keep
>> working
>>>>>> on
>>>>>>> this
>>>>>>>>> and
>>>>>>>>>> they may also get help of others once the patch is in master.
>>>>>>>>>>
>>>>>>>>>> I still have my concerns about the current limitations, but
>>>>>> these are
>>>>>>>>>> things already planned for phase 3, so some of this stuff may
>>>>>> even be
>>>>>>>> in
>>>>>>>>>> the final 2.0.
>>>>>>>>>> but as long as we have a "current limitations" section in the
>>>>>> user
>>>>>>>> guide
>>>>>>>>>> mentioning important stuff like the ones below, I'm ok with
>> it.
>>>>>>>>>>   - if you write to the table with Durability.SKIP_WALS your
>>> data
>>>>>> will
>>>>>>>> not
>>>>>>>>>> be in the incremental-backup
>>>>>>>>>>   - if you bulkload files that data will not be in the
>>> incremental
>>>>>>>> backup
>>>>>>>>>> (HBASE-14417)
>>>>>>>>>>   - the incremental backup will not only contains the data of
>>> the
>>>>>>> table
>>>>>>>>> you
>>>>>>>>>> specified but also the regions from other tables that are on
>>> the
>>>>>> same
>>>>>>>> set
>>>>>>>>>> of RSs (HBASE-14141) ...maybe a note about security around
>> this
>>>>>> topic
>>>>>>>>>>   - the incremental backup will not contains just the "latest
>>> row"
>>>>>>>> between
>>>>>>>>>> backup A and B, but it will also contains all the updates
>>>>>> occurred in
>>>>>>>>>> between. but the restore does not allow you to restore up to
>> a
>>>>>>> certain
>>>>>>>>>> point in time, the restore will always be up to the "latest
>>>>>> backup
>>>>>>>>> point".
>>>>>>>>>>   - you should limit the number of "incremental" up to N (or
>>> maybe
>>>>>>>> SIZE),
>>>>>>>>> to
>>>>>>>>>> avoid replay time becoming the bottleneck. (HBASE-14135)
>>>>>>>>>>
>>>>>>>>>> I'll be ok even with the above not being in the final 2.0,
>>>>>>>>>> but i'd like to see as blocker for the final 2.0 (not the
>>> merge)
>>>>>>>>>>   - the backup code moved in an hbase-backup module
>>>>>>>>>>   - and some more work around tools, especially to try to
>> unify
>>>>>> and
>>>>>>> make
>>>>>>>>>> simple the backup experience (simple example: in some case
>>> there
>>>>>> is a
>>>>>>>>>> backup_id argument in others a backupId argument. or things
>>>>>> like..
>>>>>>>>> restore
>>>>>>>>>> is not clear if given an incremental id it will do the full
>>>>>> restore
>>>>>>>> from
>>>>>>>>>> full up to that point or if i need to apply manually
>>> everything).
>>>>>>>>>>
>>>>>>>>>> in conclusion, I think we can open a merge vote. I'll be +1
>> on
>>>>>> it,
>>>>>>> and
>>>>>>>> I
>>>>>>>>>> think we should try to reject -1 with just a "code cleanup"
>>>>>>> motivation,
>>>>>>>>>> since there will still be work going on on the code after the
>>>>>> merge.
>>>>>>>>>>
>>>>>>>>>> Matteo
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Sun, Nov 6, 2016 at 10:54 PM, Devaraj Das <
>>>>>> ddas@hortonworks.com>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Stack and others, anything else on the patch? Merge to
>> master
>>>>>> now?
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
> 

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Stack <st...@duboce.net>]

HBASE-14414 is a JIRA with a list of random seeming issues w/ non-descript
summaries: "Add nonce support to TableBackupProcedure, BackupID must
include backup set name, ...". The last comment in that issue is from July.
It asks: "How do I figure what of backup/restore feature is going to be in
hbase-2.0.0? Thanks Vladimir Rodionov
<https://issues.apache.org/jira/secure/ViewProfile.jspa?name=vrodionov>."
to which there is no answer.  Doc update is TODO.

Where is the summary of the capability in hbase-2? What testing and at what
scale has testing been done? Is this 'stable or experimental'? If I can't
get basic info on this feature though I ask repeatedly, what hope does the
poor old operator have?

St.Ack


On Fri, Sep 8, 2017 at 1:59 PM, Vladimir Rodionov <vl...@gmail.com>
wrote:

> HBASE-14414
>
> On Fri, Sep 8, 2017 at 1:14 PM, Stack <st...@duboce.net> wrote:
>
> > Where do I go to get the current status of this feature? Looking in JIRA
> I
> > see loads of issues open against backup including some against
> hbase-2.0.0
> > and no progress being made that I can discern.
> >
> > Thanks,
> > S
> >
> >
> >
> > On Wed, Nov 23, 2016 at 8:52 AM, Stack <st...@duboce.net> wrote:
> >
> > > On Tue, Nov 22, 2016 at 6:48 PM, Stack <st...@duboce.net> wrote:
> > >
> > >> On Tue, Nov 22, 2016 at 3:17 PM, Vladimir Rodionov <
> > >> vladrodionov@gmail.com> wrote:
> > >>
> > >>> >> and/or he answered most of the review feedback
> > >>>
> > >>> No, questions are still open, but I do not see any blockers and we
> have
> > >>> HBASE-16940 to address these questions.
> > >>>
> > >>>
> > >> Agree. No blockers but stuff that should be dealt with (No one will
> pay
> > >> me any attention once merge goes in -- smile).
> > >>
> > >>
> > > Let me clarify the above. I want review addressed before merge happens.
> > > Sorry if any confusion.
> > > St.Ack
> > >
> > >
> > >
> > >
> > >
> > >
> > >> St.Ack
> > >>
> > >>
> > >>
> > >>> On Tue, Nov 22, 2016 at 3:04 PM, Devaraj Das <dd...@hortonworks.com>
> > >>> wrote:
> > >>>
> > >>> > Hi Stack, hats off to you for spending so much time on this!
> Thanks!
> > >>> From
> > >>> > my understanding, Vlad has raised follow-up jiras for the issues
> you
> > >>> > raised, and/or he answered most of the review feedback. So, do you
> > >>> think we
> > >>> > could do a merge vote now?
> > >>> > Devaraj.
> > >>> > ________________________________________
> > >>> > From: Vladimir Rodionov <vl...@gmail.com>
> > >>> > Sent: Monday, November 21, 2016 8:34 PM
> > >>> > To: dev@hbase.apache.org
> > >>> > Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch
> HBASE-7912
> > >>> >
> > >>> > >> I have spent a good bit of time reviewing and testing this
> > feature.
> > >>> I
> > >>> > would
> > >>> > >> like my review and concerns addressed and I'd like it to be
> clear
> > >>> how;
> > >>> > >> either explicit follow-on issues, pointers to where in the patch
> > or
> > >>> doc
> > >>> > my
> > >>> > >> remarks have been catered to, etc. Until then, I am against
> > commit.
> > >>> >
> > >>> > Stack, mega patch review comments will be addressed in the
> dedicated
> > >>> JIRA:
> > >>> > HBASE-16940
> > >>> > I have open several other JIRAs to address your other comments (not
> > on
> > >>> > review board).
> > >>> >
> > >>> > Details are here (end of the thread):
> > >>> > https://issues.apache.org/jira/browse/HBASE-14123
> > >>> >
> > >>> > Let me know what else should we do to move merge forward.
> > >>> >
> > >>> > -Vlad
> > >>> >
> > >>> >
> > >>> > On Fri, Nov 18, 2016 at 4:54 PM, Stack <st...@duboce.net> wrote:
> > >>> >
> > >>> > > On Fri, Nov 18, 2016 at 3:53 PM, Ted Yu <yu...@gmail.com>
> > wrote:
> > >>> > >
> > >>> > > > Thanks, Matteo.
> > >>> > > >
> > >>> > > > bq. restore is not clear if given an incremental id it will do
> > the
> > >>> full
> > >>> > > > restore from full up to that point or if i need to apply
> manually
> > >>> > > > everything
> > >>> > > >
> > >>> > > > The restore takes into consideration of the dependent
> backup(s).
> > >>> > > > So there is no need to apply preceding backup(s) manually.
> > >>> > > >
> > >>> > > >
> > >>> > > I ask this question on the issue. It is not clear from the usage
> or
> > >>> doc
> > >>> > how
> > >>> > > to run a restore from incremental. Can you fix in doc and usage
> how
> > >>> so I
> > >>> > > can be clear and try it. Currently I am stuck verifying a round
> > trip
> > >>> > backup
> > >>> > > restore made of incrementals.
> > >>> > >
> > >>> > > Thanks,
> > >>> > > S
> > >>> > >
> > >>> > >
> > >>> > >
> > >>> > > > On Fri, Nov 18, 2016 at 3:48 PM, Matteo Bertozzi <
> > >>> > > theo.bertozzi@gmail.com>
> > >>> > > > wrote:
> > >>> > > >
> > >>> > > > > I did one last pass to the mega patch. I don't see anything
> > major
> > >>> > that
> > >>> > > > > should block the merge.
> > >>> > > > >
> > >>> > > > > - most of the code is isolated in the backup package
> > >>> > > > > - all the backup code is client side
> > >>> > > > > - there are few changes to the server side, mainly for
> > cleaners,
> > >>> wal
> > >>> > > > > rolling and similar (which is ok)
> > >>> > > > > - there is a good number of tests, and an integration test
> > >>> > > > >
> > >>> > > > > the code seems to have still some left overs from the old
> > >>> > > implementation,
> > >>> > > > > and some stuff needs a cleanup. but I don't think this should
> > be
> > >>> used
> > >>> > > as
> > >>> > > > an
> > >>> > > > > argument to block the merge. I think the guys will keep
> working
> > >>> on
> > >>> > this
> > >>> > > > and
> > >>> > > > > they may also get help of others once the patch is in master.
> > >>> > > > >
> > >>> > > > > I still have my concerns about the current limitations, but
> > >>> these are
> > >>> > > > > things already planned for phase 3, so some of this stuff may
> > >>> even be
> > >>> > > in
> > >>> > > > > the final 2.0.
> > >>> > > > > but as long as we have a "current limitations" section in the
> > >>> user
> > >>> > > guide
> > >>> > > > > mentioning important stuff like the ones below, I'm ok with
> it.
> > >>> > > > >  - if you write to the table with Durability.SKIP_WALS your
> > data
> > >>> will
> > >>> > > not
> > >>> > > > > be in the incremental-backup
> > >>> > > > >  - if you bulkload files that data will not be in the
> > incremental
> > >>> > > backup
> > >>> > > > > (HBASE-14417)
> > >>> > > > >  - the incremental backup will not only contains the data of
> > the
> > >>> > table
> > >>> > > > you
> > >>> > > > > specified but also the regions from other tables that are on
> > the
> > >>> same
> > >>> > > set
> > >>> > > > > of RSs (HBASE-14141) ...maybe a note about security around
> this
> > >>> topic
> > >>> > > > >  - the incremental backup will not contains just the "latest
> > row"
> > >>> > > between
> > >>> > > > > backup A and B, but it will also contains all the updates
> > >>> occurred in
> > >>> > > > > between. but the restore does not allow you to restore up to
> a
> > >>> > certain
> > >>> > > > > point in time, the restore will always be up to the "latest
> > >>> backup
> > >>> > > > point".
> > >>> > > > >  - you should limit the number of "incremental" up to N (or
> > maybe
> > >>> > > SIZE),
> > >>> > > > to
> > >>> > > > > avoid replay time becoming the bottleneck. (HBASE-14135)
> > >>> > > > >
> > >>> > > > > I'll be ok even with the above not being in the final 2.0,
> > >>> > > > > but i'd like to see as blocker for the final 2.0 (not the
> > merge)
> > >>> > > > >  - the backup code moved in an hbase-backup module
> > >>> > > > >  - and some more work around tools, especially to try to
> unify
> > >>> and
> > >>> > make
> > >>> > > > > simple the backup experience (simple example: in some case
> > there
> > >>> is a
> > >>> > > > > backup_id argument in others a backupId argument. or things
> > >>> like..
> > >>> > > > restore
> > >>> > > > > is not clear if given an incremental id it will do the full
> > >>> restore
> > >>> > > from
> > >>> > > > > full up to that point or if i need to apply manually
> > everything).
> > >>> > > > >
> > >>> > > > > in conclusion, I think we can open a merge vote. I'll be +1
> on
> > >>> it,
> > >>> > and
> > >>> > > I
> > >>> > > > > think we should try to reject -1 with just a "code cleanup"
> > >>> > motivation,
> > >>> > > > > since there will still be work going on on the code after the
> > >>> merge.
> > >>> > > > >
> > >>> > > > > Matteo
> > >>> > > > >
> > >>> > > > >
> > >>> > > > > On Sun, Nov 6, 2016 at 10:54 PM, Devaraj Das <
> > >>> ddas@hortonworks.com>
> > >>> > > > wrote:
> > >>> > > > >
> > >>> > > > > > Stack and others, anything else on the patch? Merge to
> master
> > >>> now?
> > >>> > > > > >
> > >>> > > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> > >>
> > >>
> > >
> >
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Vladimir Rodionov <vl...@gmail.com>]

All blockers have been resolved. Some remaining JIRAs are good to have but
are not a blockers

On Fri, Sep 8, 2017 at 1:59 PM, Vladimir Rodionov <vl...@gmail.com>
wrote:

> HBASE-14414
>
> On Fri, Sep 8, 2017 at 1:14 PM, Stack <st...@duboce.net> wrote:
>
>> Where do I go to get the current status of this feature? Looking in JIRA I
>> see loads of issues open against backup including some against hbase-2.0.0
>> and no progress being made that I can discern.
>>
>> Thanks,
>> S
>>
>>
>>
>> On Wed, Nov 23, 2016 at 8:52 AM, Stack <st...@duboce.net> wrote:
>>
>> > On Tue, Nov 22, 2016 at 6:48 PM, Stack <st...@duboce.net> wrote:
>> >
>> >> On Tue, Nov 22, 2016 at 3:17 PM, Vladimir Rodionov <
>> >> vladrodionov@gmail.com> wrote:
>> >>
>> >>> >> and/or he answered most of the review feedback
>> >>>
>> >>> No, questions are still open, but I do not see any blockers and we
>> have
>> >>> HBASE-16940 to address these questions.
>> >>>
>> >>>
>> >> Agree. No blockers but stuff that should be dealt with (No one will pay
>> >> me any attention once merge goes in -- smile).
>> >>
>> >>
>> > Let me clarify the above. I want review addressed before merge happens.
>> > Sorry if any confusion.
>> > St.Ack
>> >
>> >
>> >
>> >
>> >
>> >
>> >> St.Ack
>> >>
>> >>
>> >>
>> >>> On Tue, Nov 22, 2016 at 3:04 PM, Devaraj Das <dd...@hortonworks.com>
>> >>> wrote:
>> >>>
>> >>> > Hi Stack, hats off to you for spending so much time on this! Thanks!
>> >>> From
>> >>> > my understanding, Vlad has raised follow-up jiras for the issues you
>> >>> > raised, and/or he answered most of the review feedback. So, do you
>> >>> think we
>> >>> > could do a merge vote now?
>> >>> > Devaraj.
>> >>> > ________________________________________
>> >>> > From: Vladimir Rodionov <vl...@gmail.com>
>> >>> > Sent: Monday, November 21, 2016 8:34 PM
>> >>> > To: dev@hbase.apache.org
>> >>> > Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912
>> >>> >
>> >>> > >> I have spent a good bit of time reviewing and testing this
>> feature.
>> >>> I
>> >>> > would
>> >>> > >> like my review and concerns addressed and I'd like it to be clear
>> >>> how;
>> >>> > >> either explicit follow-on issues, pointers to where in the patch
>> or
>> >>> doc
>> >>> > my
>> >>> > >> remarks have been catered to, etc. Until then, I am against
>> commit.
>> >>> >
>> >>> > Stack, mega patch review comments will be addressed in the dedicated
>> >>> JIRA:
>> >>> > HBASE-16940
>> >>> > I have open several other JIRAs to address your other comments (not
>> on
>> >>> > review board).
>> >>> >
>> >>> > Details are here (end of the thread):
>> >>> > https://issues.apache.org/jira/browse/HBASE-14123
>> >>> >
>> >>> > Let me know what else should we do to move merge forward.
>> >>> >
>> >>> > -Vlad
>> >>> >
>> >>> >
>> >>> > On Fri, Nov 18, 2016 at 4:54 PM, Stack <st...@duboce.net> wrote:
>> >>> >
>> >>> > > On Fri, Nov 18, 2016 at 3:53 PM, Ted Yu <yu...@gmail.com>
>> wrote:
>> >>> > >
>> >>> > > > Thanks, Matteo.
>> >>> > > >
>> >>> > > > bq. restore is not clear if given an incremental id it will do
>> the
>> >>> full
>> >>> > > > restore from full up to that point or if i need to apply
>> manually
>> >>> > > > everything
>> >>> > > >
>> >>> > > > The restore takes into consideration of the dependent backup(s).
>> >>> > > > So there is no need to apply preceding backup(s) manually.
>> >>> > > >
>> >>> > > >
>> >>> > > I ask this question on the issue. It is not clear from the usage
>> or
>> >>> doc
>> >>> > how
>> >>> > > to run a restore from incremental. Can you fix in doc and usage
>> how
>> >>> so I
>> >>> > > can be clear and try it. Currently I am stuck verifying a round
>> trip
>> >>> > backup
>> >>> > > restore made of incrementals.
>> >>> > >
>> >>> > > Thanks,
>> >>> > > S
>> >>> > >
>> >>> > >
>> >>> > >
>> >>> > > > On Fri, Nov 18, 2016 at 3:48 PM, Matteo Bertozzi <
>> >>> > > theo.bertozzi@gmail.com>
>> >>> > > > wrote:
>> >>> > > >
>> >>> > > > > I did one last pass to the mega patch. I don't see anything
>> major
>> >>> > that
>> >>> > > > > should block the merge.
>> >>> > > > >
>> >>> > > > > - most of the code is isolated in the backup package
>> >>> > > > > - all the backup code is client side
>> >>> > > > > - there are few changes to the server side, mainly for
>> cleaners,
>> >>> wal
>> >>> > > > > rolling and similar (which is ok)
>> >>> > > > > - there is a good number of tests, and an integration test
>> >>> > > > >
>> >>> > > > > the code seems to have still some left overs from the old
>> >>> > > implementation,
>> >>> > > > > and some stuff needs a cleanup. but I don't think this should
>> be
>> >>> used
>> >>> > > as
>> >>> > > > an
>> >>> > > > > argument to block the merge. I think the guys will keep
>> working
>> >>> on
>> >>> > this
>> >>> > > > and
>> >>> > > > > they may also get help of others once the patch is in master.
>> >>> > > > >
>> >>> > > > > I still have my concerns about the current limitations, but
>> >>> these are
>> >>> > > > > things already planned for phase 3, so some of this stuff may
>> >>> even be
>> >>> > > in
>> >>> > > > > the final 2.0.
>> >>> > > > > but as long as we have a "current limitations" section in the
>> >>> user
>> >>> > > guide
>> >>> > > > > mentioning important stuff like the ones below, I'm ok with
>> it.
>> >>> > > > >  - if you write to the table with Durability.SKIP_WALS your
>> data
>> >>> will
>> >>> > > not
>> >>> > > > > be in the incremental-backup
>> >>> > > > >  - if you bulkload files that data will not be in the
>> incremental
>> >>> > > backup
>> >>> > > > > (HBASE-14417)
>> >>> > > > >  - the incremental backup will not only contains the data of
>> the
>> >>> > table
>> >>> > > > you
>> >>> > > > > specified but also the regions from other tables that are on
>> the
>> >>> same
>> >>> > > set
>> >>> > > > > of RSs (HBASE-14141) ...maybe a note about security around
>> this
>> >>> topic
>> >>> > > > >  - the incremental backup will not contains just the "latest
>> row"
>> >>> > > between
>> >>> > > > > backup A and B, but it will also contains all the updates
>> >>> occurred in
>> >>> > > > > between. but the restore does not allow you to restore up to a
>> >>> > certain
>> >>> > > > > point in time, the restore will always be up to the "latest
>> >>> backup
>> >>> > > > point".
>> >>> > > > >  - you should limit the number of "incremental" up to N (or
>> maybe
>> >>> > > SIZE),
>> >>> > > > to
>> >>> > > > > avoid replay time becoming the bottleneck. (HBASE-14135)
>> >>> > > > >
>> >>> > > > > I'll be ok even with the above not being in the final 2.0,
>> >>> > > > > but i'd like to see as blocker for the final 2.0 (not the
>> merge)
>> >>> > > > >  - the backup code moved in an hbase-backup module
>> >>> > > > >  - and some more work around tools, especially to try to unify
>> >>> and
>> >>> > make
>> >>> > > > > simple the backup experience (simple example: in some case
>> there
>> >>> is a
>> >>> > > > > backup_id argument in others a backupId argument. or things
>> >>> like..
>> >>> > > > restore
>> >>> > > > > is not clear if given an incremental id it will do the full
>> >>> restore
>> >>> > > from
>> >>> > > > > full up to that point or if i need to apply manually
>> everything).
>> >>> > > > >
>> >>> > > > > in conclusion, I think we can open a merge vote. I'll be +1 on
>> >>> it,
>> >>> > and
>> >>> > > I
>> >>> > > > > think we should try to reject -1 with just a "code cleanup"
>> >>> > motivation,
>> >>> > > > > since there will still be work going on on the code after the
>> >>> merge.
>> >>> > > > >
>> >>> > > > > Matteo
>> >>> > > > >
>> >>> > > > >
>> >>> > > > > On Sun, Nov 6, 2016 at 10:54 PM, Devaraj Das <
>> >>> ddas@hortonworks.com>
>> >>> > > > wrote:
>> >>> > > > >
>> >>> > > > > > Stack and others, anything else on the patch? Merge to
>> master
>> >>> now?
>> >>> > > > > >
>> >>> > > > >
>> >>> > > >
>> >>> > >
>> >>> >
>> >>>
>> >>
>> >>
>> >
>>
>
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Vladimir Rodionov <vl...@gmail.com>]

HBASE-14414

On Fri, Sep 8, 2017 at 1:14 PM, Stack <st...@duboce.net> wrote:

> Where do I go to get the current status of this feature? Looking in JIRA I
> see loads of issues open against backup including some against hbase-2.0.0
> and no progress being made that I can discern.
>
> Thanks,
> S
>
>
>
> On Wed, Nov 23, 2016 at 8:52 AM, Stack <st...@duboce.net> wrote:
>
> > On Tue, Nov 22, 2016 at 6:48 PM, Stack <st...@duboce.net> wrote:
> >
> >> On Tue, Nov 22, 2016 at 3:17 PM, Vladimir Rodionov <
> >> vladrodionov@gmail.com> wrote:
> >>
> >>> >> and/or he answered most of the review feedback
> >>>
> >>> No, questions are still open, but I do not see any blockers and we have
> >>> HBASE-16940 to address these questions.
> >>>
> >>>
> >> Agree. No blockers but stuff that should be dealt with (No one will pay
> >> me any attention once merge goes in -- smile).
> >>
> >>
> > Let me clarify the above. I want review addressed before merge happens.
> > Sorry if any confusion.
> > St.Ack
> >
> >
> >
> >
> >
> >
> >> St.Ack
> >>
> >>
> >>
> >>> On Tue, Nov 22, 2016 at 3:04 PM, Devaraj Das <dd...@hortonworks.com>
> >>> wrote:
> >>>
> >>> > Hi Stack, hats off to you for spending so much time on this! Thanks!
> >>> From
> >>> > my understanding, Vlad has raised follow-up jiras for the issues you
> >>> > raised, and/or he answered most of the review feedback. So, do you
> >>> think we
> >>> > could do a merge vote now?
> >>> > Devaraj.
> >>> > ________________________________________
> >>> > From: Vladimir Rodionov <vl...@gmail.com>
> >>> > Sent: Monday, November 21, 2016 8:34 PM
> >>> > To: dev@hbase.apache.org
> >>> > Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912
> >>> >
> >>> > >> I have spent a good bit of time reviewing and testing this
> feature.
> >>> I
> >>> > would
> >>> > >> like my review and concerns addressed and I'd like it to be clear
> >>> how;
> >>> > >> either explicit follow-on issues, pointers to where in the patch
> or
> >>> doc
> >>> > my
> >>> > >> remarks have been catered to, etc. Until then, I am against
> commit.
> >>> >
> >>> > Stack, mega patch review comments will be addressed in the dedicated
> >>> JIRA:
> >>> > HBASE-16940
> >>> > I have open several other JIRAs to address your other comments (not
> on
> >>> > review board).
> >>> >
> >>> > Details are here (end of the thread):
> >>> > https://issues.apache.org/jira/browse/HBASE-14123
> >>> >
> >>> > Let me know what else should we do to move merge forward.
> >>> >
> >>> > -Vlad
> >>> >
> >>> >
> >>> > On Fri, Nov 18, 2016 at 4:54 PM, Stack <st...@duboce.net> wrote:
> >>> >
> >>> > > On Fri, Nov 18, 2016 at 3:53 PM, Ted Yu <yu...@gmail.com>
> wrote:
> >>> > >
> >>> > > > Thanks, Matteo.
> >>> > > >
> >>> > > > bq. restore is not clear if given an incremental id it will do
> the
> >>> full
> >>> > > > restore from full up to that point or if i need to apply manually
> >>> > > > everything
> >>> > > >
> >>> > > > The restore takes into consideration of the dependent backup(s).
> >>> > > > So there is no need to apply preceding backup(s) manually.
> >>> > > >
> >>> > > >
> >>> > > I ask this question on the issue. It is not clear from the usage or
> >>> doc
> >>> > how
> >>> > > to run a restore from incremental. Can you fix in doc and usage how
> >>> so I
> >>> > > can be clear and try it. Currently I am stuck verifying a round
> trip
> >>> > backup
> >>> > > restore made of incrementals.
> >>> > >
> >>> > > Thanks,
> >>> > > S
> >>> > >
> >>> > >
> >>> > >
> >>> > > > On Fri, Nov 18, 2016 at 3:48 PM, Matteo Bertozzi <
> >>> > > theo.bertozzi@gmail.com>
> >>> > > > wrote:
> >>> > > >
> >>> > > > > I did one last pass to the mega patch. I don't see anything
> major
> >>> > that
> >>> > > > > should block the merge.
> >>> > > > >
> >>> > > > > - most of the code is isolated in the backup package
> >>> > > > > - all the backup code is client side
> >>> > > > > - there are few changes to the server side, mainly for
> cleaners,
> >>> wal
> >>> > > > > rolling and similar (which is ok)
> >>> > > > > - there is a good number of tests, and an integration test
> >>> > > > >
> >>> > > > > the code seems to have still some left overs from the old
> >>> > > implementation,
> >>> > > > > and some stuff needs a cleanup. but I don't think this should
> be
> >>> used
> >>> > > as
> >>> > > > an
> >>> > > > > argument to block the merge. I think the guys will keep working
> >>> on
> >>> > this
> >>> > > > and
> >>> > > > > they may also get help of others once the patch is in master.
> >>> > > > >
> >>> > > > > I still have my concerns about the current limitations, but
> >>> these are
> >>> > > > > things already planned for phase 3, so some of this stuff may
> >>> even be
> >>> > > in
> >>> > > > > the final 2.0.
> >>> > > > > but as long as we have a "current limitations" section in the
> >>> user
> >>> > > guide
> >>> > > > > mentioning important stuff like the ones below, I'm ok with it.
> >>> > > > >  - if you write to the table with Durability.SKIP_WALS your
> data
> >>> will
> >>> > > not
> >>> > > > > be in the incremental-backup
> >>> > > > >  - if you bulkload files that data will not be in the
> incremental
> >>> > > backup
> >>> > > > > (HBASE-14417)
> >>> > > > >  - the incremental backup will not only contains the data of
> the
> >>> > table
> >>> > > > you
> >>> > > > > specified but also the regions from other tables that are on
> the
> >>> same
> >>> > > set
> >>> > > > > of RSs (HBASE-14141) ...maybe a note about security around this
> >>> topic
> >>> > > > >  - the incremental backup will not contains just the "latest
> row"
> >>> > > between
> >>> > > > > backup A and B, but it will also contains all the updates
> >>> occurred in
> >>> > > > > between. but the restore does not allow you to restore up to a
> >>> > certain
> >>> > > > > point in time, the restore will always be up to the "latest
> >>> backup
> >>> > > > point".
> >>> > > > >  - you should limit the number of "incremental" up to N (or
> maybe
> >>> > > SIZE),
> >>> > > > to
> >>> > > > > avoid replay time becoming the bottleneck. (HBASE-14135)
> >>> > > > >
> >>> > > > > I'll be ok even with the above not being in the final 2.0,
> >>> > > > > but i'd like to see as blocker for the final 2.0 (not the
> merge)
> >>> > > > >  - the backup code moved in an hbase-backup module
> >>> > > > >  - and some more work around tools, especially to try to unify
> >>> and
> >>> > make
> >>> > > > > simple the backup experience (simple example: in some case
> there
> >>> is a
> >>> > > > > backup_id argument in others a backupId argument. or things
> >>> like..
> >>> > > > restore
> >>> > > > > is not clear if given an incremental id it will do the full
> >>> restore
> >>> > > from
> >>> > > > > full up to that point or if i need to apply manually
> everything).
> >>> > > > >
> >>> > > > > in conclusion, I think we can open a merge vote. I'll be +1 on
> >>> it,
> >>> > and
> >>> > > I
> >>> > > > > think we should try to reject -1 with just a "code cleanup"
> >>> > motivation,
> >>> > > > > since there will still be work going on on the code after the
> >>> merge.
> >>> > > > >
> >>> > > > > Matteo
> >>> > > > >
> >>> > > > >
> >>> > > > > On Sun, Nov 6, 2016 at 10:54 PM, Devaraj Das <
> >>> ddas@hortonworks.com>
> >>> > > > wrote:
> >>> > > > >
> >>> > > > > > Stack and others, anything else on the patch? Merge to master
> >>> now?
> >>> > > > > >
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
> >>
> >>
> >
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Stack <st...@duboce.net>]

Where do I go to get the current status of this feature? Looking in JIRA I
see loads of issues open against backup including some against hbase-2.0.0
and no progress being made that I can discern.

Thanks,
S



On Wed, Nov 23, 2016 at 8:52 AM, Stack <st...@duboce.net> wrote:

> On Tue, Nov 22, 2016 at 6:48 PM, Stack <st...@duboce.net> wrote:
>
>> On Tue, Nov 22, 2016 at 3:17 PM, Vladimir Rodionov <
>> vladrodionov@gmail.com> wrote:
>>
>>> >> and/or he answered most of the review feedback
>>>
>>> No, questions are still open, but I do not see any blockers and we have
>>> HBASE-16940 to address these questions.
>>>
>>>
>> Agree. No blockers but stuff that should be dealt with (No one will pay
>> me any attention once merge goes in -- smile).
>>
>>
> Let me clarify the above. I want review addressed before merge happens.
> Sorry if any confusion.
> St.Ack
>
>
>
>
>
>
>> St.Ack
>>
>>
>>
>>> On Tue, Nov 22, 2016 at 3:04 PM, Devaraj Das <dd...@hortonworks.com>
>>> wrote:
>>>
>>> > Hi Stack, hats off to you for spending so much time on this! Thanks!
>>> From
>>> > my understanding, Vlad has raised follow-up jiras for the issues you
>>> > raised, and/or he answered most of the review feedback. So, do you
>>> think we
>>> > could do a merge vote now?
>>> > Devaraj.
>>> > ________________________________________
>>> > From: Vladimir Rodionov <vl...@gmail.com>
>>> > Sent: Monday, November 21, 2016 8:34 PM
>>> > To: dev@hbase.apache.org
>>> > Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912
>>> >
>>> > >> I have spent a good bit of time reviewing and testing this feature.
>>> I
>>> > would
>>> > >> like my review and concerns addressed and I'd like it to be clear
>>> how;
>>> > >> either explicit follow-on issues, pointers to where in the patch or
>>> doc
>>> > my
>>> > >> remarks have been catered to, etc. Until then, I am against commit.
>>> >
>>> > Stack, mega patch review comments will be addressed in the dedicated
>>> JIRA:
>>> > HBASE-16940
>>> > I have open several other JIRAs to address your other comments (not on
>>> > review board).
>>> >
>>> > Details are here (end of the thread):
>>> > https://issues.apache.org/jira/browse/HBASE-14123
>>> >
>>> > Let me know what else should we do to move merge forward.
>>> >
>>> > -Vlad
>>> >
>>> >
>>> > On Fri, Nov 18, 2016 at 4:54 PM, Stack <st...@duboce.net> wrote:
>>> >
>>> > > On Fri, Nov 18, 2016 at 3:53 PM, Ted Yu <yu...@gmail.com> wrote:
>>> > >
>>> > > > Thanks, Matteo.
>>> > > >
>>> > > > bq. restore is not clear if given an incremental id it will do the
>>> full
>>> > > > restore from full up to that point or if i need to apply manually
>>> > > > everything
>>> > > >
>>> > > > The restore takes into consideration of the dependent backup(s).
>>> > > > So there is no need to apply preceding backup(s) manually.
>>> > > >
>>> > > >
>>> > > I ask this question on the issue. It is not clear from the usage or
>>> doc
>>> > how
>>> > > to run a restore from incremental. Can you fix in doc and usage how
>>> so I
>>> > > can be clear and try it. Currently I am stuck verifying a round trip
>>> > backup
>>> > > restore made of incrementals.
>>> > >
>>> > > Thanks,
>>> > > S
>>> > >
>>> > >
>>> > >
>>> > > > On Fri, Nov 18, 2016 at 3:48 PM, Matteo Bertozzi <
>>> > > theo.bertozzi@gmail.com>
>>> > > > wrote:
>>> > > >
>>> > > > > I did one last pass to the mega patch. I don't see anything major
>>> > that
>>> > > > > should block the merge.
>>> > > > >
>>> > > > > - most of the code is isolated in the backup package
>>> > > > > - all the backup code is client side
>>> > > > > - there are few changes to the server side, mainly for cleaners,
>>> wal
>>> > > > > rolling and similar (which is ok)
>>> > > > > - there is a good number of tests, and an integration test
>>> > > > >
>>> > > > > the code seems to have still some left overs from the old
>>> > > implementation,
>>> > > > > and some stuff needs a cleanup. but I don't think this should be
>>> used
>>> > > as
>>> > > > an
>>> > > > > argument to block the merge. I think the guys will keep working
>>> on
>>> > this
>>> > > > and
>>> > > > > they may also get help of others once the patch is in master.
>>> > > > >
>>> > > > > I still have my concerns about the current limitations, but
>>> these are
>>> > > > > things already planned for phase 3, so some of this stuff may
>>> even be
>>> > > in
>>> > > > > the final 2.0.
>>> > > > > but as long as we have a "current limitations" section in the
>>> user
>>> > > guide
>>> > > > > mentioning important stuff like the ones below, I'm ok with it.
>>> > > > >  - if you write to the table with Durability.SKIP_WALS your data
>>> will
>>> > > not
>>> > > > > be in the incremental-backup
>>> > > > >  - if you bulkload files that data will not be in the incremental
>>> > > backup
>>> > > > > (HBASE-14417)
>>> > > > >  - the incremental backup will not only contains the data of the
>>> > table
>>> > > > you
>>> > > > > specified but also the regions from other tables that are on the
>>> same
>>> > > set
>>> > > > > of RSs (HBASE-14141) ...maybe a note about security around this
>>> topic
>>> > > > >  - the incremental backup will not contains just the "latest row"
>>> > > between
>>> > > > > backup A and B, but it will also contains all the updates
>>> occurred in
>>> > > > > between. but the restore does not allow you to restore up to a
>>> > certain
>>> > > > > point in time, the restore will always be up to the "latest
>>> backup
>>> > > > point".
>>> > > > >  - you should limit the number of "incremental" up to N (or maybe
>>> > > SIZE),
>>> > > > to
>>> > > > > avoid replay time becoming the bottleneck. (HBASE-14135)
>>> > > > >
>>> > > > > I'll be ok even with the above not being in the final 2.0,
>>> > > > > but i'd like to see as blocker for the final 2.0 (not the merge)
>>> > > > >  - the backup code moved in an hbase-backup module
>>> > > > >  - and some more work around tools, especially to try to unify
>>> and
>>> > make
>>> > > > > simple the backup experience (simple example: in some case there
>>> is a
>>> > > > > backup_id argument in others a backupId argument. or things
>>> like..
>>> > > > restore
>>> > > > > is not clear if given an incremental id it will do the full
>>> restore
>>> > > from
>>> > > > > full up to that point or if i need to apply manually everything).
>>> > > > >
>>> > > > > in conclusion, I think we can open a merge vote. I'll be +1 on
>>> it,
>>> > and
>>> > > I
>>> > > > > think we should try to reject -1 with just a "code cleanup"
>>> > motivation,
>>> > > > > since there will still be work going on on the code after the
>>> merge.
>>> > > > >
>>> > > > > Matteo
>>> > > > >
>>> > > > >
>>> > > > > On Sun, Nov 6, 2016 at 10:54 PM, Devaraj Das <
>>> ddas@hortonworks.com>
>>> > > > wrote:
>>> > > > >
>>> > > > > > Stack and others, anything else on the patch? Merge to master
>>> now?
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>>
>>
>>
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Stack <st...@duboce.net>]

On Tue, Nov 22, 2016 at 6:48 PM, Stack <st...@duboce.net> wrote:

> On Tue, Nov 22, 2016 at 3:17 PM, Vladimir Rodionov <vladrodionov@gmail.com
> > wrote:
>
>> >> and/or he answered most of the review feedback
>>
>> No, questions are still open, but I do not see any blockers and we have
>> HBASE-16940 to address these questions.
>>
>>
> Agree. No blockers but stuff that should be dealt with (No one will pay me
> any attention once merge goes in -- smile).
>
>
Let me clarify the above. I want review addressed before merge happens.
Sorry if any confusion.
St.Ack






> St.Ack
>
>
>
>> On Tue, Nov 22, 2016 at 3:04 PM, Devaraj Das <dd...@hortonworks.com>
>> wrote:
>>
>> > Hi Stack, hats off to you for spending so much time on this! Thanks!
>> From
>> > my understanding, Vlad has raised follow-up jiras for the issues you
>> > raised, and/or he answered most of the review feedback. So, do you
>> think we
>> > could do a merge vote now?
>> > Devaraj.
>> > ________________________________________
>> > From: Vladimir Rodionov <vl...@gmail.com>
>> > Sent: Monday, November 21, 2016 8:34 PM
>> > To: dev@hbase.apache.org
>> > Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912
>> >
>> > >> I have spent a good bit of time reviewing and testing this feature. I
>> > would
>> > >> like my review and concerns addressed and I'd like it to be clear
>> how;
>> > >> either explicit follow-on issues, pointers to where in the patch or
>> doc
>> > my
>> > >> remarks have been catered to, etc. Until then, I am against commit.
>> >
>> > Stack, mega patch review comments will be addressed in the dedicated
>> JIRA:
>> > HBASE-16940
>> > I have open several other JIRAs to address your other comments (not on
>> > review board).
>> >
>> > Details are here (end of the thread):
>> > https://issues.apache.org/jira/browse/HBASE-14123
>> >
>> > Let me know what else should we do to move merge forward.
>> >
>> > -Vlad
>> >
>> >
>> > On Fri, Nov 18, 2016 at 4:54 PM, Stack <st...@duboce.net> wrote:
>> >
>> > > On Fri, Nov 18, 2016 at 3:53 PM, Ted Yu <yu...@gmail.com> wrote:
>> > >
>> > > > Thanks, Matteo.
>> > > >
>> > > > bq. restore is not clear if given an incremental id it will do the
>> full
>> > > > restore from full up to that point or if i need to apply manually
>> > > > everything
>> > > >
>> > > > The restore takes into consideration of the dependent backup(s).
>> > > > So there is no need to apply preceding backup(s) manually.
>> > > >
>> > > >
>> > > I ask this question on the issue. It is not clear from the usage or
>> doc
>> > how
>> > > to run a restore from incremental. Can you fix in doc and usage how
>> so I
>> > > can be clear and try it. Currently I am stuck verifying a round trip
>> > backup
>> > > restore made of incrementals.
>> > >
>> > > Thanks,
>> > > S
>> > >
>> > >
>> > >
>> > > > On Fri, Nov 18, 2016 at 3:48 PM, Matteo Bertozzi <
>> > > theo.bertozzi@gmail.com>
>> > > > wrote:
>> > > >
>> > > > > I did one last pass to the mega patch. I don't see anything major
>> > that
>> > > > > should block the merge.
>> > > > >
>> > > > > - most of the code is isolated in the backup package
>> > > > > - all the backup code is client side
>> > > > > - there are few changes to the server side, mainly for cleaners,
>> wal
>> > > > > rolling and similar (which is ok)
>> > > > > - there is a good number of tests, and an integration test
>> > > > >
>> > > > > the code seems to have still some left overs from the old
>> > > implementation,
>> > > > > and some stuff needs a cleanup. but I don't think this should be
>> used
>> > > as
>> > > > an
>> > > > > argument to block the merge. I think the guys will keep working on
>> > this
>> > > > and
>> > > > > they may also get help of others once the patch is in master.
>> > > > >
>> > > > > I still have my concerns about the current limitations, but these
>> are
>> > > > > things already planned for phase 3, so some of this stuff may
>> even be
>> > > in
>> > > > > the final 2.0.
>> > > > > but as long as we have a "current limitations" section in the user
>> > > guide
>> > > > > mentioning important stuff like the ones below, I'm ok with it.
>> > > > >  - if you write to the table with Durability.SKIP_WALS your data
>> will
>> > > not
>> > > > > be in the incremental-backup
>> > > > >  - if you bulkload files that data will not be in the incremental
>> > > backup
>> > > > > (HBASE-14417)
>> > > > >  - the incremental backup will not only contains the data of the
>> > table
>> > > > you
>> > > > > specified but also the regions from other tables that are on the
>> same
>> > > set
>> > > > > of RSs (HBASE-14141) ...maybe a note about security around this
>> topic
>> > > > >  - the incremental backup will not contains just the "latest row"
>> > > between
>> > > > > backup A and B, but it will also contains all the updates
>> occurred in
>> > > > > between. but the restore does not allow you to restore up to a
>> > certain
>> > > > > point in time, the restore will always be up to the "latest backup
>> > > > point".
>> > > > >  - you should limit the number of "incremental" up to N (or maybe
>> > > SIZE),
>> > > > to
>> > > > > avoid replay time becoming the bottleneck. (HBASE-14135)
>> > > > >
>> > > > > I'll be ok even with the above not being in the final 2.0,
>> > > > > but i'd like to see as blocker for the final 2.0 (not the merge)
>> > > > >  - the backup code moved in an hbase-backup module
>> > > > >  - and some more work around tools, especially to try to unify and
>> > make
>> > > > > simple the backup experience (simple example: in some case there
>> is a
>> > > > > backup_id argument in others a backupId argument. or things like..
>> > > > restore
>> > > > > is not clear if given an incremental id it will do the full
>> restore
>> > > from
>> > > > > full up to that point or if i need to apply manually everything).
>> > > > >
>> > > > > in conclusion, I think we can open a merge vote. I'll be +1 on it,
>> > and
>> > > I
>> > > > > think we should try to reject -1 with just a "code cleanup"
>> > motivation,
>> > > > > since there will still be work going on on the code after the
>> merge.
>> > > > >
>> > > > > Matteo
>> > > > >
>> > > > >
>> > > > > On Sun, Nov 6, 2016 at 10:54 PM, Devaraj Das <
>> ddas@hortonworks.com>
>> > > > wrote:
>> > > > >
>> > > > > > Stack and others, anything else on the patch? Merge to master
>> now?
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Stack <st...@duboce.net>]

On Tue, Nov 22, 2016 at 3:17 PM, Vladimir Rodionov <vl...@gmail.com>
wrote:

> >> and/or he answered most of the review feedback
>
> No, questions are still open, but I do not see any blockers and we have
> HBASE-16940 to address these questions.
>
>
Agree. No blockers but stuff that should be dealt with (No one will pay me
any attention once merge goes in -- smile).

St.Ack



> On Tue, Nov 22, 2016 at 3:04 PM, Devaraj Das <dd...@hortonworks.com> wrote:
>
> > Hi Stack, hats off to you for spending so much time on this! Thanks! From
> > my understanding, Vlad has raised follow-up jiras for the issues you
> > raised, and/or he answered most of the review feedback. So, do you think
> we
> > could do a merge vote now?
> > Devaraj.
> > ________________________________________
> > From: Vladimir Rodionov <vl...@gmail.com>
> > Sent: Monday, November 21, 2016 8:34 PM
> > To: dev@hbase.apache.org
> > Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912
> >
> > >> I have spent a good bit of time reviewing and testing this feature. I
> > would
> > >> like my review and concerns addressed and I'd like it to be clear how;
> > >> either explicit follow-on issues, pointers to where in the patch or
> doc
> > my
> > >> remarks have been catered to, etc. Until then, I am against commit.
> >
> > Stack, mega patch review comments will be addressed in the dedicated
> JIRA:
> > HBASE-16940
> > I have open several other JIRAs to address your other comments (not on
> > review board).
> >
> > Details are here (end of the thread):
> > https://issues.apache.org/jira/browse/HBASE-14123
> >
> > Let me know what else should we do to move merge forward.
> >
> > -Vlad
> >
> >
> > On Fri, Nov 18, 2016 at 4:54 PM, Stack <st...@duboce.net> wrote:
> >
> > > On Fri, Nov 18, 2016 at 3:53 PM, Ted Yu <yu...@gmail.com> wrote:
> > >
> > > > Thanks, Matteo.
> > > >
> > > > bq. restore is not clear if given an incremental id it will do the
> full
> > > > restore from full up to that point or if i need to apply manually
> > > > everything
> > > >
> > > > The restore takes into consideration of the dependent backup(s).
> > > > So there is no need to apply preceding backup(s) manually.
> > > >
> > > >
> > > I ask this question on the issue. It is not clear from the usage or doc
> > how
> > > to run a restore from incremental. Can you fix in doc and usage how so
> I
> > > can be clear and try it. Currently I am stuck verifying a round trip
> > backup
> > > restore made of incrementals.
> > >
> > > Thanks,
> > > S
> > >
> > >
> > >
> > > > On Fri, Nov 18, 2016 at 3:48 PM, Matteo Bertozzi <
> > > theo.bertozzi@gmail.com>
> > > > wrote:
> > > >
> > > > > I did one last pass to the mega patch. I don't see anything major
> > that
> > > > > should block the merge.
> > > > >
> > > > > - most of the code is isolated in the backup package
> > > > > - all the backup code is client side
> > > > > - there are few changes to the server side, mainly for cleaners,
> wal
> > > > > rolling and similar (which is ok)
> > > > > - there is a good number of tests, and an integration test
> > > > >
> > > > > the code seems to have still some left overs from the old
> > > implementation,
> > > > > and some stuff needs a cleanup. but I don't think this should be
> used
> > > as
> > > > an
> > > > > argument to block the merge. I think the guys will keep working on
> > this
> > > > and
> > > > > they may also get help of others once the patch is in master.
> > > > >
> > > > > I still have my concerns about the current limitations, but these
> are
> > > > > things already planned for phase 3, so some of this stuff may even
> be
> > > in
> > > > > the final 2.0.
> > > > > but as long as we have a "current limitations" section in the user
> > > guide
> > > > > mentioning important stuff like the ones below, I'm ok with it.
> > > > >  - if you write to the table with Durability.SKIP_WALS your data
> will
> > > not
> > > > > be in the incremental-backup
> > > > >  - if you bulkload files that data will not be in the incremental
> > > backup
> > > > > (HBASE-14417)
> > > > >  - the incremental backup will not only contains the data of the
> > table
> > > > you
> > > > > specified but also the regions from other tables that are on the
> same
> > > set
> > > > > of RSs (HBASE-14141) ...maybe a note about security around this
> topic
> > > > >  - the incremental backup will not contains just the "latest row"
> > > between
> > > > > backup A and B, but it will also contains all the updates occurred
> in
> > > > > between. but the restore does not allow you to restore up to a
> > certain
> > > > > point in time, the restore will always be up to the "latest backup
> > > > point".
> > > > >  - you should limit the number of "incremental" up to N (or maybe
> > > SIZE),
> > > > to
> > > > > avoid replay time becoming the bottleneck. (HBASE-14135)
> > > > >
> > > > > I'll be ok even with the above not being in the final 2.0,
> > > > > but i'd like to see as blocker for the final 2.0 (not the merge)
> > > > >  - the backup code moved in an hbase-backup module
> > > > >  - and some more work around tools, especially to try to unify and
> > make
> > > > > simple the backup experience (simple example: in some case there
> is a
> > > > > backup_id argument in others a backupId argument. or things like..
> > > > restore
> > > > > is not clear if given an incremental id it will do the full restore
> > > from
> > > > > full up to that point or if i need to apply manually everything).
> > > > >
> > > > > in conclusion, I think we can open a merge vote. I'll be +1 on it,
> > and
> > > I
> > > > > think we should try to reject -1 with just a "code cleanup"
> > motivation,
> > > > > since there will still be work going on on the code after the
> merge.
> > > > >
> > > > > Matteo
> > > > >
> > > > >
> > > > > On Sun, Nov 6, 2016 at 10:54 PM, Devaraj Das <ddas@hortonworks.com
> >
> > > > wrote:
> > > > >
> > > > > > Stack and others, anything else on the patch? Merge to master
> now?
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Vladimir Rodionov <vl...@gmail.com>]

>> and/or he answered most of the review feedback

No, questions are still open, but I do not see any blockers and we have
HBASE-16940 to address these questions.

On Tue, Nov 22, 2016 at 3:04 PM, Devaraj Das <dd...@hortonworks.com> wrote:

> Hi Stack, hats off to you for spending so much time on this! Thanks! From
> my understanding, Vlad has raised follow-up jiras for the issues you
> raised, and/or he answered most of the review feedback. So, do you think we
> could do a merge vote now?
> Devaraj.
> ________________________________________
> From: Vladimir Rodionov <vl...@gmail.com>
> Sent: Monday, November 21, 2016 8:34 PM
> To: dev@hbase.apache.org
> Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912
>
> >> I have spent a good bit of time reviewing and testing this feature. I
> would
> >> like my review and concerns addressed and I'd like it to be clear how;
> >> either explicit follow-on issues, pointers to where in the patch or doc
> my
> >> remarks have been catered to, etc. Until then, I am against commit.
>
> Stack, mega patch review comments will be addressed in the dedicated JIRA:
> HBASE-16940
> I have open several other JIRAs to address your other comments (not on
> review board).
>
> Details are here (end of the thread):
> https://issues.apache.org/jira/browse/HBASE-14123
>
> Let me know what else should we do to move merge forward.
>
> -Vlad
>
>
> On Fri, Nov 18, 2016 at 4:54 PM, Stack <st...@duboce.net> wrote:
>
> > On Fri, Nov 18, 2016 at 3:53 PM, Ted Yu <yu...@gmail.com> wrote:
> >
> > > Thanks, Matteo.
> > >
> > > bq. restore is not clear if given an incremental id it will do the full
> > > restore from full up to that point or if i need to apply manually
> > > everything
> > >
> > > The restore takes into consideration of the dependent backup(s).
> > > So there is no need to apply preceding backup(s) manually.
> > >
> > >
> > I ask this question on the issue. It is not clear from the usage or doc
> how
> > to run a restore from incremental. Can you fix in doc and usage how so I
> > can be clear and try it. Currently I am stuck verifying a round trip
> backup
> > restore made of incrementals.
> >
> > Thanks,
> > S
> >
> >
> >
> > > On Fri, Nov 18, 2016 at 3:48 PM, Matteo Bertozzi <
> > theo.bertozzi@gmail.com>
> > > wrote:
> > >
> > > > I did one last pass to the mega patch. I don't see anything major
> that
> > > > should block the merge.
> > > >
> > > > - most of the code is isolated in the backup package
> > > > - all the backup code is client side
> > > > - there are few changes to the server side, mainly for cleaners, wal
> > > > rolling and similar (which is ok)
> > > > - there is a good number of tests, and an integration test
> > > >
> > > > the code seems to have still some left overs from the old
> > implementation,
> > > > and some stuff needs a cleanup. but I don't think this should be used
> > as
> > > an
> > > > argument to block the merge. I think the guys will keep working on
> this
> > > and
> > > > they may also get help of others once the patch is in master.
> > > >
> > > > I still have my concerns about the current limitations, but these are
> > > > things already planned for phase 3, so some of this stuff may even be
> > in
> > > > the final 2.0.
> > > > but as long as we have a "current limitations" section in the user
> > guide
> > > > mentioning important stuff like the ones below, I'm ok with it.
> > > >  - if you write to the table with Durability.SKIP_WALS your data will
> > not
> > > > be in the incremental-backup
> > > >  - if you bulkload files that data will not be in the incremental
> > backup
> > > > (HBASE-14417)
> > > >  - the incremental backup will not only contains the data of the
> table
> > > you
> > > > specified but also the regions from other tables that are on the same
> > set
> > > > of RSs (HBASE-14141) ...maybe a note about security around this topic
> > > >  - the incremental backup will not contains just the "latest row"
> > between
> > > > backup A and B, but it will also contains all the updates occurred in
> > > > between. but the restore does not allow you to restore up to a
> certain
> > > > point in time, the restore will always be up to the "latest backup
> > > point".
> > > >  - you should limit the number of "incremental" up to N (or maybe
> > SIZE),
> > > to
> > > > avoid replay time becoming the bottleneck. (HBASE-14135)
> > > >
> > > > I'll be ok even with the above not being in the final 2.0,
> > > > but i'd like to see as blocker for the final 2.0 (not the merge)
> > > >  - the backup code moved in an hbase-backup module
> > > >  - and some more work around tools, especially to try to unify and
> make
> > > > simple the backup experience (simple example: in some case there is a
> > > > backup_id argument in others a backupId argument. or things like..
> > > restore
> > > > is not clear if given an incremental id it will do the full restore
> > from
> > > > full up to that point or if i need to apply manually everything).
> > > >
> > > > in conclusion, I think we can open a merge vote. I'll be +1 on it,
> and
> > I
> > > > think we should try to reject -1 with just a "code cleanup"
> motivation,
> > > > since there will still be work going on on the code after the merge.
> > > >
> > > > Matteo
> > > >
> > > >
> > > > On Sun, Nov 6, 2016 at 10:54 PM, Devaraj Das <dd...@hortonworks.com>
> > > wrote:
> > > >
> > > > > Stack and others, anything else on the patch? Merge to master now?
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Devaraj Das <dd...@hortonworks.com>]

Hi Stack, hats off to you for spending so much time on this! Thanks! From my understanding, Vlad has raised follow-up jiras for the issues you raised, and/or he answered most of the review feedback. So, do you think we could do a merge vote now?
Devaraj.
________________________________________
From: Vladimir Rodionov <vl...@gmail.com>
Sent: Monday, November 21, 2016 8:34 PM
To: dev@hbase.apache.org
Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

>> I have spent a good bit of time reviewing and testing this feature. I
would
>> like my review and concerns addressed and I'd like it to be clear how;
>> either explicit follow-on issues, pointers to where in the patch or doc
my
>> remarks have been catered to, etc. Until then, I am against commit.

Stack, mega patch review comments will be addressed in the dedicated JIRA:
HBASE-16940
I have open several other JIRAs to address your other comments (not on
review board).

Details are here (end of the thread):
https://issues.apache.org/jira/browse/HBASE-14123

Let me know what else should we do to move merge forward.

-Vlad


On Fri, Nov 18, 2016 at 4:54 PM, Stack <st...@duboce.net> wrote:

> On Fri, Nov 18, 2016 at 3:53 PM, Ted Yu <yu...@gmail.com> wrote:
>
> > Thanks, Matteo.
> >
> > bq. restore is not clear if given an incremental id it will do the full
> > restore from full up to that point or if i need to apply manually
> > everything
> >
> > The restore takes into consideration of the dependent backup(s).
> > So there is no need to apply preceding backup(s) manually.
> >
> >
> I ask this question on the issue. It is not clear from the usage or doc how
> to run a restore from incremental. Can you fix in doc and usage how so I
> can be clear and try it. Currently I am stuck verifying a round trip backup
> restore made of incrementals.
>
> Thanks,
> S
>
>
>
> > On Fri, Nov 18, 2016 at 3:48 PM, Matteo Bertozzi <
> theo.bertozzi@gmail.com>
> > wrote:
> >
> > > I did one last pass to the mega patch. I don't see anything major that
> > > should block the merge.
> > >
> > > - most of the code is isolated in the backup package
> > > - all the backup code is client side
> > > - there are few changes to the server side, mainly for cleaners, wal
> > > rolling and similar (which is ok)
> > > - there is a good number of tests, and an integration test
> > >
> > > the code seems to have still some left overs from the old
> implementation,
> > > and some stuff needs a cleanup. but I don't think this should be used
> as
> > an
> > > argument to block the merge. I think the guys will keep working on this
> > and
> > > they may also get help of others once the patch is in master.
> > >
> > > I still have my concerns about the current limitations, but these are
> > > things already planned for phase 3, so some of this stuff may even be
> in
> > > the final 2.0.
> > > but as long as we have a "current limitations" section in the user
> guide
> > > mentioning important stuff like the ones below, I'm ok with it.
> > >  - if you write to the table with Durability.SKIP_WALS your data will
> not
> > > be in the incremental-backup
> > >  - if you bulkload files that data will not be in the incremental
> backup
> > > (HBASE-14417)
> > >  - the incremental backup will not only contains the data of the table
> > you
> > > specified but also the regions from other tables that are on the same
> set
> > > of RSs (HBASE-14141) ...maybe a note about security around this topic
> > >  - the incremental backup will not contains just the "latest row"
> between
> > > backup A and B, but it will also contains all the updates occurred in
> > > between. but the restore does not allow you to restore up to a certain
> > > point in time, the restore will always be up to the "latest backup
> > point".
> > >  - you should limit the number of "incremental" up to N (or maybe
> SIZE),
> > to
> > > avoid replay time becoming the bottleneck. (HBASE-14135)
> > >
> > > I'll be ok even with the above not being in the final 2.0,
> > > but i'd like to see as blocker for the final 2.0 (not the merge)
> > >  - the backup code moved in an hbase-backup module
> > >  - and some more work around tools, especially to try to unify and make
> > > simple the backup experience (simple example: in some case there is a
> > > backup_id argument in others a backupId argument. or things like..
> > restore
> > > is not clear if given an incremental id it will do the full restore
> from
> > > full up to that point or if i need to apply manually everything).
> > >
> > > in conclusion, I think we can open a merge vote. I'll be +1 on it, and
> I
> > > think we should try to reject -1 with just a "code cleanup" motivation,
> > > since there will still be work going on on the code after the merge.
> > >
> > > Matteo
> > >
> > >
> > > On Sun, Nov 6, 2016 at 10:54 PM, Devaraj Das <dd...@hortonworks.com>
> > wrote:
> > >
> > > > Stack and others, anything else on the patch? Merge to master now?
> > > >
> > >
> >
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Vladimir Rodionov <vl...@gmail.com>]

>> I have spent a good bit of time reviewing and testing this feature. I
would
>> like my review and concerns addressed and I'd like it to be clear how;
>> either explicit follow-on issues, pointers to where in the patch or doc
my
>> remarks have been catered to, etc. Until then, I am against commit.

Stack, mega patch review comments will be addressed in the dedicated JIRA:
HBASE-16940
I have open several other JIRAs to address your other comments (not on
review board).

Details are here (end of the thread):
https://issues.apache.org/jira/browse/HBASE-14123

Let me know what else should we do to move merge forward.

-Vlad


On Fri, Nov 18, 2016 at 4:54 PM, Stack <st...@duboce.net> wrote:

> On Fri, Nov 18, 2016 at 3:53 PM, Ted Yu <yu...@gmail.com> wrote:
>
> > Thanks, Matteo.
> >
> > bq. restore is not clear if given an incremental id it will do the full
> > restore from full up to that point or if i need to apply manually
> > everything
> >
> > The restore takes into consideration of the dependent backup(s).
> > So there is no need to apply preceding backup(s) manually.
> >
> >
> I ask this question on the issue. It is not clear from the usage or doc how
> to run a restore from incremental. Can you fix in doc and usage how so I
> can be clear and try it. Currently I am stuck verifying a round trip backup
> restore made of incrementals.
>
> Thanks,
> S
>
>
>
> > On Fri, Nov 18, 2016 at 3:48 PM, Matteo Bertozzi <
> theo.bertozzi@gmail.com>
> > wrote:
> >
> > > I did one last pass to the mega patch. I don't see anything major that
> > > should block the merge.
> > >
> > > - most of the code is isolated in the backup package
> > > - all the backup code is client side
> > > - there are few changes to the server side, mainly for cleaners, wal
> > > rolling and similar (which is ok)
> > > - there is a good number of tests, and an integration test
> > >
> > > the code seems to have still some left overs from the old
> implementation,
> > > and some stuff needs a cleanup. but I don't think this should be used
> as
> > an
> > > argument to block the merge. I think the guys will keep working on this
> > and
> > > they may also get help of others once the patch is in master.
> > >
> > > I still have my concerns about the current limitations, but these are
> > > things already planned for phase 3, so some of this stuff may even be
> in
> > > the final 2.0.
> > > but as long as we have a "current limitations" section in the user
> guide
> > > mentioning important stuff like the ones below, I'm ok with it.
> > >  - if you write to the table with Durability.SKIP_WALS your data will
> not
> > > be in the incremental-backup
> > >  - if you bulkload files that data will not be in the incremental
> backup
> > > (HBASE-14417)
> > >  - the incremental backup will not only contains the data of the table
> > you
> > > specified but also the regions from other tables that are on the same
> set
> > > of RSs (HBASE-14141) ...maybe a note about security around this topic
> > >  - the incremental backup will not contains just the "latest row"
> between
> > > backup A and B, but it will also contains all the updates occurred in
> > > between. but the restore does not allow you to restore up to a certain
> > > point in time, the restore will always be up to the "latest backup
> > point".
> > >  - you should limit the number of "incremental" up to N (or maybe
> SIZE),
> > to
> > > avoid replay time becoming the bottleneck. (HBASE-14135)
> > >
> > > I'll be ok even with the above not being in the final 2.0,
> > > but i'd like to see as blocker for the final 2.0 (not the merge)
> > >  - the backup code moved in an hbase-backup module
> > >  - and some more work around tools, especially to try to unify and make
> > > simple the backup experience (simple example: in some case there is a
> > > backup_id argument in others a backupId argument. or things like..
> > restore
> > > is not clear if given an incremental id it will do the full restore
> from
> > > full up to that point or if i need to apply manually everything).
> > >
> > > in conclusion, I think we can open a merge vote. I'll be +1 on it, and
> I
> > > think we should try to reject -1 with just a "code cleanup" motivation,
> > > since there will still be work going on on the code after the merge.
> > >
> > > Matteo
> > >
> > >
> > > On Sun, Nov 6, 2016 at 10:54 PM, Devaraj Das <dd...@hortonworks.com>
> > wrote:
> > >
> > > > Stack and others, anything else on the patch? Merge to master now?
> > > >
> > >
> >
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Stack <st...@duboce.net>]

On Fri, Nov 18, 2016 at 3:53 PM, Ted Yu <yu...@gmail.com> wrote:

> Thanks, Matteo.
>
> bq. restore is not clear if given an incremental id it will do the full
> restore from full up to that point or if i need to apply manually
> everything
>
> The restore takes into consideration of the dependent backup(s).
> So there is no need to apply preceding backup(s) manually.
>
>
I ask this question on the issue. It is not clear from the usage or doc how
to run a restore from incremental. Can you fix in doc and usage how so I
can be clear and try it. Currently I am stuck verifying a round trip backup
restore made of incrementals.

Thanks,
S



> On Fri, Nov 18, 2016 at 3:48 PM, Matteo Bertozzi <th...@gmail.com>
> wrote:
>
> > I did one last pass to the mega patch. I don't see anything major that
> > should block the merge.
> >
> > - most of the code is isolated in the backup package
> > - all the backup code is client side
> > - there are few changes to the server side, mainly for cleaners, wal
> > rolling and similar (which is ok)
> > - there is a good number of tests, and an integration test
> >
> > the code seems to have still some left overs from the old implementation,
> > and some stuff needs a cleanup. but I don't think this should be used as
> an
> > argument to block the merge. I think the guys will keep working on this
> and
> > they may also get help of others once the patch is in master.
> >
> > I still have my concerns about the current limitations, but these are
> > things already planned for phase 3, so some of this stuff may even be in
> > the final 2.0.
> > but as long as we have a "current limitations" section in the user guide
> > mentioning important stuff like the ones below, I'm ok with it.
> >  - if you write to the table with Durability.SKIP_WALS your data will not
> > be in the incremental-backup
> >  - if you bulkload files that data will not be in the incremental backup
> > (HBASE-14417)
> >  - the incremental backup will not only contains the data of the table
> you
> > specified but also the regions from other tables that are on the same set
> > of RSs (HBASE-14141) ...maybe a note about security around this topic
> >  - the incremental backup will not contains just the "latest row" between
> > backup A and B, but it will also contains all the updates occurred in
> > between. but the restore does not allow you to restore up to a certain
> > point in time, the restore will always be up to the "latest backup
> point".
> >  - you should limit the number of "incremental" up to N (or maybe SIZE),
> to
> > avoid replay time becoming the bottleneck. (HBASE-14135)
> >
> > I'll be ok even with the above not being in the final 2.0,
> > but i'd like to see as blocker for the final 2.0 (not the merge)
> >  - the backup code moved in an hbase-backup module
> >  - and some more work around tools, especially to try to unify and make
> > simple the backup experience (simple example: in some case there is a
> > backup_id argument in others a backupId argument. or things like..
> restore
> > is not clear if given an incremental id it will do the full restore from
> > full up to that point or if i need to apply manually everything).
> >
> > in conclusion, I think we can open a merge vote. I'll be +1 on it, and I
> > think we should try to reject -1 with just a "code cleanup" motivation,
> > since there will still be work going on on the code after the merge.
> >
> > Matteo
> >
> >
> > On Sun, Nov 6, 2016 at 10:54 PM, Devaraj Das <dd...@hortonworks.com>
> wrote:
> >
> > > Stack and others, anything else on the patch? Merge to master now?
> > >
> >
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Ted Yu <yu...@gmail.com>]

Thanks, Matteo.

bq. restore is not clear if given an incremental id it will do the full
restore from full up to that point or if i need to apply manually everything

The restore takes into consideration of the dependent backup(s).
So there is no need to apply preceding backup(s) manually.

On Fri, Nov 18, 2016 at 3:48 PM, Matteo Bertozzi <th...@gmail.com>
wrote:

> I did one last pass to the mega patch. I don't see anything major that
> should block the merge.
>
> - most of the code is isolated in the backup package
> - all the backup code is client side
> - there are few changes to the server side, mainly for cleaners, wal
> rolling and similar (which is ok)
> - there is a good number of tests, and an integration test
>
> the code seems to have still some left overs from the old implementation,
> and some stuff needs a cleanup. but I don't think this should be used as an
> argument to block the merge. I think the guys will keep working on this and
> they may also get help of others once the patch is in master.
>
> I still have my concerns about the current limitations, but these are
> things already planned for phase 3, so some of this stuff may even be in
> the final 2.0.
> but as long as we have a "current limitations" section in the user guide
> mentioning important stuff like the ones below, I'm ok with it.
>  - if you write to the table with Durability.SKIP_WALS your data will not
> be in the incremental-backup
>  - if you bulkload files that data will not be in the incremental backup
> (HBASE-14417)
>  - the incremental backup will not only contains the data of the table you
> specified but also the regions from other tables that are on the same set
> of RSs (HBASE-14141) ...maybe a note about security around this topic
>  - the incremental backup will not contains just the "latest row" between
> backup A and B, but it will also contains all the updates occurred in
> between. but the restore does not allow you to restore up to a certain
> point in time, the restore will always be up to the "latest backup point".
>  - you should limit the number of "incremental" up to N (or maybe SIZE), to
> avoid replay time becoming the bottleneck. (HBASE-14135)
>
> I'll be ok even with the above not being in the final 2.0,
> but i'd like to see as blocker for the final 2.0 (not the merge)
>  - the backup code moved in an hbase-backup module
>  - and some more work around tools, especially to try to unify and make
> simple the backup experience (simple example: in some case there is a
> backup_id argument in others a backupId argument. or things like.. restore
> is not clear if given an incremental id it will do the full restore from
> full up to that point or if i need to apply manually everything).
>
> in conclusion, I think we can open a merge vote. I'll be +1 on it, and I
> think we should try to reject -1 with just a "code cleanup" motivation,
> since there will still be work going on on the code after the merge.
>
> Matteo
>
>
> On Sun, Nov 6, 2016 at 10:54 PM, Devaraj Das <dd...@hortonworks.com> wrote:
>
> > Stack and others, anything else on the patch? Merge to master now?
> >
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Matteo Bertozzi <th...@gmail.com>]

I did one last pass to the mega patch. I don't see anything major that
should block the merge.

- most of the code is isolated in the backup package
- all the backup code is client side
- there are few changes to the server side, mainly for cleaners, wal
rolling and similar (which is ok)
- there is a good number of tests, and an integration test

the code seems to have still some left overs from the old implementation,
and some stuff needs a cleanup. but I don't think this should be used as an
argument to block the merge. I think the guys will keep working on this and
they may also get help of others once the patch is in master.

I still have my concerns about the current limitations, but these are
things already planned for phase 3, so some of this stuff may even be in
the final 2.0.
but as long as we have a "current limitations" section in the user guide
mentioning important stuff like the ones below, I'm ok with it.
 - if you write to the table with Durability.SKIP_WALS your data will not
be in the incremental-backup
 - if you bulkload files that data will not be in the incremental backup
(HBASE-14417)
 - the incremental backup will not only contains the data of the table you
specified but also the regions from other tables that are on the same set
of RSs (HBASE-14141) ...maybe a note about security around this topic
 - the incremental backup will not contains just the "latest row" between
backup A and B, but it will also contains all the updates occurred in
between. but the restore does not allow you to restore up to a certain
point in time, the restore will always be up to the "latest backup point".
 - you should limit the number of "incremental" up to N (or maybe SIZE), to
avoid replay time becoming the bottleneck. (HBASE-14135)

I'll be ok even with the above not being in the final 2.0,
but i'd like to see as blocker for the final 2.0 (not the merge)
 - the backup code moved in an hbase-backup module
 - and some more work around tools, especially to try to unify and make
simple the backup experience (simple example: in some case there is a
backup_id argument in others a backupId argument. or things like.. restore
is not clear if given an incremental id it will do the full restore from
full up to that point or if i need to apply manually everything).

in conclusion, I think we can open a merge vote. I'll be +1 on it, and I
think we should try to reject -1 with just a "code cleanup" motivation,
since there will still be work going on on the code after the merge.

Matteo


On Sun, Nov 6, 2016 at 10:54 PM, Devaraj Das <dd...@hortonworks.com> wrote:

> Stack and others, anything else on the patch? Merge to master now?
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Stack <st...@duboce.net>]

New version of patch was pushed end of last week. I was going to give it a
review.  Takes a couple of hours.  Sorry for delay.
S

On Sun, Nov 6, 2016 at 10:54 PM, Devaraj Das <dd...@hortonworks.com> wrote:

> Stack and others, anything else on the patch? Merge to master now?
> ________________________________________
> From: Ted Yu <yu...@gmail.com>
> Sent: Thursday, October 13, 2016 3:56 PM
> To: dev@hbase.apache.org
> Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912
>
> It is not rebased - but all backup / restore functionality is there.
>
> You can use mega patch v29 over HBASE-14123 which applies to master branch.
>
> Cheers
>
> On Thu, Oct 13, 2016 at 3:51 PM, Apekshit Sharma <ap...@cloudera.com>
> wrote:
>
> > Is HBASE-7912 the feature branch?
> > If yes, has it not been rebased to incorporate latest master changes yet?
> >
> > ~/apache/hbase  (master) → g ck apache/HBASE-7912
> > warning: refname 'apache/HBASE-7912' is ambiguous.
> > Switched to branch 'apache/HBASE-7912'
> > ~/apache/hbase  (apache/HBASE-7912) → g log HEAD..apache/master | grep
> > "^commit" | wc -l
> >      917
> >
> >
> > On Thu, Oct 13, 2016 at 7:01 AM, Ted Yu <yu...@gmail.com> wrote:
> >
> > > Once Stack's comments are addressed, we can change the master build to
> > > using Maven 3.3.3
> > >
> > > I will log JIRA for modifying refguide on the upgrade.
> > >
> > > Cheers
> > >
> > > On Thu, Oct 13, 2016 at 6:42 AM, Sean Busbey <bu...@cloudera.com>
> > wrote:
> > >
> > > > Are we requiring maven 3.3.z now? historically our required maven
> > > > version has been 3.0.5. I don't have an objection to changing it, per
> > > > se, but I'd like to make sure our docs and CI builds get updated
> > > > properly.
> > > >
> > > > On Wed, Oct 12, 2016 at 4:03 PM, Stack <st...@duboce.net> wrote:
> > > > > On Wed, Oct 12, 2016 at 1:58 PM, Stack <st...@duboce.net> wrote:
> > > > >
> > > > >> Fails again when I do this:
> > > > >>
> > > > >> $ mvn clean install -DskipTests assembly:single
> > > > >> ....
> > > > >>
> > > > >>
> > > > > Ok. Works if I use mvn 3.3.x.
> > > > > St.Ack
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >>
> > > > >> [INFO] ------------------------------
> ------------------------------
> > > > >> ------------
> > > > >> [INFO] Building Apache HBase - Server 2.0.0-SNAPSHOT
> > > > >> [INFO] ------------------------------
> ------------------------------
> > > > >> ------------
> > > > >> Downloading: file:/home/stack/hbase.git/
> hbase-server/src/main/site/
> > > > >> resources/repo/org/apache/hadoop/hadoop-distcp/2.7.1/
> > > > >> hadoop-distcp-2.7.1.pom
> > > > >> Exception in thread "pool-1-thread-1"
> ------------------------------
> > > > >> ---------------------
> > > > >> constituent[0]: file:/usr/share/maven/lib/aether-spi.jar
> > > > >> constituent[1]: file:/usr/share/maven/lib/
> > > > maven-settings-builder-3.x.jar
> > > > >> constituent[2]: file:/usr/share/maven/lib/maven-artifact-3.x.jar
> > > > >> constituent[3]: file:/usr/share/maven/lib/commons-httpclient.jar
> > > > >> constituent[4]: file:/usr/share/maven/lib/wagon-provider-api.jar
> > > > >> constituent[5]: file:/usr/share/maven/lib/maven-core-3.x.jar
> > > > >> constituent[6]: file:/usr/share/maven/lib/aether-impl.jar
> > > > >> constituent[7]: file:/usr/share/maven/lib/sisu-inject-bean.jar
> > > > >> constituent[8]: file:/usr/share/maven/lib/sisu-inject-plexus.jar
> > > > >> constituent[9]: file:/usr/share/maven/lib/wagon-file.jar
> > > > >> constituent[10]: file:/usr/share/maven/lib/guava.jar
> > > > >> constituent[11]: file:/usr/share/maven/lib/maven-compat-3.x.jar
> > > > >> constituent[12]: file:/usr/share/maven/lib/maven-model-3.x.jar
> > > > >> constituent[13]: file:/usr/share/maven/lib/
> plexus-interpolation.jar
> > > > >> constituent[14]: file:/usr/share/maven/lib/commons-codec.jar
> > > > >> constituent[15]: file:/usr/share/maven/lib/aether-util.jar
> > > > >> constituent[16]: file:/usr/share/maven/lib/aether-api.jar
> > > > >> constituent[17]: file:/usr/share/maven/lib/
> > > maven-model-builder-3.x.jar
> > > > >> constituent[18]: file:/usr/share/maven/lib/
> > > > maven-repository-metadata-3.x.
> > > > >> jar
> > > > >> constituent[19]: file:/usr/share/maven/lib/maven-settings-3.x.jar
> > > > >> constituent[20]: file:/usr/share/maven/lib/
> > > > plexus-component-annotations.
> > > > >> jar
> > > > >> constituent[21]: file:/usr/share/maven/lib/commons-cli.jar
> > > > >> constituent[22]: file:/usr/share/maven/lib/maven-embedder-3.x.jar
> > > > >> constituent[23]: file:/usr/share/maven/lib/
> maven-plugin-api-3.x.jar
> > > > >> constituent[24]: file:/usr/share/maven/lib/wagon-http-shaded.jar
> > > > >> constituent[25]: file:/usr/share/maven/lib/commons-logging.jar
> > > > >> constituent[26]: file:/usr/share/maven/lib/
> > plexus-sec-dispatcher.jar
> > > > >> constituent[27]: file:/usr/share/maven/lib/
> > aether-connector-wagon.jar
> > > > >> constituent[28]: file:/usr/share/maven/lib/plexus-utils.jar
> > > > >> constituent[29]: file:/usr/share/maven/lib/sisu-guice.jar
> > > > >> constituent[30]: file:/usr/share/maven/lib/plexus-cipher.jar
> > > > >> constituent[31]: file:/usr/share/maven/lib/
> > > > maven-aether-provider-3.x.jar
> > > > >> ---------------------------------------------------
> > > > >> java.lang.NoClassDefFoundError: org/apache/commons/lang/
> StringUtils
> > > > >> at org.apache.maven.wagon.providers.file.FileWagon.
> > > > resolveDestinationPath(
> > > > >> FileWagon.java:206)
> > > > >> at org.apache.maven.wagon.providers.file.FileWagon.
> > > > >> resourceExists(FileWagon.java:265)
> > > > >> at org.sonatype.aether.connector.wagon.WagonRepositoryConnector$
> > > > >> GetTask.run(WagonRepositoryConnector.java:577)
> > > > >> at org.sonatype.aether.util.concurrency.
> > RunnableErrorForwarder$1.run(
> > > > >> RunnableErrorForwarder.java:60)
> > > > >> at java.util.concurrent.ThreadPoolExecutor.runWorker(
> > > > >> ThreadPoolExecutor.java:1142)
> > > > >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> > > > >> ThreadPoolExecutor.java:617)
> > > > >> at java.lang.Thread.run(Thread.java:745)
> > > > >> Caused by: java.lang.ClassNotFoundException:
> > org.apache.commons.lang.
> > > > >> StringUtils
> > > > >> at org.codehaus.plexus.classworlds.strategy.
> > > > SelfFirstStrategy.loadClass(
> > > > >> SelfFirstStrategy.java:50)
> > > > >> at org.codehaus.plexus.classworlds.realm.ClassRealm.
> > > > >> unsynchronizedLoadClass(ClassRealm.java:259)
> > > > >> at org.codehaus.plexus.classworlds.realm.ClassRealm.
> > > > >> loadClass(ClassRealm.java:235)
> > > > >> at org.codehaus.plexus.classworlds.realm.ClassRealm.
> > > > >> loadClass(ClassRealm.java:227)
> > > > >> ... 7 more
> > > > >> Exception in thread "main" java.lang.NoClassDefFoundError:
> > > > >> org/apache/commons/lang/StringUtils
> > > > >> at org.apache.maven.wagon.providers.file.FileWagon.
> > > > resolveDestinationPath(
> > > > >> FileWagon.java:206)
> > > > >> at org.apache.maven.wagon.providers.file.FileWagon.
> > > > >> resourceExists(FileWagon.java:265)
> > > > >> at org.sonatype.aether.connector.wagon.WagonRepositoryConnector$
> > > > >> GetTask.run(WagonRepositoryConnector.java:577)
> > > > >> at org.sonatype.aether.util.concurrency.
> > RunnableErrorForwarder$1.run(
> > > > >> RunnableErrorForwarder.java:60)
> > > > >> at java.util.concurrent.ThreadPoolExecutor.runWorker(
> > > > >> ThreadPoolExecutor.java:1142)
> > > > >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> > > > >> ThreadPoolExecutor.java:617)
> > > > >> at java.lang.Thread.run(Thread.java:745)
> > > > >> Caused by: java.lang.ClassNotFoundException:
> > org.apache.commons.lang.
> > > > >> StringUtils
> > > > >> at org.codehaus.plexus.classworlds.strategy.
> > > > SelfFirstStrategy.loadClass(
> > > > >> SelfFirstStrategy.java:50)
> > > > >> at org.codehaus.plexus.classworlds.realm.ClassRealm.
> > > > >> unsynchronizedLoadClass(ClassRealm.java:259)
> > > > >> at org.codehaus.plexus.classworlds.realm.ClassRealm.
> > > > >> loadClass(ClassRealm.java:235)
> > > > >> at org.codehaus.plexus.classworlds.realm.ClassRealm.
> > > > >> loadClass(ClassRealm.java:227)
> > > > >>
> > > > >>
> > > > >>
> > > > >> Does it work for you lot? This is branch checkout.
> > > > >>
> > > > >> I want assembly to work so I can distribute the build around a
> > > cluster.
> > > > >>
> > > > >> Thanks,
> > > > >> St.Ack.
> > > > >>
> > > > >>
> > > > >>
> > > > >> On Wed, Oct 12, 2016 at 1:52 PM, Stack <st...@duboce.net> wrote:
> > > > >>
> > > > >>> I did a super-clean and recheck out. Now it works. Sorry for the
> > > noise.
> > > > >>> St.Ack
> > > > >>>
> > > > >>> On Wed, Oct 12, 2016 at 1:10 PM, Vladimir Rodionov <
> > > > >>> vladrodionov@gmail.com> wrote:
> > > > >>>
> > > > >>>> checked out HBASE-7912
> > > > >>>>
> > > > >>>> ran:
> > > > >>>>
> > > > >>>> mvn clean install -DskipTests
> > > > >>>>
> > > > >>>> successfully.
> > > > >>>>
> > > > >>>> -Vlad
> > > > >>>>
> > > > >>>> On Wed, Oct 12, 2016 at 1:01 PM, Vladimir Rodionov <
> > > > >>>> vladrodionov@gmail.com>
> > > > >>>> wrote:
> > > > >>>>
> > > > >>>> > I usually use:
> > > > >>>> >
> > > > >>>> > mvn clean install -DskipTests
> > > > >>>> >
> > > > >>>> > and it works.
> > > > >>>> >
> > > > >>>> > On Wed, Oct 12, 2016 at 1:01 PM, Vladimir Rodionov <
> > > > >>>> vladrodionov@gmail.com
> > > > >>>> > > wrote:
> > > > >>>> >
> > > > >>>> >> Michael,
> > > > >>>> >>
> > > > >>>> >> you can try master + latest patch on HBASE-14123 (v29). No
> need
> > > to
> > > > use
> > > > >>>> >> HBASE-7912 branch. I will double check HBASE-7912.
> > > > >>>> >>
> > > > >>>> >> -Vlad
> > > > >>>> >>
> > > > >>>> >> On Wed, Oct 12, 2016 at 12:31 PM, Stack <st...@duboce.net>
> > > wrote:
> > > > >>>> >>
> > > > >>>> >>> More info:
> > > > >>>> >>>
> > > > >>>> >>> stack@ve0524:~/hbase.git$ git checkout origin/HBASE-7912 -b
> > > > 7912v2
> > > > >>>> >>> Branch 7912v2 set up to track remote branch HBASE-7912 from
> > > > origin.
> > > > >>>> >>> Switched to a new branch '7912v2'
> > > > >>>> >>> stack@ve0524:~/hbase.git$ java -version
> > > > >>>> >>> java version "1.8.0_101"
> > > > >>>> >>> Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
> > > > >>>> >>> Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed
> > mode)
> > > > >>>> >>> stack@ve0524:~/hbase.git$ mvn clean install -DskipTests &>
> > > > >>>> /tmp/out.txt
> > > > >>>> >>>
> > > > >>>> >>> ...
> > > > >>>> >>>
> > > > >>>> >>> St.Ack
> > > > >>>> >>>
> > > > >>>> >>>
> > > > >>>> >>> On Wed, Oct 12, 2016 at 12:29 PM, Stack <st...@duboce.net>
> > > wrote:
> > > > >>>> >>>
> > > > >>>> >>> > Interesting. When I try it fails w/ below:
> > > > >>>> >>> >
> > > > >>>> >>> > [INFO] 26 warnings
> > > > >>>> >>> > 322 [INFO] ------------------------------
> > > > >>>> >>> -------------------------------
> > > > >>>> >>> > 323 [INFO] ------------------------------
> > > > >>>> >>> -------------------------------
> > > > >>>> >>> > 324 [ERROR] COMPILATION ERROR :
> > > > >>>> >>> > 325 [INFO] ------------------------------
> > > > >>>> >>> -------------------------------
> > > > >>>> >>> > 326 [ERROR] /home/stack/hbase.git/hbase-
> > > > common/src/main/java/org/
> > > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/
> > RowIndexCodecV2.j
> > > > >>>> ava:[48,8]
> > > > >>>> >>> > org.apache.hadoop.hbase.io.encoding.rowindexV2.
> > > RowIndexCodecV2
> > > > is
> > > > >>>> not
> > > > >>>> >>> > abstract and does not override abstract method
> > > > createSeeker(org.ap
> > > > >>>> >>> >  ache.hadoop.hbase.CellComparator,org.apache.
> > hadoop.hbase.io.
> > > > >>>> >>> encoding.HFileBlockDecodingContext)
> > > > >>>> >>> > in org.apache.hadoop.hbase.io.encoding.DataBlockEncoder
> > > > >>>> >>> > 327 [ERROR] /home/stack/hbase.git/hbase-
> > > > common/src/main/java/org/
> > > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/
> > RowIndexCodecV2.j
> > > > >>>> >>> ava:[143,3]
> > > > >>>> >>> > method does not override or implement a method from a
> > > supertype
> > > > >>>> >>> > 328 [ERROR] /home/stack/hbase.git/hbase-
> > > > common/src/main/java/org/
> > > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/
> > RowIndexCodecV2.j
> > > > >>>> >>> ava:[147,29]
> > > > >>>> >>> > incompatible types: java.nio.ByteBuffer cannot be
> converted
> > to
> > > > >>>> >>> > org.apache.hadoop.hbase.nio.ByteBuff
> > > > >>>> >>> > 329 [ERROR] /home/stack/hbase.git/hbase-
> > > > common/src/main/java/org/
> > > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/
> > RowIndexCodecV2.j
> > > > >>>> >>> ava:[148,33]
> > > > >>>> >>> > cannot find symbol
> > > > >>>> >>> > 330   symbol:   method getKeyDeepCopy()
> > > > >>>> >>> > 331   location: variable seeker of type
> > > > org.apache.hadoop.hbase.io
> > > > >>>> .
> > > > >>>> >>> > encoding.DataBlockEncoder.EncodedSeeker
> > > > >>>> >>> > 332 [ERROR] /home/stack/hbase.git/hbase-
> > > > common/src/main/java/org/
> > > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/
> > RowIndexCodecV2.j
> > > > >>>> >>> ava:[153,3]
> > > > >>>> >>> > method does not override or implement a method from a
> > > supertype
> > > > >>>> >>> > 333 [ERROR] /home/stack/hbase.git/hbase-
> > > > common/src/main/java/org/
> > > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV1/
> > RowIndexCodecV1.j
> > > > >>>> ava:[45,8]
> > > > >>>> >>> > org.apache.hadoop.hbase.io.encoding.rowindexV1.
> > > RowIndexCodecV1
> > > > is
> > > > >>>> not
> > > > >>>> >>> > abstract and does not override abstract method
> > > > createSeeker(org.ap
> > > > >>>> >>> >  ache.hadoop.hbase.CellComparator,org.apache.
> > hadoop.hbase.io.
> > > > >>>> >>> encoding.HFileBlockDecodingContext)
> > > > >>>> >>> > in org.apache.hadoop.hbase.io.encoding.DataBlockEncoder
> > > > >>>> >>> > 334 [ERROR] /home/stack/hbase.git/hbase-
> > > > common/src/main/java/org/
> > > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV1/
> > RowIndexCodecV1.j
> > > > >>>> >>> ava:[145,3]
> > > > >>>> >>> > method does not override or implement a method from a
> > > supertype
> > > > >>>> >>> > 335 [ERROR] /home/stack/hbase.git/hbase-
> > > > common/src/main/java/org/
> > > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV1/
> > RowIndexCodecV1.j
> > > > >>>> >>> ava:[158,3]
> > > > >>>> >>> > method does not override or implement a method from a
> > > supertype
> > > > >>>> >>> > 336 [ERROR] /home/stack/hbase.git/hbase-
> > > > common/src/main/java/org/
> > > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/
> > RowIndexSeekerV2.
> > > > >>>> >>> java:[46,8]
> > > > >>>> >>> > org.apache.hadoop.hbase.io.encoding.rowindexV2.
> > > RowIndexSeekerV2
> > > > >>>> is not
> > > > >>>> >>> > abstract and does not override abstract method
> > > compareKey(org.ap
> > > > >>>> >>> >  ache.hadoop.hbase.CellComparator,org.apache.
> > > hadoop.hbase.Cell)
> > > > in
> > > > >>>> >>> > org.apache.hadoop.hbase.io.encoding.DataBlockEncoder.
> > > > EncodedSeeker
> > > > >>>> >>> > 337 [ERROR] /home/stack/hbase.git/hbase-
> > > > common/src/main/java/org/
> > > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/
> > RowIndexSeekerV2.
> > > > >>>> >>> java:[79,3]
> > > > >>>> >>> > method does not override or implement a method from a
> > > supertype
> > > > >>>> >>> > 338 [ERROR] /home/stack/hbase.git/hbase-
> > > > common/src/main/java/org/
> > > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/
> > RowIndexSeekerV2.
> > > > >>>> >>> java:[117,3]
> > > > >>>> >>> > method does not override or implement a method from a
> > > supertype
> > > > >>>> >>> > 339 [ERROR] /home/stack/hbase.git/hbase-
> > > > common/src/main/java/org/
> > > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/
> > RowIndexSeekerV2.
> > > > >>>> >>> java:[190,3]
> > > > >>>> >>> > method does not override or implement a method from a
> > > supertype
> > > > >>>> >>> > 340 [ERROR] /home/stack/hbase.git/hbase-
> > > > common/src/main/java/org/
> > > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/
> > RowIndexSeekerV2.
> > > > >>>> >>> java:[214,3]
> > > > >>>> >>> > method does not override or implement a method from a
> > > supertype
> > > > >>>> >>> > 341 [ERROR] /home/stack/hbase.git/hbase-
> > > > common/src/main/java/org/
> > > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/
> > RowIndexSeekerV2.
> > > > >>>> >>> java:[349,3]
> > > > >>>> >>> > method does not override or implement a method from a
> > > supertype
> > > > >>>> >>> > 342 [ERROR] /home/stack/hbase.git/hbase-
> > > > common/src/main/java/org/
> > > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/
> > RowIndexSeekerV2.
> > > > >>>> >>> java:[355,3]
> > > > >>>> >>> > method does not override or implement a method from a
> > > supertype
> > > > >>>> >>> > 343 [ERROR] /home/stack/hbase.git/hbase-
> > > > common/src/main/java/org/
> > > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/
> > RowIndexSeekerV2.
> > > > >>>> >>> java:[421,36]
> > > > >>>> >>> > no suitable method found for uncompressTags(java.nio.
> > > > >>>> >>> > ByteBuffer,byte[],int,int)
> > > > >>>> >>> > 344     method org.apache.hadoop.hbase.io.
> > > > TagCompressionContext.
> > > > >>>> >>> > uncompressTags(java.io.InputStream,byte[],int,int) is not
> > > > >>>> applicable
> > > > >>>> >>> > 345       (argument mismatch; java.nio.ByteBuffer cannot
> be
> > > > >>>> converted
> > > > >>>> >>> to
> > > > >>>> >>> > java.io.InputStream)
> > > > >>>> >>> > 346     method org.apache.hadoop.hbase.io.
> > > > TagCompressionContext.
> > > > >>>> >>> > uncompressTags(org.apache.hadoop.hbase.nio.ByteBuff,
> > > > byte[],int,int)
> > > > >>>> is
> > > > >>>> >>> > not applicable
> > > > >>>> >>> > 347       (argument mismatch; java.nio.ByteBuffer cannot
> be
> > > > >>>> converted
> > > > >>>> >>> to
> > > > >>>> >>> > org.apache.hadoop.hbase.nio.ByteBuff)
> > > > >>>> >>> >
> > > > >>>> >>> > ....
> > > > >>>> >>> >
> > > > >>>> >>> > St.Ack
> > > > >>>> >>> >
> > > > >>>> >>> > On Tue, Oct 11, 2016 at 5:12 PM, Apekshit Sharma <
> > > > >>>> appy@cloudera.com>
> > > > >>>> >>> > wrote:
> > > > >>>> >>> >
> > > > >>>> >>> >> @stack, it compiled for me.
> > > > >>>> >>> >>
> > > > >>>> >>> >> Also tried few commands, and have to say, it's well
> > designed
> > > > from
> > > > >>>> user
> > > > >>>> >>> >> commands perspective.
> > > > >>>> >>> >>
> > > > >>>> >>> >> On Tue, Oct 11, 2016 at 5:08 PM, Stack <stack@duboce.net
> >
> > > > wrote:
> > > > >>>> >>> >>
> > > > >>>> >>> >> > On Tue, Oct 11, 2016 at 1:50 PM, Vladimir Rodionov <
> > > > >>>> >>> >> vladrodionov@gmail.com
> > > > >>>> >>> >> > >
> > > > >>>> >>> >> > wrote:
> > > > >>>> >>> >> >
> > > > >>>> >>> >> > > Michael,
> > > > >>>> >>> >> > >
> > > > >>>> >>> >> > > Its in HBASE-7912
> > > > >>>> >>> >> > >
> > > > >>>> >>> >> > > This is tip of git log:
> > > > >>>> >>> >> > >
> > > > >>>> >>> >> > > commit a072f6f49a26a7259ff2aaef6cb56d85eb592482
> > > > >>>> >>> >> > > Author: Frank Welsch <fw...@jps.net>
> > > > >>>> >>> >> > > Date:   Fri Sep 23 18:00:42 2016 -0400
> > > > >>>> >>> >> > >
> > > > >>>> >>> >> > >     HBASE-16574 Book updates for backup and restore
> > > > >>>> >>> >> > >
> > > > >>>> >>> >> > > commit b14e2ab1c24e65ff88dd4c579acf83cb4ed0605e
> > > > >>>> >>> >> > > Author: tedyu <yu...@gmail.com>
> > > > >>>> >>> >> > > Date:   Wed Oct 5 16:29:40 2016 -0700
> > > > >>>> >>> >> > >
> > > > >>>> >>> >> > >     HBASE-16727 Backup refactoring: remove MR
> > > dependencies
> > > > >>>> from
> > > > >>>> >>> >> HMaster
> > > > >>>> >>> >> > > (Vladimir Rodionov)
> > > > >>>> >>> >> > >
> > > > >>>> >>> >> > >
> > > > >>>> >>> >> > Thanks. I have that. I tried it and it doesn't compile
> > for
> > > > me.
> > > > >>>> Does
> > > > >>>> >>> it
> > > > >>>> >>> >> > compile for you?
> > > > >>>> >>> >> > Thanks,
> > > > >>>> >>> >> > M
> > > > >>>> >>> >> >
> > > > >>>> >>> >> >
> > > > >>>> >>> >> >
> > > > >>>> >>> >> > > -Vlad
> > > > >>>> >>> >> > >
> > > > >>>> >>> >> > > On Tue, Oct 11, 2016 at 1:44 PM, Stack <
> > stack@duboce.net
> > > >
> > > > >>>> wrote:
> > > > >>>> >>> >> > >
> > > > >>>> >>> >> > > > Which branch do I check out to try it? HBASE-7912
> is
> > > not
> > > > >>>> it. I
> > > > >>>> >>> don't
> > > > >>>> >>> >> > see
> > > > >>>> >>> >> > > an
> > > > >>>> >>> >> > > > HBASE-16727...
> > > > >>>> >>> >> > > > Thanks,
> > > > >>>> >>> >> > > > M
> > > > >>>> >>> >> > > >
> > > > >>>> >>> >> > > >
> > > > >>>> >>> >> > > > On Tue, Oct 11, 2016 at 11:41 AM, Vladimir
> Rodionov <
> > > > >>>> >>> >> > > > vladrodionov@gmail.com>
> > > > >>>> >>> >> > > > wrote:
> > > > >>>> >>> >> > > >
> > > > >>>> >>> >> > > > > The last patch is on review board:
> > > > >>>> >>> >> > > > > https://reviews.apache.org/r/52748
> > > > >>>> >>> >> > > > >
> > > > >>>> >>> >> > > > > On Mon, Oct 10, 2016 at 3:52 PM, Vladimir
> Rodionov
> > <
> > > > >>>> >>> >> > > > vladrodionov@gmail.com
> > > > >>>> >>> >> > > > > >
> > > > >>>> >>> >> > > > > wrote:
> > > > >>>> >>> >> > > > >
> > > > >>>> >>> >> > > > > > >> How hard to put in an hbase-backup module?
> > > > >>>> hbase-server
> > > > >>>> >>> is
> > > > >>>> >>> >> fat
> > > > >>>> >>> >> > > > enough
> > > > >>>> >>> >> > > > > > >> already. Could be done as a follow-up.
> > > > >>>> >>> >> > > > > >
> > > > >>>> >>> >> > > > > > https://issues.apache.org/
> > jira/browse/HBASE-16727?
> > > > >>>> >>> >> > > > > > focusedCommentId=15531237&
> > page=com.atlassian.jira.
> > > > >>>> >>> >> > > > > > plugin.system.issuetabpanels:c
> > > > >>>> >>> omment-tabpanel#comment-15531237
> > > > >>>> >>> >> > > > > >
> > > > >>>> >>> >> > > > > > Can we do merge first? Then we can discuss
> > separate
> > > > >>>> module.
> > > > >>>> >>> >> > > > > >
> > > > >>>> >>> >> > > > > > -Vlad
> > > > >>>> >>> >> > > > > >
> > > > >>>> >>> >> > > > > > On Mon, Oct 10, 2016 at 3:44 PM, Ted Yu <
> > > > >>>> >>> yuzhihong@gmail.com>
> > > > >>>> >>> >> > wrote:
> > > > >>>> >>> >> > > > > >
> > > > >>>> >>> >> > > > > >> Looks like the first quote was cut off.
> > > > >>>> >>> >> > > > > >> The original sentence was:
> > > > >>>> >>> >> > > > > >>
> > > > >>>> >>> >> > > > > >> bq. no mapreduce job launched from master or
> > > region
> > > > >>>> server.
> > > > >>>> >>> >> > > > > >>
> > > > >>>> >>> >> > > > > >> mapreduce job is launched from the node where
> > > > command
> > > > >>>> line
> > > > >>>> >>> >> tool is
> > > > >>>> >>> >> > > > run.
> > > > >>>> >>> >> > > > > >>
> > > > >>>> >>> >> > > > > >> On Mon, Oct 10, 2016 at 3:38 PM, Stack <
> > > > >>>> stack@duboce.net>
> > > > >>>> >>> >> wrote:
> > > > >>>> >>> >> > > > > >>
> > > > >>>> >>> >> > > > > >> > bq. launched from master or region server.
> > > > >>>> >>> >> > > > > >> >
> > > > >>>> >>> >> > > > > >> > What does this mean please? Has to be run
> from
> > > > >>>> Master or
> > > > >>>> >>> >> > > > RegionServer?
> > > > >>>> >>> >> > > > > >> Can
> > > > >>>> >>> >> > > > > >> > it be run from another node altogether?
> > > > >>>> >>> >> > > > > >> >
> > > > >>>> >>> >> > > > > >> > On Mon, Oct 10, 2016 at 1:44 PM, Vladimir
> > > > Rodionov <
> > > > >>>> >>> >> > > > > >> vladrodionov@gmail.com
> > > > >>>> >>> >> > > > > >> > >
> > > > >>>> >>> >> > > > > >> > wrote:
> > > > >>>> >>> >> > > > > >> >
> > > > >>>> >>> >> > > > > >> > > >> mapreduce dependency has been moved to
> > > client
> > > > >>>> side
> > > > >>>> >>> - no
> > > > >>>> >>> >> > > > mapreduce
> > > > >>>> >>> >> > > > > >> job
> > > > >>>> >>> >> > > > > >> > >
> > > > >>>> >>> >> > > > > >> > > 1. We have no code in the client module
> > > anymore,
> > > > >>>> due to
> > > > >>>> >>> >> > > dependency
> > > > >>>> >>> >> > > > > on
> > > > >>>> >>> >> > > > > >> > > internal server API (HFile and WAL
> access).
> > > > >>>> >>> >> > > > > >> > > 2. Backup/ restore are client - driven
> > > > operations,
> > > > >>>> but
> > > > >>>> >>> all
> > > > >>>> >>> >> the
> > > > >>>> >>> >> > > > code
> > > > >>>> >>> >> > > > > >> > resides
> > > > >>>> >>> >> > > > > >> > > in the server module
> > > > >>>> >>> >> > > > > >> > >
> > > > >>>> >>> >> > > > > >> >
> > > > >>>> >>> >> > > > > >> > How hard to put in an hbase-backup module?
> > > > >>>> hbase-server
> > > > >>>> >>> is
> > > > >>>> >>> >> fat
> > > > >>>> >>> >> > > > enough
> > > > >>>> >>> >> > > > > >> > already. Could be done as a follow-up.
> > > > >>>> >>> >> > > > > >> >
> > > > >>>> >>> >> > > > > >> > Thanks,
> > > > >>>> >>> >> > > > > >> > St.Ack
> > > > >>>> >>> >> > > > > >> >
> > > > >>>> >>> >> > > > > >> >
> > > > >>>> >>> >> > > > > >> >
> > > > >>>> >>> >> > > > > >> > > 3. No MR in Master, no procedure - driven
> > > > >>>> execution.
> > > > >>>> >>> >> > > > > >> > > 4. Old good MR from command-line.
> > > > >>>> >>> >> > > > > >> > > 5. Security was simplified and now only
> > > > super-user
> > > > >>>> is
> > > > >>>> >>> >> allowed
> > > > >>>> >>> >> > to
> > > > >>>> >>> >> > > > run
> > > > >>>> >>> >> > > > > >> > > backup/restores.
> > > > >>>> >>> >> > > > > >> > > 6. HBase Backup API was gone due to 1. Now
> > > only
> > > > >>>> >>> >> command-line
> > > > >>>> >>> >> > > > access
> > > > >>>> >>> >> > > > > to
> > > > >>>> >>> >> > > > > >> > > backup tools.
> > > > >>>> >>> >> > > > > >> > >
> > > > >>>> >>> >> > > > > >> > > These consequences of refactoring has been
> > > > >>>> discussed in
> > > > >>>> >>> >> > > > HBASE-16727.
> > > > >>>> >>> >> > > > > >> > >
> > > > >>>> >>> >> > > > > >> > > -Vlad
> > > > >>>> >>> >> > > > > >> > >
> > > > >>>> >>> >> > > > > >> > >
> > > > >>>> >>> >> > > > > >> > >
> > > > >>>> >>> >> > > > > >> > >
> > > > >>>> >>> >> > > > > >> > > On Mon, Oct 10, 2016 at 1:31 PM, Ted Yu <
> > > > >>>> >>> >> yuzhihong@gmail.com>
> > > > >>>> >>> >> > > > > wrote:
> > > > >>>> >>> >> > > > > >> > >
> > > > >>>> >>> >> > > > > >> > > > Reviving this thread.
> > > > >>>> >>> >> > > > > >> > > >
> > > > >>>> >>> >> > > > > >> > > > The following has taken place:
> > > > >>>> >>> >> > > > > >> > > >
> > > > >>>> >>> >> > > > > >> > > > mapreduce dependency has been moved to
> > > client
> > > > >>>> side -
> > > > >>>> >>> no
> > > > >>>> >>> >> > > > mapreduce
> > > > >>>> >>> >> > > > > >> job
> > > > >>>> >>> >> > > > > >> > > > launched from master or region server.
> > > > >>>> >>> >> > > > > >> > > > document patch (HBASE-16574) has been
> > > > integrated.
> > > > >>>> >>> >> > > > > >> > > > Updated mega patch has been attached to
> > > > >>>> HBASE-14123:
> > > > >>>> >>> this
> > > > >>>> >>> >> > > covers
> > > > >>>> >>> >> > > > > the
> > > > >>>> >>> >> > > > > >> > > > refactor in #1 above and the protobuf 3
> > > merge.
> > > > >>>> >>> >> > > > > >> > > >
> > > > >>>> >>> >> > > > > >> > > > If community has more feedback on the
> > merge
> > > > >>>> >>> proposal, I
> > > > >>>> >>> >> > would
> > > > >>>> >>> >> > > > love
> > > > >>>> >>> >> > > > > >> to
> > > > >>>> >>> >> > > > > >> > > hear
> > > > >>>> >>> >> > > > > >> > > > it.
> > > > >>>> >>> >> > > > > >> > > >
> > > > >>>> >>> >> > > > > >> > > > Thanks
> > > > >>>> >>> >> > > > > >> > > >
> > > > >>>> >>> >> > > > > >> > > > On Thu, Sep 22, 2016 at 10:31 AM, Sean
> > > Busbey
> > > > <
> > > > >>>> >>> >> > > > > busbey@cloudera.com>
> > > > >>>> >>> >> > > > > >> > > wrote:
> > > > >>>> >>> >> > > > > >> > > >
> > > > >>>> >>> >> > > > > >> > > > > I'd like to see the docs proposed on
> > > > >>>> HBASE-16574
> > > > >>>> >>> >> > integrated
> > > > >>>> >>> >> > > > into
> > > > >>>> >>> >> > > > > >> our
> > > > >>>> >>> >> > > > > >> > > > > project's documentation prior to
> merge.
> > > > >>>> >>> >> > > > > >> > > > >
> > > > >>>> >>> >> > > > > >> > > > > On Thu, Sep 22, 2016 at 9:02 AM, Ted
> Yu
> > <
> > > > >>>> >>> >> > > yuzhihong@gmail.com>
> > > > >>>> >>> >> > > > > >> wrote:
> > > > >>>> >>> >> > > > > >> > > > > > This feature can be marked
> > experimental
> > > > due
> > > > >>>> to
> > > > >>>> >>> some
> > > > >>>> >>> >> > > > > limitations
> > > > >>>> >>> >> > > > > >> > such
> > > > >>>> >>> >> > > > > >> > > as
> > > > >>>> >>> >> > > > > >> > > > > > security.
> > > > >>>> >>> >> > > > > >> > > > > >
> > > > >>>> >>> >> > > > > >> > > > > > Your previous round of comments have
> > > been
> > > > >>>> >>> addressed.
> > > > >>>> >>> >> > > > > >> > > > > > Command line tool has gone through:
> > > > >>>> >>> >> > > > > >> > > > > >
> > > > >>>> >>> >> > > > > >> > > > > > HBASE-16620 Fix backup command-line
> > tool
> > > > >>>> >>> usability
> > > > >>>> >>> >> > issues
> > > > >>>> >>> >> > > > > >> > > > > > HBASE-16655 hbase backup describe
> with
> > > > >>>> incorrect
> > > > >>>> >>> >> backup
> > > > >>>> >>> >> > id
> > > > >>>> >>> >> > > > > >> results
> > > > >>>> >>> >> > > > > >> > in
> > > > >>>> >>> >> > > > > >> > > > NPE
> > > > >>>> >>> >> > > > > >> > > > > >
> > > > >>>> >>> >> > > > > >> > > > > > The updated doc has been attached to
> > > > >>>> HBASE-16574.
> > > > >>>> >>> >> > > > > >> > > > > >
> > > > >>>> >>> >> > > > > >> > > > > > Cheers
> > > > >>>> >>> >> > > > > >> > > > > >
> > > > >>>> >>> >> > > > > >> > > > > > On Thu, Sep 22, 2016 at 8:53 AM,
> > Stack <
> > > > >>>> >>> >> > stack@duboce.net>
> > > > >>>> >>> >> > > > > >> wrote:
> > > > >>>> >>> >> > > > > >> > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> On Wed, Sep 21, 2016 at 7:43 AM,
> Ted
> > > Yu <
> > > > >>>> >>> >> > > > yuzhihong@gmail.com
> > > > >>>> >>> >> > > > > >
> > > > >>>> >>> >> > > > > >> > > wrote:
> > > > >>>> >>> >> > > > > >> > > > > >>
> > > > >>>> >>> >> > > > > >> > > > > >> > Are there more (review) comments
> ?
> > > > >>>> >>> >> > > > > >> > > > > >> >
> > > > >>>> >>> >> > > > > >> > > > > >> >
> > > > >>>> >>> >> > > > > >> > > > > >> Are outstanding comments addressed?
> > > > >>>> >>> >> > > > > >> > > > > >>
> > > > >>>> >>> >> > > > > >> > > > > >> I don't see answer to my 'is this
> > > > >>>> >>> experimental/will
> > > > >>>> >>> >> it
> > > > >>>> >>> >> > be
> > > > >>>> >>> >> > > > > >> marked
> > > > >>>> >>> >> > > > > >> > > > > >> experimental' question.
> > > > >>>> >>> >> > > > > >> > > > > >>
> > > > >>>> >>> >> > > > > >> > > > > >> I ran into some issues trying to
> use
> > > the
> > > > >>>> >>> feature and
> > > > >>>> >>> >> > > > > suggested
> > > > >>>> >>> >> > > > > >> > that
> > > > >>>> >>> >> > > > > >> > > a
> > > > >>>> >>> >> > > > > >> > > > > >> feature likes this needs polish
> else
> > > > it'll
> > > > >>>> just
> > > > >>>> >>> rot,
> > > > >>>> >>> >> > > > unused.
> > > > >>>> >>> >> > > > > >> Has
> > > > >>>> >>> >> > > > > >> > > > polish
> > > > >>>> >>> >> > > > > >> > > > > >> been applied? All ready for another
> > > > 'user'
> > > > >>>> test?
> > > > >>>> >>> >> > Suggest
> > > > >>>> >>> >> > > > that
> > > > >>>> >>> >> > > > > >> you
> > > > >>>> >>> >> > > > > >> > > > update
> > > > >>>> >>> >> > > > > >> > > > > >> here going forward for the benefit
> of
> > > > those
> > > > >>>> >>> trying
> > > > >>>> >>> >> to
> > > > >>>> >>> >> > > > follow
> > > > >>>> >>> >> > > > > >> along
> > > > >>>> >>> >> > > > > >> > > and
> > > > >>>> >>> >> > > > > >> > > > > who
> > > > >>>> >>> >> > > > > >> > > > > >> are not watching JIRA change
> fly-by.
> > > > >>>> >>> >> > > > > >> > > > > >>
> > > > >>>> >>> >> > > > > >> > > > > >> It looks like doc got a revision
> -- I
> > > > have
> > > > >>>> to
> > > > >>>> >>> check
> > > > >>>> >>> >> --
> > > > >>>> >>> >> > to
> > > > >>>> >>> >> > > > > take
> > > > >>>> >>> >> > > > > >> on
> > > > >>>> >>> >> > > > > >> > > > > >> suggestion made above but again,
> > > suggest,
> > > > >>>> that
> > > > >>>> >>> this
> > > > >>>> >>> >> > > thread
> > > > >>>> >>> >> > > > > gets
> > > > >>>> >>> >> > > > > >> > > > updated.
> > > > >>>> >>> >> > > > > >> > > > > >>
> > > > >>>> >>> >> > > > > >> > > > > >> Thanks,
> > > > >>>> >>> >> > > > > >> > > > > >> St.Ack
> > > > >>>> >>> >> > > > > >> > > > > >>
> > > > >>>> >>> >> > > > > >> > > > > >>
> > > > >>>> >>> >> > > > > >> > > > > >>
> > > > >>>> >>> >> > > > > >> > > > > >> > Thanks
> > > > >>>> >>> >> > > > > >> > > > > >> >
> > > > >>>> >>> >> > > > > >> > > > > >> > On Tue, Sep 20, 2016 at 10:02 AM,
> > > > Devaraj
> > > > >>>> Das
> > > > >>>> >>> <
> > > > >>>> >>> >> > > > > >> > > ddas@hortonworks.com
> > > > >>>> >>> >> > > > > >> > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > wrote:
> > > > >>>> >>> >> > > > > >> > > > > >> >
> > > > >>>> >>> >> > > > > >> > > > > >> > > Just reviving this thread.
> Thanks
> > > > Sean,
> > > > >>>> >>> Stack,
> > > > >>>> >>> >> > Dima,
> > > > >>>> >>> >> > > > and
> > > > >>>> >>> >> > > > > >> > others
> > > > >>>> >>> >> > > > > >> > > > for
> > > > >>>> >>> >> > > > > >> > > > > the
> > > > >>>> >>> >> > > > > >> > > > > >> > > thorough reviews and testing.
> > > Thanks
> > > > >>>> Ted and
> > > > >>>> >>> >> Vlad
> > > > >>>> >>> >> > for
> > > > >>>> >>> >> > > > > >> taking
> > > > >>>> >>> >> > > > > >> > > care
> > > > >>>> >>> >> > > > > >> > > > of
> > > > >>>> >>> >> > > > > >> > > > > >> the
> > > > >>>> >>> >> > > > > >> > > > > >> > > feedback. Are we all good to do
> > the
> > > > >>>> merge
> > > > >>>> >>> now?
> > > > >>>> >>> >> > Rather
> > > > >>>> >>> >> > > > do
> > > > >>>> >>> >> > > > > >> > sooner
> > > > >>>> >>> >> > > > > >> > > > than
> > > > >>>> >>> >> > > > > >> > > > > >> > later.
> > > > >>>> >>> >> > > > > >> > > > > >> > > ______________________________
> > > > >>>> __________
> > > > >>>> >>> >> > > > > >> > > > > >> > > From: saint.ack@gmail.com <
> > > > >>>> >>> saint.ack@gmail.com>
> > > > >>>> >>> >> on
> > > > >>>> >>> >> > > > > behalf
> > > > >>>> >>> >> > > > > >> of
> > > > >>>> >>> >> > > > > >> > > > Stack
> > > > >>>> >>> >> > > > > >> > > > > <
> > > > >>>> >>> >> > > > > >> > > > > >> > > stack@duboce.net>
> > > > >>>> >>> >> > > > > >> > > > > >> > > Sent: Monday, September 12,
> 2016
> > > > 1:18 PM
> > > > >>>> >>> >> > > > > >> > > > > >> > > To: HBase Dev List
> > > > >>>> >>> >> > > > > >> > > > > >> > > Subject: Re: [DISCUSSION] Merge
> > > > Backup /
> > > > >>>> >>> >> Restore -
> > > > >>>> >>> >> > > > Branch
> > > > >>>> >>> >> > > > > >> > > > HBASE-7912
> > > > >>>> >>> >> > > > > >> > > > > >> > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > On Mon, Sep 12, 2016 at 12:19
> PM,
> > > Ted
> > > > >>>> Yu <
> > > > >>>> >>> >> > > > > >> yuzhihong@gmail.com
> > > > >>>> >>> >> > > > > >> > >
> > > > >>>> >>> >> > > > > >> > > > > wrote:
> > > > >>>> >>> >> > > > > >> > > > > >> > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > Mega patch (rev 18) is on
> > > > HBASE-14123.
> > > > >>>> >>> >> > > > > >> > > > > >> > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > Please comment on HBASE-14123
> > on
> > > > how
> > > > >>>> you
> > > > >>>> >>> want
> > > > >>>> >>> >> to
> > > > >>>> >>> >> > > > > review.
> > > > >>>> >>> >> > > > > >> > > > > >> > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > >
> > > > >>>> >>> >> > > > > >> > > > > >> > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > Yeah. That was my lost tab.
> Last
> > rb
> > > > was
> > > > >>>> 6
> > > > >>>> >>> months
> > > > >>>> >>> >> > ago.
> > > > >>>> >>> >> > > > > >> Suggest
> > > > >>>> >>> >> > > > > >> > > > > updating
> > > > >>>> >>> >> > > > > >> > > > > >> > it.
> > > > >>>> >>> >> > > > > >> > > > > >> > > RB is pretty good for review.
> > Patch
> > > > is
> > > > >>>> only
> > > > >>>> >>> >> 1.5M so
> > > > >>>> >>> >> > > > > should
> > > > >>>> >>> >> > > > > >> be
> > > > >>>> >>> >> > > > > >> > > > fine.
> > > > >>>> >>> >> > > > > >> > > > > >> > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > St.Ack
> > > > >>>> >>> >> > > > > >> > > > > >> > >
> > > > >>>> >>> >> > > > > >> > > > > >> > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > Thanks
> > > > >>>> >>> >> > > > > >> > > > > >> > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > On Mon, Sep 12, 2016 at 12:15
> > PM,
> > > > >>>> Stack <
> > > > >>>> >>> >> > > > > >> stack@duboce.net>
> > > > >>>> >>> >> > > > > >> > > > wrote:
> > > > >>>> >>> >> > > > > >> > > > > >> > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > On review of the 'patch',
> do
> > I
> > > > just
> > > > >>>> >>> compare
> > > > >>>> >>> >> the
> > > > >>>> >>> >> > > > > branch
> > > > >>>> >>> >> > > > > >> to
> > > > >>>> >>> >> > > > > >> > > > > master or
> > > > >>>> >>> >> > > > > >> > > > > >> > is
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > there a megapatch posted
> > > > somewhere
> > > > >>>> (I
> > > > >>>> >>> think
> > > > >>>> >>> >> I
> > > > >>>> >>> >> > saw
> > > > >>>> >>> >> > > > one
> > > > >>>> >>> >> > > > > >> but
> > > > >>>> >>> >> > > > > >> > it
> > > > >>>> >>> >> > > > > >> > > > > seemed
> > > > >>>> >>> >> > > > > >> > > > > >> > > stale
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > and then I 'lost' the tab).
> > > Sorry
> > > > >>>> for
> > > > >>>> >>> dumb
> > > > >>>> >>> >> > > > question.
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > St.Ack
> > > > >>>> >>> >> > > > > >> > > > > >> > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > On Mon, Sep 12, 2016 at
> 12:01
> > > PM,
> > > > >>>> Stack
> > > > >>>> >>> <
> > > > >>>> >>> >> > > > > >> stack@duboce.net
> > > > >>>> >>> >> > > > > >> > >
> > > > >>>> >>> >> > > > > >> > > > > wrote:
> > > > >>>> >>> >> > > > > >> > > > > >> > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > > Late to the game. A few
> > > > comments
> > > > >>>> after
> > > > >>>> >>> >> > > rereading
> > > > >>>> >>> >> > > > > this
> > > > >>>> >>> >> > > > > >> > > thread
> > > > >>>> >>> >> > > > > >> > > > > as a
> > > > >>>> >>> >> > > > > >> > > > > >> > > > 'user'.
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > > + Before merge, a
> > user-facing
> > > > >>>> feature
> > > > >>>> >>> like
> > > > >>>> >>> >> > this
> > > > >>>> >>> >> > > > > >> should
> > > > >>>> >>> >> > > > > >> > > work
> > > > >>>> >>> >> > > > > >> > > > > (If
> > > > >>>> >>> >> > > > > >> > > > > >> > this
> > > > >>>> >>> >> > > > > >> > > > > >> > > is
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > "higher-bar
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > > for new features", bring
> it
> > > on
> > > > --
> > > > >>>> >>> smile).
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > > + As a user, I tried the
> > > branch
> > > > >>>> with
> > > > >>>> >>> tools
> > > > >>>> >>> >> > > after
> > > > >>>> >>> >> > > > > >> > reviewing
> > > > >>>> >>> >> > > > > >> > > > the
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > just-posted
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > > doc. I had an
> 'interesting'
> > > > >>>> experience
> > > > >>>> >>> >> (left
> > > > >>>> >>> >> > > > > >> comments up
> > > > >>>> >>> >> > > > > >> > > on
> > > > >>>> >>> >> > > > > >> > > > > >> > issue). I
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > think
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > > the tooling/doc.
> important
> > to
> > > > get
> > > > >>>> >>> right.
> > > > >>>> >>> >> If
> > > > >>>> >>> >> > it
> > > > >>>> >>> >> > > > > breaks
> > > > >>>> >>> >> > > > > >> > > easily
> > > > >>>> >>> >> > > > > >> > > > > or
> > > > >>>> >>> >> > > > > >> > > > > >> is
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > > inconsistent (or lacks
> > > > 'polish'),
> > > > >>>> >>> >> operators
> > > > >>>> >>> >> > > will
> > > > >>>> >>> >> > > > > >> judge
> > > > >>>> >>> >> > > > > >> > the
> > > > >>>> >>> >> > > > > >> > > > > whole
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > > backup/restore tooling
> > chain
> > > as
> > > > >>>> not
> > > > >>>> >>> >> > trustworthy
> > > > >>>> >>> >> > > > and
> > > > >>>> >>> >> > > > > >> > > abandon
> > > > >>>> >>> >> > > > > >> > > > > it.
> > > > >>>> >>> >> > > > > >> > > > > >> > Lets
> > > > >>>> >>> >> > > > > >> > > > > >> > > > not
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > > have this happen to this
> > > > feature.
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > > + Matteo's suggestion
> > (with a
> > > > >>>> helpful
> > > > >>>> >>> >> starter
> > > > >>>> >>> >> > > > list)
> > > > >>>> >>> >> > > > > >> that
> > > > >>>> >>> >> > > > > >> > > > there
> > > > >>>> >>> >> > > > > >> > > > > >> > needs
> > > > >>>> >>> >> > > > > >> > > > > >> > > to
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > be
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > > explicit qualification on
> > > what
> > > > is
> > > > >>>> >>> actually
> > > > >>>> >>> >> > > being
> > > > >>>> >>> >> > > > > >> > delivered
> > > > >>>> >>> >> > > > > >> > > > --
> > > > >>>> >>> >> > > > > >> > > > > >> > > > including a
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > > listing of limitations
> > (some
> > > > look
> > > > >>>> >>> serious
> > > > >>>> >>> >> > such
> > > > >>>> >>> >> > > as
> > > > >>>> >>> >> > > > > >> data
> > > > >>>> >>> >> > > > > >> > > bleed
> > > > >>>> >>> >> > > > > >> > > > > from
> > > > >>>> >>> >> > > > > >> > > > > >> > > other
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > > regions in WALs, but
> maybe
> > I
> > > > don't
> > > > >>>> >>> care
> > > > >>>> >>> >> for
> > > > >>>> >>> >> > my
> > > > >>>> >>> >> > > > use
> > > > >>>> >>> >> > > > > >> > > case...)
> > > > >>>> >>> >> > > > > >> > > > --
> > > > >>>> >>> >> > > > > >> > > > > >> > needs
> > > > >>>> >>> >> > > > > >> > > > > >> > > to
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > > accompany the merge. Lets
> > > fold
> > > > >>>> them
> > > > >>>> >>> into
> > > > >>>> >>> >> the
> > > > >>>> >>> >> > > user
> > > > >>>> >>> >> > > > > >> doc.
> > > > >>>> >>> >> > > > > >> > in
> > > > >>>> >>> >> > > > > >> > > > the
> > > > >>>> >>> >> > > > > >> > > > > >> > > technical
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > > overview area as
> suggested
> > so
> > > > user
> > > > >>>> >>> >> > expectations
> > > > >>>> >>> >> > > > are
> > > > >>>> >>> >> > > > > >> > > properly
> > > > >>>> >>> >> > > > > >> > > > > >> > managed
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > > (otherwise, they expect
> the
> > > > world
> > > > >>>> and
> > > > >>>> >>> will
> > > > >>>> >>> >> > just
> > > > >>>> >>> >> > > > > give
> > > > >>>> >>> >> > > > > >> up
> > > > >>>> >>> >> > > > > >> > > when
> > > > >>>> >>> >> > > > > >> > > > > we
> > > > >>>> >>> >> > > > > >> > > > > >> > fall
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > > short). Vladimir did a
> list
> > > of
> > > > >>>> what
> > > > >>>> >>> is in
> > > > >>>> >>> >> > each
> > > > >>>> >>> >> > > of
> > > > >>>> >>> >> > > > > the
> > > > >>>> >>> >> > > > > >> > > phases
> > > > >>>> >>> >> > > > > >> > > > > >> above
> > > > >>>> >>> >> > > > > >> > > > > >> > > > which
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > > would serve as a good
> > start.
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > > + Is this feature
> > > > 'experimental'
> > > > >>>> >>> (Matteo
> > > > >>>> >>> >> asks
> > > > >>>> >>> >> > > > > above).
> > > > >>>> >>> >> > > > > >> > I'd
> > > > >>>> >>> >> > > > > >> > > > > prefer
> > > > >>>> >>> >> > > > > >> > > > > >> it
> > > > >>>> >>> >> > > > > >> > > > > >> > > is
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > > not. If it is, it should
> be
> > > > >>>> labelled
> > > > >>>> >>> all
> > > > >>>> >>> >> over
> > > > >>>> >>> >> > > > that
> > > > >>>> >>> >> > > > > >> it is
> > > > >>>> >>> >> > > > > >> > > > so. I
> > > > >>>> >>> >> > > > > >> > > > > >> see
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > current
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > > state called out as a
> '...
> > > > >>>> technical
> > > > >>>> >>> >> preview
> > > > >>>> >>> >> > > > > >> feature'.
> > > > >>>> >>> >> > > > > >> > > Does
> > > > >>>> >>> >> > > > > >> > > > > this
> > > > >>>> >>> >> > > > > >> > > > > >> > mean
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > > not-for-users?
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > > St.Ack
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > > On Mon, Sep 12, 2016 at
> > 8:03
> > > > AM,
> > > > >>>> Ted
> > > > >>>> >>> Yu <
> > > > >>>> >>> >> > > > > >> > > > yuzhihong@gmail.com>
> > > > >>>> >>> >> > > > > >> > > > > >> > wrote:
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> Sean:
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> Do you have more
> comments
> > ?
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >>
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> Cheers
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >>
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> On Fri, Sep 9, 2016 at
> > 1:42
> > > > PM,
> > > > >>>> >>> Vladimir
> > > > >>>> >>> >> > > > Rodionov
> > > > >>>> >>> >> > > > > <
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > vladrodionov@gmail.com
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> wrote:
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >>
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > Sean,
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > Backup/Restore can
> fail
> > > due
> > > > to
> > > > >>>> >>> various
> > > > >>>> >>> >> > > > reasons:
> > > > >>>> >>> >> > > > > >> > network
> > > > >>>> >>> >> > > > > >> > > > > outage
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > (cluster
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > wide), various
> time-outs
> > > in
> > > > >>>> HBase
> > > > >>>> >>> and
> > > > >>>> >>> >> HDFS
> > > > >>>> >>> >> > > > > layer,
> > > > >>>> >>> >> > > > > >> M/R
> > > > >>>> >>> >> > > > > >> > > > > failure
> > > > >>>> >>> >> > > > > >> > > > > >> > due
> > > > >>>> >>> >> > > > > >> > > > > >> > > to
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> "HDFS
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > exceeded quota", user
> > > error
> > > > >>>> (manual
> > > > >>>> >>> >> > deletion
> > > > >>>> >>> >> > > > of
> > > > >>>> >>> >> > > > > >> data)
> > > > >>>> >>> >> > > > > >> > > and
> > > > >>>> >>> >> > > > > >> > > > > so
> > > > >>>> >>> >> > > > > >> > > > > >> on
> > > > >>>> >>> >> > > > > >> > > > > >> > so
> > > > >>>> >>> >> > > > > >> > > > > >> > > > on.
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> That
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > is impossible to
> > enumerate
> > > > all
> > > > >>>> >>> possible
> > > > >>>> >>> >> > > types
> > > > >>>> >>> >> > > > of
> > > > >>>> >>> >> > > > > >> > > failures
> > > > >>>> >>> >> > > > > >> > > > > in a
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> distributed
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > system - that is not
> our
> > > > >>>> goal/task.
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > We focus completely on
> > > > backup
> > > > >>>> >>> system
> > > > >>>> >>> >> table
> > > > >>>> >>> >> > > > > >> > consistency
> > > > >>>> >>> >> > > > > >> > > > in a
> > > > >>>> >>> >> > > > > >> > > > > >> > > presence
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > of
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> any
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > type of failure. That
> is
> > > > what I
> > > > >>>> >>> call
> > > > >>>> >>> >> > > > "tolerance
> > > > >>>> >>> >> > > > > to
> > > > >>>> >>> >> > > > > >> > > > > failures".
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > On a failure:
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > BACKUP. All backup
> > system
> > > > >>>> >>> information
> > > > >>>> >>> >> > (prior
> > > > >>>> >>> >> > > > to
> > > > >>>> >>> >> > > > > >> > backup)
> > > > >>>> >>> >> > > > > >> > > > > will
> > > > >>>> >>> >> > > > > >> > > > > >> be
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > restored
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > and all temporary
> data,
> > > > >>>> related to
> > > > >>>> >>> a
> > > > >>>> >>> >> > failed
> > > > >>>> >>> >> > > > > >> session,
> > > > >>>> >>> >> > > > > >> > in
> > > > >>>> >>> >> > > > > >> > > > > HDFS
> > > > >>>> >>> >> > > > > >> > > > > >> > will
> > > > >>>> >>> >> > > > > >> > > > > >> > > be
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > deleted
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > RESTORE. We do not
> care
> > > > about
> > > > >>>> >>> system
> > > > >>>> >>> >> data,
> > > > >>>> >>> >> > > > > because
> > > > >>>> >>> >> > > > > >> > > > restore
> > > > >>>> >>> >> > > > > >> > > > > >> does
> > > > >>>> >>> >> > > > > >> > > > > >> > > not
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> change
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > it. Temporary data in
> > HDFS
> > > > >>>> will be
> > > > >>>> >>> >> cleaned
> > > > >>>> >>> >> > > up
> > > > >>>> >>> >> > > > > and
> > > > >>>> >>> >> > > > > >> > table
> > > > >>>> >>> >> > > > > >> > > > > will
> > > > >>>> >>> >> > > > > >> > > > > >> be
> > > > >>>> >>> >> > > > > >> > > > > >> > > in a
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> state
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > back to where it was
> > > before
> > > > >>>> >>> operation
> > > > >>>> >>> >> > > started.
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > This is what user
> should
> > > > >>>> expect in
> > > > >>>> >>> case
> > > > >>>> >>> >> > of a
> > > > >>>> >>> >> > > > > >> failure.
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > -Vlad
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > -Vlad
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > On Fri, Sep 9, 2016 at
> > > 12:56
> > > > >>>> PM,
> > > > >>>> >>> Sean
> > > > >>>> >>> >> > > Busbey <
> > > > >>>> >>> >> > > > > >> > > > > >> busbey@apache.org
> > > > >>>> >>> >> > > > > >> > > > > >> > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > wrote:
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > Failing in a
> > consistent
> > > > way,
> > > > >>>> with
> > > > >>>> >>> >> docs
> > > > >>>> >>> >> > > that
> > > > >>>> >>> >> > > > > >> explain
> > > > >>>> >>> >> > > > > >> > > the
> > > > >>>> >>> >> > > > > >> > > > > >> > various
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > expected failures
> > would
> > > be
> > > > >>>> >>> >> sufficient.
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > On Fri, Sep 9, 2016
> at
> > > > 12:16
> > > > >>>> PM,
> > > > >>>> >>> >> > Vladimir
> > > > >>>> >>> >> > > > > >> Rodionov
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > <
> > vladrodionov@gmail.com
> > > >
> > > > >>>> wrote:
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > Do not worry Sean,
> > doc
> > > > is
> > > > >>>> >>> coming
> > > > >>>> >>> >> today
> > > > >>>> >>> >> > > as
> > > > >>>> >>> >> > > > a
> > > > >>>> >>> >> > > > > >> > preview
> > > > >>>> >>> >> > > > > >> > > > and
> > > > >>>> >>> >> > > > > >> > > > > >> our
> > > > >>>> >>> >> > > > > >> > > > > >> > > > writer
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > Frank
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > will be working
> on a
> > > > >>>> putting
> > > > >>>> >>> it
> > > > >>>> >>> >> into
> > > > >>>> >>> >> > > > Apache
> > > > >>>> >>> >> > > > > >> > repo.
> > > > >>>> >>> >> > > > > >> > > > > >> Timeline
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > depends
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> on
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > Franks schedule
> but
> > I
> > > > hope
> > > > >>>> we
> > > > >>>> >>> will
> > > > >>>> >>> >> get
> > > > >>>> >>> >> > > it
> > > > >>>> >>> >> > > > > >> rather
> > > > >>>> >>> >> > > > > >> > > > sooner
> > > > >>>> >>> >> > > > > >> > > > > >> than
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > later.
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > As for failure
> > > testing,
> > > > we
> > > > >>>> are
> > > > >>>> >>> >> > focusing
> > > > >>>> >>> >> > > > only
> > > > >>>> >>> >> > > > > >> on a
> > > > >>>> >>> >> > > > > >> > > > > >> consistent
> > > > >>>> >>> >> > > > > >> > > > > >> > > > state
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> of
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > backup system data
> > in
> > > a
> > > > >>>> >>> presence of
> > > > >>>> >>> >> > any
> > > > >>>> >>> >> > > > type
> > > > >>>> >>> >> > > > > >> of
> > > > >>>> >>> >> > > > > >> > > > > failures,
> > > > >>>> >>> >> > > > > >> > > > > >> We
> > > > >>>> >>> >> > > > > >> > > > > >> > > are
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > not
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > going
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > to implement
> > anything
> > > > more
> > > > >>>> >>> >> "fancy",
> > > > >>>> >>> >> > > than
> > > > >>>> >>> >> > > > > >> that.
> > > > >>>> >>> >> > > > > >> > We
> > > > >>>> >>> >> > > > > >> > > > > allow
> > > > >>>> >>> >> > > > > >> > > > > >> > both:
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> backup
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > and
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > restore to fail.
> > What
> > > we
> > > > >>>> do not
> > > > >>>> >>> >> allow
> > > > >>>> >>> >> > is
> > > > >>>> >>> >> > > > to
> > > > >>>> >>> >> > > > > >> have
> > > > >>>> >>> >> > > > > >> > > > system
> > > > >>>> >>> >> > > > > >> > > > > >> data
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> corrupted.
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > Will it suffice
> for
> > > you?
> > > > >>>> Do you
> > > > >>>> >>> >> have
> > > > >>>> >>> >> > any
> > > > >>>> >>> >> > > > > other
> > > > >>>> >>> >> > > > > >> > > > > concerns,
> > > > >>>> >>> >> > > > > >> > > > > >> you
> > > > >>>> >>> >> > > > > >> > > > > >> > > > want
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> us to
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > address?
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > -Vlad
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > On Fri, Sep 9,
> 2016
> > at
> > > > >>>> 10:56
> > > > >>>> >>> AM,
> > > > >>>> >>> >> Sean
> > > > >>>> >>> >> > > > > Busbey <
> > > > >>>> >>> >> > > > > >> > > > > >> > > busbey@apache.org
> > > > >>>> >>> >> > > > > >> > > > > >> > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > wrote:
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> "docs will come
> to
> > > > Apache
> > > > >>>> >>> soon"
> > > > >>>> >>> >> does
> > > > >>>> >>> >> > > not
> > > > >>>> >>> >> > > > > >> address
> > > > >>>> >>> >> > > > > >> > > my
> > > > >>>> >>> >> > > > > >> > > > > >> concern
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > around
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > docs
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > at
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> all, unless said
> > docs
> > > > have
> > > > >>>> >>> already
> > > > >>>> >>> >> > made
> > > > >>>> >>> >> > > > it
> > > > >>>> >>> >> > > > > >> into
> > > > >>>> >>> >> > > > > >> > > the
> > > > >>>> >>> >> > > > > >> > > > > >> project
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > repo. I
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > don't
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> want third party
> > > > >>>> resources for
> > > > >>>> >>> >> using
> > > > >>>> >>> >> > a
> > > > >>>> >>> >> > > > > major
> > > > >>>> >>> >> > > > > >> and
> > > > >>>> >>> >> > > > > >> > > > > >> important
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > feature
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> of
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > the
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> project, I want
> us
> > to
> > > > >>>> provide
> > > > >>>> >>> end
> > > > >>>> >>> >> > users
> > > > >>>> >>> >> > > > > with
> > > > >>>> >>> >> > > > > >> > what
> > > > >>>> >>> >> > > > > >> > > > they
> > > > >>>> >>> >> > > > > >> > > > > >> need
> > > > >>>> >>> >> > > > > >> > > > > >> > > to
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > get
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> the
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > job
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> done.
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >>
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> I see some calls
> > for
> > > > >>>> patience
> > > > >>>> >>> on
> > > > >>>> >>> >> the
> > > > >>>> >>> >> > > > > failure
> > > > >>>> >>> >> > > > > >> > > > testing,
> > > > >>>> >>> >> > > > > >> > > > > but
> > > > >>>> >>> >> > > > > >> > > > > >> > the
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> appeal
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > to
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > us
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> having done a bad
> > job
> > > > of
> > > > >>>> >>> requiring
> > > > >>>> >>> >> > > proper
> > > > >>>> >>> >> > > > > >> tests
> > > > >>>> >>> >> > > > > >> > of
> > > > >>>> >>> >> > > > > >> > > > > >> previous
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> features
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > just
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> makes me more
> > > concerned
> > > > >>>> about
> > > > >>>> >>> not
> > > > >>>> >>> >> > > getting
> > > > >>>> >>> >> > > > > >> them
> > > > >>>> >>> >> > > > > >> > > > here. I
> > > > >>>> >>> >> > > > > >> > > > > >> > don't
> > > > >>>> >>> >> > > > > >> > > > > >> > > > want
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> to
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > set
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> yet another bad
> > > example
> > > > >>>> that
> > > > >>>> >>> will
> > > > >>>> >>> >> > then
> > > > >>>> >>> >> > > be
> > > > >>>> >>> >> > > > > >> > pointed
> > > > >>>> >>> >> > > > > >> > > to
> > > > >>>> >>> >> > > > > >> > > > > in
> > > > >>>> >>> >> > > > > >> > > > > >> the
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > future.
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >>
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> On Sep 8, 2016
> > 10:50,
> > > > "Ted
> > > > >>>> >>> Yu" <
> > > > >>>> >>> >> > > > > >> > > yuzhihong@gmail.com
> > > > >>>> >>> >> > > > > >> > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > wrote:
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >>
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > Is there any
> > > concern
> > > > >>>> which
> > > > >>>> >>> is
> > > > >>>> >>> >> not
> > > > >>>> >>> >> > > > > >> addressed ?
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > Do we need
> > another
> > > > Vote
> > > > >>>> >>> thread ?
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > Thanks
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > On Thu, Sep 8,
> > 2016
> > > > at
> > > > >>>> 9:21
> > > > >>>> >>> AM,
> > > > >>>> >>> >> > > Andrew
> > > > >>>> >>> >> > > > > >> > Purtell <
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> apurtell@apache.org
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > wrote:
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > Vlad,
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > I apologize
> for
> > > > using
> > > > >>>> the
> > > > >>>> >>> term
> > > > >>>> >>> >> > > > > >> 'half-baked'
> > > > >>>> >>> >> > > > > >> > > in a
> > > > >>>> >>> >> > > > > >> > > > > way
> > > > >>>> >>> >> > > > > >> > > > > >> > that
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > could
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > seem a
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > description
> of
> > > > >>>> >>> HBASE-7912. I
> > > > >>>> >>> >> > meant
> > > > >>>> >>> >> > > > that
> > > > >>>> >>> >> > > > > >> as a
> > > > >>>> >>> >> > > > > >> > > > > general
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> hypothetical.
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > On Wed, Sep
> 7,
> > > > 2016 at
> > > > >>>> >>> 9:36
> > > > >>>> >>> >> AM,
> > > > >>>> >>> >> > > > > Vladimir
> > > > >>>> >>> >> > > > > >> > > > Rodionov
> > > > >>>> >>> >> > > > > >> > > > > <
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> > > > vladrodionov@gmail.com>
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > wrote:
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >> I'm not
> > sure
> > > > that
> > > > >>>> >>> "There
> > > > >>>> >>> >> is
> > > > >>>> >>> >> > > > > already
> > > > >>>> >>> >> > > > > >> > lots
> > > > >>>> >>> >> > > > > >> > > of
> > > > >>>> >>> >> > > > > >> > > > > >> > > half-baked
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> code
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > in
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > the
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > branch,
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > so what's
> the
> > > > harm
> > > > >>>> in
> > > > >>>> >>> adding
> > > > >>>> >>> >> > > more?"
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > I meant -
> not
> > > > >>>> >>> production -
> > > > >>>> >>> >> > ready
> > > > >>>> >>> >> > > > yet.
> > > > >>>> >>> >> > > > > >> This
> > > > >>>> >>> >> > > > > >> > > is
> > > > >>>> >>> >> > > > > >> > > > > 2.0
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > development
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > branch
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > and,
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > hence many
> > > > features
> > > > >>>> are
> > > > >>>> >>> in
> > > > >>>> >>> >> > works,
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > not being
> > > tested
> > > > >>>> well
> > > > >>>> >>> etc.
> > > > >>>> >>> >> I do
> > > > >>>> >>> >> > > not
> > > > >>>> >>> >> > > > > >> > consider
> > > > >>>> >>> >> > > > > >> > > > > backup
> > > > >>>> >>> >> > > > > >> > > > > >> > as
> > > > >>>> >>> >> > > > > >> > > > > >> > > > half
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > baked
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > feature -
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > it has
> passed
> > > our
> > > > >>>> >>> internal
> > > > >>>> >>> >> QA
> > > > >>>> >>> >> > and
> > > > >>>> >>> >> > > > has
> > > > >>>> >>> >> > > > > >> very
> > > > >>>> >>> >> > > > > >> > > > good
> > > > >>>> >>> >> > > > > >> > > > > >> doc,
> > > > >>>> >>> >> > > > > >> > > > > >> > > > which
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > we
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > will
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > provide
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > to Apache
> > > > shortly.
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > -Vlad
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > On Wed, Sep
> > 7,
> > > > 2016
> > > > >>>> at
> > > > >>>> >>> 9:13
> > > > >>>> >>> >> AM,
> > > > >>>> >>> >> > > > > Andrew
> > > > >>>> >>> >> > > > > >> > > > Purtell <
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > apurtell@apache.org
> >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > We
> > shouldn't
> > > > admit
> > > > >>>> >>> half
> > > > >>>> >>> >> baked
> > > > >>>> >>> >> > > > > changes
> > > > >>>> >>> >> > > > > >> > that
> > > > >>>> >>> >> > > > > >> > > > > won't
> > > > >>>> >>> >> > > > > >> > > > > >> be
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> finished.
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> However
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > in
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > this case
> > the
> > > > crew
> > > > >>>> >>> >> working on
> > > > >>>> >>> >> > > > this
> > > > >>>> >>> >> > > > > >> > feature
> > > > >>>> >>> >> > > > > >> > > > are
> > > > >>>> >>> >> > > > > >> > > > > >> long
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > timers
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> and
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > less
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > likely
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > than just
> > > about
> > > > >>>> >>> anyone to
> > > > >>>> >>> >> > leave
> > > > >>>> >>> >> > > > > >> > something
> > > > >>>> >>> >> > > > > >> > > > in a
> > > > >>>> >>> >> > > > > >> > > > > >> half
> > > > >>>> >>> >> > > > > >> > > > > >> > > > baked
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > state. Of
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > course
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > there is
> no
> > > > >>>> guarantee
> > > > >>>> >>> how
> > > > >>>> >>> >> > > > anything
> > > > >>>> >>> >> > > > > >> will
> > > > >>>> >>> >> > > > > >> > > turn
> > > > >>>> >>> >> > > > > >> > > > > out,
> > > > >>>> >>> >> > > > > >> > > > > >> > > but I
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > am
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > willing
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> to
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > take
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > a little
> on
> > > > faith
> > > > >>>> if
> > > > >>>> >>> they
> > > > >>>> >>> >> > feel
> > > > >>>> >>> >> > > > > their
> > > > >>>> >>> >> > > > > >> > best
> > > > >>>> >>> >> > > > > >> > > > path
> > > > >>>> >>> >> > > > > >> > > > > >> > > forward
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > now
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> is
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > to
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > merge
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > to
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > trunk. I
> > only
> > > > >>>> wish I
> > > > >>>> >>> had
> > > > >>>> >>> >> > > > bandwidth
> > > > >>>> >>> >> > > > > to
> > > > >>>> >>> >> > > > > >> > have
> > > > >>>> >>> >> > > > > >> > > > > done
> > > > >>>> >>> >> > > > > >> > > > > >> > some
> > > > >>>> >>> >> > > > > >> > > > > >> > > > real
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > kicking
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> of
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > the
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > tires by
> > now.
> > > > >>>> Maybe
> > > > >>>> >>> this
> > > > >>>> >>> >> > week.
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > (Yes, I'm
> > > using
> > > > >>>> some
> > > > >>>> >>> of
> > > > >>>> >>> >> that
> > > > >>>> >>> >> > > time
> > > > >>>> >>> >> > > > > for
> > > > >>>> >>> >> > > > > >> > this
> > > > >>>> >>> >> > > > > >> > > > > email
> > > > >>>> >>> >> > > > > >> > > > > >> > :-)
> > > > >>>> >>> >> > > > > >> > > > > >> > > > but
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > I
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > type
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > fast.)
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > That
> said,
> > I
> > > > would
> > > > >>>> >>> like to
> > > > >>>> >>> >> > > > agitate
> > > > >>>> >>> >> > > > > >> for
> > > > >>>> >>> >> > > > > >> > > > making
> > > > >>>> >>> >> > > > > >> > > > > 2.0
> > > > >>>> >>> >> > > > > >> > > > > >> > > more
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > real
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > and
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> spend
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > some
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > time on
> it
> > > now
> > > > >>>> that
> > > > >>>> >>> I'm
> > > > >>>> >>> >> > winding
> > > > >>>> >>> >> > > > > down
> > > > >>>> >>> >> > > > > >> > with
> > > > >>>> >>> >> > > > > >> > > > > 0.98. I
> > > > >>>> >>> >> > > > > >> > > > > >> > > think
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> that
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > means
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branching
> > for
> > > > 2.0
> > > > >>>> real
> > > > >>>> >>> >> soon
> > > > >>>> >>> >> > now
> > > > >>>> >>> >> > > > and
> > > > >>>> >>> >> > > > > >> even
> > > > >>>> >>> >> > > > > >> > > > > evicting
> > > > >>>> >>> >> > > > > >> > > > > >> > > > things
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> from
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > 2.0
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > branch
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > that
> aren't
> > > > >>>> finished
> > > > >>>> >>> or
> > > > >>>> >>> >> > stable,
> > > > >>>> >>> >> > > > > >> leaving
> > > > >>>> >>> >> > > > > >> > > them
> > > > >>>> >>> >> > > > > >> > > > > only
> > > > >>>> >>> >> > > > > >> > > > > >> > > once
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> again
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > in
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > the
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > master
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branch.
> Or,
> > > > maybe
> > > > >>>> just
> > > > >>>> >>> >> > evicting
> > > > >>>> >>> >> > > > > them.
> > > > >>>> >>> >> > > > > >> > > Let's
> > > > >>>> >>> >> > > > > >> > > > > take
> > > > >>>> >>> >> > > > > >> > > > > >> it
> > > > >>>> >>> >> > > > > >> > > > > >> > > > case
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > by
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > case.
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > I think
> > this
> > > > >>>> feature
> > > > >>>> >>> can
> > > > >>>> >>> >> come
> > > > >>>> >>> >> > > in
> > > > >>>> >>> >> > > > > >> > > relatively
> > > > >>>> >>> >> > > > > >> > > > > >> safely.
> > > > >>>> >>> >> > > > > >> > > > > >> > > As
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> added
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > insurance,
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > let's
> admit
> > > the
> > > > >>>> >>> >> possibility
> > > > >>>> >>> >> > it
> > > > >>>> >>> >> > > > > could
> > > > >>>> >>> >> > > > > >> be
> > > > >>>> >>> >> > > > > >> > > > > reverted
> > > > >>>> >>> >> > > > > >> > > > > >> on
> > > > >>>> >>> >> > > > > >> > > > > >> > > the
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > 2.0
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > branch
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> if
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > folks
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > working
> on
> > > > >>>> >>> stabilizing 2.0
> > > > >>>> >>> >> > > decide
> > > > >>>> >>> >> > > > > to
> > > > >>>> >>> >> > > > > >> > evict
> > > > >>>> >>> >> > > > > >> > > > it
> > > > >>>> >>> >> > > > > >> > > > > >> > because
> > > > >>>> >>> >> > > > > >> > > > > >> > > > it
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > is
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > unfinished
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > or
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > unstable,
> > > > because
> > > > >>>> that
> > > > >>>> >>> >> > > certainly
> > > > >>>> >>> >> > > > > can
> > > > >>>> >>> >> > > > > >> > > > happen. I
> > > > >>>> >>> >> > > > > >> > > > > >> > would
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> expect if
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > talk
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > like
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > that
> > starts,
> > > > we'd
> > > > >>>> get
> > > > >>>> >>> help
> > > > >>>> >>> >> > > > > finishing
> > > > >>>> >>> >> > > > > >> or
> > > > >>>> >>> >> > > > > >> > > > > >> stabilizing
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > what's
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > under
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > discussion
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > for
> revert.
> > > Or,
> > > > >>>> we'd
> > > > >>>> >>> have
> > > > >>>> >>> >> a
> > > > >>>> >>> >> > > > revert.
> > > > >>>> >>> >> > > > > >> > Either
> > > > >>>> >>> >> > > > > >> > > > way
> > > > >>>> >>> >> > > > > >> > > > > >> the
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > outcome
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> is
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > acceptable.
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > On Wed,
> Sep
> > > 7,
> > > > >>>> 2016 at
> > > > >>>> >>> >> 8:56
> > > > >>>> >>> >> > AM,
> > > > >>>> >>> >> > > > > Dima
> > > > >>>> >>> >> > > > > >> > > Spivak
> > > > >>>> >>> >> > > > > >> > > > <
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > >
> dimaspivak@apache.org
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > I'm not
> > > sure
> > > > >>>> that
> > > > >>>> >>> >> "There is
> > > > >>>> >>> >> > > > > already
> > > > >>>> >>> >> > > > > >> > lots
> > > > >>>> >>> >> > > > > >> > > > of
> > > > >>>> >>> >> > > > > >> > > > > >> > > > half-baked
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> code
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > in
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> the
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branch,
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > so
> what's
> > > the
> > > > >>>> harm
> > > > >>>> >>> in
> > > > >>>> >>> >> > adding
> > > > >>>> >>> >> > > > > more?"
> > > > >>>> >>> >> > > > > >> > is a
> > > > >>>> >>> >> > > > > >> > > > > good
> > > > >>>> >>> >> > > > > >> > > > > >> > code
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > commit
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > philosophy
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > for
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > a
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> > > > fault-tolerant
> > > > >>>> >>> >> distributed
> > > > >>>> >>> >> > > data
> > > > >>>> >>> >> > > > > >> store.
> > > > >>>> >>> >> > > > > >> > > ;)
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > More
> > > > seriously,
> > > > >>>> a
> > > > >>>> >>> lack
> > > > >>>> >>> >> of
> > > > >>>> >>> >> > > test
> > > > >>>> >>> >> > > > > >> > coverage
> > > > >>>> >>> >> > > > > >> > > > for
> > > > >>>> >>> >> > > > > >> > > > > >> > > existing
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > features
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > shouldn't
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > be
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > used as
> > > > >>>> >>> justification
> > > > >>>> >>> >> for
> > > > >>>> >>> >> > > > > >> introducing
> > > > >>>> >>> >> > > > > >> > > new
> > > > >>>> >>> >> > > > > >> > > > > >> > features
> > > > >>>> >>> >> > > > > >> > > > > >> > > > with
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> the
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > same
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> > > shortcomings.
> > > > >>>> >>> >> Ultimately,
> > > > >>>> >>> >> > > it's
> > > > >>>> >>> >> > > > > the
> > > > >>>> >>> >> > > > > >> end
> > > > >>>> >>> >> > > > > >> > > > user
> > > > >>>> >>> >> > > > > >> > > > > who
> > > > >>>> >>> >> > > > > >> > > > > >> > > will
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > feel
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > the
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> pain,
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > so
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> shouldn't
> > > we
> > > > do
> > > > >>>> >>> >> everything
> > > > >>>> >>> >> > we
> > > > >>>> >>> >> > > > can
> > > > >>>> >>> >> > > > > >> to
> > > > >>>> >>> >> > > > > >> > > > > mitigate
> > > > >>>> >>> >> > > > > >> > > > > >> > that?
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > -Dima
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > On Wed,
> > Sep
> > > > 7,
> > > > >>>> 2016
> > > > >>>> >>> at
> > > > >>>> >>> >> 8:46
> > > > >>>> >>> >> > > AM,
> > > > >>>> >>> >> > > > > >> > Vladimir
> > > > >>>> >>> >> > > > > >> > > > > >> > Rodionov <
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > > >>>> >>> vladrodionov@gmail.com>
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > wrote:
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Sean,
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > *
> have
> > > docs
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> Agree.
> > We
> > > > >>>> have a
> > > > >>>> >>> doc
> > > > >>>> >>> >> and
> > > > >>>> >>> >> > > > backup
> > > > >>>> >>> >> > > > > >> is
> > > > >>>> >>> >> > > > > >> > the
> > > > >>>> >>> >> > > > > >> > > > > most
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > documented
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > feature
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > :),
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > we
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > will
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> release
> > > it
> > > > >>>> >>> shortly to
> > > > >>>> >>> >> > > Apache.
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > *
> have
> > > > >>>> sunny-day
> > > > >>>> >>> >> > > correctness
> > > > >>>> >>> >> > > > > >> tests
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> Feature
> > > has
> > > > >>>> >>> close to
> > > > >>>> >>> >> 60
> > > > >>>> >>> >> > > test
> > > > >>>> >>> >> > > > > >> cases,
> > > > >>>> >>> >> > > > > >> > > > which
> > > > >>>> >>> >> > > > > >> > > > > >> run
> > > > >>>> >>> >> > > > > >> > > > > >> > > for
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> approx
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > 30
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> min.
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > We
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > can
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > add
> > more,
> > > > if
> > > > >>>> >>> >> community do
> > > > >>>> >>> >> > > not
> > > > >>>> >>> >> > > > > >> mind
> > > > >>>> >>> >> > > > > >> > :)
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > *
> have
> > > > >>>> >>> >> > > > > >> > correctness-in-face-of-failure
> > > > >>>> >>> >> > > > > >> > > > > tests
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Any
> > > > examples
> > > > >>>> of
> > > > >>>> >>> these
> > > > >>>> >>> >> > tests
> > > > >>>> >>> >> > > > in
> > > > >>>> >>> >> > > > > >> > > existing
> > > > >>>> >>> >> > > > > >> > > > > >> > features?
> > > > >>>> >>> >> > > > > >> > > > > >> > > > In
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > works,
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > we
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > have a
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > clear
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > understanding
> > > > >>>> of
> > > > >>>> >>> what
> > > > >>>> >>> >> > > should
> > > > >>>> >>> >> > > > be
> > > > >>>> >>> >> > > > > >> done
> > > > >>>> >>> >> > > > > >> > > by
> > > > >>>> >>> >> > > > > >> > > > > the
> > > > >>>> >>> >> > > > > >> > > > > >> > time
> > > > >>>> >>> >> > > > > >> > > > > >> > > of
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > 2.0
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> release.
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > That
> is
> > > > very
> > > > >>>> close
> > > > >>>> >>> >> goal
> > > > >>>> >>> >> > for
> > > > >>>> >>> >> > > > us,
> > > > >>>> >>> >> > > > > >> to
> > > > >>>> >>> >> > > > > >> > > > verify
> > > > >>>> >>> >> > > > > >> > > > > IT
> > > > >>>> >>> >> > > > > >> > > > > >> > > monkey
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > for
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> existing
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > code.
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > *
> don't
> > > > rely
> > > > >>>> on
> > > > >>>> >>> things
> > > > >>>> >>> >> > > > outside
> > > > >>>> >>> >> > > > > of
> > > > >>>> >>> >> > > > > >> > > HBase
> > > > >>>> >>> >> > > > > >> > > > > for
> > > > >>>> >>> >> > > > > >> > > > > >> > > normal
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > operation
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > (okay
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > for
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > advanced
> > > > >>>> >>> operation)
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > We do
> > > not.
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > Enormous
> > > > time
> > > > >>>> has
> > > > >>>> >>> been
> > > > >>>> >>> >> > > spent
> > > > >>>> >>> >> > > > > >> already
> > > > >>>> >>> >> > > > > >> > > on
> > > > >>>> >>> >> > > > > >> > > > > the
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > development
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > and
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > testing
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > the
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > feature,
> > > it
> > > > >>>> has
> > > > >>>> >>> passed
> > > > >>>> >>> >> > our
> > > > >>>> >>> >> > > > > >> internal
> > > > >>>> >>> >> > > > > >> > > > tests
> > > > >>>> >>> >> > > > > >> > > > > and
> > > > >>>> >>> >> > > > > >> > > > > >> > > many
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> rounds
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > of
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> code
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > reviews
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > by
> > HBase
> > > > >>>> >>> committers.
> > > > >>>> >>> >> We
> > > > >>>> >>> >> > do
> > > > >>>> >>> >> > > > not
> > > > >>>> >>> >> > > > > >> mind
> > > > >>>> >>> >> > > > > >> > if
> > > > >>>> >>> >> > > > > >> > > > > >> someone
> > > > >>>> >>> >> > > > > >> > > > > >> > > from
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> HBase
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > community
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > (outside
> > > of
> > > > >>>> HW)
> > > > >>>> >>> will
> > > > >>>> >>> >> > review
> > > > >>>> >>> >> > > > the
> > > > >>>> >>> >> > > > > >> > code,
> > > > >>>> >>> >> > > > > >> > > > but
> > > > >>>> >>> >> > > > > >> > > > > it
> > > > >>>> >>> >> > > > > >> > > > > >> > will
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> probably
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> takes
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > forever
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > to
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > wait
> > for
> > > > >>>> >>> volunteer?,
> > > > >>>> >>> >> the
> > > > >>>> >>> >> > > > > feature
> > > > >>>> >>> >> > > > > >> is
> > > > >>>> >>> >> > > > > >> > > > quite
> > > > >>>> >>> >> > > > > >> > > > > >> large
> > > > >>>> >>> >> > > > > >> > > > > >> > > > (1MB+
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> cumulative
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > patch)
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > 2.0
> > > branch
> > > > is
> > > > >>>> >>> full of
> > > > >>>> >>> >> > half
> > > > >>>> >>> >> > > > > baked
> > > > >>>> >>> >> > > > > >> > > > features,
> > > > >>>> >>> >> > > > > >> > > > > >> most
> > > > >>>> >>> >> > > > > >> > > > > >> > > of
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > them
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > are
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > in
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > active
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > development,
> > > > >>>> >>> >> therefore I
> > > > >>>> >>> >> > am
> > > > >>>> >>> >> > > > not
> > > > >>>> >>> >> > > > > >> > > > following
> > > > >>>> >>> >> > > > > >> > > > > you
> > > > >>>> >>> >> > > > > >> > > > > >> > > here,
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> Sean?
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > Why
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> HBASE-7912
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > is
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > not
> > good
> > > > >>>> enough
> > > > >>>> >>> yet
> > > > >>>> >>> >> to be
> > > > >>>> >>> >> > > > > >> integrated
> > > > >>>> >>> >> > > > > >> > > > into
> > > > >>>> >>> >> > > > > >> > > > > 2.0
> > > > >>>> >>> >> > > > > >> > > > > >> > > > branch?
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > -Vlad
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > On
> Wed,
> > > > Sep 7,
> > > > >>>> >>> 2016 at
> > > > >>>> >>> >> > 8:23
> > > > >>>> >>> >> > > > AM,
> > > > >>>> >>> >> > > > > >> Sean
> > > > >>>> >>> >> > > > > >> > > > > Busbey <
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > busbey@apache.org
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > On
> > Tue,
> > > > Sep
> > > > >>>> 6,
> > > > >>>> >>> 2016
> > > > >>>> >>> >> at
> > > > >>>> >>> >> > > > 10:36
> > > > >>>> >>> >> > > > > >> PM,
> > > > >>>> >>> >> > > > > >> > > Josh
> > > > >>>> >>> >> > > > > >> > > > > >> Elser <
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> > > > josh.elser@gmail.com>
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> wrote:
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> So,
> > > the
> > > > >>>> >>> answer to
> > > > >>>> >>> >> > > Sean's
> > > > >>>> >>> >> > > > > >> > original
> > > > >>>> >>> >> > > > > >> > > > > >> question
> > > > >>>> >>> >> > > > > >> > > > > >> > is
> > > > >>>> >>> >> > > > > >> > > > > >> > > > "as
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > robust as
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > snapshots
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> > > > presently
> > > > >>>> >>> are"?
> > > > >>>> >>> >> > > > > >> (independence of
> > > > >>>> >>> >> > > > > >> > > > > >> > > backup/restore
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > failure
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > tolerance
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > from
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> > > > snapshot
> > > > >>>> >>> failure
> > > > >>>> >>> >> > > > tolerance)
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> Is
> > > this
> > > > >>>> just a
> > > > >>>> >>> >> > question
> > > > >>>> >>> >> > > > WRT
> > > > >>>> >>> >> > > > > >> > > context
> > > > >>>> >>> >> > > > > >> > > > of
> > > > >>>> >>> >> > > > > >> > > > > >> the
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > change,
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> or
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > is it
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > means
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > for a
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> veto
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> > from
> > > > you,
> > > > >>>> >>> Sean?
> > > > >>>> >>> >> Just
> > > > >>>> >>> >> > > > trying
> > > > >>>> >>> >> > > > > >> to
> > > > >>>> >>> >> > > > > >> > > make
> > > > >>>> >>> >> > > > > >> > > > > sure
> > > > >>>> >>> >> > > > > >> > > > > >> > I'm
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> following
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> along
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > adequately.
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > I'd
> > say
> > > > ATM
> > > > >>>> I'm
> > > > >>>> >>> -0,
> > > > >>>> >>> >> > > > bordering
> > > > >>>> >>> >> > > > > >> on
> > > > >>>> >>> >> > > > > >> > -1
> > > > >>>> >>> >> > > > > >> > > > but
> > > > >>>> >>> >> > > > > >> > > > > not
> > > > >>>> >>> >> > > > > >> > > > > >> > for
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> reasons
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > I
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > can
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> > articulate
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> well.
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > Here's
> > > an
> > > > >>>> >>> attempt.
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> We've
> > > > been
> > > > >>>> >>> trying to
> > > > >>>> >>> >> > > move,
> > > > >>>> >>> >> > > > > as a
> > > > >>>> >>> >> > > > > >> > > > > community,
> > > > >>>> >>> >> > > > > >> > > > > >> > > > towards
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > minimizing
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > risk
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > to
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > > downstream
> > > > >>>> >>> folks by
> > > > >>>> >>> >> > > getting
> > > > >>>> >>> >> > > > > >> > > "complete
> > > > >>>> >>> >> > > > > >> > > > > >> enough
> > > > >>>> >>> >> > > > > >> > > > > >> > > for
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > use"
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > gates
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> in
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > place
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > before
> > > we
> > > > >>>> >>> introduce
> > > > >>>> >>> >> new
> > > > >>>> >>> >> > > > > >> features.
> > > > >>>> >>> >> > > > > >> > > This
> > > > >>>> >>> >> > > > > >> > > > > was
> > > > >>>> >>> >> > > > > >> > > > > >> > > > spurred
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> by a
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > some
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > features
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > getting
> > > > in
> > > > >>>> >>> >> half-baked
> > > > >>>> >>> >> > and
> > > > >>>> >>> >> > > > > never
> > > > >>>> >>> >> > > > > >> > > making
> > > > >>>> >>> >> > > > > >> > > > > it
> > > > >>>> >>> >> > > > > >> > > > > >> to
> > > > >>>> >>> >> > > > > >> > > > > >> > > "can
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> really
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > use"
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > status
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> (I'm
> > > > >>>> thinking of
> > > > >>>> >>> >> > > > distributed
> > > > >>>> >>> >> > > > > >> log
> > > > >>>> >>> >> > > > > >> > > > replay
> > > > >>>> >>> >> > > > > >> > > > > and
> > > > >>>> >>> >> > > > > >> > > > > >> > the
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> zk-less
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > assignment
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > stuff,
> > > I
> > > > >>>> don't
> > > > >>>> >>> >> recall
> > > > >>>> >>> >> > if
> > > > >>>> >>> >> > > > > there
> > > > >>>> >>> >> > > > > >> was
> > > > >>>> >>> >> > > > > >> > > > > more).
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The
> > > > gates,
> > > > >>>> >>> >> generally,
> > > > >>>> >>> >> > > > > included
> > > > >>>> >>> >> > > > > >> > > things
> > > > >>>> >>> >> > > > > >> > > > > like:
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > *
> > have
> > > > docs
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > *
> > have
> > > > >>>> sunny-day
> > > > >>>> >>> >> > > > correctness
> > > > >>>> >>> >> > > > > >> tests
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > *
> > have
> > > > >>>> >>> >> > > > > >> > > correctness-in-face-of-failure
> > > > >>>> >>> >> > > > > >> > > > > tests
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > *
> > don't
> > > > >>>> rely on
> > > > >>>> >>> >> things
> > > > >>>> >>> >> > > > > outside
> > > > >>>> >>> >> > > > > >> of
> > > > >>>> >>> >> > > > > >> > > > HBase
> > > > >>>> >>> >> > > > > >> > > > > for
> > > > >>>> >>> >> > > > > >> > > > > >> > > > normal
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > operation
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > (okay
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > for
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > advanced
> > > > >>>> >>> operation)
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > As
> an
> > > > >>>> example,
> > > > >>>> >>> we
> > > > >>>> >>> >> kept
> > > > >>>> >>> >> > > the
> > > > >>>> >>> >> > > > > MOB
> > > > >>>> >>> >> > > > > >> > work
> > > > >>>> >>> >> > > > > >> > > > off
> > > > >>>> >>> >> > > > > >> > > > > in
> > > > >>>> >>> >> > > > > >> > > > > >> a
> > > > >>>> >>> >> > > > > >> > > > > >> > > > branch
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> and
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > out
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> of
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > master
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> until
> > > it
> > > > >>>> could
> > > > >>>> >>> pass
> > > > >>>> >>> >> > these
> > > > >>>> >>> >> > > > > >> > criteria.
> > > > >>>> >>> >> > > > > >> > > > The
> > > > >>>> >>> >> > > > > >> > > > > big
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > exemption
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > we've
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> had
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > to
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> this
> > > was
> > > > the
> > > > >>>> >>> >> > hbase-spark
> > > > >>>> >>> >> > > > > >> > > integration,
> > > > >>>> >>> >> > > > > >> > > > > where
> > > > >>>> >>> >> > > > > >> > > > > >> > we
> > > > >>>> >>> >> > > > > >> > > > > >> > > > all
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > agreed
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > it
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > could
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> land
> > in
> > > > >>>> master
> > > > >>>> >>> >> because
> > > > >>>> >>> >> > it
> > > > >>>> >>> >> > > > was
> > > > >>>> >>> >> > > > > >> very
> > > > >>>> >>> >> > > > > >> > > > well
> > > > >>>> >>> >> > > > > >> > > > > >> > > isolated
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > (the
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > slide
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > away
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > from
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > including
> > > > >>>> docs
> > > > >>>> >>> as a
> > > > >>>> >>> >> > > > > first-class
> > > > >>>> >>> >> > > > > >> > part
> > > > >>>> >>> >> > > > > >> > > > of
> > > > >>>> >>> >> > > > > >> > > > > >> > > building
> > > > >>>> >>> >> > > > > >> > > > > >> > > > up
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> that
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > integration
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > has
> > led
> > > > me
> > > > >>>> to
> > > > >>>> >>> doubt
> > > > >>>> >>> >> the
> > > > >>>> >>> >> > > > > wisdom
> > > > >>>> >>> >> > > > > >> of
> > > > >>>> >>> >> > > > > >> > > this
> > > > >>>> >>> >> > > > > >> > > > > >> > > decision).
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> We've
> > > > also
> > > > >>>> been
> > > > >>>> >>> >> > treating
> > > > >>>> >>> >> > > > > >> inclusion
> > > > >>>> >>> >> > > > > >> > > in
> > > > >>>> >>> >> > > > > >> > > > a
> > > > >>>> >>> >> > > > > >> > > > > >> > > "probably
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> will
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > be
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > released
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > to
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > > downstream"
> > > > >>>> >>> branches
> > > > >>>> >>> >> > as a
> > > > >>>> >>> >> > > > > >> higher
> > > > >>>> >>> >> > > > > >> > > bar,
> > > > >>>> >>> >> > > > > >> > > > > >> > requiring
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > *
> > don't
> > > > >>>> >>> moderately
> > > > >>>> >>> >> > impact
> > > > >>>> >>> >> > > > > >> > > performance
> > > > >>>> >>> >> > > > > >> > > > > when
> > > > >>>> >>> >> > > > > >> > > > > >> > the
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> feature
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > isn't
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> in
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > use
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > *
> > don't
> > > > >>>> severely
> > > > >>>> >>> >> impact
> > > > >>>> >>> >> > > > > >> > performance
> > > > >>>> >>> >> > > > > >> > > > when
> > > > >>>> >>> >> > > > > >> > > > > >> the
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > feature
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> is
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > in
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> use
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > *
> > > either
> > > > >>>> >>> >> default-to-on
> > > > >>>> >>> >> > or
> > > > >>>> >>> >> > > > > show
> > > > >>>> >>> >> > > > > >> > > enough
> > > > >>>> >>> >> > > > > >> > > > > >> demand
> > > > >>>> >>> >> > > > > >> > > > > >> > to
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> believe
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > a
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > non-trivial
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > number
> > > of
> > > > >>>> folks
> > > > >>>> >>> will
> > > > >>>> >>> >> > turn
> > > > >>>> >>> >> > > > the
> > > > >>>> >>> >> > > > > >> > > feature
> > > > >>>> >>> >> > > > > >> > > > on
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The
> > > above
> > > > >>>> has
> > > > >>>> >>> kept
> > > > >>>> >>> >> MOB
> > > > >>>> >>> >> > > and
> > > > >>>> >>> >> > > > > >> > > hbase-spark
> > > > >>>> >>> >> > > > > >> > > > > >> > > > integration
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> out
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > of
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > branch-1,
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > > presumably
> > > > >>>> while
> > > > >>>> >>> >> > they've
> > > > >>>> >>> >> > > > > >> "gotten
> > > > >>>> >>> >> > > > > >> > > more
> > > > >>>> >>> >> > > > > >> > > > > >> stable"
> > > > >>>> >>> >> > > > > >> > > > > >> > > in
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> master
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > from
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > the
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > odd
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > vendor
> > > > >>>> >>> inclusion.
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Are
> > we
> > > > >>>> going to
> > > > >>>> >>> >> have a
> > > > >>>> >>> >> > > 2.0
> > > > >>>> >>> >> > > > > >> release
> > > > >>>> >>> >> > > > > >> > > > > before
> > > > >>>> >>> >> > > > > >> > > > > >> the
> > > > >>>> >>> >> > > > > >> > > > > >> > > end
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > of
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> the
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> year?
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > We're
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > coming
> > > up
> > > > >>>> on 1.5
> > > > >>>> >>> >> years
> > > > >>>> >>> >> > > > since
> > > > >>>> >>> >> > > > > >> the
> > > > >>>> >>> >> > > > > >> > > > > release of
> > > > >>>> >>> >> > > > > >> > > > > >> > > > version
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> 1.0;
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> seems
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > like
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> it's
> > > > about
> > > > >>>> time,
> > > > >>>> >>> >> > though I
> > > > >>>> >>> >> > > > > >> haven't
> > > > >>>> >>> >> > > > > >> > > seen
> > > > >>>> >>> >> > > > > >> > > > > any
> > > > >>>> >>> >> > > > > >> > > > > >> > > > concrete
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > plans
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> this
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > year.
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > Presuming
> > > > >>>> we are
> > > > >>>> >>> >> going
> > > > >>>> >>> >> > to
> > > > >>>> >>> >> > > > > have
> > > > >>>> >>> >> > > > > >> one
> > > > >>>> >>> >> > > > > >> > > by
> > > > >>>> >>> >> > > > > >> > > > > the
> > > > >>>> >>> >> > > > > >> > > > > >> end
> > > > >>>> >>> >> > > > > >> > > > > >> > > of
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > the
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > year, it
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > seems a
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > bit
> > > > close to
> > > > >>>> >>> still
> > > > >>>> >>> >> be
> > > > >>>> >>> >> > > > adding
> > > > >>>> >>> >> > > > > in
> > > > >>>> >>> >> > > > > >> > > > > "features
> > > > >>>> >>> >> > > > > >> > > > > >> > that
> > > > >>>> >>> >> > > > > >> > > > > >> > > > need
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > maturing"
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > on
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > the
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > branch.
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The
> > > lack
> > > > of
> > > > >>>> a
> > > > >>>> >>> >> concrete
> > > > >>>> >>> >> > > plan
> > > > >>>> >>> >> > > > > for
> > > > >>>> >>> >> > > > > >> > 2.0
> > > > >>>> >>> >> > > > > >> > > > > keeps
> > > > >>>> >>> >> > > > > >> > > > > >> me
> > > > >>>> >>> >> > > > > >> > > > > >> > > from
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > considering
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > these
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > things
> > > > >>>> blocker
> > > > >>>> >>> at
> > > > >>>> >>> >> the
> > > > >>>> >>> >> > > > moment.
> > > > >>>> >>> >> > > > > >> But
> > > > >>>> >>> >> > > > > >> > I
> > > > >>>> >>> >> > > > > >> > > > know
> > > > >>>> >>> >> > > > > >> > > > > >> > first
> > > > >>>> >>> >> > > > > >> > > > > >> > > > hand
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> how
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > much
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > trouble
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> folks
> > > > have
> > > > >>>> had
> > > > >>>> >>> with
> > > > >>>> >>> >> > other
> > > > >>>> >>> >> > > > > >> features
> > > > >>>> >>> >> > > > > >> > > > that
> > > > >>>> >>> >> > > > > >> > > > > >> have
> > > > >>>> >>> >> > > > > >> > > > > >> > > gone
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> into
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > downstream
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > facing
> > > > >>>> releases
> > > > >>>> >>> >> without
> > > > >>>> >>> >> > > > > >> robustness
> > > > >>>> >>> >> > > > > >> > > > > checks
> > > > >>>> >>> >> > > > > >> > > > > >> > (i.e.
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > replication),
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > and
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > I'm
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > concerned
> > > > >>>> about
> > > > >>>> >>> what
> > > > >>>> >>> >> > > we're
> > > > >>>> >>> >> > > > > >> setting
> > > > >>>> >>> >> > > > > >> > > up
> > > > >>>> >>> >> > > > > >> > > > if
> > > > >>>> >>> >> > > > > >> > > > > >> 2.0
> > > > >>>> >>> >> > > > > >> > > > > >> > > goes
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > out
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > with
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> this
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > feature
> > > > in
> > > > >>>> its
> > > > >>>> >>> >> current
> > > > >>>> >>> >> > > > state.
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > --
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > Best
> > regards,
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >    - Andy
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > Problems
> > > > worthy of
> > > > >>>> >>> attack
> > > > >>>> >>> >> > prove
> > > > >>>> >>> >> > > > > their
> > > > >>>> >>> >> > > > > >> > > worth
> > > > >>>> >>> >> > > > > >> > > > by
> > > > >>>> >>> >> > > > > >> > > > > >> > > hitting
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> back. -
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > Piet
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > Hein
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > (via Tom
> > > White)
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > --
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > Best regards,
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >    - Andy
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > Problems
> worthy
> > > of
> > > > >>>> attack
> > > > >>>> >>> >> prove
> > > > >>>> >>> >> > > their
> > > > >>>> >>> >> > > > > >> worth
> > > > >>>> >>> >> > > > > >> > by
> > > > >>>> >>> >> > > > > >> > > > > >> hitting
> > > > >>>> >>> >> > > > > >> > > > > >> > > > back.
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > -
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > Piet
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> Hein
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > (via Tom
> White)
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >>
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >>
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > > >
> > > > >>>> >>> >> > > > > >> > > > > >> > >
> > > > >>>> >>> >> > > > > >> > > > > >> >
> > > > >>>> >>> >> > > > > >> > > > > >>
> > > > >>>> >>> >> > > > > >> > > > >
> > > > >>>> >>> >> > > > > >> > > > >
> > > > >>>> >>> >> > > > > >> > > > >
> > > > >>>> >>> >> > > > > >> > > > > --
> > > > >>>> >>> >> > > > > >> > > > > busbey
> > > > >>>> >>> >> > > > > >> > > > >
> > > > >>>> >>> >> > > > > >> > > >
> > > > >>>> >>> >> > > > > >> > >
> > > > >>>> >>> >> > > > > >> >
> > > > >>>> >>> >> > > > > >>
> > > > >>>> >>> >> > > > > >
> > > > >>>> >>> >> > > > > >
> > > > >>>> >>> >> > > > >
> > > > >>>> >>> >> > > >
> > > > >>>> >>> >> > >
> > > > >>>> >>> >> >
> > > > >>>> >>> >>
> > > > >>>> >>> >>
> > > > >>>> >>> >>
> > > > >>>> >>> >> --
> > > > >>>> >>> >>
> > > > >>>> >>> >> -- Appy
> > > > >>>> >>> >>
> > > > >>>> >>> >
> > > > >>>> >>> >
> > > > >>>> >>>
> > > > >>>> >>
> > > > >>>> >>
> > > > >>>> >
> > > > >>>>
> > > > >>>
> > > > >>>
> > > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > busbey
> > > >
> > > >
> > >
> >
> >
> >
> > --
> >
> > -- Appy
> >
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Devaraj Das <dd...@hortonworks.com>]

Stack and others, anything else on the patch? Merge to master now?
________________________________________
From: Ted Yu <yu...@gmail.com>
Sent: Thursday, October 13, 2016 3:56 PM
To: dev@hbase.apache.org
Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

It is not rebased - but all backup / restore functionality is there.

You can use mega patch v29 over HBASE-14123 which applies to master branch.

Cheers

On Thu, Oct 13, 2016 at 3:51 PM, Apekshit Sharma <ap...@cloudera.com> wrote:

> Is HBASE-7912 the feature branch?
> If yes, has it not been rebased to incorporate latest master changes yet?
>
> ~/apache/hbase  (master) → g ck apache/HBASE-7912
> warning: refname 'apache/HBASE-7912' is ambiguous.
> Switched to branch 'apache/HBASE-7912'
> ~/apache/hbase  (apache/HBASE-7912) → g log HEAD..apache/master | grep
> "^commit" | wc -l
>      917
>
>
> On Thu, Oct 13, 2016 at 7:01 AM, Ted Yu <yu...@gmail.com> wrote:
>
> > Once Stack's comments are addressed, we can change the master build to
> > using Maven 3.3.3
> >
> > I will log JIRA for modifying refguide on the upgrade.
> >
> > Cheers
> >
> > On Thu, Oct 13, 2016 at 6:42 AM, Sean Busbey <bu...@cloudera.com>
> wrote:
> >
> > > Are we requiring maven 3.3.z now? historically our required maven
> > > version has been 3.0.5. I don't have an objection to changing it, per
> > > se, but I'd like to make sure our docs and CI builds get updated
> > > properly.
> > >
> > > On Wed, Oct 12, 2016 at 4:03 PM, Stack <st...@duboce.net> wrote:
> > > > On Wed, Oct 12, 2016 at 1:58 PM, Stack <st...@duboce.net> wrote:
> > > >
> > > >> Fails again when I do this:
> > > >>
> > > >> $ mvn clean install -DskipTests assembly:single
> > > >> ....
> > > >>
> > > >>
> > > > Ok. Works if I use mvn 3.3.x.
> > > > St.Ack
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >>
> > > >> [INFO] ------------------------------------------------------------
> > > >> ------------
> > > >> [INFO] Building Apache HBase - Server 2.0.0-SNAPSHOT
> > > >> [INFO] ------------------------------------------------------------
> > > >> ------------
> > > >> Downloading: file:/home/stack/hbase.git/hbase-server/src/main/site/
> > > >> resources/repo/org/apache/hadoop/hadoop-distcp/2.7.1/
> > > >> hadoop-distcp-2.7.1.pom
> > > >> Exception in thread "pool-1-thread-1" ------------------------------
> > > >> ---------------------
> > > >> constituent[0]: file:/usr/share/maven/lib/aether-spi.jar
> > > >> constituent[1]: file:/usr/share/maven/lib/
> > > maven-settings-builder-3.x.jar
> > > >> constituent[2]: file:/usr/share/maven/lib/maven-artifact-3.x.jar
> > > >> constituent[3]: file:/usr/share/maven/lib/commons-httpclient.jar
> > > >> constituent[4]: file:/usr/share/maven/lib/wagon-provider-api.jar
> > > >> constituent[5]: file:/usr/share/maven/lib/maven-core-3.x.jar
> > > >> constituent[6]: file:/usr/share/maven/lib/aether-impl.jar
> > > >> constituent[7]: file:/usr/share/maven/lib/sisu-inject-bean.jar
> > > >> constituent[8]: file:/usr/share/maven/lib/sisu-inject-plexus.jar
> > > >> constituent[9]: file:/usr/share/maven/lib/wagon-file.jar
> > > >> constituent[10]: file:/usr/share/maven/lib/guava.jar
> > > >> constituent[11]: file:/usr/share/maven/lib/maven-compat-3.x.jar
> > > >> constituent[12]: file:/usr/share/maven/lib/maven-model-3.x.jar
> > > >> constituent[13]: file:/usr/share/maven/lib/plexus-interpolation.jar
> > > >> constituent[14]: file:/usr/share/maven/lib/commons-codec.jar
> > > >> constituent[15]: file:/usr/share/maven/lib/aether-util.jar
> > > >> constituent[16]: file:/usr/share/maven/lib/aether-api.jar
> > > >> constituent[17]: file:/usr/share/maven/lib/
> > maven-model-builder-3.x.jar
> > > >> constituent[18]: file:/usr/share/maven/lib/
> > > maven-repository-metadata-3.x.
> > > >> jar
> > > >> constituent[19]: file:/usr/share/maven/lib/maven-settings-3.x.jar
> > > >> constituent[20]: file:/usr/share/maven/lib/
> > > plexus-component-annotations.
> > > >> jar
> > > >> constituent[21]: file:/usr/share/maven/lib/commons-cli.jar
> > > >> constituent[22]: file:/usr/share/maven/lib/maven-embedder-3.x.jar
> > > >> constituent[23]: file:/usr/share/maven/lib/maven-plugin-api-3.x.jar
> > > >> constituent[24]: file:/usr/share/maven/lib/wagon-http-shaded.jar
> > > >> constituent[25]: file:/usr/share/maven/lib/commons-logging.jar
> > > >> constituent[26]: file:/usr/share/maven/lib/
> plexus-sec-dispatcher.jar
> > > >> constituent[27]: file:/usr/share/maven/lib/
> aether-connector-wagon.jar
> > > >> constituent[28]: file:/usr/share/maven/lib/plexus-utils.jar
> > > >> constituent[29]: file:/usr/share/maven/lib/sisu-guice.jar
> > > >> constituent[30]: file:/usr/share/maven/lib/plexus-cipher.jar
> > > >> constituent[31]: file:/usr/share/maven/lib/
> > > maven-aether-provider-3.x.jar
> > > >> ---------------------------------------------------
> > > >> java.lang.NoClassDefFoundError: org/apache/commons/lang/StringUtils
> > > >> at org.apache.maven.wagon.providers.file.FileWagon.
> > > resolveDestinationPath(
> > > >> FileWagon.java:206)
> > > >> at org.apache.maven.wagon.providers.file.FileWagon.
> > > >> resourceExists(FileWagon.java:265)
> > > >> at org.sonatype.aether.connector.wagon.WagonRepositoryConnector$
> > > >> GetTask.run(WagonRepositoryConnector.java:577)
> > > >> at org.sonatype.aether.util.concurrency.
> RunnableErrorForwarder$1.run(
> > > >> RunnableErrorForwarder.java:60)
> > > >> at java.util.concurrent.ThreadPoolExecutor.runWorker(
> > > >> ThreadPoolExecutor.java:1142)
> > > >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> > > >> ThreadPoolExecutor.java:617)
> > > >> at java.lang.Thread.run(Thread.java:745)
> > > >> Caused by: java.lang.ClassNotFoundException:
> org.apache.commons.lang.
> > > >> StringUtils
> > > >> at org.codehaus.plexus.classworlds.strategy.
> > > SelfFirstStrategy.loadClass(
> > > >> SelfFirstStrategy.java:50)
> > > >> at org.codehaus.plexus.classworlds.realm.ClassRealm.
> > > >> unsynchronizedLoadClass(ClassRealm.java:259)
> > > >> at org.codehaus.plexus.classworlds.realm.ClassRealm.
> > > >> loadClass(ClassRealm.java:235)
> > > >> at org.codehaus.plexus.classworlds.realm.ClassRealm.
> > > >> loadClass(ClassRealm.java:227)
> > > >> ... 7 more
> > > >> Exception in thread "main" java.lang.NoClassDefFoundError:
> > > >> org/apache/commons/lang/StringUtils
> > > >> at org.apache.maven.wagon.providers.file.FileWagon.
> > > resolveDestinationPath(
> > > >> FileWagon.java:206)
> > > >> at org.apache.maven.wagon.providers.file.FileWagon.
> > > >> resourceExists(FileWagon.java:265)
> > > >> at org.sonatype.aether.connector.wagon.WagonRepositoryConnector$
> > > >> GetTask.run(WagonRepositoryConnector.java:577)
> > > >> at org.sonatype.aether.util.concurrency.
> RunnableErrorForwarder$1.run(
> > > >> RunnableErrorForwarder.java:60)
> > > >> at java.util.concurrent.ThreadPoolExecutor.runWorker(
> > > >> ThreadPoolExecutor.java:1142)
> > > >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> > > >> ThreadPoolExecutor.java:617)
> > > >> at java.lang.Thread.run(Thread.java:745)
> > > >> Caused by: java.lang.ClassNotFoundException:
> org.apache.commons.lang.
> > > >> StringUtils
> > > >> at org.codehaus.plexus.classworlds.strategy.
> > > SelfFirstStrategy.loadClass(
> > > >> SelfFirstStrategy.java:50)
> > > >> at org.codehaus.plexus.classworlds.realm.ClassRealm.
> > > >> unsynchronizedLoadClass(ClassRealm.java:259)
> > > >> at org.codehaus.plexus.classworlds.realm.ClassRealm.
> > > >> loadClass(ClassRealm.java:235)
> > > >> at org.codehaus.plexus.classworlds.realm.ClassRealm.
> > > >> loadClass(ClassRealm.java:227)
> > > >>
> > > >>
> > > >>
> > > >> Does it work for you lot? This is branch checkout.
> > > >>
> > > >> I want assembly to work so I can distribute the build around a
> > cluster.
> > > >>
> > > >> Thanks,
> > > >> St.Ack.
> > > >>
> > > >>
> > > >>
> > > >> On Wed, Oct 12, 2016 at 1:52 PM, Stack <st...@duboce.net> wrote:
> > > >>
> > > >>> I did a super-clean and recheck out. Now it works. Sorry for the
> > noise.
> > > >>> St.Ack
> > > >>>
> > > >>> On Wed, Oct 12, 2016 at 1:10 PM, Vladimir Rodionov <
> > > >>> vladrodionov@gmail.com> wrote:
> > > >>>
> > > >>>> checked out HBASE-7912
> > > >>>>
> > > >>>> ran:
> > > >>>>
> > > >>>> mvn clean install -DskipTests
> > > >>>>
> > > >>>> successfully.
> > > >>>>
> > > >>>> -Vlad
> > > >>>>
> > > >>>> On Wed, Oct 12, 2016 at 1:01 PM, Vladimir Rodionov <
> > > >>>> vladrodionov@gmail.com>
> > > >>>> wrote:
> > > >>>>
> > > >>>> > I usually use:
> > > >>>> >
> > > >>>> > mvn clean install -DskipTests
> > > >>>> >
> > > >>>> > and it works.
> > > >>>> >
> > > >>>> > On Wed, Oct 12, 2016 at 1:01 PM, Vladimir Rodionov <
> > > >>>> vladrodionov@gmail.com
> > > >>>> > > wrote:
> > > >>>> >
> > > >>>> >> Michael,
> > > >>>> >>
> > > >>>> >> you can try master + latest patch on HBASE-14123 (v29). No need
> > to
> > > use
> > > >>>> >> HBASE-7912 branch. I will double check HBASE-7912.
> > > >>>> >>
> > > >>>> >> -Vlad
> > > >>>> >>
> > > >>>> >> On Wed, Oct 12, 2016 at 12:31 PM, Stack <st...@duboce.net>
> > wrote:
> > > >>>> >>
> > > >>>> >>> More info:
> > > >>>> >>>
> > > >>>> >>> stack@ve0524:~/hbase.git$ git checkout origin/HBASE-7912 -b
> > > 7912v2
> > > >>>> >>> Branch 7912v2 set up to track remote branch HBASE-7912 from
> > > origin.
> > > >>>> >>> Switched to a new branch '7912v2'
> > > >>>> >>> stack@ve0524:~/hbase.git$ java -version
> > > >>>> >>> java version "1.8.0_101"
> > > >>>> >>> Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
> > > >>>> >>> Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed
> mode)
> > > >>>> >>> stack@ve0524:~/hbase.git$ mvn clean install -DskipTests &>
> > > >>>> /tmp/out.txt
> > > >>>> >>>
> > > >>>> >>> ...
> > > >>>> >>>
> > > >>>> >>> St.Ack
> > > >>>> >>>
> > > >>>> >>>
> > > >>>> >>> On Wed, Oct 12, 2016 at 12:29 PM, Stack <st...@duboce.net>
> > wrote:
> > > >>>> >>>
> > > >>>> >>> > Interesting. When I try it fails w/ below:
> > > >>>> >>> >
> > > >>>> >>> > [INFO] 26 warnings
> > > >>>> >>> > 322 [INFO] ------------------------------
> > > >>>> >>> -------------------------------
> > > >>>> >>> > 323 [INFO] ------------------------------
> > > >>>> >>> -------------------------------
> > > >>>> >>> > 324 [ERROR] COMPILATION ERROR :
> > > >>>> >>> > 325 [INFO] ------------------------------
> > > >>>> >>> -------------------------------
> > > >>>> >>> > 326 [ERROR] /home/stack/hbase.git/hbase-
> > > common/src/main/java/org/
> > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/
> RowIndexCodecV2.j
> > > >>>> ava:[48,8]
> > > >>>> >>> > org.apache.hadoop.hbase.io.encoding.rowindexV2.
> > RowIndexCodecV2
> > > is
> > > >>>> not
> > > >>>> >>> > abstract and does not override abstract method
> > > createSeeker(org.ap
> > > >>>> >>> >  ache.hadoop.hbase.CellComparator,org.apache.
> hadoop.hbase.io.
> > > >>>> >>> encoding.HFileBlockDecodingContext)
> > > >>>> >>> > in org.apache.hadoop.hbase.io.encoding.DataBlockEncoder
> > > >>>> >>> > 327 [ERROR] /home/stack/hbase.git/hbase-
> > > common/src/main/java/org/
> > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/
> RowIndexCodecV2.j
> > > >>>> >>> ava:[143,3]
> > > >>>> >>> > method does not override or implement a method from a
> > supertype
> > > >>>> >>> > 328 [ERROR] /home/stack/hbase.git/hbase-
> > > common/src/main/java/org/
> > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/
> RowIndexCodecV2.j
> > > >>>> >>> ava:[147,29]
> > > >>>> >>> > incompatible types: java.nio.ByteBuffer cannot be converted
> to
> > > >>>> >>> > org.apache.hadoop.hbase.nio.ByteBuff
> > > >>>> >>> > 329 [ERROR] /home/stack/hbase.git/hbase-
> > > common/src/main/java/org/
> > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/
> RowIndexCodecV2.j
> > > >>>> >>> ava:[148,33]
> > > >>>> >>> > cannot find symbol
> > > >>>> >>> > 330   symbol:   method getKeyDeepCopy()
> > > >>>> >>> > 331   location: variable seeker of type
> > > org.apache.hadoop.hbase.io
> > > >>>> .
> > > >>>> >>> > encoding.DataBlockEncoder.EncodedSeeker
> > > >>>> >>> > 332 [ERROR] /home/stack/hbase.git/hbase-
> > > common/src/main/java/org/
> > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/
> RowIndexCodecV2.j
> > > >>>> >>> ava:[153,3]
> > > >>>> >>> > method does not override or implement a method from a
> > supertype
> > > >>>> >>> > 333 [ERROR] /home/stack/hbase.git/hbase-
> > > common/src/main/java/org/
> > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV1/
> RowIndexCodecV1.j
> > > >>>> ava:[45,8]
> > > >>>> >>> > org.apache.hadoop.hbase.io.encoding.rowindexV1.
> > RowIndexCodecV1
> > > is
> > > >>>> not
> > > >>>> >>> > abstract and does not override abstract method
> > > createSeeker(org.ap
> > > >>>> >>> >  ache.hadoop.hbase.CellComparator,org.apache.
> hadoop.hbase.io.
> > > >>>> >>> encoding.HFileBlockDecodingContext)
> > > >>>> >>> > in org.apache.hadoop.hbase.io.encoding.DataBlockEncoder
> > > >>>> >>> > 334 [ERROR] /home/stack/hbase.git/hbase-
> > > common/src/main/java/org/
> > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV1/
> RowIndexCodecV1.j
> > > >>>> >>> ava:[145,3]
> > > >>>> >>> > method does not override or implement a method from a
> > supertype
> > > >>>> >>> > 335 [ERROR] /home/stack/hbase.git/hbase-
> > > common/src/main/java/org/
> > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV1/
> RowIndexCodecV1.j
> > > >>>> >>> ava:[158,3]
> > > >>>> >>> > method does not override or implement a method from a
> > supertype
> > > >>>> >>> > 336 [ERROR] /home/stack/hbase.git/hbase-
> > > common/src/main/java/org/
> > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/
> RowIndexSeekerV2.
> > > >>>> >>> java:[46,8]
> > > >>>> >>> > org.apache.hadoop.hbase.io.encoding.rowindexV2.
> > RowIndexSeekerV2
> > > >>>> is not
> > > >>>> >>> > abstract and does not override abstract method
> > compareKey(org.ap
> > > >>>> >>> >  ache.hadoop.hbase.CellComparator,org.apache.
> > hadoop.hbase.Cell)
> > > in
> > > >>>> >>> > org.apache.hadoop.hbase.io.encoding.DataBlockEncoder.
> > > EncodedSeeker
> > > >>>> >>> > 337 [ERROR] /home/stack/hbase.git/hbase-
> > > common/src/main/java/org/
> > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/
> RowIndexSeekerV2.
> > > >>>> >>> java:[79,3]
> > > >>>> >>> > method does not override or implement a method from a
> > supertype
> > > >>>> >>> > 338 [ERROR] /home/stack/hbase.git/hbase-
> > > common/src/main/java/org/
> > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/
> RowIndexSeekerV2.
> > > >>>> >>> java:[117,3]
> > > >>>> >>> > method does not override or implement a method from a
> > supertype
> > > >>>> >>> > 339 [ERROR] /home/stack/hbase.git/hbase-
> > > common/src/main/java/org/
> > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/
> RowIndexSeekerV2.
> > > >>>> >>> java:[190,3]
> > > >>>> >>> > method does not override or implement a method from a
> > supertype
> > > >>>> >>> > 340 [ERROR] /home/stack/hbase.git/hbase-
> > > common/src/main/java/org/
> > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/
> RowIndexSeekerV2.
> > > >>>> >>> java:[214,3]
> > > >>>> >>> > method does not override or implement a method from a
> > supertype
> > > >>>> >>> > 341 [ERROR] /home/stack/hbase.git/hbase-
> > > common/src/main/java/org/
> > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/
> RowIndexSeekerV2.
> > > >>>> >>> java:[349,3]
> > > >>>> >>> > method does not override or implement a method from a
> > supertype
> > > >>>> >>> > 342 [ERROR] /home/stack/hbase.git/hbase-
> > > common/src/main/java/org/
> > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/
> RowIndexSeekerV2.
> > > >>>> >>> java:[355,3]
> > > >>>> >>> > method does not override or implement a method from a
> > supertype
> > > >>>> >>> > 343 [ERROR] /home/stack/hbase.git/hbase-
> > > common/src/main/java/org/
> > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/
> RowIndexSeekerV2.
> > > >>>> >>> java:[421,36]
> > > >>>> >>> > no suitable method found for uncompressTags(java.nio.
> > > >>>> >>> > ByteBuffer,byte[],int,int)
> > > >>>> >>> > 344     method org.apache.hadoop.hbase.io.
> > > TagCompressionContext.
> > > >>>> >>> > uncompressTags(java.io.InputStream,byte[],int,int) is not
> > > >>>> applicable
> > > >>>> >>> > 345       (argument mismatch; java.nio.ByteBuffer cannot be
> > > >>>> converted
> > > >>>> >>> to
> > > >>>> >>> > java.io.InputStream)
> > > >>>> >>> > 346     method org.apache.hadoop.hbase.io.
> > > TagCompressionContext.
> > > >>>> >>> > uncompressTags(org.apache.hadoop.hbase.nio.ByteBuff,
> > > byte[],int,int)
> > > >>>> is
> > > >>>> >>> > not applicable
> > > >>>> >>> > 347       (argument mismatch; java.nio.ByteBuffer cannot be
> > > >>>> converted
> > > >>>> >>> to
> > > >>>> >>> > org.apache.hadoop.hbase.nio.ByteBuff)
> > > >>>> >>> >
> > > >>>> >>> > ....
> > > >>>> >>> >
> > > >>>> >>> > St.Ack
> > > >>>> >>> >
> > > >>>> >>> > On Tue, Oct 11, 2016 at 5:12 PM, Apekshit Sharma <
> > > >>>> appy@cloudera.com>
> > > >>>> >>> > wrote:
> > > >>>> >>> >
> > > >>>> >>> >> @stack, it compiled for me.
> > > >>>> >>> >>
> > > >>>> >>> >> Also tried few commands, and have to say, it's well
> designed
> > > from
> > > >>>> user
> > > >>>> >>> >> commands perspective.
> > > >>>> >>> >>
> > > >>>> >>> >> On Tue, Oct 11, 2016 at 5:08 PM, Stack <st...@duboce.net>
> > > wrote:
> > > >>>> >>> >>
> > > >>>> >>> >> > On Tue, Oct 11, 2016 at 1:50 PM, Vladimir Rodionov <
> > > >>>> >>> >> vladrodionov@gmail.com
> > > >>>> >>> >> > >
> > > >>>> >>> >> > wrote:
> > > >>>> >>> >> >
> > > >>>> >>> >> > > Michael,
> > > >>>> >>> >> > >
> > > >>>> >>> >> > > Its in HBASE-7912
> > > >>>> >>> >> > >
> > > >>>> >>> >> > > This is tip of git log:
> > > >>>> >>> >> > >
> > > >>>> >>> >> > > commit a072f6f49a26a7259ff2aaef6cb56d85eb592482
> > > >>>> >>> >> > > Author: Frank Welsch <fw...@jps.net>
> > > >>>> >>> >> > > Date:   Fri Sep 23 18:00:42 2016 -0400
> > > >>>> >>> >> > >
> > > >>>> >>> >> > >     HBASE-16574 Book updates for backup and restore
> > > >>>> >>> >> > >
> > > >>>> >>> >> > > commit b14e2ab1c24e65ff88dd4c579acf83cb4ed0605e
> > > >>>> >>> >> > > Author: tedyu <yu...@gmail.com>
> > > >>>> >>> >> > > Date:   Wed Oct 5 16:29:40 2016 -0700
> > > >>>> >>> >> > >
> > > >>>> >>> >> > >     HBASE-16727 Backup refactoring: remove MR
> > dependencies
> > > >>>> from
> > > >>>> >>> >> HMaster
> > > >>>> >>> >> > > (Vladimir Rodionov)
> > > >>>> >>> >> > >
> > > >>>> >>> >> > >
> > > >>>> >>> >> > Thanks. I have that. I tried it and it doesn't compile
> for
> > > me.
> > > >>>> Does
> > > >>>> >>> it
> > > >>>> >>> >> > compile for you?
> > > >>>> >>> >> > Thanks,
> > > >>>> >>> >> > M
> > > >>>> >>> >> >
> > > >>>> >>> >> >
> > > >>>> >>> >> >
> > > >>>> >>> >> > > -Vlad
> > > >>>> >>> >> > >
> > > >>>> >>> >> > > On Tue, Oct 11, 2016 at 1:44 PM, Stack <
> stack@duboce.net
> > >
> > > >>>> wrote:
> > > >>>> >>> >> > >
> > > >>>> >>> >> > > > Which branch do I check out to try it? HBASE-7912 is
> > not
> > > >>>> it. I
> > > >>>> >>> don't
> > > >>>> >>> >> > see
> > > >>>> >>> >> > > an
> > > >>>> >>> >> > > > HBASE-16727...
> > > >>>> >>> >> > > > Thanks,
> > > >>>> >>> >> > > > M
> > > >>>> >>> >> > > >
> > > >>>> >>> >> > > >
> > > >>>> >>> >> > > > On Tue, Oct 11, 2016 at 11:41 AM, Vladimir Rodionov <
> > > >>>> >>> >> > > > vladrodionov@gmail.com>
> > > >>>> >>> >> > > > wrote:
> > > >>>> >>> >> > > >
> > > >>>> >>> >> > > > > The last patch is on review board:
> > > >>>> >>> >> > > > > https://reviews.apache.org/r/52748
> > > >>>> >>> >> > > > >
> > > >>>> >>> >> > > > > On Mon, Oct 10, 2016 at 3:52 PM, Vladimir Rodionov
> <
> > > >>>> >>> >> > > > vladrodionov@gmail.com
> > > >>>> >>> >> > > > > >
> > > >>>> >>> >> > > > > wrote:
> > > >>>> >>> >> > > > >
> > > >>>> >>> >> > > > > > >> How hard to put in an hbase-backup module?
> > > >>>> hbase-server
> > > >>>> >>> is
> > > >>>> >>> >> fat
> > > >>>> >>> >> > > > enough
> > > >>>> >>> >> > > > > > >> already. Could be done as a follow-up.
> > > >>>> >>> >> > > > > >
> > > >>>> >>> >> > > > > > https://issues.apache.org/
> jira/browse/HBASE-16727?
> > > >>>> >>> >> > > > > > focusedCommentId=15531237&
> page=com.atlassian.jira.
> > > >>>> >>> >> > > > > > plugin.system.issuetabpanels:c
> > > >>>> >>> omment-tabpanel#comment-15531237
> > > >>>> >>> >> > > > > >
> > > >>>> >>> >> > > > > > Can we do merge first? Then we can discuss
> separate
> > > >>>> module.
> > > >>>> >>> >> > > > > >
> > > >>>> >>> >> > > > > > -Vlad
> > > >>>> >>> >> > > > > >
> > > >>>> >>> >> > > > > > On Mon, Oct 10, 2016 at 3:44 PM, Ted Yu <
> > > >>>> >>> yuzhihong@gmail.com>
> > > >>>> >>> >> > wrote:
> > > >>>> >>> >> > > > > >
> > > >>>> >>> >> > > > > >> Looks like the first quote was cut off.
> > > >>>> >>> >> > > > > >> The original sentence was:
> > > >>>> >>> >> > > > > >>
> > > >>>> >>> >> > > > > >> bq. no mapreduce job launched from master or
> > region
> > > >>>> server.
> > > >>>> >>> >> > > > > >>
> > > >>>> >>> >> > > > > >> mapreduce job is launched from the node where
> > > command
> > > >>>> line
> > > >>>> >>> >> tool is
> > > >>>> >>> >> > > > run.
> > > >>>> >>> >> > > > > >>
> > > >>>> >>> >> > > > > >> On Mon, Oct 10, 2016 at 3:38 PM, Stack <
> > > >>>> stack@duboce.net>
> > > >>>> >>> >> wrote:
> > > >>>> >>> >> > > > > >>
> > > >>>> >>> >> > > > > >> > bq. launched from master or region server.
> > > >>>> >>> >> > > > > >> >
> > > >>>> >>> >> > > > > >> > What does this mean please? Has to be run from
> > > >>>> Master or
> > > >>>> >>> >> > > > RegionServer?
> > > >>>> >>> >> > > > > >> Can
> > > >>>> >>> >> > > > > >> > it be run from another node altogether?
> > > >>>> >>> >> > > > > >> >
> > > >>>> >>> >> > > > > >> > On Mon, Oct 10, 2016 at 1:44 PM, Vladimir
> > > Rodionov <
> > > >>>> >>> >> > > > > >> vladrodionov@gmail.com
> > > >>>> >>> >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > wrote:
> > > >>>> >>> >> > > > > >> >
> > > >>>> >>> >> > > > > >> > > >> mapreduce dependency has been moved to
> > client
> > > >>>> side
> > > >>>> >>> - no
> > > >>>> >>> >> > > > mapreduce
> > > >>>> >>> >> > > > > >> job
> > > >>>> >>> >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > > 1. We have no code in the client module
> > anymore,
> > > >>>> due to
> > > >>>> >>> >> > > dependency
> > > >>>> >>> >> > > > > on
> > > >>>> >>> >> > > > > >> > > internal server API (HFile and WAL access).
> > > >>>> >>> >> > > > > >> > > 2. Backup/ restore are client - driven
> > > operations,
> > > >>>> but
> > > >>>> >>> all
> > > >>>> >>> >> the
> > > >>>> >>> >> > > > code
> > > >>>> >>> >> > > > > >> > resides
> > > >>>> >>> >> > > > > >> > > in the server module
> > > >>>> >>> >> > > > > >> > >
> > > >>>> >>> >> > > > > >> >
> > > >>>> >>> >> > > > > >> > How hard to put in an hbase-backup module?
> > > >>>> hbase-server
> > > >>>> >>> is
> > > >>>> >>> >> fat
> > > >>>> >>> >> > > > enough
> > > >>>> >>> >> > > > > >> > already. Could be done as a follow-up.
> > > >>>> >>> >> > > > > >> >
> > > >>>> >>> >> > > > > >> > Thanks,
> > > >>>> >>> >> > > > > >> > St.Ack
> > > >>>> >>> >> > > > > >> >
> > > >>>> >>> >> > > > > >> >
> > > >>>> >>> >> > > > > >> >
> > > >>>> >>> >> > > > > >> > > 3. No MR in Master, no procedure - driven
> > > >>>> execution.
> > > >>>> >>> >> > > > > >> > > 4. Old good MR from command-line.
> > > >>>> >>> >> > > > > >> > > 5. Security was simplified and now only
> > > super-user
> > > >>>> is
> > > >>>> >>> >> allowed
> > > >>>> >>> >> > to
> > > >>>> >>> >> > > > run
> > > >>>> >>> >> > > > > >> > > backup/restores.
> > > >>>> >>> >> > > > > >> > > 6. HBase Backup API was gone due to 1. Now
> > only
> > > >>>> >>> >> command-line
> > > >>>> >>> >> > > > access
> > > >>>> >>> >> > > > > to
> > > >>>> >>> >> > > > > >> > > backup tools.
> > > >>>> >>> >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > > These consequences of refactoring has been
> > > >>>> discussed in
> > > >>>> >>> >> > > > HBASE-16727.
> > > >>>> >>> >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > > -Vlad
> > > >>>> >>> >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > > On Mon, Oct 10, 2016 at 1:31 PM, Ted Yu <
> > > >>>> >>> >> yuzhihong@gmail.com>
> > > >>>> >>> >> > > > > wrote:
> > > >>>> >>> >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > > > Reviving this thread.
> > > >>>> >>> >> > > > > >> > > >
> > > >>>> >>> >> > > > > >> > > > The following has taken place:
> > > >>>> >>> >> > > > > >> > > >
> > > >>>> >>> >> > > > > >> > > > mapreduce dependency has been moved to
> > client
> > > >>>> side -
> > > >>>> >>> no
> > > >>>> >>> >> > > > mapreduce
> > > >>>> >>> >> > > > > >> job
> > > >>>> >>> >> > > > > >> > > > launched from master or region server.
> > > >>>> >>> >> > > > > >> > > > document patch (HBASE-16574) has been
> > > integrated.
> > > >>>> >>> >> > > > > >> > > > Updated mega patch has been attached to
> > > >>>> HBASE-14123:
> > > >>>> >>> this
> > > >>>> >>> >> > > covers
> > > >>>> >>> >> > > > > the
> > > >>>> >>> >> > > > > >> > > > refactor in #1 above and the protobuf 3
> > merge.
> > > >>>> >>> >> > > > > >> > > >
> > > >>>> >>> >> > > > > >> > > > If community has more feedback on the
> merge
> > > >>>> >>> proposal, I
> > > >>>> >>> >> > would
> > > >>>> >>> >> > > > love
> > > >>>> >>> >> > > > > >> to
> > > >>>> >>> >> > > > > >> > > hear
> > > >>>> >>> >> > > > > >> > > > it.
> > > >>>> >>> >> > > > > >> > > >
> > > >>>> >>> >> > > > > >> > > > Thanks
> > > >>>> >>> >> > > > > >> > > >
> > > >>>> >>> >> > > > > >> > > > On Thu, Sep 22, 2016 at 10:31 AM, Sean
> > Busbey
> > > <
> > > >>>> >>> >> > > > > busbey@cloudera.com>
> > > >>>> >>> >> > > > > >> > > wrote:
> > > >>>> >>> >> > > > > >> > > >
> > > >>>> >>> >> > > > > >> > > > > I'd like to see the docs proposed on
> > > >>>> HBASE-16574
> > > >>>> >>> >> > integrated
> > > >>>> >>> >> > > > into
> > > >>>> >>> >> > > > > >> our
> > > >>>> >>> >> > > > > >> > > > > project's documentation prior to merge.
> > > >>>> >>> >> > > > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > > On Thu, Sep 22, 2016 at 9:02 AM, Ted Yu
> <
> > > >>>> >>> >> > > yuzhihong@gmail.com>
> > > >>>> >>> >> > > > > >> wrote:
> > > >>>> >>> >> > > > > >> > > > > > This feature can be marked
> experimental
> > > due
> > > >>>> to
> > > >>>> >>> some
> > > >>>> >>> >> > > > > limitations
> > > >>>> >>> >> > > > > >> > such
> > > >>>> >>> >> > > > > >> > > as
> > > >>>> >>> >> > > > > >> > > > > > security.
> > > >>>> >>> >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > > Your previous round of comments have
> > been
> > > >>>> >>> addressed.
> > > >>>> >>> >> > > > > >> > > > > > Command line tool has gone through:
> > > >>>> >>> >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > > HBASE-16620 Fix backup command-line
> tool
> > > >>>> >>> usability
> > > >>>> >>> >> > issues
> > > >>>> >>> >> > > > > >> > > > > > HBASE-16655 hbase backup describe with
> > > >>>> incorrect
> > > >>>> >>> >> backup
> > > >>>> >>> >> > id
> > > >>>> >>> >> > > > > >> results
> > > >>>> >>> >> > > > > >> > in
> > > >>>> >>> >> > > > > >> > > > NPE
> > > >>>> >>> >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > > The updated doc has been attached to
> > > >>>> HBASE-16574.
> > > >>>> >>> >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > > Cheers
> > > >>>> >>> >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > > On Thu, Sep 22, 2016 at 8:53 AM,
> Stack <
> > > >>>> >>> >> > stack@duboce.net>
> > > >>>> >>> >> > > > > >> wrote:
> > > >>>> >>> >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> On Wed, Sep 21, 2016 at 7:43 AM, Ted
> > Yu <
> > > >>>> >>> >> > > > yuzhihong@gmail.com
> > > >>>> >>> >> > > > > >
> > > >>>> >>> >> > > > > >> > > wrote:
> > > >>>> >>> >> > > > > >> > > > > >>
> > > >>>> >>> >> > > > > >> > > > > >> > Are there more (review) comments ?
> > > >>>> >>> >> > > > > >> > > > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> Are outstanding comments addressed?
> > > >>>> >>> >> > > > > >> > > > > >>
> > > >>>> >>> >> > > > > >> > > > > >> I don't see answer to my 'is this
> > > >>>> >>> experimental/will
> > > >>>> >>> >> it
> > > >>>> >>> >> > be
> > > >>>> >>> >> > > > > >> marked
> > > >>>> >>> >> > > > > >> > > > > >> experimental' question.
> > > >>>> >>> >> > > > > >> > > > > >>
> > > >>>> >>> >> > > > > >> > > > > >> I ran into some issues trying to use
> > the
> > > >>>> >>> feature and
> > > >>>> >>> >> > > > > suggested
> > > >>>> >>> >> > > > > >> > that
> > > >>>> >>> >> > > > > >> > > a
> > > >>>> >>> >> > > > > >> > > > > >> feature likes this needs polish else
> > > it'll
> > > >>>> just
> > > >>>> >>> rot,
> > > >>>> >>> >> > > > unused.
> > > >>>> >>> >> > > > > >> Has
> > > >>>> >>> >> > > > > >> > > > polish
> > > >>>> >>> >> > > > > >> > > > > >> been applied? All ready for another
> > > 'user'
> > > >>>> test?
> > > >>>> >>> >> > Suggest
> > > >>>> >>> >> > > > that
> > > >>>> >>> >> > > > > >> you
> > > >>>> >>> >> > > > > >> > > > update
> > > >>>> >>> >> > > > > >> > > > > >> here going forward for the benefit of
> > > those
> > > >>>> >>> trying
> > > >>>> >>> >> to
> > > >>>> >>> >> > > > follow
> > > >>>> >>> >> > > > > >> along
> > > >>>> >>> >> > > > > >> > > and
> > > >>>> >>> >> > > > > >> > > > > who
> > > >>>> >>> >> > > > > >> > > > > >> are not watching JIRA change fly-by.
> > > >>>> >>> >> > > > > >> > > > > >>
> > > >>>> >>> >> > > > > >> > > > > >> It looks like doc got a revision -- I
> > > have
> > > >>>> to
> > > >>>> >>> check
> > > >>>> >>> >> --
> > > >>>> >>> >> > to
> > > >>>> >>> >> > > > > take
> > > >>>> >>> >> > > > > >> on
> > > >>>> >>> >> > > > > >> > > > > >> suggestion made above but again,
> > suggest,
> > > >>>> that
> > > >>>> >>> this
> > > >>>> >>> >> > > thread
> > > >>>> >>> >> > > > > gets
> > > >>>> >>> >> > > > > >> > > > updated.
> > > >>>> >>> >> > > > > >> > > > > >>
> > > >>>> >>> >> > > > > >> > > > > >> Thanks,
> > > >>>> >>> >> > > > > >> > > > > >> St.Ack
> > > >>>> >>> >> > > > > >> > > > > >>
> > > >>>> >>> >> > > > > >> > > > > >>
> > > >>>> >>> >> > > > > >> > > > > >>
> > > >>>> >>> >> > > > > >> > > > > >> > Thanks
> > > >>>> >>> >> > > > > >> > > > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > On Tue, Sep 20, 2016 at 10:02 AM,
> > > Devaraj
> > > >>>> Das
> > > >>>> >>> <
> > > >>>> >>> >> > > > > >> > > ddas@hortonworks.com
> > > >>>> >>> >> > > > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > wrote:
> > > >>>> >>> >> > > > > >> > > > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > > Just reviving this thread. Thanks
> > > Sean,
> > > >>>> >>> Stack,
> > > >>>> >>> >> > Dima,
> > > >>>> >>> >> > > > and
> > > >>>> >>> >> > > > > >> > others
> > > >>>> >>> >> > > > > >> > > > for
> > > >>>> >>> >> > > > > >> > > > > the
> > > >>>> >>> >> > > > > >> > > > > >> > > thorough reviews and testing.
> > Thanks
> > > >>>> Ted and
> > > >>>> >>> >> Vlad
> > > >>>> >>> >> > for
> > > >>>> >>> >> > > > > >> taking
> > > >>>> >>> >> > > > > >> > > care
> > > >>>> >>> >> > > > > >> > > > of
> > > >>>> >>> >> > > > > >> > > > > >> the
> > > >>>> >>> >> > > > > >> > > > > >> > > feedback. Are we all good to do
> the
> > > >>>> merge
> > > >>>> >>> now?
> > > >>>> >>> >> > Rather
> > > >>>> >>> >> > > > do
> > > >>>> >>> >> > > > > >> > sooner
> > > >>>> >>> >> > > > > >> > > > than
> > > >>>> >>> >> > > > > >> > > > > >> > later.
> > > >>>> >>> >> > > > > >> > > > > >> > > ______________________________
> > > >>>> __________
> > > >>>> >>> >> > > > > >> > > > > >> > > From: saint.ack@gmail.com <
> > > >>>> >>> saint.ack@gmail.com>
> > > >>>> >>> >> on
> > > >>>> >>> >> > > > > behalf
> > > >>>> >>> >> > > > > >> of
> > > >>>> >>> >> > > > > >> > > > Stack
> > > >>>> >>> >> > > > > >> > > > > <
> > > >>>> >>> >> > > > > >> > > > > >> > > stack@duboce.net>
> > > >>>> >>> >> > > > > >> > > > > >> > > Sent: Monday, September 12, 2016
> > > 1:18 PM
> > > >>>> >>> >> > > > > >> > > > > >> > > To: HBase Dev List
> > > >>>> >>> >> > > > > >> > > > > >> > > Subject: Re: [DISCUSSION] Merge
> > > Backup /
> > > >>>> >>> >> Restore -
> > > >>>> >>> >> > > > Branch
> > > >>>> >>> >> > > > > >> > > > HBASE-7912
> > > >>>> >>> >> > > > > >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> > > On Mon, Sep 12, 2016 at 12:19 PM,
> > Ted
> > > >>>> Yu <
> > > >>>> >>> >> > > > > >> yuzhihong@gmail.com
> > > >>>> >>> >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > > > > wrote:
> > > >>>> >>> >> > > > > >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > Mega patch (rev 18) is on
> > > HBASE-14123.
> > > >>>> >>> >> > > > > >> > > > > >> > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > Please comment on HBASE-14123
> on
> > > how
> > > >>>> you
> > > >>>> >>> want
> > > >>>> >>> >> to
> > > >>>> >>> >> > > > > review.
> > > >>>> >>> >> > > > > >> > > > > >> > > >
> > > >>>> >>> >> > > > > >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> > > Yeah. That was my lost tab. Last
> rb
> > > was
> > > >>>> 6
> > > >>>> >>> months
> > > >>>> >>> >> > ago.
> > > >>>> >>> >> > > > > >> Suggest
> > > >>>> >>> >> > > > > >> > > > > updating
> > > >>>> >>> >> > > > > >> > > > > >> > it.
> > > >>>> >>> >> > > > > >> > > > > >> > > RB is pretty good for review.
> Patch
> > > is
> > > >>>> only
> > > >>>> >>> >> 1.5M so
> > > >>>> >>> >> > > > > should
> > > >>>> >>> >> > > > > >> be
> > > >>>> >>> >> > > > > >> > > > fine.
> > > >>>> >>> >> > > > > >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> > > St.Ack
> > > >>>> >>> >> > > > > >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > Thanks
> > > >>>> >>> >> > > > > >> > > > > >> > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > On Mon, Sep 12, 2016 at 12:15
> PM,
> > > >>>> Stack <
> > > >>>> >>> >> > > > > >> stack@duboce.net>
> > > >>>> >>> >> > > > > >> > > > wrote:
> > > >>>> >>> >> > > > > >> > > > > >> > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > On review of the 'patch', do
> I
> > > just
> > > >>>> >>> compare
> > > >>>> >>> >> the
> > > >>>> >>> >> > > > > branch
> > > >>>> >>> >> > > > > >> to
> > > >>>> >>> >> > > > > >> > > > > master or
> > > >>>> >>> >> > > > > >> > > > > >> > is
> > > >>>> >>> >> > > > > >> > > > > >> > > > > there a megapatch posted
> > > somewhere
> > > >>>> (I
> > > >>>> >>> think
> > > >>>> >>> >> I
> > > >>>> >>> >> > saw
> > > >>>> >>> >> > > > one
> > > >>>> >>> >> > > > > >> but
> > > >>>> >>> >> > > > > >> > it
> > > >>>> >>> >> > > > > >> > > > > seemed
> > > >>>> >>> >> > > > > >> > > > > >> > > stale
> > > >>>> >>> >> > > > > >> > > > > >> > > > > and then I 'lost' the tab).
> > Sorry
> > > >>>> for
> > > >>>> >>> dumb
> > > >>>> >>> >> > > > question.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > St.Ack
> > > >>>> >>> >> > > > > >> > > > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > On Mon, Sep 12, 2016 at 12:01
> > PM,
> > > >>>> Stack
> > > >>>> >>> <
> > > >>>> >>> >> > > > > >> stack@duboce.net
> > > >>>> >>> >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > > > > wrote:
> > > >>>> >>> >> > > > > >> > > > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > Late to the game. A few
> > > comments
> > > >>>> after
> > > >>>> >>> >> > > rereading
> > > >>>> >>> >> > > > > this
> > > >>>> >>> >> > > > > >> > > thread
> > > >>>> >>> >> > > > > >> > > > > as a
> > > >>>> >>> >> > > > > >> > > > > >> > > > 'user'.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > + Before merge, a
> user-facing
> > > >>>> feature
> > > >>>> >>> like
> > > >>>> >>> >> > this
> > > >>>> >>> >> > > > > >> should
> > > >>>> >>> >> > > > > >> > > work
> > > >>>> >>> >> > > > > >> > > > > (If
> > > >>>> >>> >> > > > > >> > > > > >> > this
> > > >>>> >>> >> > > > > >> > > > > >> > > is
> > > >>>> >>> >> > > > > >> > > > > >> > > > > "higher-bar
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > for new features", bring it
> > on
> > > --
> > > >>>> >>> smile).
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > + As a user, I tried the
> > branch
> > > >>>> with
> > > >>>> >>> tools
> > > >>>> >>> >> > > after
> > > >>>> >>> >> > > > > >> > reviewing
> > > >>>> >>> >> > > > > >> > > > the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > just-posted
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > doc. I had an 'interesting'
> > > >>>> experience
> > > >>>> >>> >> (left
> > > >>>> >>> >> > > > > >> comments up
> > > >>>> >>> >> > > > > >> > > on
> > > >>>> >>> >> > > > > >> > > > > >> > issue). I
> > > >>>> >>> >> > > > > >> > > > > >> > > > > think
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > the tooling/doc. important
> to
> > > get
> > > >>>> >>> right.
> > > >>>> >>> >> If
> > > >>>> >>> >> > it
> > > >>>> >>> >> > > > > breaks
> > > >>>> >>> >> > > > > >> > > easily
> > > >>>> >>> >> > > > > >> > > > > or
> > > >>>> >>> >> > > > > >> > > > > >> is
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > inconsistent (or lacks
> > > 'polish'),
> > > >>>> >>> >> operators
> > > >>>> >>> >> > > will
> > > >>>> >>> >> > > > > >> judge
> > > >>>> >>> >> > > > > >> > the
> > > >>>> >>> >> > > > > >> > > > > whole
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > backup/restore tooling
> chain
> > as
> > > >>>> not
> > > >>>> >>> >> > trustworthy
> > > >>>> >>> >> > > > and
> > > >>>> >>> >> > > > > >> > > abandon
> > > >>>> >>> >> > > > > >> > > > > it.
> > > >>>> >>> >> > > > > >> > > > > >> > Lets
> > > >>>> >>> >> > > > > >> > > > > >> > > > not
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > have this happen to this
> > > feature.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > + Matteo's suggestion
> (with a
> > > >>>> helpful
> > > >>>> >>> >> starter
> > > >>>> >>> >> > > > list)
> > > >>>> >>> >> > > > > >> that
> > > >>>> >>> >> > > > > >> > > > there
> > > >>>> >>> >> > > > > >> > > > > >> > needs
> > > >>>> >>> >> > > > > >> > > > > >> > > to
> > > >>>> >>> >> > > > > >> > > > > >> > > > > be
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > explicit qualification on
> > what
> > > is
> > > >>>> >>> actually
> > > >>>> >>> >> > > being
> > > >>>> >>> >> > > > > >> > delivered
> > > >>>> >>> >> > > > > >> > > > --
> > > >>>> >>> >> > > > > >> > > > > >> > > > including a
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > listing of limitations
> (some
> > > look
> > > >>>> >>> serious
> > > >>>> >>> >> > such
> > > >>>> >>> >> > > as
> > > >>>> >>> >> > > > > >> data
> > > >>>> >>> >> > > > > >> > > bleed
> > > >>>> >>> >> > > > > >> > > > > from
> > > >>>> >>> >> > > > > >> > > > > >> > > other
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > regions in WALs, but maybe
> I
> > > don't
> > > >>>> >>> care
> > > >>>> >>> >> for
> > > >>>> >>> >> > my
> > > >>>> >>> >> > > > use
> > > >>>> >>> >> > > > > >> > > case...)
> > > >>>> >>> >> > > > > >> > > > --
> > > >>>> >>> >> > > > > >> > > > > >> > needs
> > > >>>> >>> >> > > > > >> > > > > >> > > to
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > accompany the merge. Lets
> > fold
> > > >>>> them
> > > >>>> >>> into
> > > >>>> >>> >> the
> > > >>>> >>> >> > > user
> > > >>>> >>> >> > > > > >> doc.
> > > >>>> >>> >> > > > > >> > in
> > > >>>> >>> >> > > > > >> > > > the
> > > >>>> >>> >> > > > > >> > > > > >> > > technical
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > overview area as suggested
> so
> > > user
> > > >>>> >>> >> > expectations
> > > >>>> >>> >> > > > are
> > > >>>> >>> >> > > > > >> > > properly
> > > >>>> >>> >> > > > > >> > > > > >> > managed
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > (otherwise, they expect the
> > > world
> > > >>>> and
> > > >>>> >>> will
> > > >>>> >>> >> > just
> > > >>>> >>> >> > > > > give
> > > >>>> >>> >> > > > > >> up
> > > >>>> >>> >> > > > > >> > > when
> > > >>>> >>> >> > > > > >> > > > > we
> > > >>>> >>> >> > > > > >> > > > > >> > fall
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > short). Vladimir did a list
> > of
> > > >>>> what
> > > >>>> >>> is in
> > > >>>> >>> >> > each
> > > >>>> >>> >> > > of
> > > >>>> >>> >> > > > > the
> > > >>>> >>> >> > > > > >> > > phases
> > > >>>> >>> >> > > > > >> > > > > >> above
> > > >>>> >>> >> > > > > >> > > > > >> > > > which
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > would serve as a good
> start.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > + Is this feature
> > > 'experimental'
> > > >>>> >>> (Matteo
> > > >>>> >>> >> asks
> > > >>>> >>> >> > > > > above).
> > > >>>> >>> >> > > > > >> > I'd
> > > >>>> >>> >> > > > > >> > > > > prefer
> > > >>>> >>> >> > > > > >> > > > > >> it
> > > >>>> >>> >> > > > > >> > > > > >> > > is
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > not. If it is, it should be
> > > >>>> labelled
> > > >>>> >>> all
> > > >>>> >>> >> over
> > > >>>> >>> >> > > > that
> > > >>>> >>> >> > > > > >> it is
> > > >>>> >>> >> > > > > >> > > > so. I
> > > >>>> >>> >> > > > > >> > > > > >> see
> > > >>>> >>> >> > > > > >> > > > > >> > > > > current
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > state called out as a '...
> > > >>>> technical
> > > >>>> >>> >> preview
> > > >>>> >>> >> > > > > >> feature'.
> > > >>>> >>> >> > > > > >> > > Does
> > > >>>> >>> >> > > > > >> > > > > this
> > > >>>> >>> >> > > > > >> > > > > >> > mean
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > not-for-users?
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > St.Ack
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > On Mon, Sep 12, 2016 at
> 8:03
> > > AM,
> > > >>>> Ted
> > > >>>> >>> Yu <
> > > >>>> >>> >> > > > > >> > > > yuzhihong@gmail.com>
> > > >>>> >>> >> > > > > >> > > > > >> > wrote:
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> Sean:
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> Do you have more comments
> ?
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >>
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> Cheers
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >>
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> On Fri, Sep 9, 2016 at
> 1:42
> > > PM,
> > > >>>> >>> Vladimir
> > > >>>> >>> >> > > > Rodionov
> > > >>>> >>> >> > > > > <
> > > >>>> >>> >> > > > > >> > > > > >> > > > > vladrodionov@gmail.com
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> wrote:
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >>
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > Sean,
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > Backup/Restore can fail
> > due
> > > to
> > > >>>> >>> various
> > > >>>> >>> >> > > > reasons:
> > > >>>> >>> >> > > > > >> > network
> > > >>>> >>> >> > > > > >> > > > > outage
> > > >>>> >>> >> > > > > >> > > > > >> > > > > (cluster
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > wide), various time-outs
> > in
> > > >>>> HBase
> > > >>>> >>> and
> > > >>>> >>> >> HDFS
> > > >>>> >>> >> > > > > layer,
> > > >>>> >>> >> > > > > >> M/R
> > > >>>> >>> >> > > > > >> > > > > failure
> > > >>>> >>> >> > > > > >> > > > > >> > due
> > > >>>> >>> >> > > > > >> > > > > >> > > to
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> "HDFS
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > exceeded quota", user
> > error
> > > >>>> (manual
> > > >>>> >>> >> > deletion
> > > >>>> >>> >> > > > of
> > > >>>> >>> >> > > > > >> data)
> > > >>>> >>> >> > > > > >> > > and
> > > >>>> >>> >> > > > > >> > > > > so
> > > >>>> >>> >> > > > > >> > > > > >> on
> > > >>>> >>> >> > > > > >> > > > > >> > so
> > > >>>> >>> >> > > > > >> > > > > >> > > > on.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> That
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > is impossible to
> enumerate
> > > all
> > > >>>> >>> possible
> > > >>>> >>> >> > > types
> > > >>>> >>> >> > > > of
> > > >>>> >>> >> > > > > >> > > failures
> > > >>>> >>> >> > > > > >> > > > > in a
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> distributed
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > system - that is not our
> > > >>>> goal/task.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > We focus completely on
> > > backup
> > > >>>> >>> system
> > > >>>> >>> >> table
> > > >>>> >>> >> > > > > >> > consistency
> > > >>>> >>> >> > > > > >> > > > in a
> > > >>>> >>> >> > > > > >> > > > > >> > > presence
> > > >>>> >>> >> > > > > >> > > > > >> > > > > of
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> any
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > type of failure. That is
> > > what I
> > > >>>> >>> call
> > > >>>> >>> >> > > > "tolerance
> > > >>>> >>> >> > > > > to
> > > >>>> >>> >> > > > > >> > > > > failures".
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > On a failure:
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > BACKUP. All backup
> system
> > > >>>> >>> information
> > > >>>> >>> >> > (prior
> > > >>>> >>> >> > > > to
> > > >>>> >>> >> > > > > >> > backup)
> > > >>>> >>> >> > > > > >> > > > > will
> > > >>>> >>> >> > > > > >> > > > > >> be
> > > >>>> >>> >> > > > > >> > > > > >> > > > > restored
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > and all temporary data,
> > > >>>> related to
> > > >>>> >>> a
> > > >>>> >>> >> > failed
> > > >>>> >>> >> > > > > >> session,
> > > >>>> >>> >> > > > > >> > in
> > > >>>> >>> >> > > > > >> > > > > HDFS
> > > >>>> >>> >> > > > > >> > > > > >> > will
> > > >>>> >>> >> > > > > >> > > > > >> > > be
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > deleted
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > RESTORE. We do not care
> > > about
> > > >>>> >>> system
> > > >>>> >>> >> data,
> > > >>>> >>> >> > > > > because
> > > >>>> >>> >> > > > > >> > > > restore
> > > >>>> >>> >> > > > > >> > > > > >> does
> > > >>>> >>> >> > > > > >> > > > > >> > > not
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> change
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > it. Temporary data in
> HDFS
> > > >>>> will be
> > > >>>> >>> >> cleaned
> > > >>>> >>> >> > > up
> > > >>>> >>> >> > > > > and
> > > >>>> >>> >> > > > > >> > table
> > > >>>> >>> >> > > > > >> > > > > will
> > > >>>> >>> >> > > > > >> > > > > >> be
> > > >>>> >>> >> > > > > >> > > > > >> > > in a
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> state
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > back to where it was
> > before
> > > >>>> >>> operation
> > > >>>> >>> >> > > started.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > This is what user should
> > > >>>> expect in
> > > >>>> >>> case
> > > >>>> >>> >> > of a
> > > >>>> >>> >> > > > > >> failure.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > -Vlad
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > -Vlad
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > On Fri, Sep 9, 2016 at
> > 12:56
> > > >>>> PM,
> > > >>>> >>> Sean
> > > >>>> >>> >> > > Busbey <
> > > >>>> >>> >> > > > > >> > > > > >> busbey@apache.org
> > > >>>> >>> >> > > > > >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > wrote:
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > Failing in a
> consistent
> > > way,
> > > >>>> with
> > > >>>> >>> >> docs
> > > >>>> >>> >> > > that
> > > >>>> >>> >> > > > > >> explain
> > > >>>> >>> >> > > > > >> > > the
> > > >>>> >>> >> > > > > >> > > > > >> > various
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > expected failures
> would
> > be
> > > >>>> >>> >> sufficient.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > On Fri, Sep 9, 2016 at
> > > 12:16
> > > >>>> PM,
> > > >>>> >>> >> > Vladimir
> > > >>>> >>> >> > > > > >> Rodionov
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > <
> vladrodionov@gmail.com
> > >
> > > >>>> wrote:
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > Do not worry Sean,
> doc
> > > is
> > > >>>> >>> coming
> > > >>>> >>> >> today
> > > >>>> >>> >> > > as
> > > >>>> >>> >> > > > a
> > > >>>> >>> >> > > > > >> > preview
> > > >>>> >>> >> > > > > >> > > > and
> > > >>>> >>> >> > > > > >> > > > > >> our
> > > >>>> >>> >> > > > > >> > > > > >> > > > writer
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > Frank
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > will be working on a
> > > >>>> putting
> > > >>>> >>> it
> > > >>>> >>> >> into
> > > >>>> >>> >> > > > Apache
> > > >>>> >>> >> > > > > >> > repo.
> > > >>>> >>> >> > > > > >> > > > > >> Timeline
> > > >>>> >>> >> > > > > >> > > > > >> > > > > depends
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> on
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > Franks schedule but
> I
> > > hope
> > > >>>> we
> > > >>>> >>> will
> > > >>>> >>> >> get
> > > >>>> >>> >> > > it
> > > >>>> >>> >> > > > > >> rather
> > > >>>> >>> >> > > > > >> > > > sooner
> > > >>>> >>> >> > > > > >> > > > > >> than
> > > >>>> >>> >> > > > > >> > > > > >> > > > > later.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > As for failure
> > testing,
> > > we
> > > >>>> are
> > > >>>> >>> >> > focusing
> > > >>>> >>> >> > > > only
> > > >>>> >>> >> > > > > >> on a
> > > >>>> >>> >> > > > > >> > > > > >> consistent
> > > >>>> >>> >> > > > > >> > > > > >> > > > state
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> of
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > backup system data
> in
> > a
> > > >>>> >>> presence of
> > > >>>> >>> >> > any
> > > >>>> >>> >> > > > type
> > > >>>> >>> >> > > > > >> of
> > > >>>> >>> >> > > > > >> > > > > failures,
> > > >>>> >>> >> > > > > >> > > > > >> We
> > > >>>> >>> >> > > > > >> > > > > >> > > are
> > > >>>> >>> >> > > > > >> > > > > >> > > > > not
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > going
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > to implement
> anything
> > > more
> > > >>>> >>> >> "fancy",
> > > >>>> >>> >> > > than
> > > >>>> >>> >> > > > > >> that.
> > > >>>> >>> >> > > > > >> > We
> > > >>>> >>> >> > > > > >> > > > > allow
> > > >>>> >>> >> > > > > >> > > > > >> > both:
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> backup
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > and
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > restore to fail.
> What
> > we
> > > >>>> do not
> > > >>>> >>> >> allow
> > > >>>> >>> >> > is
> > > >>>> >>> >> > > > to
> > > >>>> >>> >> > > > > >> have
> > > >>>> >>> >> > > > > >> > > > system
> > > >>>> >>> >> > > > > >> > > > > >> data
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> corrupted.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > Will it suffice for
> > you?
> > > >>>> Do you
> > > >>>> >>> >> have
> > > >>>> >>> >> > any
> > > >>>> >>> >> > > > > other
> > > >>>> >>> >> > > > > >> > > > > concerns,
> > > >>>> >>> >> > > > > >> > > > > >> you
> > > >>>> >>> >> > > > > >> > > > > >> > > > want
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> us to
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > address?
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > -Vlad
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > On Fri, Sep 9, 2016
> at
> > > >>>> 10:56
> > > >>>> >>> AM,
> > > >>>> >>> >> Sean
> > > >>>> >>> >> > > > > Busbey <
> > > >>>> >>> >> > > > > >> > > > > >> > > busbey@apache.org
> > > >>>> >>> >> > > > > >> > > > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > wrote:
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> "docs will come to
> > > Apache
> > > >>>> >>> soon"
> > > >>>> >>> >> does
> > > >>>> >>> >> > > not
> > > >>>> >>> >> > > > > >> address
> > > >>>> >>> >> > > > > >> > > my
> > > >>>> >>> >> > > > > >> > > > > >> concern
> > > >>>> >>> >> > > > > >> > > > > >> > > > > around
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > docs
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > at
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> all, unless said
> docs
> > > have
> > > >>>> >>> already
> > > >>>> >>> >> > made
> > > >>>> >>> >> > > > it
> > > >>>> >>> >> > > > > >> into
> > > >>>> >>> >> > > > > >> > > the
> > > >>>> >>> >> > > > > >> > > > > >> project
> > > >>>> >>> >> > > > > >> > > > > >> > > > > repo. I
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > don't
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> want third party
> > > >>>> resources for
> > > >>>> >>> >> using
> > > >>>> >>> >> > a
> > > >>>> >>> >> > > > > major
> > > >>>> >>> >> > > > > >> and
> > > >>>> >>> >> > > > > >> > > > > >> important
> > > >>>> >>> >> > > > > >> > > > > >> > > > > feature
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> of
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> project, I want us
> to
> > > >>>> provide
> > > >>>> >>> end
> > > >>>> >>> >> > users
> > > >>>> >>> >> > > > > with
> > > >>>> >>> >> > > > > >> > what
> > > >>>> >>> >> > > > > >> > > > they
> > > >>>> >>> >> > > > > >> > > > > >> need
> > > >>>> >>> >> > > > > >> > > > > >> > > to
> > > >>>> >>> >> > > > > >> > > > > >> > > > > get
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > job
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> done.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >>
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> I see some calls
> for
> > > >>>> patience
> > > >>>> >>> on
> > > >>>> >>> >> the
> > > >>>> >>> >> > > > > failure
> > > >>>> >>> >> > > > > >> > > > testing,
> > > >>>> >>> >> > > > > >> > > > > but
> > > >>>> >>> >> > > > > >> > > > > >> > the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> appeal
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > to
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > us
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> having done a bad
> job
> > > of
> > > >>>> >>> requiring
> > > >>>> >>> >> > > proper
> > > >>>> >>> >> > > > > >> tests
> > > >>>> >>> >> > > > > >> > of
> > > >>>> >>> >> > > > > >> > > > > >> previous
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> features
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > just
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> makes me more
> > concerned
> > > >>>> about
> > > >>>> >>> not
> > > >>>> >>> >> > > getting
> > > >>>> >>> >> > > > > >> them
> > > >>>> >>> >> > > > > >> > > > here. I
> > > >>>> >>> >> > > > > >> > > > > >> > don't
> > > >>>> >>> >> > > > > >> > > > > >> > > > want
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> to
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > set
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> yet another bad
> > example
> > > >>>> that
> > > >>>> >>> will
> > > >>>> >>> >> > then
> > > >>>> >>> >> > > be
> > > >>>> >>> >> > > > > >> > pointed
> > > >>>> >>> >> > > > > >> > > to
> > > >>>> >>> >> > > > > >> > > > > in
> > > >>>> >>> >> > > > > >> > > > > >> the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > future.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >>
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> On Sep 8, 2016
> 10:50,
> > > "Ted
> > > >>>> >>> Yu" <
> > > >>>> >>> >> > > > > >> > > yuzhihong@gmail.com
> > > >>>> >>> >> > > > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > wrote:
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >>
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > Is there any
> > concern
> > > >>>> which
> > > >>>> >>> is
> > > >>>> >>> >> not
> > > >>>> >>> >> > > > > >> addressed ?
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > Do we need
> another
> > > Vote
> > > >>>> >>> thread ?
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > Thanks
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > On Thu, Sep 8,
> 2016
> > > at
> > > >>>> 9:21
> > > >>>> >>> AM,
> > > >>>> >>> >> > > Andrew
> > > >>>> >>> >> > > > > >> > Purtell <
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> apurtell@apache.org
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > wrote:
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > Vlad,
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > I apologize for
> > > using
> > > >>>> the
> > > >>>> >>> term
> > > >>>> >>> >> > > > > >> 'half-baked'
> > > >>>> >>> >> > > > > >> > > in a
> > > >>>> >>> >> > > > > >> > > > > way
> > > >>>> >>> >> > > > > >> > > > > >> > that
> > > >>>> >>> >> > > > > >> > > > > >> > > > > could
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > seem a
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > description of
> > > >>>> >>> HBASE-7912. I
> > > >>>> >>> >> > meant
> > > >>>> >>> >> > > > that
> > > >>>> >>> >> > > > > >> as a
> > > >>>> >>> >> > > > > >> > > > > general
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> hypothetical.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > On Wed, Sep 7,
> > > 2016 at
> > > >>>> >>> 9:36
> > > >>>> >>> >> AM,
> > > >>>> >>> >> > > > > Vladimir
> > > >>>> >>> >> > > > > >> > > > Rodionov
> > > >>>> >>> >> > > > > >> > > > > <
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> > > vladrodionov@gmail.com>
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > wrote:
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >> I'm not
> sure
> > > that
> > > >>>> >>> "There
> > > >>>> >>> >> is
> > > >>>> >>> >> > > > > already
> > > >>>> >>> >> > > > > >> > lots
> > > >>>> >>> >> > > > > >> > > of
> > > >>>> >>> >> > > > > >> > > > > >> > > half-baked
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> code
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > in
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > branch,
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > so what's the
> > > harm
> > > >>>> in
> > > >>>> >>> adding
> > > >>>> >>> >> > > more?"
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > I meant - not
> > > >>>> >>> production -
> > > >>>> >>> >> > ready
> > > >>>> >>> >> > > > yet.
> > > >>>> >>> >> > > > > >> This
> > > >>>> >>> >> > > > > >> > > is
> > > >>>> >>> >> > > > > >> > > > > 2.0
> > > >>>> >>> >> > > > > >> > > > > >> > > > > development
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > branch
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > and,
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > hence many
> > > features
> > > >>>> are
> > > >>>> >>> in
> > > >>>> >>> >> > works,
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > not being
> > tested
> > > >>>> well
> > > >>>> >>> etc.
> > > >>>> >>> >> I do
> > > >>>> >>> >> > > not
> > > >>>> >>> >> > > > > >> > consider
> > > >>>> >>> >> > > > > >> > > > > backup
> > > >>>> >>> >> > > > > >> > > > > >> > as
> > > >>>> >>> >> > > > > >> > > > > >> > > > half
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > baked
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > feature -
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > it has passed
> > our
> > > >>>> >>> internal
> > > >>>> >>> >> QA
> > > >>>> >>> >> > and
> > > >>>> >>> >> > > > has
> > > >>>> >>> >> > > > > >> very
> > > >>>> >>> >> > > > > >> > > > good
> > > >>>> >>> >> > > > > >> > > > > >> doc,
> > > >>>> >>> >> > > > > >> > > > > >> > > > which
> > > >>>> >>> >> > > > > >> > > > > >> > > > > we
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > will
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > provide
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > to Apache
> > > shortly.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > -Vlad
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > On Wed, Sep
> 7,
> > > 2016
> > > >>>> at
> > > >>>> >>> 9:13
> > > >>>> >>> >> AM,
> > > >>>> >>> >> > > > > Andrew
> > > >>>> >>> >> > > > > >> > > > Purtell <
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > apurtell@apache.org>
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > We
> shouldn't
> > > admit
> > > >>>> >>> half
> > > >>>> >>> >> baked
> > > >>>> >>> >> > > > > changes
> > > >>>> >>> >> > > > > >> > that
> > > >>>> >>> >> > > > > >> > > > > won't
> > > >>>> >>> >> > > > > >> > > > > >> be
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> finished.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> However
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > in
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > this case
> the
> > > crew
> > > >>>> >>> >> working on
> > > >>>> >>> >> > > > this
> > > >>>> >>> >> > > > > >> > feature
> > > >>>> >>> >> > > > > >> > > > are
> > > >>>> >>> >> > > > > >> > > > > >> long
> > > >>>> >>> >> > > > > >> > > > > >> > > > > timers
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> and
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > less
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > likely
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > than just
> > about
> > > >>>> >>> anyone to
> > > >>>> >>> >> > leave
> > > >>>> >>> >> > > > > >> > something
> > > >>>> >>> >> > > > > >> > > > in a
> > > >>>> >>> >> > > > > >> > > > > >> half
> > > >>>> >>> >> > > > > >> > > > > >> > > > baked
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > state. Of
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > course
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > there is no
> > > >>>> guarantee
> > > >>>> >>> how
> > > >>>> >>> >> > > > anything
> > > >>>> >>> >> > > > > >> will
> > > >>>> >>> >> > > > > >> > > turn
> > > >>>> >>> >> > > > > >> > > > > out,
> > > >>>> >>> >> > > > > >> > > > > >> > > but I
> > > >>>> >>> >> > > > > >> > > > > >> > > > > am
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > willing
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> to
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > take
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > a little on
> > > faith
> > > >>>> if
> > > >>>> >>> they
> > > >>>> >>> >> > feel
> > > >>>> >>> >> > > > > their
> > > >>>> >>> >> > > > > >> > best
> > > >>>> >>> >> > > > > >> > > > path
> > > >>>> >>> >> > > > > >> > > > > >> > > forward
> > > >>>> >>> >> > > > > >> > > > > >> > > > > now
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> is
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > to
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > merge
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > to
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > trunk. I
> only
> > > >>>> wish I
> > > >>>> >>> had
> > > >>>> >>> >> > > > bandwidth
> > > >>>> >>> >> > > > > to
> > > >>>> >>> >> > > > > >> > have
> > > >>>> >>> >> > > > > >> > > > > done
> > > >>>> >>> >> > > > > >> > > > > >> > some
> > > >>>> >>> >> > > > > >> > > > > >> > > > real
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > kicking
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> of
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > tires by
> now.
> > > >>>> Maybe
> > > >>>> >>> this
> > > >>>> >>> >> > week.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > (Yes, I'm
> > using
> > > >>>> some
> > > >>>> >>> of
> > > >>>> >>> >> that
> > > >>>> >>> >> > > time
> > > >>>> >>> >> > > > > for
> > > >>>> >>> >> > > > > >> > this
> > > >>>> >>> >> > > > > >> > > > > email
> > > >>>> >>> >> > > > > >> > > > > >> > :-)
> > > >>>> >>> >> > > > > >> > > > > >> > > > but
> > > >>>> >>> >> > > > > >> > > > > >> > > > > I
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > type
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > fast.)
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > That said,
> I
> > > would
> > > >>>> >>> like to
> > > >>>> >>> >> > > > agitate
> > > >>>> >>> >> > > > > >> for
> > > >>>> >>> >> > > > > >> > > > making
> > > >>>> >>> >> > > > > >> > > > > 2.0
> > > >>>> >>> >> > > > > >> > > > > >> > > more
> > > >>>> >>> >> > > > > >> > > > > >> > > > > real
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > and
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> spend
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > some
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > time on it
> > now
> > > >>>> that
> > > >>>> >>> I'm
> > > >>>> >>> >> > winding
> > > >>>> >>> >> > > > > down
> > > >>>> >>> >> > > > > >> > with
> > > >>>> >>> >> > > > > >> > > > > 0.98. I
> > > >>>> >>> >> > > > > >> > > > > >> > > think
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> that
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > means
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branching
> for
> > > 2.0
> > > >>>> real
> > > >>>> >>> >> soon
> > > >>>> >>> >> > now
> > > >>>> >>> >> > > > and
> > > >>>> >>> >> > > > > >> even
> > > >>>> >>> >> > > > > >> > > > > evicting
> > > >>>> >>> >> > > > > >> > > > > >> > > > things
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> from
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > 2.0
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > branch
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > that aren't
> > > >>>> finished
> > > >>>> >>> or
> > > >>>> >>> >> > stable,
> > > >>>> >>> >> > > > > >> leaving
> > > >>>> >>> >> > > > > >> > > them
> > > >>>> >>> >> > > > > >> > > > > only
> > > >>>> >>> >> > > > > >> > > > > >> > > once
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> again
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > in
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > master
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branch. Or,
> > > maybe
> > > >>>> just
> > > >>>> >>> >> > evicting
> > > >>>> >>> >> > > > > them.
> > > >>>> >>> >> > > > > >> > > Let's
> > > >>>> >>> >> > > > > >> > > > > take
> > > >>>> >>> >> > > > > >> > > > > >> it
> > > >>>> >>> >> > > > > >> > > > > >> > > > case
> > > >>>> >>> >> > > > > >> > > > > >> > > > > by
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > case.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > I think
> this
> > > >>>> feature
> > > >>>> >>> can
> > > >>>> >>> >> come
> > > >>>> >>> >> > > in
> > > >>>> >>> >> > > > > >> > > relatively
> > > >>>> >>> >> > > > > >> > > > > >> safely.
> > > >>>> >>> >> > > > > >> > > > > >> > > As
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> added
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > insurance,
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > let's admit
> > the
> > > >>>> >>> >> possibility
> > > >>>> >>> >> > it
> > > >>>> >>> >> > > > > could
> > > >>>> >>> >> > > > > >> be
> > > >>>> >>> >> > > > > >> > > > > reverted
> > > >>>> >>> >> > > > > >> > > > > >> on
> > > >>>> >>> >> > > > > >> > > > > >> > > the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > 2.0
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > branch
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> if
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > folks
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > working on
> > > >>>> >>> stabilizing 2.0
> > > >>>> >>> >> > > decide
> > > >>>> >>> >> > > > > to
> > > >>>> >>> >> > > > > >> > evict
> > > >>>> >>> >> > > > > >> > > > it
> > > >>>> >>> >> > > > > >> > > > > >> > because
> > > >>>> >>> >> > > > > >> > > > > >> > > > it
> > > >>>> >>> >> > > > > >> > > > > >> > > > > is
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > unfinished
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > or
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > unstable,
> > > because
> > > >>>> that
> > > >>>> >>> >> > > certainly
> > > >>>> >>> >> > > > > can
> > > >>>> >>> >> > > > > >> > > > happen. I
> > > >>>> >>> >> > > > > >> > > > > >> > would
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> expect if
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > talk
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > like
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > that
> starts,
> > > we'd
> > > >>>> get
> > > >>>> >>> help
> > > >>>> >>> >> > > > > finishing
> > > >>>> >>> >> > > > > >> or
> > > >>>> >>> >> > > > > >> > > > > >> stabilizing
> > > >>>> >>> >> > > > > >> > > > > >> > > > > what's
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > under
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > discussion
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > for revert.
> > Or,
> > > >>>> we'd
> > > >>>> >>> have
> > > >>>> >>> >> a
> > > >>>> >>> >> > > > revert.
> > > >>>> >>> >> > > > > >> > Either
> > > >>>> >>> >> > > > > >> > > > way
> > > >>>> >>> >> > > > > >> > > > > >> the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > outcome
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> is
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > acceptable.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > On Wed, Sep
> > 7,
> > > >>>> 2016 at
> > > >>>> >>> >> 8:56
> > > >>>> >>> >> > AM,
> > > >>>> >>> >> > > > > Dima
> > > >>>> >>> >> > > > > >> > > Spivak
> > > >>>> >>> >> > > > > >> > > > <
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > dimaspivak@apache.org
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > I'm not
> > sure
> > > >>>> that
> > > >>>> >>> >> "There is
> > > >>>> >>> >> > > > > already
> > > >>>> >>> >> > > > > >> > lots
> > > >>>> >>> >> > > > > >> > > > of
> > > >>>> >>> >> > > > > >> > > > > >> > > > half-baked
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> code
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > in
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branch,
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > so what's
> > the
> > > >>>> harm
> > > >>>> >>> in
> > > >>>> >>> >> > adding
> > > >>>> >>> >> > > > > more?"
> > > >>>> >>> >> > > > > >> > is a
> > > >>>> >>> >> > > > > >> > > > > good
> > > >>>> >>> >> > > > > >> > > > > >> > code
> > > >>>> >>> >> > > > > >> > > > > >> > > > > commit
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > philosophy
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > for
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > a
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> > > fault-tolerant
> > > >>>> >>> >> distributed
> > > >>>> >>> >> > > data
> > > >>>> >>> >> > > > > >> store.
> > > >>>> >>> >> > > > > >> > > ;)
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > More
> > > seriously,
> > > >>>> a
> > > >>>> >>> lack
> > > >>>> >>> >> of
> > > >>>> >>> >> > > test
> > > >>>> >>> >> > > > > >> > coverage
> > > >>>> >>> >> > > > > >> > > > for
> > > >>>> >>> >> > > > > >> > > > > >> > > existing
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > features
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > shouldn't
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > be
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > used as
> > > >>>> >>> justification
> > > >>>> >>> >> for
> > > >>>> >>> >> > > > > >> introducing
> > > >>>> >>> >> > > > > >> > > new
> > > >>>> >>> >> > > > > >> > > > > >> > features
> > > >>>> >>> >> > > > > >> > > > > >> > > > with
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > same
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> > shortcomings.
> > > >>>> >>> >> Ultimately,
> > > >>>> >>> >> > > it's
> > > >>>> >>> >> > > > > the
> > > >>>> >>> >> > > > > >> end
> > > >>>> >>> >> > > > > >> > > > user
> > > >>>> >>> >> > > > > >> > > > > who
> > > >>>> >>> >> > > > > >> > > > > >> > > will
> > > >>>> >>> >> > > > > >> > > > > >> > > > > feel
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> pain,
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > so
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > shouldn't
> > we
> > > do
> > > >>>> >>> >> everything
> > > >>>> >>> >> > we
> > > >>>> >>> >> > > > can
> > > >>>> >>> >> > > > > >> to
> > > >>>> >>> >> > > > > >> > > > > mitigate
> > > >>>> >>> >> > > > > >> > > > > >> > that?
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > -Dima
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > On Wed,
> Sep
> > > 7,
> > > >>>> 2016
> > > >>>> >>> at
> > > >>>> >>> >> 8:46
> > > >>>> >>> >> > > AM,
> > > >>>> >>> >> > > > > >> > Vladimir
> > > >>>> >>> >> > > > > >> > > > > >> > Rodionov <
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > >>>> >>> vladrodionov@gmail.com>
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > wrote:
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Sean,
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have
> > docs
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Agree.
> We
> > > >>>> have a
> > > >>>> >>> doc
> > > >>>> >>> >> and
> > > >>>> >>> >> > > > backup
> > > >>>> >>> >> > > > > >> is
> > > >>>> >>> >> > > > > >> > the
> > > >>>> >>> >> > > > > >> > > > > most
> > > >>>> >>> >> > > > > >> > > > > >> > > > > documented
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > feature
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > :),
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > we
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > will
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > release
> > it
> > > >>>> >>> shortly to
> > > >>>> >>> >> > > Apache.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have
> > > >>>> sunny-day
> > > >>>> >>> >> > > correctness
> > > >>>> >>> >> > > > > >> tests
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Feature
> > has
> > > >>>> >>> close to
> > > >>>> >>> >> 60
> > > >>>> >>> >> > > test
> > > >>>> >>> >> > > > > >> cases,
> > > >>>> >>> >> > > > > >> > > > which
> > > >>>> >>> >> > > > > >> > > > > >> run
> > > >>>> >>> >> > > > > >> > > > > >> > > for
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> approx
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > 30
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> min.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > We
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > can
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > add
> more,
> > > if
> > > >>>> >>> >> community do
> > > >>>> >>> >> > > not
> > > >>>> >>> >> > > > > >> mind
> > > >>>> >>> >> > > > > >> > :)
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have
> > > >>>> >>> >> > > > > >> > correctness-in-face-of-failure
> > > >>>> >>> >> > > > > >> > > > > tests
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Any
> > > examples
> > > >>>> of
> > > >>>> >>> these
> > > >>>> >>> >> > tests
> > > >>>> >>> >> > > > in
> > > >>>> >>> >> > > > > >> > > existing
> > > >>>> >>> >> > > > > >> > > > > >> > features?
> > > >>>> >>> >> > > > > >> > > > > >> > > > In
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > works,
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > we
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > have a
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > clear
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > understanding
> > > >>>> of
> > > >>>> >>> what
> > > >>>> >>> >> > > should
> > > >>>> >>> >> > > > be
> > > >>>> >>> >> > > > > >> done
> > > >>>> >>> >> > > > > >> > > by
> > > >>>> >>> >> > > > > >> > > > > the
> > > >>>> >>> >> > > > > >> > > > > >> > time
> > > >>>> >>> >> > > > > >> > > > > >> > > of
> > > >>>> >>> >> > > > > >> > > > > >> > > > > 2.0
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> release.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > That is
> > > very
> > > >>>> close
> > > >>>> >>> >> goal
> > > >>>> >>> >> > for
> > > >>>> >>> >> > > > us,
> > > >>>> >>> >> > > > > >> to
> > > >>>> >>> >> > > > > >> > > > verify
> > > >>>> >>> >> > > > > >> > > > > IT
> > > >>>> >>> >> > > > > >> > > > > >> > > monkey
> > > >>>> >>> >> > > > > >> > > > > >> > > > > for
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> existing
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > code.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * don't
> > > rely
> > > >>>> on
> > > >>>> >>> things
> > > >>>> >>> >> > > > outside
> > > >>>> >>> >> > > > > of
> > > >>>> >>> >> > > > > >> > > HBase
> > > >>>> >>> >> > > > > >> > > > > for
> > > >>>> >>> >> > > > > >> > > > > >> > > normal
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > operation
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > (okay
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > for
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> advanced
> > > >>>> >>> operation)
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > We do
> > not.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> Enormous
> > > time
> > > >>>> has
> > > >>>> >>> been
> > > >>>> >>> >> > > spent
> > > >>>> >>> >> > > > > >> already
> > > >>>> >>> >> > > > > >> > > on
> > > >>>> >>> >> > > > > >> > > > > the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > development
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > and
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > testing
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> feature,
> > it
> > > >>>> has
> > > >>>> >>> passed
> > > >>>> >>> >> > our
> > > >>>> >>> >> > > > > >> internal
> > > >>>> >>> >> > > > > >> > > > tests
> > > >>>> >>> >> > > > > >> > > > > and
> > > >>>> >>> >> > > > > >> > > > > >> > > many
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> rounds
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > of
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> code
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > reviews
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > by
> HBase
> > > >>>> >>> committers.
> > > >>>> >>> >> We
> > > >>>> >>> >> > do
> > > >>>> >>> >> > > > not
> > > >>>> >>> >> > > > > >> mind
> > > >>>> >>> >> > > > > >> > if
> > > >>>> >>> >> > > > > >> > > > > >> someone
> > > >>>> >>> >> > > > > >> > > > > >> > > from
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> HBase
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > community
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> (outside
> > of
> > > >>>> HW)
> > > >>>> >>> will
> > > >>>> >>> >> > review
> > > >>>> >>> >> > > > the
> > > >>>> >>> >> > > > > >> > code,
> > > >>>> >>> >> > > > > >> > > > but
> > > >>>> >>> >> > > > > >> > > > > it
> > > >>>> >>> >> > > > > >> > > > > >> > will
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> probably
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> takes
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > forever
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > to
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > wait
> for
> > > >>>> >>> volunteer?,
> > > >>>> >>> >> the
> > > >>>> >>> >> > > > > feature
> > > >>>> >>> >> > > > > >> is
> > > >>>> >>> >> > > > > >> > > > quite
> > > >>>> >>> >> > > > > >> > > > > >> large
> > > >>>> >>> >> > > > > >> > > > > >> > > > (1MB+
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> cumulative
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > patch)
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > 2.0
> > branch
> > > is
> > > >>>> >>> full of
> > > >>>> >>> >> > half
> > > >>>> >>> >> > > > > baked
> > > >>>> >>> >> > > > > >> > > > features,
> > > >>>> >>> >> > > > > >> > > > > >> most
> > > >>>> >>> >> > > > > >> > > > > >> > > of
> > > >>>> >>> >> > > > > >> > > > > >> > > > > them
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > are
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > in
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > active
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > development,
> > > >>>> >>> >> therefore I
> > > >>>> >>> >> > am
> > > >>>> >>> >> > > > not
> > > >>>> >>> >> > > > > >> > > > following
> > > >>>> >>> >> > > > > >> > > > > you
> > > >>>> >>> >> > > > > >> > > > > >> > > here,
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> Sean?
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > Why
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > HBASE-7912
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > is
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > not
> good
> > > >>>> enough
> > > >>>> >>> yet
> > > >>>> >>> >> to be
> > > >>>> >>> >> > > > > >> integrated
> > > >>>> >>> >> > > > > >> > > > into
> > > >>>> >>> >> > > > > >> > > > > 2.0
> > > >>>> >>> >> > > > > >> > > > > >> > > > branch?
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > -Vlad
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > On Wed,
> > > Sep 7,
> > > >>>> >>> 2016 at
> > > >>>> >>> >> > 8:23
> > > >>>> >>> >> > > > AM,
> > > >>>> >>> >> > > > > >> Sean
> > > >>>> >>> >> > > > > >> > > > > Busbey <
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > busbey@apache.org
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > On
> Tue,
> > > Sep
> > > >>>> 6,
> > > >>>> >>> 2016
> > > >>>> >>> >> at
> > > >>>> >>> >> > > > 10:36
> > > >>>> >>> >> > > > > >> PM,
> > > >>>> >>> >> > > > > >> > > Josh
> > > >>>> >>> >> > > > > >> > > > > >> Elser <
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> > > josh.elser@gmail.com>
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > wrote:
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > So,
> > the
> > > >>>> >>> answer to
> > > >>>> >>> >> > > Sean's
> > > >>>> >>> >> > > > > >> > original
> > > >>>> >>> >> > > > > >> > > > > >> question
> > > >>>> >>> >> > > > > >> > > > > >> > is
> > > >>>> >>> >> > > > > >> > > > > >> > > > "as
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > robust as
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > snapshots
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> > > presently
> > > >>>> >>> are"?
> > > >>>> >>> >> > > > > >> (independence of
> > > >>>> >>> >> > > > > >> > > > > >> > > backup/restore
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > failure
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > tolerance
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > from
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> > > snapshot
> > > >>>> >>> failure
> > > >>>> >>> >> > > > tolerance)
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > Is
> > this
> > > >>>> just a
> > > >>>> >>> >> > question
> > > >>>> >>> >> > > > WRT
> > > >>>> >>> >> > > > > >> > > context
> > > >>>> >>> >> > > > > >> > > > of
> > > >>>> >>> >> > > > > >> > > > > >> the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > change,
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> or
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > is it
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > means
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > for a
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > veto
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> from
> > > you,
> > > >>>> >>> Sean?
> > > >>>> >>> >> Just
> > > >>>> >>> >> > > > trying
> > > >>>> >>> >> > > > > >> to
> > > >>>> >>> >> > > > > >> > > make
> > > >>>> >>> >> > > > > >> > > > > sure
> > > >>>> >>> >> > > > > >> > > > > >> > I'm
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> following
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> along
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > adequately.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > I'd
> say
> > > ATM
> > > >>>> I'm
> > > >>>> >>> -0,
> > > >>>> >>> >> > > > bordering
> > > >>>> >>> >> > > > > >> on
> > > >>>> >>> >> > > > > >> > -1
> > > >>>> >>> >> > > > > >> > > > but
> > > >>>> >>> >> > > > > >> > > > > not
> > > >>>> >>> >> > > > > >> > > > > >> > for
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> reasons
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > I
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > can
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> articulate
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > well.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> Here's
> > an
> > > >>>> >>> attempt.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > We've
> > > been
> > > >>>> >>> trying to
> > > >>>> >>> >> > > move,
> > > >>>> >>> >> > > > > as a
> > > >>>> >>> >> > > > > >> > > > > community,
> > > >>>> >>> >> > > > > >> > > > > >> > > > towards
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > minimizing
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > risk
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > to
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > downstream
> > > >>>> >>> folks by
> > > >>>> >>> >> > > getting
> > > >>>> >>> >> > > > > >> > > "complete
> > > >>>> >>> >> > > > > >> > > > > >> enough
> > > >>>> >>> >> > > > > >> > > > > >> > > for
> > > >>>> >>> >> > > > > >> > > > > >> > > > > use"
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > gates
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> in
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > place
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> before
> > we
> > > >>>> >>> introduce
> > > >>>> >>> >> new
> > > >>>> >>> >> > > > > >> features.
> > > >>>> >>> >> > > > > >> > > This
> > > >>>> >>> >> > > > > >> > > > > was
> > > >>>> >>> >> > > > > >> > > > > >> > > > spurred
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> by a
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > some
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > features
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> getting
> > > in
> > > >>>> >>> >> half-baked
> > > >>>> >>> >> > and
> > > >>>> >>> >> > > > > never
> > > >>>> >>> >> > > > > >> > > making
> > > >>>> >>> >> > > > > >> > > > > it
> > > >>>> >>> >> > > > > >> > > > > >> to
> > > >>>> >>> >> > > > > >> > > > > >> > > "can
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> really
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > use"
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > status
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > (I'm
> > > >>>> thinking of
> > > >>>> >>> >> > > > distributed
> > > >>>> >>> >> > > > > >> log
> > > >>>> >>> >> > > > > >> > > > replay
> > > >>>> >>> >> > > > > >> > > > > and
> > > >>>> >>> >> > > > > >> > > > > >> > the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> zk-less
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > assignment
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> stuff,
> > I
> > > >>>> don't
> > > >>>> >>> >> recall
> > > >>>> >>> >> > if
> > > >>>> >>> >> > > > > there
> > > >>>> >>> >> > > > > >> was
> > > >>>> >>> >> > > > > >> > > > > more).
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The
> > > gates,
> > > >>>> >>> >> generally,
> > > >>>> >>> >> > > > > included
> > > >>>> >>> >> > > > > >> > > things
> > > >>>> >>> >> > > > > >> > > > > like:
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > *
> have
> > > docs
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > *
> have
> > > >>>> sunny-day
> > > >>>> >>> >> > > > correctness
> > > >>>> >>> >> > > > > >> tests
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > *
> have
> > > >>>> >>> >> > > > > >> > > correctness-in-face-of-failure
> > > >>>> >>> >> > > > > >> > > > > tests
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > *
> don't
> > > >>>> rely on
> > > >>>> >>> >> things
> > > >>>> >>> >> > > > > outside
> > > >>>> >>> >> > > > > >> of
> > > >>>> >>> >> > > > > >> > > > HBase
> > > >>>> >>> >> > > > > >> > > > > for
> > > >>>> >>> >> > > > > >> > > > > >> > > > normal
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > operation
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > (okay
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > for
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > advanced
> > > >>>> >>> operation)
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > As an
> > > >>>> example,
> > > >>>> >>> we
> > > >>>> >>> >> kept
> > > >>>> >>> >> > > the
> > > >>>> >>> >> > > > > MOB
> > > >>>> >>> >> > > > > >> > work
> > > >>>> >>> >> > > > > >> > > > off
> > > >>>> >>> >> > > > > >> > > > > in
> > > >>>> >>> >> > > > > >> > > > > >> a
> > > >>>> >>> >> > > > > >> > > > > >> > > > branch
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> and
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > out
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> of
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > master
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > until
> > it
> > > >>>> could
> > > >>>> >>> pass
> > > >>>> >>> >> > these
> > > >>>> >>> >> > > > > >> > criteria.
> > > >>>> >>> >> > > > > >> > > > The
> > > >>>> >>> >> > > > > >> > > > > big
> > > >>>> >>> >> > > > > >> > > > > >> > > > > exemption
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > we've
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> had
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > to
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > this
> > was
> > > the
> > > >>>> >>> >> > hbase-spark
> > > >>>> >>> >> > > > > >> > > integration,
> > > >>>> >>> >> > > > > >> > > > > where
> > > >>>> >>> >> > > > > >> > > > > >> > we
> > > >>>> >>> >> > > > > >> > > > > >> > > > all
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > agreed
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > it
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > could
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > land
> in
> > > >>>> master
> > > >>>> >>> >> because
> > > >>>> >>> >> > it
> > > >>>> >>> >> > > > was
> > > >>>> >>> >> > > > > >> very
> > > >>>> >>> >> > > > > >> > > > well
> > > >>>> >>> >> > > > > >> > > > > >> > > isolated
> > > >>>> >>> >> > > > > >> > > > > >> > > > > (the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > slide
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > away
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > from
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > including
> > > >>>> docs
> > > >>>> >>> as a
> > > >>>> >>> >> > > > > first-class
> > > >>>> >>> >> > > > > >> > part
> > > >>>> >>> >> > > > > >> > > > of
> > > >>>> >>> >> > > > > >> > > > > >> > > building
> > > >>>> >>> >> > > > > >> > > > > >> > > > up
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> that
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > integration
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > has
> led
> > > me
> > > >>>> to
> > > >>>> >>> doubt
> > > >>>> >>> >> the
> > > >>>> >>> >> > > > > wisdom
> > > >>>> >>> >> > > > > >> of
> > > >>>> >>> >> > > > > >> > > this
> > > >>>> >>> >> > > > > >> > > > > >> > > decision).
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > We've
> > > also
> > > >>>> been
> > > >>>> >>> >> > treating
> > > >>>> >>> >> > > > > >> inclusion
> > > >>>> >>> >> > > > > >> > > in
> > > >>>> >>> >> > > > > >> > > > a
> > > >>>> >>> >> > > > > >> > > > > >> > > "probably
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> will
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > be
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > released
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > to
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > downstream"
> > > >>>> >>> branches
> > > >>>> >>> >> > as a
> > > >>>> >>> >> > > > > >> higher
> > > >>>> >>> >> > > > > >> > > bar,
> > > >>>> >>> >> > > > > >> > > > > >> > requiring
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > *
> don't
> > > >>>> >>> moderately
> > > >>>> >>> >> > impact
> > > >>>> >>> >> > > > > >> > > performance
> > > >>>> >>> >> > > > > >> > > > > when
> > > >>>> >>> >> > > > > >> > > > > >> > the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> feature
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > isn't
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> in
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > use
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > *
> don't
> > > >>>> severely
> > > >>>> >>> >> impact
> > > >>>> >>> >> > > > > >> > performance
> > > >>>> >>> >> > > > > >> > > > when
> > > >>>> >>> >> > > > > >> > > > > >> the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > feature
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> is
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > in
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> use
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > *
> > either
> > > >>>> >>> >> default-to-on
> > > >>>> >>> >> > or
> > > >>>> >>> >> > > > > show
> > > >>>> >>> >> > > > > >> > > enough
> > > >>>> >>> >> > > > > >> > > > > >> demand
> > > >>>> >>> >> > > > > >> > > > > >> > to
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> believe
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > a
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > non-trivial
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> number
> > of
> > > >>>> folks
> > > >>>> >>> will
> > > >>>> >>> >> > turn
> > > >>>> >>> >> > > > the
> > > >>>> >>> >> > > > > >> > > feature
> > > >>>> >>> >> > > > > >> > > > on
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The
> > above
> > > >>>> has
> > > >>>> >>> kept
> > > >>>> >>> >> MOB
> > > >>>> >>> >> > > and
> > > >>>> >>> >> > > > > >> > > hbase-spark
> > > >>>> >>> >> > > > > >> > > > > >> > > > integration
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> out
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > of
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > branch-1,
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > presumably
> > > >>>> while
> > > >>>> >>> >> > they've
> > > >>>> >>> >> > > > > >> "gotten
> > > >>>> >>> >> > > > > >> > > more
> > > >>>> >>> >> > > > > >> > > > > >> stable"
> > > >>>> >>> >> > > > > >> > > > > >> > > in
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> master
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > from
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > odd
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> vendor
> > > >>>> >>> inclusion.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Are
> we
> > > >>>> going to
> > > >>>> >>> >> have a
> > > >>>> >>> >> > > 2.0
> > > >>>> >>> >> > > > > >> release
> > > >>>> >>> >> > > > > >> > > > > before
> > > >>>> >>> >> > > > > >> > > > > >> the
> > > >>>> >>> >> > > > > >> > > > > >> > > end
> > > >>>> >>> >> > > > > >> > > > > >> > > > > of
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> year?
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > We're
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> coming
> > up
> > > >>>> on 1.5
> > > >>>> >>> >> years
> > > >>>> >>> >> > > > since
> > > >>>> >>> >> > > > > >> the
> > > >>>> >>> >> > > > > >> > > > > release of
> > > >>>> >>> >> > > > > >> > > > > >> > > > version
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> 1.0;
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> seems
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > like
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > it's
> > > about
> > > >>>> time,
> > > >>>> >>> >> > though I
> > > >>>> >>> >> > > > > >> haven't
> > > >>>> >>> >> > > > > >> > > seen
> > > >>>> >>> >> > > > > >> > > > > any
> > > >>>> >>> >> > > > > >> > > > > >> > > > concrete
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > plans
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> this
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > year.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > Presuming
> > > >>>> we are
> > > >>>> >>> >> going
> > > >>>> >>> >> > to
> > > >>>> >>> >> > > > > have
> > > >>>> >>> >> > > > > >> one
> > > >>>> >>> >> > > > > >> > > by
> > > >>>> >>> >> > > > > >> > > > > the
> > > >>>> >>> >> > > > > >> > > > > >> end
> > > >>>> >>> >> > > > > >> > > > > >> > > of
> > > >>>> >>> >> > > > > >> > > > > >> > > > > the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > year, it
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > seems a
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > bit
> > > close to
> > > >>>> >>> still
> > > >>>> >>> >> be
> > > >>>> >>> >> > > > adding
> > > >>>> >>> >> > > > > in
> > > >>>> >>> >> > > > > >> > > > > "features
> > > >>>> >>> >> > > > > >> > > > > >> > that
> > > >>>> >>> >> > > > > >> > > > > >> > > > need
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > maturing"
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > on
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> branch.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The
> > lack
> > > of
> > > >>>> a
> > > >>>> >>> >> concrete
> > > >>>> >>> >> > > plan
> > > >>>> >>> >> > > > > for
> > > >>>> >>> >> > > > > >> > 2.0
> > > >>>> >>> >> > > > > >> > > > > keeps
> > > >>>> >>> >> > > > > >> > > > > >> me
> > > >>>> >>> >> > > > > >> > > > > >> > > from
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > considering
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > these
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> things
> > > >>>> blocker
> > > >>>> >>> at
> > > >>>> >>> >> the
> > > >>>> >>> >> > > > moment.
> > > >>>> >>> >> > > > > >> But
> > > >>>> >>> >> > > > > >> > I
> > > >>>> >>> >> > > > > >> > > > know
> > > >>>> >>> >> > > > > >> > > > > >> > first
> > > >>>> >>> >> > > > > >> > > > > >> > > > hand
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> how
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > much
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > trouble
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > folks
> > > have
> > > >>>> had
> > > >>>> >>> with
> > > >>>> >>> >> > other
> > > >>>> >>> >> > > > > >> features
> > > >>>> >>> >> > > > > >> > > > that
> > > >>>> >>> >> > > > > >> > > > > >> have
> > > >>>> >>> >> > > > > >> > > > > >> > > gone
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> into
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > downstream
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> facing
> > > >>>> releases
> > > >>>> >>> >> without
> > > >>>> >>> >> > > > > >> robustness
> > > >>>> >>> >> > > > > >> > > > > checks
> > > >>>> >>> >> > > > > >> > > > > >> > (i.e.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > replication),
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > and
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > I'm
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > concerned
> > > >>>> about
> > > >>>> >>> what
> > > >>>> >>> >> > > we're
> > > >>>> >>> >> > > > > >> setting
> > > >>>> >>> >> > > > > >> > > up
> > > >>>> >>> >> > > > > >> > > > if
> > > >>>> >>> >> > > > > >> > > > > >> 2.0
> > > >>>> >>> >> > > > > >> > > > > >> > > goes
> > > >>>> >>> >> > > > > >> > > > > >> > > > > out
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > with
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> this
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> feature
> > > in
> > > >>>> its
> > > >>>> >>> >> current
> > > >>>> >>> >> > > > state.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > --
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > Best
> regards,
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >    - Andy
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > Problems
> > > worthy of
> > > >>>> >>> attack
> > > >>>> >>> >> > prove
> > > >>>> >>> >> > > > > their
> > > >>>> >>> >> > > > > >> > > worth
> > > >>>> >>> >> > > > > >> > > > by
> > > >>>> >>> >> > > > > >> > > > > >> > > hitting
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> back. -
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > Piet
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > Hein
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > (via Tom
> > White)
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > --
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > Best regards,
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >    - Andy
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > Problems worthy
> > of
> > > >>>> attack
> > > >>>> >>> >> prove
> > > >>>> >>> >> > > their
> > > >>>> >>> >> > > > > >> worth
> > > >>>> >>> >> > > > > >> > by
> > > >>>> >>> >> > > > > >> > > > > >> hitting
> > > >>>> >>> >> > > > > >> > > > > >> > > > back.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > -
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > Piet
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> Hein
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > (via Tom White)
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >>
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >>
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > >
> > > >>>> >>> >> > > > > >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> >
> > > >>>> >>> >> > > > > >> > > > > >>
> > > >>>> >>> >> > > > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > > --
> > > >>>> >>> >> > > > > >> > > > > busbey
> > > >>>> >>> >> > > > > >> > > > >
> > > >>>> >>> >> > > > > >> > > >
> > > >>>> >>> >> > > > > >> > >
> > > >>>> >>> >> > > > > >> >
> > > >>>> >>> >> > > > > >>
> > > >>>> >>> >> > > > > >
> > > >>>> >>> >> > > > > >
> > > >>>> >>> >> > > > >
> > > >>>> >>> >> > > >
> > > >>>> >>> >> > >
> > > >>>> >>> >> >
> > > >>>> >>> >>
> > > >>>> >>> >>
> > > >>>> >>> >>
> > > >>>> >>> >> --
> > > >>>> >>> >>
> > > >>>> >>> >> -- Appy
> > > >>>> >>> >>
> > > >>>> >>> >
> > > >>>> >>> >
> > > >>>> >>>
> > > >>>> >>
> > > >>>> >>
> > > >>>> >
> > > >>>>
> > > >>>
> > > >>>
> > > >>
> > >
> > >
> > >
> > > --
> > > busbey
> > >
> > >
> >
>
>
>
> --
>
> -- Appy
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Ted Yu <yu...@gmail.com>]

It is not rebased - but all backup / restore functionality is there.

You can use mega patch v29 over HBASE-14123 which applies to master branch.

Cheers

On Thu, Oct 13, 2016 at 3:51 PM, Apekshit Sharma <ap...@cloudera.com> wrote:

> Is HBASE-7912 the feature branch?
> If yes, has it not been rebased to incorporate latest master changes yet?
>
> ~/apache/hbase  (master) → g ck apache/HBASE-7912
> warning: refname 'apache/HBASE-7912' is ambiguous.
> Switched to branch 'apache/HBASE-7912'
> ~/apache/hbase  (apache/HBASE-7912) → g log HEAD..apache/master | grep
> "^commit" | wc -l
>      917
>
>
> On Thu, Oct 13, 2016 at 7:01 AM, Ted Yu <yu...@gmail.com> wrote:
>
> > Once Stack's comments are addressed, we can change the master build to
> > using Maven 3.3.3
> >
> > I will log JIRA for modifying refguide on the upgrade.
> >
> > Cheers
> >
> > On Thu, Oct 13, 2016 at 6:42 AM, Sean Busbey <bu...@cloudera.com>
> wrote:
> >
> > > Are we requiring maven 3.3.z now? historically our required maven
> > > version has been 3.0.5. I don't have an objection to changing it, per
> > > se, but I'd like to make sure our docs and CI builds get updated
> > > properly.
> > >
> > > On Wed, Oct 12, 2016 at 4:03 PM, Stack <st...@duboce.net> wrote:
> > > > On Wed, Oct 12, 2016 at 1:58 PM, Stack <st...@duboce.net> wrote:
> > > >
> > > >> Fails again when I do this:
> > > >>
> > > >> $ mvn clean install -DskipTests assembly:single
> > > >> ....
> > > >>
> > > >>
> > > > Ok. Works if I use mvn 3.3.x.
> > > > St.Ack
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >>
> > > >> [INFO] ------------------------------------------------------------
> > > >> ------------
> > > >> [INFO] Building Apache HBase - Server 2.0.0-SNAPSHOT
> > > >> [INFO] ------------------------------------------------------------
> > > >> ------------
> > > >> Downloading: file:/home/stack/hbase.git/hbase-server/src/main/site/
> > > >> resources/repo/org/apache/hadoop/hadoop-distcp/2.7.1/
> > > >> hadoop-distcp-2.7.1.pom
> > > >> Exception in thread "pool-1-thread-1" ------------------------------
> > > >> ---------------------
> > > >> constituent[0]: file:/usr/share/maven/lib/aether-spi.jar
> > > >> constituent[1]: file:/usr/share/maven/lib/
> > > maven-settings-builder-3.x.jar
> > > >> constituent[2]: file:/usr/share/maven/lib/maven-artifact-3.x.jar
> > > >> constituent[3]: file:/usr/share/maven/lib/commons-httpclient.jar
> > > >> constituent[4]: file:/usr/share/maven/lib/wagon-provider-api.jar
> > > >> constituent[5]: file:/usr/share/maven/lib/maven-core-3.x.jar
> > > >> constituent[6]: file:/usr/share/maven/lib/aether-impl.jar
> > > >> constituent[7]: file:/usr/share/maven/lib/sisu-inject-bean.jar
> > > >> constituent[8]: file:/usr/share/maven/lib/sisu-inject-plexus.jar
> > > >> constituent[9]: file:/usr/share/maven/lib/wagon-file.jar
> > > >> constituent[10]: file:/usr/share/maven/lib/guava.jar
> > > >> constituent[11]: file:/usr/share/maven/lib/maven-compat-3.x.jar
> > > >> constituent[12]: file:/usr/share/maven/lib/maven-model-3.x.jar
> > > >> constituent[13]: file:/usr/share/maven/lib/plexus-interpolation.jar
> > > >> constituent[14]: file:/usr/share/maven/lib/commons-codec.jar
> > > >> constituent[15]: file:/usr/share/maven/lib/aether-util.jar
> > > >> constituent[16]: file:/usr/share/maven/lib/aether-api.jar
> > > >> constituent[17]: file:/usr/share/maven/lib/
> > maven-model-builder-3.x.jar
> > > >> constituent[18]: file:/usr/share/maven/lib/
> > > maven-repository-metadata-3.x.
> > > >> jar
> > > >> constituent[19]: file:/usr/share/maven/lib/maven-settings-3.x.jar
> > > >> constituent[20]: file:/usr/share/maven/lib/
> > > plexus-component-annotations.
> > > >> jar
> > > >> constituent[21]: file:/usr/share/maven/lib/commons-cli.jar
> > > >> constituent[22]: file:/usr/share/maven/lib/maven-embedder-3.x.jar
> > > >> constituent[23]: file:/usr/share/maven/lib/maven-plugin-api-3.x.jar
> > > >> constituent[24]: file:/usr/share/maven/lib/wagon-http-shaded.jar
> > > >> constituent[25]: file:/usr/share/maven/lib/commons-logging.jar
> > > >> constituent[26]: file:/usr/share/maven/lib/
> plexus-sec-dispatcher.jar
> > > >> constituent[27]: file:/usr/share/maven/lib/
> aether-connector-wagon.jar
> > > >> constituent[28]: file:/usr/share/maven/lib/plexus-utils.jar
> > > >> constituent[29]: file:/usr/share/maven/lib/sisu-guice.jar
> > > >> constituent[30]: file:/usr/share/maven/lib/plexus-cipher.jar
> > > >> constituent[31]: file:/usr/share/maven/lib/
> > > maven-aether-provider-3.x.jar
> > > >> ---------------------------------------------------
> > > >> java.lang.NoClassDefFoundError: org/apache/commons/lang/StringUtils
> > > >> at org.apache.maven.wagon.providers.file.FileWagon.
> > > resolveDestinationPath(
> > > >> FileWagon.java:206)
> > > >> at org.apache.maven.wagon.providers.file.FileWagon.
> > > >> resourceExists(FileWagon.java:265)
> > > >> at org.sonatype.aether.connector.wagon.WagonRepositoryConnector$
> > > >> GetTask.run(WagonRepositoryConnector.java:577)
> > > >> at org.sonatype.aether.util.concurrency.
> RunnableErrorForwarder$1.run(
> > > >> RunnableErrorForwarder.java:60)
> > > >> at java.util.concurrent.ThreadPoolExecutor.runWorker(
> > > >> ThreadPoolExecutor.java:1142)
> > > >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> > > >> ThreadPoolExecutor.java:617)
> > > >> at java.lang.Thread.run(Thread.java:745)
> > > >> Caused by: java.lang.ClassNotFoundException:
> org.apache.commons.lang.
> > > >> StringUtils
> > > >> at org.codehaus.plexus.classworlds.strategy.
> > > SelfFirstStrategy.loadClass(
> > > >> SelfFirstStrategy.java:50)
> > > >> at org.codehaus.plexus.classworlds.realm.ClassRealm.
> > > >> unsynchronizedLoadClass(ClassRealm.java:259)
> > > >> at org.codehaus.plexus.classworlds.realm.ClassRealm.
> > > >> loadClass(ClassRealm.java:235)
> > > >> at org.codehaus.plexus.classworlds.realm.ClassRealm.
> > > >> loadClass(ClassRealm.java:227)
> > > >> ... 7 more
> > > >> Exception in thread "main" java.lang.NoClassDefFoundError:
> > > >> org/apache/commons/lang/StringUtils
> > > >> at org.apache.maven.wagon.providers.file.FileWagon.
> > > resolveDestinationPath(
> > > >> FileWagon.java:206)
> > > >> at org.apache.maven.wagon.providers.file.FileWagon.
> > > >> resourceExists(FileWagon.java:265)
> > > >> at org.sonatype.aether.connector.wagon.WagonRepositoryConnector$
> > > >> GetTask.run(WagonRepositoryConnector.java:577)
> > > >> at org.sonatype.aether.util.concurrency.
> RunnableErrorForwarder$1.run(
> > > >> RunnableErrorForwarder.java:60)
> > > >> at java.util.concurrent.ThreadPoolExecutor.runWorker(
> > > >> ThreadPoolExecutor.java:1142)
> > > >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> > > >> ThreadPoolExecutor.java:617)
> > > >> at java.lang.Thread.run(Thread.java:745)
> > > >> Caused by: java.lang.ClassNotFoundException:
> org.apache.commons.lang.
> > > >> StringUtils
> > > >> at org.codehaus.plexus.classworlds.strategy.
> > > SelfFirstStrategy.loadClass(
> > > >> SelfFirstStrategy.java:50)
> > > >> at org.codehaus.plexus.classworlds.realm.ClassRealm.
> > > >> unsynchronizedLoadClass(ClassRealm.java:259)
> > > >> at org.codehaus.plexus.classworlds.realm.ClassRealm.
> > > >> loadClass(ClassRealm.java:235)
> > > >> at org.codehaus.plexus.classworlds.realm.ClassRealm.
> > > >> loadClass(ClassRealm.java:227)
> > > >>
> > > >>
> > > >>
> > > >> Does it work for you lot? This is branch checkout.
> > > >>
> > > >> I want assembly to work so I can distribute the build around a
> > cluster.
> > > >>
> > > >> Thanks,
> > > >> St.Ack.
> > > >>
> > > >>
> > > >>
> > > >> On Wed, Oct 12, 2016 at 1:52 PM, Stack <st...@duboce.net> wrote:
> > > >>
> > > >>> I did a super-clean and recheck out. Now it works. Sorry for the
> > noise.
> > > >>> St.Ack
> > > >>>
> > > >>> On Wed, Oct 12, 2016 at 1:10 PM, Vladimir Rodionov <
> > > >>> vladrodionov@gmail.com> wrote:
> > > >>>
> > > >>>> checked out HBASE-7912
> > > >>>>
> > > >>>> ran:
> > > >>>>
> > > >>>> mvn clean install -DskipTests
> > > >>>>
> > > >>>> successfully.
> > > >>>>
> > > >>>> -Vlad
> > > >>>>
> > > >>>> On Wed, Oct 12, 2016 at 1:01 PM, Vladimir Rodionov <
> > > >>>> vladrodionov@gmail.com>
> > > >>>> wrote:
> > > >>>>
> > > >>>> > I usually use:
> > > >>>> >
> > > >>>> > mvn clean install -DskipTests
> > > >>>> >
> > > >>>> > and it works.
> > > >>>> >
> > > >>>> > On Wed, Oct 12, 2016 at 1:01 PM, Vladimir Rodionov <
> > > >>>> vladrodionov@gmail.com
> > > >>>> > > wrote:
> > > >>>> >
> > > >>>> >> Michael,
> > > >>>> >>
> > > >>>> >> you can try master + latest patch on HBASE-14123 (v29). No need
> > to
> > > use
> > > >>>> >> HBASE-7912 branch. I will double check HBASE-7912.
> > > >>>> >>
> > > >>>> >> -Vlad
> > > >>>> >>
> > > >>>> >> On Wed, Oct 12, 2016 at 12:31 PM, Stack <st...@duboce.net>
> > wrote:
> > > >>>> >>
> > > >>>> >>> More info:
> > > >>>> >>>
> > > >>>> >>> stack@ve0524:~/hbase.git$ git checkout origin/HBASE-7912 -b
> > > 7912v2
> > > >>>> >>> Branch 7912v2 set up to track remote branch HBASE-7912 from
> > > origin.
> > > >>>> >>> Switched to a new branch '7912v2'
> > > >>>> >>> stack@ve0524:~/hbase.git$ java -version
> > > >>>> >>> java version "1.8.0_101"
> > > >>>> >>> Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
> > > >>>> >>> Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed
> mode)
> > > >>>> >>> stack@ve0524:~/hbase.git$ mvn clean install -DskipTests &>
> > > >>>> /tmp/out.txt
> > > >>>> >>>
> > > >>>> >>> ...
> > > >>>> >>>
> > > >>>> >>> St.Ack
> > > >>>> >>>
> > > >>>> >>>
> > > >>>> >>> On Wed, Oct 12, 2016 at 12:29 PM, Stack <st...@duboce.net>
> > wrote:
> > > >>>> >>>
> > > >>>> >>> > Interesting. When I try it fails w/ below:
> > > >>>> >>> >
> > > >>>> >>> > [INFO] 26 warnings
> > > >>>> >>> > 322 [INFO] ------------------------------
> > > >>>> >>> -------------------------------
> > > >>>> >>> > 323 [INFO] ------------------------------
> > > >>>> >>> -------------------------------
> > > >>>> >>> > 324 [ERROR] COMPILATION ERROR :
> > > >>>> >>> > 325 [INFO] ------------------------------
> > > >>>> >>> -------------------------------
> > > >>>> >>> > 326 [ERROR] /home/stack/hbase.git/hbase-
> > > common/src/main/java/org/
> > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/
> RowIndexCodecV2.j
> > > >>>> ava:[48,8]
> > > >>>> >>> > org.apache.hadoop.hbase.io.encoding.rowindexV2.
> > RowIndexCodecV2
> > > is
> > > >>>> not
> > > >>>> >>> > abstract and does not override abstract method
> > > createSeeker(org.ap
> > > >>>> >>> >  ache.hadoop.hbase.CellComparator,org.apache.
> hadoop.hbase.io.
> > > >>>> >>> encoding.HFileBlockDecodingContext)
> > > >>>> >>> > in org.apache.hadoop.hbase.io.encoding.DataBlockEncoder
> > > >>>> >>> > 327 [ERROR] /home/stack/hbase.git/hbase-
> > > common/src/main/java/org/
> > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/
> RowIndexCodecV2.j
> > > >>>> >>> ava:[143,3]
> > > >>>> >>> > method does not override or implement a method from a
> > supertype
> > > >>>> >>> > 328 [ERROR] /home/stack/hbase.git/hbase-
> > > common/src/main/java/org/
> > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/
> RowIndexCodecV2.j
> > > >>>> >>> ava:[147,29]
> > > >>>> >>> > incompatible types: java.nio.ByteBuffer cannot be converted
> to
> > > >>>> >>> > org.apache.hadoop.hbase.nio.ByteBuff
> > > >>>> >>> > 329 [ERROR] /home/stack/hbase.git/hbase-
> > > common/src/main/java/org/
> > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/
> RowIndexCodecV2.j
> > > >>>> >>> ava:[148,33]
> > > >>>> >>> > cannot find symbol
> > > >>>> >>> > 330   symbol:   method getKeyDeepCopy()
> > > >>>> >>> > 331   location: variable seeker of type
> > > org.apache.hadoop.hbase.io
> > > >>>> .
> > > >>>> >>> > encoding.DataBlockEncoder.EncodedSeeker
> > > >>>> >>> > 332 [ERROR] /home/stack/hbase.git/hbase-
> > > common/src/main/java/org/
> > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/
> RowIndexCodecV2.j
> > > >>>> >>> ava:[153,3]
> > > >>>> >>> > method does not override or implement a method from a
> > supertype
> > > >>>> >>> > 333 [ERROR] /home/stack/hbase.git/hbase-
> > > common/src/main/java/org/
> > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV1/
> RowIndexCodecV1.j
> > > >>>> ava:[45,8]
> > > >>>> >>> > org.apache.hadoop.hbase.io.encoding.rowindexV1.
> > RowIndexCodecV1
> > > is
> > > >>>> not
> > > >>>> >>> > abstract and does not override abstract method
> > > createSeeker(org.ap
> > > >>>> >>> >  ache.hadoop.hbase.CellComparator,org.apache.
> hadoop.hbase.io.
> > > >>>> >>> encoding.HFileBlockDecodingContext)
> > > >>>> >>> > in org.apache.hadoop.hbase.io.encoding.DataBlockEncoder
> > > >>>> >>> > 334 [ERROR] /home/stack/hbase.git/hbase-
> > > common/src/main/java/org/
> > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV1/
> RowIndexCodecV1.j
> > > >>>> >>> ava:[145,3]
> > > >>>> >>> > method does not override or implement a method from a
> > supertype
> > > >>>> >>> > 335 [ERROR] /home/stack/hbase.git/hbase-
> > > common/src/main/java/org/
> > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV1/
> RowIndexCodecV1.j
> > > >>>> >>> ava:[158,3]
> > > >>>> >>> > method does not override or implement a method from a
> > supertype
> > > >>>> >>> > 336 [ERROR] /home/stack/hbase.git/hbase-
> > > common/src/main/java/org/
> > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/
> RowIndexSeekerV2.
> > > >>>> >>> java:[46,8]
> > > >>>> >>> > org.apache.hadoop.hbase.io.encoding.rowindexV2.
> > RowIndexSeekerV2
> > > >>>> is not
> > > >>>> >>> > abstract and does not override abstract method
> > compareKey(org.ap
> > > >>>> >>> >  ache.hadoop.hbase.CellComparator,org.apache.
> > hadoop.hbase.Cell)
> > > in
> > > >>>> >>> > org.apache.hadoop.hbase.io.encoding.DataBlockEncoder.
> > > EncodedSeeker
> > > >>>> >>> > 337 [ERROR] /home/stack/hbase.git/hbase-
> > > common/src/main/java/org/
> > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/
> RowIndexSeekerV2.
> > > >>>> >>> java:[79,3]
> > > >>>> >>> > method does not override or implement a method from a
> > supertype
> > > >>>> >>> > 338 [ERROR] /home/stack/hbase.git/hbase-
> > > common/src/main/java/org/
> > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/
> RowIndexSeekerV2.
> > > >>>> >>> java:[117,3]
> > > >>>> >>> > method does not override or implement a method from a
> > supertype
> > > >>>> >>> > 339 [ERROR] /home/stack/hbase.git/hbase-
> > > common/src/main/java/org/
> > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/
> RowIndexSeekerV2.
> > > >>>> >>> java:[190,3]
> > > >>>> >>> > method does not override or implement a method from a
> > supertype
> > > >>>> >>> > 340 [ERROR] /home/stack/hbase.git/hbase-
> > > common/src/main/java/org/
> > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/
> RowIndexSeekerV2.
> > > >>>> >>> java:[214,3]
> > > >>>> >>> > method does not override or implement a method from a
> > supertype
> > > >>>> >>> > 341 [ERROR] /home/stack/hbase.git/hbase-
> > > common/src/main/java/org/
> > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/
> RowIndexSeekerV2.
> > > >>>> >>> java:[349,3]
> > > >>>> >>> > method does not override or implement a method from a
> > supertype
> > > >>>> >>> > 342 [ERROR] /home/stack/hbase.git/hbase-
> > > common/src/main/java/org/
> > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/
> RowIndexSeekerV2.
> > > >>>> >>> java:[355,3]
> > > >>>> >>> > method does not override or implement a method from a
> > supertype
> > > >>>> >>> > 343 [ERROR] /home/stack/hbase.git/hbase-
> > > common/src/main/java/org/
> > > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/
> RowIndexSeekerV2.
> > > >>>> >>> java:[421,36]
> > > >>>> >>> > no suitable method found for uncompressTags(java.nio.
> > > >>>> >>> > ByteBuffer,byte[],int,int)
> > > >>>> >>> > 344     method org.apache.hadoop.hbase.io.
> > > TagCompressionContext.
> > > >>>> >>> > uncompressTags(java.io.InputStream,byte[],int,int) is not
> > > >>>> applicable
> > > >>>> >>> > 345       (argument mismatch; java.nio.ByteBuffer cannot be
> > > >>>> converted
> > > >>>> >>> to
> > > >>>> >>> > java.io.InputStream)
> > > >>>> >>> > 346     method org.apache.hadoop.hbase.io.
> > > TagCompressionContext.
> > > >>>> >>> > uncompressTags(org.apache.hadoop.hbase.nio.ByteBuff,
> > > byte[],int,int)
> > > >>>> is
> > > >>>> >>> > not applicable
> > > >>>> >>> > 347       (argument mismatch; java.nio.ByteBuffer cannot be
> > > >>>> converted
> > > >>>> >>> to
> > > >>>> >>> > org.apache.hadoop.hbase.nio.ByteBuff)
> > > >>>> >>> >
> > > >>>> >>> > ....
> > > >>>> >>> >
> > > >>>> >>> > St.Ack
> > > >>>> >>> >
> > > >>>> >>> > On Tue, Oct 11, 2016 at 5:12 PM, Apekshit Sharma <
> > > >>>> appy@cloudera.com>
> > > >>>> >>> > wrote:
> > > >>>> >>> >
> > > >>>> >>> >> @stack, it compiled for me.
> > > >>>> >>> >>
> > > >>>> >>> >> Also tried few commands, and have to say, it's well
> designed
> > > from
> > > >>>> user
> > > >>>> >>> >> commands perspective.
> > > >>>> >>> >>
> > > >>>> >>> >> On Tue, Oct 11, 2016 at 5:08 PM, Stack <st...@duboce.net>
> > > wrote:
> > > >>>> >>> >>
> > > >>>> >>> >> > On Tue, Oct 11, 2016 at 1:50 PM, Vladimir Rodionov <
> > > >>>> >>> >> vladrodionov@gmail.com
> > > >>>> >>> >> > >
> > > >>>> >>> >> > wrote:
> > > >>>> >>> >> >
> > > >>>> >>> >> > > Michael,
> > > >>>> >>> >> > >
> > > >>>> >>> >> > > Its in HBASE-7912
> > > >>>> >>> >> > >
> > > >>>> >>> >> > > This is tip of git log:
> > > >>>> >>> >> > >
> > > >>>> >>> >> > > commit a072f6f49a26a7259ff2aaef6cb56d85eb592482
> > > >>>> >>> >> > > Author: Frank Welsch <fw...@jps.net>
> > > >>>> >>> >> > > Date:   Fri Sep 23 18:00:42 2016 -0400
> > > >>>> >>> >> > >
> > > >>>> >>> >> > >     HBASE-16574 Book updates for backup and restore
> > > >>>> >>> >> > >
> > > >>>> >>> >> > > commit b14e2ab1c24e65ff88dd4c579acf83cb4ed0605e
> > > >>>> >>> >> > > Author: tedyu <yu...@gmail.com>
> > > >>>> >>> >> > > Date:   Wed Oct 5 16:29:40 2016 -0700
> > > >>>> >>> >> > >
> > > >>>> >>> >> > >     HBASE-16727 Backup refactoring: remove MR
> > dependencies
> > > >>>> from
> > > >>>> >>> >> HMaster
> > > >>>> >>> >> > > (Vladimir Rodionov)
> > > >>>> >>> >> > >
> > > >>>> >>> >> > >
> > > >>>> >>> >> > Thanks. I have that. I tried it and it doesn't compile
> for
> > > me.
> > > >>>> Does
> > > >>>> >>> it
> > > >>>> >>> >> > compile for you?
> > > >>>> >>> >> > Thanks,
> > > >>>> >>> >> > M
> > > >>>> >>> >> >
> > > >>>> >>> >> >
> > > >>>> >>> >> >
> > > >>>> >>> >> > > -Vlad
> > > >>>> >>> >> > >
> > > >>>> >>> >> > > On Tue, Oct 11, 2016 at 1:44 PM, Stack <
> stack@duboce.net
> > >
> > > >>>> wrote:
> > > >>>> >>> >> > >
> > > >>>> >>> >> > > > Which branch do I check out to try it? HBASE-7912 is
> > not
> > > >>>> it. I
> > > >>>> >>> don't
> > > >>>> >>> >> > see
> > > >>>> >>> >> > > an
> > > >>>> >>> >> > > > HBASE-16727...
> > > >>>> >>> >> > > > Thanks,
> > > >>>> >>> >> > > > M
> > > >>>> >>> >> > > >
> > > >>>> >>> >> > > >
> > > >>>> >>> >> > > > On Tue, Oct 11, 2016 at 11:41 AM, Vladimir Rodionov <
> > > >>>> >>> >> > > > vladrodionov@gmail.com>
> > > >>>> >>> >> > > > wrote:
> > > >>>> >>> >> > > >
> > > >>>> >>> >> > > > > The last patch is on review board:
> > > >>>> >>> >> > > > > https://reviews.apache.org/r/52748
> > > >>>> >>> >> > > > >
> > > >>>> >>> >> > > > > On Mon, Oct 10, 2016 at 3:52 PM, Vladimir Rodionov
> <
> > > >>>> >>> >> > > > vladrodionov@gmail.com
> > > >>>> >>> >> > > > > >
> > > >>>> >>> >> > > > > wrote:
> > > >>>> >>> >> > > > >
> > > >>>> >>> >> > > > > > >> How hard to put in an hbase-backup module?
> > > >>>> hbase-server
> > > >>>> >>> is
> > > >>>> >>> >> fat
> > > >>>> >>> >> > > > enough
> > > >>>> >>> >> > > > > > >> already. Could be done as a follow-up.
> > > >>>> >>> >> > > > > >
> > > >>>> >>> >> > > > > > https://issues.apache.org/
> jira/browse/HBASE-16727?
> > > >>>> >>> >> > > > > > focusedCommentId=15531237&
> page=com.atlassian.jira.
> > > >>>> >>> >> > > > > > plugin.system.issuetabpanels:c
> > > >>>> >>> omment-tabpanel#comment-15531237
> > > >>>> >>> >> > > > > >
> > > >>>> >>> >> > > > > > Can we do merge first? Then we can discuss
> separate
> > > >>>> module.
> > > >>>> >>> >> > > > > >
> > > >>>> >>> >> > > > > > -Vlad
> > > >>>> >>> >> > > > > >
> > > >>>> >>> >> > > > > > On Mon, Oct 10, 2016 at 3:44 PM, Ted Yu <
> > > >>>> >>> yuzhihong@gmail.com>
> > > >>>> >>> >> > wrote:
> > > >>>> >>> >> > > > > >
> > > >>>> >>> >> > > > > >> Looks like the first quote was cut off.
> > > >>>> >>> >> > > > > >> The original sentence was:
> > > >>>> >>> >> > > > > >>
> > > >>>> >>> >> > > > > >> bq. no mapreduce job launched from master or
> > region
> > > >>>> server.
> > > >>>> >>> >> > > > > >>
> > > >>>> >>> >> > > > > >> mapreduce job is launched from the node where
> > > command
> > > >>>> line
> > > >>>> >>> >> tool is
> > > >>>> >>> >> > > > run.
> > > >>>> >>> >> > > > > >>
> > > >>>> >>> >> > > > > >> On Mon, Oct 10, 2016 at 3:38 PM, Stack <
> > > >>>> stack@duboce.net>
> > > >>>> >>> >> wrote:
> > > >>>> >>> >> > > > > >>
> > > >>>> >>> >> > > > > >> > bq. launched from master or region server.
> > > >>>> >>> >> > > > > >> >
> > > >>>> >>> >> > > > > >> > What does this mean please? Has to be run from
> > > >>>> Master or
> > > >>>> >>> >> > > > RegionServer?
> > > >>>> >>> >> > > > > >> Can
> > > >>>> >>> >> > > > > >> > it be run from another node altogether?
> > > >>>> >>> >> > > > > >> >
> > > >>>> >>> >> > > > > >> > On Mon, Oct 10, 2016 at 1:44 PM, Vladimir
> > > Rodionov <
> > > >>>> >>> >> > > > > >> vladrodionov@gmail.com
> > > >>>> >>> >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > wrote:
> > > >>>> >>> >> > > > > >> >
> > > >>>> >>> >> > > > > >> > > >> mapreduce dependency has been moved to
> > client
> > > >>>> side
> > > >>>> >>> - no
> > > >>>> >>> >> > > > mapreduce
> > > >>>> >>> >> > > > > >> job
> > > >>>> >>> >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > > 1. We have no code in the client module
> > anymore,
> > > >>>> due to
> > > >>>> >>> >> > > dependency
> > > >>>> >>> >> > > > > on
> > > >>>> >>> >> > > > > >> > > internal server API (HFile and WAL access).
> > > >>>> >>> >> > > > > >> > > 2. Backup/ restore are client - driven
> > > operations,
> > > >>>> but
> > > >>>> >>> all
> > > >>>> >>> >> the
> > > >>>> >>> >> > > > code
> > > >>>> >>> >> > > > > >> > resides
> > > >>>> >>> >> > > > > >> > > in the server module
> > > >>>> >>> >> > > > > >> > >
> > > >>>> >>> >> > > > > >> >
> > > >>>> >>> >> > > > > >> > How hard to put in an hbase-backup module?
> > > >>>> hbase-server
> > > >>>> >>> is
> > > >>>> >>> >> fat
> > > >>>> >>> >> > > > enough
> > > >>>> >>> >> > > > > >> > already. Could be done as a follow-up.
> > > >>>> >>> >> > > > > >> >
> > > >>>> >>> >> > > > > >> > Thanks,
> > > >>>> >>> >> > > > > >> > St.Ack
> > > >>>> >>> >> > > > > >> >
> > > >>>> >>> >> > > > > >> >
> > > >>>> >>> >> > > > > >> >
> > > >>>> >>> >> > > > > >> > > 3. No MR in Master, no procedure - driven
> > > >>>> execution.
> > > >>>> >>> >> > > > > >> > > 4. Old good MR from command-line.
> > > >>>> >>> >> > > > > >> > > 5. Security was simplified and now only
> > > super-user
> > > >>>> is
> > > >>>> >>> >> allowed
> > > >>>> >>> >> > to
> > > >>>> >>> >> > > > run
> > > >>>> >>> >> > > > > >> > > backup/restores.
> > > >>>> >>> >> > > > > >> > > 6. HBase Backup API was gone due to 1. Now
> > only
> > > >>>> >>> >> command-line
> > > >>>> >>> >> > > > access
> > > >>>> >>> >> > > > > to
> > > >>>> >>> >> > > > > >> > > backup tools.
> > > >>>> >>> >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > > These consequences of refactoring has been
> > > >>>> discussed in
> > > >>>> >>> >> > > > HBASE-16727.
> > > >>>> >>> >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > > -Vlad
> > > >>>> >>> >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > > On Mon, Oct 10, 2016 at 1:31 PM, Ted Yu <
> > > >>>> >>> >> yuzhihong@gmail.com>
> > > >>>> >>> >> > > > > wrote:
> > > >>>> >>> >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > > > Reviving this thread.
> > > >>>> >>> >> > > > > >> > > >
> > > >>>> >>> >> > > > > >> > > > The following has taken place:
> > > >>>> >>> >> > > > > >> > > >
> > > >>>> >>> >> > > > > >> > > > mapreduce dependency has been moved to
> > client
> > > >>>> side -
> > > >>>> >>> no
> > > >>>> >>> >> > > > mapreduce
> > > >>>> >>> >> > > > > >> job
> > > >>>> >>> >> > > > > >> > > > launched from master or region server.
> > > >>>> >>> >> > > > > >> > > > document patch (HBASE-16574) has been
> > > integrated.
> > > >>>> >>> >> > > > > >> > > > Updated mega patch has been attached to
> > > >>>> HBASE-14123:
> > > >>>> >>> this
> > > >>>> >>> >> > > covers
> > > >>>> >>> >> > > > > the
> > > >>>> >>> >> > > > > >> > > > refactor in #1 above and the protobuf 3
> > merge.
> > > >>>> >>> >> > > > > >> > > >
> > > >>>> >>> >> > > > > >> > > > If community has more feedback on the
> merge
> > > >>>> >>> proposal, I
> > > >>>> >>> >> > would
> > > >>>> >>> >> > > > love
> > > >>>> >>> >> > > > > >> to
> > > >>>> >>> >> > > > > >> > > hear
> > > >>>> >>> >> > > > > >> > > > it.
> > > >>>> >>> >> > > > > >> > > >
> > > >>>> >>> >> > > > > >> > > > Thanks
> > > >>>> >>> >> > > > > >> > > >
> > > >>>> >>> >> > > > > >> > > > On Thu, Sep 22, 2016 at 10:31 AM, Sean
> > Busbey
> > > <
> > > >>>> >>> >> > > > > busbey@cloudera.com>
> > > >>>> >>> >> > > > > >> > > wrote:
> > > >>>> >>> >> > > > > >> > > >
> > > >>>> >>> >> > > > > >> > > > > I'd like to see the docs proposed on
> > > >>>> HBASE-16574
> > > >>>> >>> >> > integrated
> > > >>>> >>> >> > > > into
> > > >>>> >>> >> > > > > >> our
> > > >>>> >>> >> > > > > >> > > > > project's documentation prior to merge.
> > > >>>> >>> >> > > > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > > On Thu, Sep 22, 2016 at 9:02 AM, Ted Yu
> <
> > > >>>> >>> >> > > yuzhihong@gmail.com>
> > > >>>> >>> >> > > > > >> wrote:
> > > >>>> >>> >> > > > > >> > > > > > This feature can be marked
> experimental
> > > due
> > > >>>> to
> > > >>>> >>> some
> > > >>>> >>> >> > > > > limitations
> > > >>>> >>> >> > > > > >> > such
> > > >>>> >>> >> > > > > >> > > as
> > > >>>> >>> >> > > > > >> > > > > > security.
> > > >>>> >>> >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > > Your previous round of comments have
> > been
> > > >>>> >>> addressed.
> > > >>>> >>> >> > > > > >> > > > > > Command line tool has gone through:
> > > >>>> >>> >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > > HBASE-16620 Fix backup command-line
> tool
> > > >>>> >>> usability
> > > >>>> >>> >> > issues
> > > >>>> >>> >> > > > > >> > > > > > HBASE-16655 hbase backup describe with
> > > >>>> incorrect
> > > >>>> >>> >> backup
> > > >>>> >>> >> > id
> > > >>>> >>> >> > > > > >> results
> > > >>>> >>> >> > > > > >> > in
> > > >>>> >>> >> > > > > >> > > > NPE
> > > >>>> >>> >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > > The updated doc has been attached to
> > > >>>> HBASE-16574.
> > > >>>> >>> >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > > Cheers
> > > >>>> >>> >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > > On Thu, Sep 22, 2016 at 8:53 AM,
> Stack <
> > > >>>> >>> >> > stack@duboce.net>
> > > >>>> >>> >> > > > > >> wrote:
> > > >>>> >>> >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> On Wed, Sep 21, 2016 at 7:43 AM, Ted
> > Yu <
> > > >>>> >>> >> > > > yuzhihong@gmail.com
> > > >>>> >>> >> > > > > >
> > > >>>> >>> >> > > > > >> > > wrote:
> > > >>>> >>> >> > > > > >> > > > > >>
> > > >>>> >>> >> > > > > >> > > > > >> > Are there more (review) comments ?
> > > >>>> >>> >> > > > > >> > > > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> Are outstanding comments addressed?
> > > >>>> >>> >> > > > > >> > > > > >>
> > > >>>> >>> >> > > > > >> > > > > >> I don't see answer to my 'is this
> > > >>>> >>> experimental/will
> > > >>>> >>> >> it
> > > >>>> >>> >> > be
> > > >>>> >>> >> > > > > >> marked
> > > >>>> >>> >> > > > > >> > > > > >> experimental' question.
> > > >>>> >>> >> > > > > >> > > > > >>
> > > >>>> >>> >> > > > > >> > > > > >> I ran into some issues trying to use
> > the
> > > >>>> >>> feature and
> > > >>>> >>> >> > > > > suggested
> > > >>>> >>> >> > > > > >> > that
> > > >>>> >>> >> > > > > >> > > a
> > > >>>> >>> >> > > > > >> > > > > >> feature likes this needs polish else
> > > it'll
> > > >>>> just
> > > >>>> >>> rot,
> > > >>>> >>> >> > > > unused.
> > > >>>> >>> >> > > > > >> Has
> > > >>>> >>> >> > > > > >> > > > polish
> > > >>>> >>> >> > > > > >> > > > > >> been applied? All ready for another
> > > 'user'
> > > >>>> test?
> > > >>>> >>> >> > Suggest
> > > >>>> >>> >> > > > that
> > > >>>> >>> >> > > > > >> you
> > > >>>> >>> >> > > > > >> > > > update
> > > >>>> >>> >> > > > > >> > > > > >> here going forward for the benefit of
> > > those
> > > >>>> >>> trying
> > > >>>> >>> >> to
> > > >>>> >>> >> > > > follow
> > > >>>> >>> >> > > > > >> along
> > > >>>> >>> >> > > > > >> > > and
> > > >>>> >>> >> > > > > >> > > > > who
> > > >>>> >>> >> > > > > >> > > > > >> are not watching JIRA change fly-by.
> > > >>>> >>> >> > > > > >> > > > > >>
> > > >>>> >>> >> > > > > >> > > > > >> It looks like doc got a revision -- I
> > > have
> > > >>>> to
> > > >>>> >>> check
> > > >>>> >>> >> --
> > > >>>> >>> >> > to
> > > >>>> >>> >> > > > > take
> > > >>>> >>> >> > > > > >> on
> > > >>>> >>> >> > > > > >> > > > > >> suggestion made above but again,
> > suggest,
> > > >>>> that
> > > >>>> >>> this
> > > >>>> >>> >> > > thread
> > > >>>> >>> >> > > > > gets
> > > >>>> >>> >> > > > > >> > > > updated.
> > > >>>> >>> >> > > > > >> > > > > >>
> > > >>>> >>> >> > > > > >> > > > > >> Thanks,
> > > >>>> >>> >> > > > > >> > > > > >> St.Ack
> > > >>>> >>> >> > > > > >> > > > > >>
> > > >>>> >>> >> > > > > >> > > > > >>
> > > >>>> >>> >> > > > > >> > > > > >>
> > > >>>> >>> >> > > > > >> > > > > >> > Thanks
> > > >>>> >>> >> > > > > >> > > > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > On Tue, Sep 20, 2016 at 10:02 AM,
> > > Devaraj
> > > >>>> Das
> > > >>>> >>> <
> > > >>>> >>> >> > > > > >> > > ddas@hortonworks.com
> > > >>>> >>> >> > > > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > wrote:
> > > >>>> >>> >> > > > > >> > > > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > > Just reviving this thread. Thanks
> > > Sean,
> > > >>>> >>> Stack,
> > > >>>> >>> >> > Dima,
> > > >>>> >>> >> > > > and
> > > >>>> >>> >> > > > > >> > others
> > > >>>> >>> >> > > > > >> > > > for
> > > >>>> >>> >> > > > > >> > > > > the
> > > >>>> >>> >> > > > > >> > > > > >> > > thorough reviews and testing.
> > Thanks
> > > >>>> Ted and
> > > >>>> >>> >> Vlad
> > > >>>> >>> >> > for
> > > >>>> >>> >> > > > > >> taking
> > > >>>> >>> >> > > > > >> > > care
> > > >>>> >>> >> > > > > >> > > > of
> > > >>>> >>> >> > > > > >> > > > > >> the
> > > >>>> >>> >> > > > > >> > > > > >> > > feedback. Are we all good to do
> the
> > > >>>> merge
> > > >>>> >>> now?
> > > >>>> >>> >> > Rather
> > > >>>> >>> >> > > > do
> > > >>>> >>> >> > > > > >> > sooner
> > > >>>> >>> >> > > > > >> > > > than
> > > >>>> >>> >> > > > > >> > > > > >> > later.
> > > >>>> >>> >> > > > > >> > > > > >> > > ______________________________
> > > >>>> __________
> > > >>>> >>> >> > > > > >> > > > > >> > > From: saint.ack@gmail.com <
> > > >>>> >>> saint.ack@gmail.com>
> > > >>>> >>> >> on
> > > >>>> >>> >> > > > > behalf
> > > >>>> >>> >> > > > > >> of
> > > >>>> >>> >> > > > > >> > > > Stack
> > > >>>> >>> >> > > > > >> > > > > <
> > > >>>> >>> >> > > > > >> > > > > >> > > stack@duboce.net>
> > > >>>> >>> >> > > > > >> > > > > >> > > Sent: Monday, September 12, 2016
> > > 1:18 PM
> > > >>>> >>> >> > > > > >> > > > > >> > > To: HBase Dev List
> > > >>>> >>> >> > > > > >> > > > > >> > > Subject: Re: [DISCUSSION] Merge
> > > Backup /
> > > >>>> >>> >> Restore -
> > > >>>> >>> >> > > > Branch
> > > >>>> >>> >> > > > > >> > > > HBASE-7912
> > > >>>> >>> >> > > > > >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> > > On Mon, Sep 12, 2016 at 12:19 PM,
> > Ted
> > > >>>> Yu <
> > > >>>> >>> >> > > > > >> yuzhihong@gmail.com
> > > >>>> >>> >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > > > > wrote:
> > > >>>> >>> >> > > > > >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > Mega patch (rev 18) is on
> > > HBASE-14123.
> > > >>>> >>> >> > > > > >> > > > > >> > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > Please comment on HBASE-14123
> on
> > > how
> > > >>>> you
> > > >>>> >>> want
> > > >>>> >>> >> to
> > > >>>> >>> >> > > > > review.
> > > >>>> >>> >> > > > > >> > > > > >> > > >
> > > >>>> >>> >> > > > > >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> > > Yeah. That was my lost tab. Last
> rb
> > > was
> > > >>>> 6
> > > >>>> >>> months
> > > >>>> >>> >> > ago.
> > > >>>> >>> >> > > > > >> Suggest
> > > >>>> >>> >> > > > > >> > > > > updating
> > > >>>> >>> >> > > > > >> > > > > >> > it.
> > > >>>> >>> >> > > > > >> > > > > >> > > RB is pretty good for review.
> Patch
> > > is
> > > >>>> only
> > > >>>> >>> >> 1.5M so
> > > >>>> >>> >> > > > > should
> > > >>>> >>> >> > > > > >> be
> > > >>>> >>> >> > > > > >> > > > fine.
> > > >>>> >>> >> > > > > >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> > > St.Ack
> > > >>>> >>> >> > > > > >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > Thanks
> > > >>>> >>> >> > > > > >> > > > > >> > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > On Mon, Sep 12, 2016 at 12:15
> PM,
> > > >>>> Stack <
> > > >>>> >>> >> > > > > >> stack@duboce.net>
> > > >>>> >>> >> > > > > >> > > > wrote:
> > > >>>> >>> >> > > > > >> > > > > >> > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > On review of the 'patch', do
> I
> > > just
> > > >>>> >>> compare
> > > >>>> >>> >> the
> > > >>>> >>> >> > > > > branch
> > > >>>> >>> >> > > > > >> to
> > > >>>> >>> >> > > > > >> > > > > master or
> > > >>>> >>> >> > > > > >> > > > > >> > is
> > > >>>> >>> >> > > > > >> > > > > >> > > > > there a megapatch posted
> > > somewhere
> > > >>>> (I
> > > >>>> >>> think
> > > >>>> >>> >> I
> > > >>>> >>> >> > saw
> > > >>>> >>> >> > > > one
> > > >>>> >>> >> > > > > >> but
> > > >>>> >>> >> > > > > >> > it
> > > >>>> >>> >> > > > > >> > > > > seemed
> > > >>>> >>> >> > > > > >> > > > > >> > > stale
> > > >>>> >>> >> > > > > >> > > > > >> > > > > and then I 'lost' the tab).
> > Sorry
> > > >>>> for
> > > >>>> >>> dumb
> > > >>>> >>> >> > > > question.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > St.Ack
> > > >>>> >>> >> > > > > >> > > > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > On Mon, Sep 12, 2016 at 12:01
> > PM,
> > > >>>> Stack
> > > >>>> >>> <
> > > >>>> >>> >> > > > > >> stack@duboce.net
> > > >>>> >>> >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > > > > wrote:
> > > >>>> >>> >> > > > > >> > > > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > Late to the game. A few
> > > comments
> > > >>>> after
> > > >>>> >>> >> > > rereading
> > > >>>> >>> >> > > > > this
> > > >>>> >>> >> > > > > >> > > thread
> > > >>>> >>> >> > > > > >> > > > > as a
> > > >>>> >>> >> > > > > >> > > > > >> > > > 'user'.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > + Before merge, a
> user-facing
> > > >>>> feature
> > > >>>> >>> like
> > > >>>> >>> >> > this
> > > >>>> >>> >> > > > > >> should
> > > >>>> >>> >> > > > > >> > > work
> > > >>>> >>> >> > > > > >> > > > > (If
> > > >>>> >>> >> > > > > >> > > > > >> > this
> > > >>>> >>> >> > > > > >> > > > > >> > > is
> > > >>>> >>> >> > > > > >> > > > > >> > > > > "higher-bar
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > for new features", bring it
> > on
> > > --
> > > >>>> >>> smile).
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > + As a user, I tried the
> > branch
> > > >>>> with
> > > >>>> >>> tools
> > > >>>> >>> >> > > after
> > > >>>> >>> >> > > > > >> > reviewing
> > > >>>> >>> >> > > > > >> > > > the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > just-posted
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > doc. I had an 'interesting'
> > > >>>> experience
> > > >>>> >>> >> (left
> > > >>>> >>> >> > > > > >> comments up
> > > >>>> >>> >> > > > > >> > > on
> > > >>>> >>> >> > > > > >> > > > > >> > issue). I
> > > >>>> >>> >> > > > > >> > > > > >> > > > > think
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > the tooling/doc. important
> to
> > > get
> > > >>>> >>> right.
> > > >>>> >>> >> If
> > > >>>> >>> >> > it
> > > >>>> >>> >> > > > > breaks
> > > >>>> >>> >> > > > > >> > > easily
> > > >>>> >>> >> > > > > >> > > > > or
> > > >>>> >>> >> > > > > >> > > > > >> is
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > inconsistent (or lacks
> > > 'polish'),
> > > >>>> >>> >> operators
> > > >>>> >>> >> > > will
> > > >>>> >>> >> > > > > >> judge
> > > >>>> >>> >> > > > > >> > the
> > > >>>> >>> >> > > > > >> > > > > whole
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > backup/restore tooling
> chain
> > as
> > > >>>> not
> > > >>>> >>> >> > trustworthy
> > > >>>> >>> >> > > > and
> > > >>>> >>> >> > > > > >> > > abandon
> > > >>>> >>> >> > > > > >> > > > > it.
> > > >>>> >>> >> > > > > >> > > > > >> > Lets
> > > >>>> >>> >> > > > > >> > > > > >> > > > not
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > have this happen to this
> > > feature.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > + Matteo's suggestion
> (with a
> > > >>>> helpful
> > > >>>> >>> >> starter
> > > >>>> >>> >> > > > list)
> > > >>>> >>> >> > > > > >> that
> > > >>>> >>> >> > > > > >> > > > there
> > > >>>> >>> >> > > > > >> > > > > >> > needs
> > > >>>> >>> >> > > > > >> > > > > >> > > to
> > > >>>> >>> >> > > > > >> > > > > >> > > > > be
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > explicit qualification on
> > what
> > > is
> > > >>>> >>> actually
> > > >>>> >>> >> > > being
> > > >>>> >>> >> > > > > >> > delivered
> > > >>>> >>> >> > > > > >> > > > --
> > > >>>> >>> >> > > > > >> > > > > >> > > > including a
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > listing of limitations
> (some
> > > look
> > > >>>> >>> serious
> > > >>>> >>> >> > such
> > > >>>> >>> >> > > as
> > > >>>> >>> >> > > > > >> data
> > > >>>> >>> >> > > > > >> > > bleed
> > > >>>> >>> >> > > > > >> > > > > from
> > > >>>> >>> >> > > > > >> > > > > >> > > other
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > regions in WALs, but maybe
> I
> > > don't
> > > >>>> >>> care
> > > >>>> >>> >> for
> > > >>>> >>> >> > my
> > > >>>> >>> >> > > > use
> > > >>>> >>> >> > > > > >> > > case...)
> > > >>>> >>> >> > > > > >> > > > --
> > > >>>> >>> >> > > > > >> > > > > >> > needs
> > > >>>> >>> >> > > > > >> > > > > >> > > to
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > accompany the merge. Lets
> > fold
> > > >>>> them
> > > >>>> >>> into
> > > >>>> >>> >> the
> > > >>>> >>> >> > > user
> > > >>>> >>> >> > > > > >> doc.
> > > >>>> >>> >> > > > > >> > in
> > > >>>> >>> >> > > > > >> > > > the
> > > >>>> >>> >> > > > > >> > > > > >> > > technical
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > overview area as suggested
> so
> > > user
> > > >>>> >>> >> > expectations
> > > >>>> >>> >> > > > are
> > > >>>> >>> >> > > > > >> > > properly
> > > >>>> >>> >> > > > > >> > > > > >> > managed
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > (otherwise, they expect the
> > > world
> > > >>>> and
> > > >>>> >>> will
> > > >>>> >>> >> > just
> > > >>>> >>> >> > > > > give
> > > >>>> >>> >> > > > > >> up
> > > >>>> >>> >> > > > > >> > > when
> > > >>>> >>> >> > > > > >> > > > > we
> > > >>>> >>> >> > > > > >> > > > > >> > fall
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > short). Vladimir did a list
> > of
> > > >>>> what
> > > >>>> >>> is in
> > > >>>> >>> >> > each
> > > >>>> >>> >> > > of
> > > >>>> >>> >> > > > > the
> > > >>>> >>> >> > > > > >> > > phases
> > > >>>> >>> >> > > > > >> > > > > >> above
> > > >>>> >>> >> > > > > >> > > > > >> > > > which
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > would serve as a good
> start.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > + Is this feature
> > > 'experimental'
> > > >>>> >>> (Matteo
> > > >>>> >>> >> asks
> > > >>>> >>> >> > > > > above).
> > > >>>> >>> >> > > > > >> > I'd
> > > >>>> >>> >> > > > > >> > > > > prefer
> > > >>>> >>> >> > > > > >> > > > > >> it
> > > >>>> >>> >> > > > > >> > > > > >> > > is
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > not. If it is, it should be
> > > >>>> labelled
> > > >>>> >>> all
> > > >>>> >>> >> over
> > > >>>> >>> >> > > > that
> > > >>>> >>> >> > > > > >> it is
> > > >>>> >>> >> > > > > >> > > > so. I
> > > >>>> >>> >> > > > > >> > > > > >> see
> > > >>>> >>> >> > > > > >> > > > > >> > > > > current
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > state called out as a '...
> > > >>>> technical
> > > >>>> >>> >> preview
> > > >>>> >>> >> > > > > >> feature'.
> > > >>>> >>> >> > > > > >> > > Does
> > > >>>> >>> >> > > > > >> > > > > this
> > > >>>> >>> >> > > > > >> > > > > >> > mean
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > not-for-users?
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > St.Ack
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > > On Mon, Sep 12, 2016 at
> 8:03
> > > AM,
> > > >>>> Ted
> > > >>>> >>> Yu <
> > > >>>> >>> >> > > > > >> > > > yuzhihong@gmail.com>
> > > >>>> >>> >> > > > > >> > > > > >> > wrote:
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> Sean:
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> Do you have more comments
> ?
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >>
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> Cheers
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >>
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> On Fri, Sep 9, 2016 at
> 1:42
> > > PM,
> > > >>>> >>> Vladimir
> > > >>>> >>> >> > > > Rodionov
> > > >>>> >>> >> > > > > <
> > > >>>> >>> >> > > > > >> > > > > >> > > > > vladrodionov@gmail.com
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> wrote:
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >>
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > Sean,
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > Backup/Restore can fail
> > due
> > > to
> > > >>>> >>> various
> > > >>>> >>> >> > > > reasons:
> > > >>>> >>> >> > > > > >> > network
> > > >>>> >>> >> > > > > >> > > > > outage
> > > >>>> >>> >> > > > > >> > > > > >> > > > > (cluster
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > wide), various time-outs
> > in
> > > >>>> HBase
> > > >>>> >>> and
> > > >>>> >>> >> HDFS
> > > >>>> >>> >> > > > > layer,
> > > >>>> >>> >> > > > > >> M/R
> > > >>>> >>> >> > > > > >> > > > > failure
> > > >>>> >>> >> > > > > >> > > > > >> > due
> > > >>>> >>> >> > > > > >> > > > > >> > > to
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> "HDFS
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > exceeded quota", user
> > error
> > > >>>> (manual
> > > >>>> >>> >> > deletion
> > > >>>> >>> >> > > > of
> > > >>>> >>> >> > > > > >> data)
> > > >>>> >>> >> > > > > >> > > and
> > > >>>> >>> >> > > > > >> > > > > so
> > > >>>> >>> >> > > > > >> > > > > >> on
> > > >>>> >>> >> > > > > >> > > > > >> > so
> > > >>>> >>> >> > > > > >> > > > > >> > > > on.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> That
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > is impossible to
> enumerate
> > > all
> > > >>>> >>> possible
> > > >>>> >>> >> > > types
> > > >>>> >>> >> > > > of
> > > >>>> >>> >> > > > > >> > > failures
> > > >>>> >>> >> > > > > >> > > > > in a
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> distributed
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > system - that is not our
> > > >>>> goal/task.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > We focus completely on
> > > backup
> > > >>>> >>> system
> > > >>>> >>> >> table
> > > >>>> >>> >> > > > > >> > consistency
> > > >>>> >>> >> > > > > >> > > > in a
> > > >>>> >>> >> > > > > >> > > > > >> > > presence
> > > >>>> >>> >> > > > > >> > > > > >> > > > > of
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> any
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > type of failure. That is
> > > what I
> > > >>>> >>> call
> > > >>>> >>> >> > > > "tolerance
> > > >>>> >>> >> > > > > to
> > > >>>> >>> >> > > > > >> > > > > failures".
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > On a failure:
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > BACKUP. All backup
> system
> > > >>>> >>> information
> > > >>>> >>> >> > (prior
> > > >>>> >>> >> > > > to
> > > >>>> >>> >> > > > > >> > backup)
> > > >>>> >>> >> > > > > >> > > > > will
> > > >>>> >>> >> > > > > >> > > > > >> be
> > > >>>> >>> >> > > > > >> > > > > >> > > > > restored
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > and all temporary data,
> > > >>>> related to
> > > >>>> >>> a
> > > >>>> >>> >> > failed
> > > >>>> >>> >> > > > > >> session,
> > > >>>> >>> >> > > > > >> > in
> > > >>>> >>> >> > > > > >> > > > > HDFS
> > > >>>> >>> >> > > > > >> > > > > >> > will
> > > >>>> >>> >> > > > > >> > > > > >> > > be
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > deleted
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > RESTORE. We do not care
> > > about
> > > >>>> >>> system
> > > >>>> >>> >> data,
> > > >>>> >>> >> > > > > because
> > > >>>> >>> >> > > > > >> > > > restore
> > > >>>> >>> >> > > > > >> > > > > >> does
> > > >>>> >>> >> > > > > >> > > > > >> > > not
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> change
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > it. Temporary data in
> HDFS
> > > >>>> will be
> > > >>>> >>> >> cleaned
> > > >>>> >>> >> > > up
> > > >>>> >>> >> > > > > and
> > > >>>> >>> >> > > > > >> > table
> > > >>>> >>> >> > > > > >> > > > > will
> > > >>>> >>> >> > > > > >> > > > > >> be
> > > >>>> >>> >> > > > > >> > > > > >> > > in a
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> state
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > back to where it was
> > before
> > > >>>> >>> operation
> > > >>>> >>> >> > > started.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > This is what user should
> > > >>>> expect in
> > > >>>> >>> case
> > > >>>> >>> >> > of a
> > > >>>> >>> >> > > > > >> failure.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > -Vlad
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > -Vlad
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > On Fri, Sep 9, 2016 at
> > 12:56
> > > >>>> PM,
> > > >>>> >>> Sean
> > > >>>> >>> >> > > Busbey <
> > > >>>> >>> >> > > > > >> > > > > >> busbey@apache.org
> > > >>>> >>> >> > > > > >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > wrote:
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > Failing in a
> consistent
> > > way,
> > > >>>> with
> > > >>>> >>> >> docs
> > > >>>> >>> >> > > that
> > > >>>> >>> >> > > > > >> explain
> > > >>>> >>> >> > > > > >> > > the
> > > >>>> >>> >> > > > > >> > > > > >> > various
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > expected failures
> would
> > be
> > > >>>> >>> >> sufficient.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > On Fri, Sep 9, 2016 at
> > > 12:16
> > > >>>> PM,
> > > >>>> >>> >> > Vladimir
> > > >>>> >>> >> > > > > >> Rodionov
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > <
> vladrodionov@gmail.com
> > >
> > > >>>> wrote:
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > Do not worry Sean,
> doc
> > > is
> > > >>>> >>> coming
> > > >>>> >>> >> today
> > > >>>> >>> >> > > as
> > > >>>> >>> >> > > > a
> > > >>>> >>> >> > > > > >> > preview
> > > >>>> >>> >> > > > > >> > > > and
> > > >>>> >>> >> > > > > >> > > > > >> our
> > > >>>> >>> >> > > > > >> > > > > >> > > > writer
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > Frank
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > will be working on a
> > > >>>> putting
> > > >>>> >>> it
> > > >>>> >>> >> into
> > > >>>> >>> >> > > > Apache
> > > >>>> >>> >> > > > > >> > repo.
> > > >>>> >>> >> > > > > >> > > > > >> Timeline
> > > >>>> >>> >> > > > > >> > > > > >> > > > > depends
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> on
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > Franks schedule but
> I
> > > hope
> > > >>>> we
> > > >>>> >>> will
> > > >>>> >>> >> get
> > > >>>> >>> >> > > it
> > > >>>> >>> >> > > > > >> rather
> > > >>>> >>> >> > > > > >> > > > sooner
> > > >>>> >>> >> > > > > >> > > > > >> than
> > > >>>> >>> >> > > > > >> > > > > >> > > > > later.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > As for failure
> > testing,
> > > we
> > > >>>> are
> > > >>>> >>> >> > focusing
> > > >>>> >>> >> > > > only
> > > >>>> >>> >> > > > > >> on a
> > > >>>> >>> >> > > > > >> > > > > >> consistent
> > > >>>> >>> >> > > > > >> > > > > >> > > > state
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> of
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > backup system data
> in
> > a
> > > >>>> >>> presence of
> > > >>>> >>> >> > any
> > > >>>> >>> >> > > > type
> > > >>>> >>> >> > > > > >> of
> > > >>>> >>> >> > > > > >> > > > > failures,
> > > >>>> >>> >> > > > > >> > > > > >> We
> > > >>>> >>> >> > > > > >> > > > > >> > > are
> > > >>>> >>> >> > > > > >> > > > > >> > > > > not
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > going
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > to implement
> anything
> > > more
> > > >>>> >>> >> "fancy",
> > > >>>> >>> >> > > than
> > > >>>> >>> >> > > > > >> that.
> > > >>>> >>> >> > > > > >> > We
> > > >>>> >>> >> > > > > >> > > > > allow
> > > >>>> >>> >> > > > > >> > > > > >> > both:
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> backup
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > and
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > restore to fail.
> What
> > we
> > > >>>> do not
> > > >>>> >>> >> allow
> > > >>>> >>> >> > is
> > > >>>> >>> >> > > > to
> > > >>>> >>> >> > > > > >> have
> > > >>>> >>> >> > > > > >> > > > system
> > > >>>> >>> >> > > > > >> > > > > >> data
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> corrupted.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > Will it suffice for
> > you?
> > > >>>> Do you
> > > >>>> >>> >> have
> > > >>>> >>> >> > any
> > > >>>> >>> >> > > > > other
> > > >>>> >>> >> > > > > >> > > > > concerns,
> > > >>>> >>> >> > > > > >> > > > > >> you
> > > >>>> >>> >> > > > > >> > > > > >> > > > want
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> us to
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > address?
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > -Vlad
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > On Fri, Sep 9, 2016
> at
> > > >>>> 10:56
> > > >>>> >>> AM,
> > > >>>> >>> >> Sean
> > > >>>> >>> >> > > > > Busbey <
> > > >>>> >>> >> > > > > >> > > > > >> > > busbey@apache.org
> > > >>>> >>> >> > > > > >> > > > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > wrote:
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> "docs will come to
> > > Apache
> > > >>>> >>> soon"
> > > >>>> >>> >> does
> > > >>>> >>> >> > > not
> > > >>>> >>> >> > > > > >> address
> > > >>>> >>> >> > > > > >> > > my
> > > >>>> >>> >> > > > > >> > > > > >> concern
> > > >>>> >>> >> > > > > >> > > > > >> > > > > around
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > docs
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > at
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> all, unless said
> docs
> > > have
> > > >>>> >>> already
> > > >>>> >>> >> > made
> > > >>>> >>> >> > > > it
> > > >>>> >>> >> > > > > >> into
> > > >>>> >>> >> > > > > >> > > the
> > > >>>> >>> >> > > > > >> > > > > >> project
> > > >>>> >>> >> > > > > >> > > > > >> > > > > repo. I
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > don't
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> want third party
> > > >>>> resources for
> > > >>>> >>> >> using
> > > >>>> >>> >> > a
> > > >>>> >>> >> > > > > major
> > > >>>> >>> >> > > > > >> and
> > > >>>> >>> >> > > > > >> > > > > >> important
> > > >>>> >>> >> > > > > >> > > > > >> > > > > feature
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> of
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> project, I want us
> to
> > > >>>> provide
> > > >>>> >>> end
> > > >>>> >>> >> > users
> > > >>>> >>> >> > > > > with
> > > >>>> >>> >> > > > > >> > what
> > > >>>> >>> >> > > > > >> > > > they
> > > >>>> >>> >> > > > > >> > > > > >> need
> > > >>>> >>> >> > > > > >> > > > > >> > > to
> > > >>>> >>> >> > > > > >> > > > > >> > > > > get
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > job
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> done.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >>
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> I see some calls
> for
> > > >>>> patience
> > > >>>> >>> on
> > > >>>> >>> >> the
> > > >>>> >>> >> > > > > failure
> > > >>>> >>> >> > > > > >> > > > testing,
> > > >>>> >>> >> > > > > >> > > > > but
> > > >>>> >>> >> > > > > >> > > > > >> > the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> appeal
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > to
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > us
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> having done a bad
> job
> > > of
> > > >>>> >>> requiring
> > > >>>> >>> >> > > proper
> > > >>>> >>> >> > > > > >> tests
> > > >>>> >>> >> > > > > >> > of
> > > >>>> >>> >> > > > > >> > > > > >> previous
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> features
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > just
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> makes me more
> > concerned
> > > >>>> about
> > > >>>> >>> not
> > > >>>> >>> >> > > getting
> > > >>>> >>> >> > > > > >> them
> > > >>>> >>> >> > > > > >> > > > here. I
> > > >>>> >>> >> > > > > >> > > > > >> > don't
> > > >>>> >>> >> > > > > >> > > > > >> > > > want
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> to
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > set
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> yet another bad
> > example
> > > >>>> that
> > > >>>> >>> will
> > > >>>> >>> >> > then
> > > >>>> >>> >> > > be
> > > >>>> >>> >> > > > > >> > pointed
> > > >>>> >>> >> > > > > >> > > to
> > > >>>> >>> >> > > > > >> > > > > in
> > > >>>> >>> >> > > > > >> > > > > >> the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > future.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >>
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> On Sep 8, 2016
> 10:50,
> > > "Ted
> > > >>>> >>> Yu" <
> > > >>>> >>> >> > > > > >> > > yuzhihong@gmail.com
> > > >>>> >>> >> > > > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > wrote:
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >>
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > Is there any
> > concern
> > > >>>> which
> > > >>>> >>> is
> > > >>>> >>> >> not
> > > >>>> >>> >> > > > > >> addressed ?
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > Do we need
> another
> > > Vote
> > > >>>> >>> thread ?
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > Thanks
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > On Thu, Sep 8,
> 2016
> > > at
> > > >>>> 9:21
> > > >>>> >>> AM,
> > > >>>> >>> >> > > Andrew
> > > >>>> >>> >> > > > > >> > Purtell <
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> apurtell@apache.org
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > wrote:
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > Vlad,
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > I apologize for
> > > using
> > > >>>> the
> > > >>>> >>> term
> > > >>>> >>> >> > > > > >> 'half-baked'
> > > >>>> >>> >> > > > > >> > > in a
> > > >>>> >>> >> > > > > >> > > > > way
> > > >>>> >>> >> > > > > >> > > > > >> > that
> > > >>>> >>> >> > > > > >> > > > > >> > > > > could
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > seem a
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > description of
> > > >>>> >>> HBASE-7912. I
> > > >>>> >>> >> > meant
> > > >>>> >>> >> > > > that
> > > >>>> >>> >> > > > > >> as a
> > > >>>> >>> >> > > > > >> > > > > general
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> hypothetical.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > On Wed, Sep 7,
> > > 2016 at
> > > >>>> >>> 9:36
> > > >>>> >>> >> AM,
> > > >>>> >>> >> > > > > Vladimir
> > > >>>> >>> >> > > > > >> > > > Rodionov
> > > >>>> >>> >> > > > > >> > > > > <
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> > > vladrodionov@gmail.com>
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > wrote:
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >> I'm not
> sure
> > > that
> > > >>>> >>> "There
> > > >>>> >>> >> is
> > > >>>> >>> >> > > > > already
> > > >>>> >>> >> > > > > >> > lots
> > > >>>> >>> >> > > > > >> > > of
> > > >>>> >>> >> > > > > >> > > > > >> > > half-baked
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> code
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > in
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > branch,
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > so what's the
> > > harm
> > > >>>> in
> > > >>>> >>> adding
> > > >>>> >>> >> > > more?"
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > I meant - not
> > > >>>> >>> production -
> > > >>>> >>> >> > ready
> > > >>>> >>> >> > > > yet.
> > > >>>> >>> >> > > > > >> This
> > > >>>> >>> >> > > > > >> > > is
> > > >>>> >>> >> > > > > >> > > > > 2.0
> > > >>>> >>> >> > > > > >> > > > > >> > > > > development
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > branch
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > and,
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > hence many
> > > features
> > > >>>> are
> > > >>>> >>> in
> > > >>>> >>> >> > works,
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > not being
> > tested
> > > >>>> well
> > > >>>> >>> etc.
> > > >>>> >>> >> I do
> > > >>>> >>> >> > > not
> > > >>>> >>> >> > > > > >> > consider
> > > >>>> >>> >> > > > > >> > > > > backup
> > > >>>> >>> >> > > > > >> > > > > >> > as
> > > >>>> >>> >> > > > > >> > > > > >> > > > half
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > baked
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > feature -
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > it has passed
> > our
> > > >>>> >>> internal
> > > >>>> >>> >> QA
> > > >>>> >>> >> > and
> > > >>>> >>> >> > > > has
> > > >>>> >>> >> > > > > >> very
> > > >>>> >>> >> > > > > >> > > > good
> > > >>>> >>> >> > > > > >> > > > > >> doc,
> > > >>>> >>> >> > > > > >> > > > > >> > > > which
> > > >>>> >>> >> > > > > >> > > > > >> > > > > we
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > will
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > provide
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > to Apache
> > > shortly.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > -Vlad
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > On Wed, Sep
> 7,
> > > 2016
> > > >>>> at
> > > >>>> >>> 9:13
> > > >>>> >>> >> AM,
> > > >>>> >>> >> > > > > Andrew
> > > >>>> >>> >> > > > > >> > > > Purtell <
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > apurtell@apache.org>
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > We
> shouldn't
> > > admit
> > > >>>> >>> half
> > > >>>> >>> >> baked
> > > >>>> >>> >> > > > > changes
> > > >>>> >>> >> > > > > >> > that
> > > >>>> >>> >> > > > > >> > > > > won't
> > > >>>> >>> >> > > > > >> > > > > >> be
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> finished.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> However
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > in
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > this case
> the
> > > crew
> > > >>>> >>> >> working on
> > > >>>> >>> >> > > > this
> > > >>>> >>> >> > > > > >> > feature
> > > >>>> >>> >> > > > > >> > > > are
> > > >>>> >>> >> > > > > >> > > > > >> long
> > > >>>> >>> >> > > > > >> > > > > >> > > > > timers
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> and
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > less
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > likely
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > than just
> > about
> > > >>>> >>> anyone to
> > > >>>> >>> >> > leave
> > > >>>> >>> >> > > > > >> > something
> > > >>>> >>> >> > > > > >> > > > in a
> > > >>>> >>> >> > > > > >> > > > > >> half
> > > >>>> >>> >> > > > > >> > > > > >> > > > baked
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > state. Of
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > course
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > there is no
> > > >>>> guarantee
> > > >>>> >>> how
> > > >>>> >>> >> > > > anything
> > > >>>> >>> >> > > > > >> will
> > > >>>> >>> >> > > > > >> > > turn
> > > >>>> >>> >> > > > > >> > > > > out,
> > > >>>> >>> >> > > > > >> > > > > >> > > but I
> > > >>>> >>> >> > > > > >> > > > > >> > > > > am
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > willing
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> to
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > take
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > a little on
> > > faith
> > > >>>> if
> > > >>>> >>> they
> > > >>>> >>> >> > feel
> > > >>>> >>> >> > > > > their
> > > >>>> >>> >> > > > > >> > best
> > > >>>> >>> >> > > > > >> > > > path
> > > >>>> >>> >> > > > > >> > > > > >> > > forward
> > > >>>> >>> >> > > > > >> > > > > >> > > > > now
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> is
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > to
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > merge
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > to
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > trunk. I
> only
> > > >>>> wish I
> > > >>>> >>> had
> > > >>>> >>> >> > > > bandwidth
> > > >>>> >>> >> > > > > to
> > > >>>> >>> >> > > > > >> > have
> > > >>>> >>> >> > > > > >> > > > > done
> > > >>>> >>> >> > > > > >> > > > > >> > some
> > > >>>> >>> >> > > > > >> > > > > >> > > > real
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > kicking
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> of
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > tires by
> now.
> > > >>>> Maybe
> > > >>>> >>> this
> > > >>>> >>> >> > week.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > (Yes, I'm
> > using
> > > >>>> some
> > > >>>> >>> of
> > > >>>> >>> >> that
> > > >>>> >>> >> > > time
> > > >>>> >>> >> > > > > for
> > > >>>> >>> >> > > > > >> > this
> > > >>>> >>> >> > > > > >> > > > > email
> > > >>>> >>> >> > > > > >> > > > > >> > :-)
> > > >>>> >>> >> > > > > >> > > > > >> > > > but
> > > >>>> >>> >> > > > > >> > > > > >> > > > > I
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > type
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > fast.)
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > That said,
> I
> > > would
> > > >>>> >>> like to
> > > >>>> >>> >> > > > agitate
> > > >>>> >>> >> > > > > >> for
> > > >>>> >>> >> > > > > >> > > > making
> > > >>>> >>> >> > > > > >> > > > > 2.0
> > > >>>> >>> >> > > > > >> > > > > >> > > more
> > > >>>> >>> >> > > > > >> > > > > >> > > > > real
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > and
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> spend
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > some
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > time on it
> > now
> > > >>>> that
> > > >>>> >>> I'm
> > > >>>> >>> >> > winding
> > > >>>> >>> >> > > > > down
> > > >>>> >>> >> > > > > >> > with
> > > >>>> >>> >> > > > > >> > > > > 0.98. I
> > > >>>> >>> >> > > > > >> > > > > >> > > think
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> that
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > means
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branching
> for
> > > 2.0
> > > >>>> real
> > > >>>> >>> >> soon
> > > >>>> >>> >> > now
> > > >>>> >>> >> > > > and
> > > >>>> >>> >> > > > > >> even
> > > >>>> >>> >> > > > > >> > > > > evicting
> > > >>>> >>> >> > > > > >> > > > > >> > > > things
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> from
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > 2.0
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > branch
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > that aren't
> > > >>>> finished
> > > >>>> >>> or
> > > >>>> >>> >> > stable,
> > > >>>> >>> >> > > > > >> leaving
> > > >>>> >>> >> > > > > >> > > them
> > > >>>> >>> >> > > > > >> > > > > only
> > > >>>> >>> >> > > > > >> > > > > >> > > once
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> again
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > in
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > master
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branch. Or,
> > > maybe
> > > >>>> just
> > > >>>> >>> >> > evicting
> > > >>>> >>> >> > > > > them.
> > > >>>> >>> >> > > > > >> > > Let's
> > > >>>> >>> >> > > > > >> > > > > take
> > > >>>> >>> >> > > > > >> > > > > >> it
> > > >>>> >>> >> > > > > >> > > > > >> > > > case
> > > >>>> >>> >> > > > > >> > > > > >> > > > > by
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > case.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > I think
> this
> > > >>>> feature
> > > >>>> >>> can
> > > >>>> >>> >> come
> > > >>>> >>> >> > > in
> > > >>>> >>> >> > > > > >> > > relatively
> > > >>>> >>> >> > > > > >> > > > > >> safely.
> > > >>>> >>> >> > > > > >> > > > > >> > > As
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> added
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > insurance,
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > let's admit
> > the
> > > >>>> >>> >> possibility
> > > >>>> >>> >> > it
> > > >>>> >>> >> > > > > could
> > > >>>> >>> >> > > > > >> be
> > > >>>> >>> >> > > > > >> > > > > reverted
> > > >>>> >>> >> > > > > >> > > > > >> on
> > > >>>> >>> >> > > > > >> > > > > >> > > the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > 2.0
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > branch
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> if
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > folks
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > working on
> > > >>>> >>> stabilizing 2.0
> > > >>>> >>> >> > > decide
> > > >>>> >>> >> > > > > to
> > > >>>> >>> >> > > > > >> > evict
> > > >>>> >>> >> > > > > >> > > > it
> > > >>>> >>> >> > > > > >> > > > > >> > because
> > > >>>> >>> >> > > > > >> > > > > >> > > > it
> > > >>>> >>> >> > > > > >> > > > > >> > > > > is
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > unfinished
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > or
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > unstable,
> > > because
> > > >>>> that
> > > >>>> >>> >> > > certainly
> > > >>>> >>> >> > > > > can
> > > >>>> >>> >> > > > > >> > > > happen. I
> > > >>>> >>> >> > > > > >> > > > > >> > would
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> expect if
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > talk
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > like
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > that
> starts,
> > > we'd
> > > >>>> get
> > > >>>> >>> help
> > > >>>> >>> >> > > > > finishing
> > > >>>> >>> >> > > > > >> or
> > > >>>> >>> >> > > > > >> > > > > >> stabilizing
> > > >>>> >>> >> > > > > >> > > > > >> > > > > what's
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > under
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > discussion
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > for revert.
> > Or,
> > > >>>> we'd
> > > >>>> >>> have
> > > >>>> >>> >> a
> > > >>>> >>> >> > > > revert.
> > > >>>> >>> >> > > > > >> > Either
> > > >>>> >>> >> > > > > >> > > > way
> > > >>>> >>> >> > > > > >> > > > > >> the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > outcome
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> is
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > acceptable.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > On Wed, Sep
> > 7,
> > > >>>> 2016 at
> > > >>>> >>> >> 8:56
> > > >>>> >>> >> > AM,
> > > >>>> >>> >> > > > > Dima
> > > >>>> >>> >> > > > > >> > > Spivak
> > > >>>> >>> >> > > > > >> > > > <
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > dimaspivak@apache.org
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > I'm not
> > sure
> > > >>>> that
> > > >>>> >>> >> "There is
> > > >>>> >>> >> > > > > already
> > > >>>> >>> >> > > > > >> > lots
> > > >>>> >>> >> > > > > >> > > > of
> > > >>>> >>> >> > > > > >> > > > > >> > > > half-baked
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> code
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > in
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branch,
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > so what's
> > the
> > > >>>> harm
> > > >>>> >>> in
> > > >>>> >>> >> > adding
> > > >>>> >>> >> > > > > more?"
> > > >>>> >>> >> > > > > >> > is a
> > > >>>> >>> >> > > > > >> > > > > good
> > > >>>> >>> >> > > > > >> > > > > >> > code
> > > >>>> >>> >> > > > > >> > > > > >> > > > > commit
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > philosophy
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > for
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > a
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> > > fault-tolerant
> > > >>>> >>> >> distributed
> > > >>>> >>> >> > > data
> > > >>>> >>> >> > > > > >> store.
> > > >>>> >>> >> > > > > >> > > ;)
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > More
> > > seriously,
> > > >>>> a
> > > >>>> >>> lack
> > > >>>> >>> >> of
> > > >>>> >>> >> > > test
> > > >>>> >>> >> > > > > >> > coverage
> > > >>>> >>> >> > > > > >> > > > for
> > > >>>> >>> >> > > > > >> > > > > >> > > existing
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > features
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > shouldn't
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > be
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > used as
> > > >>>> >>> justification
> > > >>>> >>> >> for
> > > >>>> >>> >> > > > > >> introducing
> > > >>>> >>> >> > > > > >> > > new
> > > >>>> >>> >> > > > > >> > > > > >> > features
> > > >>>> >>> >> > > > > >> > > > > >> > > > with
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > same
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> > shortcomings.
> > > >>>> >>> >> Ultimately,
> > > >>>> >>> >> > > it's
> > > >>>> >>> >> > > > > the
> > > >>>> >>> >> > > > > >> end
> > > >>>> >>> >> > > > > >> > > > user
> > > >>>> >>> >> > > > > >> > > > > who
> > > >>>> >>> >> > > > > >> > > > > >> > > will
> > > >>>> >>> >> > > > > >> > > > > >> > > > > feel
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> pain,
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > so
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > shouldn't
> > we
> > > do
> > > >>>> >>> >> everything
> > > >>>> >>> >> > we
> > > >>>> >>> >> > > > can
> > > >>>> >>> >> > > > > >> to
> > > >>>> >>> >> > > > > >> > > > > mitigate
> > > >>>> >>> >> > > > > >> > > > > >> > that?
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > -Dima
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > On Wed,
> Sep
> > > 7,
> > > >>>> 2016
> > > >>>> >>> at
> > > >>>> >>> >> 8:46
> > > >>>> >>> >> > > AM,
> > > >>>> >>> >> > > > > >> > Vladimir
> > > >>>> >>> >> > > > > >> > > > > >> > Rodionov <
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > >>>> >>> vladrodionov@gmail.com>
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > wrote:
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Sean,
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have
> > docs
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Agree.
> We
> > > >>>> have a
> > > >>>> >>> doc
> > > >>>> >>> >> and
> > > >>>> >>> >> > > > backup
> > > >>>> >>> >> > > > > >> is
> > > >>>> >>> >> > > > > >> > the
> > > >>>> >>> >> > > > > >> > > > > most
> > > >>>> >>> >> > > > > >> > > > > >> > > > > documented
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > feature
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > :),
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > we
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > will
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > release
> > it
> > > >>>> >>> shortly to
> > > >>>> >>> >> > > Apache.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have
> > > >>>> sunny-day
> > > >>>> >>> >> > > correctness
> > > >>>> >>> >> > > > > >> tests
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Feature
> > has
> > > >>>> >>> close to
> > > >>>> >>> >> 60
> > > >>>> >>> >> > > test
> > > >>>> >>> >> > > > > >> cases,
> > > >>>> >>> >> > > > > >> > > > which
> > > >>>> >>> >> > > > > >> > > > > >> run
> > > >>>> >>> >> > > > > >> > > > > >> > > for
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> approx
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > 30
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> min.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > We
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > can
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > add
> more,
> > > if
> > > >>>> >>> >> community do
> > > >>>> >>> >> > > not
> > > >>>> >>> >> > > > > >> mind
> > > >>>> >>> >> > > > > >> > :)
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have
> > > >>>> >>> >> > > > > >> > correctness-in-face-of-failure
> > > >>>> >>> >> > > > > >> > > > > tests
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Any
> > > examples
> > > >>>> of
> > > >>>> >>> these
> > > >>>> >>> >> > tests
> > > >>>> >>> >> > > > in
> > > >>>> >>> >> > > > > >> > > existing
> > > >>>> >>> >> > > > > >> > > > > >> > features?
> > > >>>> >>> >> > > > > >> > > > > >> > > > In
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > works,
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > we
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > have a
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > clear
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > understanding
> > > >>>> of
> > > >>>> >>> what
> > > >>>> >>> >> > > should
> > > >>>> >>> >> > > > be
> > > >>>> >>> >> > > > > >> done
> > > >>>> >>> >> > > > > >> > > by
> > > >>>> >>> >> > > > > >> > > > > the
> > > >>>> >>> >> > > > > >> > > > > >> > time
> > > >>>> >>> >> > > > > >> > > > > >> > > of
> > > >>>> >>> >> > > > > >> > > > > >> > > > > 2.0
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> release.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > That is
> > > very
> > > >>>> close
> > > >>>> >>> >> goal
> > > >>>> >>> >> > for
> > > >>>> >>> >> > > > us,
> > > >>>> >>> >> > > > > >> to
> > > >>>> >>> >> > > > > >> > > > verify
> > > >>>> >>> >> > > > > >> > > > > IT
> > > >>>> >>> >> > > > > >> > > > > >> > > monkey
> > > >>>> >>> >> > > > > >> > > > > >> > > > > for
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> existing
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > code.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * don't
> > > rely
> > > >>>> on
> > > >>>> >>> things
> > > >>>> >>> >> > > > outside
> > > >>>> >>> >> > > > > of
> > > >>>> >>> >> > > > > >> > > HBase
> > > >>>> >>> >> > > > > >> > > > > for
> > > >>>> >>> >> > > > > >> > > > > >> > > normal
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > operation
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > (okay
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > for
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> advanced
> > > >>>> >>> operation)
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > We do
> > not.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> Enormous
> > > time
> > > >>>> has
> > > >>>> >>> been
> > > >>>> >>> >> > > spent
> > > >>>> >>> >> > > > > >> already
> > > >>>> >>> >> > > > > >> > > on
> > > >>>> >>> >> > > > > >> > > > > the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > development
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > and
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > testing
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> feature,
> > it
> > > >>>> has
> > > >>>> >>> passed
> > > >>>> >>> >> > our
> > > >>>> >>> >> > > > > >> internal
> > > >>>> >>> >> > > > > >> > > > tests
> > > >>>> >>> >> > > > > >> > > > > and
> > > >>>> >>> >> > > > > >> > > > > >> > > many
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> rounds
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > of
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> code
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > reviews
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > by
> HBase
> > > >>>> >>> committers.
> > > >>>> >>> >> We
> > > >>>> >>> >> > do
> > > >>>> >>> >> > > > not
> > > >>>> >>> >> > > > > >> mind
> > > >>>> >>> >> > > > > >> > if
> > > >>>> >>> >> > > > > >> > > > > >> someone
> > > >>>> >>> >> > > > > >> > > > > >> > > from
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> HBase
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > community
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> (outside
> > of
> > > >>>> HW)
> > > >>>> >>> will
> > > >>>> >>> >> > review
> > > >>>> >>> >> > > > the
> > > >>>> >>> >> > > > > >> > code,
> > > >>>> >>> >> > > > > >> > > > but
> > > >>>> >>> >> > > > > >> > > > > it
> > > >>>> >>> >> > > > > >> > > > > >> > will
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> probably
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> takes
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > forever
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > to
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > wait
> for
> > > >>>> >>> volunteer?,
> > > >>>> >>> >> the
> > > >>>> >>> >> > > > > feature
> > > >>>> >>> >> > > > > >> is
> > > >>>> >>> >> > > > > >> > > > quite
> > > >>>> >>> >> > > > > >> > > > > >> large
> > > >>>> >>> >> > > > > >> > > > > >> > > > (1MB+
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> cumulative
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > patch)
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > 2.0
> > branch
> > > is
> > > >>>> >>> full of
> > > >>>> >>> >> > half
> > > >>>> >>> >> > > > > baked
> > > >>>> >>> >> > > > > >> > > > features,
> > > >>>> >>> >> > > > > >> > > > > >> most
> > > >>>> >>> >> > > > > >> > > > > >> > > of
> > > >>>> >>> >> > > > > >> > > > > >> > > > > them
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > are
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > in
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > active
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > development,
> > > >>>> >>> >> therefore I
> > > >>>> >>> >> > am
> > > >>>> >>> >> > > > not
> > > >>>> >>> >> > > > > >> > > > following
> > > >>>> >>> >> > > > > >> > > > > you
> > > >>>> >>> >> > > > > >> > > > > >> > > here,
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> Sean?
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > Why
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > HBASE-7912
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > is
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > not
> good
> > > >>>> enough
> > > >>>> >>> yet
> > > >>>> >>> >> to be
> > > >>>> >>> >> > > > > >> integrated
> > > >>>> >>> >> > > > > >> > > > into
> > > >>>> >>> >> > > > > >> > > > > 2.0
> > > >>>> >>> >> > > > > >> > > > > >> > > > branch?
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > -Vlad
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > On Wed,
> > > Sep 7,
> > > >>>> >>> 2016 at
> > > >>>> >>> >> > 8:23
> > > >>>> >>> >> > > > AM,
> > > >>>> >>> >> > > > > >> Sean
> > > >>>> >>> >> > > > > >> > > > > Busbey <
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > busbey@apache.org
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > On
> Tue,
> > > Sep
> > > >>>> 6,
> > > >>>> >>> 2016
> > > >>>> >>> >> at
> > > >>>> >>> >> > > > 10:36
> > > >>>> >>> >> > > > > >> PM,
> > > >>>> >>> >> > > > > >> > > Josh
> > > >>>> >>> >> > > > > >> > > > > >> Elser <
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> > > josh.elser@gmail.com>
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > wrote:
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > So,
> > the
> > > >>>> >>> answer to
> > > >>>> >>> >> > > Sean's
> > > >>>> >>> >> > > > > >> > original
> > > >>>> >>> >> > > > > >> > > > > >> question
> > > >>>> >>> >> > > > > >> > > > > >> > is
> > > >>>> >>> >> > > > > >> > > > > >> > > > "as
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > robust as
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > snapshots
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> > > presently
> > > >>>> >>> are"?
> > > >>>> >>> >> > > > > >> (independence of
> > > >>>> >>> >> > > > > >> > > > > >> > > backup/restore
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > failure
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > tolerance
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > from
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> > > snapshot
> > > >>>> >>> failure
> > > >>>> >>> >> > > > tolerance)
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > Is
> > this
> > > >>>> just a
> > > >>>> >>> >> > question
> > > >>>> >>> >> > > > WRT
> > > >>>> >>> >> > > > > >> > > context
> > > >>>> >>> >> > > > > >> > > > of
> > > >>>> >>> >> > > > > >> > > > > >> the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > change,
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> or
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > is it
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > means
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > for a
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > veto
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> from
> > > you,
> > > >>>> >>> Sean?
> > > >>>> >>> >> Just
> > > >>>> >>> >> > > > trying
> > > >>>> >>> >> > > > > >> to
> > > >>>> >>> >> > > > > >> > > make
> > > >>>> >>> >> > > > > >> > > > > sure
> > > >>>> >>> >> > > > > >> > > > > >> > I'm
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> following
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> along
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > adequately.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > I'd
> say
> > > ATM
> > > >>>> I'm
> > > >>>> >>> -0,
> > > >>>> >>> >> > > > bordering
> > > >>>> >>> >> > > > > >> on
> > > >>>> >>> >> > > > > >> > -1
> > > >>>> >>> >> > > > > >> > > > but
> > > >>>> >>> >> > > > > >> > > > > not
> > > >>>> >>> >> > > > > >> > > > > >> > for
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> reasons
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > I
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > can
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> articulate
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > well.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> Here's
> > an
> > > >>>> >>> attempt.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > We've
> > > been
> > > >>>> >>> trying to
> > > >>>> >>> >> > > move,
> > > >>>> >>> >> > > > > as a
> > > >>>> >>> >> > > > > >> > > > > community,
> > > >>>> >>> >> > > > > >> > > > > >> > > > towards
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > minimizing
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > risk
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > to
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > downstream
> > > >>>> >>> folks by
> > > >>>> >>> >> > > getting
> > > >>>> >>> >> > > > > >> > > "complete
> > > >>>> >>> >> > > > > >> > > > > >> enough
> > > >>>> >>> >> > > > > >> > > > > >> > > for
> > > >>>> >>> >> > > > > >> > > > > >> > > > > use"
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > gates
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> in
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > place
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> before
> > we
> > > >>>> >>> introduce
> > > >>>> >>> >> new
> > > >>>> >>> >> > > > > >> features.
> > > >>>> >>> >> > > > > >> > > This
> > > >>>> >>> >> > > > > >> > > > > was
> > > >>>> >>> >> > > > > >> > > > > >> > > > spurred
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> by a
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > some
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > features
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> getting
> > > in
> > > >>>> >>> >> half-baked
> > > >>>> >>> >> > and
> > > >>>> >>> >> > > > > never
> > > >>>> >>> >> > > > > >> > > making
> > > >>>> >>> >> > > > > >> > > > > it
> > > >>>> >>> >> > > > > >> > > > > >> to
> > > >>>> >>> >> > > > > >> > > > > >> > > "can
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> really
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > use"
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > status
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > (I'm
> > > >>>> thinking of
> > > >>>> >>> >> > > > distributed
> > > >>>> >>> >> > > > > >> log
> > > >>>> >>> >> > > > > >> > > > replay
> > > >>>> >>> >> > > > > >> > > > > and
> > > >>>> >>> >> > > > > >> > > > > >> > the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> zk-less
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > assignment
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> stuff,
> > I
> > > >>>> don't
> > > >>>> >>> >> recall
> > > >>>> >>> >> > if
> > > >>>> >>> >> > > > > there
> > > >>>> >>> >> > > > > >> was
> > > >>>> >>> >> > > > > >> > > > > more).
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The
> > > gates,
> > > >>>> >>> >> generally,
> > > >>>> >>> >> > > > > included
> > > >>>> >>> >> > > > > >> > > things
> > > >>>> >>> >> > > > > >> > > > > like:
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > *
> have
> > > docs
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > *
> have
> > > >>>> sunny-day
> > > >>>> >>> >> > > > correctness
> > > >>>> >>> >> > > > > >> tests
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > *
> have
> > > >>>> >>> >> > > > > >> > > correctness-in-face-of-failure
> > > >>>> >>> >> > > > > >> > > > > tests
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > *
> don't
> > > >>>> rely on
> > > >>>> >>> >> things
> > > >>>> >>> >> > > > > outside
> > > >>>> >>> >> > > > > >> of
> > > >>>> >>> >> > > > > >> > > > HBase
> > > >>>> >>> >> > > > > >> > > > > for
> > > >>>> >>> >> > > > > >> > > > > >> > > > normal
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > operation
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > (okay
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > for
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > advanced
> > > >>>> >>> operation)
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > As an
> > > >>>> example,
> > > >>>> >>> we
> > > >>>> >>> >> kept
> > > >>>> >>> >> > > the
> > > >>>> >>> >> > > > > MOB
> > > >>>> >>> >> > > > > >> > work
> > > >>>> >>> >> > > > > >> > > > off
> > > >>>> >>> >> > > > > >> > > > > in
> > > >>>> >>> >> > > > > >> > > > > >> a
> > > >>>> >>> >> > > > > >> > > > > >> > > > branch
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> and
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > out
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> of
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > master
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > until
> > it
> > > >>>> could
> > > >>>> >>> pass
> > > >>>> >>> >> > these
> > > >>>> >>> >> > > > > >> > criteria.
> > > >>>> >>> >> > > > > >> > > > The
> > > >>>> >>> >> > > > > >> > > > > big
> > > >>>> >>> >> > > > > >> > > > > >> > > > > exemption
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > we've
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> had
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > to
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > this
> > was
> > > the
> > > >>>> >>> >> > hbase-spark
> > > >>>> >>> >> > > > > >> > > integration,
> > > >>>> >>> >> > > > > >> > > > > where
> > > >>>> >>> >> > > > > >> > > > > >> > we
> > > >>>> >>> >> > > > > >> > > > > >> > > > all
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > agreed
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > it
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > could
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > land
> in
> > > >>>> master
> > > >>>> >>> >> because
> > > >>>> >>> >> > it
> > > >>>> >>> >> > > > was
> > > >>>> >>> >> > > > > >> very
> > > >>>> >>> >> > > > > >> > > > well
> > > >>>> >>> >> > > > > >> > > > > >> > > isolated
> > > >>>> >>> >> > > > > >> > > > > >> > > > > (the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > slide
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > away
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > from
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > including
> > > >>>> docs
> > > >>>> >>> as a
> > > >>>> >>> >> > > > > first-class
> > > >>>> >>> >> > > > > >> > part
> > > >>>> >>> >> > > > > >> > > > of
> > > >>>> >>> >> > > > > >> > > > > >> > > building
> > > >>>> >>> >> > > > > >> > > > > >> > > > up
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> that
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > integration
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > has
> led
> > > me
> > > >>>> to
> > > >>>> >>> doubt
> > > >>>> >>> >> the
> > > >>>> >>> >> > > > > wisdom
> > > >>>> >>> >> > > > > >> of
> > > >>>> >>> >> > > > > >> > > this
> > > >>>> >>> >> > > > > >> > > > > >> > > decision).
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > We've
> > > also
> > > >>>> been
> > > >>>> >>> >> > treating
> > > >>>> >>> >> > > > > >> inclusion
> > > >>>> >>> >> > > > > >> > > in
> > > >>>> >>> >> > > > > >> > > > a
> > > >>>> >>> >> > > > > >> > > > > >> > > "probably
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> will
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > be
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > released
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > to
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > downstream"
> > > >>>> >>> branches
> > > >>>> >>> >> > as a
> > > >>>> >>> >> > > > > >> higher
> > > >>>> >>> >> > > > > >> > > bar,
> > > >>>> >>> >> > > > > >> > > > > >> > requiring
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > *
> don't
> > > >>>> >>> moderately
> > > >>>> >>> >> > impact
> > > >>>> >>> >> > > > > >> > > performance
> > > >>>> >>> >> > > > > >> > > > > when
> > > >>>> >>> >> > > > > >> > > > > >> > the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> feature
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > isn't
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> in
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > use
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > *
> don't
> > > >>>> severely
> > > >>>> >>> >> impact
> > > >>>> >>> >> > > > > >> > performance
> > > >>>> >>> >> > > > > >> > > > when
> > > >>>> >>> >> > > > > >> > > > > >> the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > feature
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> is
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > in
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> use
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > *
> > either
> > > >>>> >>> >> default-to-on
> > > >>>> >>> >> > or
> > > >>>> >>> >> > > > > show
> > > >>>> >>> >> > > > > >> > > enough
> > > >>>> >>> >> > > > > >> > > > > >> demand
> > > >>>> >>> >> > > > > >> > > > > >> > to
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> believe
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > a
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > non-trivial
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> number
> > of
> > > >>>> folks
> > > >>>> >>> will
> > > >>>> >>> >> > turn
> > > >>>> >>> >> > > > the
> > > >>>> >>> >> > > > > >> > > feature
> > > >>>> >>> >> > > > > >> > > > on
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The
> > above
> > > >>>> has
> > > >>>> >>> kept
> > > >>>> >>> >> MOB
> > > >>>> >>> >> > > and
> > > >>>> >>> >> > > > > >> > > hbase-spark
> > > >>>> >>> >> > > > > >> > > > > >> > > > integration
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> out
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > of
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > branch-1,
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > presumably
> > > >>>> while
> > > >>>> >>> >> > they've
> > > >>>> >>> >> > > > > >> "gotten
> > > >>>> >>> >> > > > > >> > > more
> > > >>>> >>> >> > > > > >> > > > > >> stable"
> > > >>>> >>> >> > > > > >> > > > > >> > > in
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> master
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > from
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > odd
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> vendor
> > > >>>> >>> inclusion.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Are
> we
> > > >>>> going to
> > > >>>> >>> >> have a
> > > >>>> >>> >> > > 2.0
> > > >>>> >>> >> > > > > >> release
> > > >>>> >>> >> > > > > >> > > > > before
> > > >>>> >>> >> > > > > >> > > > > >> the
> > > >>>> >>> >> > > > > >> > > > > >> > > end
> > > >>>> >>> >> > > > > >> > > > > >> > > > > of
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> year?
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > We're
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> coming
> > up
> > > >>>> on 1.5
> > > >>>> >>> >> years
> > > >>>> >>> >> > > > since
> > > >>>> >>> >> > > > > >> the
> > > >>>> >>> >> > > > > >> > > > > release of
> > > >>>> >>> >> > > > > >> > > > > >> > > > version
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> 1.0;
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> seems
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > like
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > it's
> > > about
> > > >>>> time,
> > > >>>> >>> >> > though I
> > > >>>> >>> >> > > > > >> haven't
> > > >>>> >>> >> > > > > >> > > seen
> > > >>>> >>> >> > > > > >> > > > > any
> > > >>>> >>> >> > > > > >> > > > > >> > > > concrete
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > plans
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> this
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > year.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > Presuming
> > > >>>> we are
> > > >>>> >>> >> going
> > > >>>> >>> >> > to
> > > >>>> >>> >> > > > > have
> > > >>>> >>> >> > > > > >> one
> > > >>>> >>> >> > > > > >> > > by
> > > >>>> >>> >> > > > > >> > > > > the
> > > >>>> >>> >> > > > > >> > > > > >> end
> > > >>>> >>> >> > > > > >> > > > > >> > > of
> > > >>>> >>> >> > > > > >> > > > > >> > > > > the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > year, it
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > seems a
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > bit
> > > close to
> > > >>>> >>> still
> > > >>>> >>> >> be
> > > >>>> >>> >> > > > adding
> > > >>>> >>> >> > > > > in
> > > >>>> >>> >> > > > > >> > > > > "features
> > > >>>> >>> >> > > > > >> > > > > >> > that
> > > >>>> >>> >> > > > > >> > > > > >> > > > need
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > maturing"
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > on
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > the
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> branch.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The
> > lack
> > > of
> > > >>>> a
> > > >>>> >>> >> concrete
> > > >>>> >>> >> > > plan
> > > >>>> >>> >> > > > > for
> > > >>>> >>> >> > > > > >> > 2.0
> > > >>>> >>> >> > > > > >> > > > > keeps
> > > >>>> >>> >> > > > > >> > > > > >> me
> > > >>>> >>> >> > > > > >> > > > > >> > > from
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > considering
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > these
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> things
> > > >>>> blocker
> > > >>>> >>> at
> > > >>>> >>> >> the
> > > >>>> >>> >> > > > moment.
> > > >>>> >>> >> > > > > >> But
> > > >>>> >>> >> > > > > >> > I
> > > >>>> >>> >> > > > > >> > > > know
> > > >>>> >>> >> > > > > >> > > > > >> > first
> > > >>>> >>> >> > > > > >> > > > > >> > > > hand
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> how
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > much
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > trouble
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > folks
> > > have
> > > >>>> had
> > > >>>> >>> with
> > > >>>> >>> >> > other
> > > >>>> >>> >> > > > > >> features
> > > >>>> >>> >> > > > > >> > > > that
> > > >>>> >>> >> > > > > >> > > > > >> have
> > > >>>> >>> >> > > > > >> > > > > >> > > gone
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> into
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > downstream
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> facing
> > > >>>> releases
> > > >>>> >>> >> without
> > > >>>> >>> >> > > > > >> robustness
> > > >>>> >>> >> > > > > >> > > > > checks
> > > >>>> >>> >> > > > > >> > > > > >> > (i.e.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > replication),
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > and
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > I'm
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > concerned
> > > >>>> about
> > > >>>> >>> what
> > > >>>> >>> >> > > we're
> > > >>>> >>> >> > > > > >> setting
> > > >>>> >>> >> > > > > >> > > up
> > > >>>> >>> >> > > > > >> > > > if
> > > >>>> >>> >> > > > > >> > > > > >> 2.0
> > > >>>> >>> >> > > > > >> > > > > >> > > goes
> > > >>>> >>> >> > > > > >> > > > > >> > > > > out
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > with
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> this
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> feature
> > > in
> > > >>>> its
> > > >>>> >>> >> current
> > > >>>> >>> >> > > > state.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > --
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > Best
> regards,
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >    - Andy
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > Problems
> > > worthy of
> > > >>>> >>> attack
> > > >>>> >>> >> > prove
> > > >>>> >>> >> > > > > their
> > > >>>> >>> >> > > > > >> > > worth
> > > >>>> >>> >> > > > > >> > > > by
> > > >>>> >>> >> > > > > >> > > > > >> > > hitting
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> back. -
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > Piet
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > Hein
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > (via Tom
> > White)
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > --
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > Best regards,
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >    - Andy
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > Problems worthy
> > of
> > > >>>> attack
> > > >>>> >>> >> prove
> > > >>>> >>> >> > > their
> > > >>>> >>> >> > > > > >> worth
> > > >>>> >>> >> > > > > >> > by
> > > >>>> >>> >> > > > > >> > > > > >> hitting
> > > >>>> >>> >> > > > > >> > > > > >> > > > back.
> > > >>>> >>> >> > > > > >> > > > > >> > > > > -
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > Piet
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> Hein
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > (via Tom White)
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >>
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >>
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > > >> > > >
> > > >>>> >>> >> > > > > >> > > > > >> > >
> > > >>>> >>> >> > > > > >> > > > > >> >
> > > >>>> >>> >> > > > > >> > > > > >>
> > > >>>> >>> >> > > > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > >
> > > >>>> >>> >> > > > > >> > > > > --
> > > >>>> >>> >> > > > > >> > > > > busbey
> > > >>>> >>> >> > > > > >> > > > >
> > > >>>> >>> >> > > > > >> > > >
> > > >>>> >>> >> > > > > >> > >
> > > >>>> >>> >> > > > > >> >
> > > >>>> >>> >> > > > > >>
> > > >>>> >>> >> > > > > >
> > > >>>> >>> >> > > > > >
> > > >>>> >>> >> > > > >
> > > >>>> >>> >> > > >
> > > >>>> >>> >> > >
> > > >>>> >>> >> >
> > > >>>> >>> >>
> > > >>>> >>> >>
> > > >>>> >>> >>
> > > >>>> >>> >> --
> > > >>>> >>> >>
> > > >>>> >>> >> -- Appy
> > > >>>> >>> >>
> > > >>>> >>> >
> > > >>>> >>> >
> > > >>>> >>>
> > > >>>> >>
> > > >>>> >>
> > > >>>> >
> > > >>>>
> > > >>>
> > > >>>
> > > >>
> > >
> > >
> > >
> > > --
> > > busbey
> > >
> > >
> >
>
>
>
> --
>
> -- Appy
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Apekshit Sharma <ap...@cloudera.com>]

Is HBASE-7912 the feature branch?
If yes, has it not been rebased to incorporate latest master changes yet?

~/apache/hbase  (master) → g ck apache/HBASE-7912
warning: refname 'apache/HBASE-7912' is ambiguous.
Switched to branch 'apache/HBASE-7912'
~/apache/hbase  (apache/HBASE-7912) → g log HEAD..apache/master | grep
"^commit" | wc -l
     917


On Thu, Oct 13, 2016 at 7:01 AM, Ted Yu <yu...@gmail.com> wrote:

> Once Stack's comments are addressed, we can change the master build to
> using Maven 3.3.3
>
> I will log JIRA for modifying refguide on the upgrade.
>
> Cheers
>
> On Thu, Oct 13, 2016 at 6:42 AM, Sean Busbey <bu...@cloudera.com> wrote:
>
> > Are we requiring maven 3.3.z now? historically our required maven
> > version has been 3.0.5. I don't have an objection to changing it, per
> > se, but I'd like to make sure our docs and CI builds get updated
> > properly.
> >
> > On Wed, Oct 12, 2016 at 4:03 PM, Stack <st...@duboce.net> wrote:
> > > On Wed, Oct 12, 2016 at 1:58 PM, Stack <st...@duboce.net> wrote:
> > >
> > >> Fails again when I do this:
> > >>
> > >> $ mvn clean install -DskipTests assembly:single
> > >> ....
> > >>
> > >>
> > > Ok. Works if I use mvn 3.3.x.
> > > St.Ack
> > >
> > >
> > >
> > >
> > >
> > >>
> > >> [INFO] ------------------------------------------------------------
> > >> ------------
> > >> [INFO] Building Apache HBase - Server 2.0.0-SNAPSHOT
> > >> [INFO] ------------------------------------------------------------
> > >> ------------
> > >> Downloading: file:/home/stack/hbase.git/hbase-server/src/main/site/
> > >> resources/repo/org/apache/hadoop/hadoop-distcp/2.7.1/
> > >> hadoop-distcp-2.7.1.pom
> > >> Exception in thread "pool-1-thread-1" ------------------------------
> > >> ---------------------
> > >> constituent[0]: file:/usr/share/maven/lib/aether-spi.jar
> > >> constituent[1]: file:/usr/share/maven/lib/
> > maven-settings-builder-3.x.jar
> > >> constituent[2]: file:/usr/share/maven/lib/maven-artifact-3.x.jar
> > >> constituent[3]: file:/usr/share/maven/lib/commons-httpclient.jar
> > >> constituent[4]: file:/usr/share/maven/lib/wagon-provider-api.jar
> > >> constituent[5]: file:/usr/share/maven/lib/maven-core-3.x.jar
> > >> constituent[6]: file:/usr/share/maven/lib/aether-impl.jar
> > >> constituent[7]: file:/usr/share/maven/lib/sisu-inject-bean.jar
> > >> constituent[8]: file:/usr/share/maven/lib/sisu-inject-plexus.jar
> > >> constituent[9]: file:/usr/share/maven/lib/wagon-file.jar
> > >> constituent[10]: file:/usr/share/maven/lib/guava.jar
> > >> constituent[11]: file:/usr/share/maven/lib/maven-compat-3.x.jar
> > >> constituent[12]: file:/usr/share/maven/lib/maven-model-3.x.jar
> > >> constituent[13]: file:/usr/share/maven/lib/plexus-interpolation.jar
> > >> constituent[14]: file:/usr/share/maven/lib/commons-codec.jar
> > >> constituent[15]: file:/usr/share/maven/lib/aether-util.jar
> > >> constituent[16]: file:/usr/share/maven/lib/aether-api.jar
> > >> constituent[17]: file:/usr/share/maven/lib/
> maven-model-builder-3.x.jar
> > >> constituent[18]: file:/usr/share/maven/lib/
> > maven-repository-metadata-3.x.
> > >> jar
> > >> constituent[19]: file:/usr/share/maven/lib/maven-settings-3.x.jar
> > >> constituent[20]: file:/usr/share/maven/lib/
> > plexus-component-annotations.
> > >> jar
> > >> constituent[21]: file:/usr/share/maven/lib/commons-cli.jar
> > >> constituent[22]: file:/usr/share/maven/lib/maven-embedder-3.x.jar
> > >> constituent[23]: file:/usr/share/maven/lib/maven-plugin-api-3.x.jar
> > >> constituent[24]: file:/usr/share/maven/lib/wagon-http-shaded.jar
> > >> constituent[25]: file:/usr/share/maven/lib/commons-logging.jar
> > >> constituent[26]: file:/usr/share/maven/lib/plexus-sec-dispatcher.jar
> > >> constituent[27]: file:/usr/share/maven/lib/aether-connector-wagon.jar
> > >> constituent[28]: file:/usr/share/maven/lib/plexus-utils.jar
> > >> constituent[29]: file:/usr/share/maven/lib/sisu-guice.jar
> > >> constituent[30]: file:/usr/share/maven/lib/plexus-cipher.jar
> > >> constituent[31]: file:/usr/share/maven/lib/
> > maven-aether-provider-3.x.jar
> > >> ---------------------------------------------------
> > >> java.lang.NoClassDefFoundError: org/apache/commons/lang/StringUtils
> > >> at org.apache.maven.wagon.providers.file.FileWagon.
> > resolveDestinationPath(
> > >> FileWagon.java:206)
> > >> at org.apache.maven.wagon.providers.file.FileWagon.
> > >> resourceExists(FileWagon.java:265)
> > >> at org.sonatype.aether.connector.wagon.WagonRepositoryConnector$
> > >> GetTask.run(WagonRepositoryConnector.java:577)
> > >> at org.sonatype.aether.util.concurrency.RunnableErrorForwarder$1.run(
> > >> RunnableErrorForwarder.java:60)
> > >> at java.util.concurrent.ThreadPoolExecutor.runWorker(
> > >> ThreadPoolExecutor.java:1142)
> > >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> > >> ThreadPoolExecutor.java:617)
> > >> at java.lang.Thread.run(Thread.java:745)
> > >> Caused by: java.lang.ClassNotFoundException: org.apache.commons.lang.
> > >> StringUtils
> > >> at org.codehaus.plexus.classworlds.strategy.
> > SelfFirstStrategy.loadClass(
> > >> SelfFirstStrategy.java:50)
> > >> at org.codehaus.plexus.classworlds.realm.ClassRealm.
> > >> unsynchronizedLoadClass(ClassRealm.java:259)
> > >> at org.codehaus.plexus.classworlds.realm.ClassRealm.
> > >> loadClass(ClassRealm.java:235)
> > >> at org.codehaus.plexus.classworlds.realm.ClassRealm.
> > >> loadClass(ClassRealm.java:227)
> > >> ... 7 more
> > >> Exception in thread "main" java.lang.NoClassDefFoundError:
> > >> org/apache/commons/lang/StringUtils
> > >> at org.apache.maven.wagon.providers.file.FileWagon.
> > resolveDestinationPath(
> > >> FileWagon.java:206)
> > >> at org.apache.maven.wagon.providers.file.FileWagon.
> > >> resourceExists(FileWagon.java:265)
> > >> at org.sonatype.aether.connector.wagon.WagonRepositoryConnector$
> > >> GetTask.run(WagonRepositoryConnector.java:577)
> > >> at org.sonatype.aether.util.concurrency.RunnableErrorForwarder$1.run(
> > >> RunnableErrorForwarder.java:60)
> > >> at java.util.concurrent.ThreadPoolExecutor.runWorker(
> > >> ThreadPoolExecutor.java:1142)
> > >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> > >> ThreadPoolExecutor.java:617)
> > >> at java.lang.Thread.run(Thread.java:745)
> > >> Caused by: java.lang.ClassNotFoundException: org.apache.commons.lang.
> > >> StringUtils
> > >> at org.codehaus.plexus.classworlds.strategy.
> > SelfFirstStrategy.loadClass(
> > >> SelfFirstStrategy.java:50)
> > >> at org.codehaus.plexus.classworlds.realm.ClassRealm.
> > >> unsynchronizedLoadClass(ClassRealm.java:259)
> > >> at org.codehaus.plexus.classworlds.realm.ClassRealm.
> > >> loadClass(ClassRealm.java:235)
> > >> at org.codehaus.plexus.classworlds.realm.ClassRealm.
> > >> loadClass(ClassRealm.java:227)
> > >>
> > >>
> > >>
> > >> Does it work for you lot? This is branch checkout.
> > >>
> > >> I want assembly to work so I can distribute the build around a
> cluster.
> > >>
> > >> Thanks,
> > >> St.Ack.
> > >>
> > >>
> > >>
> > >> On Wed, Oct 12, 2016 at 1:52 PM, Stack <st...@duboce.net> wrote:
> > >>
> > >>> I did a super-clean and recheck out. Now it works. Sorry for the
> noise.
> > >>> St.Ack
> > >>>
> > >>> On Wed, Oct 12, 2016 at 1:10 PM, Vladimir Rodionov <
> > >>> vladrodionov@gmail.com> wrote:
> > >>>
> > >>>> checked out HBASE-7912
> > >>>>
> > >>>> ran:
> > >>>>
> > >>>> mvn clean install -DskipTests
> > >>>>
> > >>>> successfully.
> > >>>>
> > >>>> -Vlad
> > >>>>
> > >>>> On Wed, Oct 12, 2016 at 1:01 PM, Vladimir Rodionov <
> > >>>> vladrodionov@gmail.com>
> > >>>> wrote:
> > >>>>
> > >>>> > I usually use:
> > >>>> >
> > >>>> > mvn clean install -DskipTests
> > >>>> >
> > >>>> > and it works.
> > >>>> >
> > >>>> > On Wed, Oct 12, 2016 at 1:01 PM, Vladimir Rodionov <
> > >>>> vladrodionov@gmail.com
> > >>>> > > wrote:
> > >>>> >
> > >>>> >> Michael,
> > >>>> >>
> > >>>> >> you can try master + latest patch on HBASE-14123 (v29). No need
> to
> > use
> > >>>> >> HBASE-7912 branch. I will double check HBASE-7912.
> > >>>> >>
> > >>>> >> -Vlad
> > >>>> >>
> > >>>> >> On Wed, Oct 12, 2016 at 12:31 PM, Stack <st...@duboce.net>
> wrote:
> > >>>> >>
> > >>>> >>> More info:
> > >>>> >>>
> > >>>> >>> stack@ve0524:~/hbase.git$ git checkout origin/HBASE-7912 -b
> > 7912v2
> > >>>> >>> Branch 7912v2 set up to track remote branch HBASE-7912 from
> > origin.
> > >>>> >>> Switched to a new branch '7912v2'
> > >>>> >>> stack@ve0524:~/hbase.git$ java -version
> > >>>> >>> java version "1.8.0_101"
> > >>>> >>> Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
> > >>>> >>> Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode)
> > >>>> >>> stack@ve0524:~/hbase.git$ mvn clean install -DskipTests &>
> > >>>> /tmp/out.txt
> > >>>> >>>
> > >>>> >>> ...
> > >>>> >>>
> > >>>> >>> St.Ack
> > >>>> >>>
> > >>>> >>>
> > >>>> >>> On Wed, Oct 12, 2016 at 12:29 PM, Stack <st...@duboce.net>
> wrote:
> > >>>> >>>
> > >>>> >>> > Interesting. When I try it fails w/ below:
> > >>>> >>> >
> > >>>> >>> > [INFO] 26 warnings
> > >>>> >>> > 322 [INFO] ------------------------------
> > >>>> >>> -------------------------------
> > >>>> >>> > 323 [INFO] ------------------------------
> > >>>> >>> -------------------------------
> > >>>> >>> > 324 [ERROR] COMPILATION ERROR :
> > >>>> >>> > 325 [INFO] ------------------------------
> > >>>> >>> -------------------------------
> > >>>> >>> > 326 [ERROR] /home/stack/hbase.git/hbase-
> > common/src/main/java/org/
> > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.j
> > >>>> ava:[48,8]
> > >>>> >>> > org.apache.hadoop.hbase.io.encoding.rowindexV2.
> RowIndexCodecV2
> > is
> > >>>> not
> > >>>> >>> > abstract and does not override abstract method
> > createSeeker(org.ap
> > >>>> >>> >  ache.hadoop.hbase.CellComparator,org.apache.hadoop.hbase.io.
> > >>>> >>> encoding.HFileBlockDecodingContext)
> > >>>> >>> > in org.apache.hadoop.hbase.io.encoding.DataBlockEncoder
> > >>>> >>> > 327 [ERROR] /home/stack/hbase.git/hbase-
> > common/src/main/java/org/
> > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.j
> > >>>> >>> ava:[143,3]
> > >>>> >>> > method does not override or implement a method from a
> supertype
> > >>>> >>> > 328 [ERROR] /home/stack/hbase.git/hbase-
> > common/src/main/java/org/
> > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.j
> > >>>> >>> ava:[147,29]
> > >>>> >>> > incompatible types: java.nio.ByteBuffer cannot be converted to
> > >>>> >>> > org.apache.hadoop.hbase.nio.ByteBuff
> > >>>> >>> > 329 [ERROR] /home/stack/hbase.git/hbase-
> > common/src/main/java/org/
> > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.j
> > >>>> >>> ava:[148,33]
> > >>>> >>> > cannot find symbol
> > >>>> >>> > 330   symbol:   method getKeyDeepCopy()
> > >>>> >>> > 331   location: variable seeker of type
> > org.apache.hadoop.hbase.io
> > >>>> .
> > >>>> >>> > encoding.DataBlockEncoder.EncodedSeeker
> > >>>> >>> > 332 [ERROR] /home/stack/hbase.git/hbase-
> > common/src/main/java/org/
> > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.j
> > >>>> >>> ava:[153,3]
> > >>>> >>> > method does not override or implement a method from a
> supertype
> > >>>> >>> > 333 [ERROR] /home/stack/hbase.git/hbase-
> > common/src/main/java/org/
> > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV1/RowIndexCodecV1.j
> > >>>> ava:[45,8]
> > >>>> >>> > org.apache.hadoop.hbase.io.encoding.rowindexV1.
> RowIndexCodecV1
> > is
> > >>>> not
> > >>>> >>> > abstract and does not override abstract method
> > createSeeker(org.ap
> > >>>> >>> >  ache.hadoop.hbase.CellComparator,org.apache.hadoop.hbase.io.
> > >>>> >>> encoding.HFileBlockDecodingContext)
> > >>>> >>> > in org.apache.hadoop.hbase.io.encoding.DataBlockEncoder
> > >>>> >>> > 334 [ERROR] /home/stack/hbase.git/hbase-
> > common/src/main/java/org/
> > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV1/RowIndexCodecV1.j
> > >>>> >>> ava:[145,3]
> > >>>> >>> > method does not override or implement a method from a
> supertype
> > >>>> >>> > 335 [ERROR] /home/stack/hbase.git/hbase-
> > common/src/main/java/org/
> > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV1/RowIndexCodecV1.j
> > >>>> >>> ava:[158,3]
> > >>>> >>> > method does not override or implement a method from a
> supertype
> > >>>> >>> > 336 [ERROR] /home/stack/hbase.git/hbase-
> > common/src/main/java/org/
> > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
> > >>>> >>> java:[46,8]
> > >>>> >>> > org.apache.hadoop.hbase.io.encoding.rowindexV2.
> RowIndexSeekerV2
> > >>>> is not
> > >>>> >>> > abstract and does not override abstract method
> compareKey(org.ap
> > >>>> >>> >  ache.hadoop.hbase.CellComparator,org.apache.
> hadoop.hbase.Cell)
> > in
> > >>>> >>> > org.apache.hadoop.hbase.io.encoding.DataBlockEncoder.
> > EncodedSeeker
> > >>>> >>> > 337 [ERROR] /home/stack/hbase.git/hbase-
> > common/src/main/java/org/
> > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
> > >>>> >>> java:[79,3]
> > >>>> >>> > method does not override or implement a method from a
> supertype
> > >>>> >>> > 338 [ERROR] /home/stack/hbase.git/hbase-
> > common/src/main/java/org/
> > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
> > >>>> >>> java:[117,3]
> > >>>> >>> > method does not override or implement a method from a
> supertype
> > >>>> >>> > 339 [ERROR] /home/stack/hbase.git/hbase-
> > common/src/main/java/org/
> > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
> > >>>> >>> java:[190,3]
> > >>>> >>> > method does not override or implement a method from a
> supertype
> > >>>> >>> > 340 [ERROR] /home/stack/hbase.git/hbase-
> > common/src/main/java/org/
> > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
> > >>>> >>> java:[214,3]
> > >>>> >>> > method does not override or implement a method from a
> supertype
> > >>>> >>> > 341 [ERROR] /home/stack/hbase.git/hbase-
> > common/src/main/java/org/
> > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
> > >>>> >>> java:[349,3]
> > >>>> >>> > method does not override or implement a method from a
> supertype
> > >>>> >>> > 342 [ERROR] /home/stack/hbase.git/hbase-
> > common/src/main/java/org/
> > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
> > >>>> >>> java:[355,3]
> > >>>> >>> > method does not override or implement a method from a
> supertype
> > >>>> >>> > 343 [ERROR] /home/stack/hbase.git/hbase-
> > common/src/main/java/org/
> > >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
> > >>>> >>> java:[421,36]
> > >>>> >>> > no suitable method found for uncompressTags(java.nio.
> > >>>> >>> > ByteBuffer,byte[],int,int)
> > >>>> >>> > 344     method org.apache.hadoop.hbase.io.
> > TagCompressionContext.
> > >>>> >>> > uncompressTags(java.io.InputStream,byte[],int,int) is not
> > >>>> applicable
> > >>>> >>> > 345       (argument mismatch; java.nio.ByteBuffer cannot be
> > >>>> converted
> > >>>> >>> to
> > >>>> >>> > java.io.InputStream)
> > >>>> >>> > 346     method org.apache.hadoop.hbase.io.
> > TagCompressionContext.
> > >>>> >>> > uncompressTags(org.apache.hadoop.hbase.nio.ByteBuff,
> > byte[],int,int)
> > >>>> is
> > >>>> >>> > not applicable
> > >>>> >>> > 347       (argument mismatch; java.nio.ByteBuffer cannot be
> > >>>> converted
> > >>>> >>> to
> > >>>> >>> > org.apache.hadoop.hbase.nio.ByteBuff)
> > >>>> >>> >
> > >>>> >>> > ....
> > >>>> >>> >
> > >>>> >>> > St.Ack
> > >>>> >>> >
> > >>>> >>> > On Tue, Oct 11, 2016 at 5:12 PM, Apekshit Sharma <
> > >>>> appy@cloudera.com>
> > >>>> >>> > wrote:
> > >>>> >>> >
> > >>>> >>> >> @stack, it compiled for me.
> > >>>> >>> >>
> > >>>> >>> >> Also tried few commands, and have to say, it's well designed
> > from
> > >>>> user
> > >>>> >>> >> commands perspective.
> > >>>> >>> >>
> > >>>> >>> >> On Tue, Oct 11, 2016 at 5:08 PM, Stack <st...@duboce.net>
> > wrote:
> > >>>> >>> >>
> > >>>> >>> >> > On Tue, Oct 11, 2016 at 1:50 PM, Vladimir Rodionov <
> > >>>> >>> >> vladrodionov@gmail.com
> > >>>> >>> >> > >
> > >>>> >>> >> > wrote:
> > >>>> >>> >> >
> > >>>> >>> >> > > Michael,
> > >>>> >>> >> > >
> > >>>> >>> >> > > Its in HBASE-7912
> > >>>> >>> >> > >
> > >>>> >>> >> > > This is tip of git log:
> > >>>> >>> >> > >
> > >>>> >>> >> > > commit a072f6f49a26a7259ff2aaef6cb56d85eb592482
> > >>>> >>> >> > > Author: Frank Welsch <fw...@jps.net>
> > >>>> >>> >> > > Date:   Fri Sep 23 18:00:42 2016 -0400
> > >>>> >>> >> > >
> > >>>> >>> >> > >     HBASE-16574 Book updates for backup and restore
> > >>>> >>> >> > >
> > >>>> >>> >> > > commit b14e2ab1c24e65ff88dd4c579acf83cb4ed0605e
> > >>>> >>> >> > > Author: tedyu <yu...@gmail.com>
> > >>>> >>> >> > > Date:   Wed Oct 5 16:29:40 2016 -0700
> > >>>> >>> >> > >
> > >>>> >>> >> > >     HBASE-16727 Backup refactoring: remove MR
> dependencies
> > >>>> from
> > >>>> >>> >> HMaster
> > >>>> >>> >> > > (Vladimir Rodionov)
> > >>>> >>> >> > >
> > >>>> >>> >> > >
> > >>>> >>> >> > Thanks. I have that. I tried it and it doesn't compile for
> > me.
> > >>>> Does
> > >>>> >>> it
> > >>>> >>> >> > compile for you?
> > >>>> >>> >> > Thanks,
> > >>>> >>> >> > M
> > >>>> >>> >> >
> > >>>> >>> >> >
> > >>>> >>> >> >
> > >>>> >>> >> > > -Vlad
> > >>>> >>> >> > >
> > >>>> >>> >> > > On Tue, Oct 11, 2016 at 1:44 PM, Stack <stack@duboce.net
> >
> > >>>> wrote:
> > >>>> >>> >> > >
> > >>>> >>> >> > > > Which branch do I check out to try it? HBASE-7912 is
> not
> > >>>> it. I
> > >>>> >>> don't
> > >>>> >>> >> > see
> > >>>> >>> >> > > an
> > >>>> >>> >> > > > HBASE-16727...
> > >>>> >>> >> > > > Thanks,
> > >>>> >>> >> > > > M
> > >>>> >>> >> > > >
> > >>>> >>> >> > > >
> > >>>> >>> >> > > > On Tue, Oct 11, 2016 at 11:41 AM, Vladimir Rodionov <
> > >>>> >>> >> > > > vladrodionov@gmail.com>
> > >>>> >>> >> > > > wrote:
> > >>>> >>> >> > > >
> > >>>> >>> >> > > > > The last patch is on review board:
> > >>>> >>> >> > > > > https://reviews.apache.org/r/52748
> > >>>> >>> >> > > > >
> > >>>> >>> >> > > > > On Mon, Oct 10, 2016 at 3:52 PM, Vladimir Rodionov <
> > >>>> >>> >> > > > vladrodionov@gmail.com
> > >>>> >>> >> > > > > >
> > >>>> >>> >> > > > > wrote:
> > >>>> >>> >> > > > >
> > >>>> >>> >> > > > > > >> How hard to put in an hbase-backup module?
> > >>>> hbase-server
> > >>>> >>> is
> > >>>> >>> >> fat
> > >>>> >>> >> > > > enough
> > >>>> >>> >> > > > > > >> already. Could be done as a follow-up.
> > >>>> >>> >> > > > > >
> > >>>> >>> >> > > > > > https://issues.apache.org/jira/browse/HBASE-16727?
> > >>>> >>> >> > > > > > focusedCommentId=15531237&page=com.atlassian.jira.
> > >>>> >>> >> > > > > > plugin.system.issuetabpanels:c
> > >>>> >>> omment-tabpanel#comment-15531237
> > >>>> >>> >> > > > > >
> > >>>> >>> >> > > > > > Can we do merge first? Then we can discuss separate
> > >>>> module.
> > >>>> >>> >> > > > > >
> > >>>> >>> >> > > > > > -Vlad
> > >>>> >>> >> > > > > >
> > >>>> >>> >> > > > > > On Mon, Oct 10, 2016 at 3:44 PM, Ted Yu <
> > >>>> >>> yuzhihong@gmail.com>
> > >>>> >>> >> > wrote:
> > >>>> >>> >> > > > > >
> > >>>> >>> >> > > > > >> Looks like the first quote was cut off.
> > >>>> >>> >> > > > > >> The original sentence was:
> > >>>> >>> >> > > > > >>
> > >>>> >>> >> > > > > >> bq. no mapreduce job launched from master or
> region
> > >>>> server.
> > >>>> >>> >> > > > > >>
> > >>>> >>> >> > > > > >> mapreduce job is launched from the node where
> > command
> > >>>> line
> > >>>> >>> >> tool is
> > >>>> >>> >> > > > run.
> > >>>> >>> >> > > > > >>
> > >>>> >>> >> > > > > >> On Mon, Oct 10, 2016 at 3:38 PM, Stack <
> > >>>> stack@duboce.net>
> > >>>> >>> >> wrote:
> > >>>> >>> >> > > > > >>
> > >>>> >>> >> > > > > >> > bq. launched from master or region server.
> > >>>> >>> >> > > > > >> >
> > >>>> >>> >> > > > > >> > What does this mean please? Has to be run from
> > >>>> Master or
> > >>>> >>> >> > > > RegionServer?
> > >>>> >>> >> > > > > >> Can
> > >>>> >>> >> > > > > >> > it be run from another node altogether?
> > >>>> >>> >> > > > > >> >
> > >>>> >>> >> > > > > >> > On Mon, Oct 10, 2016 at 1:44 PM, Vladimir
> > Rodionov <
> > >>>> >>> >> > > > > >> vladrodionov@gmail.com
> > >>>> >>> >> > > > > >> > >
> > >>>> >>> >> > > > > >> > wrote:
> > >>>> >>> >> > > > > >> >
> > >>>> >>> >> > > > > >> > > >> mapreduce dependency has been moved to
> client
> > >>>> side
> > >>>> >>> - no
> > >>>> >>> >> > > > mapreduce
> > >>>> >>> >> > > > > >> job
> > >>>> >>> >> > > > > >> > >
> > >>>> >>> >> > > > > >> > > 1. We have no code in the client module
> anymore,
> > >>>> due to
> > >>>> >>> >> > > dependency
> > >>>> >>> >> > > > > on
> > >>>> >>> >> > > > > >> > > internal server API (HFile and WAL access).
> > >>>> >>> >> > > > > >> > > 2. Backup/ restore are client - driven
> > operations,
> > >>>> but
> > >>>> >>> all
> > >>>> >>> >> the
> > >>>> >>> >> > > > code
> > >>>> >>> >> > > > > >> > resides
> > >>>> >>> >> > > > > >> > > in the server module
> > >>>> >>> >> > > > > >> > >
> > >>>> >>> >> > > > > >> >
> > >>>> >>> >> > > > > >> > How hard to put in an hbase-backup module?
> > >>>> hbase-server
> > >>>> >>> is
> > >>>> >>> >> fat
> > >>>> >>> >> > > > enough
> > >>>> >>> >> > > > > >> > already. Could be done as a follow-up.
> > >>>> >>> >> > > > > >> >
> > >>>> >>> >> > > > > >> > Thanks,
> > >>>> >>> >> > > > > >> > St.Ack
> > >>>> >>> >> > > > > >> >
> > >>>> >>> >> > > > > >> >
> > >>>> >>> >> > > > > >> >
> > >>>> >>> >> > > > > >> > > 3. No MR in Master, no procedure - driven
> > >>>> execution.
> > >>>> >>> >> > > > > >> > > 4. Old good MR from command-line.
> > >>>> >>> >> > > > > >> > > 5. Security was simplified and now only
> > super-user
> > >>>> is
> > >>>> >>> >> allowed
> > >>>> >>> >> > to
> > >>>> >>> >> > > > run
> > >>>> >>> >> > > > > >> > > backup/restores.
> > >>>> >>> >> > > > > >> > > 6. HBase Backup API was gone due to 1. Now
> only
> > >>>> >>> >> command-line
> > >>>> >>> >> > > > access
> > >>>> >>> >> > > > > to
> > >>>> >>> >> > > > > >> > > backup tools.
> > >>>> >>> >> > > > > >> > >
> > >>>> >>> >> > > > > >> > > These consequences of refactoring has been
> > >>>> discussed in
> > >>>> >>> >> > > > HBASE-16727.
> > >>>> >>> >> > > > > >> > >
> > >>>> >>> >> > > > > >> > > -Vlad
> > >>>> >>> >> > > > > >> > >
> > >>>> >>> >> > > > > >> > >
> > >>>> >>> >> > > > > >> > >
> > >>>> >>> >> > > > > >> > >
> > >>>> >>> >> > > > > >> > > On Mon, Oct 10, 2016 at 1:31 PM, Ted Yu <
> > >>>> >>> >> yuzhihong@gmail.com>
> > >>>> >>> >> > > > > wrote:
> > >>>> >>> >> > > > > >> > >
> > >>>> >>> >> > > > > >> > > > Reviving this thread.
> > >>>> >>> >> > > > > >> > > >
> > >>>> >>> >> > > > > >> > > > The following has taken place:
> > >>>> >>> >> > > > > >> > > >
> > >>>> >>> >> > > > > >> > > > mapreduce dependency has been moved to
> client
> > >>>> side -
> > >>>> >>> no
> > >>>> >>> >> > > > mapreduce
> > >>>> >>> >> > > > > >> job
> > >>>> >>> >> > > > > >> > > > launched from master or region server.
> > >>>> >>> >> > > > > >> > > > document patch (HBASE-16574) has been
> > integrated.
> > >>>> >>> >> > > > > >> > > > Updated mega patch has been attached to
> > >>>> HBASE-14123:
> > >>>> >>> this
> > >>>> >>> >> > > covers
> > >>>> >>> >> > > > > the
> > >>>> >>> >> > > > > >> > > > refactor in #1 above and the protobuf 3
> merge.
> > >>>> >>> >> > > > > >> > > >
> > >>>> >>> >> > > > > >> > > > If community has more feedback on the merge
> > >>>> >>> proposal, I
> > >>>> >>> >> > would
> > >>>> >>> >> > > > love
> > >>>> >>> >> > > > > >> to
> > >>>> >>> >> > > > > >> > > hear
> > >>>> >>> >> > > > > >> > > > it.
> > >>>> >>> >> > > > > >> > > >
> > >>>> >>> >> > > > > >> > > > Thanks
> > >>>> >>> >> > > > > >> > > >
> > >>>> >>> >> > > > > >> > > > On Thu, Sep 22, 2016 at 10:31 AM, Sean
> Busbey
> > <
> > >>>> >>> >> > > > > busbey@cloudera.com>
> > >>>> >>> >> > > > > >> > > wrote:
> > >>>> >>> >> > > > > >> > > >
> > >>>> >>> >> > > > > >> > > > > I'd like to see the docs proposed on
> > >>>> HBASE-16574
> > >>>> >>> >> > integrated
> > >>>> >>> >> > > > into
> > >>>> >>> >> > > > > >> our
> > >>>> >>> >> > > > > >> > > > > project's documentation prior to merge.
> > >>>> >>> >> > > > > >> > > > >
> > >>>> >>> >> > > > > >> > > > > On Thu, Sep 22, 2016 at 9:02 AM, Ted Yu <
> > >>>> >>> >> > > yuzhihong@gmail.com>
> > >>>> >>> >> > > > > >> wrote:
> > >>>> >>> >> > > > > >> > > > > > This feature can be marked experimental
> > due
> > >>>> to
> > >>>> >>> some
> > >>>> >>> >> > > > > limitations
> > >>>> >>> >> > > > > >> > such
> > >>>> >>> >> > > > > >> > > as
> > >>>> >>> >> > > > > >> > > > > > security.
> > >>>> >>> >> > > > > >> > > > > >
> > >>>> >>> >> > > > > >> > > > > > Your previous round of comments have
> been
> > >>>> >>> addressed.
> > >>>> >>> >> > > > > >> > > > > > Command line tool has gone through:
> > >>>> >>> >> > > > > >> > > > > >
> > >>>> >>> >> > > > > >> > > > > > HBASE-16620 Fix backup command-line tool
> > >>>> >>> usability
> > >>>> >>> >> > issues
> > >>>> >>> >> > > > > >> > > > > > HBASE-16655 hbase backup describe with
> > >>>> incorrect
> > >>>> >>> >> backup
> > >>>> >>> >> > id
> > >>>> >>> >> > > > > >> results
> > >>>> >>> >> > > > > >> > in
> > >>>> >>> >> > > > > >> > > > NPE
> > >>>> >>> >> > > > > >> > > > > >
> > >>>> >>> >> > > > > >> > > > > > The updated doc has been attached to
> > >>>> HBASE-16574.
> > >>>> >>> >> > > > > >> > > > > >
> > >>>> >>> >> > > > > >> > > > > > Cheers
> > >>>> >>> >> > > > > >> > > > > >
> > >>>> >>> >> > > > > >> > > > > > On Thu, Sep 22, 2016 at 8:53 AM, Stack <
> > >>>> >>> >> > stack@duboce.net>
> > >>>> >>> >> > > > > >> wrote:
> > >>>> >>> >> > > > > >> > > > > >
> > >>>> >>> >> > > > > >> > > > > >> On Wed, Sep 21, 2016 at 7:43 AM, Ted
> Yu <
> > >>>> >>> >> > > > yuzhihong@gmail.com
> > >>>> >>> >> > > > > >
> > >>>> >>> >> > > > > >> > > wrote:
> > >>>> >>> >> > > > > >> > > > > >>
> > >>>> >>> >> > > > > >> > > > > >> > Are there more (review) comments ?
> > >>>> >>> >> > > > > >> > > > > >> >
> > >>>> >>> >> > > > > >> > > > > >> >
> > >>>> >>> >> > > > > >> > > > > >> Are outstanding comments addressed?
> > >>>> >>> >> > > > > >> > > > > >>
> > >>>> >>> >> > > > > >> > > > > >> I don't see answer to my 'is this
> > >>>> >>> experimental/will
> > >>>> >>> >> it
> > >>>> >>> >> > be
> > >>>> >>> >> > > > > >> marked
> > >>>> >>> >> > > > > >> > > > > >> experimental' question.
> > >>>> >>> >> > > > > >> > > > > >>
> > >>>> >>> >> > > > > >> > > > > >> I ran into some issues trying to use
> the
> > >>>> >>> feature and
> > >>>> >>> >> > > > > suggested
> > >>>> >>> >> > > > > >> > that
> > >>>> >>> >> > > > > >> > > a
> > >>>> >>> >> > > > > >> > > > > >> feature likes this needs polish else
> > it'll
> > >>>> just
> > >>>> >>> rot,
> > >>>> >>> >> > > > unused.
> > >>>> >>> >> > > > > >> Has
> > >>>> >>> >> > > > > >> > > > polish
> > >>>> >>> >> > > > > >> > > > > >> been applied? All ready for another
> > 'user'
> > >>>> test?
> > >>>> >>> >> > Suggest
> > >>>> >>> >> > > > that
> > >>>> >>> >> > > > > >> you
> > >>>> >>> >> > > > > >> > > > update
> > >>>> >>> >> > > > > >> > > > > >> here going forward for the benefit of
> > those
> > >>>> >>> trying
> > >>>> >>> >> to
> > >>>> >>> >> > > > follow
> > >>>> >>> >> > > > > >> along
> > >>>> >>> >> > > > > >> > > and
> > >>>> >>> >> > > > > >> > > > > who
> > >>>> >>> >> > > > > >> > > > > >> are not watching JIRA change fly-by.
> > >>>> >>> >> > > > > >> > > > > >>
> > >>>> >>> >> > > > > >> > > > > >> It looks like doc got a revision -- I
> > have
> > >>>> to
> > >>>> >>> check
> > >>>> >>> >> --
> > >>>> >>> >> > to
> > >>>> >>> >> > > > > take
> > >>>> >>> >> > > > > >> on
> > >>>> >>> >> > > > > >> > > > > >> suggestion made above but again,
> suggest,
> > >>>> that
> > >>>> >>> this
> > >>>> >>> >> > > thread
> > >>>> >>> >> > > > > gets
> > >>>> >>> >> > > > > >> > > > updated.
> > >>>> >>> >> > > > > >> > > > > >>
> > >>>> >>> >> > > > > >> > > > > >> Thanks,
> > >>>> >>> >> > > > > >> > > > > >> St.Ack
> > >>>> >>> >> > > > > >> > > > > >>
> > >>>> >>> >> > > > > >> > > > > >>
> > >>>> >>> >> > > > > >> > > > > >>
> > >>>> >>> >> > > > > >> > > > > >> > Thanks
> > >>>> >>> >> > > > > >> > > > > >> >
> > >>>> >>> >> > > > > >> > > > > >> > On Tue, Sep 20, 2016 at 10:02 AM,
> > Devaraj
> > >>>> Das
> > >>>> >>> <
> > >>>> >>> >> > > > > >> > > ddas@hortonworks.com
> > >>>> >>> >> > > > > >> > > > >
> > >>>> >>> >> > > > > >> > > > > >> > wrote:
> > >>>> >>> >> > > > > >> > > > > >> >
> > >>>> >>> >> > > > > >> > > > > >> > > Just reviving this thread. Thanks
> > Sean,
> > >>>> >>> Stack,
> > >>>> >>> >> > Dima,
> > >>>> >>> >> > > > and
> > >>>> >>> >> > > > > >> > others
> > >>>> >>> >> > > > > >> > > > for
> > >>>> >>> >> > > > > >> > > > > the
> > >>>> >>> >> > > > > >> > > > > >> > > thorough reviews and testing.
> Thanks
> > >>>> Ted and
> > >>>> >>> >> Vlad
> > >>>> >>> >> > for
> > >>>> >>> >> > > > > >> taking
> > >>>> >>> >> > > > > >> > > care
> > >>>> >>> >> > > > > >> > > > of
> > >>>> >>> >> > > > > >> > > > > >> the
> > >>>> >>> >> > > > > >> > > > > >> > > feedback. Are we all good to do the
> > >>>> merge
> > >>>> >>> now?
> > >>>> >>> >> > Rather
> > >>>> >>> >> > > > do
> > >>>> >>> >> > > > > >> > sooner
> > >>>> >>> >> > > > > >> > > > than
> > >>>> >>> >> > > > > >> > > > > >> > later.
> > >>>> >>> >> > > > > >> > > > > >> > > ______________________________
> > >>>> __________
> > >>>> >>> >> > > > > >> > > > > >> > > From: saint.ack@gmail.com <
> > >>>> >>> saint.ack@gmail.com>
> > >>>> >>> >> on
> > >>>> >>> >> > > > > behalf
> > >>>> >>> >> > > > > >> of
> > >>>> >>> >> > > > > >> > > > Stack
> > >>>> >>> >> > > > > >> > > > > <
> > >>>> >>> >> > > > > >> > > > > >> > > stack@duboce.net>
> > >>>> >>> >> > > > > >> > > > > >> > > Sent: Monday, September 12, 2016
> > 1:18 PM
> > >>>> >>> >> > > > > >> > > > > >> > > To: HBase Dev List
> > >>>> >>> >> > > > > >> > > > > >> > > Subject: Re: [DISCUSSION] Merge
> > Backup /
> > >>>> >>> >> Restore -
> > >>>> >>> >> > > > Branch
> > >>>> >>> >> > > > > >> > > > HBASE-7912
> > >>>> >>> >> > > > > >> > > > > >> > >
> > >>>> >>> >> > > > > >> > > > > >> > > On Mon, Sep 12, 2016 at 12:19 PM,
> Ted
> > >>>> Yu <
> > >>>> >>> >> > > > > >> yuzhihong@gmail.com
> > >>>> >>> >> > > > > >> > >
> > >>>> >>> >> > > > > >> > > > > wrote:
> > >>>> >>> >> > > > > >> > > > > >> > >
> > >>>> >>> >> > > > > >> > > > > >> > > > Mega patch (rev 18) is on
> > HBASE-14123.
> > >>>> >>> >> > > > > >> > > > > >> > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > Please comment on HBASE-14123 on
> > how
> > >>>> you
> > >>>> >>> want
> > >>>> >>> >> to
> > >>>> >>> >> > > > > review.
> > >>>> >>> >> > > > > >> > > > > >> > > >
> > >>>> >>> >> > > > > >> > > > > >> > >
> > >>>> >>> >> > > > > >> > > > > >> > >
> > >>>> >>> >> > > > > >> > > > > >> > > Yeah. That was my lost tab. Last rb
> > was
> > >>>> 6
> > >>>> >>> months
> > >>>> >>> >> > ago.
> > >>>> >>> >> > > > > >> Suggest
> > >>>> >>> >> > > > > >> > > > > updating
> > >>>> >>> >> > > > > >> > > > > >> > it.
> > >>>> >>> >> > > > > >> > > > > >> > > RB is pretty good for review. Patch
> > is
> > >>>> only
> > >>>> >>> >> 1.5M so
> > >>>> >>> >> > > > > should
> > >>>> >>> >> > > > > >> be
> > >>>> >>> >> > > > > >> > > > fine.
> > >>>> >>> >> > > > > >> > > > > >> > >
> > >>>> >>> >> > > > > >> > > > > >> > > St.Ack
> > >>>> >>> >> > > > > >> > > > > >> > >
> > >>>> >>> >> > > > > >> > > > > >> > >
> > >>>> >>> >> > > > > >> > > > > >> > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > Thanks
> > >>>> >>> >> > > > > >> > > > > >> > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > On Mon, Sep 12, 2016 at 12:15 PM,
> > >>>> Stack <
> > >>>> >>> >> > > > > >> stack@duboce.net>
> > >>>> >>> >> > > > > >> > > > wrote:
> > >>>> >>> >> > > > > >> > > > > >> > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > On review of the 'patch', do I
> > just
> > >>>> >>> compare
> > >>>> >>> >> the
> > >>>> >>> >> > > > > branch
> > >>>> >>> >> > > > > >> to
> > >>>> >>> >> > > > > >> > > > > master or
> > >>>> >>> >> > > > > >> > > > > >> > is
> > >>>> >>> >> > > > > >> > > > > >> > > > > there a megapatch posted
> > somewhere
> > >>>> (I
> > >>>> >>> think
> > >>>> >>> >> I
> > >>>> >>> >> > saw
> > >>>> >>> >> > > > one
> > >>>> >>> >> > > > > >> but
> > >>>> >>> >> > > > > >> > it
> > >>>> >>> >> > > > > >> > > > > seemed
> > >>>> >>> >> > > > > >> > > > > >> > > stale
> > >>>> >>> >> > > > > >> > > > > >> > > > > and then I 'lost' the tab).
> Sorry
> > >>>> for
> > >>>> >>> dumb
> > >>>> >>> >> > > > question.
> > >>>> >>> >> > > > > >> > > > > >> > > > > St.Ack
> > >>>> >>> >> > > > > >> > > > > >> > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > On Mon, Sep 12, 2016 at 12:01
> PM,
> > >>>> Stack
> > >>>> >>> <
> > >>>> >>> >> > > > > >> stack@duboce.net
> > >>>> >>> >> > > > > >> > >
> > >>>> >>> >> > > > > >> > > > > wrote:
> > >>>> >>> >> > > > > >> > > > > >> > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > > Late to the game. A few
> > comments
> > >>>> after
> > >>>> >>> >> > > rereading
> > >>>> >>> >> > > > > this
> > >>>> >>> >> > > > > >> > > thread
> > >>>> >>> >> > > > > >> > > > > as a
> > >>>> >>> >> > > > > >> > > > > >> > > > 'user'.
> > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > > + Before merge, a user-facing
> > >>>> feature
> > >>>> >>> like
> > >>>> >>> >> > this
> > >>>> >>> >> > > > > >> should
> > >>>> >>> >> > > > > >> > > work
> > >>>> >>> >> > > > > >> > > > > (If
> > >>>> >>> >> > > > > >> > > > > >> > this
> > >>>> >>> >> > > > > >> > > > > >> > > is
> > >>>> >>> >> > > > > >> > > > > >> > > > > "higher-bar
> > >>>> >>> >> > > > > >> > > > > >> > > > > > for new features", bring it
> on
> > --
> > >>>> >>> smile).
> > >>>> >>> >> > > > > >> > > > > >> > > > > > + As a user, I tried the
> branch
> > >>>> with
> > >>>> >>> tools
> > >>>> >>> >> > > after
> > >>>> >>> >> > > > > >> > reviewing
> > >>>> >>> >> > > > > >> > > > the
> > >>>> >>> >> > > > > >> > > > > >> > > > > just-posted
> > >>>> >>> >> > > > > >> > > > > >> > > > > > doc. I had an 'interesting'
> > >>>> experience
> > >>>> >>> >> (left
> > >>>> >>> >> > > > > >> comments up
> > >>>> >>> >> > > > > >> > > on
> > >>>> >>> >> > > > > >> > > > > >> > issue). I
> > >>>> >>> >> > > > > >> > > > > >> > > > > think
> > >>>> >>> >> > > > > >> > > > > >> > > > > > the tooling/doc. important to
> > get
> > >>>> >>> right.
> > >>>> >>> >> If
> > >>>> >>> >> > it
> > >>>> >>> >> > > > > breaks
> > >>>> >>> >> > > > > >> > > easily
> > >>>> >>> >> > > > > >> > > > > or
> > >>>> >>> >> > > > > >> > > > > >> is
> > >>>> >>> >> > > > > >> > > > > >> > > > > > inconsistent (or lacks
> > 'polish'),
> > >>>> >>> >> operators
> > >>>> >>> >> > > will
> > >>>> >>> >> > > > > >> judge
> > >>>> >>> >> > > > > >> > the
> > >>>> >>> >> > > > > >> > > > > whole
> > >>>> >>> >> > > > > >> > > > > >> > > > > > backup/restore tooling chain
> as
> > >>>> not
> > >>>> >>> >> > trustworthy
> > >>>> >>> >> > > > and
> > >>>> >>> >> > > > > >> > > abandon
> > >>>> >>> >> > > > > >> > > > > it.
> > >>>> >>> >> > > > > >> > > > > >> > Lets
> > >>>> >>> >> > > > > >> > > > > >> > > > not
> > >>>> >>> >> > > > > >> > > > > >> > > > > > have this happen to this
> > feature.
> > >>>> >>> >> > > > > >> > > > > >> > > > > > + Matteo's suggestion (with a
> > >>>> helpful
> > >>>> >>> >> starter
> > >>>> >>> >> > > > list)
> > >>>> >>> >> > > > > >> that
> > >>>> >>> >> > > > > >> > > > there
> > >>>> >>> >> > > > > >> > > > > >> > needs
> > >>>> >>> >> > > > > >> > > > > >> > > to
> > >>>> >>> >> > > > > >> > > > > >> > > > > be
> > >>>> >>> >> > > > > >> > > > > >> > > > > > explicit qualification on
> what
> > is
> > >>>> >>> actually
> > >>>> >>> >> > > being
> > >>>> >>> >> > > > > >> > delivered
> > >>>> >>> >> > > > > >> > > > --
> > >>>> >>> >> > > > > >> > > > > >> > > > including a
> > >>>> >>> >> > > > > >> > > > > >> > > > > > listing of limitations (some
> > look
> > >>>> >>> serious
> > >>>> >>> >> > such
> > >>>> >>> >> > > as
> > >>>> >>> >> > > > > >> data
> > >>>> >>> >> > > > > >> > > bleed
> > >>>> >>> >> > > > > >> > > > > from
> > >>>> >>> >> > > > > >> > > > > >> > > other
> > >>>> >>> >> > > > > >> > > > > >> > > > > > regions in WALs, but maybe I
> > don't
> > >>>> >>> care
> > >>>> >>> >> for
> > >>>> >>> >> > my
> > >>>> >>> >> > > > use
> > >>>> >>> >> > > > > >> > > case...)
> > >>>> >>> >> > > > > >> > > > --
> > >>>> >>> >> > > > > >> > > > > >> > needs
> > >>>> >>> >> > > > > >> > > > > >> > > to
> > >>>> >>> >> > > > > >> > > > > >> > > > > > accompany the merge. Lets
> fold
> > >>>> them
> > >>>> >>> into
> > >>>> >>> >> the
> > >>>> >>> >> > > user
> > >>>> >>> >> > > > > >> doc.
> > >>>> >>> >> > > > > >> > in
> > >>>> >>> >> > > > > >> > > > the
> > >>>> >>> >> > > > > >> > > > > >> > > technical
> > >>>> >>> >> > > > > >> > > > > >> > > > > > overview area as suggested so
> > user
> > >>>> >>> >> > expectations
> > >>>> >>> >> > > > are
> > >>>> >>> >> > > > > >> > > properly
> > >>>> >>> >> > > > > >> > > > > >> > managed
> > >>>> >>> >> > > > > >> > > > > >> > > > > > (otherwise, they expect the
> > world
> > >>>> and
> > >>>> >>> will
> > >>>> >>> >> > just
> > >>>> >>> >> > > > > give
> > >>>> >>> >> > > > > >> up
> > >>>> >>> >> > > > > >> > > when
> > >>>> >>> >> > > > > >> > > > > we
> > >>>> >>> >> > > > > >> > > > > >> > fall
> > >>>> >>> >> > > > > >> > > > > >> > > > > > short). Vladimir did a list
> of
> > >>>> what
> > >>>> >>> is in
> > >>>> >>> >> > each
> > >>>> >>> >> > > of
> > >>>> >>> >> > > > > the
> > >>>> >>> >> > > > > >> > > phases
> > >>>> >>> >> > > > > >> > > > > >> above
> > >>>> >>> >> > > > > >> > > > > >> > > > which
> > >>>> >>> >> > > > > >> > > > > >> > > > > > would serve as a good start.
> > >>>> >>> >> > > > > >> > > > > >> > > > > > + Is this feature
> > 'experimental'
> > >>>> >>> (Matteo
> > >>>> >>> >> asks
> > >>>> >>> >> > > > > above).
> > >>>> >>> >> > > > > >> > I'd
> > >>>> >>> >> > > > > >> > > > > prefer
> > >>>> >>> >> > > > > >> > > > > >> it
> > >>>> >>> >> > > > > >> > > > > >> > > is
> > >>>> >>> >> > > > > >> > > > > >> > > > > > not. If it is, it should be
> > >>>> labelled
> > >>>> >>> all
> > >>>> >>> >> over
> > >>>> >>> >> > > > that
> > >>>> >>> >> > > > > >> it is
> > >>>> >>> >> > > > > >> > > > so. I
> > >>>> >>> >> > > > > >> > > > > >> see
> > >>>> >>> >> > > > > >> > > > > >> > > > > current
> > >>>> >>> >> > > > > >> > > > > >> > > > > > state called out as a '...
> > >>>> technical
> > >>>> >>> >> preview
> > >>>> >>> >> > > > > >> feature'.
> > >>>> >>> >> > > > > >> > > Does
> > >>>> >>> >> > > > > >> > > > > this
> > >>>> >>> >> > > > > >> > > > > >> > mean
> > >>>> >>> >> > > > > >> > > > > >> > > > > > not-for-users?
> > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > > St.Ack
> > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > > On Mon, Sep 12, 2016 at 8:03
> > AM,
> > >>>> Ted
> > >>>> >>> Yu <
> > >>>> >>> >> > > > > >> > > > yuzhihong@gmail.com>
> > >>>> >>> >> > > > > >> > > > > >> > wrote:
> > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> Sean:
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> Do you have more comments ?
> > >>>> >>> >> > > > > >> > > > > >> > > > > >>
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> Cheers
> > >>>> >>> >> > > > > >> > > > > >> > > > > >>
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> On Fri, Sep 9, 2016 at 1:42
> > PM,
> > >>>> >>> Vladimir
> > >>>> >>> >> > > > Rodionov
> > >>>> >>> >> > > > > <
> > >>>> >>> >> > > > > >> > > > > >> > > > > vladrodionov@gmail.com
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> wrote:
> > >>>> >>> >> > > > > >> > > > > >> > > > > >>
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > Sean,
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > Backup/Restore can fail
> due
> > to
> > >>>> >>> various
> > >>>> >>> >> > > > reasons:
> > >>>> >>> >> > > > > >> > network
> > >>>> >>> >> > > > > >> > > > > outage
> > >>>> >>> >> > > > > >> > > > > >> > > > > (cluster
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > wide), various time-outs
> in
> > >>>> HBase
> > >>>> >>> and
> > >>>> >>> >> HDFS
> > >>>> >>> >> > > > > layer,
> > >>>> >>> >> > > > > >> M/R
> > >>>> >>> >> > > > > >> > > > > failure
> > >>>> >>> >> > > > > >> > > > > >> > due
> > >>>> >>> >> > > > > >> > > > > >> > > to
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> "HDFS
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > exceeded quota", user
> error
> > >>>> (manual
> > >>>> >>> >> > deletion
> > >>>> >>> >> > > > of
> > >>>> >>> >> > > > > >> data)
> > >>>> >>> >> > > > > >> > > and
> > >>>> >>> >> > > > > >> > > > > so
> > >>>> >>> >> > > > > >> > > > > >> on
> > >>>> >>> >> > > > > >> > > > > >> > so
> > >>>> >>> >> > > > > >> > > > > >> > > > on.
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> That
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > is impossible to enumerate
> > all
> > >>>> >>> possible
> > >>>> >>> >> > > types
> > >>>> >>> >> > > > of
> > >>>> >>> >> > > > > >> > > failures
> > >>>> >>> >> > > > > >> > > > > in a
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> distributed
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > system - that is not our
> > >>>> goal/task.
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > We focus completely on
> > backup
> > >>>> >>> system
> > >>>> >>> >> table
> > >>>> >>> >> > > > > >> > consistency
> > >>>> >>> >> > > > > >> > > > in a
> > >>>> >>> >> > > > > >> > > > > >> > > presence
> > >>>> >>> >> > > > > >> > > > > >> > > > > of
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> any
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > type of failure. That is
> > what I
> > >>>> >>> call
> > >>>> >>> >> > > > "tolerance
> > >>>> >>> >> > > > > to
> > >>>> >>> >> > > > > >> > > > > failures".
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > On a failure:
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > BACKUP. All backup system
> > >>>> >>> information
> > >>>> >>> >> > (prior
> > >>>> >>> >> > > > to
> > >>>> >>> >> > > > > >> > backup)
> > >>>> >>> >> > > > > >> > > > > will
> > >>>> >>> >> > > > > >> > > > > >> be
> > >>>> >>> >> > > > > >> > > > > >> > > > > restored
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > and all temporary data,
> > >>>> related to
> > >>>> >>> a
> > >>>> >>> >> > failed
> > >>>> >>> >> > > > > >> session,
> > >>>> >>> >> > > > > >> > in
> > >>>> >>> >> > > > > >> > > > > HDFS
> > >>>> >>> >> > > > > >> > > > > >> > will
> > >>>> >>> >> > > > > >> > > > > >> > > be
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > deleted
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > RESTORE. We do not care
> > about
> > >>>> >>> system
> > >>>> >>> >> data,
> > >>>> >>> >> > > > > because
> > >>>> >>> >> > > > > >> > > > restore
> > >>>> >>> >> > > > > >> > > > > >> does
> > >>>> >>> >> > > > > >> > > > > >> > > not
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> change
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > it. Temporary data in HDFS
> > >>>> will be
> > >>>> >>> >> cleaned
> > >>>> >>> >> > > up
> > >>>> >>> >> > > > > and
> > >>>> >>> >> > > > > >> > table
> > >>>> >>> >> > > > > >> > > > > will
> > >>>> >>> >> > > > > >> > > > > >> be
> > >>>> >>> >> > > > > >> > > > > >> > > in a
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> state
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > back to where it was
> before
> > >>>> >>> operation
> > >>>> >>> >> > > started.
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > This is what user should
> > >>>> expect in
> > >>>> >>> case
> > >>>> >>> >> > of a
> > >>>> >>> >> > > > > >> failure.
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > -Vlad
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > -Vlad
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > On Fri, Sep 9, 2016 at
> 12:56
> > >>>> PM,
> > >>>> >>> Sean
> > >>>> >>> >> > > Busbey <
> > >>>> >>> >> > > > > >> > > > > >> busbey@apache.org
> > >>>> >>> >> > > > > >> > > > > >> > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > wrote:
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > Failing in a consistent
> > way,
> > >>>> with
> > >>>> >>> >> docs
> > >>>> >>> >> > > that
> > >>>> >>> >> > > > > >> explain
> > >>>> >>> >> > > > > >> > > the
> > >>>> >>> >> > > > > >> > > > > >> > various
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > expected failures would
> be
> > >>>> >>> >> sufficient.
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > On Fri, Sep 9, 2016 at
> > 12:16
> > >>>> PM,
> > >>>> >>> >> > Vladimir
> > >>>> >>> >> > > > > >> Rodionov
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > <vladrodionov@gmail.com
> >
> > >>>> wrote:
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > Do not worry Sean, doc
> > is
> > >>>> >>> coming
> > >>>> >>> >> today
> > >>>> >>> >> > > as
> > >>>> >>> >> > > > a
> > >>>> >>> >> > > > > >> > preview
> > >>>> >>> >> > > > > >> > > > and
> > >>>> >>> >> > > > > >> > > > > >> our
> > >>>> >>> >> > > > > >> > > > > >> > > > writer
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > Frank
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > will be working on a
> > >>>> putting
> > >>>> >>> it
> > >>>> >>> >> into
> > >>>> >>> >> > > > Apache
> > >>>> >>> >> > > > > >> > repo.
> > >>>> >>> >> > > > > >> > > > > >> Timeline
> > >>>> >>> >> > > > > >> > > > > >> > > > > depends
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> on
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > Franks schedule but I
> > hope
> > >>>> we
> > >>>> >>> will
> > >>>> >>> >> get
> > >>>> >>> >> > > it
> > >>>> >>> >> > > > > >> rather
> > >>>> >>> >> > > > > >> > > > sooner
> > >>>> >>> >> > > > > >> > > > > >> than
> > >>>> >>> >> > > > > >> > > > > >> > > > > later.
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > As for failure
> testing,
> > we
> > >>>> are
> > >>>> >>> >> > focusing
> > >>>> >>> >> > > > only
> > >>>> >>> >> > > > > >> on a
> > >>>> >>> >> > > > > >> > > > > >> consistent
> > >>>> >>> >> > > > > >> > > > > >> > > > state
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> of
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > backup system data in
> a
> > >>>> >>> presence of
> > >>>> >>> >> > any
> > >>>> >>> >> > > > type
> > >>>> >>> >> > > > > >> of
> > >>>> >>> >> > > > > >> > > > > failures,
> > >>>> >>> >> > > > > >> > > > > >> We
> > >>>> >>> >> > > > > >> > > > > >> > > are
> > >>>> >>> >> > > > > >> > > > > >> > > > > not
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > going
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > to implement  anything
> > more
> > >>>> >>> >> "fancy",
> > >>>> >>> >> > > than
> > >>>> >>> >> > > > > >> that.
> > >>>> >>> >> > > > > >> > We
> > >>>> >>> >> > > > > >> > > > > allow
> > >>>> >>> >> > > > > >> > > > > >> > both:
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> backup
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > and
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > restore to fail. What
> we
> > >>>> do not
> > >>>> >>> >> allow
> > >>>> >>> >> > is
> > >>>> >>> >> > > > to
> > >>>> >>> >> > > > > >> have
> > >>>> >>> >> > > > > >> > > > system
> > >>>> >>> >> > > > > >> > > > > >> data
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> corrupted.
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > Will it suffice for
> you?
> > >>>> Do you
> > >>>> >>> >> have
> > >>>> >>> >> > any
> > >>>> >>> >> > > > > other
> > >>>> >>> >> > > > > >> > > > > concerns,
> > >>>> >>> >> > > > > >> > > > > >> you
> > >>>> >>> >> > > > > >> > > > > >> > > > want
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> us to
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > address?
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > -Vlad
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > On Fri, Sep 9, 2016 at
> > >>>> 10:56
> > >>>> >>> AM,
> > >>>> >>> >> Sean
> > >>>> >>> >> > > > > Busbey <
> > >>>> >>> >> > > > > >> > > > > >> > > busbey@apache.org
> > >>>> >>> >> > > > > >> > > > > >> > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > wrote:
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> "docs will come to
> > Apache
> > >>>> >>> soon"
> > >>>> >>> >> does
> > >>>> >>> >> > > not
> > >>>> >>> >> > > > > >> address
> > >>>> >>> >> > > > > >> > > my
> > >>>> >>> >> > > > > >> > > > > >> concern
> > >>>> >>> >> > > > > >> > > > > >> > > > > around
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > docs
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > at
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> all, unless said docs
> > have
> > >>>> >>> already
> > >>>> >>> >> > made
> > >>>> >>> >> > > > it
> > >>>> >>> >> > > > > >> into
> > >>>> >>> >> > > > > >> > > the
> > >>>> >>> >> > > > > >> > > > > >> project
> > >>>> >>> >> > > > > >> > > > > >> > > > > repo. I
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > don't
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> want third party
> > >>>> resources for
> > >>>> >>> >> using
> > >>>> >>> >> > a
> > >>>> >>> >> > > > > major
> > >>>> >>> >> > > > > >> and
> > >>>> >>> >> > > > > >> > > > > >> important
> > >>>> >>> >> > > > > >> > > > > >> > > > > feature
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> of
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > the
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> project, I want us to
> > >>>> provide
> > >>>> >>> end
> > >>>> >>> >> > users
> > >>>> >>> >> > > > > with
> > >>>> >>> >> > > > > >> > what
> > >>>> >>> >> > > > > >> > > > they
> > >>>> >>> >> > > > > >> > > > > >> need
> > >>>> >>> >> > > > > >> > > > > >> > > to
> > >>>> >>> >> > > > > >> > > > > >> > > > > get
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> the
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > job
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> done.
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >>
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> I see some calls for
> > >>>> patience
> > >>>> >>> on
> > >>>> >>> >> the
> > >>>> >>> >> > > > > failure
> > >>>> >>> >> > > > > >> > > > testing,
> > >>>> >>> >> > > > > >> > > > > but
> > >>>> >>> >> > > > > >> > > > > >> > the
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> appeal
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > to
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > us
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> having done a bad job
> > of
> > >>>> >>> requiring
> > >>>> >>> >> > > proper
> > >>>> >>> >> > > > > >> tests
> > >>>> >>> >> > > > > >> > of
> > >>>> >>> >> > > > > >> > > > > >> previous
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> features
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > just
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> makes me more
> concerned
> > >>>> about
> > >>>> >>> not
> > >>>> >>> >> > > getting
> > >>>> >>> >> > > > > >> them
> > >>>> >>> >> > > > > >> > > > here. I
> > >>>> >>> >> > > > > >> > > > > >> > don't
> > >>>> >>> >> > > > > >> > > > > >> > > > want
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> to
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > set
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> yet another bad
> example
> > >>>> that
> > >>>> >>> will
> > >>>> >>> >> > then
> > >>>> >>> >> > > be
> > >>>> >>> >> > > > > >> > pointed
> > >>>> >>> >> > > > > >> > > to
> > >>>> >>> >> > > > > >> > > > > in
> > >>>> >>> >> > > > > >> > > > > >> the
> > >>>> >>> >> > > > > >> > > > > >> > > > > future.
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >>
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> On Sep 8, 2016 10:50,
> > "Ted
> > >>>> >>> Yu" <
> > >>>> >>> >> > > > > >> > > yuzhihong@gmail.com
> > >>>> >>> >> > > > > >> > > > >
> > >>>> >>> >> > > > > >> > > > > >> > wrote:
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >>
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > Is there any
> concern
> > >>>> which
> > >>>> >>> is
> > >>>> >>> >> not
> > >>>> >>> >> > > > > >> addressed ?
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > Do we need another
> > Vote
> > >>>> >>> thread ?
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > Thanks
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > On Thu, Sep 8, 2016
> > at
> > >>>> 9:21
> > >>>> >>> AM,
> > >>>> >>> >> > > Andrew
> > >>>> >>> >> > > > > >> > Purtell <
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> apurtell@apache.org
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > wrote:
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > Vlad,
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > I apologize for
> > using
> > >>>> the
> > >>>> >>> term
> > >>>> >>> >> > > > > >> 'half-baked'
> > >>>> >>> >> > > > > >> > > in a
> > >>>> >>> >> > > > > >> > > > > way
> > >>>> >>> >> > > > > >> > > > > >> > that
> > >>>> >>> >> > > > > >> > > > > >> > > > > could
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > seem a
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > description of
> > >>>> >>> HBASE-7912. I
> > >>>> >>> >> > meant
> > >>>> >>> >> > > > that
> > >>>> >>> >> > > > > >> as a
> > >>>> >>> >> > > > > >> > > > > general
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> hypothetical.
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > On Wed, Sep 7,
> > 2016 at
> > >>>> >>> 9:36
> > >>>> >>> >> AM,
> > >>>> >>> >> > > > > Vladimir
> > >>>> >>> >> > > > > >> > > > Rodionov
> > >>>> >>> >> > > > > >> > > > > <
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> > vladrodionov@gmail.com>
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > wrote:
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >> I'm not sure
> > that
> > >>>> >>> "There
> > >>>> >>> >> is
> > >>>> >>> >> > > > > already
> > >>>> >>> >> > > > > >> > lots
> > >>>> >>> >> > > > > >> > > of
> > >>>> >>> >> > > > > >> > > > > >> > > half-baked
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> code
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > in
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > the
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > branch,
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > so what's the
> > harm
> > >>>> in
> > >>>> >>> adding
> > >>>> >>> >> > > more?"
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > I meant - not
> > >>>> >>> production -
> > >>>> >>> >> > ready
> > >>>> >>> >> > > > yet.
> > >>>> >>> >> > > > > >> This
> > >>>> >>> >> > > > > >> > > is
> > >>>> >>> >> > > > > >> > > > > 2.0
> > >>>> >>> >> > > > > >> > > > > >> > > > > development
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > branch
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > and,
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > hence many
> > features
> > >>>> are
> > >>>> >>> in
> > >>>> >>> >> > works,
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > not being
> tested
> > >>>> well
> > >>>> >>> etc.
> > >>>> >>> >> I do
> > >>>> >>> >> > > not
> > >>>> >>> >> > > > > >> > consider
> > >>>> >>> >> > > > > >> > > > > backup
> > >>>> >>> >> > > > > >> > > > > >> > as
> > >>>> >>> >> > > > > >> > > > > >> > > > half
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > baked
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > feature -
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > it has passed
> our
> > >>>> >>> internal
> > >>>> >>> >> QA
> > >>>> >>> >> > and
> > >>>> >>> >> > > > has
> > >>>> >>> >> > > > > >> very
> > >>>> >>> >> > > > > >> > > > good
> > >>>> >>> >> > > > > >> > > > > >> doc,
> > >>>> >>> >> > > > > >> > > > > >> > > > which
> > >>>> >>> >> > > > > >> > > > > >> > > > > we
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > will
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > provide
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > to Apache
> > shortly.
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > -Vlad
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > On Wed, Sep 7,
> > 2016
> > >>>> at
> > >>>> >>> 9:13
> > >>>> >>> >> AM,
> > >>>> >>> >> > > > > Andrew
> > >>>> >>> >> > > > > >> > > > Purtell <
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > apurtell@apache.org>
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > We shouldn't
> > admit
> > >>>> >>> half
> > >>>> >>> >> baked
> > >>>> >>> >> > > > > changes
> > >>>> >>> >> > > > > >> > that
> > >>>> >>> >> > > > > >> > > > > won't
> > >>>> >>> >> > > > > >> > > > > >> be
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> finished.
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> However
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > in
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > this case the
> > crew
> > >>>> >>> >> working on
> > >>>> >>> >> > > > this
> > >>>> >>> >> > > > > >> > feature
> > >>>> >>> >> > > > > >> > > > are
> > >>>> >>> >> > > > > >> > > > > >> long
> > >>>> >>> >> > > > > >> > > > > >> > > > > timers
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> and
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > less
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > likely
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > than just
> about
> > >>>> >>> anyone to
> > >>>> >>> >> > leave
> > >>>> >>> >> > > > > >> > something
> > >>>> >>> >> > > > > >> > > > in a
> > >>>> >>> >> > > > > >> > > > > >> half
> > >>>> >>> >> > > > > >> > > > > >> > > > baked
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > state. Of
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > course
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > there is no
> > >>>> guarantee
> > >>>> >>> how
> > >>>> >>> >> > > > anything
> > >>>> >>> >> > > > > >> will
> > >>>> >>> >> > > > > >> > > turn
> > >>>> >>> >> > > > > >> > > > > out,
> > >>>> >>> >> > > > > >> > > > > >> > > but I
> > >>>> >>> >> > > > > >> > > > > >> > > > > am
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > willing
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> to
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > take
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > a little on
> > faith
> > >>>> if
> > >>>> >>> they
> > >>>> >>> >> > feel
> > >>>> >>> >> > > > > their
> > >>>> >>> >> > > > > >> > best
> > >>>> >>> >> > > > > >> > > > path
> > >>>> >>> >> > > > > >> > > > > >> > > forward
> > >>>> >>> >> > > > > >> > > > > >> > > > > now
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> is
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > to
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > merge
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > to
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > trunk. I only
> > >>>> wish I
> > >>>> >>> had
> > >>>> >>> >> > > > bandwidth
> > >>>> >>> >> > > > > to
> > >>>> >>> >> > > > > >> > have
> > >>>> >>> >> > > > > >> > > > > done
> > >>>> >>> >> > > > > >> > > > > >> > some
> > >>>> >>> >> > > > > >> > > > > >> > > > real
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > kicking
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> of
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > the
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > tires by now.
> > >>>> Maybe
> > >>>> >>> this
> > >>>> >>> >> > week.
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > (Yes, I'm
> using
> > >>>> some
> > >>>> >>> of
> > >>>> >>> >> that
> > >>>> >>> >> > > time
> > >>>> >>> >> > > > > for
> > >>>> >>> >> > > > > >> > this
> > >>>> >>> >> > > > > >> > > > > email
> > >>>> >>> >> > > > > >> > > > > >> > :-)
> > >>>> >>> >> > > > > >> > > > > >> > > > but
> > >>>> >>> >> > > > > >> > > > > >> > > > > I
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > type
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > fast.)
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > That said, I
> > would
> > >>>> >>> like to
> > >>>> >>> >> > > > agitate
> > >>>> >>> >> > > > > >> for
> > >>>> >>> >> > > > > >> > > > making
> > >>>> >>> >> > > > > >> > > > > 2.0
> > >>>> >>> >> > > > > >> > > > > >> > > more
> > >>>> >>> >> > > > > >> > > > > >> > > > > real
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > and
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> spend
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > some
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > time on it
> now
> > >>>> that
> > >>>> >>> I'm
> > >>>> >>> >> > winding
> > >>>> >>> >> > > > > down
> > >>>> >>> >> > > > > >> > with
> > >>>> >>> >> > > > > >> > > > > 0.98. I
> > >>>> >>> >> > > > > >> > > > > >> > > think
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> that
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > means
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branching for
> > 2.0
> > >>>> real
> > >>>> >>> >> soon
> > >>>> >>> >> > now
> > >>>> >>> >> > > > and
> > >>>> >>> >> > > > > >> even
> > >>>> >>> >> > > > > >> > > > > evicting
> > >>>> >>> >> > > > > >> > > > > >> > > > things
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> from
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > 2.0
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > branch
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > that aren't
> > >>>> finished
> > >>>> >>> or
> > >>>> >>> >> > stable,
> > >>>> >>> >> > > > > >> leaving
> > >>>> >>> >> > > > > >> > > them
> > >>>> >>> >> > > > > >> > > > > only
> > >>>> >>> >> > > > > >> > > > > >> > > once
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> again
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > in
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > the
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > master
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branch. Or,
> > maybe
> > >>>> just
> > >>>> >>> >> > evicting
> > >>>> >>> >> > > > > them.
> > >>>> >>> >> > > > > >> > > Let's
> > >>>> >>> >> > > > > >> > > > > take
> > >>>> >>> >> > > > > >> > > > > >> it
> > >>>> >>> >> > > > > >> > > > > >> > > > case
> > >>>> >>> >> > > > > >> > > > > >> > > > > by
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > case.
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > I think this
> > >>>> feature
> > >>>> >>> can
> > >>>> >>> >> come
> > >>>> >>> >> > > in
> > >>>> >>> >> > > > > >> > > relatively
> > >>>> >>> >> > > > > >> > > > > >> safely.
> > >>>> >>> >> > > > > >> > > > > >> > > As
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> added
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > insurance,
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > let's admit
> the
> > >>>> >>> >> possibility
> > >>>> >>> >> > it
> > >>>> >>> >> > > > > could
> > >>>> >>> >> > > > > >> be
> > >>>> >>> >> > > > > >> > > > > reverted
> > >>>> >>> >> > > > > >> > > > > >> on
> > >>>> >>> >> > > > > >> > > > > >> > > the
> > >>>> >>> >> > > > > >> > > > > >> > > > > 2.0
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > branch
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> if
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > folks
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > working on
> > >>>> >>> stabilizing 2.0
> > >>>> >>> >> > > decide
> > >>>> >>> >> > > > > to
> > >>>> >>> >> > > > > >> > evict
> > >>>> >>> >> > > > > >> > > > it
> > >>>> >>> >> > > > > >> > > > > >> > because
> > >>>> >>> >> > > > > >> > > > > >> > > > it
> > >>>> >>> >> > > > > >> > > > > >> > > > > is
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > unfinished
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > or
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > unstable,
> > because
> > >>>> that
> > >>>> >>> >> > > certainly
> > >>>> >>> >> > > > > can
> > >>>> >>> >> > > > > >> > > > happen. I
> > >>>> >>> >> > > > > >> > > > > >> > would
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> expect if
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > talk
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > like
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > that starts,
> > we'd
> > >>>> get
> > >>>> >>> help
> > >>>> >>> >> > > > > finishing
> > >>>> >>> >> > > > > >> or
> > >>>> >>> >> > > > > >> > > > > >> stabilizing
> > >>>> >>> >> > > > > >> > > > > >> > > > > what's
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > under
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > discussion
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > for revert.
> Or,
> > >>>> we'd
> > >>>> >>> have
> > >>>> >>> >> a
> > >>>> >>> >> > > > revert.
> > >>>> >>> >> > > > > >> > Either
> > >>>> >>> >> > > > > >> > > > way
> > >>>> >>> >> > > > > >> > > > > >> the
> > >>>> >>> >> > > > > >> > > > > >> > > > > outcome
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> is
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > acceptable.
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > On Wed, Sep
> 7,
> > >>>> 2016 at
> > >>>> >>> >> 8:56
> > >>>> >>> >> > AM,
> > >>>> >>> >> > > > > Dima
> > >>>> >>> >> > > > > >> > > Spivak
> > >>>> >>> >> > > > > >> > > > <
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > dimaspivak@apache.org
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > I'm not
> sure
> > >>>> that
> > >>>> >>> >> "There is
> > >>>> >>> >> > > > > already
> > >>>> >>> >> > > > > >> > lots
> > >>>> >>> >> > > > > >> > > > of
> > >>>> >>> >> > > > > >> > > > > >> > > > half-baked
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> code
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > in
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> the
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branch,
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > so what's
> the
> > >>>> harm
> > >>>> >>> in
> > >>>> >>> >> > adding
> > >>>> >>> >> > > > > more?"
> > >>>> >>> >> > > > > >> > is a
> > >>>> >>> >> > > > > >> > > > > good
> > >>>> >>> >> > > > > >> > > > > >> > code
> > >>>> >>> >> > > > > >> > > > > >> > > > > commit
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > philosophy
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > for
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > a
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> > fault-tolerant
> > >>>> >>> >> distributed
> > >>>> >>> >> > > data
> > >>>> >>> >> > > > > >> store.
> > >>>> >>> >> > > > > >> > > ;)
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > More
> > seriously,
> > >>>> a
> > >>>> >>> lack
> > >>>> >>> >> of
> > >>>> >>> >> > > test
> > >>>> >>> >> > > > > >> > coverage
> > >>>> >>> >> > > > > >> > > > for
> > >>>> >>> >> > > > > >> > > > > >> > > existing
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > features
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > shouldn't
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > be
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > used as
> > >>>> >>> justification
> > >>>> >>> >> for
> > >>>> >>> >> > > > > >> introducing
> > >>>> >>> >> > > > > >> > > new
> > >>>> >>> >> > > > > >> > > > > >> > features
> > >>>> >>> >> > > > > >> > > > > >> > > > with
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> the
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > same
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> shortcomings.
> > >>>> >>> >> Ultimately,
> > >>>> >>> >> > > it's
> > >>>> >>> >> > > > > the
> > >>>> >>> >> > > > > >> end
> > >>>> >>> >> > > > > >> > > > user
> > >>>> >>> >> > > > > >> > > > > who
> > >>>> >>> >> > > > > >> > > > > >> > > will
> > >>>> >>> >> > > > > >> > > > > >> > > > > feel
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > the
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> pain,
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > so
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > shouldn't
> we
> > do
> > >>>> >>> >> everything
> > >>>> >>> >> > we
> > >>>> >>> >> > > > can
> > >>>> >>> >> > > > > >> to
> > >>>> >>> >> > > > > >> > > > > mitigate
> > >>>> >>> >> > > > > >> > > > > >> > that?
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > -Dima
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > On Wed, Sep
> > 7,
> > >>>> 2016
> > >>>> >>> at
> > >>>> >>> >> 8:46
> > >>>> >>> >> > > AM,
> > >>>> >>> >> > > > > >> > Vladimir
> > >>>> >>> >> > > > > >> > > > > >> > Rodionov <
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > >>>> >>> vladrodionov@gmail.com>
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > wrote:
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Sean,
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have
> docs
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Agree. We
> > >>>> have a
> > >>>> >>> doc
> > >>>> >>> >> and
> > >>>> >>> >> > > > backup
> > >>>> >>> >> > > > > >> is
> > >>>> >>> >> > > > > >> > the
> > >>>> >>> >> > > > > >> > > > > most
> > >>>> >>> >> > > > > >> > > > > >> > > > > documented
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > feature
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > :),
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > we
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > will
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > release
> it
> > >>>> >>> shortly to
> > >>>> >>> >> > > Apache.
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have
> > >>>> sunny-day
> > >>>> >>> >> > > correctness
> > >>>> >>> >> > > > > >> tests
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Feature
> has
> > >>>> >>> close to
> > >>>> >>> >> 60
> > >>>> >>> >> > > test
> > >>>> >>> >> > > > > >> cases,
> > >>>> >>> >> > > > > >> > > > which
> > >>>> >>> >> > > > > >> > > > > >> run
> > >>>> >>> >> > > > > >> > > > > >> > > for
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> approx
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > 30
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> min.
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > We
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > can
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > add more,
> > if
> > >>>> >>> >> community do
> > >>>> >>> >> > > not
> > >>>> >>> >> > > > > >> mind
> > >>>> >>> >> > > > > >> > :)
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have
> > >>>> >>> >> > > > > >> > correctness-in-face-of-failure
> > >>>> >>> >> > > > > >> > > > > tests
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Any
> > examples
> > >>>> of
> > >>>> >>> these
> > >>>> >>> >> > tests
> > >>>> >>> >> > > > in
> > >>>> >>> >> > > > > >> > > existing
> > >>>> >>> >> > > > > >> > > > > >> > features?
> > >>>> >>> >> > > > > >> > > > > >> > > > In
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > works,
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > we
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > have a
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > clear
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > understanding
> > >>>> of
> > >>>> >>> what
> > >>>> >>> >> > > should
> > >>>> >>> >> > > > be
> > >>>> >>> >> > > > > >> done
> > >>>> >>> >> > > > > >> > > by
> > >>>> >>> >> > > > > >> > > > > the
> > >>>> >>> >> > > > > >> > > > > >> > time
> > >>>> >>> >> > > > > >> > > > > >> > > of
> > >>>> >>> >> > > > > >> > > > > >> > > > > 2.0
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> release.
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > That is
> > very
> > >>>> close
> > >>>> >>> >> goal
> > >>>> >>> >> > for
> > >>>> >>> >> > > > us,
> > >>>> >>> >> > > > > >> to
> > >>>> >>> >> > > > > >> > > > verify
> > >>>> >>> >> > > > > >> > > > > IT
> > >>>> >>> >> > > > > >> > > > > >> > > monkey
> > >>>> >>> >> > > > > >> > > > > >> > > > > for
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> existing
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > code.
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * don't
> > rely
> > >>>> on
> > >>>> >>> things
> > >>>> >>> >> > > > outside
> > >>>> >>> >> > > > > of
> > >>>> >>> >> > > > > >> > > HBase
> > >>>> >>> >> > > > > >> > > > > for
> > >>>> >>> >> > > > > >> > > > > >> > > normal
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > operation
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > (okay
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > for
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > advanced
> > >>>> >>> operation)
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > We do
> not.
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Enormous
> > time
> > >>>> has
> > >>>> >>> been
> > >>>> >>> >> > > spent
> > >>>> >>> >> > > > > >> already
> > >>>> >>> >> > > > > >> > > on
> > >>>> >>> >> > > > > >> > > > > the
> > >>>> >>> >> > > > > >> > > > > >> > > > > development
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > and
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > testing
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > the
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > feature,
> it
> > >>>> has
> > >>>> >>> passed
> > >>>> >>> >> > our
> > >>>> >>> >> > > > > >> internal
> > >>>> >>> >> > > > > >> > > > tests
> > >>>> >>> >> > > > > >> > > > > and
> > >>>> >>> >> > > > > >> > > > > >> > > many
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> rounds
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > of
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> code
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > reviews
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > by HBase
> > >>>> >>> committers.
> > >>>> >>> >> We
> > >>>> >>> >> > do
> > >>>> >>> >> > > > not
> > >>>> >>> >> > > > > >> mind
> > >>>> >>> >> > > > > >> > if
> > >>>> >>> >> > > > > >> > > > > >> someone
> > >>>> >>> >> > > > > >> > > > > >> > > from
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> HBase
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > community
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > (outside
> of
> > >>>> HW)
> > >>>> >>> will
> > >>>> >>> >> > review
> > >>>> >>> >> > > > the
> > >>>> >>> >> > > > > >> > code,
> > >>>> >>> >> > > > > >> > > > but
> > >>>> >>> >> > > > > >> > > > > it
> > >>>> >>> >> > > > > >> > > > > >> > will
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> probably
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> takes
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > forever
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > to
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > wait for
> > >>>> >>> volunteer?,
> > >>>> >>> >> the
> > >>>> >>> >> > > > > feature
> > >>>> >>> >> > > > > >> is
> > >>>> >>> >> > > > > >> > > > quite
> > >>>> >>> >> > > > > >> > > > > >> large
> > >>>> >>> >> > > > > >> > > > > >> > > > (1MB+
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> cumulative
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > patch)
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > 2.0
> branch
> > is
> > >>>> >>> full of
> > >>>> >>> >> > half
> > >>>> >>> >> > > > > baked
> > >>>> >>> >> > > > > >> > > > features,
> > >>>> >>> >> > > > > >> > > > > >> most
> > >>>> >>> >> > > > > >> > > > > >> > > of
> > >>>> >>> >> > > > > >> > > > > >> > > > > them
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > are
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > in
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > active
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > development,
> > >>>> >>> >> therefore I
> > >>>> >>> >> > am
> > >>>> >>> >> > > > not
> > >>>> >>> >> > > > > >> > > > following
> > >>>> >>> >> > > > > >> > > > > you
> > >>>> >>> >> > > > > >> > > > > >> > > here,
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> Sean?
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > Why
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > HBASE-7912
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > is
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > not good
> > >>>> enough
> > >>>> >>> yet
> > >>>> >>> >> to be
> > >>>> >>> >> > > > > >> integrated
> > >>>> >>> >> > > > > >> > > > into
> > >>>> >>> >> > > > > >> > > > > 2.0
> > >>>> >>> >> > > > > >> > > > > >> > > > branch?
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > -Vlad
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > On Wed,
> > Sep 7,
> > >>>> >>> 2016 at
> > >>>> >>> >> > 8:23
> > >>>> >>> >> > > > AM,
> > >>>> >>> >> > > > > >> Sean
> > >>>> >>> >> > > > > >> > > > > Busbey <
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > busbey@apache.org
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > On Tue,
> > Sep
> > >>>> 6,
> > >>>> >>> 2016
> > >>>> >>> >> at
> > >>>> >>> >> > > > 10:36
> > >>>> >>> >> > > > > >> PM,
> > >>>> >>> >> > > > > >> > > Josh
> > >>>> >>> >> > > > > >> > > > > >> Elser <
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> > josh.elser@gmail.com>
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > wrote:
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > So,
> the
> > >>>> >>> answer to
> > >>>> >>> >> > > Sean's
> > >>>> >>> >> > > > > >> > original
> > >>>> >>> >> > > > > >> > > > > >> question
> > >>>> >>> >> > > > > >> > > > > >> > is
> > >>>> >>> >> > > > > >> > > > > >> > > > "as
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > robust as
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > snapshots
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> > presently
> > >>>> >>> are"?
> > >>>> >>> >> > > > > >> (independence of
> > >>>> >>> >> > > > > >> > > > > >> > > backup/restore
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > failure
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > tolerance
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > from
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> > snapshot
> > >>>> >>> failure
> > >>>> >>> >> > > > tolerance)
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > Is
> this
> > >>>> just a
> > >>>> >>> >> > question
> > >>>> >>> >> > > > WRT
> > >>>> >>> >> > > > > >> > > context
> > >>>> >>> >> > > > > >> > > > of
> > >>>> >>> >> > > > > >> > > > > >> the
> > >>>> >>> >> > > > > >> > > > > >> > > > > change,
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> or
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > is it
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > means
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > for a
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > veto
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > from
> > you,
> > >>>> >>> Sean?
> > >>>> >>> >> Just
> > >>>> >>> >> > > > trying
> > >>>> >>> >> > > > > >> to
> > >>>> >>> >> > > > > >> > > make
> > >>>> >>> >> > > > > >> > > > > sure
> > >>>> >>> >> > > > > >> > > > > >> > I'm
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> following
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> along
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> adequately.
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > I'd say
> > ATM
> > >>>> I'm
> > >>>> >>> -0,
> > >>>> >>> >> > > > bordering
> > >>>> >>> >> > > > > >> on
> > >>>> >>> >> > > > > >> > -1
> > >>>> >>> >> > > > > >> > > > but
> > >>>> >>> >> > > > > >> > > > > not
> > >>>> >>> >> > > > > >> > > > > >> > for
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> reasons
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > I
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > can
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > articulate
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > well.
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Here's
> an
> > >>>> >>> attempt.
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > We've
> > been
> > >>>> >>> trying to
> > >>>> >>> >> > > move,
> > >>>> >>> >> > > > > as a
> > >>>> >>> >> > > > > >> > > > > community,
> > >>>> >>> >> > > > > >> > > > > >> > > > towards
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > minimizing
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > risk
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > to
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > downstream
> > >>>> >>> folks by
> > >>>> >>> >> > > getting
> > >>>> >>> >> > > > > >> > > "complete
> > >>>> >>> >> > > > > >> > > > > >> enough
> > >>>> >>> >> > > > > >> > > > > >> > > for
> > >>>> >>> >> > > > > >> > > > > >> > > > > use"
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > gates
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> in
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > place
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > before
> we
> > >>>> >>> introduce
> > >>>> >>> >> new
> > >>>> >>> >> > > > > >> features.
> > >>>> >>> >> > > > > >> > > This
> > >>>> >>> >> > > > > >> > > > > was
> > >>>> >>> >> > > > > >> > > > > >> > > > spurred
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> by a
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > some
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > features
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > getting
> > in
> > >>>> >>> >> half-baked
> > >>>> >>> >> > and
> > >>>> >>> >> > > > > never
> > >>>> >>> >> > > > > >> > > making
> > >>>> >>> >> > > > > >> > > > > it
> > >>>> >>> >> > > > > >> > > > > >> to
> > >>>> >>> >> > > > > >> > > > > >> > > "can
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> really
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > use"
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > status
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > (I'm
> > >>>> thinking of
> > >>>> >>> >> > > > distributed
> > >>>> >>> >> > > > > >> log
> > >>>> >>> >> > > > > >> > > > replay
> > >>>> >>> >> > > > > >> > > > > and
> > >>>> >>> >> > > > > >> > > > > >> > the
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> zk-less
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > assignment
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > stuff,
> I
> > >>>> don't
> > >>>> >>> >> recall
> > >>>> >>> >> > if
> > >>>> >>> >> > > > > there
> > >>>> >>> >> > > > > >> was
> > >>>> >>> >> > > > > >> > > > > more).
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The
> > gates,
> > >>>> >>> >> generally,
> > >>>> >>> >> > > > > included
> > >>>> >>> >> > > > > >> > > things
> > >>>> >>> >> > > > > >> > > > > like:
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have
> > docs
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have
> > >>>> sunny-day
> > >>>> >>> >> > > > correctness
> > >>>> >>> >> > > > > >> tests
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have
> > >>>> >>> >> > > > > >> > > correctness-in-face-of-failure
> > >>>> >>> >> > > > > >> > > > > tests
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't
> > >>>> rely on
> > >>>> >>> >> things
> > >>>> >>> >> > > > > outside
> > >>>> >>> >> > > > > >> of
> > >>>> >>> >> > > > > >> > > > HBase
> > >>>> >>> >> > > > > >> > > > > for
> > >>>> >>> >> > > > > >> > > > > >> > > > normal
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > operation
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > (okay
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > for
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> advanced
> > >>>> >>> operation)
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > As an
> > >>>> example,
> > >>>> >>> we
> > >>>> >>> >> kept
> > >>>> >>> >> > > the
> > >>>> >>> >> > > > > MOB
> > >>>> >>> >> > > > > >> > work
> > >>>> >>> >> > > > > >> > > > off
> > >>>> >>> >> > > > > >> > > > > in
> > >>>> >>> >> > > > > >> > > > > >> a
> > >>>> >>> >> > > > > >> > > > > >> > > > branch
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> and
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > out
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> of
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > master
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > until
> it
> > >>>> could
> > >>>> >>> pass
> > >>>> >>> >> > these
> > >>>> >>> >> > > > > >> > criteria.
> > >>>> >>> >> > > > > >> > > > The
> > >>>> >>> >> > > > > >> > > > > big
> > >>>> >>> >> > > > > >> > > > > >> > > > > exemption
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > we've
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> had
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > to
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > this
> was
> > the
> > >>>> >>> >> > hbase-spark
> > >>>> >>> >> > > > > >> > > integration,
> > >>>> >>> >> > > > > >> > > > > where
> > >>>> >>> >> > > > > >> > > > > >> > we
> > >>>> >>> >> > > > > >> > > > > >> > > > all
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > agreed
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > it
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > could
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > land in
> > >>>> master
> > >>>> >>> >> because
> > >>>> >>> >> > it
> > >>>> >>> >> > > > was
> > >>>> >>> >> > > > > >> very
> > >>>> >>> >> > > > > >> > > > well
> > >>>> >>> >> > > > > >> > > > > >> > > isolated
> > >>>> >>> >> > > > > >> > > > > >> > > > > (the
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > slide
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > away
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > from
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> including
> > >>>> docs
> > >>>> >>> as a
> > >>>> >>> >> > > > > first-class
> > >>>> >>> >> > > > > >> > part
> > >>>> >>> >> > > > > >> > > > of
> > >>>> >>> >> > > > > >> > > > > >> > > building
> > >>>> >>> >> > > > > >> > > > > >> > > > up
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> that
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > integration
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > has led
> > me
> > >>>> to
> > >>>> >>> doubt
> > >>>> >>> >> the
> > >>>> >>> >> > > > > wisdom
> > >>>> >>> >> > > > > >> of
> > >>>> >>> >> > > > > >> > > this
> > >>>> >>> >> > > > > >> > > > > >> > > decision).
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > We've
> > also
> > >>>> been
> > >>>> >>> >> > treating
> > >>>> >>> >> > > > > >> inclusion
> > >>>> >>> >> > > > > >> > > in
> > >>>> >>> >> > > > > >> > > > a
> > >>>> >>> >> > > > > >> > > > > >> > > "probably
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> will
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > be
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > released
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > to
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > downstream"
> > >>>> >>> branches
> > >>>> >>> >> > as a
> > >>>> >>> >> > > > > >> higher
> > >>>> >>> >> > > > > >> > > bar,
> > >>>> >>> >> > > > > >> > > > > >> > requiring
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't
> > >>>> >>> moderately
> > >>>> >>> >> > impact
> > >>>> >>> >> > > > > >> > > performance
> > >>>> >>> >> > > > > >> > > > > when
> > >>>> >>> >> > > > > >> > > > > >> > the
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> feature
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > isn't
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> in
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > use
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't
> > >>>> severely
> > >>>> >>> >> impact
> > >>>> >>> >> > > > > >> > performance
> > >>>> >>> >> > > > > >> > > > when
> > >>>> >>> >> > > > > >> > > > > >> the
> > >>>> >>> >> > > > > >> > > > > >> > > > > feature
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> is
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > in
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> use
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > *
> either
> > >>>> >>> >> default-to-on
> > >>>> >>> >> > or
> > >>>> >>> >> > > > > show
> > >>>> >>> >> > > > > >> > > enough
> > >>>> >>> >> > > > > >> > > > > >> demand
> > >>>> >>> >> > > > > >> > > > > >> > to
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> believe
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > a
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > non-trivial
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > number
> of
> > >>>> folks
> > >>>> >>> will
> > >>>> >>> >> > turn
> > >>>> >>> >> > > > the
> > >>>> >>> >> > > > > >> > > feature
> > >>>> >>> >> > > > > >> > > > on
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The
> above
> > >>>> has
> > >>>> >>> kept
> > >>>> >>> >> MOB
> > >>>> >>> >> > > and
> > >>>> >>> >> > > > > >> > > hbase-spark
> > >>>> >>> >> > > > > >> > > > > >> > > > integration
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> out
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > of
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > branch-1,
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > presumably
> > >>>> while
> > >>>> >>> >> > they've
> > >>>> >>> >> > > > > >> "gotten
> > >>>> >>> >> > > > > >> > > more
> > >>>> >>> >> > > > > >> > > > > >> stable"
> > >>>> >>> >> > > > > >> > > > > >> > > in
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> master
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > from
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > the
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > odd
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > vendor
> > >>>> >>> inclusion.
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Are we
> > >>>> going to
> > >>>> >>> >> have a
> > >>>> >>> >> > > 2.0
> > >>>> >>> >> > > > > >> release
> > >>>> >>> >> > > > > >> > > > > before
> > >>>> >>> >> > > > > >> > > > > >> the
> > >>>> >>> >> > > > > >> > > > > >> > > end
> > >>>> >>> >> > > > > >> > > > > >> > > > > of
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> the
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> year?
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > We're
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > coming
> up
> > >>>> on 1.5
> > >>>> >>> >> years
> > >>>> >>> >> > > > since
> > >>>> >>> >> > > > > >> the
> > >>>> >>> >> > > > > >> > > > > release of
> > >>>> >>> >> > > > > >> > > > > >> > > > version
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> 1.0;
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> seems
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > like
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > it's
> > about
> > >>>> time,
> > >>>> >>> >> > though I
> > >>>> >>> >> > > > > >> haven't
> > >>>> >>> >> > > > > >> > > seen
> > >>>> >>> >> > > > > >> > > > > any
> > >>>> >>> >> > > > > >> > > > > >> > > > concrete
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > plans
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> this
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > year.
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> Presuming
> > >>>> we are
> > >>>> >>> >> going
> > >>>> >>> >> > to
> > >>>> >>> >> > > > > have
> > >>>> >>> >> > > > > >> one
> > >>>> >>> >> > > > > >> > > by
> > >>>> >>> >> > > > > >> > > > > the
> > >>>> >>> >> > > > > >> > > > > >> end
> > >>>> >>> >> > > > > >> > > > > >> > > of
> > >>>> >>> >> > > > > >> > > > > >> > > > > the
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > year, it
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > seems a
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > bit
> > close to
> > >>>> >>> still
> > >>>> >>> >> be
> > >>>> >>> >> > > > adding
> > >>>> >>> >> > > > > in
> > >>>> >>> >> > > > > >> > > > > "features
> > >>>> >>> >> > > > > >> > > > > >> > that
> > >>>> >>> >> > > > > >> > > > > >> > > > need
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > maturing"
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > on
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > the
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > branch.
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The
> lack
> > of
> > >>>> a
> > >>>> >>> >> concrete
> > >>>> >>> >> > > plan
> > >>>> >>> >> > > > > for
> > >>>> >>> >> > > > > >> > 2.0
> > >>>> >>> >> > > > > >> > > > > keeps
> > >>>> >>> >> > > > > >> > > > > >> me
> > >>>> >>> >> > > > > >> > > > > >> > > from
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > considering
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > these
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > things
> > >>>> blocker
> > >>>> >>> at
> > >>>> >>> >> the
> > >>>> >>> >> > > > moment.
> > >>>> >>> >> > > > > >> But
> > >>>> >>> >> > > > > >> > I
> > >>>> >>> >> > > > > >> > > > know
> > >>>> >>> >> > > > > >> > > > > >> > first
> > >>>> >>> >> > > > > >> > > > > >> > > > hand
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> how
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > much
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > trouble
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > folks
> > have
> > >>>> had
> > >>>> >>> with
> > >>>> >>> >> > other
> > >>>> >>> >> > > > > >> features
> > >>>> >>> >> > > > > >> > > > that
> > >>>> >>> >> > > > > >> > > > > >> have
> > >>>> >>> >> > > > > >> > > > > >> > > gone
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> into
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > downstream
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > facing
> > >>>> releases
> > >>>> >>> >> without
> > >>>> >>> >> > > > > >> robustness
> > >>>> >>> >> > > > > >> > > > > checks
> > >>>> >>> >> > > > > >> > > > > >> > (i.e.
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > replication),
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > and
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > I'm
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> concerned
> > >>>> about
> > >>>> >>> what
> > >>>> >>> >> > > we're
> > >>>> >>> >> > > > > >> setting
> > >>>> >>> >> > > > > >> > > up
> > >>>> >>> >> > > > > >> > > > if
> > >>>> >>> >> > > > > >> > > > > >> 2.0
> > >>>> >>> >> > > > > >> > > > > >> > > goes
> > >>>> >>> >> > > > > >> > > > > >> > > > > out
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > with
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> this
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > feature
> > in
> > >>>> its
> > >>>> >>> >> current
> > >>>> >>> >> > > > state.
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > --
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > Best regards,
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >    - Andy
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > Problems
> > worthy of
> > >>>> >>> attack
> > >>>> >>> >> > prove
> > >>>> >>> >> > > > > their
> > >>>> >>> >> > > > > >> > > worth
> > >>>> >>> >> > > > > >> > > > by
> > >>>> >>> >> > > > > >> > > > > >> > > hitting
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> back. -
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > Piet
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > Hein
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > (via Tom
> White)
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > --
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > Best regards,
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >    - Andy
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > Problems worthy
> of
> > >>>> attack
> > >>>> >>> >> prove
> > >>>> >>> >> > > their
> > >>>> >>> >> > > > > >> worth
> > >>>> >>> >> > > > > >> > by
> > >>>> >>> >> > > > > >> > > > > >> hitting
> > >>>> >>> >> > > > > >> > > > > >> > > > back.
> > >>>> >>> >> > > > > >> > > > > >> > > > > -
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > Piet
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> Hein
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > (via Tom White)
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >>
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >>
> > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > > >
> > >>>> >>> >> > > > > >> > > > > >> > > >
> > >>>> >>> >> > > > > >> > > > > >> > >
> > >>>> >>> >> > > > > >> > > > > >> >
> > >>>> >>> >> > > > > >> > > > > >>
> > >>>> >>> >> > > > > >> > > > >
> > >>>> >>> >> > > > > >> > > > >
> > >>>> >>> >> > > > > >> > > > >
> > >>>> >>> >> > > > > >> > > > > --
> > >>>> >>> >> > > > > >> > > > > busbey
> > >>>> >>> >> > > > > >> > > > >
> > >>>> >>> >> > > > > >> > > >
> > >>>> >>> >> > > > > >> > >
> > >>>> >>> >> > > > > >> >
> > >>>> >>> >> > > > > >>
> > >>>> >>> >> > > > > >
> > >>>> >>> >> > > > > >
> > >>>> >>> >> > > > >
> > >>>> >>> >> > > >
> > >>>> >>> >> > >
> > >>>> >>> >> >
> > >>>> >>> >>
> > >>>> >>> >>
> > >>>> >>> >>
> > >>>> >>> >> --
> > >>>> >>> >>
> > >>>> >>> >> -- Appy
> > >>>> >>> >>
> > >>>> >>> >
> > >>>> >>> >
> > >>>> >>>
> > >>>> >>
> > >>>> >>
> > >>>> >
> > >>>>
> > >>>
> > >>>
> > >>
> >
> >
> >
> > --
> > busbey
> >
> >
>



-- 

-- Appy

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Ted Yu <yu...@gmail.com>]

Once Stack's comments are addressed, we can change the master build to
using Maven 3.3.3

I will log JIRA for modifying refguide on the upgrade.

Cheers

On Thu, Oct 13, 2016 at 6:42 AM, Sean Busbey <bu...@cloudera.com> wrote:

> Are we requiring maven 3.3.z now? historically our required maven
> version has been 3.0.5. I don't have an objection to changing it, per
> se, but I'd like to make sure our docs and CI builds get updated
> properly.
>
> On Wed, Oct 12, 2016 at 4:03 PM, Stack <st...@duboce.net> wrote:
> > On Wed, Oct 12, 2016 at 1:58 PM, Stack <st...@duboce.net> wrote:
> >
> >> Fails again when I do this:
> >>
> >> $ mvn clean install -DskipTests assembly:single
> >> ....
> >>
> >>
> > Ok. Works if I use mvn 3.3.x.
> > St.Ack
> >
> >
> >
> >
> >
> >>
> >> [INFO] ------------------------------------------------------------
> >> ------------
> >> [INFO] Building Apache HBase - Server 2.0.0-SNAPSHOT
> >> [INFO] ------------------------------------------------------------
> >> ------------
> >> Downloading: file:/home/stack/hbase.git/hbase-server/src/main/site/
> >> resources/repo/org/apache/hadoop/hadoop-distcp/2.7.1/
> >> hadoop-distcp-2.7.1.pom
> >> Exception in thread "pool-1-thread-1" ------------------------------
> >> ---------------------
> >> constituent[0]: file:/usr/share/maven/lib/aether-spi.jar
> >> constituent[1]: file:/usr/share/maven/lib/
> maven-settings-builder-3.x.jar
> >> constituent[2]: file:/usr/share/maven/lib/maven-artifact-3.x.jar
> >> constituent[3]: file:/usr/share/maven/lib/commons-httpclient.jar
> >> constituent[4]: file:/usr/share/maven/lib/wagon-provider-api.jar
> >> constituent[5]: file:/usr/share/maven/lib/maven-core-3.x.jar
> >> constituent[6]: file:/usr/share/maven/lib/aether-impl.jar
> >> constituent[7]: file:/usr/share/maven/lib/sisu-inject-bean.jar
> >> constituent[8]: file:/usr/share/maven/lib/sisu-inject-plexus.jar
> >> constituent[9]: file:/usr/share/maven/lib/wagon-file.jar
> >> constituent[10]: file:/usr/share/maven/lib/guava.jar
> >> constituent[11]: file:/usr/share/maven/lib/maven-compat-3.x.jar
> >> constituent[12]: file:/usr/share/maven/lib/maven-model-3.x.jar
> >> constituent[13]: file:/usr/share/maven/lib/plexus-interpolation.jar
> >> constituent[14]: file:/usr/share/maven/lib/commons-codec.jar
> >> constituent[15]: file:/usr/share/maven/lib/aether-util.jar
> >> constituent[16]: file:/usr/share/maven/lib/aether-api.jar
> >> constituent[17]: file:/usr/share/maven/lib/maven-model-builder-3.x.jar
> >> constituent[18]: file:/usr/share/maven/lib/
> maven-repository-metadata-3.x.
> >> jar
> >> constituent[19]: file:/usr/share/maven/lib/maven-settings-3.x.jar
> >> constituent[20]: file:/usr/share/maven/lib/
> plexus-component-annotations.
> >> jar
> >> constituent[21]: file:/usr/share/maven/lib/commons-cli.jar
> >> constituent[22]: file:/usr/share/maven/lib/maven-embedder-3.x.jar
> >> constituent[23]: file:/usr/share/maven/lib/maven-plugin-api-3.x.jar
> >> constituent[24]: file:/usr/share/maven/lib/wagon-http-shaded.jar
> >> constituent[25]: file:/usr/share/maven/lib/commons-logging.jar
> >> constituent[26]: file:/usr/share/maven/lib/plexus-sec-dispatcher.jar
> >> constituent[27]: file:/usr/share/maven/lib/aether-connector-wagon.jar
> >> constituent[28]: file:/usr/share/maven/lib/plexus-utils.jar
> >> constituent[29]: file:/usr/share/maven/lib/sisu-guice.jar
> >> constituent[30]: file:/usr/share/maven/lib/plexus-cipher.jar
> >> constituent[31]: file:/usr/share/maven/lib/
> maven-aether-provider-3.x.jar
> >> ---------------------------------------------------
> >> java.lang.NoClassDefFoundError: org/apache/commons/lang/StringUtils
> >> at org.apache.maven.wagon.providers.file.FileWagon.
> resolveDestinationPath(
> >> FileWagon.java:206)
> >> at org.apache.maven.wagon.providers.file.FileWagon.
> >> resourceExists(FileWagon.java:265)
> >> at org.sonatype.aether.connector.wagon.WagonRepositoryConnector$
> >> GetTask.run(WagonRepositoryConnector.java:577)
> >> at org.sonatype.aether.util.concurrency.RunnableErrorForwarder$1.run(
> >> RunnableErrorForwarder.java:60)
> >> at java.util.concurrent.ThreadPoolExecutor.runWorker(
> >> ThreadPoolExecutor.java:1142)
> >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> >> ThreadPoolExecutor.java:617)
> >> at java.lang.Thread.run(Thread.java:745)
> >> Caused by: java.lang.ClassNotFoundException: org.apache.commons.lang.
> >> StringUtils
> >> at org.codehaus.plexus.classworlds.strategy.
> SelfFirstStrategy.loadClass(
> >> SelfFirstStrategy.java:50)
> >> at org.codehaus.plexus.classworlds.realm.ClassRealm.
> >> unsynchronizedLoadClass(ClassRealm.java:259)
> >> at org.codehaus.plexus.classworlds.realm.ClassRealm.
> >> loadClass(ClassRealm.java:235)
> >> at org.codehaus.plexus.classworlds.realm.ClassRealm.
> >> loadClass(ClassRealm.java:227)
> >> ... 7 more
> >> Exception in thread "main" java.lang.NoClassDefFoundError:
> >> org/apache/commons/lang/StringUtils
> >> at org.apache.maven.wagon.providers.file.FileWagon.
> resolveDestinationPath(
> >> FileWagon.java:206)
> >> at org.apache.maven.wagon.providers.file.FileWagon.
> >> resourceExists(FileWagon.java:265)
> >> at org.sonatype.aether.connector.wagon.WagonRepositoryConnector$
> >> GetTask.run(WagonRepositoryConnector.java:577)
> >> at org.sonatype.aether.util.concurrency.RunnableErrorForwarder$1.run(
> >> RunnableErrorForwarder.java:60)
> >> at java.util.concurrent.ThreadPoolExecutor.runWorker(
> >> ThreadPoolExecutor.java:1142)
> >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> >> ThreadPoolExecutor.java:617)
> >> at java.lang.Thread.run(Thread.java:745)
> >> Caused by: java.lang.ClassNotFoundException: org.apache.commons.lang.
> >> StringUtils
> >> at org.codehaus.plexus.classworlds.strategy.
> SelfFirstStrategy.loadClass(
> >> SelfFirstStrategy.java:50)
> >> at org.codehaus.plexus.classworlds.realm.ClassRealm.
> >> unsynchronizedLoadClass(ClassRealm.java:259)
> >> at org.codehaus.plexus.classworlds.realm.ClassRealm.
> >> loadClass(ClassRealm.java:235)
> >> at org.codehaus.plexus.classworlds.realm.ClassRealm.
> >> loadClass(ClassRealm.java:227)
> >>
> >>
> >>
> >> Does it work for you lot? This is branch checkout.
> >>
> >> I want assembly to work so I can distribute the build around a cluster.
> >>
> >> Thanks,
> >> St.Ack.
> >>
> >>
> >>
> >> On Wed, Oct 12, 2016 at 1:52 PM, Stack <st...@duboce.net> wrote:
> >>
> >>> I did a super-clean and recheck out. Now it works. Sorry for the noise.
> >>> St.Ack
> >>>
> >>> On Wed, Oct 12, 2016 at 1:10 PM, Vladimir Rodionov <
> >>> vladrodionov@gmail.com> wrote:
> >>>
> >>>> checked out HBASE-7912
> >>>>
> >>>> ran:
> >>>>
> >>>> mvn clean install -DskipTests
> >>>>
> >>>> successfully.
> >>>>
> >>>> -Vlad
> >>>>
> >>>> On Wed, Oct 12, 2016 at 1:01 PM, Vladimir Rodionov <
> >>>> vladrodionov@gmail.com>
> >>>> wrote:
> >>>>
> >>>> > I usually use:
> >>>> >
> >>>> > mvn clean install -DskipTests
> >>>> >
> >>>> > and it works.
> >>>> >
> >>>> > On Wed, Oct 12, 2016 at 1:01 PM, Vladimir Rodionov <
> >>>> vladrodionov@gmail.com
> >>>> > > wrote:
> >>>> >
> >>>> >> Michael,
> >>>> >>
> >>>> >> you can try master + latest patch on HBASE-14123 (v29). No need to
> use
> >>>> >> HBASE-7912 branch. I will double check HBASE-7912.
> >>>> >>
> >>>> >> -Vlad
> >>>> >>
> >>>> >> On Wed, Oct 12, 2016 at 12:31 PM, Stack <st...@duboce.net> wrote:
> >>>> >>
> >>>> >>> More info:
> >>>> >>>
> >>>> >>> stack@ve0524:~/hbase.git$ git checkout origin/HBASE-7912 -b
> 7912v2
> >>>> >>> Branch 7912v2 set up to track remote branch HBASE-7912 from
> origin.
> >>>> >>> Switched to a new branch '7912v2'
> >>>> >>> stack@ve0524:~/hbase.git$ java -version
> >>>> >>> java version "1.8.0_101"
> >>>> >>> Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
> >>>> >>> Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode)
> >>>> >>> stack@ve0524:~/hbase.git$ mvn clean install -DskipTests &>
> >>>> /tmp/out.txt
> >>>> >>>
> >>>> >>> ...
> >>>> >>>
> >>>> >>> St.Ack
> >>>> >>>
> >>>> >>>
> >>>> >>> On Wed, Oct 12, 2016 at 12:29 PM, Stack <st...@duboce.net> wrote:
> >>>> >>>
> >>>> >>> > Interesting. When I try it fails w/ below:
> >>>> >>> >
> >>>> >>> > [INFO] 26 warnings
> >>>> >>> > 322 [INFO] ------------------------------
> >>>> >>> -------------------------------
> >>>> >>> > 323 [INFO] ------------------------------
> >>>> >>> -------------------------------
> >>>> >>> > 324 [ERROR] COMPILATION ERROR :
> >>>> >>> > 325 [INFO] ------------------------------
> >>>> >>> -------------------------------
> >>>> >>> > 326 [ERROR] /home/stack/hbase.git/hbase-
> common/src/main/java/org/
> >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.j
> >>>> ava:[48,8]
> >>>> >>> > org.apache.hadoop.hbase.io.encoding.rowindexV2.RowIndexCodecV2
> is
> >>>> not
> >>>> >>> > abstract and does not override abstract method
> createSeeker(org.ap
> >>>> >>> >  ache.hadoop.hbase.CellComparator,org.apache.hadoop.hbase.io.
> >>>> >>> encoding.HFileBlockDecodingContext)
> >>>> >>> > in org.apache.hadoop.hbase.io.encoding.DataBlockEncoder
> >>>> >>> > 327 [ERROR] /home/stack/hbase.git/hbase-
> common/src/main/java/org/
> >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.j
> >>>> >>> ava:[143,3]
> >>>> >>> > method does not override or implement a method from a supertype
> >>>> >>> > 328 [ERROR] /home/stack/hbase.git/hbase-
> common/src/main/java/org/
> >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.j
> >>>> >>> ava:[147,29]
> >>>> >>> > incompatible types: java.nio.ByteBuffer cannot be converted to
> >>>> >>> > org.apache.hadoop.hbase.nio.ByteBuff
> >>>> >>> > 329 [ERROR] /home/stack/hbase.git/hbase-
> common/src/main/java/org/
> >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.j
> >>>> >>> ava:[148,33]
> >>>> >>> > cannot find symbol
> >>>> >>> > 330   symbol:   method getKeyDeepCopy()
> >>>> >>> > 331   location: variable seeker of type
> org.apache.hadoop.hbase.io
> >>>> .
> >>>> >>> > encoding.DataBlockEncoder.EncodedSeeker
> >>>> >>> > 332 [ERROR] /home/stack/hbase.git/hbase-
> common/src/main/java/org/
> >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.j
> >>>> >>> ava:[153,3]
> >>>> >>> > method does not override or implement a method from a supertype
> >>>> >>> > 333 [ERROR] /home/stack/hbase.git/hbase-
> common/src/main/java/org/
> >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV1/RowIndexCodecV1.j
> >>>> ava:[45,8]
> >>>> >>> > org.apache.hadoop.hbase.io.encoding.rowindexV1.RowIndexCodecV1
> is
> >>>> not
> >>>> >>> > abstract and does not override abstract method
> createSeeker(org.ap
> >>>> >>> >  ache.hadoop.hbase.CellComparator,org.apache.hadoop.hbase.io.
> >>>> >>> encoding.HFileBlockDecodingContext)
> >>>> >>> > in org.apache.hadoop.hbase.io.encoding.DataBlockEncoder
> >>>> >>> > 334 [ERROR] /home/stack/hbase.git/hbase-
> common/src/main/java/org/
> >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV1/RowIndexCodecV1.j
> >>>> >>> ava:[145,3]
> >>>> >>> > method does not override or implement a method from a supertype
> >>>> >>> > 335 [ERROR] /home/stack/hbase.git/hbase-
> common/src/main/java/org/
> >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV1/RowIndexCodecV1.j
> >>>> >>> ava:[158,3]
> >>>> >>> > method does not override or implement a method from a supertype
> >>>> >>> > 336 [ERROR] /home/stack/hbase.git/hbase-
> common/src/main/java/org/
> >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
> >>>> >>> java:[46,8]
> >>>> >>> > org.apache.hadoop.hbase.io.encoding.rowindexV2.RowIndexSeekerV2
> >>>> is not
> >>>> >>> > abstract and does not override abstract method compareKey(org.ap
> >>>> >>> >  ache.hadoop.hbase.CellComparator,org.apache.hadoop.hbase.Cell)
> in
> >>>> >>> > org.apache.hadoop.hbase.io.encoding.DataBlockEncoder.
> EncodedSeeker
> >>>> >>> > 337 [ERROR] /home/stack/hbase.git/hbase-
> common/src/main/java/org/
> >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
> >>>> >>> java:[79,3]
> >>>> >>> > method does not override or implement a method from a supertype
> >>>> >>> > 338 [ERROR] /home/stack/hbase.git/hbase-
> common/src/main/java/org/
> >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
> >>>> >>> java:[117,3]
> >>>> >>> > method does not override or implement a method from a supertype
> >>>> >>> > 339 [ERROR] /home/stack/hbase.git/hbase-
> common/src/main/java/org/
> >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
> >>>> >>> java:[190,3]
> >>>> >>> > method does not override or implement a method from a supertype
> >>>> >>> > 340 [ERROR] /home/stack/hbase.git/hbase-
> common/src/main/java/org/
> >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
> >>>> >>> java:[214,3]
> >>>> >>> > method does not override or implement a method from a supertype
> >>>> >>> > 341 [ERROR] /home/stack/hbase.git/hbase-
> common/src/main/java/org/
> >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
> >>>> >>> java:[349,3]
> >>>> >>> > method does not override or implement a method from a supertype
> >>>> >>> > 342 [ERROR] /home/stack/hbase.git/hbase-
> common/src/main/java/org/
> >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
> >>>> >>> java:[355,3]
> >>>> >>> > method does not override or implement a method from a supertype
> >>>> >>> > 343 [ERROR] /home/stack/hbase.git/hbase-
> common/src/main/java/org/
> >>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
> >>>> >>> java:[421,36]
> >>>> >>> > no suitable method found for uncompressTags(java.nio.
> >>>> >>> > ByteBuffer,byte[],int,int)
> >>>> >>> > 344     method org.apache.hadoop.hbase.io.
> TagCompressionContext.
> >>>> >>> > uncompressTags(java.io.InputStream,byte[],int,int) is not
> >>>> applicable
> >>>> >>> > 345       (argument mismatch; java.nio.ByteBuffer cannot be
> >>>> converted
> >>>> >>> to
> >>>> >>> > java.io.InputStream)
> >>>> >>> > 346     method org.apache.hadoop.hbase.io.
> TagCompressionContext.
> >>>> >>> > uncompressTags(org.apache.hadoop.hbase.nio.ByteBuff,
> byte[],int,int)
> >>>> is
> >>>> >>> > not applicable
> >>>> >>> > 347       (argument mismatch; java.nio.ByteBuffer cannot be
> >>>> converted
> >>>> >>> to
> >>>> >>> > org.apache.hadoop.hbase.nio.ByteBuff)
> >>>> >>> >
> >>>> >>> > ....
> >>>> >>> >
> >>>> >>> > St.Ack
> >>>> >>> >
> >>>> >>> > On Tue, Oct 11, 2016 at 5:12 PM, Apekshit Sharma <
> >>>> appy@cloudera.com>
> >>>> >>> > wrote:
> >>>> >>> >
> >>>> >>> >> @stack, it compiled for me.
> >>>> >>> >>
> >>>> >>> >> Also tried few commands, and have to say, it's well designed
> from
> >>>> user
> >>>> >>> >> commands perspective.
> >>>> >>> >>
> >>>> >>> >> On Tue, Oct 11, 2016 at 5:08 PM, Stack <st...@duboce.net>
> wrote:
> >>>> >>> >>
> >>>> >>> >> > On Tue, Oct 11, 2016 at 1:50 PM, Vladimir Rodionov <
> >>>> >>> >> vladrodionov@gmail.com
> >>>> >>> >> > >
> >>>> >>> >> > wrote:
> >>>> >>> >> >
> >>>> >>> >> > > Michael,
> >>>> >>> >> > >
> >>>> >>> >> > > Its in HBASE-7912
> >>>> >>> >> > >
> >>>> >>> >> > > This is tip of git log:
> >>>> >>> >> > >
> >>>> >>> >> > > commit a072f6f49a26a7259ff2aaef6cb56d85eb592482
> >>>> >>> >> > > Author: Frank Welsch <fw...@jps.net>
> >>>> >>> >> > > Date:   Fri Sep 23 18:00:42 2016 -0400
> >>>> >>> >> > >
> >>>> >>> >> > >     HBASE-16574 Book updates for backup and restore
> >>>> >>> >> > >
> >>>> >>> >> > > commit b14e2ab1c24e65ff88dd4c579acf83cb4ed0605e
> >>>> >>> >> > > Author: tedyu <yu...@gmail.com>
> >>>> >>> >> > > Date:   Wed Oct 5 16:29:40 2016 -0700
> >>>> >>> >> > >
> >>>> >>> >> > >     HBASE-16727 Backup refactoring: remove MR dependencies
> >>>> from
> >>>> >>> >> HMaster
> >>>> >>> >> > > (Vladimir Rodionov)
> >>>> >>> >> > >
> >>>> >>> >> > >
> >>>> >>> >> > Thanks. I have that. I tried it and it doesn't compile for
> me.
> >>>> Does
> >>>> >>> it
> >>>> >>> >> > compile for you?
> >>>> >>> >> > Thanks,
> >>>> >>> >> > M
> >>>> >>> >> >
> >>>> >>> >> >
> >>>> >>> >> >
> >>>> >>> >> > > -Vlad
> >>>> >>> >> > >
> >>>> >>> >> > > On Tue, Oct 11, 2016 at 1:44 PM, Stack <st...@duboce.net>
> >>>> wrote:
> >>>> >>> >> > >
> >>>> >>> >> > > > Which branch do I check out to try it? HBASE-7912 is not
> >>>> it. I
> >>>> >>> don't
> >>>> >>> >> > see
> >>>> >>> >> > > an
> >>>> >>> >> > > > HBASE-16727...
> >>>> >>> >> > > > Thanks,
> >>>> >>> >> > > > M
> >>>> >>> >> > > >
> >>>> >>> >> > > >
> >>>> >>> >> > > > On Tue, Oct 11, 2016 at 11:41 AM, Vladimir Rodionov <
> >>>> >>> >> > > > vladrodionov@gmail.com>
> >>>> >>> >> > > > wrote:
> >>>> >>> >> > > >
> >>>> >>> >> > > > > The last patch is on review board:
> >>>> >>> >> > > > > https://reviews.apache.org/r/52748
> >>>> >>> >> > > > >
> >>>> >>> >> > > > > On Mon, Oct 10, 2016 at 3:52 PM, Vladimir Rodionov <
> >>>> >>> >> > > > vladrodionov@gmail.com
> >>>> >>> >> > > > > >
> >>>> >>> >> > > > > wrote:
> >>>> >>> >> > > > >
> >>>> >>> >> > > > > > >> How hard to put in an hbase-backup module?
> >>>> hbase-server
> >>>> >>> is
> >>>> >>> >> fat
> >>>> >>> >> > > > enough
> >>>> >>> >> > > > > > >> already. Could be done as a follow-up.
> >>>> >>> >> > > > > >
> >>>> >>> >> > > > > > https://issues.apache.org/jira/browse/HBASE-16727?
> >>>> >>> >> > > > > > focusedCommentId=15531237&page=com.atlassian.jira.
> >>>> >>> >> > > > > > plugin.system.issuetabpanels:c
> >>>> >>> omment-tabpanel#comment-15531237
> >>>> >>> >> > > > > >
> >>>> >>> >> > > > > > Can we do merge first? Then we can discuss separate
> >>>> module.
> >>>> >>> >> > > > > >
> >>>> >>> >> > > > > > -Vlad
> >>>> >>> >> > > > > >
> >>>> >>> >> > > > > > On Mon, Oct 10, 2016 at 3:44 PM, Ted Yu <
> >>>> >>> yuzhihong@gmail.com>
> >>>> >>> >> > wrote:
> >>>> >>> >> > > > > >
> >>>> >>> >> > > > > >> Looks like the first quote was cut off.
> >>>> >>> >> > > > > >> The original sentence was:
> >>>> >>> >> > > > > >>
> >>>> >>> >> > > > > >> bq. no mapreduce job launched from master or region
> >>>> server.
> >>>> >>> >> > > > > >>
> >>>> >>> >> > > > > >> mapreduce job is launched from the node where
> command
> >>>> line
> >>>> >>> >> tool is
> >>>> >>> >> > > > run.
> >>>> >>> >> > > > > >>
> >>>> >>> >> > > > > >> On Mon, Oct 10, 2016 at 3:38 PM, Stack <
> >>>> stack@duboce.net>
> >>>> >>> >> wrote:
> >>>> >>> >> > > > > >>
> >>>> >>> >> > > > > >> > bq. launched from master or region server.
> >>>> >>> >> > > > > >> >
> >>>> >>> >> > > > > >> > What does this mean please? Has to be run from
> >>>> Master or
> >>>> >>> >> > > > RegionServer?
> >>>> >>> >> > > > > >> Can
> >>>> >>> >> > > > > >> > it be run from another node altogether?
> >>>> >>> >> > > > > >> >
> >>>> >>> >> > > > > >> > On Mon, Oct 10, 2016 at 1:44 PM, Vladimir
> Rodionov <
> >>>> >>> >> > > > > >> vladrodionov@gmail.com
> >>>> >>> >> > > > > >> > >
> >>>> >>> >> > > > > >> > wrote:
> >>>> >>> >> > > > > >> >
> >>>> >>> >> > > > > >> > > >> mapreduce dependency has been moved to client
> >>>> side
> >>>> >>> - no
> >>>> >>> >> > > > mapreduce
> >>>> >>> >> > > > > >> job
> >>>> >>> >> > > > > >> > >
> >>>> >>> >> > > > > >> > > 1. We have no code in the client module anymore,
> >>>> due to
> >>>> >>> >> > > dependency
> >>>> >>> >> > > > > on
> >>>> >>> >> > > > > >> > > internal server API (HFile and WAL access).
> >>>> >>> >> > > > > >> > > 2. Backup/ restore are client - driven
> operations,
> >>>> but
> >>>> >>> all
> >>>> >>> >> the
> >>>> >>> >> > > > code
> >>>> >>> >> > > > > >> > resides
> >>>> >>> >> > > > > >> > > in the server module
> >>>> >>> >> > > > > >> > >
> >>>> >>> >> > > > > >> >
> >>>> >>> >> > > > > >> > How hard to put in an hbase-backup module?
> >>>> hbase-server
> >>>> >>> is
> >>>> >>> >> fat
> >>>> >>> >> > > > enough
> >>>> >>> >> > > > > >> > already. Could be done as a follow-up.
> >>>> >>> >> > > > > >> >
> >>>> >>> >> > > > > >> > Thanks,
> >>>> >>> >> > > > > >> > St.Ack
> >>>> >>> >> > > > > >> >
> >>>> >>> >> > > > > >> >
> >>>> >>> >> > > > > >> >
> >>>> >>> >> > > > > >> > > 3. No MR in Master, no procedure - driven
> >>>> execution.
> >>>> >>> >> > > > > >> > > 4. Old good MR from command-line.
> >>>> >>> >> > > > > >> > > 5. Security was simplified and now only
> super-user
> >>>> is
> >>>> >>> >> allowed
> >>>> >>> >> > to
> >>>> >>> >> > > > run
> >>>> >>> >> > > > > >> > > backup/restores.
> >>>> >>> >> > > > > >> > > 6. HBase Backup API was gone due to 1. Now only
> >>>> >>> >> command-line
> >>>> >>> >> > > > access
> >>>> >>> >> > > > > to
> >>>> >>> >> > > > > >> > > backup tools.
> >>>> >>> >> > > > > >> > >
> >>>> >>> >> > > > > >> > > These consequences of refactoring has been
> >>>> discussed in
> >>>> >>> >> > > > HBASE-16727.
> >>>> >>> >> > > > > >> > >
> >>>> >>> >> > > > > >> > > -Vlad
> >>>> >>> >> > > > > >> > >
> >>>> >>> >> > > > > >> > >
> >>>> >>> >> > > > > >> > >
> >>>> >>> >> > > > > >> > >
> >>>> >>> >> > > > > >> > > On Mon, Oct 10, 2016 at 1:31 PM, Ted Yu <
> >>>> >>> >> yuzhihong@gmail.com>
> >>>> >>> >> > > > > wrote:
> >>>> >>> >> > > > > >> > >
> >>>> >>> >> > > > > >> > > > Reviving this thread.
> >>>> >>> >> > > > > >> > > >
> >>>> >>> >> > > > > >> > > > The following has taken place:
> >>>> >>> >> > > > > >> > > >
> >>>> >>> >> > > > > >> > > > mapreduce dependency has been moved to client
> >>>> side -
> >>>> >>> no
> >>>> >>> >> > > > mapreduce
> >>>> >>> >> > > > > >> job
> >>>> >>> >> > > > > >> > > > launched from master or region server.
> >>>> >>> >> > > > > >> > > > document patch (HBASE-16574) has been
> integrated.
> >>>> >>> >> > > > > >> > > > Updated mega patch has been attached to
> >>>> HBASE-14123:
> >>>> >>> this
> >>>> >>> >> > > covers
> >>>> >>> >> > > > > the
> >>>> >>> >> > > > > >> > > > refactor in #1 above and the protobuf 3 merge.
> >>>> >>> >> > > > > >> > > >
> >>>> >>> >> > > > > >> > > > If community has more feedback on the merge
> >>>> >>> proposal, I
> >>>> >>> >> > would
> >>>> >>> >> > > > love
> >>>> >>> >> > > > > >> to
> >>>> >>> >> > > > > >> > > hear
> >>>> >>> >> > > > > >> > > > it.
> >>>> >>> >> > > > > >> > > >
> >>>> >>> >> > > > > >> > > > Thanks
> >>>> >>> >> > > > > >> > > >
> >>>> >>> >> > > > > >> > > > On Thu, Sep 22, 2016 at 10:31 AM, Sean Busbey
> <
> >>>> >>> >> > > > > busbey@cloudera.com>
> >>>> >>> >> > > > > >> > > wrote:
> >>>> >>> >> > > > > >> > > >
> >>>> >>> >> > > > > >> > > > > I'd like to see the docs proposed on
> >>>> HBASE-16574
> >>>> >>> >> > integrated
> >>>> >>> >> > > > into
> >>>> >>> >> > > > > >> our
> >>>> >>> >> > > > > >> > > > > project's documentation prior to merge.
> >>>> >>> >> > > > > >> > > > >
> >>>> >>> >> > > > > >> > > > > On Thu, Sep 22, 2016 at 9:02 AM, Ted Yu <
> >>>> >>> >> > > yuzhihong@gmail.com>
> >>>> >>> >> > > > > >> wrote:
> >>>> >>> >> > > > > >> > > > > > This feature can be marked experimental
> due
> >>>> to
> >>>> >>> some
> >>>> >>> >> > > > > limitations
> >>>> >>> >> > > > > >> > such
> >>>> >>> >> > > > > >> > > as
> >>>> >>> >> > > > > >> > > > > > security.
> >>>> >>> >> > > > > >> > > > > >
> >>>> >>> >> > > > > >> > > > > > Your previous round of comments have been
> >>>> >>> addressed.
> >>>> >>> >> > > > > >> > > > > > Command line tool has gone through:
> >>>> >>> >> > > > > >> > > > > >
> >>>> >>> >> > > > > >> > > > > > HBASE-16620 Fix backup command-line tool
> >>>> >>> usability
> >>>> >>> >> > issues
> >>>> >>> >> > > > > >> > > > > > HBASE-16655 hbase backup describe with
> >>>> incorrect
> >>>> >>> >> backup
> >>>> >>> >> > id
> >>>> >>> >> > > > > >> results
> >>>> >>> >> > > > > >> > in
> >>>> >>> >> > > > > >> > > > NPE
> >>>> >>> >> > > > > >> > > > > >
> >>>> >>> >> > > > > >> > > > > > The updated doc has been attached to
> >>>> HBASE-16574.
> >>>> >>> >> > > > > >> > > > > >
> >>>> >>> >> > > > > >> > > > > > Cheers
> >>>> >>> >> > > > > >> > > > > >
> >>>> >>> >> > > > > >> > > > > > On Thu, Sep 22, 2016 at 8:53 AM, Stack <
> >>>> >>> >> > stack@duboce.net>
> >>>> >>> >> > > > > >> wrote:
> >>>> >>> >> > > > > >> > > > > >
> >>>> >>> >> > > > > >> > > > > >> On Wed, Sep 21, 2016 at 7:43 AM, Ted Yu <
> >>>> >>> >> > > > yuzhihong@gmail.com
> >>>> >>> >> > > > > >
> >>>> >>> >> > > > > >> > > wrote:
> >>>> >>> >> > > > > >> > > > > >>
> >>>> >>> >> > > > > >> > > > > >> > Are there more (review) comments ?
> >>>> >>> >> > > > > >> > > > > >> >
> >>>> >>> >> > > > > >> > > > > >> >
> >>>> >>> >> > > > > >> > > > > >> Are outstanding comments addressed?
> >>>> >>> >> > > > > >> > > > > >>
> >>>> >>> >> > > > > >> > > > > >> I don't see answer to my 'is this
> >>>> >>> experimental/will
> >>>> >>> >> it
> >>>> >>> >> > be
> >>>> >>> >> > > > > >> marked
> >>>> >>> >> > > > > >> > > > > >> experimental' question.
> >>>> >>> >> > > > > >> > > > > >>
> >>>> >>> >> > > > > >> > > > > >> I ran into some issues trying to use the
> >>>> >>> feature and
> >>>> >>> >> > > > > suggested
> >>>> >>> >> > > > > >> > that
> >>>> >>> >> > > > > >> > > a
> >>>> >>> >> > > > > >> > > > > >> feature likes this needs polish else
> it'll
> >>>> just
> >>>> >>> rot,
> >>>> >>> >> > > > unused.
> >>>> >>> >> > > > > >> Has
> >>>> >>> >> > > > > >> > > > polish
> >>>> >>> >> > > > > >> > > > > >> been applied? All ready for another
> 'user'
> >>>> test?
> >>>> >>> >> > Suggest
> >>>> >>> >> > > > that
> >>>> >>> >> > > > > >> you
> >>>> >>> >> > > > > >> > > > update
> >>>> >>> >> > > > > >> > > > > >> here going forward for the benefit of
> those
> >>>> >>> trying
> >>>> >>> >> to
> >>>> >>> >> > > > follow
> >>>> >>> >> > > > > >> along
> >>>> >>> >> > > > > >> > > and
> >>>> >>> >> > > > > >> > > > > who
> >>>> >>> >> > > > > >> > > > > >> are not watching JIRA change fly-by.
> >>>> >>> >> > > > > >> > > > > >>
> >>>> >>> >> > > > > >> > > > > >> It looks like doc got a revision -- I
> have
> >>>> to
> >>>> >>> check
> >>>> >>> >> --
> >>>> >>> >> > to
> >>>> >>> >> > > > > take
> >>>> >>> >> > > > > >> on
> >>>> >>> >> > > > > >> > > > > >> suggestion made above but again, suggest,
> >>>> that
> >>>> >>> this
> >>>> >>> >> > > thread
> >>>> >>> >> > > > > gets
> >>>> >>> >> > > > > >> > > > updated.
> >>>> >>> >> > > > > >> > > > > >>
> >>>> >>> >> > > > > >> > > > > >> Thanks,
> >>>> >>> >> > > > > >> > > > > >> St.Ack
> >>>> >>> >> > > > > >> > > > > >>
> >>>> >>> >> > > > > >> > > > > >>
> >>>> >>> >> > > > > >> > > > > >>
> >>>> >>> >> > > > > >> > > > > >> > Thanks
> >>>> >>> >> > > > > >> > > > > >> >
> >>>> >>> >> > > > > >> > > > > >> > On Tue, Sep 20, 2016 at 10:02 AM,
> Devaraj
> >>>> Das
> >>>> >>> <
> >>>> >>> >> > > > > >> > > ddas@hortonworks.com
> >>>> >>> >> > > > > >> > > > >
> >>>> >>> >> > > > > >> > > > > >> > wrote:
> >>>> >>> >> > > > > >> > > > > >> >
> >>>> >>> >> > > > > >> > > > > >> > > Just reviving this thread. Thanks
> Sean,
> >>>> >>> Stack,
> >>>> >>> >> > Dima,
> >>>> >>> >> > > > and
> >>>> >>> >> > > > > >> > others
> >>>> >>> >> > > > > >> > > > for
> >>>> >>> >> > > > > >> > > > > the
> >>>> >>> >> > > > > >> > > > > >> > > thorough reviews and testing. Thanks
> >>>> Ted and
> >>>> >>> >> Vlad
> >>>> >>> >> > for
> >>>> >>> >> > > > > >> taking
> >>>> >>> >> > > > > >> > > care
> >>>> >>> >> > > > > >> > > > of
> >>>> >>> >> > > > > >> > > > > >> the
> >>>> >>> >> > > > > >> > > > > >> > > feedback. Are we all good to do the
> >>>> merge
> >>>> >>> now?
> >>>> >>> >> > Rather
> >>>> >>> >> > > > do
> >>>> >>> >> > > > > >> > sooner
> >>>> >>> >> > > > > >> > > > than
> >>>> >>> >> > > > > >> > > > > >> > later.
> >>>> >>> >> > > > > >> > > > > >> > > ______________________________
> >>>> __________
> >>>> >>> >> > > > > >> > > > > >> > > From: saint.ack@gmail.com <
> >>>> >>> saint.ack@gmail.com>
> >>>> >>> >> on
> >>>> >>> >> > > > > behalf
> >>>> >>> >> > > > > >> of
> >>>> >>> >> > > > > >> > > > Stack
> >>>> >>> >> > > > > >> > > > > <
> >>>> >>> >> > > > > >> > > > > >> > > stack@duboce.net>
> >>>> >>> >> > > > > >> > > > > >> > > Sent: Monday, September 12, 2016
> 1:18 PM
> >>>> >>> >> > > > > >> > > > > >> > > To: HBase Dev List
> >>>> >>> >> > > > > >> > > > > >> > > Subject: Re: [DISCUSSION] Merge
> Backup /
> >>>> >>> >> Restore -
> >>>> >>> >> > > > Branch
> >>>> >>> >> > > > > >> > > > HBASE-7912
> >>>> >>> >> > > > > >> > > > > >> > >
> >>>> >>> >> > > > > >> > > > > >> > > On Mon, Sep 12, 2016 at 12:19 PM, Ted
> >>>> Yu <
> >>>> >>> >> > > > > >> yuzhihong@gmail.com
> >>>> >>> >> > > > > >> > >
> >>>> >>> >> > > > > >> > > > > wrote:
> >>>> >>> >> > > > > >> > > > > >> > >
> >>>> >>> >> > > > > >> > > > > >> > > > Mega patch (rev 18) is on
> HBASE-14123.
> >>>> >>> >> > > > > >> > > > > >> > > >
> >>>> >>> >> > > > > >> > > > > >> > > > Please comment on HBASE-14123 on
> how
> >>>> you
> >>>> >>> want
> >>>> >>> >> to
> >>>> >>> >> > > > > review.
> >>>> >>> >> > > > > >> > > > > >> > > >
> >>>> >>> >> > > > > >> > > > > >> > >
> >>>> >>> >> > > > > >> > > > > >> > >
> >>>> >>> >> > > > > >> > > > > >> > > Yeah. That was my lost tab. Last rb
> was
> >>>> 6
> >>>> >>> months
> >>>> >>> >> > ago.
> >>>> >>> >> > > > > >> Suggest
> >>>> >>> >> > > > > >> > > > > updating
> >>>> >>> >> > > > > >> > > > > >> > it.
> >>>> >>> >> > > > > >> > > > > >> > > RB is pretty good for review. Patch
> is
> >>>> only
> >>>> >>> >> 1.5M so
> >>>> >>> >> > > > > should
> >>>> >>> >> > > > > >> be
> >>>> >>> >> > > > > >> > > > fine.
> >>>> >>> >> > > > > >> > > > > >> > >
> >>>> >>> >> > > > > >> > > > > >> > > St.Ack
> >>>> >>> >> > > > > >> > > > > >> > >
> >>>> >>> >> > > > > >> > > > > >> > >
> >>>> >>> >> > > > > >> > > > > >> > > >
> >>>> >>> >> > > > > >> > > > > >> > > > Thanks
> >>>> >>> >> > > > > >> > > > > >> > > >
> >>>> >>> >> > > > > >> > > > > >> > > > On Mon, Sep 12, 2016 at 12:15 PM,
> >>>> Stack <
> >>>> >>> >> > > > > >> stack@duboce.net>
> >>>> >>> >> > > > > >> > > > wrote:
> >>>> >>> >> > > > > >> > > > > >> > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > On review of the 'patch', do I
> just
> >>>> >>> compare
> >>>> >>> >> the
> >>>> >>> >> > > > > branch
> >>>> >>> >> > > > > >> to
> >>>> >>> >> > > > > >> > > > > master or
> >>>> >>> >> > > > > >> > > > > >> > is
> >>>> >>> >> > > > > >> > > > > >> > > > > there a megapatch posted
> somewhere
> >>>> (I
> >>>> >>> think
> >>>> >>> >> I
> >>>> >>> >> > saw
> >>>> >>> >> > > > one
> >>>> >>> >> > > > > >> but
> >>>> >>> >> > > > > >> > it
> >>>> >>> >> > > > > >> > > > > seemed
> >>>> >>> >> > > > > >> > > > > >> > > stale
> >>>> >>> >> > > > > >> > > > > >> > > > > and then I 'lost' the tab). Sorry
> >>>> for
> >>>> >>> dumb
> >>>> >>> >> > > > question.
> >>>> >>> >> > > > > >> > > > > >> > > > > St.Ack
> >>>> >>> >> > > > > >> > > > > >> > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > On Mon, Sep 12, 2016 at 12:01 PM,
> >>>> Stack
> >>>> >>> <
> >>>> >>> >> > > > > >> stack@duboce.net
> >>>> >>> >> > > > > >> > >
> >>>> >>> >> > > > > >> > > > > wrote:
> >>>> >>> >> > > > > >> > > > > >> > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > > Late to the game. A few
> comments
> >>>> after
> >>>> >>> >> > > rereading
> >>>> >>> >> > > > > this
> >>>> >>> >> > > > > >> > > thread
> >>>> >>> >> > > > > >> > > > > as a
> >>>> >>> >> > > > > >> > > > > >> > > > 'user'.
> >>>> >>> >> > > > > >> > > > > >> > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > > + Before merge, a user-facing
> >>>> feature
> >>>> >>> like
> >>>> >>> >> > this
> >>>> >>> >> > > > > >> should
> >>>> >>> >> > > > > >> > > work
> >>>> >>> >> > > > > >> > > > > (If
> >>>> >>> >> > > > > >> > > > > >> > this
> >>>> >>> >> > > > > >> > > > > >> > > is
> >>>> >>> >> > > > > >> > > > > >> > > > > "higher-bar
> >>>> >>> >> > > > > >> > > > > >> > > > > > for new features", bring it on
> --
> >>>> >>> smile).
> >>>> >>> >> > > > > >> > > > > >> > > > > > + As a user, I tried the branch
> >>>> with
> >>>> >>> tools
> >>>> >>> >> > > after
> >>>> >>> >> > > > > >> > reviewing
> >>>> >>> >> > > > > >> > > > the
> >>>> >>> >> > > > > >> > > > > >> > > > > just-posted
> >>>> >>> >> > > > > >> > > > > >> > > > > > doc. I had an 'interesting'
> >>>> experience
> >>>> >>> >> (left
> >>>> >>> >> > > > > >> comments up
> >>>> >>> >> > > > > >> > > on
> >>>> >>> >> > > > > >> > > > > >> > issue). I
> >>>> >>> >> > > > > >> > > > > >> > > > > think
> >>>> >>> >> > > > > >> > > > > >> > > > > > the tooling/doc. important to
> get
> >>>> >>> right.
> >>>> >>> >> If
> >>>> >>> >> > it
> >>>> >>> >> > > > > breaks
> >>>> >>> >> > > > > >> > > easily
> >>>> >>> >> > > > > >> > > > > or
> >>>> >>> >> > > > > >> > > > > >> is
> >>>> >>> >> > > > > >> > > > > >> > > > > > inconsistent (or lacks
> 'polish'),
> >>>> >>> >> operators
> >>>> >>> >> > > will
> >>>> >>> >> > > > > >> judge
> >>>> >>> >> > > > > >> > the
> >>>> >>> >> > > > > >> > > > > whole
> >>>> >>> >> > > > > >> > > > > >> > > > > > backup/restore tooling chain as
> >>>> not
> >>>> >>> >> > trustworthy
> >>>> >>> >> > > > and
> >>>> >>> >> > > > > >> > > abandon
> >>>> >>> >> > > > > >> > > > > it.
> >>>> >>> >> > > > > >> > > > > >> > Lets
> >>>> >>> >> > > > > >> > > > > >> > > > not
> >>>> >>> >> > > > > >> > > > > >> > > > > > have this happen to this
> feature.
> >>>> >>> >> > > > > >> > > > > >> > > > > > + Matteo's suggestion (with a
> >>>> helpful
> >>>> >>> >> starter
> >>>> >>> >> > > > list)
> >>>> >>> >> > > > > >> that
> >>>> >>> >> > > > > >> > > > there
> >>>> >>> >> > > > > >> > > > > >> > needs
> >>>> >>> >> > > > > >> > > > > >> > > to
> >>>> >>> >> > > > > >> > > > > >> > > > > be
> >>>> >>> >> > > > > >> > > > > >> > > > > > explicit qualification on what
> is
> >>>> >>> actually
> >>>> >>> >> > > being
> >>>> >>> >> > > > > >> > delivered
> >>>> >>> >> > > > > >> > > > --
> >>>> >>> >> > > > > >> > > > > >> > > > including a
> >>>> >>> >> > > > > >> > > > > >> > > > > > listing of limitations (some
> look
> >>>> >>> serious
> >>>> >>> >> > such
> >>>> >>> >> > > as
> >>>> >>> >> > > > > >> data
> >>>> >>> >> > > > > >> > > bleed
> >>>> >>> >> > > > > >> > > > > from
> >>>> >>> >> > > > > >> > > > > >> > > other
> >>>> >>> >> > > > > >> > > > > >> > > > > > regions in WALs, but maybe I
> don't
> >>>> >>> care
> >>>> >>> >> for
> >>>> >>> >> > my
> >>>> >>> >> > > > use
> >>>> >>> >> > > > > >> > > case...)
> >>>> >>> >> > > > > >> > > > --
> >>>> >>> >> > > > > >> > > > > >> > needs
> >>>> >>> >> > > > > >> > > > > >> > > to
> >>>> >>> >> > > > > >> > > > > >> > > > > > accompany the merge. Lets fold
> >>>> them
> >>>> >>> into
> >>>> >>> >> the
> >>>> >>> >> > > user
> >>>> >>> >> > > > > >> doc.
> >>>> >>> >> > > > > >> > in
> >>>> >>> >> > > > > >> > > > the
> >>>> >>> >> > > > > >> > > > > >> > > technical
> >>>> >>> >> > > > > >> > > > > >> > > > > > overview area as suggested so
> user
> >>>> >>> >> > expectations
> >>>> >>> >> > > > are
> >>>> >>> >> > > > > >> > > properly
> >>>> >>> >> > > > > >> > > > > >> > managed
> >>>> >>> >> > > > > >> > > > > >> > > > > > (otherwise, they expect the
> world
> >>>> and
> >>>> >>> will
> >>>> >>> >> > just
> >>>> >>> >> > > > > give
> >>>> >>> >> > > > > >> up
> >>>> >>> >> > > > > >> > > when
> >>>> >>> >> > > > > >> > > > > we
> >>>> >>> >> > > > > >> > > > > >> > fall
> >>>> >>> >> > > > > >> > > > > >> > > > > > short). Vladimir did a list of
> >>>> what
> >>>> >>> is in
> >>>> >>> >> > each
> >>>> >>> >> > > of
> >>>> >>> >> > > > > the
> >>>> >>> >> > > > > >> > > phases
> >>>> >>> >> > > > > >> > > > > >> above
> >>>> >>> >> > > > > >> > > > > >> > > > which
> >>>> >>> >> > > > > >> > > > > >> > > > > > would serve as a good start.
> >>>> >>> >> > > > > >> > > > > >> > > > > > + Is this feature
> 'experimental'
> >>>> >>> (Matteo
> >>>> >>> >> asks
> >>>> >>> >> > > > > above).
> >>>> >>> >> > > > > >> > I'd
> >>>> >>> >> > > > > >> > > > > prefer
> >>>> >>> >> > > > > >> > > > > >> it
> >>>> >>> >> > > > > >> > > > > >> > > is
> >>>> >>> >> > > > > >> > > > > >> > > > > > not. If it is, it should be
> >>>> labelled
> >>>> >>> all
> >>>> >>> >> over
> >>>> >>> >> > > > that
> >>>> >>> >> > > > > >> it is
> >>>> >>> >> > > > > >> > > > so. I
> >>>> >>> >> > > > > >> > > > > >> see
> >>>> >>> >> > > > > >> > > > > >> > > > > current
> >>>> >>> >> > > > > >> > > > > >> > > > > > state called out as a '...
> >>>> technical
> >>>> >>> >> preview
> >>>> >>> >> > > > > >> feature'.
> >>>> >>> >> > > > > >> > > Does
> >>>> >>> >> > > > > >> > > > > this
> >>>> >>> >> > > > > >> > > > > >> > mean
> >>>> >>> >> > > > > >> > > > > >> > > > > > not-for-users?
> >>>> >>> >> > > > > >> > > > > >> > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > > St.Ack
> >>>> >>> >> > > > > >> > > > > >> > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > > On Mon, Sep 12, 2016 at 8:03
> AM,
> >>>> Ted
> >>>> >>> Yu <
> >>>> >>> >> > > > > >> > > > yuzhihong@gmail.com>
> >>>> >>> >> > > > > >> > > > > >> > wrote:
> >>>> >>> >> > > > > >> > > > > >> > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> Sean:
> >>>> >>> >> > > > > >> > > > > >> > > > > >> Do you have more comments ?
> >>>> >>> >> > > > > >> > > > > >> > > > > >>
> >>>> >>> >> > > > > >> > > > > >> > > > > >> Cheers
> >>>> >>> >> > > > > >> > > > > >> > > > > >>
> >>>> >>> >> > > > > >> > > > > >> > > > > >> On Fri, Sep 9, 2016 at 1:42
> PM,
> >>>> >>> Vladimir
> >>>> >>> >> > > > Rodionov
> >>>> >>> >> > > > > <
> >>>> >>> >> > > > > >> > > > > >> > > > > vladrodionov@gmail.com
> >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> wrote:
> >>>> >>> >> > > > > >> > > > > >> > > > > >>
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > Sean,
> >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > Backup/Restore can fail due
> to
> >>>> >>> various
> >>>> >>> >> > > > reasons:
> >>>> >>> >> > > > > >> > network
> >>>> >>> >> > > > > >> > > > > outage
> >>>> >>> >> > > > > >> > > > > >> > > > > (cluster
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > wide), various time-outs in
> >>>> HBase
> >>>> >>> and
> >>>> >>> >> HDFS
> >>>> >>> >> > > > > layer,
> >>>> >>> >> > > > > >> M/R
> >>>> >>> >> > > > > >> > > > > failure
> >>>> >>> >> > > > > >> > > > > >> > due
> >>>> >>> >> > > > > >> > > > > >> > > to
> >>>> >>> >> > > > > >> > > > > >> > > > > >> "HDFS
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > exceeded quota", user error
> >>>> (manual
> >>>> >>> >> > deletion
> >>>> >>> >> > > > of
> >>>> >>> >> > > > > >> data)
> >>>> >>> >> > > > > >> > > and
> >>>> >>> >> > > > > >> > > > > so
> >>>> >>> >> > > > > >> > > > > >> on
> >>>> >>> >> > > > > >> > > > > >> > so
> >>>> >>> >> > > > > >> > > > > >> > > > on.
> >>>> >>> >> > > > > >> > > > > >> > > > > >> That
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > is impossible to enumerate
> all
> >>>> >>> possible
> >>>> >>> >> > > types
> >>>> >>> >> > > > of
> >>>> >>> >> > > > > >> > > failures
> >>>> >>> >> > > > > >> > > > > in a
> >>>> >>> >> > > > > >> > > > > >> > > > > >> distributed
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > system - that is not our
> >>>> goal/task.
> >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > We focus completely on
> backup
> >>>> >>> system
> >>>> >>> >> table
> >>>> >>> >> > > > > >> > consistency
> >>>> >>> >> > > > > >> > > > in a
> >>>> >>> >> > > > > >> > > > > >> > > presence
> >>>> >>> >> > > > > >> > > > > >> > > > > of
> >>>> >>> >> > > > > >> > > > > >> > > > > >> any
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > type of failure. That is
> what I
> >>>> >>> call
> >>>> >>> >> > > > "tolerance
> >>>> >>> >> > > > > to
> >>>> >>> >> > > > > >> > > > > failures".
> >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > On a failure:
> >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > BACKUP. All backup system
> >>>> >>> information
> >>>> >>> >> > (prior
> >>>> >>> >> > > > to
> >>>> >>> >> > > > > >> > backup)
> >>>> >>> >> > > > > >> > > > > will
> >>>> >>> >> > > > > >> > > > > >> be
> >>>> >>> >> > > > > >> > > > > >> > > > > restored
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > and all temporary data,
> >>>> related to
> >>>> >>> a
> >>>> >>> >> > failed
> >>>> >>> >> > > > > >> session,
> >>>> >>> >> > > > > >> > in
> >>>> >>> >> > > > > >> > > > > HDFS
> >>>> >>> >> > > > > >> > > > > >> > will
> >>>> >>> >> > > > > >> > > > > >> > > be
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > deleted
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > RESTORE. We do not care
> about
> >>>> >>> system
> >>>> >>> >> data,
> >>>> >>> >> > > > > because
> >>>> >>> >> > > > > >> > > > restore
> >>>> >>> >> > > > > >> > > > > >> does
> >>>> >>> >> > > > > >> > > > > >> > > not
> >>>> >>> >> > > > > >> > > > > >> > > > > >> change
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > it. Temporary data in HDFS
> >>>> will be
> >>>> >>> >> cleaned
> >>>> >>> >> > > up
> >>>> >>> >> > > > > and
> >>>> >>> >> > > > > >> > table
> >>>> >>> >> > > > > >> > > > > will
> >>>> >>> >> > > > > >> > > > > >> be
> >>>> >>> >> > > > > >> > > > > >> > > in a
> >>>> >>> >> > > > > >> > > > > >> > > > > >> state
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > back to where it was before
> >>>> >>> operation
> >>>> >>> >> > > started.
> >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > This is what user should
> >>>> expect in
> >>>> >>> case
> >>>> >>> >> > of a
> >>>> >>> >> > > > > >> failure.
> >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > -Vlad
> >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > -Vlad
> >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > On Fri, Sep 9, 2016 at 12:56
> >>>> PM,
> >>>> >>> Sean
> >>>> >>> >> > > Busbey <
> >>>> >>> >> > > > > >> > > > > >> busbey@apache.org
> >>>> >>> >> > > > > >> > > > > >> > >
> >>>> >>> >> > > > > >> > > > > >> > > > > wrote:
> >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > Failing in a consistent
> way,
> >>>> with
> >>>> >>> >> docs
> >>>> >>> >> > > that
> >>>> >>> >> > > > > >> explain
> >>>> >>> >> > > > > >> > > the
> >>>> >>> >> > > > > >> > > > > >> > various
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > expected failures would be
> >>>> >>> >> sufficient.
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > On Fri, Sep 9, 2016 at
> 12:16
> >>>> PM,
> >>>> >>> >> > Vladimir
> >>>> >>> >> > > > > >> Rodionov
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > <vl...@gmail.com>
> >>>> wrote:
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > Do not worry Sean, doc
> is
> >>>> >>> coming
> >>>> >>> >> today
> >>>> >>> >> > > as
> >>>> >>> >> > > > a
> >>>> >>> >> > > > > >> > preview
> >>>> >>> >> > > > > >> > > > and
> >>>> >>> >> > > > > >> > > > > >> our
> >>>> >>> >> > > > > >> > > > > >> > > > writer
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > Frank
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > will be working on a
> >>>> putting
> >>>> >>> it
> >>>> >>> >> into
> >>>> >>> >> > > > Apache
> >>>> >>> >> > > > > >> > repo.
> >>>> >>> >> > > > > >> > > > > >> Timeline
> >>>> >>> >> > > > > >> > > > > >> > > > > depends
> >>>> >>> >> > > > > >> > > > > >> > > > > >> on
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > Franks schedule but I
> hope
> >>>> we
> >>>> >>> will
> >>>> >>> >> get
> >>>> >>> >> > > it
> >>>> >>> >> > > > > >> rather
> >>>> >>> >> > > > > >> > > > sooner
> >>>> >>> >> > > > > >> > > > > >> than
> >>>> >>> >> > > > > >> > > > > >> > > > > later.
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > As for failure testing,
> we
> >>>> are
> >>>> >>> >> > focusing
> >>>> >>> >> > > > only
> >>>> >>> >> > > > > >> on a
> >>>> >>> >> > > > > >> > > > > >> consistent
> >>>> >>> >> > > > > >> > > > > >> > > > state
> >>>> >>> >> > > > > >> > > > > >> > > > > >> of
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > backup system data in a
> >>>> >>> presence of
> >>>> >>> >> > any
> >>>> >>> >> > > > type
> >>>> >>> >> > > > > >> of
> >>>> >>> >> > > > > >> > > > > failures,
> >>>> >>> >> > > > > >> > > > > >> We
> >>>> >>> >> > > > > >> > > > > >> > > are
> >>>> >>> >> > > > > >> > > > > >> > > > > not
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > going
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > to implement  anything
> more
> >>>> >>> >> "fancy",
> >>>> >>> >> > > than
> >>>> >>> >> > > > > >> that.
> >>>> >>> >> > > > > >> > We
> >>>> >>> >> > > > > >> > > > > allow
> >>>> >>> >> > > > > >> > > > > >> > both:
> >>>> >>> >> > > > > >> > > > > >> > > > > >> backup
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > and
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > restore to fail. What we
> >>>> do not
> >>>> >>> >> allow
> >>>> >>> >> > is
> >>>> >>> >> > > > to
> >>>> >>> >> > > > > >> have
> >>>> >>> >> > > > > >> > > > system
> >>>> >>> >> > > > > >> > > > > >> data
> >>>> >>> >> > > > > >> > > > > >> > > > > >> corrupted.
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > Will it suffice for you?
> >>>> Do you
> >>>> >>> >> have
> >>>> >>> >> > any
> >>>> >>> >> > > > > other
> >>>> >>> >> > > > > >> > > > > concerns,
> >>>> >>> >> > > > > >> > > > > >> you
> >>>> >>> >> > > > > >> > > > > >> > > > want
> >>>> >>> >> > > > > >> > > > > >> > > > > >> us to
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > address?
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > -Vlad
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > > On Fri, Sep 9, 2016 at
> >>>> 10:56
> >>>> >>> AM,
> >>>> >>> >> Sean
> >>>> >>> >> > > > > Busbey <
> >>>> >>> >> > > > > >> > > > > >> > > busbey@apache.org
> >>>> >>> >> > > > > >> > > > > >> > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > wrote:
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> "docs will come to
> Apache
> >>>> >>> soon"
> >>>> >>> >> does
> >>>> >>> >> > > not
> >>>> >>> >> > > > > >> address
> >>>> >>> >> > > > > >> > > my
> >>>> >>> >> > > > > >> > > > > >> concern
> >>>> >>> >> > > > > >> > > > > >> > > > > around
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > docs
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > at
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> all, unless said docs
> have
> >>>> >>> already
> >>>> >>> >> > made
> >>>> >>> >> > > > it
> >>>> >>> >> > > > > >> into
> >>>> >>> >> > > > > >> > > the
> >>>> >>> >> > > > > >> > > > > >> project
> >>>> >>> >> > > > > >> > > > > >> > > > > repo. I
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > don't
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> want third party
> >>>> resources for
> >>>> >>> >> using
> >>>> >>> >> > a
> >>>> >>> >> > > > > major
> >>>> >>> >> > > > > >> and
> >>>> >>> >> > > > > >> > > > > >> important
> >>>> >>> >> > > > > >> > > > > >> > > > > feature
> >>>> >>> >> > > > > >> > > > > >> > > > > >> of
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > the
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> project, I want us to
> >>>> provide
> >>>> >>> end
> >>>> >>> >> > users
> >>>> >>> >> > > > > with
> >>>> >>> >> > > > > >> > what
> >>>> >>> >> > > > > >> > > > they
> >>>> >>> >> > > > > >> > > > > >> need
> >>>> >>> >> > > > > >> > > > > >> > > to
> >>>> >>> >> > > > > >> > > > > >> > > > > get
> >>>> >>> >> > > > > >> > > > > >> > > > > >> the
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > job
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> done.
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >>
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> I see some calls for
> >>>> patience
> >>>> >>> on
> >>>> >>> >> the
> >>>> >>> >> > > > > failure
> >>>> >>> >> > > > > >> > > > testing,
> >>>> >>> >> > > > > >> > > > > but
> >>>> >>> >> > > > > >> > > > > >> > the
> >>>> >>> >> > > > > >> > > > > >> > > > > >> appeal
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > to
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > us
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> having done a bad job
> of
> >>>> >>> requiring
> >>>> >>> >> > > proper
> >>>> >>> >> > > > > >> tests
> >>>> >>> >> > > > > >> > of
> >>>> >>> >> > > > > >> > > > > >> previous
> >>>> >>> >> > > > > >> > > > > >> > > > > >> features
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > just
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> makes me more concerned
> >>>> about
> >>>> >>> not
> >>>> >>> >> > > getting
> >>>> >>> >> > > > > >> them
> >>>> >>> >> > > > > >> > > > here. I
> >>>> >>> >> > > > > >> > > > > >> > don't
> >>>> >>> >> > > > > >> > > > > >> > > > want
> >>>> >>> >> > > > > >> > > > > >> > > > > >> to
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > set
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> yet another bad example
> >>>> that
> >>>> >>> will
> >>>> >>> >> > then
> >>>> >>> >> > > be
> >>>> >>> >> > > > > >> > pointed
> >>>> >>> >> > > > > >> > > to
> >>>> >>> >> > > > > >> > > > > in
> >>>> >>> >> > > > > >> > > > > >> the
> >>>> >>> >> > > > > >> > > > > >> > > > > future.
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >>
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> On Sep 8, 2016 10:50,
> "Ted
> >>>> >>> Yu" <
> >>>> >>> >> > > > > >> > > yuzhihong@gmail.com
> >>>> >>> >> > > > > >> > > > >
> >>>> >>> >> > > > > >> > > > > >> > wrote:
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >>
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > Is there any concern
> >>>> which
> >>>> >>> is
> >>>> >>> >> not
> >>>> >>> >> > > > > >> addressed ?
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > Do we need another
> Vote
> >>>> >>> thread ?
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > Thanks
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > On Thu, Sep 8, 2016
> at
> >>>> 9:21
> >>>> >>> AM,
> >>>> >>> >> > > Andrew
> >>>> >>> >> > > > > >> > Purtell <
> >>>> >>> >> > > > > >> > > > > >> > > > > >> apurtell@apache.org
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > wrote:
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > Vlad,
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > I apologize for
> using
> >>>> the
> >>>> >>> term
> >>>> >>> >> > > > > >> 'half-baked'
> >>>> >>> >> > > > > >> > > in a
> >>>> >>> >> > > > > >> > > > > way
> >>>> >>> >> > > > > >> > > > > >> > that
> >>>> >>> >> > > > > >> > > > > >> > > > > could
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > seem a
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > description of
> >>>> >>> HBASE-7912. I
> >>>> >>> >> > meant
> >>>> >>> >> > > > that
> >>>> >>> >> > > > > >> as a
> >>>> >>> >> > > > > >> > > > > general
> >>>> >>> >> > > > > >> > > > > >> > > > > >> hypothetical.
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > On Wed, Sep 7,
> 2016 at
> >>>> >>> 9:36
> >>>> >>> >> AM,
> >>>> >>> >> > > > > Vladimir
> >>>> >>> >> > > > > >> > > > Rodionov
> >>>> >>> >> > > > > >> > > > > <
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> vladrodionov@gmail.com>
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > wrote:
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >> I'm not sure
> that
> >>>> >>> "There
> >>>> >>> >> is
> >>>> >>> >> > > > > already
> >>>> >>> >> > > > > >> > lots
> >>>> >>> >> > > > > >> > > of
> >>>> >>> >> > > > > >> > > > > >> > > half-baked
> >>>> >>> >> > > > > >> > > > > >> > > > > >> code
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > in
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > the
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > branch,
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > so what's the
> harm
> >>>> in
> >>>> >>> adding
> >>>> >>> >> > > more?"
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > I meant - not
> >>>> >>> production -
> >>>> >>> >> > ready
> >>>> >>> >> > > > yet.
> >>>> >>> >> > > > > >> This
> >>>> >>> >> > > > > >> > > is
> >>>> >>> >> > > > > >> > > > > 2.0
> >>>> >>> >> > > > > >> > > > > >> > > > > development
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > branch
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > and,
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > hence many
> features
> >>>> are
> >>>> >>> in
> >>>> >>> >> > works,
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > not being tested
> >>>> well
> >>>> >>> etc.
> >>>> >>> >> I do
> >>>> >>> >> > > not
> >>>> >>> >> > > > > >> > consider
> >>>> >>> >> > > > > >> > > > > backup
> >>>> >>> >> > > > > >> > > > > >> > as
> >>>> >>> >> > > > > >> > > > > >> > > > half
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > baked
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > feature -
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > it has passed our
> >>>> >>> internal
> >>>> >>> >> QA
> >>>> >>> >> > and
> >>>> >>> >> > > > has
> >>>> >>> >> > > > > >> very
> >>>> >>> >> > > > > >> > > > good
> >>>> >>> >> > > > > >> > > > > >> doc,
> >>>> >>> >> > > > > >> > > > > >> > > > which
> >>>> >>> >> > > > > >> > > > > >> > > > > we
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > will
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > provide
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > to Apache
> shortly.
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > -Vlad
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > On Wed, Sep 7,
> 2016
> >>>> at
> >>>> >>> 9:13
> >>>> >>> >> AM,
> >>>> >>> >> > > > > Andrew
> >>>> >>> >> > > > > >> > > > Purtell <
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > apurtell@apache.org>
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > We shouldn't
> admit
> >>>> >>> half
> >>>> >>> >> baked
> >>>> >>> >> > > > > changes
> >>>> >>> >> > > > > >> > that
> >>>> >>> >> > > > > >> > > > > won't
> >>>> >>> >> > > > > >> > > > > >> be
> >>>> >>> >> > > > > >> > > > > >> > > > > >> finished.
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> However
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > in
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > this case the
> crew
> >>>> >>> >> working on
> >>>> >>> >> > > > this
> >>>> >>> >> > > > > >> > feature
> >>>> >>> >> > > > > >> > > > are
> >>>> >>> >> > > > > >> > > > > >> long
> >>>> >>> >> > > > > >> > > > > >> > > > > timers
> >>>> >>> >> > > > > >> > > > > >> > > > > >> and
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > less
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > likely
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > than just about
> >>>> >>> anyone to
> >>>> >>> >> > leave
> >>>> >>> >> > > > > >> > something
> >>>> >>> >> > > > > >> > > > in a
> >>>> >>> >> > > > > >> > > > > >> half
> >>>> >>> >> > > > > >> > > > > >> > > > baked
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > state. Of
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > course
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > there is no
> >>>> guarantee
> >>>> >>> how
> >>>> >>> >> > > > anything
> >>>> >>> >> > > > > >> will
> >>>> >>> >> > > > > >> > > turn
> >>>> >>> >> > > > > >> > > > > out,
> >>>> >>> >> > > > > >> > > > > >> > > but I
> >>>> >>> >> > > > > >> > > > > >> > > > > am
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > willing
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> to
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > take
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > a little on
> faith
> >>>> if
> >>>> >>> they
> >>>> >>> >> > feel
> >>>> >>> >> > > > > their
> >>>> >>> >> > > > > >> > best
> >>>> >>> >> > > > > >> > > > path
> >>>> >>> >> > > > > >> > > > > >> > > forward
> >>>> >>> >> > > > > >> > > > > >> > > > > now
> >>>> >>> >> > > > > >> > > > > >> > > > > >> is
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > to
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > merge
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > to
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > trunk. I only
> >>>> wish I
> >>>> >>> had
> >>>> >>> >> > > > bandwidth
> >>>> >>> >> > > > > to
> >>>> >>> >> > > > > >> > have
> >>>> >>> >> > > > > >> > > > > done
> >>>> >>> >> > > > > >> > > > > >> > some
> >>>> >>> >> > > > > >> > > > > >> > > > real
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > kicking
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> of
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > the
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > tires by now.
> >>>> Maybe
> >>>> >>> this
> >>>> >>> >> > week.
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > (Yes, I'm using
> >>>> some
> >>>> >>> of
> >>>> >>> >> that
> >>>> >>> >> > > time
> >>>> >>> >> > > > > for
> >>>> >>> >> > > > > >> > this
> >>>> >>> >> > > > > >> > > > > email
> >>>> >>> >> > > > > >> > > > > >> > :-)
> >>>> >>> >> > > > > >> > > > > >> > > > but
> >>>> >>> >> > > > > >> > > > > >> > > > > I
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > type
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > fast.)
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > That said, I
> would
> >>>> >>> like to
> >>>> >>> >> > > > agitate
> >>>> >>> >> > > > > >> for
> >>>> >>> >> > > > > >> > > > making
> >>>> >>> >> > > > > >> > > > > 2.0
> >>>> >>> >> > > > > >> > > > > >> > > more
> >>>> >>> >> > > > > >> > > > > >> > > > > real
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > and
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> spend
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > some
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > time on it now
> >>>> that
> >>>> >>> I'm
> >>>> >>> >> > winding
> >>>> >>> >> > > > > down
> >>>> >>> >> > > > > >> > with
> >>>> >>> >> > > > > >> > > > > 0.98. I
> >>>> >>> >> > > > > >> > > > > >> > > think
> >>>> >>> >> > > > > >> > > > > >> > > > > >> that
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > means
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branching for
> 2.0
> >>>> real
> >>>> >>> >> soon
> >>>> >>> >> > now
> >>>> >>> >> > > > and
> >>>> >>> >> > > > > >> even
> >>>> >>> >> > > > > >> > > > > evicting
> >>>> >>> >> > > > > >> > > > > >> > > > things
> >>>> >>> >> > > > > >> > > > > >> > > > > >> from
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > 2.0
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > branch
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > that aren't
> >>>> finished
> >>>> >>> or
> >>>> >>> >> > stable,
> >>>> >>> >> > > > > >> leaving
> >>>> >>> >> > > > > >> > > them
> >>>> >>> >> > > > > >> > > > > only
> >>>> >>> >> > > > > >> > > > > >> > > once
> >>>> >>> >> > > > > >> > > > > >> > > > > >> again
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > in
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > the
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > master
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branch. Or,
> maybe
> >>>> just
> >>>> >>> >> > evicting
> >>>> >>> >> > > > > them.
> >>>> >>> >> > > > > >> > > Let's
> >>>> >>> >> > > > > >> > > > > take
> >>>> >>> >> > > > > >> > > > > >> it
> >>>> >>> >> > > > > >> > > > > >> > > > case
> >>>> >>> >> > > > > >> > > > > >> > > > > by
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > case.
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > I think this
> >>>> feature
> >>>> >>> can
> >>>> >>> >> come
> >>>> >>> >> > > in
> >>>> >>> >> > > > > >> > > relatively
> >>>> >>> >> > > > > >> > > > > >> safely.
> >>>> >>> >> > > > > >> > > > > >> > > As
> >>>> >>> >> > > > > >> > > > > >> > > > > >> added
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > insurance,
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > let's admit the
> >>>> >>> >> possibility
> >>>> >>> >> > it
> >>>> >>> >> > > > > could
> >>>> >>> >> > > > > >> be
> >>>> >>> >> > > > > >> > > > > reverted
> >>>> >>> >> > > > > >> > > > > >> on
> >>>> >>> >> > > > > >> > > > > >> > > the
> >>>> >>> >> > > > > >> > > > > >> > > > > 2.0
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > branch
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> if
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > folks
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > working on
> >>>> >>> stabilizing 2.0
> >>>> >>> >> > > decide
> >>>> >>> >> > > > > to
> >>>> >>> >> > > > > >> > evict
> >>>> >>> >> > > > > >> > > > it
> >>>> >>> >> > > > > >> > > > > >> > because
> >>>> >>> >> > > > > >> > > > > >> > > > it
> >>>> >>> >> > > > > >> > > > > >> > > > > is
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > unfinished
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > or
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > unstable,
> because
> >>>> that
> >>>> >>> >> > > certainly
> >>>> >>> >> > > > > can
> >>>> >>> >> > > > > >> > > > happen. I
> >>>> >>> >> > > > > >> > > > > >> > would
> >>>> >>> >> > > > > >> > > > > >> > > > > >> expect if
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > talk
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > like
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > that starts,
> we'd
> >>>> get
> >>>> >>> help
> >>>> >>> >> > > > > finishing
> >>>> >>> >> > > > > >> or
> >>>> >>> >> > > > > >> > > > > >> stabilizing
> >>>> >>> >> > > > > >> > > > > >> > > > > what's
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > under
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > discussion
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > for revert. Or,
> >>>> we'd
> >>>> >>> have
> >>>> >>> >> a
> >>>> >>> >> > > > revert.
> >>>> >>> >> > > > > >> > Either
> >>>> >>> >> > > > > >> > > > way
> >>>> >>> >> > > > > >> > > > > >> the
> >>>> >>> >> > > > > >> > > > > >> > > > > outcome
> >>>> >>> >> > > > > >> > > > > >> > > > > >> is
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > acceptable.
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > On Wed, Sep 7,
> >>>> 2016 at
> >>>> >>> >> 8:56
> >>>> >>> >> > AM,
> >>>> >>> >> > > > > Dima
> >>>> >>> >> > > > > >> > > Spivak
> >>>> >>> >> > > > > >> > > > <
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > dimaspivak@apache.org
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > I'm not sure
> >>>> that
> >>>> >>> >> "There is
> >>>> >>> >> > > > > already
> >>>> >>> >> > > > > >> > lots
> >>>> >>> >> > > > > >> > > > of
> >>>> >>> >> > > > > >> > > > > >> > > > half-baked
> >>>> >>> >> > > > > >> > > > > >> > > > > >> code
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > in
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> the
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branch,
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > so what's the
> >>>> harm
> >>>> >>> in
> >>>> >>> >> > adding
> >>>> >>> >> > > > > more?"
> >>>> >>> >> > > > > >> > is a
> >>>> >>> >> > > > > >> > > > > good
> >>>> >>> >> > > > > >> > > > > >> > code
> >>>> >>> >> > > > > >> > > > > >> > > > > commit
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > philosophy
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > for
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > a
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> fault-tolerant
> >>>> >>> >> distributed
> >>>> >>> >> > > data
> >>>> >>> >> > > > > >> store.
> >>>> >>> >> > > > > >> > > ;)
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > More
> seriously,
> >>>> a
> >>>> >>> lack
> >>>> >>> >> of
> >>>> >>> >> > > test
> >>>> >>> >> > > > > >> > coverage
> >>>> >>> >> > > > > >> > > > for
> >>>> >>> >> > > > > >> > > > > >> > > existing
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > features
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > shouldn't
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > be
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > used as
> >>>> >>> justification
> >>>> >>> >> for
> >>>> >>> >> > > > > >> introducing
> >>>> >>> >> > > > > >> > > new
> >>>> >>> >> > > > > >> > > > > >> > features
> >>>> >>> >> > > > > >> > > > > >> > > > with
> >>>> >>> >> > > > > >> > > > > >> > > > > >> the
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > same
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > shortcomings.
> >>>> >>> >> Ultimately,
> >>>> >>> >> > > it's
> >>>> >>> >> > > > > the
> >>>> >>> >> > > > > >> end
> >>>> >>> >> > > > > >> > > > user
> >>>> >>> >> > > > > >> > > > > who
> >>>> >>> >> > > > > >> > > > > >> > > will
> >>>> >>> >> > > > > >> > > > > >> > > > > feel
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > the
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> pain,
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > so
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > shouldn't we
> do
> >>>> >>> >> everything
> >>>> >>> >> > we
> >>>> >>> >> > > > can
> >>>> >>> >> > > > > >> to
> >>>> >>> >> > > > > >> > > > > mitigate
> >>>> >>> >> > > > > >> > > > > >> > that?
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > -Dima
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > On Wed, Sep
> 7,
> >>>> 2016
> >>>> >>> at
> >>>> >>> >> 8:46
> >>>> >>> >> > > AM,
> >>>> >>> >> > > > > >> > Vladimir
> >>>> >>> >> > > > > >> > > > > >> > Rodionov <
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >>>> >>> vladrodionov@gmail.com>
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > wrote:
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Sean,
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have docs
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Agree. We
> >>>> have a
> >>>> >>> doc
> >>>> >>> >> and
> >>>> >>> >> > > > backup
> >>>> >>> >> > > > > >> is
> >>>> >>> >> > > > > >> > the
> >>>> >>> >> > > > > >> > > > > most
> >>>> >>> >> > > > > >> > > > > >> > > > > documented
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > feature
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > :),
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > we
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > will
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > release it
> >>>> >>> shortly to
> >>>> >>> >> > > Apache.
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have
> >>>> sunny-day
> >>>> >>> >> > > correctness
> >>>> >>> >> > > > > >> tests
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Feature has
> >>>> >>> close to
> >>>> >>> >> 60
> >>>> >>> >> > > test
> >>>> >>> >> > > > > >> cases,
> >>>> >>> >> > > > > >> > > > which
> >>>> >>> >> > > > > >> > > > > >> run
> >>>> >>> >> > > > > >> > > > > >> > > for
> >>>> >>> >> > > > > >> > > > > >> > > > > >> approx
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > 30
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> min.
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > We
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > can
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > add more,
> if
> >>>> >>> >> community do
> >>>> >>> >> > > not
> >>>> >>> >> > > > > >> mind
> >>>> >>> >> > > > > >> > :)
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have
> >>>> >>> >> > > > > >> > correctness-in-face-of-failure
> >>>> >>> >> > > > > >> > > > > tests
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Any
> examples
> >>>> of
> >>>> >>> these
> >>>> >>> >> > tests
> >>>> >>> >> > > > in
> >>>> >>> >> > > > > >> > > existing
> >>>> >>> >> > > > > >> > > > > >> > features?
> >>>> >>> >> > > > > >> > > > > >> > > > In
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > works,
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > we
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > have a
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > clear
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> understanding
> >>>> of
> >>>> >>> what
> >>>> >>> >> > > should
> >>>> >>> >> > > > be
> >>>> >>> >> > > > > >> done
> >>>> >>> >> > > > > >> > > by
> >>>> >>> >> > > > > >> > > > > the
> >>>> >>> >> > > > > >> > > > > >> > time
> >>>> >>> >> > > > > >> > > > > >> > > of
> >>>> >>> >> > > > > >> > > > > >> > > > > 2.0
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> release.
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > That is
> very
> >>>> close
> >>>> >>> >> goal
> >>>> >>> >> > for
> >>>> >>> >> > > > us,
> >>>> >>> >> > > > > >> to
> >>>> >>> >> > > > > >> > > > verify
> >>>> >>> >> > > > > >> > > > > IT
> >>>> >>> >> > > > > >> > > > > >> > > monkey
> >>>> >>> >> > > > > >> > > > > >> > > > > for
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> existing
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > code.
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * don't
> rely
> >>>> on
> >>>> >>> things
> >>>> >>> >> > > > outside
> >>>> >>> >> > > > > of
> >>>> >>> >> > > > > >> > > HBase
> >>>> >>> >> > > > > >> > > > > for
> >>>> >>> >> > > > > >> > > > > >> > > normal
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > operation
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > (okay
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > for
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > advanced
> >>>> >>> operation)
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > We do not.
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Enormous
> time
> >>>> has
> >>>> >>> been
> >>>> >>> >> > > spent
> >>>> >>> >> > > > > >> already
> >>>> >>> >> > > > > >> > > on
> >>>> >>> >> > > > > >> > > > > the
> >>>> >>> >> > > > > >> > > > > >> > > > > development
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > and
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > testing
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > the
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > feature, it
> >>>> has
> >>>> >>> passed
> >>>> >>> >> > our
> >>>> >>> >> > > > > >> internal
> >>>> >>> >> > > > > >> > > > tests
> >>>> >>> >> > > > > >> > > > > and
> >>>> >>> >> > > > > >> > > > > >> > > many
> >>>> >>> >> > > > > >> > > > > >> > > > > >> rounds
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > of
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> code
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > reviews
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > by HBase
> >>>> >>> committers.
> >>>> >>> >> We
> >>>> >>> >> > do
> >>>> >>> >> > > > not
> >>>> >>> >> > > > > >> mind
> >>>> >>> >> > > > > >> > if
> >>>> >>> >> > > > > >> > > > > >> someone
> >>>> >>> >> > > > > >> > > > > >> > > from
> >>>> >>> >> > > > > >> > > > > >> > > > > >> HBase
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > community
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > (outside of
> >>>> HW)
> >>>> >>> will
> >>>> >>> >> > review
> >>>> >>> >> > > > the
> >>>> >>> >> > > > > >> > code,
> >>>> >>> >> > > > > >> > > > but
> >>>> >>> >> > > > > >> > > > > it
> >>>> >>> >> > > > > >> > > > > >> > will
> >>>> >>> >> > > > > >> > > > > >> > > > > >> probably
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> takes
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > forever
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > to
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > wait for
> >>>> >>> volunteer?,
> >>>> >>> >> the
> >>>> >>> >> > > > > feature
> >>>> >>> >> > > > > >> is
> >>>> >>> >> > > > > >> > > > quite
> >>>> >>> >> > > > > >> > > > > >> large
> >>>> >>> >> > > > > >> > > > > >> > > > (1MB+
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> cumulative
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > patch)
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > 2.0 branch
> is
> >>>> >>> full of
> >>>> >>> >> > half
> >>>> >>> >> > > > > baked
> >>>> >>> >> > > > > >> > > > features,
> >>>> >>> >> > > > > >> > > > > >> most
> >>>> >>> >> > > > > >> > > > > >> > > of
> >>>> >>> >> > > > > >> > > > > >> > > > > them
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > are
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > in
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > active
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> development,
> >>>> >>> >> therefore I
> >>>> >>> >> > am
> >>>> >>> >> > > > not
> >>>> >>> >> > > > > >> > > > following
> >>>> >>> >> > > > > >> > > > > you
> >>>> >>> >> > > > > >> > > > > >> > > here,
> >>>> >>> >> > > > > >> > > > > >> > > > > >> Sean?
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > Why
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > HBASE-7912
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > is
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > not good
> >>>> enough
> >>>> >>> yet
> >>>> >>> >> to be
> >>>> >>> >> > > > > >> integrated
> >>>> >>> >> > > > > >> > > > into
> >>>> >>> >> > > > > >> > > > > 2.0
> >>>> >>> >> > > > > >> > > > > >> > > > branch?
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > -Vlad
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > On Wed,
> Sep 7,
> >>>> >>> 2016 at
> >>>> >>> >> > 8:23
> >>>> >>> >> > > > AM,
> >>>> >>> >> > > > > >> Sean
> >>>> >>> >> > > > > >> > > > > Busbey <
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > busbey@apache.org
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > On Tue,
> Sep
> >>>> 6,
> >>>> >>> 2016
> >>>> >>> >> at
> >>>> >>> >> > > > 10:36
> >>>> >>> >> > > > > >> PM,
> >>>> >>> >> > > > > >> > > Josh
> >>>> >>> >> > > > > >> > > > > >> Elser <
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> josh.elser@gmail.com>
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > wrote:
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > So, the
> >>>> >>> answer to
> >>>> >>> >> > > Sean's
> >>>> >>> >> > > > > >> > original
> >>>> >>> >> > > > > >> > > > > >> question
> >>>> >>> >> > > > > >> > > > > >> > is
> >>>> >>> >> > > > > >> > > > > >> > > > "as
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > robust as
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > snapshots
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> presently
> >>>> >>> are"?
> >>>> >>> >> > > > > >> (independence of
> >>>> >>> >> > > > > >> > > > > >> > > backup/restore
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > failure
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > tolerance
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > from
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> snapshot
> >>>> >>> failure
> >>>> >>> >> > > > tolerance)
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > Is this
> >>>> just a
> >>>> >>> >> > question
> >>>> >>> >> > > > WRT
> >>>> >>> >> > > > > >> > > context
> >>>> >>> >> > > > > >> > > > of
> >>>> >>> >> > > > > >> > > > > >> the
> >>>> >>> >> > > > > >> > > > > >> > > > > change,
> >>>> >>> >> > > > > >> > > > > >> > > > > >> or
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > is it
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > means
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > for a
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > veto
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > from
> you,
> >>>> >>> Sean?
> >>>> >>> >> Just
> >>>> >>> >> > > > trying
> >>>> >>> >> > > > > >> to
> >>>> >>> >> > > > > >> > > make
> >>>> >>> >> > > > > >> > > > > sure
> >>>> >>> >> > > > > >> > > > > >> > I'm
> >>>> >>> >> > > > > >> > > > > >> > > > > >> following
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> along
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > adequately.
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > I'd say
> ATM
> >>>> I'm
> >>>> >>> -0,
> >>>> >>> >> > > > bordering
> >>>> >>> >> > > > > >> on
> >>>> >>> >> > > > > >> > -1
> >>>> >>> >> > > > > >> > > > but
> >>>> >>> >> > > > > >> > > > > not
> >>>> >>> >> > > > > >> > > > > >> > for
> >>>> >>> >> > > > > >> > > > > >> > > > > >> reasons
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > I
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > can
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > articulate
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > well.
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Here's an
> >>>> >>> attempt.
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > We've
> been
> >>>> >>> trying to
> >>>> >>> >> > > move,
> >>>> >>> >> > > > > as a
> >>>> >>> >> > > > > >> > > > > community,
> >>>> >>> >> > > > > >> > > > > >> > > > towards
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > minimizing
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > risk
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > to
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> downstream
> >>>> >>> folks by
> >>>> >>> >> > > getting
> >>>> >>> >> > > > > >> > > "complete
> >>>> >>> >> > > > > >> > > > > >> enough
> >>>> >>> >> > > > > >> > > > > >> > > for
> >>>> >>> >> > > > > >> > > > > >> > > > > use"
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > gates
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> in
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > place
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > before we
> >>>> >>> introduce
> >>>> >>> >> new
> >>>> >>> >> > > > > >> features.
> >>>> >>> >> > > > > >> > > This
> >>>> >>> >> > > > > >> > > > > was
> >>>> >>> >> > > > > >> > > > > >> > > > spurred
> >>>> >>> >> > > > > >> > > > > >> > > > > >> by a
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > some
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > features
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > getting
> in
> >>>> >>> >> half-baked
> >>>> >>> >> > and
> >>>> >>> >> > > > > never
> >>>> >>> >> > > > > >> > > making
> >>>> >>> >> > > > > >> > > > > it
> >>>> >>> >> > > > > >> > > > > >> to
> >>>> >>> >> > > > > >> > > > > >> > > "can
> >>>> >>> >> > > > > >> > > > > >> > > > > >> really
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > use"
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > status
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > (I'm
> >>>> thinking of
> >>>> >>> >> > > > distributed
> >>>> >>> >> > > > > >> log
> >>>> >>> >> > > > > >> > > > replay
> >>>> >>> >> > > > > >> > > > > and
> >>>> >>> >> > > > > >> > > > > >> > the
> >>>> >>> >> > > > > >> > > > > >> > > > > >> zk-less
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > assignment
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > stuff, I
> >>>> don't
> >>>> >>> >> recall
> >>>> >>> >> > if
> >>>> >>> >> > > > > there
> >>>> >>> >> > > > > >> was
> >>>> >>> >> > > > > >> > > > > more).
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The
> gates,
> >>>> >>> >> generally,
> >>>> >>> >> > > > > included
> >>>> >>> >> > > > > >> > > things
> >>>> >>> >> > > > > >> > > > > like:
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have
> docs
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have
> >>>> sunny-day
> >>>> >>> >> > > > correctness
> >>>> >>> >> > > > > >> tests
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have
> >>>> >>> >> > > > > >> > > correctness-in-face-of-failure
> >>>> >>> >> > > > > >> > > > > tests
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't
> >>>> rely on
> >>>> >>> >> things
> >>>> >>> >> > > > > outside
> >>>> >>> >> > > > > >> of
> >>>> >>> >> > > > > >> > > > HBase
> >>>> >>> >> > > > > >> > > > > for
> >>>> >>> >> > > > > >> > > > > >> > > > normal
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > operation
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > (okay
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > for
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > advanced
> >>>> >>> operation)
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > As an
> >>>> example,
> >>>> >>> we
> >>>> >>> >> kept
> >>>> >>> >> > > the
> >>>> >>> >> > > > > MOB
> >>>> >>> >> > > > > >> > work
> >>>> >>> >> > > > > >> > > > off
> >>>> >>> >> > > > > >> > > > > in
> >>>> >>> >> > > > > >> > > > > >> a
> >>>> >>> >> > > > > >> > > > > >> > > > branch
> >>>> >>> >> > > > > >> > > > > >> > > > > >> and
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > out
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> of
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > master
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > until it
> >>>> could
> >>>> >>> pass
> >>>> >>> >> > these
> >>>> >>> >> > > > > >> > criteria.
> >>>> >>> >> > > > > >> > > > The
> >>>> >>> >> > > > > >> > > > > big
> >>>> >>> >> > > > > >> > > > > >> > > > > exemption
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > we've
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> had
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > to
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > this was
> the
> >>>> >>> >> > hbase-spark
> >>>> >>> >> > > > > >> > > integration,
> >>>> >>> >> > > > > >> > > > > where
> >>>> >>> >> > > > > >> > > > > >> > we
> >>>> >>> >> > > > > >> > > > > >> > > > all
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > agreed
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > it
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > could
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > land in
> >>>> master
> >>>> >>> >> because
> >>>> >>> >> > it
> >>>> >>> >> > > > was
> >>>> >>> >> > > > > >> very
> >>>> >>> >> > > > > >> > > > well
> >>>> >>> >> > > > > >> > > > > >> > > isolated
> >>>> >>> >> > > > > >> > > > > >> > > > > (the
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > slide
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > away
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > from
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > including
> >>>> docs
> >>>> >>> as a
> >>>> >>> >> > > > > first-class
> >>>> >>> >> > > > > >> > part
> >>>> >>> >> > > > > >> > > > of
> >>>> >>> >> > > > > >> > > > > >> > > building
> >>>> >>> >> > > > > >> > > > > >> > > > up
> >>>> >>> >> > > > > >> > > > > >> > > > > >> that
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > integration
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > has led
> me
> >>>> to
> >>>> >>> doubt
> >>>> >>> >> the
> >>>> >>> >> > > > > wisdom
> >>>> >>> >> > > > > >> of
> >>>> >>> >> > > > > >> > > this
> >>>> >>> >> > > > > >> > > > > >> > > decision).
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > We've
> also
> >>>> been
> >>>> >>> >> > treating
> >>>> >>> >> > > > > >> inclusion
> >>>> >>> >> > > > > >> > > in
> >>>> >>> >> > > > > >> > > > a
> >>>> >>> >> > > > > >> > > > > >> > > "probably
> >>>> >>> >> > > > > >> > > > > >> > > > > >> will
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > be
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > released
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > to
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> downstream"
> >>>> >>> branches
> >>>> >>> >> > as a
> >>>> >>> >> > > > > >> higher
> >>>> >>> >> > > > > >> > > bar,
> >>>> >>> >> > > > > >> > > > > >> > requiring
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't
> >>>> >>> moderately
> >>>> >>> >> > impact
> >>>> >>> >> > > > > >> > > performance
> >>>> >>> >> > > > > >> > > > > when
> >>>> >>> >> > > > > >> > > > > >> > the
> >>>> >>> >> > > > > >> > > > > >> > > > > >> feature
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > isn't
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> in
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > use
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't
> >>>> severely
> >>>> >>> >> impact
> >>>> >>> >> > > > > >> > performance
> >>>> >>> >> > > > > >> > > > when
> >>>> >>> >> > > > > >> > > > > >> the
> >>>> >>> >> > > > > >> > > > > >> > > > > feature
> >>>> >>> >> > > > > >> > > > > >> > > > > >> is
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > in
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> use
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * either
> >>>> >>> >> default-to-on
> >>>> >>> >> > or
> >>>> >>> >> > > > > show
> >>>> >>> >> > > > > >> > > enough
> >>>> >>> >> > > > > >> > > > > >> demand
> >>>> >>> >> > > > > >> > > > > >> > to
> >>>> >>> >> > > > > >> > > > > >> > > > > >> believe
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > a
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > non-trivial
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > number of
> >>>> folks
> >>>> >>> will
> >>>> >>> >> > turn
> >>>> >>> >> > > > the
> >>>> >>> >> > > > > >> > > feature
> >>>> >>> >> > > > > >> > > > on
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The above
> >>>> has
> >>>> >>> kept
> >>>> >>> >> MOB
> >>>> >>> >> > > and
> >>>> >>> >> > > > > >> > > hbase-spark
> >>>> >>> >> > > > > >> > > > > >> > > > integration
> >>>> >>> >> > > > > >> > > > > >> > > > > >> out
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > of
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > branch-1,
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> presumably
> >>>> while
> >>>> >>> >> > they've
> >>>> >>> >> > > > > >> "gotten
> >>>> >>> >> > > > > >> > > more
> >>>> >>> >> > > > > >> > > > > >> stable"
> >>>> >>> >> > > > > >> > > > > >> > > in
> >>>> >>> >> > > > > >> > > > > >> > > > > >> master
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > from
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > the
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > odd
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > vendor
> >>>> >>> inclusion.
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Are we
> >>>> going to
> >>>> >>> >> have a
> >>>> >>> >> > > 2.0
> >>>> >>> >> > > > > >> release
> >>>> >>> >> > > > > >> > > > > before
> >>>> >>> >> > > > > >> > > > > >> the
> >>>> >>> >> > > > > >> > > > > >> > > end
> >>>> >>> >> > > > > >> > > > > >> > > > > of
> >>>> >>> >> > > > > >> > > > > >> > > > > >> the
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> year?
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > We're
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > coming up
> >>>> on 1.5
> >>>> >>> >> years
> >>>> >>> >> > > > since
> >>>> >>> >> > > > > >> the
> >>>> >>> >> > > > > >> > > > > release of
> >>>> >>> >> > > > > >> > > > > >> > > > version
> >>>> >>> >> > > > > >> > > > > >> > > > > >> 1.0;
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> seems
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > like
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > it's
> about
> >>>> time,
> >>>> >>> >> > though I
> >>>> >>> >> > > > > >> haven't
> >>>> >>> >> > > > > >> > > seen
> >>>> >>> >> > > > > >> > > > > any
> >>>> >>> >> > > > > >> > > > > >> > > > concrete
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > plans
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> this
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > year.
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Presuming
> >>>> we are
> >>>> >>> >> going
> >>>> >>> >> > to
> >>>> >>> >> > > > > have
> >>>> >>> >> > > > > >> one
> >>>> >>> >> > > > > >> > > by
> >>>> >>> >> > > > > >> > > > > the
> >>>> >>> >> > > > > >> > > > > >> end
> >>>> >>> >> > > > > >> > > > > >> > > of
> >>>> >>> >> > > > > >> > > > > >> > > > > the
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > year, it
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > seems a
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > bit
> close to
> >>>> >>> still
> >>>> >>> >> be
> >>>> >>> >> > > > adding
> >>>> >>> >> > > > > in
> >>>> >>> >> > > > > >> > > > > "features
> >>>> >>> >> > > > > >> > > > > >> > that
> >>>> >>> >> > > > > >> > > > > >> > > > need
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > maturing"
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > on
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > the
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > branch.
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The lack
> of
> >>>> a
> >>>> >>> >> concrete
> >>>> >>> >> > > plan
> >>>> >>> >> > > > > for
> >>>> >>> >> > > > > >> > 2.0
> >>>> >>> >> > > > > >> > > > > keeps
> >>>> >>> >> > > > > >> > > > > >> me
> >>>> >>> >> > > > > >> > > > > >> > > from
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > considering
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > these
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > things
> >>>> blocker
> >>>> >>> at
> >>>> >>> >> the
> >>>> >>> >> > > > moment.
> >>>> >>> >> > > > > >> But
> >>>> >>> >> > > > > >> > I
> >>>> >>> >> > > > > >> > > > know
> >>>> >>> >> > > > > >> > > > > >> > first
> >>>> >>> >> > > > > >> > > > > >> > > > hand
> >>>> >>> >> > > > > >> > > > > >> > > > > >> how
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > much
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > trouble
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > folks
> have
> >>>> had
> >>>> >>> with
> >>>> >>> >> > other
> >>>> >>> >> > > > > >> features
> >>>> >>> >> > > > > >> > > > that
> >>>> >>> >> > > > > >> > > > > >> have
> >>>> >>> >> > > > > >> > > > > >> > > gone
> >>>> >>> >> > > > > >> > > > > >> > > > > >> into
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > downstream
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > facing
> >>>> releases
> >>>> >>> >> without
> >>>> >>> >> > > > > >> robustness
> >>>> >>> >> > > > > >> > > > > checks
> >>>> >>> >> > > > > >> > > > > >> > (i.e.
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > replication),
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > and
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > I'm
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > concerned
> >>>> about
> >>>> >>> what
> >>>> >>> >> > > we're
> >>>> >>> >> > > > > >> setting
> >>>> >>> >> > > > > >> > > up
> >>>> >>> >> > > > > >> > > > if
> >>>> >>> >> > > > > >> > > > > >> 2.0
> >>>> >>> >> > > > > >> > > > > >> > > goes
> >>>> >>> >> > > > > >> > > > > >> > > > > out
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > with
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> this
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > feature
> in
> >>>> its
> >>>> >>> >> current
> >>>> >>> >> > > > state.
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > --
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > Best regards,
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >    - Andy
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > Problems
> worthy of
> >>>> >>> attack
> >>>> >>> >> > prove
> >>>> >>> >> > > > > their
> >>>> >>> >> > > > > >> > > worth
> >>>> >>> >> > > > > >> > > > by
> >>>> >>> >> > > > > >> > > > > >> > > hitting
> >>>> >>> >> > > > > >> > > > > >> > > > > >> back. -
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > Piet
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > Hein
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > (via Tom White)
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > --
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > Best regards,
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >    - Andy
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > Problems worthy of
> >>>> attack
> >>>> >>> >> prove
> >>>> >>> >> > > their
> >>>> >>> >> > > > > >> worth
> >>>> >>> >> > > > > >> > by
> >>>> >>> >> > > > > >> > > > > >> hitting
> >>>> >>> >> > > > > >> > > > > >> > > > back.
> >>>> >>> >> > > > > >> > > > > >> > > > > -
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > Piet
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> Hein
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > (via Tom White)
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > > >>
> >>>> >>> >> > > > > >> > > > > >> > > > > >> > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >> >
> >>>> >>> >> > > > > >> > > > > >> > > > > >>
> >>>> >>> >> > > > > >> > > > > >> > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > > >
> >>>> >>> >> > > > > >> > > > > >> > > > >
> >>>> >>> >> > > > > >> > > > > >> > > >
> >>>> >>> >> > > > > >> > > > > >> > >
> >>>> >>> >> > > > > >> > > > > >> >
> >>>> >>> >> > > > > >> > > > > >>
> >>>> >>> >> > > > > >> > > > >
> >>>> >>> >> > > > > >> > > > >
> >>>> >>> >> > > > > >> > > > >
> >>>> >>> >> > > > > >> > > > > --
> >>>> >>> >> > > > > >> > > > > busbey
> >>>> >>> >> > > > > >> > > > >
> >>>> >>> >> > > > > >> > > >
> >>>> >>> >> > > > > >> > >
> >>>> >>> >> > > > > >> >
> >>>> >>> >> > > > > >>
> >>>> >>> >> > > > > >
> >>>> >>> >> > > > > >
> >>>> >>> >> > > > >
> >>>> >>> >> > > >
> >>>> >>> >> > >
> >>>> >>> >> >
> >>>> >>> >>
> >>>> >>> >>
> >>>> >>> >>
> >>>> >>> >> --
> >>>> >>> >>
> >>>> >>> >> -- Appy
> >>>> >>> >>
> >>>> >>> >
> >>>> >>> >
> >>>> >>>
> >>>> >>
> >>>> >>
> >>>> >
> >>>>
> >>>
> >>>
> >>
>
>
>
> --
> busbey
>
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Sean Busbey <bu...@cloudera.com>]

Are we requiring maven 3.3.z now? historically our required maven
version has been 3.0.5. I don't have an objection to changing it, per
se, but I'd like to make sure our docs and CI builds get updated
properly.

On Wed, Oct 12, 2016 at 4:03 PM, Stack <st...@duboce.net> wrote:
> On Wed, Oct 12, 2016 at 1:58 PM, Stack <st...@duboce.net> wrote:
>
>> Fails again when I do this:
>>
>> $ mvn clean install -DskipTests assembly:single
>> ....
>>
>>
> Ok. Works if I use mvn 3.3.x.
> St.Ack
>
>
>
>
>
>>
>> [INFO] ------------------------------------------------------------
>> ------------
>> [INFO] Building Apache HBase - Server 2.0.0-SNAPSHOT
>> [INFO] ------------------------------------------------------------
>> ------------
>> Downloading: file:/home/stack/hbase.git/hbase-server/src/main/site/
>> resources/repo/org/apache/hadoop/hadoop-distcp/2.7.1/
>> hadoop-distcp-2.7.1.pom
>> Exception in thread "pool-1-thread-1" ------------------------------
>> ---------------------
>> constituent[0]: file:/usr/share/maven/lib/aether-spi.jar
>> constituent[1]: file:/usr/share/maven/lib/maven-settings-builder-3.x.jar
>> constituent[2]: file:/usr/share/maven/lib/maven-artifact-3.x.jar
>> constituent[3]: file:/usr/share/maven/lib/commons-httpclient.jar
>> constituent[4]: file:/usr/share/maven/lib/wagon-provider-api.jar
>> constituent[5]: file:/usr/share/maven/lib/maven-core-3.x.jar
>> constituent[6]: file:/usr/share/maven/lib/aether-impl.jar
>> constituent[7]: file:/usr/share/maven/lib/sisu-inject-bean.jar
>> constituent[8]: file:/usr/share/maven/lib/sisu-inject-plexus.jar
>> constituent[9]: file:/usr/share/maven/lib/wagon-file.jar
>> constituent[10]: file:/usr/share/maven/lib/guava.jar
>> constituent[11]: file:/usr/share/maven/lib/maven-compat-3.x.jar
>> constituent[12]: file:/usr/share/maven/lib/maven-model-3.x.jar
>> constituent[13]: file:/usr/share/maven/lib/plexus-interpolation.jar
>> constituent[14]: file:/usr/share/maven/lib/commons-codec.jar
>> constituent[15]: file:/usr/share/maven/lib/aether-util.jar
>> constituent[16]: file:/usr/share/maven/lib/aether-api.jar
>> constituent[17]: file:/usr/share/maven/lib/maven-model-builder-3.x.jar
>> constituent[18]: file:/usr/share/maven/lib/maven-repository-metadata-3.x.
>> jar
>> constituent[19]: file:/usr/share/maven/lib/maven-settings-3.x.jar
>> constituent[20]: file:/usr/share/maven/lib/plexus-component-annotations.
>> jar
>> constituent[21]: file:/usr/share/maven/lib/commons-cli.jar
>> constituent[22]: file:/usr/share/maven/lib/maven-embedder-3.x.jar
>> constituent[23]: file:/usr/share/maven/lib/maven-plugin-api-3.x.jar
>> constituent[24]: file:/usr/share/maven/lib/wagon-http-shaded.jar
>> constituent[25]: file:/usr/share/maven/lib/commons-logging.jar
>> constituent[26]: file:/usr/share/maven/lib/plexus-sec-dispatcher.jar
>> constituent[27]: file:/usr/share/maven/lib/aether-connector-wagon.jar
>> constituent[28]: file:/usr/share/maven/lib/plexus-utils.jar
>> constituent[29]: file:/usr/share/maven/lib/sisu-guice.jar
>> constituent[30]: file:/usr/share/maven/lib/plexus-cipher.jar
>> constituent[31]: file:/usr/share/maven/lib/maven-aether-provider-3.x.jar
>> ---------------------------------------------------
>> java.lang.NoClassDefFoundError: org/apache/commons/lang/StringUtils
>> at org.apache.maven.wagon.providers.file.FileWagon.resolveDestinationPath(
>> FileWagon.java:206)
>> at org.apache.maven.wagon.providers.file.FileWagon.
>> resourceExists(FileWagon.java:265)
>> at org.sonatype.aether.connector.wagon.WagonRepositoryConnector$
>> GetTask.run(WagonRepositoryConnector.java:577)
>> at org.sonatype.aether.util.concurrency.RunnableErrorForwarder$1.run(
>> RunnableErrorForwarder.java:60)
>> at java.util.concurrent.ThreadPoolExecutor.runWorker(
>> ThreadPoolExecutor.java:1142)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
>> ThreadPoolExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:745)
>> Caused by: java.lang.ClassNotFoundException: org.apache.commons.lang.
>> StringUtils
>> at org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy.loadClass(
>> SelfFirstStrategy.java:50)
>> at org.codehaus.plexus.classworlds.realm.ClassRealm.
>> unsynchronizedLoadClass(ClassRealm.java:259)
>> at org.codehaus.plexus.classworlds.realm.ClassRealm.
>> loadClass(ClassRealm.java:235)
>> at org.codehaus.plexus.classworlds.realm.ClassRealm.
>> loadClass(ClassRealm.java:227)
>> ... 7 more
>> Exception in thread "main" java.lang.NoClassDefFoundError:
>> org/apache/commons/lang/StringUtils
>> at org.apache.maven.wagon.providers.file.FileWagon.resolveDestinationPath(
>> FileWagon.java:206)
>> at org.apache.maven.wagon.providers.file.FileWagon.
>> resourceExists(FileWagon.java:265)
>> at org.sonatype.aether.connector.wagon.WagonRepositoryConnector$
>> GetTask.run(WagonRepositoryConnector.java:577)
>> at org.sonatype.aether.util.concurrency.RunnableErrorForwarder$1.run(
>> RunnableErrorForwarder.java:60)
>> at java.util.concurrent.ThreadPoolExecutor.runWorker(
>> ThreadPoolExecutor.java:1142)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
>> ThreadPoolExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:745)
>> Caused by: java.lang.ClassNotFoundException: org.apache.commons.lang.
>> StringUtils
>> at org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy.loadClass(
>> SelfFirstStrategy.java:50)
>> at org.codehaus.plexus.classworlds.realm.ClassRealm.
>> unsynchronizedLoadClass(ClassRealm.java:259)
>> at org.codehaus.plexus.classworlds.realm.ClassRealm.
>> loadClass(ClassRealm.java:235)
>> at org.codehaus.plexus.classworlds.realm.ClassRealm.
>> loadClass(ClassRealm.java:227)
>>
>>
>>
>> Does it work for you lot? This is branch checkout.
>>
>> I want assembly to work so I can distribute the build around a cluster.
>>
>> Thanks,
>> St.Ack.
>>
>>
>>
>> On Wed, Oct 12, 2016 at 1:52 PM, Stack <st...@duboce.net> wrote:
>>
>>> I did a super-clean and recheck out. Now it works. Sorry for the noise.
>>> St.Ack
>>>
>>> On Wed, Oct 12, 2016 at 1:10 PM, Vladimir Rodionov <
>>> vladrodionov@gmail.com> wrote:
>>>
>>>> checked out HBASE-7912
>>>>
>>>> ran:
>>>>
>>>> mvn clean install -DskipTests
>>>>
>>>> successfully.
>>>>
>>>> -Vlad
>>>>
>>>> On Wed, Oct 12, 2016 at 1:01 PM, Vladimir Rodionov <
>>>> vladrodionov@gmail.com>
>>>> wrote:
>>>>
>>>> > I usually use:
>>>> >
>>>> > mvn clean install -DskipTests
>>>> >
>>>> > and it works.
>>>> >
>>>> > On Wed, Oct 12, 2016 at 1:01 PM, Vladimir Rodionov <
>>>> vladrodionov@gmail.com
>>>> > > wrote:
>>>> >
>>>> >> Michael,
>>>> >>
>>>> >> you can try master + latest patch on HBASE-14123 (v29). No need to use
>>>> >> HBASE-7912 branch. I will double check HBASE-7912.
>>>> >>
>>>> >> -Vlad
>>>> >>
>>>> >> On Wed, Oct 12, 2016 at 12:31 PM, Stack <st...@duboce.net> wrote:
>>>> >>
>>>> >>> More info:
>>>> >>>
>>>> >>> stack@ve0524:~/hbase.git$ git checkout origin/HBASE-7912 -b 7912v2
>>>> >>> Branch 7912v2 set up to track remote branch HBASE-7912 from origin.
>>>> >>> Switched to a new branch '7912v2'
>>>> >>> stack@ve0524:~/hbase.git$ java -version
>>>> >>> java version "1.8.0_101"
>>>> >>> Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
>>>> >>> Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode)
>>>> >>> stack@ve0524:~/hbase.git$ mvn clean install -DskipTests &>
>>>> /tmp/out.txt
>>>> >>>
>>>> >>> ...
>>>> >>>
>>>> >>> St.Ack
>>>> >>>
>>>> >>>
>>>> >>> On Wed, Oct 12, 2016 at 12:29 PM, Stack <st...@duboce.net> wrote:
>>>> >>>
>>>> >>> > Interesting. When I try it fails w/ below:
>>>> >>> >
>>>> >>> > [INFO] 26 warnings
>>>> >>> > 322 [INFO] ------------------------------
>>>> >>> -------------------------------
>>>> >>> > 323 [INFO] ------------------------------
>>>> >>> -------------------------------
>>>> >>> > 324 [ERROR] COMPILATION ERROR :
>>>> >>> > 325 [INFO] ------------------------------
>>>> >>> -------------------------------
>>>> >>> > 326 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.j
>>>> ava:[48,8]
>>>> >>> > org.apache.hadoop.hbase.io.encoding.rowindexV2.RowIndexCodecV2 is
>>>> not
>>>> >>> > abstract and does not override abstract method createSeeker(org.ap
>>>> >>> >  ache.hadoop.hbase.CellComparator,org.apache.hadoop.hbase.io.
>>>> >>> encoding.HFileBlockDecodingContext)
>>>> >>> > in org.apache.hadoop.hbase.io.encoding.DataBlockEncoder
>>>> >>> > 327 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.j
>>>> >>> ava:[143,3]
>>>> >>> > method does not override or implement a method from a supertype
>>>> >>> > 328 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.j
>>>> >>> ava:[147,29]
>>>> >>> > incompatible types: java.nio.ByteBuffer cannot be converted to
>>>> >>> > org.apache.hadoop.hbase.nio.ByteBuff
>>>> >>> > 329 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.j
>>>> >>> ava:[148,33]
>>>> >>> > cannot find symbol
>>>> >>> > 330   symbol:   method getKeyDeepCopy()
>>>> >>> > 331   location: variable seeker of type org.apache.hadoop.hbase.io
>>>> .
>>>> >>> > encoding.DataBlockEncoder.EncodedSeeker
>>>> >>> > 332 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.j
>>>> >>> ava:[153,3]
>>>> >>> > method does not override or implement a method from a supertype
>>>> >>> > 333 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV1/RowIndexCodecV1.j
>>>> ava:[45,8]
>>>> >>> > org.apache.hadoop.hbase.io.encoding.rowindexV1.RowIndexCodecV1 is
>>>> not
>>>> >>> > abstract and does not override abstract method createSeeker(org.ap
>>>> >>> >  ache.hadoop.hbase.CellComparator,org.apache.hadoop.hbase.io.
>>>> >>> encoding.HFileBlockDecodingContext)
>>>> >>> > in org.apache.hadoop.hbase.io.encoding.DataBlockEncoder
>>>> >>> > 334 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV1/RowIndexCodecV1.j
>>>> >>> ava:[145,3]
>>>> >>> > method does not override or implement a method from a supertype
>>>> >>> > 335 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV1/RowIndexCodecV1.j
>>>> >>> ava:[158,3]
>>>> >>> > method does not override or implement a method from a supertype
>>>> >>> > 336 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>>>> >>> java:[46,8]
>>>> >>> > org.apache.hadoop.hbase.io.encoding.rowindexV2.RowIndexSeekerV2
>>>> is not
>>>> >>> > abstract and does not override abstract method compareKey(org.ap
>>>> >>> >  ache.hadoop.hbase.CellComparator,org.apache.hadoop.hbase.Cell) in
>>>> >>> > org.apache.hadoop.hbase.io.encoding.DataBlockEncoder.EncodedSeeker
>>>> >>> > 337 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>>>> >>> java:[79,3]
>>>> >>> > method does not override or implement a method from a supertype
>>>> >>> > 338 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>>>> >>> java:[117,3]
>>>> >>> > method does not override or implement a method from a supertype
>>>> >>> > 339 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>>>> >>> java:[190,3]
>>>> >>> > method does not override or implement a method from a supertype
>>>> >>> > 340 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>>>> >>> java:[214,3]
>>>> >>> > method does not override or implement a method from a supertype
>>>> >>> > 341 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>>>> >>> java:[349,3]
>>>> >>> > method does not override or implement a method from a supertype
>>>> >>> > 342 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>>>> >>> java:[355,3]
>>>> >>> > method does not override or implement a method from a supertype
>>>> >>> > 343 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>>>> >>> java:[421,36]
>>>> >>> > no suitable method found for uncompressTags(java.nio.
>>>> >>> > ByteBuffer,byte[],int,int)
>>>> >>> > 344     method org.apache.hadoop.hbase.io.TagCompressionContext.
>>>> >>> > uncompressTags(java.io.InputStream,byte[],int,int) is not
>>>> applicable
>>>> >>> > 345       (argument mismatch; java.nio.ByteBuffer cannot be
>>>> converted
>>>> >>> to
>>>> >>> > java.io.InputStream)
>>>> >>> > 346     method org.apache.hadoop.hbase.io.TagCompressionContext.
>>>> >>> > uncompressTags(org.apache.hadoop.hbase.nio.ByteBuff,byte[],int,int)
>>>> is
>>>> >>> > not applicable
>>>> >>> > 347       (argument mismatch; java.nio.ByteBuffer cannot be
>>>> converted
>>>> >>> to
>>>> >>> > org.apache.hadoop.hbase.nio.ByteBuff)
>>>> >>> >
>>>> >>> > ....
>>>> >>> >
>>>> >>> > St.Ack
>>>> >>> >
>>>> >>> > On Tue, Oct 11, 2016 at 5:12 PM, Apekshit Sharma <
>>>> appy@cloudera.com>
>>>> >>> > wrote:
>>>> >>> >
>>>> >>> >> @stack, it compiled for me.
>>>> >>> >>
>>>> >>> >> Also tried few commands, and have to say, it's well designed from
>>>> user
>>>> >>> >> commands perspective.
>>>> >>> >>
>>>> >>> >> On Tue, Oct 11, 2016 at 5:08 PM, Stack <st...@duboce.net> wrote:
>>>> >>> >>
>>>> >>> >> > On Tue, Oct 11, 2016 at 1:50 PM, Vladimir Rodionov <
>>>> >>> >> vladrodionov@gmail.com
>>>> >>> >> > >
>>>> >>> >> > wrote:
>>>> >>> >> >
>>>> >>> >> > > Michael,
>>>> >>> >> > >
>>>> >>> >> > > Its in HBASE-7912
>>>> >>> >> > >
>>>> >>> >> > > This is tip of git log:
>>>> >>> >> > >
>>>> >>> >> > > commit a072f6f49a26a7259ff2aaef6cb56d85eb592482
>>>> >>> >> > > Author: Frank Welsch <fw...@jps.net>
>>>> >>> >> > > Date:   Fri Sep 23 18:00:42 2016 -0400
>>>> >>> >> > >
>>>> >>> >> > >     HBASE-16574 Book updates for backup and restore
>>>> >>> >> > >
>>>> >>> >> > > commit b14e2ab1c24e65ff88dd4c579acf83cb4ed0605e
>>>> >>> >> > > Author: tedyu <yu...@gmail.com>
>>>> >>> >> > > Date:   Wed Oct 5 16:29:40 2016 -0700
>>>> >>> >> > >
>>>> >>> >> > >     HBASE-16727 Backup refactoring: remove MR dependencies
>>>> from
>>>> >>> >> HMaster
>>>> >>> >> > > (Vladimir Rodionov)
>>>> >>> >> > >
>>>> >>> >> > >
>>>> >>> >> > Thanks. I have that. I tried it and it doesn't compile for me.
>>>> Does
>>>> >>> it
>>>> >>> >> > compile for you?
>>>> >>> >> > Thanks,
>>>> >>> >> > M
>>>> >>> >> >
>>>> >>> >> >
>>>> >>> >> >
>>>> >>> >> > > -Vlad
>>>> >>> >> > >
>>>> >>> >> > > On Tue, Oct 11, 2016 at 1:44 PM, Stack <st...@duboce.net>
>>>> wrote:
>>>> >>> >> > >
>>>> >>> >> > > > Which branch do I check out to try it? HBASE-7912 is not
>>>> it. I
>>>> >>> don't
>>>> >>> >> > see
>>>> >>> >> > > an
>>>> >>> >> > > > HBASE-16727...
>>>> >>> >> > > > Thanks,
>>>> >>> >> > > > M
>>>> >>> >> > > >
>>>> >>> >> > > >
>>>> >>> >> > > > On Tue, Oct 11, 2016 at 11:41 AM, Vladimir Rodionov <
>>>> >>> >> > > > vladrodionov@gmail.com>
>>>> >>> >> > > > wrote:
>>>> >>> >> > > >
>>>> >>> >> > > > > The last patch is on review board:
>>>> >>> >> > > > > https://reviews.apache.org/r/52748
>>>> >>> >> > > > >
>>>> >>> >> > > > > On Mon, Oct 10, 2016 at 3:52 PM, Vladimir Rodionov <
>>>> >>> >> > > > vladrodionov@gmail.com
>>>> >>> >> > > > > >
>>>> >>> >> > > > > wrote:
>>>> >>> >> > > > >
>>>> >>> >> > > > > > >> How hard to put in an hbase-backup module?
>>>> hbase-server
>>>> >>> is
>>>> >>> >> fat
>>>> >>> >> > > > enough
>>>> >>> >> > > > > > >> already. Could be done as a follow-up.
>>>> >>> >> > > > > >
>>>> >>> >> > > > > > https://issues.apache.org/jira/browse/HBASE-16727?
>>>> >>> >> > > > > > focusedCommentId=15531237&page=com.atlassian.jira.
>>>> >>> >> > > > > > plugin.system.issuetabpanels:c
>>>> >>> omment-tabpanel#comment-15531237
>>>> >>> >> > > > > >
>>>> >>> >> > > > > > Can we do merge first? Then we can discuss separate
>>>> module.
>>>> >>> >> > > > > >
>>>> >>> >> > > > > > -Vlad
>>>> >>> >> > > > > >
>>>> >>> >> > > > > > On Mon, Oct 10, 2016 at 3:44 PM, Ted Yu <
>>>> >>> yuzhihong@gmail.com>
>>>> >>> >> > wrote:
>>>> >>> >> > > > > >
>>>> >>> >> > > > > >> Looks like the first quote was cut off.
>>>> >>> >> > > > > >> The original sentence was:
>>>> >>> >> > > > > >>
>>>> >>> >> > > > > >> bq. no mapreduce job launched from master or region
>>>> server.
>>>> >>> >> > > > > >>
>>>> >>> >> > > > > >> mapreduce job is launched from the node where command
>>>> line
>>>> >>> >> tool is
>>>> >>> >> > > > run.
>>>> >>> >> > > > > >>
>>>> >>> >> > > > > >> On Mon, Oct 10, 2016 at 3:38 PM, Stack <
>>>> stack@duboce.net>
>>>> >>> >> wrote:
>>>> >>> >> > > > > >>
>>>> >>> >> > > > > >> > bq. launched from master or region server.
>>>> >>> >> > > > > >> >
>>>> >>> >> > > > > >> > What does this mean please? Has to be run from
>>>> Master or
>>>> >>> >> > > > RegionServer?
>>>> >>> >> > > > > >> Can
>>>> >>> >> > > > > >> > it be run from another node altogether?
>>>> >>> >> > > > > >> >
>>>> >>> >> > > > > >> > On Mon, Oct 10, 2016 at 1:44 PM, Vladimir Rodionov <
>>>> >>> >> > > > > >> vladrodionov@gmail.com
>>>> >>> >> > > > > >> > >
>>>> >>> >> > > > > >> > wrote:
>>>> >>> >> > > > > >> >
>>>> >>> >> > > > > >> > > >> mapreduce dependency has been moved to client
>>>> side
>>>> >>> - no
>>>> >>> >> > > > mapreduce
>>>> >>> >> > > > > >> job
>>>> >>> >> > > > > >> > >
>>>> >>> >> > > > > >> > > 1. We have no code in the client module anymore,
>>>> due to
>>>> >>> >> > > dependency
>>>> >>> >> > > > > on
>>>> >>> >> > > > > >> > > internal server API (HFile and WAL access).
>>>> >>> >> > > > > >> > > 2. Backup/ restore are client - driven operations,
>>>> but
>>>> >>> all
>>>> >>> >> the
>>>> >>> >> > > > code
>>>> >>> >> > > > > >> > resides
>>>> >>> >> > > > > >> > > in the server module
>>>> >>> >> > > > > >> > >
>>>> >>> >> > > > > >> >
>>>> >>> >> > > > > >> > How hard to put in an hbase-backup module?
>>>> hbase-server
>>>> >>> is
>>>> >>> >> fat
>>>> >>> >> > > > enough
>>>> >>> >> > > > > >> > already. Could be done as a follow-up.
>>>> >>> >> > > > > >> >
>>>> >>> >> > > > > >> > Thanks,
>>>> >>> >> > > > > >> > St.Ack
>>>> >>> >> > > > > >> >
>>>> >>> >> > > > > >> >
>>>> >>> >> > > > > >> >
>>>> >>> >> > > > > >> > > 3. No MR in Master, no procedure - driven
>>>> execution.
>>>> >>> >> > > > > >> > > 4. Old good MR from command-line.
>>>> >>> >> > > > > >> > > 5. Security was simplified and now only super-user
>>>> is
>>>> >>> >> allowed
>>>> >>> >> > to
>>>> >>> >> > > > run
>>>> >>> >> > > > > >> > > backup/restores.
>>>> >>> >> > > > > >> > > 6. HBase Backup API was gone due to 1. Now only
>>>> >>> >> command-line
>>>> >>> >> > > > access
>>>> >>> >> > > > > to
>>>> >>> >> > > > > >> > > backup tools.
>>>> >>> >> > > > > >> > >
>>>> >>> >> > > > > >> > > These consequences of refactoring has been
>>>> discussed in
>>>> >>> >> > > > HBASE-16727.
>>>> >>> >> > > > > >> > >
>>>> >>> >> > > > > >> > > -Vlad
>>>> >>> >> > > > > >> > >
>>>> >>> >> > > > > >> > >
>>>> >>> >> > > > > >> > >
>>>> >>> >> > > > > >> > >
>>>> >>> >> > > > > >> > > On Mon, Oct 10, 2016 at 1:31 PM, Ted Yu <
>>>> >>> >> yuzhihong@gmail.com>
>>>> >>> >> > > > > wrote:
>>>> >>> >> > > > > >> > >
>>>> >>> >> > > > > >> > > > Reviving this thread.
>>>> >>> >> > > > > >> > > >
>>>> >>> >> > > > > >> > > > The following has taken place:
>>>> >>> >> > > > > >> > > >
>>>> >>> >> > > > > >> > > > mapreduce dependency has been moved to client
>>>> side -
>>>> >>> no
>>>> >>> >> > > > mapreduce
>>>> >>> >> > > > > >> job
>>>> >>> >> > > > > >> > > > launched from master or region server.
>>>> >>> >> > > > > >> > > > document patch (HBASE-16574) has been integrated.
>>>> >>> >> > > > > >> > > > Updated mega patch has been attached to
>>>> HBASE-14123:
>>>> >>> this
>>>> >>> >> > > covers
>>>> >>> >> > > > > the
>>>> >>> >> > > > > >> > > > refactor in #1 above and the protobuf 3 merge.
>>>> >>> >> > > > > >> > > >
>>>> >>> >> > > > > >> > > > If community has more feedback on the merge
>>>> >>> proposal, I
>>>> >>> >> > would
>>>> >>> >> > > > love
>>>> >>> >> > > > > >> to
>>>> >>> >> > > > > >> > > hear
>>>> >>> >> > > > > >> > > > it.
>>>> >>> >> > > > > >> > > >
>>>> >>> >> > > > > >> > > > Thanks
>>>> >>> >> > > > > >> > > >
>>>> >>> >> > > > > >> > > > On Thu, Sep 22, 2016 at 10:31 AM, Sean Busbey <
>>>> >>> >> > > > > busbey@cloudera.com>
>>>> >>> >> > > > > >> > > wrote:
>>>> >>> >> > > > > >> > > >
>>>> >>> >> > > > > >> > > > > I'd like to see the docs proposed on
>>>> HBASE-16574
>>>> >>> >> > integrated
>>>> >>> >> > > > into
>>>> >>> >> > > > > >> our
>>>> >>> >> > > > > >> > > > > project's documentation prior to merge.
>>>> >>> >> > > > > >> > > > >
>>>> >>> >> > > > > >> > > > > On Thu, Sep 22, 2016 at 9:02 AM, Ted Yu <
>>>> >>> >> > > yuzhihong@gmail.com>
>>>> >>> >> > > > > >> wrote:
>>>> >>> >> > > > > >> > > > > > This feature can be marked experimental due
>>>> to
>>>> >>> some
>>>> >>> >> > > > > limitations
>>>> >>> >> > > > > >> > such
>>>> >>> >> > > > > >> > > as
>>>> >>> >> > > > > >> > > > > > security.
>>>> >>> >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > > Your previous round of comments have been
>>>> >>> addressed.
>>>> >>> >> > > > > >> > > > > > Command line tool has gone through:
>>>> >>> >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > > HBASE-16620 Fix backup command-line tool
>>>> >>> usability
>>>> >>> >> > issues
>>>> >>> >> > > > > >> > > > > > HBASE-16655 hbase backup describe with
>>>> incorrect
>>>> >>> >> backup
>>>> >>> >> > id
>>>> >>> >> > > > > >> results
>>>> >>> >> > > > > >> > in
>>>> >>> >> > > > > >> > > > NPE
>>>> >>> >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > > The updated doc has been attached to
>>>> HBASE-16574.
>>>> >>> >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > > Cheers
>>>> >>> >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > > On Thu, Sep 22, 2016 at 8:53 AM, Stack <
>>>> >>> >> > stack@duboce.net>
>>>> >>> >> > > > > >> wrote:
>>>> >>> >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> On Wed, Sep 21, 2016 at 7:43 AM, Ted Yu <
>>>> >>> >> > > > yuzhihong@gmail.com
>>>> >>> >> > > > > >
>>>> >>> >> > > > > >> > > wrote:
>>>> >>> >> > > > > >> > > > > >>
>>>> >>> >> > > > > >> > > > > >> > Are there more (review) comments ?
>>>> >>> >> > > > > >> > > > > >> >
>>>> >>> >> > > > > >> > > > > >> >
>>>> >>> >> > > > > >> > > > > >> Are outstanding comments addressed?
>>>> >>> >> > > > > >> > > > > >>
>>>> >>> >> > > > > >> > > > > >> I don't see answer to my 'is this
>>>> >>> experimental/will
>>>> >>> >> it
>>>> >>> >> > be
>>>> >>> >> > > > > >> marked
>>>> >>> >> > > > > >> > > > > >> experimental' question.
>>>> >>> >> > > > > >> > > > > >>
>>>> >>> >> > > > > >> > > > > >> I ran into some issues trying to use the
>>>> >>> feature and
>>>> >>> >> > > > > suggested
>>>> >>> >> > > > > >> > that
>>>> >>> >> > > > > >> > > a
>>>> >>> >> > > > > >> > > > > >> feature likes this needs polish else it'll
>>>> just
>>>> >>> rot,
>>>> >>> >> > > > unused.
>>>> >>> >> > > > > >> Has
>>>> >>> >> > > > > >> > > > polish
>>>> >>> >> > > > > >> > > > > >> been applied? All ready for another 'user'
>>>> test?
>>>> >>> >> > Suggest
>>>> >>> >> > > > that
>>>> >>> >> > > > > >> you
>>>> >>> >> > > > > >> > > > update
>>>> >>> >> > > > > >> > > > > >> here going forward for the benefit of those
>>>> >>> trying
>>>> >>> >> to
>>>> >>> >> > > > follow
>>>> >>> >> > > > > >> along
>>>> >>> >> > > > > >> > > and
>>>> >>> >> > > > > >> > > > > who
>>>> >>> >> > > > > >> > > > > >> are not watching JIRA change fly-by.
>>>> >>> >> > > > > >> > > > > >>
>>>> >>> >> > > > > >> > > > > >> It looks like doc got a revision -- I have
>>>> to
>>>> >>> check
>>>> >>> >> --
>>>> >>> >> > to
>>>> >>> >> > > > > take
>>>> >>> >> > > > > >> on
>>>> >>> >> > > > > >> > > > > >> suggestion made above but again, suggest,
>>>> that
>>>> >>> this
>>>> >>> >> > > thread
>>>> >>> >> > > > > gets
>>>> >>> >> > > > > >> > > > updated.
>>>> >>> >> > > > > >> > > > > >>
>>>> >>> >> > > > > >> > > > > >> Thanks,
>>>> >>> >> > > > > >> > > > > >> St.Ack
>>>> >>> >> > > > > >> > > > > >>
>>>> >>> >> > > > > >> > > > > >>
>>>> >>> >> > > > > >> > > > > >>
>>>> >>> >> > > > > >> > > > > >> > Thanks
>>>> >>> >> > > > > >> > > > > >> >
>>>> >>> >> > > > > >> > > > > >> > On Tue, Sep 20, 2016 at 10:02 AM, Devaraj
>>>> Das
>>>> >>> <
>>>> >>> >> > > > > >> > > ddas@hortonworks.com
>>>> >>> >> > > > > >> > > > >
>>>> >>> >> > > > > >> > > > > >> > wrote:
>>>> >>> >> > > > > >> > > > > >> >
>>>> >>> >> > > > > >> > > > > >> > > Just reviving this thread. Thanks Sean,
>>>> >>> Stack,
>>>> >>> >> > Dima,
>>>> >>> >> > > > and
>>>> >>> >> > > > > >> > others
>>>> >>> >> > > > > >> > > > for
>>>> >>> >> > > > > >> > > > > the
>>>> >>> >> > > > > >> > > > > >> > > thorough reviews and testing. Thanks
>>>> Ted and
>>>> >>> >> Vlad
>>>> >>> >> > for
>>>> >>> >> > > > > >> taking
>>>> >>> >> > > > > >> > > care
>>>> >>> >> > > > > >> > > > of
>>>> >>> >> > > > > >> > > > > >> the
>>>> >>> >> > > > > >> > > > > >> > > feedback. Are we all good to do the
>>>> merge
>>>> >>> now?
>>>> >>> >> > Rather
>>>> >>> >> > > > do
>>>> >>> >> > > > > >> > sooner
>>>> >>> >> > > > > >> > > > than
>>>> >>> >> > > > > >> > > > > >> > later.
>>>> >>> >> > > > > >> > > > > >> > > ______________________________
>>>> __________
>>>> >>> >> > > > > >> > > > > >> > > From: saint.ack@gmail.com <
>>>> >>> saint.ack@gmail.com>
>>>> >>> >> on
>>>> >>> >> > > > > behalf
>>>> >>> >> > > > > >> of
>>>> >>> >> > > > > >> > > > Stack
>>>> >>> >> > > > > >> > > > > <
>>>> >>> >> > > > > >> > > > > >> > > stack@duboce.net>
>>>> >>> >> > > > > >> > > > > >> > > Sent: Monday, September 12, 2016 1:18 PM
>>>> >>> >> > > > > >> > > > > >> > > To: HBase Dev List
>>>> >>> >> > > > > >> > > > > >> > > Subject: Re: [DISCUSSION] Merge Backup /
>>>> >>> >> Restore -
>>>> >>> >> > > > Branch
>>>> >>> >> > > > > >> > > > HBASE-7912
>>>> >>> >> > > > > >> > > > > >> > >
>>>> >>> >> > > > > >> > > > > >> > > On Mon, Sep 12, 2016 at 12:19 PM, Ted
>>>> Yu <
>>>> >>> >> > > > > >> yuzhihong@gmail.com
>>>> >>> >> > > > > >> > >
>>>> >>> >> > > > > >> > > > > wrote:
>>>> >>> >> > > > > >> > > > > >> > >
>>>> >>> >> > > > > >> > > > > >> > > > Mega patch (rev 18) is on HBASE-14123.
>>>> >>> >> > > > > >> > > > > >> > > >
>>>> >>> >> > > > > >> > > > > >> > > > Please comment on HBASE-14123 on how
>>>> you
>>>> >>> want
>>>> >>> >> to
>>>> >>> >> > > > > review.
>>>> >>> >> > > > > >> > > > > >> > > >
>>>> >>> >> > > > > >> > > > > >> > >
>>>> >>> >> > > > > >> > > > > >> > >
>>>> >>> >> > > > > >> > > > > >> > > Yeah. That was my lost tab. Last rb was
>>>> 6
>>>> >>> months
>>>> >>> >> > ago.
>>>> >>> >> > > > > >> Suggest
>>>> >>> >> > > > > >> > > > > updating
>>>> >>> >> > > > > >> > > > > >> > it.
>>>> >>> >> > > > > >> > > > > >> > > RB is pretty good for review. Patch is
>>>> only
>>>> >>> >> 1.5M so
>>>> >>> >> > > > > should
>>>> >>> >> > > > > >> be
>>>> >>> >> > > > > >> > > > fine.
>>>> >>> >> > > > > >> > > > > >> > >
>>>> >>> >> > > > > >> > > > > >> > > St.Ack
>>>> >>> >> > > > > >> > > > > >> > >
>>>> >>> >> > > > > >> > > > > >> > >
>>>> >>> >> > > > > >> > > > > >> > > >
>>>> >>> >> > > > > >> > > > > >> > > > Thanks
>>>> >>> >> > > > > >> > > > > >> > > >
>>>> >>> >> > > > > >> > > > > >> > > > On Mon, Sep 12, 2016 at 12:15 PM,
>>>> Stack <
>>>> >>> >> > > > > >> stack@duboce.net>
>>>> >>> >> > > > > >> > > > wrote:
>>>> >>> >> > > > > >> > > > > >> > > >
>>>> >>> >> > > > > >> > > > > >> > > > > On review of the 'patch', do I just
>>>> >>> compare
>>>> >>> >> the
>>>> >>> >> > > > > branch
>>>> >>> >> > > > > >> to
>>>> >>> >> > > > > >> > > > > master or
>>>> >>> >> > > > > >> > > > > >> > is
>>>> >>> >> > > > > >> > > > > >> > > > > there a megapatch posted somewhere
>>>> (I
>>>> >>> think
>>>> >>> >> I
>>>> >>> >> > saw
>>>> >>> >> > > > one
>>>> >>> >> > > > > >> but
>>>> >>> >> > > > > >> > it
>>>> >>> >> > > > > >> > > > > seemed
>>>> >>> >> > > > > >> > > > > >> > > stale
>>>> >>> >> > > > > >> > > > > >> > > > > and then I 'lost' the tab). Sorry
>>>> for
>>>> >>> dumb
>>>> >>> >> > > > question.
>>>> >>> >> > > > > >> > > > > >> > > > > St.Ack
>>>> >>> >> > > > > >> > > > > >> > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > On Mon, Sep 12, 2016 at 12:01 PM,
>>>> Stack
>>>> >>> <
>>>> >>> >> > > > > >> stack@duboce.net
>>>> >>> >> > > > > >> > >
>>>> >>> >> > > > > >> > > > > wrote:
>>>> >>> >> > > > > >> > > > > >> > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > > Late to the game. A few comments
>>>> after
>>>> >>> >> > > rereading
>>>> >>> >> > > > > this
>>>> >>> >> > > > > >> > > thread
>>>> >>> >> > > > > >> > > > > as a
>>>> >>> >> > > > > >> > > > > >> > > > 'user'.
>>>> >>> >> > > > > >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > > + Before merge, a user-facing
>>>> feature
>>>> >>> like
>>>> >>> >> > this
>>>> >>> >> > > > > >> should
>>>> >>> >> > > > > >> > > work
>>>> >>> >> > > > > >> > > > > (If
>>>> >>> >> > > > > >> > > > > >> > this
>>>> >>> >> > > > > >> > > > > >> > > is
>>>> >>> >> > > > > >> > > > > >> > > > > "higher-bar
>>>> >>> >> > > > > >> > > > > >> > > > > > for new features", bring it on --
>>>> >>> smile).
>>>> >>> >> > > > > >> > > > > >> > > > > > + As a user, I tried the branch
>>>> with
>>>> >>> tools
>>>> >>> >> > > after
>>>> >>> >> > > > > >> > reviewing
>>>> >>> >> > > > > >> > > > the
>>>> >>> >> > > > > >> > > > > >> > > > > just-posted
>>>> >>> >> > > > > >> > > > > >> > > > > > doc. I had an 'interesting'
>>>> experience
>>>> >>> >> (left
>>>> >>> >> > > > > >> comments up
>>>> >>> >> > > > > >> > > on
>>>> >>> >> > > > > >> > > > > >> > issue). I
>>>> >>> >> > > > > >> > > > > >> > > > > think
>>>> >>> >> > > > > >> > > > > >> > > > > > the tooling/doc. important to get
>>>> >>> right.
>>>> >>> >> If
>>>> >>> >> > it
>>>> >>> >> > > > > breaks
>>>> >>> >> > > > > >> > > easily
>>>> >>> >> > > > > >> > > > > or
>>>> >>> >> > > > > >> > > > > >> is
>>>> >>> >> > > > > >> > > > > >> > > > > > inconsistent (or lacks 'polish'),
>>>> >>> >> operators
>>>> >>> >> > > will
>>>> >>> >> > > > > >> judge
>>>> >>> >> > > > > >> > the
>>>> >>> >> > > > > >> > > > > whole
>>>> >>> >> > > > > >> > > > > >> > > > > > backup/restore tooling chain as
>>>> not
>>>> >>> >> > trustworthy
>>>> >>> >> > > > and
>>>> >>> >> > > > > >> > > abandon
>>>> >>> >> > > > > >> > > > > it.
>>>> >>> >> > > > > >> > > > > >> > Lets
>>>> >>> >> > > > > >> > > > > >> > > > not
>>>> >>> >> > > > > >> > > > > >> > > > > > have this happen to this feature.
>>>> >>> >> > > > > >> > > > > >> > > > > > + Matteo's suggestion (with a
>>>> helpful
>>>> >>> >> starter
>>>> >>> >> > > > list)
>>>> >>> >> > > > > >> that
>>>> >>> >> > > > > >> > > > there
>>>> >>> >> > > > > >> > > > > >> > needs
>>>> >>> >> > > > > >> > > > > >> > > to
>>>> >>> >> > > > > >> > > > > >> > > > > be
>>>> >>> >> > > > > >> > > > > >> > > > > > explicit qualification on what is
>>>> >>> actually
>>>> >>> >> > > being
>>>> >>> >> > > > > >> > delivered
>>>> >>> >> > > > > >> > > > --
>>>> >>> >> > > > > >> > > > > >> > > > including a
>>>> >>> >> > > > > >> > > > > >> > > > > > listing of limitations (some look
>>>> >>> serious
>>>> >>> >> > such
>>>> >>> >> > > as
>>>> >>> >> > > > > >> data
>>>> >>> >> > > > > >> > > bleed
>>>> >>> >> > > > > >> > > > > from
>>>> >>> >> > > > > >> > > > > >> > > other
>>>> >>> >> > > > > >> > > > > >> > > > > > regions in WALs, but maybe I don't
>>>> >>> care
>>>> >>> >> for
>>>> >>> >> > my
>>>> >>> >> > > > use
>>>> >>> >> > > > > >> > > case...)
>>>> >>> >> > > > > >> > > > --
>>>> >>> >> > > > > >> > > > > >> > needs
>>>> >>> >> > > > > >> > > > > >> > > to
>>>> >>> >> > > > > >> > > > > >> > > > > > accompany the merge. Lets fold
>>>> them
>>>> >>> into
>>>> >>> >> the
>>>> >>> >> > > user
>>>> >>> >> > > > > >> doc.
>>>> >>> >> > > > > >> > in
>>>> >>> >> > > > > >> > > > the
>>>> >>> >> > > > > >> > > > > >> > > technical
>>>> >>> >> > > > > >> > > > > >> > > > > > overview area as suggested so user
>>>> >>> >> > expectations
>>>> >>> >> > > > are
>>>> >>> >> > > > > >> > > properly
>>>> >>> >> > > > > >> > > > > >> > managed
>>>> >>> >> > > > > >> > > > > >> > > > > > (otherwise, they expect the world
>>>> and
>>>> >>> will
>>>> >>> >> > just
>>>> >>> >> > > > > give
>>>> >>> >> > > > > >> up
>>>> >>> >> > > > > >> > > when
>>>> >>> >> > > > > >> > > > > we
>>>> >>> >> > > > > >> > > > > >> > fall
>>>> >>> >> > > > > >> > > > > >> > > > > > short). Vladimir did a list of
>>>> what
>>>> >>> is in
>>>> >>> >> > each
>>>> >>> >> > > of
>>>> >>> >> > > > > the
>>>> >>> >> > > > > >> > > phases
>>>> >>> >> > > > > >> > > > > >> above
>>>> >>> >> > > > > >> > > > > >> > > > which
>>>> >>> >> > > > > >> > > > > >> > > > > > would serve as a good start.
>>>> >>> >> > > > > >> > > > > >> > > > > > + Is this feature 'experimental'
>>>> >>> (Matteo
>>>> >>> >> asks
>>>> >>> >> > > > > above).
>>>> >>> >> > > > > >> > I'd
>>>> >>> >> > > > > >> > > > > prefer
>>>> >>> >> > > > > >> > > > > >> it
>>>> >>> >> > > > > >> > > > > >> > > is
>>>> >>> >> > > > > >> > > > > >> > > > > > not. If it is, it should be
>>>> labelled
>>>> >>> all
>>>> >>> >> over
>>>> >>> >> > > > that
>>>> >>> >> > > > > >> it is
>>>> >>> >> > > > > >> > > > so. I
>>>> >>> >> > > > > >> > > > > >> see
>>>> >>> >> > > > > >> > > > > >> > > > > current
>>>> >>> >> > > > > >> > > > > >> > > > > > state called out as a '...
>>>> technical
>>>> >>> >> preview
>>>> >>> >> > > > > >> feature'.
>>>> >>> >> > > > > >> > > Does
>>>> >>> >> > > > > >> > > > > this
>>>> >>> >> > > > > >> > > > > >> > mean
>>>> >>> >> > > > > >> > > > > >> > > > > > not-for-users?
>>>> >>> >> > > > > >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > > St.Ack
>>>> >>> >> > > > > >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > > On Mon, Sep 12, 2016 at 8:03 AM,
>>>> Ted
>>>> >>> Yu <
>>>> >>> >> > > > > >> > > > yuzhihong@gmail.com>
>>>> >>> >> > > > > >> > > > > >> > wrote:
>>>> >>> >> > > > > >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> Sean:
>>>> >>> >> > > > > >> > > > > >> > > > > >> Do you have more comments ?
>>>> >>> >> > > > > >> > > > > >> > > > > >>
>>>> >>> >> > > > > >> > > > > >> > > > > >> Cheers
>>>> >>> >> > > > > >> > > > > >> > > > > >>
>>>> >>> >> > > > > >> > > > > >> > > > > >> On Fri, Sep 9, 2016 at 1:42 PM,
>>>> >>> Vladimir
>>>> >>> >> > > > Rodionov
>>>> >>> >> > > > > <
>>>> >>> >> > > > > >> > > > > >> > > > > vladrodionov@gmail.com
>>>> >>> >> > > > > >> > > > > >> > > > > >> >
>>>> >>> >> > > > > >> > > > > >> > > > > >> wrote:
>>>> >>> >> > > > > >> > > > > >> > > > > >>
>>>> >>> >> > > > > >> > > > > >> > > > > >> > Sean,
>>>> >>> >> > > > > >> > > > > >> > > > > >> >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > Backup/Restore can fail due to
>>>> >>> various
>>>> >>> >> > > > reasons:
>>>> >>> >> > > > > >> > network
>>>> >>> >> > > > > >> > > > > outage
>>>> >>> >> > > > > >> > > > > >> > > > > (cluster
>>>> >>> >> > > > > >> > > > > >> > > > > >> > wide), various time-outs in
>>>> HBase
>>>> >>> and
>>>> >>> >> HDFS
>>>> >>> >> > > > > layer,
>>>> >>> >> > > > > >> M/R
>>>> >>> >> > > > > >> > > > > failure
>>>> >>> >> > > > > >> > > > > >> > due
>>>> >>> >> > > > > >> > > > > >> > > to
>>>> >>> >> > > > > >> > > > > >> > > > > >> "HDFS
>>>> >>> >> > > > > >> > > > > >> > > > > >> > exceeded quota", user error
>>>> (manual
>>>> >>> >> > deletion
>>>> >>> >> > > > of
>>>> >>> >> > > > > >> data)
>>>> >>> >> > > > > >> > > and
>>>> >>> >> > > > > >> > > > > so
>>>> >>> >> > > > > >> > > > > >> on
>>>> >>> >> > > > > >> > > > > >> > so
>>>> >>> >> > > > > >> > > > > >> > > > on.
>>>> >>> >> > > > > >> > > > > >> > > > > >> That
>>>> >>> >> > > > > >> > > > > >> > > > > >> > is impossible to enumerate all
>>>> >>> possible
>>>> >>> >> > > types
>>>> >>> >> > > > of
>>>> >>> >> > > > > >> > > failures
>>>> >>> >> > > > > >> > > > > in a
>>>> >>> >> > > > > >> > > > > >> > > > > >> distributed
>>>> >>> >> > > > > >> > > > > >> > > > > >> > system - that is not our
>>>> goal/task.
>>>> >>> >> > > > > >> > > > > >> > > > > >> >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > We focus completely on backup
>>>> >>> system
>>>> >>> >> table
>>>> >>> >> > > > > >> > consistency
>>>> >>> >> > > > > >> > > > in a
>>>> >>> >> > > > > >> > > > > >> > > presence
>>>> >>> >> > > > > >> > > > > >> > > > > of
>>>> >>> >> > > > > >> > > > > >> > > > > >> any
>>>> >>> >> > > > > >> > > > > >> > > > > >> > type of failure. That is what I
>>>> >>> call
>>>> >>> >> > > > "tolerance
>>>> >>> >> > > > > to
>>>> >>> >> > > > > >> > > > > failures".
>>>> >>> >> > > > > >> > > > > >> > > > > >> >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > On a failure:
>>>> >>> >> > > > > >> > > > > >> > > > > >> >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > BACKUP. All backup system
>>>> >>> information
>>>> >>> >> > (prior
>>>> >>> >> > > > to
>>>> >>> >> > > > > >> > backup)
>>>> >>> >> > > > > >> > > > > will
>>>> >>> >> > > > > >> > > > > >> be
>>>> >>> >> > > > > >> > > > > >> > > > > restored
>>>> >>> >> > > > > >> > > > > >> > > > > >> > and all temporary data,
>>>> related to
>>>> >>> a
>>>> >>> >> > failed
>>>> >>> >> > > > > >> session,
>>>> >>> >> > > > > >> > in
>>>> >>> >> > > > > >> > > > > HDFS
>>>> >>> >> > > > > >> > > > > >> > will
>>>> >>> >> > > > > >> > > > > >> > > be
>>>> >>> >> > > > > >> > > > > >> > > > > >> > deleted
>>>> >>> >> > > > > >> > > > > >> > > > > >> > RESTORE. We do not care about
>>>> >>> system
>>>> >>> >> data,
>>>> >>> >> > > > > because
>>>> >>> >> > > > > >> > > > restore
>>>> >>> >> > > > > >> > > > > >> does
>>>> >>> >> > > > > >> > > > > >> > > not
>>>> >>> >> > > > > >> > > > > >> > > > > >> change
>>>> >>> >> > > > > >> > > > > >> > > > > >> > it. Temporary data in HDFS
>>>> will be
>>>> >>> >> cleaned
>>>> >>> >> > > up
>>>> >>> >> > > > > and
>>>> >>> >> > > > > >> > table
>>>> >>> >> > > > > >> > > > > will
>>>> >>> >> > > > > >> > > > > >> be
>>>> >>> >> > > > > >> > > > > >> > > in a
>>>> >>> >> > > > > >> > > > > >> > > > > >> state
>>>> >>> >> > > > > >> > > > > >> > > > > >> > back to where it was before
>>>> >>> operation
>>>> >>> >> > > started.
>>>> >>> >> > > > > >> > > > > >> > > > > >> >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > This is what user should
>>>> expect in
>>>> >>> case
>>>> >>> >> > of a
>>>> >>> >> > > > > >> failure.
>>>> >>> >> > > > > >> > > > > >> > > > > >> >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > -Vlad
>>>> >>> >> > > > > >> > > > > >> > > > > >> >
>>>> >>> >> > > > > >> > > > > >> > > > > >> >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > -Vlad
>>>> >>> >> > > > > >> > > > > >> > > > > >> >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > On Fri, Sep 9, 2016 at 12:56
>>>> PM,
>>>> >>> Sean
>>>> >>> >> > > Busbey <
>>>> >>> >> > > > > >> > > > > >> busbey@apache.org
>>>> >>> >> > > > > >> > > > > >> > >
>>>> >>> >> > > > > >> > > > > >> > > > > wrote:
>>>> >>> >> > > > > >> > > > > >> > > > > >> >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > Failing in a consistent way,
>>>> with
>>>> >>> >> docs
>>>> >>> >> > > that
>>>> >>> >> > > > > >> explain
>>>> >>> >> > > > > >> > > the
>>>> >>> >> > > > > >> > > > > >> > various
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > expected failures would be
>>>> >>> >> sufficient.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > On Fri, Sep 9, 2016 at 12:16
>>>> PM,
>>>> >>> >> > Vladimir
>>>> >>> >> > > > > >> Rodionov
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > <vl...@gmail.com>
>>>> wrote:
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > > Do not worry Sean, doc is
>>>> >>> coming
>>>> >>> >> today
>>>> >>> >> > > as
>>>> >>> >> > > > a
>>>> >>> >> > > > > >> > preview
>>>> >>> >> > > > > >> > > > and
>>>> >>> >> > > > > >> > > > > >> our
>>>> >>> >> > > > > >> > > > > >> > > > writer
>>>> >>> >> > > > > >> > > > > >> > > > > >> > Frank
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > > will be working on a
>>>> putting
>>>> >>> it
>>>> >>> >> into
>>>> >>> >> > > > Apache
>>>> >>> >> > > > > >> > repo.
>>>> >>> >> > > > > >> > > > > >> Timeline
>>>> >>> >> > > > > >> > > > > >> > > > > depends
>>>> >>> >> > > > > >> > > > > >> > > > > >> on
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > > Franks schedule but I hope
>>>> we
>>>> >>> will
>>>> >>> >> get
>>>> >>> >> > > it
>>>> >>> >> > > > > >> rather
>>>> >>> >> > > > > >> > > > sooner
>>>> >>> >> > > > > >> > > > > >> than
>>>> >>> >> > > > > >> > > > > >> > > > > later.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > > As for failure testing, we
>>>> are
>>>> >>> >> > focusing
>>>> >>> >> > > > only
>>>> >>> >> > > > > >> on a
>>>> >>> >> > > > > >> > > > > >> consistent
>>>> >>> >> > > > > >> > > > > >> > > > state
>>>> >>> >> > > > > >> > > > > >> > > > > >> of
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > > backup system data in a
>>>> >>> presence of
>>>> >>> >> > any
>>>> >>> >> > > > type
>>>> >>> >> > > > > >> of
>>>> >>> >> > > > > >> > > > > failures,
>>>> >>> >> > > > > >> > > > > >> We
>>>> >>> >> > > > > >> > > > > >> > > are
>>>> >>> >> > > > > >> > > > > >> > > > > not
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > going
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > > to implement  anything more
>>>> >>> >> "fancy",
>>>> >>> >> > > than
>>>> >>> >> > > > > >> that.
>>>> >>> >> > > > > >> > We
>>>> >>> >> > > > > >> > > > > allow
>>>> >>> >> > > > > >> > > > > >> > both:
>>>> >>> >> > > > > >> > > > > >> > > > > >> backup
>>>> >>> >> > > > > >> > > > > >> > > > > >> > and
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > > restore to fail. What we
>>>> do not
>>>> >>> >> allow
>>>> >>> >> > is
>>>> >>> >> > > > to
>>>> >>> >> > > > > >> have
>>>> >>> >> > > > > >> > > > system
>>>> >>> >> > > > > >> > > > > >> data
>>>> >>> >> > > > > >> > > > > >> > > > > >> corrupted.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > > Will it suffice for you?
>>>> Do you
>>>> >>> >> have
>>>> >>> >> > any
>>>> >>> >> > > > > other
>>>> >>> >> > > > > >> > > > > concerns,
>>>> >>> >> > > > > >> > > > > >> you
>>>> >>> >> > > > > >> > > > > >> > > > want
>>>> >>> >> > > > > >> > > > > >> > > > > >> us to
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > > address?
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > > -Vlad
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > > On Fri, Sep 9, 2016 at
>>>> 10:56
>>>> >>> AM,
>>>> >>> >> Sean
>>>> >>> >> > > > > Busbey <
>>>> >>> >> > > > > >> > > > > >> > > busbey@apache.org
>>>> >>> >> > > > > >> > > > > >> > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > wrote:
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> "docs will come to Apache
>>>> >>> soon"
>>>> >>> >> does
>>>> >>> >> > > not
>>>> >>> >> > > > > >> address
>>>> >>> >> > > > > >> > > my
>>>> >>> >> > > > > >> > > > > >> concern
>>>> >>> >> > > > > >> > > > > >> > > > > around
>>>> >>> >> > > > > >> > > > > >> > > > > >> > docs
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > at
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> all, unless said docs have
>>>> >>> already
>>>> >>> >> > made
>>>> >>> >> > > > it
>>>> >>> >> > > > > >> into
>>>> >>> >> > > > > >> > > the
>>>> >>> >> > > > > >> > > > > >> project
>>>> >>> >> > > > > >> > > > > >> > > > > repo. I
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > don't
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> want third party
>>>> resources for
>>>> >>> >> using
>>>> >>> >> > a
>>>> >>> >> > > > > major
>>>> >>> >> > > > > >> and
>>>> >>> >> > > > > >> > > > > >> important
>>>> >>> >> > > > > >> > > > > >> > > > > feature
>>>> >>> >> > > > > >> > > > > >> > > > > >> of
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > the
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> project, I want us to
>>>> provide
>>>> >>> end
>>>> >>> >> > users
>>>> >>> >> > > > > with
>>>> >>> >> > > > > >> > what
>>>> >>> >> > > > > >> > > > they
>>>> >>> >> > > > > >> > > > > >> need
>>>> >>> >> > > > > >> > > > > >> > > to
>>>> >>> >> > > > > >> > > > > >> > > > > get
>>>> >>> >> > > > > >> > > > > >> > > > > >> the
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > job
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> done.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >>
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> I see some calls for
>>>> patience
>>>> >>> on
>>>> >>> >> the
>>>> >>> >> > > > > failure
>>>> >>> >> > > > > >> > > > testing,
>>>> >>> >> > > > > >> > > > > but
>>>> >>> >> > > > > >> > > > > >> > the
>>>> >>> >> > > > > >> > > > > >> > > > > >> appeal
>>>> >>> >> > > > > >> > > > > >> > > > > >> > to
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > us
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> having done a bad job of
>>>> >>> requiring
>>>> >>> >> > > proper
>>>> >>> >> > > > > >> tests
>>>> >>> >> > > > > >> > of
>>>> >>> >> > > > > >> > > > > >> previous
>>>> >>> >> > > > > >> > > > > >> > > > > >> features
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > just
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> makes me more concerned
>>>> about
>>>> >>> not
>>>> >>> >> > > getting
>>>> >>> >> > > > > >> them
>>>> >>> >> > > > > >> > > > here. I
>>>> >>> >> > > > > >> > > > > >> > don't
>>>> >>> >> > > > > >> > > > > >> > > > want
>>>> >>> >> > > > > >> > > > > >> > > > > >> to
>>>> >>> >> > > > > >> > > > > >> > > > > >> > set
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> yet another bad example
>>>> that
>>>> >>> will
>>>> >>> >> > then
>>>> >>> >> > > be
>>>> >>> >> > > > > >> > pointed
>>>> >>> >> > > > > >> > > to
>>>> >>> >> > > > > >> > > > > in
>>>> >>> >> > > > > >> > > > > >> the
>>>> >>> >> > > > > >> > > > > >> > > > > future.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >>
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> On Sep 8, 2016 10:50, "Ted
>>>> >>> Yu" <
>>>> >>> >> > > > > >> > > yuzhihong@gmail.com
>>>> >>> >> > > > > >> > > > >
>>>> >>> >> > > > > >> > > > > >> > wrote:
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >>
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > Is there any concern
>>>> which
>>>> >>> is
>>>> >>> >> not
>>>> >>> >> > > > > >> addressed ?
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > Do we need another Vote
>>>> >>> thread ?
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > Thanks
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > On Thu, Sep 8, 2016 at
>>>> 9:21
>>>> >>> AM,
>>>> >>> >> > > Andrew
>>>> >>> >> > > > > >> > Purtell <
>>>> >>> >> > > > > >> > > > > >> > > > > >> apurtell@apache.org
>>>> >>> >> > > > > >> > > > > >> > > > > >> > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > wrote:
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > Vlad,
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > I apologize for using
>>>> the
>>>> >>> term
>>>> >>> >> > > > > >> 'half-baked'
>>>> >>> >> > > > > >> > > in a
>>>> >>> >> > > > > >> > > > > way
>>>> >>> >> > > > > >> > > > > >> > that
>>>> >>> >> > > > > >> > > > > >> > > > > could
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > seem a
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > description of
>>>> >>> HBASE-7912. I
>>>> >>> >> > meant
>>>> >>> >> > > > that
>>>> >>> >> > > > > >> as a
>>>> >>> >> > > > > >> > > > > general
>>>> >>> >> > > > > >> > > > > >> > > > > >> hypothetical.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > On Wed, Sep 7, 2016 at
>>>> >>> 9:36
>>>> >>> >> AM,
>>>> >>> >> > > > > Vladimir
>>>> >>> >> > > > > >> > > > Rodionov
>>>> >>> >> > > > > >> > > > > <
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > vladrodionov@gmail.com>
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > wrote:
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >> I'm not sure that
>>>> >>> "There
>>>> >>> >> is
>>>> >>> >> > > > > already
>>>> >>> >> > > > > >> > lots
>>>> >>> >> > > > > >> > > of
>>>> >>> >> > > > > >> > > > > >> > > half-baked
>>>> >>> >> > > > > >> > > > > >> > > > > >> code
>>>> >>> >> > > > > >> > > > > >> > > > > >> > in
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > the
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > branch,
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > so what's the harm
>>>> in
>>>> >>> adding
>>>> >>> >> > > more?"
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > I meant - not
>>>> >>> production -
>>>> >>> >> > ready
>>>> >>> >> > > > yet.
>>>> >>> >> > > > > >> This
>>>> >>> >> > > > > >> > > is
>>>> >>> >> > > > > >> > > > > 2.0
>>>> >>> >> > > > > >> > > > > >> > > > > development
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > branch
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > and,
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > hence many features
>>>> are
>>>> >>> in
>>>> >>> >> > works,
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > not being tested
>>>> well
>>>> >>> etc.
>>>> >>> >> I do
>>>> >>> >> > > not
>>>> >>> >> > > > > >> > consider
>>>> >>> >> > > > > >> > > > > backup
>>>> >>> >> > > > > >> > > > > >> > as
>>>> >>> >> > > > > >> > > > > >> > > > half
>>>> >>> >> > > > > >> > > > > >> > > > > >> > baked
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > feature -
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > it has passed our
>>>> >>> internal
>>>> >>> >> QA
>>>> >>> >> > and
>>>> >>> >> > > > has
>>>> >>> >> > > > > >> very
>>>> >>> >> > > > > >> > > > good
>>>> >>> >> > > > > >> > > > > >> doc,
>>>> >>> >> > > > > >> > > > > >> > > > which
>>>> >>> >> > > > > >> > > > > >> > > > > we
>>>> >>> >> > > > > >> > > > > >> > > > > >> > will
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > provide
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > to Apache shortly.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > -Vlad
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > On Wed, Sep 7, 2016
>>>> at
>>>> >>> 9:13
>>>> >>> >> AM,
>>>> >>> >> > > > > Andrew
>>>> >>> >> > > > > >> > > > Purtell <
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > apurtell@apache.org>
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > We shouldn't admit
>>>> >>> half
>>>> >>> >> baked
>>>> >>> >> > > > > changes
>>>> >>> >> > > > > >> > that
>>>> >>> >> > > > > >> > > > > won't
>>>> >>> >> > > > > >> > > > > >> be
>>>> >>> >> > > > > >> > > > > >> > > > > >> finished.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> However
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > in
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > this case the crew
>>>> >>> >> working on
>>>> >>> >> > > > this
>>>> >>> >> > > > > >> > feature
>>>> >>> >> > > > > >> > > > are
>>>> >>> >> > > > > >> > > > > >> long
>>>> >>> >> > > > > >> > > > > >> > > > > timers
>>>> >>> >> > > > > >> > > > > >> > > > > >> and
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > less
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > likely
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > than just about
>>>> >>> anyone to
>>>> >>> >> > leave
>>>> >>> >> > > > > >> > something
>>>> >>> >> > > > > >> > > > in a
>>>> >>> >> > > > > >> > > > > >> half
>>>> >>> >> > > > > >> > > > > >> > > > baked
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > state. Of
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > course
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > there is no
>>>> guarantee
>>>> >>> how
>>>> >>> >> > > > anything
>>>> >>> >> > > > > >> will
>>>> >>> >> > > > > >> > > turn
>>>> >>> >> > > > > >> > > > > out,
>>>> >>> >> > > > > >> > > > > >> > > but I
>>>> >>> >> > > > > >> > > > > >> > > > > am
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > willing
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> to
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > take
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > a little on faith
>>>> if
>>>> >>> they
>>>> >>> >> > feel
>>>> >>> >> > > > > their
>>>> >>> >> > > > > >> > best
>>>> >>> >> > > > > >> > > > path
>>>> >>> >> > > > > >> > > > > >> > > forward
>>>> >>> >> > > > > >> > > > > >> > > > > now
>>>> >>> >> > > > > >> > > > > >> > > > > >> is
>>>> >>> >> > > > > >> > > > > >> > > > > >> > to
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > merge
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > to
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > trunk. I only
>>>> wish I
>>>> >>> had
>>>> >>> >> > > > bandwidth
>>>> >>> >> > > > > to
>>>> >>> >> > > > > >> > have
>>>> >>> >> > > > > >> > > > > done
>>>> >>> >> > > > > >> > > > > >> > some
>>>> >>> >> > > > > >> > > > > >> > > > real
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > kicking
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> of
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > the
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > tires by now.
>>>> Maybe
>>>> >>> this
>>>> >>> >> > week.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > (Yes, I'm using
>>>> some
>>>> >>> of
>>>> >>> >> that
>>>> >>> >> > > time
>>>> >>> >> > > > > for
>>>> >>> >> > > > > >> > this
>>>> >>> >> > > > > >> > > > > email
>>>> >>> >> > > > > >> > > > > >> > :-)
>>>> >>> >> > > > > >> > > > > >> > > > but
>>>> >>> >> > > > > >> > > > > >> > > > > I
>>>> >>> >> > > > > >> > > > > >> > > > > >> > type
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > fast.)
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > That said, I would
>>>> >>> like to
>>>> >>> >> > > > agitate
>>>> >>> >> > > > > >> for
>>>> >>> >> > > > > >> > > > making
>>>> >>> >> > > > > >> > > > > 2.0
>>>> >>> >> > > > > >> > > > > >> > > more
>>>> >>> >> > > > > >> > > > > >> > > > > real
>>>> >>> >> > > > > >> > > > > >> > > > > >> > and
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> spend
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > some
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > time on it now
>>>> that
>>>> >>> I'm
>>>> >>> >> > winding
>>>> >>> >> > > > > down
>>>> >>> >> > > > > >> > with
>>>> >>> >> > > > > >> > > > > 0.98. I
>>>> >>> >> > > > > >> > > > > >> > > think
>>>> >>> >> > > > > >> > > > > >> > > > > >> that
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > means
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branching for 2.0
>>>> real
>>>> >>> >> soon
>>>> >>> >> > now
>>>> >>> >> > > > and
>>>> >>> >> > > > > >> even
>>>> >>> >> > > > > >> > > > > evicting
>>>> >>> >> > > > > >> > > > > >> > > > things
>>>> >>> >> > > > > >> > > > > >> > > > > >> from
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > 2.0
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > branch
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > that aren't
>>>> finished
>>>> >>> or
>>>> >>> >> > stable,
>>>> >>> >> > > > > >> leaving
>>>> >>> >> > > > > >> > > them
>>>> >>> >> > > > > >> > > > > only
>>>> >>> >> > > > > >> > > > > >> > > once
>>>> >>> >> > > > > >> > > > > >> > > > > >> again
>>>> >>> >> > > > > >> > > > > >> > > > > >> > in
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > the
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > master
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branch. Or, maybe
>>>> just
>>>> >>> >> > evicting
>>>> >>> >> > > > > them.
>>>> >>> >> > > > > >> > > Let's
>>>> >>> >> > > > > >> > > > > take
>>>> >>> >> > > > > >> > > > > >> it
>>>> >>> >> > > > > >> > > > > >> > > > case
>>>> >>> >> > > > > >> > > > > >> > > > > by
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > case.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > I think this
>>>> feature
>>>> >>> can
>>>> >>> >> come
>>>> >>> >> > > in
>>>> >>> >> > > > > >> > > relatively
>>>> >>> >> > > > > >> > > > > >> safely.
>>>> >>> >> > > > > >> > > > > >> > > As
>>>> >>> >> > > > > >> > > > > >> > > > > >> added
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > insurance,
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > let's admit the
>>>> >>> >> possibility
>>>> >>> >> > it
>>>> >>> >> > > > > could
>>>> >>> >> > > > > >> be
>>>> >>> >> > > > > >> > > > > reverted
>>>> >>> >> > > > > >> > > > > >> on
>>>> >>> >> > > > > >> > > > > >> > > the
>>>> >>> >> > > > > >> > > > > >> > > > > 2.0
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > branch
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> if
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > folks
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > working on
>>>> >>> stabilizing 2.0
>>>> >>> >> > > decide
>>>> >>> >> > > > > to
>>>> >>> >> > > > > >> > evict
>>>> >>> >> > > > > >> > > > it
>>>> >>> >> > > > > >> > > > > >> > because
>>>> >>> >> > > > > >> > > > > >> > > > it
>>>> >>> >> > > > > >> > > > > >> > > > > is
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > unfinished
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > or
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > unstable, because
>>>> that
>>>> >>> >> > > certainly
>>>> >>> >> > > > > can
>>>> >>> >> > > > > >> > > > happen. I
>>>> >>> >> > > > > >> > > > > >> > would
>>>> >>> >> > > > > >> > > > > >> > > > > >> expect if
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > talk
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > like
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > that starts, we'd
>>>> get
>>>> >>> help
>>>> >>> >> > > > > finishing
>>>> >>> >> > > > > >> or
>>>> >>> >> > > > > >> > > > > >> stabilizing
>>>> >>> >> > > > > >> > > > > >> > > > > what's
>>>> >>> >> > > > > >> > > > > >> > > > > >> > under
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > discussion
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > for revert. Or,
>>>> we'd
>>>> >>> have
>>>> >>> >> a
>>>> >>> >> > > > revert.
>>>> >>> >> > > > > >> > Either
>>>> >>> >> > > > > >> > > > way
>>>> >>> >> > > > > >> > > > > >> the
>>>> >>> >> > > > > >> > > > > >> > > > > outcome
>>>> >>> >> > > > > >> > > > > >> > > > > >> is
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > acceptable.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > On Wed, Sep 7,
>>>> 2016 at
>>>> >>> >> 8:56
>>>> >>> >> > AM,
>>>> >>> >> > > > > Dima
>>>> >>> >> > > > > >> > > Spivak
>>>> >>> >> > > > > >> > > > <
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > dimaspivak@apache.org
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > I'm not sure
>>>> that
>>>> >>> >> "There is
>>>> >>> >> > > > > already
>>>> >>> >> > > > > >> > lots
>>>> >>> >> > > > > >> > > > of
>>>> >>> >> > > > > >> > > > > >> > > > half-baked
>>>> >>> >> > > > > >> > > > > >> > > > > >> code
>>>> >>> >> > > > > >> > > > > >> > > > > >> > in
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> the
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branch,
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > so what's the
>>>> harm
>>>> >>> in
>>>> >>> >> > adding
>>>> >>> >> > > > > more?"
>>>> >>> >> > > > > >> > is a
>>>> >>> >> > > > > >> > > > > good
>>>> >>> >> > > > > >> > > > > >> > code
>>>> >>> >> > > > > >> > > > > >> > > > > commit
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > philosophy
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > for
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > a
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > fault-tolerant
>>>> >>> >> distributed
>>>> >>> >> > > data
>>>> >>> >> > > > > >> store.
>>>> >>> >> > > > > >> > > ;)
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > More seriously,
>>>> a
>>>> >>> lack
>>>> >>> >> of
>>>> >>> >> > > test
>>>> >>> >> > > > > >> > coverage
>>>> >>> >> > > > > >> > > > for
>>>> >>> >> > > > > >> > > > > >> > > existing
>>>> >>> >> > > > > >> > > > > >> > > > > >> > features
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > shouldn't
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > be
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > used as
>>>> >>> justification
>>>> >>> >> for
>>>> >>> >> > > > > >> introducing
>>>> >>> >> > > > > >> > > new
>>>> >>> >> > > > > >> > > > > >> > features
>>>> >>> >> > > > > >> > > > > >> > > > with
>>>> >>> >> > > > > >> > > > > >> > > > > >> the
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > same
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > shortcomings.
>>>> >>> >> Ultimately,
>>>> >>> >> > > it's
>>>> >>> >> > > > > the
>>>> >>> >> > > > > >> end
>>>> >>> >> > > > > >> > > > user
>>>> >>> >> > > > > >> > > > > who
>>>> >>> >> > > > > >> > > > > >> > > will
>>>> >>> >> > > > > >> > > > > >> > > > > feel
>>>> >>> >> > > > > >> > > > > >> > > > > >> > the
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> pain,
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > so
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > shouldn't we do
>>>> >>> >> everything
>>>> >>> >> > we
>>>> >>> >> > > > can
>>>> >>> >> > > > > >> to
>>>> >>> >> > > > > >> > > > > mitigate
>>>> >>> >> > > > > >> > > > > >> > that?
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > -Dima
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > On Wed, Sep 7,
>>>> 2016
>>>> >>> at
>>>> >>> >> 8:46
>>>> >>> >> > > AM,
>>>> >>> >> > > > > >> > Vladimir
>>>> >>> >> > > > > >> > > > > >> > Rodionov <
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>>> >>> vladrodionov@gmail.com>
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > wrote:
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Sean,
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have docs
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Agree. We
>>>> have a
>>>> >>> doc
>>>> >>> >> and
>>>> >>> >> > > > backup
>>>> >>> >> > > > > >> is
>>>> >>> >> > > > > >> > the
>>>> >>> >> > > > > >> > > > > most
>>>> >>> >> > > > > >> > > > > >> > > > > documented
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > feature
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > :),
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > we
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > will
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > release it
>>>> >>> shortly to
>>>> >>> >> > > Apache.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have
>>>> sunny-day
>>>> >>> >> > > correctness
>>>> >>> >> > > > > >> tests
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Feature has
>>>> >>> close to
>>>> >>> >> 60
>>>> >>> >> > > test
>>>> >>> >> > > > > >> cases,
>>>> >>> >> > > > > >> > > > which
>>>> >>> >> > > > > >> > > > > >> run
>>>> >>> >> > > > > >> > > > > >> > > for
>>>> >>> >> > > > > >> > > > > >> > > > > >> approx
>>>> >>> >> > > > > >> > > > > >> > > > > >> > 30
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> min.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > We
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > can
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > add more, if
>>>> >>> >> community do
>>>> >>> >> > > not
>>>> >>> >> > > > > >> mind
>>>> >>> >> > > > > >> > :)
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have
>>>> >>> >> > > > > >> > correctness-in-face-of-failure
>>>> >>> >> > > > > >> > > > > tests
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Any examples
>>>> of
>>>> >>> these
>>>> >>> >> > tests
>>>> >>> >> > > > in
>>>> >>> >> > > > > >> > > existing
>>>> >>> >> > > > > >> > > > > >> > features?
>>>> >>> >> > > > > >> > > > > >> > > > In
>>>> >>> >> > > > > >> > > > > >> > > > > >> > works,
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > we
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > have a
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > clear
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > understanding
>>>> of
>>>> >>> what
>>>> >>> >> > > should
>>>> >>> >> > > > be
>>>> >>> >> > > > > >> done
>>>> >>> >> > > > > >> > > by
>>>> >>> >> > > > > >> > > > > the
>>>> >>> >> > > > > >> > > > > >> > time
>>>> >>> >> > > > > >> > > > > >> > > of
>>>> >>> >> > > > > >> > > > > >> > > > > 2.0
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> release.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > That is very
>>>> close
>>>> >>> >> goal
>>>> >>> >> > for
>>>> >>> >> > > > us,
>>>> >>> >> > > > > >> to
>>>> >>> >> > > > > >> > > > verify
>>>> >>> >> > > > > >> > > > > IT
>>>> >>> >> > > > > >> > > > > >> > > monkey
>>>> >>> >> > > > > >> > > > > >> > > > > for
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> existing
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > code.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * don't rely
>>>> on
>>>> >>> things
>>>> >>> >> > > > outside
>>>> >>> >> > > > > of
>>>> >>> >> > > > > >> > > HBase
>>>> >>> >> > > > > >> > > > > for
>>>> >>> >> > > > > >> > > > > >> > > normal
>>>> >>> >> > > > > >> > > > > >> > > > > >> > operation
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > (okay
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > for
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > advanced
>>>> >>> operation)
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > We do not.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Enormous time
>>>> has
>>>> >>> been
>>>> >>> >> > > spent
>>>> >>> >> > > > > >> already
>>>> >>> >> > > > > >> > > on
>>>> >>> >> > > > > >> > > > > the
>>>> >>> >> > > > > >> > > > > >> > > > > development
>>>> >>> >> > > > > >> > > > > >> > > > > >> > and
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > testing
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > the
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > feature, it
>>>> has
>>>> >>> passed
>>>> >>> >> > our
>>>> >>> >> > > > > >> internal
>>>> >>> >> > > > > >> > > > tests
>>>> >>> >> > > > > >> > > > > and
>>>> >>> >> > > > > >> > > > > >> > > many
>>>> >>> >> > > > > >> > > > > >> > > > > >> rounds
>>>> >>> >> > > > > >> > > > > >> > > > > >> > of
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> code
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > reviews
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > by HBase
>>>> >>> committers.
>>>> >>> >> We
>>>> >>> >> > do
>>>> >>> >> > > > not
>>>> >>> >> > > > > >> mind
>>>> >>> >> > > > > >> > if
>>>> >>> >> > > > > >> > > > > >> someone
>>>> >>> >> > > > > >> > > > > >> > > from
>>>> >>> >> > > > > >> > > > > >> > > > > >> HBase
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > community
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > (outside of
>>>> HW)
>>>> >>> will
>>>> >>> >> > review
>>>> >>> >> > > > the
>>>> >>> >> > > > > >> > code,
>>>> >>> >> > > > > >> > > > but
>>>> >>> >> > > > > >> > > > > it
>>>> >>> >> > > > > >> > > > > >> > will
>>>> >>> >> > > > > >> > > > > >> > > > > >> probably
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> takes
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > forever
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > to
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > wait for
>>>> >>> volunteer?,
>>>> >>> >> the
>>>> >>> >> > > > > feature
>>>> >>> >> > > > > >> is
>>>> >>> >> > > > > >> > > > quite
>>>> >>> >> > > > > >> > > > > >> large
>>>> >>> >> > > > > >> > > > > >> > > > (1MB+
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> cumulative
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > patch)
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > 2.0 branch is
>>>> >>> full of
>>>> >>> >> > half
>>>> >>> >> > > > > baked
>>>> >>> >> > > > > >> > > > features,
>>>> >>> >> > > > > >> > > > > >> most
>>>> >>> >> > > > > >> > > > > >> > > of
>>>> >>> >> > > > > >> > > > > >> > > > > them
>>>> >>> >> > > > > >> > > > > >> > > > > >> > are
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > in
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > active
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > development,
>>>> >>> >> therefore I
>>>> >>> >> > am
>>>> >>> >> > > > not
>>>> >>> >> > > > > >> > > > following
>>>> >>> >> > > > > >> > > > > you
>>>> >>> >> > > > > >> > > > > >> > > here,
>>>> >>> >> > > > > >> > > > > >> > > > > >> Sean?
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > Why
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > HBASE-7912
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > is
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > not good
>>>> enough
>>>> >>> yet
>>>> >>> >> to be
>>>> >>> >> > > > > >> integrated
>>>> >>> >> > > > > >> > > > into
>>>> >>> >> > > > > >> > > > > 2.0
>>>> >>> >> > > > > >> > > > > >> > > > branch?
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > -Vlad
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > On Wed, Sep 7,
>>>> >>> 2016 at
>>>> >>> >> > 8:23
>>>> >>> >> > > > AM,
>>>> >>> >> > > > > >> Sean
>>>> >>> >> > > > > >> > > > > Busbey <
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > busbey@apache.org
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > On Tue, Sep
>>>> 6,
>>>> >>> 2016
>>>> >>> >> at
>>>> >>> >> > > > 10:36
>>>> >>> >> > > > > >> PM,
>>>> >>> >> > > > > >> > > Josh
>>>> >>> >> > > > > >> > > > > >> Elser <
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > josh.elser@gmail.com>
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > wrote:
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > So, the
>>>> >>> answer to
>>>> >>> >> > > Sean's
>>>> >>> >> > > > > >> > original
>>>> >>> >> > > > > >> > > > > >> question
>>>> >>> >> > > > > >> > > > > >> > is
>>>> >>> >> > > > > >> > > > > >> > > > "as
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > robust as
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > snapshots
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > presently
>>>> >>> are"?
>>>> >>> >> > > > > >> (independence of
>>>> >>> >> > > > > >> > > > > >> > > backup/restore
>>>> >>> >> > > > > >> > > > > >> > > > > >> > failure
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > tolerance
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > from
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > snapshot
>>>> >>> failure
>>>> >>> >> > > > tolerance)
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > Is this
>>>> just a
>>>> >>> >> > question
>>>> >>> >> > > > WRT
>>>> >>> >> > > > > >> > > context
>>>> >>> >> > > > > >> > > > of
>>>> >>> >> > > > > >> > > > > >> the
>>>> >>> >> > > > > >> > > > > >> > > > > change,
>>>> >>> >> > > > > >> > > > > >> > > > > >> or
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > is it
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > means
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > for a
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > veto
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > from you,
>>>> >>> Sean?
>>>> >>> >> Just
>>>> >>> >> > > > trying
>>>> >>> >> > > > > >> to
>>>> >>> >> > > > > >> > > make
>>>> >>> >> > > > > >> > > > > sure
>>>> >>> >> > > > > >> > > > > >> > I'm
>>>> >>> >> > > > > >> > > > > >> > > > > >> following
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> along
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > adequately.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > I'd say ATM
>>>> I'm
>>>> >>> -0,
>>>> >>> >> > > > bordering
>>>> >>> >> > > > > >> on
>>>> >>> >> > > > > >> > -1
>>>> >>> >> > > > > >> > > > but
>>>> >>> >> > > > > >> > > > > not
>>>> >>> >> > > > > >> > > > > >> > for
>>>> >>> >> > > > > >> > > > > >> > > > > >> reasons
>>>> >>> >> > > > > >> > > > > >> > > > > >> > I
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > can
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > articulate
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > well.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Here's an
>>>> >>> attempt.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > We've been
>>>> >>> trying to
>>>> >>> >> > > move,
>>>> >>> >> > > > > as a
>>>> >>> >> > > > > >> > > > > community,
>>>> >>> >> > > > > >> > > > > >> > > > towards
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > minimizing
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > risk
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > to
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > downstream
>>>> >>> folks by
>>>> >>> >> > > getting
>>>> >>> >> > > > > >> > > "complete
>>>> >>> >> > > > > >> > > > > >> enough
>>>> >>> >> > > > > >> > > > > >> > > for
>>>> >>> >> > > > > >> > > > > >> > > > > use"
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > gates
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> in
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > place
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > before we
>>>> >>> introduce
>>>> >>> >> new
>>>> >>> >> > > > > >> features.
>>>> >>> >> > > > > >> > > This
>>>> >>> >> > > > > >> > > > > was
>>>> >>> >> > > > > >> > > > > >> > > > spurred
>>>> >>> >> > > > > >> > > > > >> > > > > >> by a
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > some
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > features
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > getting in
>>>> >>> >> half-baked
>>>> >>> >> > and
>>>> >>> >> > > > > never
>>>> >>> >> > > > > >> > > making
>>>> >>> >> > > > > >> > > > > it
>>>> >>> >> > > > > >> > > > > >> to
>>>> >>> >> > > > > >> > > > > >> > > "can
>>>> >>> >> > > > > >> > > > > >> > > > > >> really
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > use"
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > status
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > (I'm
>>>> thinking of
>>>> >>> >> > > > distributed
>>>> >>> >> > > > > >> log
>>>> >>> >> > > > > >> > > > replay
>>>> >>> >> > > > > >> > > > > and
>>>> >>> >> > > > > >> > > > > >> > the
>>>> >>> >> > > > > >> > > > > >> > > > > >> zk-less
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > assignment
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > stuff, I
>>>> don't
>>>> >>> >> recall
>>>> >>> >> > if
>>>> >>> >> > > > > there
>>>> >>> >> > > > > >> was
>>>> >>> >> > > > > >> > > > > more).
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The gates,
>>>> >>> >> generally,
>>>> >>> >> > > > > included
>>>> >>> >> > > > > >> > > things
>>>> >>> >> > > > > >> > > > > like:
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have docs
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have
>>>> sunny-day
>>>> >>> >> > > > correctness
>>>> >>> >> > > > > >> tests
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have
>>>> >>> >> > > > > >> > > correctness-in-face-of-failure
>>>> >>> >> > > > > >> > > > > tests
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't
>>>> rely on
>>>> >>> >> things
>>>> >>> >> > > > > outside
>>>> >>> >> > > > > >> of
>>>> >>> >> > > > > >> > > > HBase
>>>> >>> >> > > > > >> > > > > for
>>>> >>> >> > > > > >> > > > > >> > > > normal
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > operation
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > (okay
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > for
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > advanced
>>>> >>> operation)
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > As an
>>>> example,
>>>> >>> we
>>>> >>> >> kept
>>>> >>> >> > > the
>>>> >>> >> > > > > MOB
>>>> >>> >> > > > > >> > work
>>>> >>> >> > > > > >> > > > off
>>>> >>> >> > > > > >> > > > > in
>>>> >>> >> > > > > >> > > > > >> a
>>>> >>> >> > > > > >> > > > > >> > > > branch
>>>> >>> >> > > > > >> > > > > >> > > > > >> and
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > out
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> of
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > master
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > until it
>>>> could
>>>> >>> pass
>>>> >>> >> > these
>>>> >>> >> > > > > >> > criteria.
>>>> >>> >> > > > > >> > > > The
>>>> >>> >> > > > > >> > > > > big
>>>> >>> >> > > > > >> > > > > >> > > > > exemption
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > we've
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> had
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > to
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > this was the
>>>> >>> >> > hbase-spark
>>>> >>> >> > > > > >> > > integration,
>>>> >>> >> > > > > >> > > > > where
>>>> >>> >> > > > > >> > > > > >> > we
>>>> >>> >> > > > > >> > > > > >> > > > all
>>>> >>> >> > > > > >> > > > > >> > > > > >> > agreed
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > it
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > could
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > land in
>>>> master
>>>> >>> >> because
>>>> >>> >> > it
>>>> >>> >> > > > was
>>>> >>> >> > > > > >> very
>>>> >>> >> > > > > >> > > > well
>>>> >>> >> > > > > >> > > > > >> > > isolated
>>>> >>> >> > > > > >> > > > > >> > > > > (the
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > slide
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > away
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > from
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > including
>>>> docs
>>>> >>> as a
>>>> >>> >> > > > > first-class
>>>> >>> >> > > > > >> > part
>>>> >>> >> > > > > >> > > > of
>>>> >>> >> > > > > >> > > > > >> > > building
>>>> >>> >> > > > > >> > > > > >> > > > up
>>>> >>> >> > > > > >> > > > > >> > > > > >> that
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > integration
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > has led me
>>>> to
>>>> >>> doubt
>>>> >>> >> the
>>>> >>> >> > > > > wisdom
>>>> >>> >> > > > > >> of
>>>> >>> >> > > > > >> > > this
>>>> >>> >> > > > > >> > > > > >> > > decision).
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > We've also
>>>> been
>>>> >>> >> > treating
>>>> >>> >> > > > > >> inclusion
>>>> >>> >> > > > > >> > > in
>>>> >>> >> > > > > >> > > > a
>>>> >>> >> > > > > >> > > > > >> > > "probably
>>>> >>> >> > > > > >> > > > > >> > > > > >> will
>>>> >>> >> > > > > >> > > > > >> > > > > >> > be
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > released
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > to
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > downstream"
>>>> >>> branches
>>>> >>> >> > as a
>>>> >>> >> > > > > >> higher
>>>> >>> >> > > > > >> > > bar,
>>>> >>> >> > > > > >> > > > > >> > requiring
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't
>>>> >>> moderately
>>>> >>> >> > impact
>>>> >>> >> > > > > >> > > performance
>>>> >>> >> > > > > >> > > > > when
>>>> >>> >> > > > > >> > > > > >> > the
>>>> >>> >> > > > > >> > > > > >> > > > > >> feature
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > isn't
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> in
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > use
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't
>>>> severely
>>>> >>> >> impact
>>>> >>> >> > > > > >> > performance
>>>> >>> >> > > > > >> > > > when
>>>> >>> >> > > > > >> > > > > >> the
>>>> >>> >> > > > > >> > > > > >> > > > > feature
>>>> >>> >> > > > > >> > > > > >> > > > > >> is
>>>> >>> >> > > > > >> > > > > >> > > > > >> > in
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> use
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * either
>>>> >>> >> default-to-on
>>>> >>> >> > or
>>>> >>> >> > > > > show
>>>> >>> >> > > > > >> > > enough
>>>> >>> >> > > > > >> > > > > >> demand
>>>> >>> >> > > > > >> > > > > >> > to
>>>> >>> >> > > > > >> > > > > >> > > > > >> believe
>>>> >>> >> > > > > >> > > > > >> > > > > >> > a
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > non-trivial
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > number of
>>>> folks
>>>> >>> will
>>>> >>> >> > turn
>>>> >>> >> > > > the
>>>> >>> >> > > > > >> > > feature
>>>> >>> >> > > > > >> > > > on
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The above
>>>> has
>>>> >>> kept
>>>> >>> >> MOB
>>>> >>> >> > > and
>>>> >>> >> > > > > >> > > hbase-spark
>>>> >>> >> > > > > >> > > > > >> > > > integration
>>>> >>> >> > > > > >> > > > > >> > > > > >> out
>>>> >>> >> > > > > >> > > > > >> > > > > >> > of
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > branch-1,
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > presumably
>>>> while
>>>> >>> >> > they've
>>>> >>> >> > > > > >> "gotten
>>>> >>> >> > > > > >> > > more
>>>> >>> >> > > > > >> > > > > >> stable"
>>>> >>> >> > > > > >> > > > > >> > > in
>>>> >>> >> > > > > >> > > > > >> > > > > >> master
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > from
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > the
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > odd
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > vendor
>>>> >>> inclusion.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Are we
>>>> going to
>>>> >>> >> have a
>>>> >>> >> > > 2.0
>>>> >>> >> > > > > >> release
>>>> >>> >> > > > > >> > > > > before
>>>> >>> >> > > > > >> > > > > >> the
>>>> >>> >> > > > > >> > > > > >> > > end
>>>> >>> >> > > > > >> > > > > >> > > > > of
>>>> >>> >> > > > > >> > > > > >> > > > > >> the
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> year?
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > We're
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > coming up
>>>> on 1.5
>>>> >>> >> years
>>>> >>> >> > > > since
>>>> >>> >> > > > > >> the
>>>> >>> >> > > > > >> > > > > release of
>>>> >>> >> > > > > >> > > > > >> > > > version
>>>> >>> >> > > > > >> > > > > >> > > > > >> 1.0;
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> seems
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > like
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > it's about
>>>> time,
>>>> >>> >> > though I
>>>> >>> >> > > > > >> haven't
>>>> >>> >> > > > > >> > > seen
>>>> >>> >> > > > > >> > > > > any
>>>> >>> >> > > > > >> > > > > >> > > > concrete
>>>> >>> >> > > > > >> > > > > >> > > > > >> > plans
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> this
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > year.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Presuming
>>>> we are
>>>> >>> >> going
>>>> >>> >> > to
>>>> >>> >> > > > > have
>>>> >>> >> > > > > >> one
>>>> >>> >> > > > > >> > > by
>>>> >>> >> > > > > >> > > > > the
>>>> >>> >> > > > > >> > > > > >> end
>>>> >>> >> > > > > >> > > > > >> > > of
>>>> >>> >> > > > > >> > > > > >> > > > > the
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > year, it
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > seems a
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > bit close to
>>>> >>> still
>>>> >>> >> be
>>>> >>> >> > > > adding
>>>> >>> >> > > > > in
>>>> >>> >> > > > > >> > > > > "features
>>>> >>> >> > > > > >> > > > > >> > that
>>>> >>> >> > > > > >> > > > > >> > > > need
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > maturing"
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > on
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > the
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > branch.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The lack of
>>>> a
>>>> >>> >> concrete
>>>> >>> >> > > plan
>>>> >>> >> > > > > for
>>>> >>> >> > > > > >> > 2.0
>>>> >>> >> > > > > >> > > > > keeps
>>>> >>> >> > > > > >> > > > > >> me
>>>> >>> >> > > > > >> > > > > >> > > from
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > considering
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > these
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > things
>>>> blocker
>>>> >>> at
>>>> >>> >> the
>>>> >>> >> > > > moment.
>>>> >>> >> > > > > >> But
>>>> >>> >> > > > > >> > I
>>>> >>> >> > > > > >> > > > know
>>>> >>> >> > > > > >> > > > > >> > first
>>>> >>> >> > > > > >> > > > > >> > > > hand
>>>> >>> >> > > > > >> > > > > >> > > > > >> how
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > much
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > trouble
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > folks have
>>>> had
>>>> >>> with
>>>> >>> >> > other
>>>> >>> >> > > > > >> features
>>>> >>> >> > > > > >> > > > that
>>>> >>> >> > > > > >> > > > > >> have
>>>> >>> >> > > > > >> > > > > >> > > gone
>>>> >>> >> > > > > >> > > > > >> > > > > >> into
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > downstream
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > facing
>>>> releases
>>>> >>> >> without
>>>> >>> >> > > > > >> robustness
>>>> >>> >> > > > > >> > > > > checks
>>>> >>> >> > > > > >> > > > > >> > (i.e.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > replication),
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > and
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > I'm
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > concerned
>>>> about
>>>> >>> what
>>>> >>> >> > > we're
>>>> >>> >> > > > > >> setting
>>>> >>> >> > > > > >> > > up
>>>> >>> >> > > > > >> > > > if
>>>> >>> >> > > > > >> > > > > >> 2.0
>>>> >>> >> > > > > >> > > > > >> > > goes
>>>> >>> >> > > > > >> > > > > >> > > > > out
>>>> >>> >> > > > > >> > > > > >> > > > > >> > with
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> this
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > feature in
>>>> its
>>>> >>> >> current
>>>> >>> >> > > > state.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > --
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > Best regards,
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >    - Andy
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > Problems worthy of
>>>> >>> attack
>>>> >>> >> > prove
>>>> >>> >> > > > > their
>>>> >>> >> > > > > >> > > worth
>>>> >>> >> > > > > >> > > > by
>>>> >>> >> > > > > >> > > > > >> > > hitting
>>>> >>> >> > > > > >> > > > > >> > > > > >> back. -
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > Piet
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > Hein
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > (via Tom White)
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > --
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > Best regards,
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >    - Andy
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > Problems worthy of
>>>> attack
>>>> >>> >> prove
>>>> >>> >> > > their
>>>> >>> >> > > > > >> worth
>>>> >>> >> > > > > >> > by
>>>> >>> >> > > > > >> > > > > >> hitting
>>>> >>> >> > > > > >> > > > > >> > > > back.
>>>> >>> >> > > > > >> > > > > >> > > > > -
>>>> >>> >> > > > > >> > > > > >> > > > > >> > Piet
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> Hein
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > (via Tom White)
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >>
>>>> >>> >> > > > > >> > > > > >> > > > > >> > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> >
>>>> >>> >> > > > > >> > > > > >> > > > > >>
>>>> >>> >> > > > > >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > >
>>>> >>> >> > > > > >> > > > > >> > > >
>>>> >>> >> > > > > >> > > > > >> > >
>>>> >>> >> > > > > >> > > > > >> >
>>>> >>> >> > > > > >> > > > > >>
>>>> >>> >> > > > > >> > > > >
>>>> >>> >> > > > > >> > > > >
>>>> >>> >> > > > > >> > > > >
>>>> >>> >> > > > > >> > > > > --
>>>> >>> >> > > > > >> > > > > busbey
>>>> >>> >> > > > > >> > > > >
>>>> >>> >> > > > > >> > > >
>>>> >>> >> > > > > >> > >
>>>> >>> >> > > > > >> >
>>>> >>> >> > > > > >>
>>>> >>> >> > > > > >
>>>> >>> >> > > > > >
>>>> >>> >> > > > >
>>>> >>> >> > > >
>>>> >>> >> > >
>>>> >>> >> >
>>>> >>> >>
>>>> >>> >>
>>>> >>> >>
>>>> >>> >> --
>>>> >>> >>
>>>> >>> >> -- Appy
>>>> >>> >>
>>>> >>> >
>>>> >>> >
>>>> >>>
>>>> >>
>>>> >>
>>>> >
>>>>
>>>
>>>
>>



-- 
busbey

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Stack <st...@duboce.net>]

I put some notes up in https://issues.apache.org/jira/browse/HBASE-7912
St.Ack

On Wed, Oct 12, 2016 at 2:03 PM, Stack <st...@duboce.net> wrote:

> On Wed, Oct 12, 2016 at 1:58 PM, Stack <st...@duboce.net> wrote:
>
>> Fails again when I do this:
>>
>> $ mvn clean install -DskipTests assembly:single
>> ....
>>
>>
> Ok. Works if I use mvn 3.3.x.
> St.Ack
>
>
>
>
>
>>
>> [INFO] ------------------------------------------------------------
>> ------------
>> [INFO] Building Apache HBase - Server 2.0.0-SNAPSHOT
>> [INFO] ------------------------------------------------------------
>> ------------
>> Downloading: file:/home/stack/hbase.git/hbase-server/src/main/site/resour
>> ces/repo/org/apache/hadoop/hadoop-distcp/2.7.1/hadoop-distcp-2.7.1.pom
>> Exception in thread "pool-1-thread-1" ------------------------------
>> ---------------------
>> constituent[0]: file:/usr/share/maven/lib/aether-spi.jar
>> constituent[1]: file:/usr/share/maven/lib/maven-settings-builder-3.x.jar
>> constituent[2]: file:/usr/share/maven/lib/maven-artifact-3.x.jar
>> constituent[3]: file:/usr/share/maven/lib/commons-httpclient.jar
>> constituent[4]: file:/usr/share/maven/lib/wagon-provider-api.jar
>> constituent[5]: file:/usr/share/maven/lib/maven-core-3.x.jar
>> constituent[6]: file:/usr/share/maven/lib/aether-impl.jar
>> constituent[7]: file:/usr/share/maven/lib/sisu-inject-bean.jar
>> constituent[8]: file:/usr/share/maven/lib/sisu-inject-plexus.jar
>> constituent[9]: file:/usr/share/maven/lib/wagon-file.jar
>> constituent[10]: file:/usr/share/maven/lib/guava.jar
>> constituent[11]: file:/usr/share/maven/lib/maven-compat-3.x.jar
>> constituent[12]: file:/usr/share/maven/lib/maven-model-3.x.jar
>> constituent[13]: file:/usr/share/maven/lib/plexus-interpolation.jar
>> constituent[14]: file:/usr/share/maven/lib/commons-codec.jar
>> constituent[15]: file:/usr/share/maven/lib/aether-util.jar
>> constituent[16]: file:/usr/share/maven/lib/aether-api.jar
>> constituent[17]: file:/usr/share/maven/lib/maven-model-builder-3.x.jar
>> constituent[18]: file:/usr/share/maven/lib/mave
>> n-repository-metadata-3.x.jar
>> constituent[19]: file:/usr/share/maven/lib/maven-settings-3.x.jar
>> constituent[20]: file:/usr/share/maven/lib/plex
>> us-component-annotations.jar
>> constituent[21]: file:/usr/share/maven/lib/commons-cli.jar
>> constituent[22]: file:/usr/share/maven/lib/maven-embedder-3.x.jar
>> constituent[23]: file:/usr/share/maven/lib/maven-plugin-api-3.x.jar
>> constituent[24]: file:/usr/share/maven/lib/wagon-http-shaded.jar
>> constituent[25]: file:/usr/share/maven/lib/commons-logging.jar
>> constituent[26]: file:/usr/share/maven/lib/plexus-sec-dispatcher.jar
>> constituent[27]: file:/usr/share/maven/lib/aether-connector-wagon.jar
>> constituent[28]: file:/usr/share/maven/lib/plexus-utils.jar
>> constituent[29]: file:/usr/share/maven/lib/sisu-guice.jar
>> constituent[30]: file:/usr/share/maven/lib/plexus-cipher.jar
>> constituent[31]: file:/usr/share/maven/lib/maven-aether-provider-3.x.jar
>> ---------------------------------------------------
>> java.lang.NoClassDefFoundError: org/apache/commons/lang/StringUtils
>> at org.apache.maven.wagon.providers.file.FileWagon.resolveDesti
>> nationPath(FileWagon.java:206)
>> at org.apache.maven.wagon.providers.file.FileWagon.resourceExis
>> ts(FileWagon.java:265)
>> at org.sonatype.aether.connector.wagon.WagonRepositoryConnector
>> $GetTask.run(WagonRepositoryConnector.java:577)
>> at org.sonatype.aether.util.concurrency.RunnableErrorForwarder$
>> 1.run(RunnableErrorForwarder.java:60)
>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
>> Executor.java:1142)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
>> lExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:745)
>> Caused by: java.lang.ClassNotFoundException:
>> org.apache.commons.lang.StringUtils
>> at org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy.
>> loadClass(SelfFirstStrategy.java:50)
>> at org.codehaus.plexus.classworlds.realm.ClassRealm.unsynchroni
>> zedLoadClass(ClassRealm.java:259)
>> at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(
>> ClassRealm.java:235)
>> at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(
>> ClassRealm.java:227)
>> ... 7 more
>> Exception in thread "main" java.lang.NoClassDefFoundError:
>> org/apache/commons/lang/StringUtils
>> at org.apache.maven.wagon.providers.file.FileWagon.resolveDesti
>> nationPath(FileWagon.java:206)
>> at org.apache.maven.wagon.providers.file.FileWagon.resourceExis
>> ts(FileWagon.java:265)
>> at org.sonatype.aether.connector.wagon.WagonRepositoryConnector
>> $GetTask.run(WagonRepositoryConnector.java:577)
>> at org.sonatype.aether.util.concurrency.RunnableErrorForwarder$
>> 1.run(RunnableErrorForwarder.java:60)
>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
>> Executor.java:1142)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
>> lExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:745)
>> Caused by: java.lang.ClassNotFoundException:
>> org.apache.commons.lang.StringUtils
>> at org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy.
>> loadClass(SelfFirstStrategy.java:50)
>> at org.codehaus.plexus.classworlds.realm.ClassRealm.unsynchroni
>> zedLoadClass(ClassRealm.java:259)
>> at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(
>> ClassRealm.java:235)
>> at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(
>> ClassRealm.java:227)
>>
>>
>>
>> Does it work for you lot? This is branch checkout.
>>
>> I want assembly to work so I can distribute the build around a cluster.
>>
>> Thanks,
>> St.Ack.
>>
>>
>>
>> On Wed, Oct 12, 2016 at 1:52 PM, Stack <st...@duboce.net> wrote:
>>
>>> I did a super-clean and recheck out. Now it works. Sorry for the noise.
>>> St.Ack
>>>
>>> On Wed, Oct 12, 2016 at 1:10 PM, Vladimir Rodionov <
>>> vladrodionov@gmail.com> wrote:
>>>
>>>> checked out HBASE-7912
>>>>
>>>> ran:
>>>>
>>>> mvn clean install -DskipTests
>>>>
>>>> successfully.
>>>>
>>>> -Vlad
>>>>
>>>> On Wed, Oct 12, 2016 at 1:01 PM, Vladimir Rodionov <
>>>> vladrodionov@gmail.com>
>>>> wrote:
>>>>
>>>> > I usually use:
>>>> >
>>>> > mvn clean install -DskipTests
>>>> >
>>>> > and it works.
>>>> >
>>>> > On Wed, Oct 12, 2016 at 1:01 PM, Vladimir Rodionov <
>>>> vladrodionov@gmail.com
>>>> > > wrote:
>>>> >
>>>> >> Michael,
>>>> >>
>>>> >> you can try master + latest patch on HBASE-14123 (v29). No need to
>>>> use
>>>> >> HBASE-7912 branch. I will double check HBASE-7912.
>>>> >>
>>>> >> -Vlad
>>>> >>
>>>> >> On Wed, Oct 12, 2016 at 12:31 PM, Stack <st...@duboce.net> wrote:
>>>> >>
>>>> >>> More info:
>>>> >>>
>>>> >>> stack@ve0524:~/hbase.git$ git checkout origin/HBASE-7912 -b 7912v2
>>>> >>> Branch 7912v2 set up to track remote branch HBASE-7912 from origin.
>>>> >>> Switched to a new branch '7912v2'
>>>> >>> stack@ve0524:~/hbase.git$ java -version
>>>> >>> java version "1.8.0_101"
>>>> >>> Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
>>>> >>> Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode)
>>>> >>> stack@ve0524:~/hbase.git$ mvn clean install -DskipTests &>
>>>> /tmp/out.txt
>>>> >>>
>>>> >>> ...
>>>> >>>
>>>> >>> St.Ack
>>>> >>>
>>>> >>>
>>>> >>> On Wed, Oct 12, 2016 at 12:29 PM, Stack <st...@duboce.net> wrote:
>>>> >>>
>>>> >>> > Interesting. When I try it fails w/ below:
>>>> >>> >
>>>> >>> > [INFO] 26 warnings
>>>> >>> > 322 [INFO] ------------------------------
>>>> >>> -------------------------------
>>>> >>> > 323 [INFO] ------------------------------
>>>> >>> -------------------------------
>>>> >>> > 324 [ERROR] COMPILATION ERROR :
>>>> >>> > 325 [INFO] ------------------------------
>>>> >>> -------------------------------
>>>> >>> > 326 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.j
>>>> ava:[48,8]
>>>> >>> > org.apache.hadoop.hbase.io.encoding.rowindexV2.RowIndexCodecV2
>>>> is not
>>>> >>> > abstract and does not override abstract method createSeeker(org.ap
>>>> >>> >  ache.hadoop.hbase.CellComparator,org.apache.hadoop.hbase.io.
>>>> >>> encoding.HFileBlockDecodingContext)
>>>> >>> > in org.apache.hadoop.hbase.io.encoding.DataBlockEncoder
>>>> >>> > 327 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.j
>>>> >>> ava:[143,3]
>>>> >>> > method does not override or implement a method from a supertype
>>>> >>> > 328 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.j
>>>> >>> ava:[147,29]
>>>> >>> > incompatible types: java.nio.ByteBuffer cannot be converted to
>>>> >>> > org.apache.hadoop.hbase.nio.ByteBuff
>>>> >>> > 329 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.j
>>>> >>> ava:[148,33]
>>>> >>> > cannot find symbol
>>>> >>> > 330   symbol:   method getKeyDeepCopy()
>>>> >>> > 331   location: variable seeker of type
>>>> org.apache.hadoop.hbase.io.
>>>> >>> > encoding.DataBlockEncoder.EncodedSeeker
>>>> >>> > 332 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.j
>>>> >>> ava:[153,3]
>>>> >>> > method does not override or implement a method from a supertype
>>>> >>> > 333 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV1/RowIndexCodecV1.j
>>>> ava:[45,8]
>>>> >>> > org.apache.hadoop.hbase.io.encoding.rowindexV1.RowIndexCodecV1
>>>> is not
>>>> >>> > abstract and does not override abstract method createSeeker(org.ap
>>>> >>> >  ache.hadoop.hbase.CellComparator,org.apache.hadoop.hbase.io.
>>>> >>> encoding.HFileBlockDecodingContext)
>>>> >>> > in org.apache.hadoop.hbase.io.encoding.DataBlockEncoder
>>>> >>> > 334 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV1/RowIndexCodecV1.j
>>>> >>> ava:[145,3]
>>>> >>> > method does not override or implement a method from a supertype
>>>> >>> > 335 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV1/RowIndexCodecV1.j
>>>> >>> ava:[158,3]
>>>> >>> > method does not override or implement a method from a supertype
>>>> >>> > 336 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>>>> >>> java:[46,8]
>>>> >>> > org.apache.hadoop.hbase.io.encoding.rowindexV2.RowIndexSeekerV2
>>>> is not
>>>> >>> > abstract and does not override abstract method compareKey(org.ap
>>>> >>> >  ache.hadoop.hbase.CellComparator,org.apache.hadoop.hbase.Cell)
>>>> in
>>>> >>> > org.apache.hadoop.hbase.io.encoding.DataBlockEncoder.Encoded
>>>> Seeker
>>>> >>> > 337 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>>>> >>> java:[79,3]
>>>> >>> > method does not override or implement a method from a supertype
>>>> >>> > 338 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>>>> >>> java:[117,3]
>>>> >>> > method does not override or implement a method from a supertype
>>>> >>> > 339 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>>>> >>> java:[190,3]
>>>> >>> > method does not override or implement a method from a supertype
>>>> >>> > 340 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>>>> >>> java:[214,3]
>>>> >>> > method does not override or implement a method from a supertype
>>>> >>> > 341 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>>>> >>> java:[349,3]
>>>> >>> > method does not override or implement a method from a supertype
>>>> >>> > 342 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>>>> >>> java:[355,3]
>>>> >>> > method does not override or implement a method from a supertype
>>>> >>> > 343 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>>>> >>> java:[421,36]
>>>> >>> > no suitable method found for uncompressTags(java.nio.
>>>> >>> > ByteBuffer,byte[],int,int)
>>>> >>> > 344     method org.apache.hadoop.hbase.io.TagCompressionContext.
>>>> >>> > uncompressTags(java.io.InputStream,byte[],int,int) is not
>>>> applicable
>>>> >>> > 345       (argument mismatch; java.nio.ByteBuffer cannot be
>>>> converted
>>>> >>> to
>>>> >>> > java.io.InputStream)
>>>> >>> > 346     method org.apache.hadoop.hbase.io.TagCompressionContext.
>>>> >>> > uncompressTags(org.apache.hadoop.hbase.nio.ByteBuff,byte[],int,int)
>>>> is
>>>> >>> > not applicable
>>>> >>> > 347       (argument mismatch; java.nio.ByteBuffer cannot be
>>>> converted
>>>> >>> to
>>>> >>> > org.apache.hadoop.hbase.nio.ByteBuff)
>>>> >>> >
>>>> >>> > ....
>>>> >>> >
>>>> >>> > St.Ack
>>>> >>> >
>>>> >>> > On Tue, Oct 11, 2016 at 5:12 PM, Apekshit Sharma <
>>>> appy@cloudera.com>
>>>> >>> > wrote:
>>>> >>> >
>>>> >>> >> @stack, it compiled for me.
>>>> >>> >>
>>>> >>> >> Also tried few commands, and have to say, it's well designed
>>>> from user
>>>> >>> >> commands perspective.
>>>> >>> >>
>>>> >>> >> On Tue, Oct 11, 2016 at 5:08 PM, Stack <st...@duboce.net> wrote:
>>>> >>> >>
>>>> >>> >> > On Tue, Oct 11, 2016 at 1:50 PM, Vladimir Rodionov <
>>>> >>> >> vladrodionov@gmail.com
>>>> >>> >> > >
>>>> >>> >> > wrote:
>>>> >>> >> >
>>>> >>> >> > > Michael,
>>>> >>> >> > >
>>>> >>> >> > > Its in HBASE-7912
>>>> >>> >> > >
>>>> >>> >> > > This is tip of git log:
>>>> >>> >> > >
>>>> >>> >> > > commit a072f6f49a26a7259ff2aaef6cb56d85eb592482
>>>> >>> >> > > Author: Frank Welsch <fw...@jps.net>
>>>> >>> >> > > Date:   Fri Sep 23 18:00:42 2016 -0400
>>>> >>> >> > >
>>>> >>> >> > >     HBASE-16574 Book updates for backup and restore
>>>> >>> >> > >
>>>> >>> >> > > commit b14e2ab1c24e65ff88dd4c579acf83cb4ed0605e
>>>> >>> >> > > Author: tedyu <yu...@gmail.com>
>>>> >>> >> > > Date:   Wed Oct 5 16:29:40 2016 -0700
>>>> >>> >> > >
>>>> >>> >> > >     HBASE-16727 Backup refactoring: remove MR dependencies
>>>> from
>>>> >>> >> HMaster
>>>> >>> >> > > (Vladimir Rodionov)
>>>> >>> >> > >
>>>> >>> >> > >
>>>> >>> >> > Thanks. I have that. I tried it and it doesn't compile for me.
>>>> Does
>>>> >>> it
>>>> >>> >> > compile for you?
>>>> >>> >> > Thanks,
>>>> >>> >> > M
>>>> >>> >> >
>>>> >>> >> >
>>>> >>> >> >
>>>> >>> >> > > -Vlad
>>>> >>> >> > >
>>>> >>> >> > > On Tue, Oct 11, 2016 at 1:44 PM, Stack <st...@duboce.net>
>>>> wrote:
>>>> >>> >> > >
>>>> >>> >> > > > Which branch do I check out to try it? HBASE-7912 is not
>>>> it. I
>>>> >>> don't
>>>> >>> >> > see
>>>> >>> >> > > an
>>>> >>> >> > > > HBASE-16727...
>>>> >>> >> > > > Thanks,
>>>> >>> >> > > > M
>>>> >>> >> > > >
>>>> >>> >> > > >
>>>> >>> >> > > > On Tue, Oct 11, 2016 at 11:41 AM, Vladimir Rodionov <
>>>> >>> >> > > > vladrodionov@gmail.com>
>>>> >>> >> > > > wrote:
>>>> >>> >> > > >
>>>> >>> >> > > > > The last patch is on review board:
>>>> >>> >> > > > > https://reviews.apache.org/r/52748
>>>> >>> >> > > > >
>>>> >>> >> > > > > On Mon, Oct 10, 2016 at 3:52 PM, Vladimir Rodionov <
>>>> >>> >> > > > vladrodionov@gmail.com
>>>> >>> >> > > > > >
>>>> >>> >> > > > > wrote:
>>>> >>> >> > > > >
>>>> >>> >> > > > > > >> How hard to put in an hbase-backup module?
>>>> hbase-server
>>>> >>> is
>>>> >>> >> fat
>>>> >>> >> > > > enough
>>>> >>> >> > > > > > >> already. Could be done as a follow-up.
>>>> >>> >> > > > > >
>>>> >>> >> > > > > > https://issues.apache.org/jira/browse/HBASE-16727?
>>>> >>> >> > > > > > focusedCommentId=15531237&page=com.atlassian.jira.
>>>> >>> >> > > > > > plugin.system.issuetabpanels:c
>>>> >>> omment-tabpanel#comment-15531237
>>>> >>> >> > > > > >
>>>> >>> >> > > > > > Can we do merge first? Then we can discuss separate
>>>> module.
>>>> >>> >> > > > > >
>>>> >>> >> > > > > > -Vlad
>>>> >>> >> > > > > >
>>>> >>> >> > > > > > On Mon, Oct 10, 2016 at 3:44 PM, Ted Yu <
>>>> >>> yuzhihong@gmail.com>
>>>> >>> >> > wrote:
>>>> >>> >> > > > > >
>>>> >>> >> > > > > >> Looks like the first quote was cut off.
>>>> >>> >> > > > > >> The original sentence was:
>>>> >>> >> > > > > >>
>>>> >>> >> > > > > >> bq. no mapreduce job launched from master or region
>>>> server.
>>>> >>> >> > > > > >>
>>>> >>> >> > > > > >> mapreduce job is launched from the node where command
>>>> line
>>>> >>> >> tool is
>>>> >>> >> > > > run.
>>>> >>> >> > > > > >>
>>>> >>> >> > > > > >> On Mon, Oct 10, 2016 at 3:38 PM, Stack <
>>>> stack@duboce.net>
>>>> >>> >> wrote:
>>>> >>> >> > > > > >>
>>>> >>> >> > > > > >> > bq. launched from master or region server.
>>>> >>> >> > > > > >> >
>>>> >>> >> > > > > >> > What does this mean please? Has to be run from
>>>> Master or
>>>> >>> >> > > > RegionServer?
>>>> >>> >> > > > > >> Can
>>>> >>> >> > > > > >> > it be run from another node altogether?
>>>> >>> >> > > > > >> >
>>>> >>> >> > > > > >> > On Mon, Oct 10, 2016 at 1:44 PM, Vladimir Rodionov <
>>>> >>> >> > > > > >> vladrodionov@gmail.com
>>>> >>> >> > > > > >> > >
>>>> >>> >> > > > > >> > wrote:
>>>> >>> >> > > > > >> >
>>>> >>> >> > > > > >> > > >> mapreduce dependency has been moved to client
>>>> side
>>>> >>> - no
>>>> >>> >> > > > mapreduce
>>>> >>> >> > > > > >> job
>>>> >>> >> > > > > >> > >
>>>> >>> >> > > > > >> > > 1. We have no code in the client module anymore,
>>>> due to
>>>> >>> >> > > dependency
>>>> >>> >> > > > > on
>>>> >>> >> > > > > >> > > internal server API (HFile and WAL access).
>>>> >>> >> > > > > >> > > 2. Backup/ restore are client - driven
>>>> operations, but
>>>> >>> all
>>>> >>> >> the
>>>> >>> >> > > > code
>>>> >>> >> > > > > >> > resides
>>>> >>> >> > > > > >> > > in the server module
>>>> >>> >> > > > > >> > >
>>>> >>> >> > > > > >> >
>>>> >>> >> > > > > >> > How hard to put in an hbase-backup module?
>>>> hbase-server
>>>> >>> is
>>>> >>> >> fat
>>>> >>> >> > > > enough
>>>> >>> >> > > > > >> > already. Could be done as a follow-up.
>>>> >>> >> > > > > >> >
>>>> >>> >> > > > > >> > Thanks,
>>>> >>> >> > > > > >> > St.Ack
>>>> >>> >> > > > > >> >
>>>> >>> >> > > > > >> >
>>>> >>> >> > > > > >> >
>>>> >>> >> > > > > >> > > 3. No MR in Master, no procedure - driven
>>>> execution.
>>>> >>> >> > > > > >> > > 4. Old good MR from command-line.
>>>> >>> >> > > > > >> > > 5. Security was simplified and now only
>>>> super-user is
>>>> >>> >> allowed
>>>> >>> >> > to
>>>> >>> >> > > > run
>>>> >>> >> > > > > >> > > backup/restores.
>>>> >>> >> > > > > >> > > 6. HBase Backup API was gone due to 1. Now only
>>>> >>> >> command-line
>>>> >>> >> > > > access
>>>> >>> >> > > > > to
>>>> >>> >> > > > > >> > > backup tools.
>>>> >>> >> > > > > >> > >
>>>> >>> >> > > > > >> > > These consequences of refactoring has been
>>>> discussed in
>>>> >>> >> > > > HBASE-16727.
>>>> >>> >> > > > > >> > >
>>>> >>> >> > > > > >> > > -Vlad
>>>> >>> >> > > > > >> > >
>>>> >>> >> > > > > >> > >
>>>> >>> >> > > > > >> > >
>>>> >>> >> > > > > >> > >
>>>> >>> >> > > > > >> > > On Mon, Oct 10, 2016 at 1:31 PM, Ted Yu <
>>>> >>> >> yuzhihong@gmail.com>
>>>> >>> >> > > > > wrote:
>>>> >>> >> > > > > >> > >
>>>> >>> >> > > > > >> > > > Reviving this thread.
>>>> >>> >> > > > > >> > > >
>>>> >>> >> > > > > >> > > > The following has taken place:
>>>> >>> >> > > > > >> > > >
>>>> >>> >> > > > > >> > > > mapreduce dependency has been moved to client
>>>> side -
>>>> >>> no
>>>> >>> >> > > > mapreduce
>>>> >>> >> > > > > >> job
>>>> >>> >> > > > > >> > > > launched from master or region server.
>>>> >>> >> > > > > >> > > > document patch (HBASE-16574) has been
>>>> integrated.
>>>> >>> >> > > > > >> > > > Updated mega patch has been attached to
>>>> HBASE-14123:
>>>> >>> this
>>>> >>> >> > > covers
>>>> >>> >> > > > > the
>>>> >>> >> > > > > >> > > > refactor in #1 above and the protobuf 3 merge.
>>>> >>> >> > > > > >> > > >
>>>> >>> >> > > > > >> > > > If community has more feedback on the merge
>>>> >>> proposal, I
>>>> >>> >> > would
>>>> >>> >> > > > love
>>>> >>> >> > > > > >> to
>>>> >>> >> > > > > >> > > hear
>>>> >>> >> > > > > >> > > > it.
>>>> >>> >> > > > > >> > > >
>>>> >>> >> > > > > >> > > > Thanks
>>>> >>> >> > > > > >> > > >
>>>> >>> >> > > > > >> > > > On Thu, Sep 22, 2016 at 10:31 AM, Sean Busbey <
>>>> >>> >> > > > > busbey@cloudera.com>
>>>> >>> >> > > > > >> > > wrote:
>>>> >>> >> > > > > >> > > >
>>>> >>> >> > > > > >> > > > > I'd like to see the docs proposed on
>>>> HBASE-16574
>>>> >>> >> > integrated
>>>> >>> >> > > > into
>>>> >>> >> > > > > >> our
>>>> >>> >> > > > > >> > > > > project's documentation prior to merge.
>>>> >>> >> > > > > >> > > > >
>>>> >>> >> > > > > >> > > > > On Thu, Sep 22, 2016 at 9:02 AM, Ted Yu <
>>>> >>> >> > > yuzhihong@gmail.com>
>>>> >>> >> > > > > >> wrote:
>>>> >>> >> > > > > >> > > > > > This feature can be marked experimental due
>>>> to
>>>> >>> some
>>>> >>> >> > > > > limitations
>>>> >>> >> > > > > >> > such
>>>> >>> >> > > > > >> > > as
>>>> >>> >> > > > > >> > > > > > security.
>>>> >>> >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > > Your previous round of comments have been
>>>> >>> addressed.
>>>> >>> >> > > > > >> > > > > > Command line tool has gone through:
>>>> >>> >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > > HBASE-16620 Fix backup command-line tool
>>>> >>> usability
>>>> >>> >> > issues
>>>> >>> >> > > > > >> > > > > > HBASE-16655 hbase backup describe with
>>>> incorrect
>>>> >>> >> backup
>>>> >>> >> > id
>>>> >>> >> > > > > >> results
>>>> >>> >> > > > > >> > in
>>>> >>> >> > > > > >> > > > NPE
>>>> >>> >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > > The updated doc has been attached to
>>>> HBASE-16574.
>>>> >>> >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > > Cheers
>>>> >>> >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > > On Thu, Sep 22, 2016 at 8:53 AM, Stack <
>>>> >>> >> > stack@duboce.net>
>>>> >>> >> > > > > >> wrote:
>>>> >>> >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> On Wed, Sep 21, 2016 at 7:43 AM, Ted Yu <
>>>> >>> >> > > > yuzhihong@gmail.com
>>>> >>> >> > > > > >
>>>> >>> >> > > > > >> > > wrote:
>>>> >>> >> > > > > >> > > > > >>
>>>> >>> >> > > > > >> > > > > >> > Are there more (review) comments ?
>>>> >>> >> > > > > >> > > > > >> >
>>>> >>> >> > > > > >> > > > > >> >
>>>> >>> >> > > > > >> > > > > >> Are outstanding comments addressed?
>>>> >>> >> > > > > >> > > > > >>
>>>> >>> >> > > > > >> > > > > >> I don't see answer to my 'is this
>>>> >>> experimental/will
>>>> >>> >> it
>>>> >>> >> > be
>>>> >>> >> > > > > >> marked
>>>> >>> >> > > > > >> > > > > >> experimental' question.
>>>> >>> >> > > > > >> > > > > >>
>>>> >>> >> > > > > >> > > > > >> I ran into some issues trying to use the
>>>> >>> feature and
>>>> >>> >> > > > > suggested
>>>> >>> >> > > > > >> > that
>>>> >>> >> > > > > >> > > a
>>>> >>> >> > > > > >> > > > > >> feature likes this needs polish else it'll
>>>> just
>>>> >>> rot,
>>>> >>> >> > > > unused.
>>>> >>> >> > > > > >> Has
>>>> >>> >> > > > > >> > > > polish
>>>> >>> >> > > > > >> > > > > >> been applied? All ready for another 'user'
>>>> test?
>>>> >>> >> > Suggest
>>>> >>> >> > > > that
>>>> >>> >> > > > > >> you
>>>> >>> >> > > > > >> > > > update
>>>> >>> >> > > > > >> > > > > >> here going forward for the benefit of those
>>>> >>> trying
>>>> >>> >> to
>>>> >>> >> > > > follow
>>>> >>> >> > > > > >> along
>>>> >>> >> > > > > >> > > and
>>>> >>> >> > > > > >> > > > > who
>>>> >>> >> > > > > >> > > > > >> are not watching JIRA change fly-by.
>>>> >>> >> > > > > >> > > > > >>
>>>> >>> >> > > > > >> > > > > >> It looks like doc got a revision -- I have
>>>> to
>>>> >>> check
>>>> >>> >> --
>>>> >>> >> > to
>>>> >>> >> > > > > take
>>>> >>> >> > > > > >> on
>>>> >>> >> > > > > >> > > > > >> suggestion made above but again, suggest,
>>>> that
>>>> >>> this
>>>> >>> >> > > thread
>>>> >>> >> > > > > gets
>>>> >>> >> > > > > >> > > > updated.
>>>> >>> >> > > > > >> > > > > >>
>>>> >>> >> > > > > >> > > > > >> Thanks,
>>>> >>> >> > > > > >> > > > > >> St.Ack
>>>> >>> >> > > > > >> > > > > >>
>>>> >>> >> > > > > >> > > > > >>
>>>> >>> >> > > > > >> > > > > >>
>>>> >>> >> > > > > >> > > > > >> > Thanks
>>>> >>> >> > > > > >> > > > > >> >
>>>> >>> >> > > > > >> > > > > >> > On Tue, Sep 20, 2016 at 10:02 AM,
>>>> Devaraj Das
>>>> >>> <
>>>> >>> >> > > > > >> > > ddas@hortonworks.com
>>>> >>> >> > > > > >> > > > >
>>>> >>> >> > > > > >> > > > > >> > wrote:
>>>> >>> >> > > > > >> > > > > >> >
>>>> >>> >> > > > > >> > > > > >> > > Just reviving this thread. Thanks Sean,
>>>> >>> Stack,
>>>> >>> >> > Dima,
>>>> >>> >> > > > and
>>>> >>> >> > > > > >> > others
>>>> >>> >> > > > > >> > > > for
>>>> >>> >> > > > > >> > > > > the
>>>> >>> >> > > > > >> > > > > >> > > thorough reviews and testing. Thanks
>>>> Ted and
>>>> >>> >> Vlad
>>>> >>> >> > for
>>>> >>> >> > > > > >> taking
>>>> >>> >> > > > > >> > > care
>>>> >>> >> > > > > >> > > > of
>>>> >>> >> > > > > >> > > > > >> the
>>>> >>> >> > > > > >> > > > > >> > > feedback. Are we all good to do the
>>>> merge
>>>> >>> now?
>>>> >>> >> > Rather
>>>> >>> >> > > > do
>>>> >>> >> > > > > >> > sooner
>>>> >>> >> > > > > >> > > > than
>>>> >>> >> > > > > >> > > > > >> > later.
>>>> >>> >> > > > > >> > > > > >> > > ______________________________
>>>> __________
>>>> >>> >> > > > > >> > > > > >> > > From: saint.ack@gmail.com <
>>>> >>> saint.ack@gmail.com>
>>>> >>> >> on
>>>> >>> >> > > > > behalf
>>>> >>> >> > > > > >> of
>>>> >>> >> > > > > >> > > > Stack
>>>> >>> >> > > > > >> > > > > <
>>>> >>> >> > > > > >> > > > > >> > > stack@duboce.net>
>>>> >>> >> > > > > >> > > > > >> > > Sent: Monday, September 12, 2016 1:18
>>>> PM
>>>> >>> >> > > > > >> > > > > >> > > To: HBase Dev List
>>>> >>> >> > > > > >> > > > > >> > > Subject: Re: [DISCUSSION] Merge Backup
>>>> /
>>>> >>> >> Restore -
>>>> >>> >> > > > Branch
>>>> >>> >> > > > > >> > > > HBASE-7912
>>>> >>> >> > > > > >> > > > > >> > >
>>>> >>> >> > > > > >> > > > > >> > > On Mon, Sep 12, 2016 at 12:19 PM, Ted
>>>> Yu <
>>>> >>> >> > > > > >> yuzhihong@gmail.com
>>>> >>> >> > > > > >> > >
>>>> >>> >> > > > > >> > > > > wrote:
>>>> >>> >> > > > > >> > > > > >> > >
>>>> >>> >> > > > > >> > > > > >> > > > Mega patch (rev 18) is on
>>>> HBASE-14123.
>>>> >>> >> > > > > >> > > > > >> > > >
>>>> >>> >> > > > > >> > > > > >> > > > Please comment on HBASE-14123 on how
>>>> you
>>>> >>> want
>>>> >>> >> to
>>>> >>> >> > > > > review.
>>>> >>> >> > > > > >> > > > > >> > > >
>>>> >>> >> > > > > >> > > > > >> > >
>>>> >>> >> > > > > >> > > > > >> > >
>>>> >>> >> > > > > >> > > > > >> > > Yeah. That was my lost tab. Last rb
>>>> was 6
>>>> >>> months
>>>> >>> >> > ago.
>>>> >>> >> > > > > >> Suggest
>>>> >>> >> > > > > >> > > > > updating
>>>> >>> >> > > > > >> > > > > >> > it.
>>>> >>> >> > > > > >> > > > > >> > > RB is pretty good for review. Patch is
>>>> only
>>>> >>> >> 1.5M so
>>>> >>> >> > > > > should
>>>> >>> >> > > > > >> be
>>>> >>> >> > > > > >> > > > fine.
>>>> >>> >> > > > > >> > > > > >> > >
>>>> >>> >> > > > > >> > > > > >> > > St.Ack
>>>> >>> >> > > > > >> > > > > >> > >
>>>> >>> >> > > > > >> > > > > >> > >
>>>> >>> >> > > > > >> > > > > >> > > >
>>>> >>> >> > > > > >> > > > > >> > > > Thanks
>>>> >>> >> > > > > >> > > > > >> > > >
>>>> >>> >> > > > > >> > > > > >> > > > On Mon, Sep 12, 2016 at 12:15 PM,
>>>> Stack <
>>>> >>> >> > > > > >> stack@duboce.net>
>>>> >>> >> > > > > >> > > > wrote:
>>>> >>> >> > > > > >> > > > > >> > > >
>>>> >>> >> > > > > >> > > > > >> > > > > On review of the 'patch', do I just
>>>> >>> compare
>>>> >>> >> the
>>>> >>> >> > > > > branch
>>>> >>> >> > > > > >> to
>>>> >>> >> > > > > >> > > > > master or
>>>> >>> >> > > > > >> > > > > >> > is
>>>> >>> >> > > > > >> > > > > >> > > > > there a megapatch posted somewhere
>>>> (I
>>>> >>> think
>>>> >>> >> I
>>>> >>> >> > saw
>>>> >>> >> > > > one
>>>> >>> >> > > > > >> but
>>>> >>> >> > > > > >> > it
>>>> >>> >> > > > > >> > > > > seemed
>>>> >>> >> > > > > >> > > > > >> > > stale
>>>> >>> >> > > > > >> > > > > >> > > > > and then I 'lost' the tab). Sorry
>>>> for
>>>> >>> dumb
>>>> >>> >> > > > question.
>>>> >>> >> > > > > >> > > > > >> > > > > St.Ack
>>>> >>> >> > > > > >> > > > > >> > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > On Mon, Sep 12, 2016 at 12:01 PM,
>>>> Stack
>>>> >>> <
>>>> >>> >> > > > > >> stack@duboce.net
>>>> >>> >> > > > > >> > >
>>>> >>> >> > > > > >> > > > > wrote:
>>>> >>> >> > > > > >> > > > > >> > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > > Late to the game. A few comments
>>>> after
>>>> >>> >> > > rereading
>>>> >>> >> > > > > this
>>>> >>> >> > > > > >> > > thread
>>>> >>> >> > > > > >> > > > > as a
>>>> >>> >> > > > > >> > > > > >> > > > 'user'.
>>>> >>> >> > > > > >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > > + Before merge, a user-facing
>>>> feature
>>>> >>> like
>>>> >>> >> > this
>>>> >>> >> > > > > >> should
>>>> >>> >> > > > > >> > > work
>>>> >>> >> > > > > >> > > > > (If
>>>> >>> >> > > > > >> > > > > >> > this
>>>> >>> >> > > > > >> > > > > >> > > is
>>>> >>> >> > > > > >> > > > > >> > > > > "higher-bar
>>>> >>> >> > > > > >> > > > > >> > > > > > for new features", bring it on --
>>>> >>> smile).
>>>> >>> >> > > > > >> > > > > >> > > > > > + As a user, I tried the branch
>>>> with
>>>> >>> tools
>>>> >>> >> > > after
>>>> >>> >> > > > > >> > reviewing
>>>> >>> >> > > > > >> > > > the
>>>> >>> >> > > > > >> > > > > >> > > > > just-posted
>>>> >>> >> > > > > >> > > > > >> > > > > > doc. I had an 'interesting'
>>>> experience
>>>> >>> >> (left
>>>> >>> >> > > > > >> comments up
>>>> >>> >> > > > > >> > > on
>>>> >>> >> > > > > >> > > > > >> > issue). I
>>>> >>> >> > > > > >> > > > > >> > > > > think
>>>> >>> >> > > > > >> > > > > >> > > > > > the tooling/doc. important to get
>>>> >>> right.
>>>> >>> >> If
>>>> >>> >> > it
>>>> >>> >> > > > > breaks
>>>> >>> >> > > > > >> > > easily
>>>> >>> >> > > > > >> > > > > or
>>>> >>> >> > > > > >> > > > > >> is
>>>> >>> >> > > > > >> > > > > >> > > > > > inconsistent (or lacks 'polish'),
>>>> >>> >> operators
>>>> >>> >> > > will
>>>> >>> >> > > > > >> judge
>>>> >>> >> > > > > >> > the
>>>> >>> >> > > > > >> > > > > whole
>>>> >>> >> > > > > >> > > > > >> > > > > > backup/restore tooling chain as
>>>> not
>>>> >>> >> > trustworthy
>>>> >>> >> > > > and
>>>> >>> >> > > > > >> > > abandon
>>>> >>> >> > > > > >> > > > > it.
>>>> >>> >> > > > > >> > > > > >> > Lets
>>>> >>> >> > > > > >> > > > > >> > > > not
>>>> >>> >> > > > > >> > > > > >> > > > > > have this happen to this feature.
>>>> >>> >> > > > > >> > > > > >> > > > > > + Matteo's suggestion (with a
>>>> helpful
>>>> >>> >> starter
>>>> >>> >> > > > list)
>>>> >>> >> > > > > >> that
>>>> >>> >> > > > > >> > > > there
>>>> >>> >> > > > > >> > > > > >> > needs
>>>> >>> >> > > > > >> > > > > >> > > to
>>>> >>> >> > > > > >> > > > > >> > > > > be
>>>> >>> >> > > > > >> > > > > >> > > > > > explicit qualification on what is
>>>> >>> actually
>>>> >>> >> > > being
>>>> >>> >> > > > > >> > delivered
>>>> >>> >> > > > > >> > > > --
>>>> >>> >> > > > > >> > > > > >> > > > including a
>>>> >>> >> > > > > >> > > > > >> > > > > > listing of limitations (some look
>>>> >>> serious
>>>> >>> >> > such
>>>> >>> >> > > as
>>>> >>> >> > > > > >> data
>>>> >>> >> > > > > >> > > bleed
>>>> >>> >> > > > > >> > > > > from
>>>> >>> >> > > > > >> > > > > >> > > other
>>>> >>> >> > > > > >> > > > > >> > > > > > regions in WALs, but maybe I
>>>> don't
>>>> >>> care
>>>> >>> >> for
>>>> >>> >> > my
>>>> >>> >> > > > use
>>>> >>> >> > > > > >> > > case...)
>>>> >>> >> > > > > >> > > > --
>>>> >>> >> > > > > >> > > > > >> > needs
>>>> >>> >> > > > > >> > > > > >> > > to
>>>> >>> >> > > > > >> > > > > >> > > > > > accompany the merge. Lets fold
>>>> them
>>>> >>> into
>>>> >>> >> the
>>>> >>> >> > > user
>>>> >>> >> > > > > >> doc.
>>>> >>> >> > > > > >> > in
>>>> >>> >> > > > > >> > > > the
>>>> >>> >> > > > > >> > > > > >> > > technical
>>>> >>> >> > > > > >> > > > > >> > > > > > overview area as suggested so
>>>> user
>>>> >>> >> > expectations
>>>> >>> >> > > > are
>>>> >>> >> > > > > >> > > properly
>>>> >>> >> > > > > >> > > > > >> > managed
>>>> >>> >> > > > > >> > > > > >> > > > > > (otherwise, they expect the
>>>> world and
>>>> >>> will
>>>> >>> >> > just
>>>> >>> >> > > > > give
>>>> >>> >> > > > > >> up
>>>> >>> >> > > > > >> > > when
>>>> >>> >> > > > > >> > > > > we
>>>> >>> >> > > > > >> > > > > >> > fall
>>>> >>> >> > > > > >> > > > > >> > > > > > short). Vladimir did a list of
>>>> what
>>>> >>> is in
>>>> >>> >> > each
>>>> >>> >> > > of
>>>> >>> >> > > > > the
>>>> >>> >> > > > > >> > > phases
>>>> >>> >> > > > > >> > > > > >> above
>>>> >>> >> > > > > >> > > > > >> > > > which
>>>> >>> >> > > > > >> > > > > >> > > > > > would serve as a good start.
>>>> >>> >> > > > > >> > > > > >> > > > > > + Is this feature 'experimental'
>>>> >>> (Matteo
>>>> >>> >> asks
>>>> >>> >> > > > > above).
>>>> >>> >> > > > > >> > I'd
>>>> >>> >> > > > > >> > > > > prefer
>>>> >>> >> > > > > >> > > > > >> it
>>>> >>> >> > > > > >> > > > > >> > > is
>>>> >>> >> > > > > >> > > > > >> > > > > > not. If it is, it should be
>>>> labelled
>>>> >>> all
>>>> >>> >> over
>>>> >>> >> > > > that
>>>> >>> >> > > > > >> it is
>>>> >>> >> > > > > >> > > > so. I
>>>> >>> >> > > > > >> > > > > >> see
>>>> >>> >> > > > > >> > > > > >> > > > > current
>>>> >>> >> > > > > >> > > > > >> > > > > > state called out as a '...
>>>> technical
>>>> >>> >> preview
>>>> >>> >> > > > > >> feature'.
>>>> >>> >> > > > > >> > > Does
>>>> >>> >> > > > > >> > > > > this
>>>> >>> >> > > > > >> > > > > >> > mean
>>>> >>> >> > > > > >> > > > > >> > > > > > not-for-users?
>>>> >>> >> > > > > >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > > St.Ack
>>>> >>> >> > > > > >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > > On Mon, Sep 12, 2016 at 8:03 AM,
>>>> Ted
>>>> >>> Yu <
>>>> >>> >> > > > > >> > > > yuzhihong@gmail.com>
>>>> >>> >> > > > > >> > > > > >> > wrote:
>>>> >>> >> > > > > >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> Sean:
>>>> >>> >> > > > > >> > > > > >> > > > > >> Do you have more comments ?
>>>> >>> >> > > > > >> > > > > >> > > > > >>
>>>> >>> >> > > > > >> > > > > >> > > > > >> Cheers
>>>> >>> >> > > > > >> > > > > >> > > > > >>
>>>> >>> >> > > > > >> > > > > >> > > > > >> On Fri, Sep 9, 2016 at 1:42 PM,
>>>> >>> Vladimir
>>>> >>> >> > > > Rodionov
>>>> >>> >> > > > > <
>>>> >>> >> > > > > >> > > > > >> > > > > vladrodionov@gmail.com
>>>> >>> >> > > > > >> > > > > >> > > > > >> >
>>>> >>> >> > > > > >> > > > > >> > > > > >> wrote:
>>>> >>> >> > > > > >> > > > > >> > > > > >>
>>>> >>> >> > > > > >> > > > > >> > > > > >> > Sean,
>>>> >>> >> > > > > >> > > > > >> > > > > >> >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > Backup/Restore can fail due to
>>>> >>> various
>>>> >>> >> > > > reasons:
>>>> >>> >> > > > > >> > network
>>>> >>> >> > > > > >> > > > > outage
>>>> >>> >> > > > > >> > > > > >> > > > > (cluster
>>>> >>> >> > > > > >> > > > > >> > > > > >> > wide), various time-outs in
>>>> HBase
>>>> >>> and
>>>> >>> >> HDFS
>>>> >>> >> > > > > layer,
>>>> >>> >> > > > > >> M/R
>>>> >>> >> > > > > >> > > > > failure
>>>> >>> >> > > > > >> > > > > >> > due
>>>> >>> >> > > > > >> > > > > >> > > to
>>>> >>> >> > > > > >> > > > > >> > > > > >> "HDFS
>>>> >>> >> > > > > >> > > > > >> > > > > >> > exceeded quota", user error
>>>> (manual
>>>> >>> >> > deletion
>>>> >>> >> > > > of
>>>> >>> >> > > > > >> data)
>>>> >>> >> > > > > >> > > and
>>>> >>> >> > > > > >> > > > > so
>>>> >>> >> > > > > >> > > > > >> on
>>>> >>> >> > > > > >> > > > > >> > so
>>>> >>> >> > > > > >> > > > > >> > > > on.
>>>> >>> >> > > > > >> > > > > >> > > > > >> That
>>>> >>> >> > > > > >> > > > > >> > > > > >> > is impossible to enumerate all
>>>> >>> possible
>>>> >>> >> > > types
>>>> >>> >> > > > of
>>>> >>> >> > > > > >> > > failures
>>>> >>> >> > > > > >> > > > > in a
>>>> >>> >> > > > > >> > > > > >> > > > > >> distributed
>>>> >>> >> > > > > >> > > > > >> > > > > >> > system - that is not our
>>>> goal/task.
>>>> >>> >> > > > > >> > > > > >> > > > > >> >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > We focus completely on backup
>>>> >>> system
>>>> >>> >> table
>>>> >>> >> > > > > >> > consistency
>>>> >>> >> > > > > >> > > > in a
>>>> >>> >> > > > > >> > > > > >> > > presence
>>>> >>> >> > > > > >> > > > > >> > > > > of
>>>> >>> >> > > > > >> > > > > >> > > > > >> any
>>>> >>> >> > > > > >> > > > > >> > > > > >> > type of failure. That is what
>>>> I
>>>> >>> call
>>>> >>> >> > > > "tolerance
>>>> >>> >> > > > > to
>>>> >>> >> > > > > >> > > > > failures".
>>>> >>> >> > > > > >> > > > > >> > > > > >> >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > On a failure:
>>>> >>> >> > > > > >> > > > > >> > > > > >> >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > BACKUP. All backup system
>>>> >>> information
>>>> >>> >> > (prior
>>>> >>> >> > > > to
>>>> >>> >> > > > > >> > backup)
>>>> >>> >> > > > > >> > > > > will
>>>> >>> >> > > > > >> > > > > >> be
>>>> >>> >> > > > > >> > > > > >> > > > > restored
>>>> >>> >> > > > > >> > > > > >> > > > > >> > and all temporary data,
>>>> related to
>>>> >>> a
>>>> >>> >> > failed
>>>> >>> >> > > > > >> session,
>>>> >>> >> > > > > >> > in
>>>> >>> >> > > > > >> > > > > HDFS
>>>> >>> >> > > > > >> > > > > >> > will
>>>> >>> >> > > > > >> > > > > >> > > be
>>>> >>> >> > > > > >> > > > > >> > > > > >> > deleted
>>>> >>> >> > > > > >> > > > > >> > > > > >> > RESTORE. We do not care about
>>>> >>> system
>>>> >>> >> data,
>>>> >>> >> > > > > because
>>>> >>> >> > > > > >> > > > restore
>>>> >>> >> > > > > >> > > > > >> does
>>>> >>> >> > > > > >> > > > > >> > > not
>>>> >>> >> > > > > >> > > > > >> > > > > >> change
>>>> >>> >> > > > > >> > > > > >> > > > > >> > it. Temporary data in HDFS
>>>> will be
>>>> >>> >> cleaned
>>>> >>> >> > > up
>>>> >>> >> > > > > and
>>>> >>> >> > > > > >> > table
>>>> >>> >> > > > > >> > > > > will
>>>> >>> >> > > > > >> > > > > >> be
>>>> >>> >> > > > > >> > > > > >> > > in a
>>>> >>> >> > > > > >> > > > > >> > > > > >> state
>>>> >>> >> > > > > >> > > > > >> > > > > >> > back to where it was before
>>>> >>> operation
>>>> >>> >> > > started.
>>>> >>> >> > > > > >> > > > > >> > > > > >> >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > This is what user should
>>>> expect in
>>>> >>> case
>>>> >>> >> > of a
>>>> >>> >> > > > > >> failure.
>>>> >>> >> > > > > >> > > > > >> > > > > >> >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > -Vlad
>>>> >>> >> > > > > >> > > > > >> > > > > >> >
>>>> >>> >> > > > > >> > > > > >> > > > > >> >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > -Vlad
>>>> >>> >> > > > > >> > > > > >> > > > > >> >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > On Fri, Sep 9, 2016 at 12:56
>>>> PM,
>>>> >>> Sean
>>>> >>> >> > > Busbey <
>>>> >>> >> > > > > >> > > > > >> busbey@apache.org
>>>> >>> >> > > > > >> > > > > >> > >
>>>> >>> >> > > > > >> > > > > >> > > > > wrote:
>>>> >>> >> > > > > >> > > > > >> > > > > >> >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > Failing in a consistent
>>>> way, with
>>>> >>> >> docs
>>>> >>> >> > > that
>>>> >>> >> > > > > >> explain
>>>> >>> >> > > > > >> > > the
>>>> >>> >> > > > > >> > > > > >> > various
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > expected failures would be
>>>> >>> >> sufficient.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > On Fri, Sep 9, 2016 at
>>>> 12:16 PM,
>>>> >>> >> > Vladimir
>>>> >>> >> > > > > >> Rodionov
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > <vl...@gmail.com>
>>>> wrote:
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > > Do not worry Sean, doc is
>>>> >>> coming
>>>> >>> >> today
>>>> >>> >> > > as
>>>> >>> >> > > > a
>>>> >>> >> > > > > >> > preview
>>>> >>> >> > > > > >> > > > and
>>>> >>> >> > > > > >> > > > > >> our
>>>> >>> >> > > > > >> > > > > >> > > > writer
>>>> >>> >> > > > > >> > > > > >> > > > > >> > Frank
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > > will be working on a
>>>> putting
>>>> >>> it
>>>> >>> >> into
>>>> >>> >> > > > Apache
>>>> >>> >> > > > > >> > repo.
>>>> >>> >> > > > > >> > > > > >> Timeline
>>>> >>> >> > > > > >> > > > > >> > > > > depends
>>>> >>> >> > > > > >> > > > > >> > > > > >> on
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > > Franks schedule but I
>>>> hope we
>>>> >>> will
>>>> >>> >> get
>>>> >>> >> > > it
>>>> >>> >> > > > > >> rather
>>>> >>> >> > > > > >> > > > sooner
>>>> >>> >> > > > > >> > > > > >> than
>>>> >>> >> > > > > >> > > > > >> > > > > later.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > > As for failure testing,
>>>> we are
>>>> >>> >> > focusing
>>>> >>> >> > > > only
>>>> >>> >> > > > > >> on a
>>>> >>> >> > > > > >> > > > > >> consistent
>>>> >>> >> > > > > >> > > > > >> > > > state
>>>> >>> >> > > > > >> > > > > >> > > > > >> of
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > > backup system data in a
>>>> >>> presence of
>>>> >>> >> > any
>>>> >>> >> > > > type
>>>> >>> >> > > > > >> of
>>>> >>> >> > > > > >> > > > > failures,
>>>> >>> >> > > > > >> > > > > >> We
>>>> >>> >> > > > > >> > > > > >> > > are
>>>> >>> >> > > > > >> > > > > >> > > > > not
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > going
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > > to implement  anything
>>>> more
>>>> >>> >> "fancy",
>>>> >>> >> > > than
>>>> >>> >> > > > > >> that.
>>>> >>> >> > > > > >> > We
>>>> >>> >> > > > > >> > > > > allow
>>>> >>> >> > > > > >> > > > > >> > both:
>>>> >>> >> > > > > >> > > > > >> > > > > >> backup
>>>> >>> >> > > > > >> > > > > >> > > > > >> > and
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > > restore to fail. What we
>>>> do not
>>>> >>> >> allow
>>>> >>> >> > is
>>>> >>> >> > > > to
>>>> >>> >> > > > > >> have
>>>> >>> >> > > > > >> > > > system
>>>> >>> >> > > > > >> > > > > >> data
>>>> >>> >> > > > > >> > > > > >> > > > > >> corrupted.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > > Will it suffice for you?
>>>> Do you
>>>> >>> >> have
>>>> >>> >> > any
>>>> >>> >> > > > > other
>>>> >>> >> > > > > >> > > > > concerns,
>>>> >>> >> > > > > >> > > > > >> you
>>>> >>> >> > > > > >> > > > > >> > > > want
>>>> >>> >> > > > > >> > > > > >> > > > > >> us to
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > > address?
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > > -Vlad
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > > On Fri, Sep 9, 2016 at
>>>> 10:56
>>>> >>> AM,
>>>> >>> >> Sean
>>>> >>> >> > > > > Busbey <
>>>> >>> >> > > > > >> > > > > >> > > busbey@apache.org
>>>> >>> >> > > > > >> > > > > >> > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > wrote:
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> "docs will come to Apache
>>>> >>> soon"
>>>> >>> >> does
>>>> >>> >> > > not
>>>> >>> >> > > > > >> address
>>>> >>> >> > > > > >> > > my
>>>> >>> >> > > > > >> > > > > >> concern
>>>> >>> >> > > > > >> > > > > >> > > > > around
>>>> >>> >> > > > > >> > > > > >> > > > > >> > docs
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > at
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> all, unless said docs
>>>> have
>>>> >>> already
>>>> >>> >> > made
>>>> >>> >> > > > it
>>>> >>> >> > > > > >> into
>>>> >>> >> > > > > >> > > the
>>>> >>> >> > > > > >> > > > > >> project
>>>> >>> >> > > > > >> > > > > >> > > > > repo. I
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > don't
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> want third party
>>>> resources for
>>>> >>> >> using
>>>> >>> >> > a
>>>> >>> >> > > > > major
>>>> >>> >> > > > > >> and
>>>> >>> >> > > > > >> > > > > >> important
>>>> >>> >> > > > > >> > > > > >> > > > > feature
>>>> >>> >> > > > > >> > > > > >> > > > > >> of
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > the
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> project, I want us to
>>>> provide
>>>> >>> end
>>>> >>> >> > users
>>>> >>> >> > > > > with
>>>> >>> >> > > > > >> > what
>>>> >>> >> > > > > >> > > > they
>>>> >>> >> > > > > >> > > > > >> need
>>>> >>> >> > > > > >> > > > > >> > > to
>>>> >>> >> > > > > >> > > > > >> > > > > get
>>>> >>> >> > > > > >> > > > > >> > > > > >> the
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > job
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> done.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >>
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> I see some calls for
>>>> patience
>>>> >>> on
>>>> >>> >> the
>>>> >>> >> > > > > failure
>>>> >>> >> > > > > >> > > > testing,
>>>> >>> >> > > > > >> > > > > but
>>>> >>> >> > > > > >> > > > > >> > the
>>>> >>> >> > > > > >> > > > > >> > > > > >> appeal
>>>> >>> >> > > > > >> > > > > >> > > > > >> > to
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > us
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> having done a bad job of
>>>> >>> requiring
>>>> >>> >> > > proper
>>>> >>> >> > > > > >> tests
>>>> >>> >> > > > > >> > of
>>>> >>> >> > > > > >> > > > > >> previous
>>>> >>> >> > > > > >> > > > > >> > > > > >> features
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > just
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> makes me more concerned
>>>> about
>>>> >>> not
>>>> >>> >> > > getting
>>>> >>> >> > > > > >> them
>>>> >>> >> > > > > >> > > > here. I
>>>> >>> >> > > > > >> > > > > >> > don't
>>>> >>> >> > > > > >> > > > > >> > > > want
>>>> >>> >> > > > > >> > > > > >> > > > > >> to
>>>> >>> >> > > > > >> > > > > >> > > > > >> > set
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> yet another bad example
>>>> that
>>>> >>> will
>>>> >>> >> > then
>>>> >>> >> > > be
>>>> >>> >> > > > > >> > pointed
>>>> >>> >> > > > > >> > > to
>>>> >>> >> > > > > >> > > > > in
>>>> >>> >> > > > > >> > > > > >> the
>>>> >>> >> > > > > >> > > > > >> > > > > future.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >>
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> On Sep 8, 2016 10:50,
>>>> "Ted
>>>> >>> Yu" <
>>>> >>> >> > > > > >> > > yuzhihong@gmail.com
>>>> >>> >> > > > > >> > > > >
>>>> >>> >> > > > > >> > > > > >> > wrote:
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >>
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > Is there any concern
>>>> which
>>>> >>> is
>>>> >>> >> not
>>>> >>> >> > > > > >> addressed ?
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > Do we need another Vote
>>>> >>> thread ?
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > Thanks
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > On Thu, Sep 8, 2016 at
>>>> 9:21
>>>> >>> AM,
>>>> >>> >> > > Andrew
>>>> >>> >> > > > > >> > Purtell <
>>>> >>> >> > > > > >> > > > > >> > > > > >> apurtell@apache.org
>>>> >>> >> > > > > >> > > > > >> > > > > >> > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > wrote:
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > Vlad,
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > I apologize for
>>>> using the
>>>> >>> term
>>>> >>> >> > > > > >> 'half-baked'
>>>> >>> >> > > > > >> > > in a
>>>> >>> >> > > > > >> > > > > way
>>>> >>> >> > > > > >> > > > > >> > that
>>>> >>> >> > > > > >> > > > > >> > > > > could
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > seem a
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > description of
>>>> >>> HBASE-7912. I
>>>> >>> >> > meant
>>>> >>> >> > > > that
>>>> >>> >> > > > > >> as a
>>>> >>> >> > > > > >> > > > > general
>>>> >>> >> > > > > >> > > > > >> > > > > >> hypothetical.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > On Wed, Sep 7, 2016
>>>> at
>>>> >>> 9:36
>>>> >>> >> AM,
>>>> >>> >> > > > > Vladimir
>>>> >>> >> > > > > >> > > > Rodionov
>>>> >>> >> > > > > >> > > > > <
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > vladrodionov@gmail.com
>>>> >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > wrote:
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >> I'm not sure
>>>> that
>>>> >>> "There
>>>> >>> >> is
>>>> >>> >> > > > > already
>>>> >>> >> > > > > >> > lots
>>>> >>> >> > > > > >> > > of
>>>> >>> >> > > > > >> > > > > >> > > half-baked
>>>> >>> >> > > > > >> > > > > >> > > > > >> code
>>>> >>> >> > > > > >> > > > > >> > > > > >> > in
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > the
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > branch,
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > so what's the harm
>>>> in
>>>> >>> adding
>>>> >>> >> > > more?"
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > I meant - not
>>>> >>> production -
>>>> >>> >> > ready
>>>> >>> >> > > > yet.
>>>> >>> >> > > > > >> This
>>>> >>> >> > > > > >> > > is
>>>> >>> >> > > > > >> > > > > 2.0
>>>> >>> >> > > > > >> > > > > >> > > > > development
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > branch
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > and,
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > hence many
>>>> features are
>>>> >>> in
>>>> >>> >> > works,
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > not being tested
>>>> well
>>>> >>> etc.
>>>> >>> >> I do
>>>> >>> >> > > not
>>>> >>> >> > > > > >> > consider
>>>> >>> >> > > > > >> > > > > backup
>>>> >>> >> > > > > >> > > > > >> > as
>>>> >>> >> > > > > >> > > > > >> > > > half
>>>> >>> >> > > > > >> > > > > >> > > > > >> > baked
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > feature -
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > it has passed our
>>>> >>> internal
>>>> >>> >> QA
>>>> >>> >> > and
>>>> >>> >> > > > has
>>>> >>> >> > > > > >> very
>>>> >>> >> > > > > >> > > > good
>>>> >>> >> > > > > >> > > > > >> doc,
>>>> >>> >> > > > > >> > > > > >> > > > which
>>>> >>> >> > > > > >> > > > > >> > > > > we
>>>> >>> >> > > > > >> > > > > >> > > > > >> > will
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > provide
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > to Apache shortly.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > -Vlad
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > On Wed, Sep 7,
>>>> 2016 at
>>>> >>> 9:13
>>>> >>> >> AM,
>>>> >>> >> > > > > Andrew
>>>> >>> >> > > > > >> > > > Purtell <
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > apurtell@apache.org>
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > We shouldn't
>>>> admit
>>>> >>> half
>>>> >>> >> baked
>>>> >>> >> > > > > changes
>>>> >>> >> > > > > >> > that
>>>> >>> >> > > > > >> > > > > won't
>>>> >>> >> > > > > >> > > > > >> be
>>>> >>> >> > > > > >> > > > > >> > > > > >> finished.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> However
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > in
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > this case the
>>>> crew
>>>> >>> >> working on
>>>> >>> >> > > > this
>>>> >>> >> > > > > >> > feature
>>>> >>> >> > > > > >> > > > are
>>>> >>> >> > > > > >> > > > > >> long
>>>> >>> >> > > > > >> > > > > >> > > > > timers
>>>> >>> >> > > > > >> > > > > >> > > > > >> and
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > less
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > likely
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > than just about
>>>> >>> anyone to
>>>> >>> >> > leave
>>>> >>> >> > > > > >> > something
>>>> >>> >> > > > > >> > > > in a
>>>> >>> >> > > > > >> > > > > >> half
>>>> >>> >> > > > > >> > > > > >> > > > baked
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > state. Of
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > course
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > there is no
>>>> guarantee
>>>> >>> how
>>>> >>> >> > > > anything
>>>> >>> >> > > > > >> will
>>>> >>> >> > > > > >> > > turn
>>>> >>> >> > > > > >> > > > > out,
>>>> >>> >> > > > > >> > > > > >> > > but I
>>>> >>> >> > > > > >> > > > > >> > > > > am
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > willing
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> to
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > take
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > a little on
>>>> faith if
>>>> >>> they
>>>> >>> >> > feel
>>>> >>> >> > > > > their
>>>> >>> >> > > > > >> > best
>>>> >>> >> > > > > >> > > > path
>>>> >>> >> > > > > >> > > > > >> > > forward
>>>> >>> >> > > > > >> > > > > >> > > > > now
>>>> >>> >> > > > > >> > > > > >> > > > > >> is
>>>> >>> >> > > > > >> > > > > >> > > > > >> > to
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > merge
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > to
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > trunk. I only
>>>> wish I
>>>> >>> had
>>>> >>> >> > > > bandwidth
>>>> >>> >> > > > > to
>>>> >>> >> > > > > >> > have
>>>> >>> >> > > > > >> > > > > done
>>>> >>> >> > > > > >> > > > > >> > some
>>>> >>> >> > > > > >> > > > > >> > > > real
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > kicking
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> of
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > the
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > tires by now.
>>>> Maybe
>>>> >>> this
>>>> >>> >> > week.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > (Yes, I'm using
>>>> some
>>>> >>> of
>>>> >>> >> that
>>>> >>> >> > > time
>>>> >>> >> > > > > for
>>>> >>> >> > > > > >> > this
>>>> >>> >> > > > > >> > > > > email
>>>> >>> >> > > > > >> > > > > >> > :-)
>>>> >>> >> > > > > >> > > > > >> > > > but
>>>> >>> >> > > > > >> > > > > >> > > > > I
>>>> >>> >> > > > > >> > > > > >> > > > > >> > type
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > fast.)
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > That said, I
>>>> would
>>>> >>> like to
>>>> >>> >> > > > agitate
>>>> >>> >> > > > > >> for
>>>> >>> >> > > > > >> > > > making
>>>> >>> >> > > > > >> > > > > 2.0
>>>> >>> >> > > > > >> > > > > >> > > more
>>>> >>> >> > > > > >> > > > > >> > > > > real
>>>> >>> >> > > > > >> > > > > >> > > > > >> > and
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> spend
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > some
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > time on it now
>>>> that
>>>> >>> I'm
>>>> >>> >> > winding
>>>> >>> >> > > > > down
>>>> >>> >> > > > > >> > with
>>>> >>> >> > > > > >> > > > > 0.98. I
>>>> >>> >> > > > > >> > > > > >> > > think
>>>> >>> >> > > > > >> > > > > >> > > > > >> that
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > means
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branching for
>>>> 2.0 real
>>>> >>> >> soon
>>>> >>> >> > now
>>>> >>> >> > > > and
>>>> >>> >> > > > > >> even
>>>> >>> >> > > > > >> > > > > evicting
>>>> >>> >> > > > > >> > > > > >> > > > things
>>>> >>> >> > > > > >> > > > > >> > > > > >> from
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > 2.0
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > branch
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > that aren't
>>>> finished
>>>> >>> or
>>>> >>> >> > stable,
>>>> >>> >> > > > > >> leaving
>>>> >>> >> > > > > >> > > them
>>>> >>> >> > > > > >> > > > > only
>>>> >>> >> > > > > >> > > > > >> > > once
>>>> >>> >> > > > > >> > > > > >> > > > > >> again
>>>> >>> >> > > > > >> > > > > >> > > > > >> > in
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > the
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > master
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branch. Or,
>>>> maybe just
>>>> >>> >> > evicting
>>>> >>> >> > > > > them.
>>>> >>> >> > > > > >> > > Let's
>>>> >>> >> > > > > >> > > > > take
>>>> >>> >> > > > > >> > > > > >> it
>>>> >>> >> > > > > >> > > > > >> > > > case
>>>> >>> >> > > > > >> > > > > >> > > > > by
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > case.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > I think this
>>>> feature
>>>> >>> can
>>>> >>> >> come
>>>> >>> >> > > in
>>>> >>> >> > > > > >> > > relatively
>>>> >>> >> > > > > >> > > > > >> safely.
>>>> >>> >> > > > > >> > > > > >> > > As
>>>> >>> >> > > > > >> > > > > >> > > > > >> added
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > insurance,
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > let's admit the
>>>> >>> >> possibility
>>>> >>> >> > it
>>>> >>> >> > > > > could
>>>> >>> >> > > > > >> be
>>>> >>> >> > > > > >> > > > > reverted
>>>> >>> >> > > > > >> > > > > >> on
>>>> >>> >> > > > > >> > > > > >> > > the
>>>> >>> >> > > > > >> > > > > >> > > > > 2.0
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > branch
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> if
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > folks
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > working on
>>>> >>> stabilizing 2.0
>>>> >>> >> > > decide
>>>> >>> >> > > > > to
>>>> >>> >> > > > > >> > evict
>>>> >>> >> > > > > >> > > > it
>>>> >>> >> > > > > >> > > > > >> > because
>>>> >>> >> > > > > >> > > > > >> > > > it
>>>> >>> >> > > > > >> > > > > >> > > > > is
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > unfinished
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > or
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > unstable,
>>>> because that
>>>> >>> >> > > certainly
>>>> >>> >> > > > > can
>>>> >>> >> > > > > >> > > > happen. I
>>>> >>> >> > > > > >> > > > > >> > would
>>>> >>> >> > > > > >> > > > > >> > > > > >> expect if
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > talk
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > like
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > that starts,
>>>> we'd get
>>>> >>> help
>>>> >>> >> > > > > finishing
>>>> >>> >> > > > > >> or
>>>> >>> >> > > > > >> > > > > >> stabilizing
>>>> >>> >> > > > > >> > > > > >> > > > > what's
>>>> >>> >> > > > > >> > > > > >> > > > > >> > under
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > discussion
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > for revert. Or,
>>>> we'd
>>>> >>> have
>>>> >>> >> a
>>>> >>> >> > > > revert.
>>>> >>> >> > > > > >> > Either
>>>> >>> >> > > > > >> > > > way
>>>> >>> >> > > > > >> > > > > >> the
>>>> >>> >> > > > > >> > > > > >> > > > > outcome
>>>> >>> >> > > > > >> > > > > >> > > > > >> is
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > acceptable.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > On Wed, Sep 7,
>>>> 2016 at
>>>> >>> >> 8:56
>>>> >>> >> > AM,
>>>> >>> >> > > > > Dima
>>>> >>> >> > > > > >> > > Spivak
>>>> >>> >> > > > > >> > > > <
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > dimaspivak@apache.org
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > I'm not sure
>>>> that
>>>> >>> >> "There is
>>>> >>> >> > > > > already
>>>> >>> >> > > > > >> > lots
>>>> >>> >> > > > > >> > > > of
>>>> >>> >> > > > > >> > > > > >> > > > half-baked
>>>> >>> >> > > > > >> > > > > >> > > > > >> code
>>>> >>> >> > > > > >> > > > > >> > > > > >> > in
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> the
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branch,
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > so what's the
>>>> harm
>>>> >>> in
>>>> >>> >> > adding
>>>> >>> >> > > > > more?"
>>>> >>> >> > > > > >> > is a
>>>> >>> >> > > > > >> > > > > good
>>>> >>> >> > > > > >> > > > > >> > code
>>>> >>> >> > > > > >> > > > > >> > > > > commit
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > philosophy
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > for
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > a
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > fault-tolerant
>>>> >>> >> distributed
>>>> >>> >> > > data
>>>> >>> >> > > > > >> store.
>>>> >>> >> > > > > >> > > ;)
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > More
>>>> seriously, a
>>>> >>> lack
>>>> >>> >> of
>>>> >>> >> > > test
>>>> >>> >> > > > > >> > coverage
>>>> >>> >> > > > > >> > > > for
>>>> >>> >> > > > > >> > > > > >> > > existing
>>>> >>> >> > > > > >> > > > > >> > > > > >> > features
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > shouldn't
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > be
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > used as
>>>> >>> justification
>>>> >>> >> for
>>>> >>> >> > > > > >> introducing
>>>> >>> >> > > > > >> > > new
>>>> >>> >> > > > > >> > > > > >> > features
>>>> >>> >> > > > > >> > > > > >> > > > with
>>>> >>> >> > > > > >> > > > > >> > > > > >> the
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > same
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > shortcomings.
>>>> >>> >> Ultimately,
>>>> >>> >> > > it's
>>>> >>> >> > > > > the
>>>> >>> >> > > > > >> end
>>>> >>> >> > > > > >> > > > user
>>>> >>> >> > > > > >> > > > > who
>>>> >>> >> > > > > >> > > > > >> > > will
>>>> >>> >> > > > > >> > > > > >> > > > > feel
>>>> >>> >> > > > > >> > > > > >> > > > > >> > the
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> pain,
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > so
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > shouldn't we do
>>>> >>> >> everything
>>>> >>> >> > we
>>>> >>> >> > > > can
>>>> >>> >> > > > > >> to
>>>> >>> >> > > > > >> > > > > mitigate
>>>> >>> >> > > > > >> > > > > >> > that?
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > -Dima
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > On Wed, Sep 7,
>>>> 2016
>>>> >>> at
>>>> >>> >> 8:46
>>>> >>> >> > > AM,
>>>> >>> >> > > > > >> > Vladimir
>>>> >>> >> > > > > >> > > > > >> > Rodionov <
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>>> >>> vladrodionov@gmail.com>
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > wrote:
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Sean,
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have docs
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Agree. We
>>>> have a
>>>> >>> doc
>>>> >>> >> and
>>>> >>> >> > > > backup
>>>> >>> >> > > > > >> is
>>>> >>> >> > > > > >> > the
>>>> >>> >> > > > > >> > > > > most
>>>> >>> >> > > > > >> > > > > >> > > > > documented
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > feature
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > :),
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > we
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > will
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > release it
>>>> >>> shortly to
>>>> >>> >> > > Apache.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have
>>>> sunny-day
>>>> >>> >> > > correctness
>>>> >>> >> > > > > >> tests
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Feature has
>>>> >>> close to
>>>> >>> >> 60
>>>> >>> >> > > test
>>>> >>> >> > > > > >> cases,
>>>> >>> >> > > > > >> > > > which
>>>> >>> >> > > > > >> > > > > >> run
>>>> >>> >> > > > > >> > > > > >> > > for
>>>> >>> >> > > > > >> > > > > >> > > > > >> approx
>>>> >>> >> > > > > >> > > > > >> > > > > >> > 30
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> min.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > We
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > can
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > add more, if
>>>> >>> >> community do
>>>> >>> >> > > not
>>>> >>> >> > > > > >> mind
>>>> >>> >> > > > > >> > :)
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have
>>>> >>> >> > > > > >> > correctness-in-face-of-failure
>>>> >>> >> > > > > >> > > > > tests
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Any examples
>>>> of
>>>> >>> these
>>>> >>> >> > tests
>>>> >>> >> > > > in
>>>> >>> >> > > > > >> > > existing
>>>> >>> >> > > > > >> > > > > >> > features?
>>>> >>> >> > > > > >> > > > > >> > > > In
>>>> >>> >> > > > > >> > > > > >> > > > > >> > works,
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > we
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > have a
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > clear
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>>> understanding of
>>>> >>> what
>>>> >>> >> > > should
>>>> >>> >> > > > be
>>>> >>> >> > > > > >> done
>>>> >>> >> > > > > >> > > by
>>>> >>> >> > > > > >> > > > > the
>>>> >>> >> > > > > >> > > > > >> > time
>>>> >>> >> > > > > >> > > > > >> > > of
>>>> >>> >> > > > > >> > > > > >> > > > > 2.0
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> release.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > That is very
>>>> close
>>>> >>> >> goal
>>>> >>> >> > for
>>>> >>> >> > > > us,
>>>> >>> >> > > > > >> to
>>>> >>> >> > > > > >> > > > verify
>>>> >>> >> > > > > >> > > > > IT
>>>> >>> >> > > > > >> > > > > >> > > monkey
>>>> >>> >> > > > > >> > > > > >> > > > > for
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> existing
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > code.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * don't rely
>>>> on
>>>> >>> things
>>>> >>> >> > > > outside
>>>> >>> >> > > > > of
>>>> >>> >> > > > > >> > > HBase
>>>> >>> >> > > > > >> > > > > for
>>>> >>> >> > > > > >> > > > > >> > > normal
>>>> >>> >> > > > > >> > > > > >> > > > > >> > operation
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > (okay
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > for
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > advanced
>>>> >>> operation)
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > We do not.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Enormous
>>>> time has
>>>> >>> been
>>>> >>> >> > > spent
>>>> >>> >> > > > > >> already
>>>> >>> >> > > > > >> > > on
>>>> >>> >> > > > > >> > > > > the
>>>> >>> >> > > > > >> > > > > >> > > > > development
>>>> >>> >> > > > > >> > > > > >> > > > > >> > and
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > testing
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > the
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > feature, it
>>>> has
>>>> >>> passed
>>>> >>> >> > our
>>>> >>> >> > > > > >> internal
>>>> >>> >> > > > > >> > > > tests
>>>> >>> >> > > > > >> > > > > and
>>>> >>> >> > > > > >> > > > > >> > > many
>>>> >>> >> > > > > >> > > > > >> > > > > >> rounds
>>>> >>> >> > > > > >> > > > > >> > > > > >> > of
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> code
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > reviews
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > by HBase
>>>> >>> committers.
>>>> >>> >> We
>>>> >>> >> > do
>>>> >>> >> > > > not
>>>> >>> >> > > > > >> mind
>>>> >>> >> > > > > >> > if
>>>> >>> >> > > > > >> > > > > >> someone
>>>> >>> >> > > > > >> > > > > >> > > from
>>>> >>> >> > > > > >> > > > > >> > > > > >> HBase
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > community
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > (outside of
>>>> HW)
>>>> >>> will
>>>> >>> >> > review
>>>> >>> >> > > > the
>>>> >>> >> > > > > >> > code,
>>>> >>> >> > > > > >> > > > but
>>>> >>> >> > > > > >> > > > > it
>>>> >>> >> > > > > >> > > > > >> > will
>>>> >>> >> > > > > >> > > > > >> > > > > >> probably
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> takes
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > forever
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > to
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > wait for
>>>> >>> volunteer?,
>>>> >>> >> the
>>>> >>> >> > > > > feature
>>>> >>> >> > > > > >> is
>>>> >>> >> > > > > >> > > > quite
>>>> >>> >> > > > > >> > > > > >> large
>>>> >>> >> > > > > >> > > > > >> > > > (1MB+
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> cumulative
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > patch)
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > 2.0 branch is
>>>> >>> full of
>>>> >>> >> > half
>>>> >>> >> > > > > baked
>>>> >>> >> > > > > >> > > > features,
>>>> >>> >> > > > > >> > > > > >> most
>>>> >>> >> > > > > >> > > > > >> > > of
>>>> >>> >> > > > > >> > > > > >> > > > > them
>>>> >>> >> > > > > >> > > > > >> > > > > >> > are
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > in
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > active
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > development,
>>>> >>> >> therefore I
>>>> >>> >> > am
>>>> >>> >> > > > not
>>>> >>> >> > > > > >> > > > following
>>>> >>> >> > > > > >> > > > > you
>>>> >>> >> > > > > >> > > > > >> > > here,
>>>> >>> >> > > > > >> > > > > >> > > > > >> Sean?
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > Why
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > HBASE-7912
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > is
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > not good
>>>> enough
>>>> >>> yet
>>>> >>> >> to be
>>>> >>> >> > > > > >> integrated
>>>> >>> >> > > > > >> > > > into
>>>> >>> >> > > > > >> > > > > 2.0
>>>> >>> >> > > > > >> > > > > >> > > > branch?
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > -Vlad
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > On Wed, Sep
>>>> 7,
>>>> >>> 2016 at
>>>> >>> >> > 8:23
>>>> >>> >> > > > AM,
>>>> >>> >> > > > > >> Sean
>>>> >>> >> > > > > >> > > > > Busbey <
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > busbey@apache.org
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > On Tue,
>>>> Sep 6,
>>>> >>> 2016
>>>> >>> >> at
>>>> >>> >> > > > 10:36
>>>> >>> >> > > > > >> PM,
>>>> >>> >> > > > > >> > > Josh
>>>> >>> >> > > > > >> > > > > >> Elser <
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > josh.elser@gmail.com
>>>> >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > wrote:
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > So, the
>>>> >>> answer to
>>>> >>> >> > > Sean's
>>>> >>> >> > > > > >> > original
>>>> >>> >> > > > > >> > > > > >> question
>>>> >>> >> > > > > >> > > > > >> > is
>>>> >>> >> > > > > >> > > > > >> > > > "as
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > robust as
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > snapshots
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > presently
>>>> >>> are"?
>>>> >>> >> > > > > >> (independence of
>>>> >>> >> > > > > >> > > > > >> > > backup/restore
>>>> >>> >> > > > > >> > > > > >> > > > > >> > failure
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > tolerance
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > from
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > snapshot
>>>> >>> failure
>>>> >>> >> > > > tolerance)
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > Is this
>>>> just a
>>>> >>> >> > question
>>>> >>> >> > > > WRT
>>>> >>> >> > > > > >> > > context
>>>> >>> >> > > > > >> > > > of
>>>> >>> >> > > > > >> > > > > >> the
>>>> >>> >> > > > > >> > > > > >> > > > > change,
>>>> >>> >> > > > > >> > > > > >> > > > > >> or
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > is it
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > means
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > for a
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > veto
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > from you,
>>>> >>> Sean?
>>>> >>> >> Just
>>>> >>> >> > > > trying
>>>> >>> >> > > > > >> to
>>>> >>> >> > > > > >> > > make
>>>> >>> >> > > > > >> > > > > sure
>>>> >>> >> > > > > >> > > > > >> > I'm
>>>> >>> >> > > > > >> > > > > >> > > > > >> following
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> along
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > adequately.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > I'd say
>>>> ATM I'm
>>>> >>> -0,
>>>> >>> >> > > > bordering
>>>> >>> >> > > > > >> on
>>>> >>> >> > > > > >> > -1
>>>> >>> >> > > > > >> > > > but
>>>> >>> >> > > > > >> > > > > not
>>>> >>> >> > > > > >> > > > > >> > for
>>>> >>> >> > > > > >> > > > > >> > > > > >> reasons
>>>> >>> >> > > > > >> > > > > >> > > > > >> > I
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > can
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > articulate
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > well.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Here's an
>>>> >>> attempt.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > We've been
>>>> >>> trying to
>>>> >>> >> > > move,
>>>> >>> >> > > > > as a
>>>> >>> >> > > > > >> > > > > community,
>>>> >>> >> > > > > >> > > > > >> > > > towards
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > minimizing
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > risk
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > to
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > downstream
>>>> >>> folks by
>>>> >>> >> > > getting
>>>> >>> >> > > > > >> > > "complete
>>>> >>> >> > > > > >> > > > > >> enough
>>>> >>> >> > > > > >> > > > > >> > > for
>>>> >>> >> > > > > >> > > > > >> > > > > use"
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > gates
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> in
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > place
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > before we
>>>> >>> introduce
>>>> >>> >> new
>>>> >>> >> > > > > >> features.
>>>> >>> >> > > > > >> > > This
>>>> >>> >> > > > > >> > > > > was
>>>> >>> >> > > > > >> > > > > >> > > > spurred
>>>> >>> >> > > > > >> > > > > >> > > > > >> by a
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > some
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > features
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > getting in
>>>> >>> >> half-baked
>>>> >>> >> > and
>>>> >>> >> > > > > never
>>>> >>> >> > > > > >> > > making
>>>> >>> >> > > > > >> > > > > it
>>>> >>> >> > > > > >> > > > > >> to
>>>> >>> >> > > > > >> > > > > >> > > "can
>>>> >>> >> > > > > >> > > > > >> > > > > >> really
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > use"
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > status
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > (I'm
>>>> thinking of
>>>> >>> >> > > > distributed
>>>> >>> >> > > > > >> log
>>>> >>> >> > > > > >> > > > replay
>>>> >>> >> > > > > >> > > > > and
>>>> >>> >> > > > > >> > > > > >> > the
>>>> >>> >> > > > > >> > > > > >> > > > > >> zk-less
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > assignment
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > stuff, I
>>>> don't
>>>> >>> >> recall
>>>> >>> >> > if
>>>> >>> >> > > > > there
>>>> >>> >> > > > > >> was
>>>> >>> >> > > > > >> > > > > more).
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The gates,
>>>> >>> >> generally,
>>>> >>> >> > > > > included
>>>> >>> >> > > > > >> > > things
>>>> >>> >> > > > > >> > > > > like:
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have docs
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have
>>>> sunny-day
>>>> >>> >> > > > correctness
>>>> >>> >> > > > > >> tests
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have
>>>> >>> >> > > > > >> > > correctness-in-face-of-failure
>>>> >>> >> > > > > >> > > > > tests
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't
>>>> rely on
>>>> >>> >> things
>>>> >>> >> > > > > outside
>>>> >>> >> > > > > >> of
>>>> >>> >> > > > > >> > > > HBase
>>>> >>> >> > > > > >> > > > > for
>>>> >>> >> > > > > >> > > > > >> > > > normal
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > operation
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > (okay
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > for
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > advanced
>>>> >>> operation)
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > As an
>>>> example,
>>>> >>> we
>>>> >>> >> kept
>>>> >>> >> > > the
>>>> >>> >> > > > > MOB
>>>> >>> >> > > > > >> > work
>>>> >>> >> > > > > >> > > > off
>>>> >>> >> > > > > >> > > > > in
>>>> >>> >> > > > > >> > > > > >> a
>>>> >>> >> > > > > >> > > > > >> > > > branch
>>>> >>> >> > > > > >> > > > > >> > > > > >> and
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > out
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> of
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > master
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > until it
>>>> could
>>>> >>> pass
>>>> >>> >> > these
>>>> >>> >> > > > > >> > criteria.
>>>> >>> >> > > > > >> > > > The
>>>> >>> >> > > > > >> > > > > big
>>>> >>> >> > > > > >> > > > > >> > > > > exemption
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > we've
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> had
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > to
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > this was
>>>> the
>>>> >>> >> > hbase-spark
>>>> >>> >> > > > > >> > > integration,
>>>> >>> >> > > > > >> > > > > where
>>>> >>> >> > > > > >> > > > > >> > we
>>>> >>> >> > > > > >> > > > > >> > > > all
>>>> >>> >> > > > > >> > > > > >> > > > > >> > agreed
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > it
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > could
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > land in
>>>> master
>>>> >>> >> because
>>>> >>> >> > it
>>>> >>> >> > > > was
>>>> >>> >> > > > > >> very
>>>> >>> >> > > > > >> > > > well
>>>> >>> >> > > > > >> > > > > >> > > isolated
>>>> >>> >> > > > > >> > > > > >> > > > > (the
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > slide
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > away
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > from
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > including
>>>> docs
>>>> >>> as a
>>>> >>> >> > > > > first-class
>>>> >>> >> > > > > >> > part
>>>> >>> >> > > > > >> > > > of
>>>> >>> >> > > > > >> > > > > >> > > building
>>>> >>> >> > > > > >> > > > > >> > > > up
>>>> >>> >> > > > > >> > > > > >> > > > > >> that
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > integration
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > has led me
>>>> to
>>>> >>> doubt
>>>> >>> >> the
>>>> >>> >> > > > > wisdom
>>>> >>> >> > > > > >> of
>>>> >>> >> > > > > >> > > this
>>>> >>> >> > > > > >> > > > > >> > > decision).
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > We've also
>>>> been
>>>> >>> >> > treating
>>>> >>> >> > > > > >> inclusion
>>>> >>> >> > > > > >> > > in
>>>> >>> >> > > > > >> > > > a
>>>> >>> >> > > > > >> > > > > >> > > "probably
>>>> >>> >> > > > > >> > > > > >> > > > > >> will
>>>> >>> >> > > > > >> > > > > >> > > > > >> > be
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > released
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > to
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > downstream"
>>>> >>> branches
>>>> >>> >> > as a
>>>> >>> >> > > > > >> higher
>>>> >>> >> > > > > >> > > bar,
>>>> >>> >> > > > > >> > > > > >> > requiring
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't
>>>> >>> moderately
>>>> >>> >> > impact
>>>> >>> >> > > > > >> > > performance
>>>> >>> >> > > > > >> > > > > when
>>>> >>> >> > > > > >> > > > > >> > the
>>>> >>> >> > > > > >> > > > > >> > > > > >> feature
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > isn't
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> in
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > use
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't
>>>> severely
>>>> >>> >> impact
>>>> >>> >> > > > > >> > performance
>>>> >>> >> > > > > >> > > > when
>>>> >>> >> > > > > >> > > > > >> the
>>>> >>> >> > > > > >> > > > > >> > > > > feature
>>>> >>> >> > > > > >> > > > > >> > > > > >> is
>>>> >>> >> > > > > >> > > > > >> > > > > >> > in
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> use
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * either
>>>> >>> >> default-to-on
>>>> >>> >> > or
>>>> >>> >> > > > > show
>>>> >>> >> > > > > >> > > enough
>>>> >>> >> > > > > >> > > > > >> demand
>>>> >>> >> > > > > >> > > > > >> > to
>>>> >>> >> > > > > >> > > > > >> > > > > >> believe
>>>> >>> >> > > > > >> > > > > >> > > > > >> > a
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > non-trivial
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > number of
>>>> folks
>>>> >>> will
>>>> >>> >> > turn
>>>> >>> >> > > > the
>>>> >>> >> > > > > >> > > feature
>>>> >>> >> > > > > >> > > > on
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The above
>>>> has
>>>> >>> kept
>>>> >>> >> MOB
>>>> >>> >> > > and
>>>> >>> >> > > > > >> > > hbase-spark
>>>> >>> >> > > > > >> > > > > >> > > > integration
>>>> >>> >> > > > > >> > > > > >> > > > > >> out
>>>> >>> >> > > > > >> > > > > >> > > > > >> > of
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > branch-1,
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > presumably
>>>> while
>>>> >>> >> > they've
>>>> >>> >> > > > > >> "gotten
>>>> >>> >> > > > > >> > > more
>>>> >>> >> > > > > >> > > > > >> stable"
>>>> >>> >> > > > > >> > > > > >> > > in
>>>> >>> >> > > > > >> > > > > >> > > > > >> master
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > from
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > the
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > odd
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > vendor
>>>> >>> inclusion.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Are we
>>>> going to
>>>> >>> >> have a
>>>> >>> >> > > 2.0
>>>> >>> >> > > > > >> release
>>>> >>> >> > > > > >> > > > > before
>>>> >>> >> > > > > >> > > > > >> the
>>>> >>> >> > > > > >> > > > > >> > > end
>>>> >>> >> > > > > >> > > > > >> > > > > of
>>>> >>> >> > > > > >> > > > > >> > > > > >> the
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> year?
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > We're
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > coming up
>>>> on 1.5
>>>> >>> >> years
>>>> >>> >> > > > since
>>>> >>> >> > > > > >> the
>>>> >>> >> > > > > >> > > > > release of
>>>> >>> >> > > > > >> > > > > >> > > > version
>>>> >>> >> > > > > >> > > > > >> > > > > >> 1.0;
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> seems
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > like
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > it's about
>>>> time,
>>>> >>> >> > though I
>>>> >>> >> > > > > >> haven't
>>>> >>> >> > > > > >> > > seen
>>>> >>> >> > > > > >> > > > > any
>>>> >>> >> > > > > >> > > > > >> > > > concrete
>>>> >>> >> > > > > >> > > > > >> > > > > >> > plans
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> this
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > year.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Presuming
>>>> we are
>>>> >>> >> going
>>>> >>> >> > to
>>>> >>> >> > > > > have
>>>> >>> >> > > > > >> one
>>>> >>> >> > > > > >> > > by
>>>> >>> >> > > > > >> > > > > the
>>>> >>> >> > > > > >> > > > > >> end
>>>> >>> >> > > > > >> > > > > >> > > of
>>>> >>> >> > > > > >> > > > > >> > > > > the
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > year, it
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > seems a
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > bit close
>>>> to
>>>> >>> still
>>>> >>> >> be
>>>> >>> >> > > > adding
>>>> >>> >> > > > > in
>>>> >>> >> > > > > >> > > > > "features
>>>> >>> >> > > > > >> > > > > >> > that
>>>> >>> >> > > > > >> > > > > >> > > > need
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > maturing"
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > on
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > the
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > branch.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The lack
>>>> of a
>>>> >>> >> concrete
>>>> >>> >> > > plan
>>>> >>> >> > > > > for
>>>> >>> >> > > > > >> > 2.0
>>>> >>> >> > > > > >> > > > > keeps
>>>> >>> >> > > > > >> > > > > >> me
>>>> >>> >> > > > > >> > > > > >> > > from
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > considering
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > these
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > things
>>>> blocker
>>>> >>> at
>>>> >>> >> the
>>>> >>> >> > > > moment.
>>>> >>> >> > > > > >> But
>>>> >>> >> > > > > >> > I
>>>> >>> >> > > > > >> > > > know
>>>> >>> >> > > > > >> > > > > >> > first
>>>> >>> >> > > > > >> > > > > >> > > > hand
>>>> >>> >> > > > > >> > > > > >> > > > > >> how
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > much
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > trouble
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > folks have
>>>> had
>>>> >>> with
>>>> >>> >> > other
>>>> >>> >> > > > > >> features
>>>> >>> >> > > > > >> > > > that
>>>> >>> >> > > > > >> > > > > >> have
>>>> >>> >> > > > > >> > > > > >> > > gone
>>>> >>> >> > > > > >> > > > > >> > > > > >> into
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > downstream
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > facing
>>>> releases
>>>> >>> >> without
>>>> >>> >> > > > > >> robustness
>>>> >>> >> > > > > >> > > > > checks
>>>> >>> >> > > > > >> > > > > >> > (i.e.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > replication),
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > and
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > I'm
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > concerned
>>>> about
>>>> >>> what
>>>> >>> >> > > we're
>>>> >>> >> > > > > >> setting
>>>> >>> >> > > > > >> > > up
>>>> >>> >> > > > > >> > > > if
>>>> >>> >> > > > > >> > > > > >> 2.0
>>>> >>> >> > > > > >> > > > > >> > > goes
>>>> >>> >> > > > > >> > > > > >> > > > > out
>>>> >>> >> > > > > >> > > > > >> > > > > >> > with
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> this
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > feature in
>>>> its
>>>> >>> >> current
>>>> >>> >> > > > state.
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > --
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > Best regards,
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >    - Andy
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > Problems worthy
>>>> of
>>>> >>> attack
>>>> >>> >> > prove
>>>> >>> >> > > > > their
>>>> >>> >> > > > > >> > > worth
>>>> >>> >> > > > > >> > > > by
>>>> >>> >> > > > > >> > > > > >> > > hitting
>>>> >>> >> > > > > >> > > > > >> > > > > >> back. -
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > Piet
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > Hein
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > (via Tom White)
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > --
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > Best regards,
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >    - Andy
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > Problems worthy of
>>>> attack
>>>> >>> >> prove
>>>> >>> >> > > their
>>>> >>> >> > > > > >> worth
>>>> >>> >> > > > > >> > by
>>>> >>> >> > > > > >> > > > > >> hitting
>>>> >>> >> > > > > >> > > > > >> > > > back.
>>>> >>> >> > > > > >> > > > > >> > > > > -
>>>> >>> >> > > > > >> > > > > >> > > > > >> > Piet
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> Hein
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > (via Tom White)
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
>>>> >>> >> > > > > >> > > > > >> > > > > >> > > >>
>>>> >>> >> > > > > >> > > > > >> > > > > >> > >
>>>> >>> >> > > > > >> > > > > >> > > > > >> >
>>>> >>> >> > > > > >> > > > > >> > > > > >>
>>>> >>> >> > > > > >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > > >
>>>> >>> >> > > > > >> > > > > >> > > > >
>>>> >>> >> > > > > >> > > > > >> > > >
>>>> >>> >> > > > > >> > > > > >> > >
>>>> >>> >> > > > > >> > > > > >> >
>>>> >>> >> > > > > >> > > > > >>
>>>> >>> >> > > > > >> > > > >
>>>> >>> >> > > > > >> > > > >
>>>> >>> >> > > > > >> > > > >
>>>> >>> >> > > > > >> > > > > --
>>>> >>> >> > > > > >> > > > > busbey
>>>> >>> >> > > > > >> > > > >
>>>> >>> >> > > > > >> > > >
>>>> >>> >> > > > > >> > >
>>>> >>> >> > > > > >> >
>>>> >>> >> > > > > >>
>>>> >>> >> > > > > >
>>>> >>> >> > > > > >
>>>> >>> >> > > > >
>>>> >>> >> > > >
>>>> >>> >> > >
>>>> >>> >> >
>>>> >>> >>
>>>> >>> >>
>>>> >>> >>
>>>> >>> >> --
>>>> >>> >>
>>>> >>> >> -- Appy
>>>> >>> >>
>>>> >>> >
>>>> >>> >
>>>> >>>
>>>> >>
>>>> >>
>>>> >
>>>>
>>>
>>>
>>
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Stack <st...@duboce.net>]

On Wed, Oct 12, 2016 at 1:58 PM, Stack <st...@duboce.net> wrote:

> Fails again when I do this:
>
> $ mvn clean install -DskipTests assembly:single
> ....
>
>
Ok. Works if I use mvn 3.3.x.
St.Ack





>
> [INFO] ------------------------------------------------------------
> ------------
> [INFO] Building Apache HBase - Server 2.0.0-SNAPSHOT
> [INFO] ------------------------------------------------------------
> ------------
> Downloading: file:/home/stack/hbase.git/hbase-server/src/main/site/
> resources/repo/org/apache/hadoop/hadoop-distcp/2.7.1/
> hadoop-distcp-2.7.1.pom
> Exception in thread "pool-1-thread-1" ------------------------------
> ---------------------
> constituent[0]: file:/usr/share/maven/lib/aether-spi.jar
> constituent[1]: file:/usr/share/maven/lib/maven-settings-builder-3.x.jar
> constituent[2]: file:/usr/share/maven/lib/maven-artifact-3.x.jar
> constituent[3]: file:/usr/share/maven/lib/commons-httpclient.jar
> constituent[4]: file:/usr/share/maven/lib/wagon-provider-api.jar
> constituent[5]: file:/usr/share/maven/lib/maven-core-3.x.jar
> constituent[6]: file:/usr/share/maven/lib/aether-impl.jar
> constituent[7]: file:/usr/share/maven/lib/sisu-inject-bean.jar
> constituent[8]: file:/usr/share/maven/lib/sisu-inject-plexus.jar
> constituent[9]: file:/usr/share/maven/lib/wagon-file.jar
> constituent[10]: file:/usr/share/maven/lib/guava.jar
> constituent[11]: file:/usr/share/maven/lib/maven-compat-3.x.jar
> constituent[12]: file:/usr/share/maven/lib/maven-model-3.x.jar
> constituent[13]: file:/usr/share/maven/lib/plexus-interpolation.jar
> constituent[14]: file:/usr/share/maven/lib/commons-codec.jar
> constituent[15]: file:/usr/share/maven/lib/aether-util.jar
> constituent[16]: file:/usr/share/maven/lib/aether-api.jar
> constituent[17]: file:/usr/share/maven/lib/maven-model-builder-3.x.jar
> constituent[18]: file:/usr/share/maven/lib/maven-repository-metadata-3.x.
> jar
> constituent[19]: file:/usr/share/maven/lib/maven-settings-3.x.jar
> constituent[20]: file:/usr/share/maven/lib/plexus-component-annotations.
> jar
> constituent[21]: file:/usr/share/maven/lib/commons-cli.jar
> constituent[22]: file:/usr/share/maven/lib/maven-embedder-3.x.jar
> constituent[23]: file:/usr/share/maven/lib/maven-plugin-api-3.x.jar
> constituent[24]: file:/usr/share/maven/lib/wagon-http-shaded.jar
> constituent[25]: file:/usr/share/maven/lib/commons-logging.jar
> constituent[26]: file:/usr/share/maven/lib/plexus-sec-dispatcher.jar
> constituent[27]: file:/usr/share/maven/lib/aether-connector-wagon.jar
> constituent[28]: file:/usr/share/maven/lib/plexus-utils.jar
> constituent[29]: file:/usr/share/maven/lib/sisu-guice.jar
> constituent[30]: file:/usr/share/maven/lib/plexus-cipher.jar
> constituent[31]: file:/usr/share/maven/lib/maven-aether-provider-3.x.jar
> ---------------------------------------------------
> java.lang.NoClassDefFoundError: org/apache/commons/lang/StringUtils
> at org.apache.maven.wagon.providers.file.FileWagon.resolveDestinationPath(
> FileWagon.java:206)
> at org.apache.maven.wagon.providers.file.FileWagon.
> resourceExists(FileWagon.java:265)
> at org.sonatype.aether.connector.wagon.WagonRepositoryConnector$
> GetTask.run(WagonRepositoryConnector.java:577)
> at org.sonatype.aether.util.concurrency.RunnableErrorForwarder$1.run(
> RunnableErrorForwarder.java:60)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.ClassNotFoundException: org.apache.commons.lang.
> StringUtils
> at org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy.loadClass(
> SelfFirstStrategy.java:50)
> at org.codehaus.plexus.classworlds.realm.ClassRealm.
> unsynchronizedLoadClass(ClassRealm.java:259)
> at org.codehaus.plexus.classworlds.realm.ClassRealm.
> loadClass(ClassRealm.java:235)
> at org.codehaus.plexus.classworlds.realm.ClassRealm.
> loadClass(ClassRealm.java:227)
> ... 7 more
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/commons/lang/StringUtils
> at org.apache.maven.wagon.providers.file.FileWagon.resolveDestinationPath(
> FileWagon.java:206)
> at org.apache.maven.wagon.providers.file.FileWagon.
> resourceExists(FileWagon.java:265)
> at org.sonatype.aether.connector.wagon.WagonRepositoryConnector$
> GetTask.run(WagonRepositoryConnector.java:577)
> at org.sonatype.aether.util.concurrency.RunnableErrorForwarder$1.run(
> RunnableErrorForwarder.java:60)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.ClassNotFoundException: org.apache.commons.lang.
> StringUtils
> at org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy.loadClass(
> SelfFirstStrategy.java:50)
> at org.codehaus.plexus.classworlds.realm.ClassRealm.
> unsynchronizedLoadClass(ClassRealm.java:259)
> at org.codehaus.plexus.classworlds.realm.ClassRealm.
> loadClass(ClassRealm.java:235)
> at org.codehaus.plexus.classworlds.realm.ClassRealm.
> loadClass(ClassRealm.java:227)
>
>
>
> Does it work for you lot? This is branch checkout.
>
> I want assembly to work so I can distribute the build around a cluster.
>
> Thanks,
> St.Ack.
>
>
>
> On Wed, Oct 12, 2016 at 1:52 PM, Stack <st...@duboce.net> wrote:
>
>> I did a super-clean and recheck out. Now it works. Sorry for the noise.
>> St.Ack
>>
>> On Wed, Oct 12, 2016 at 1:10 PM, Vladimir Rodionov <
>> vladrodionov@gmail.com> wrote:
>>
>>> checked out HBASE-7912
>>>
>>> ran:
>>>
>>> mvn clean install -DskipTests
>>>
>>> successfully.
>>>
>>> -Vlad
>>>
>>> On Wed, Oct 12, 2016 at 1:01 PM, Vladimir Rodionov <
>>> vladrodionov@gmail.com>
>>> wrote:
>>>
>>> > I usually use:
>>> >
>>> > mvn clean install -DskipTests
>>> >
>>> > and it works.
>>> >
>>> > On Wed, Oct 12, 2016 at 1:01 PM, Vladimir Rodionov <
>>> vladrodionov@gmail.com
>>> > > wrote:
>>> >
>>> >> Michael,
>>> >>
>>> >> you can try master + latest patch on HBASE-14123 (v29). No need to use
>>> >> HBASE-7912 branch. I will double check HBASE-7912.
>>> >>
>>> >> -Vlad
>>> >>
>>> >> On Wed, Oct 12, 2016 at 12:31 PM, Stack <st...@duboce.net> wrote:
>>> >>
>>> >>> More info:
>>> >>>
>>> >>> stack@ve0524:~/hbase.git$ git checkout origin/HBASE-7912 -b 7912v2
>>> >>> Branch 7912v2 set up to track remote branch HBASE-7912 from origin.
>>> >>> Switched to a new branch '7912v2'
>>> >>> stack@ve0524:~/hbase.git$ java -version
>>> >>> java version "1.8.0_101"
>>> >>> Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
>>> >>> Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode)
>>> >>> stack@ve0524:~/hbase.git$ mvn clean install -DskipTests &>
>>> /tmp/out.txt
>>> >>>
>>> >>> ...
>>> >>>
>>> >>> St.Ack
>>> >>>
>>> >>>
>>> >>> On Wed, Oct 12, 2016 at 12:29 PM, Stack <st...@duboce.net> wrote:
>>> >>>
>>> >>> > Interesting. When I try it fails w/ below:
>>> >>> >
>>> >>> > [INFO] 26 warnings
>>> >>> > 322 [INFO] ------------------------------
>>> >>> -------------------------------
>>> >>> > 323 [INFO] ------------------------------
>>> >>> -------------------------------
>>> >>> > 324 [ERROR] COMPILATION ERROR :
>>> >>> > 325 [INFO] ------------------------------
>>> >>> -------------------------------
>>> >>> > 326 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.j
>>> ava:[48,8]
>>> >>> > org.apache.hadoop.hbase.io.encoding.rowindexV2.RowIndexCodecV2 is
>>> not
>>> >>> > abstract and does not override abstract method createSeeker(org.ap
>>> >>> >  ache.hadoop.hbase.CellComparator,org.apache.hadoop.hbase.io.
>>> >>> encoding.HFileBlockDecodingContext)
>>> >>> > in org.apache.hadoop.hbase.io.encoding.DataBlockEncoder
>>> >>> > 327 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.j
>>> >>> ava:[143,3]
>>> >>> > method does not override or implement a method from a supertype
>>> >>> > 328 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.j
>>> >>> ava:[147,29]
>>> >>> > incompatible types: java.nio.ByteBuffer cannot be converted to
>>> >>> > org.apache.hadoop.hbase.nio.ByteBuff
>>> >>> > 329 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.j
>>> >>> ava:[148,33]
>>> >>> > cannot find symbol
>>> >>> > 330   symbol:   method getKeyDeepCopy()
>>> >>> > 331   location: variable seeker of type org.apache.hadoop.hbase.io
>>> .
>>> >>> > encoding.DataBlockEncoder.EncodedSeeker
>>> >>> > 332 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.j
>>> >>> ava:[153,3]
>>> >>> > method does not override or implement a method from a supertype
>>> >>> > 333 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV1/RowIndexCodecV1.j
>>> ava:[45,8]
>>> >>> > org.apache.hadoop.hbase.io.encoding.rowindexV1.RowIndexCodecV1 is
>>> not
>>> >>> > abstract and does not override abstract method createSeeker(org.ap
>>> >>> >  ache.hadoop.hbase.CellComparator,org.apache.hadoop.hbase.io.
>>> >>> encoding.HFileBlockDecodingContext)
>>> >>> > in org.apache.hadoop.hbase.io.encoding.DataBlockEncoder
>>> >>> > 334 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV1/RowIndexCodecV1.j
>>> >>> ava:[145,3]
>>> >>> > method does not override or implement a method from a supertype
>>> >>> > 335 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV1/RowIndexCodecV1.j
>>> >>> ava:[158,3]
>>> >>> > method does not override or implement a method from a supertype
>>> >>> > 336 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>>> >>> java:[46,8]
>>> >>> > org.apache.hadoop.hbase.io.encoding.rowindexV2.RowIndexSeekerV2
>>> is not
>>> >>> > abstract and does not override abstract method compareKey(org.ap
>>> >>> >  ache.hadoop.hbase.CellComparator,org.apache.hadoop.hbase.Cell) in
>>> >>> > org.apache.hadoop.hbase.io.encoding.DataBlockEncoder.EncodedSeeker
>>> >>> > 337 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>>> >>> java:[79,3]
>>> >>> > method does not override or implement a method from a supertype
>>> >>> > 338 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>>> >>> java:[117,3]
>>> >>> > method does not override or implement a method from a supertype
>>> >>> > 339 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>>> >>> java:[190,3]
>>> >>> > method does not override or implement a method from a supertype
>>> >>> > 340 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>>> >>> java:[214,3]
>>> >>> > method does not override or implement a method from a supertype
>>> >>> > 341 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>>> >>> java:[349,3]
>>> >>> > method does not override or implement a method from a supertype
>>> >>> > 342 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>>> >>> java:[355,3]
>>> >>> > method does not override or implement a method from a supertype
>>> >>> > 343 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>>> >>> java:[421,36]
>>> >>> > no suitable method found for uncompressTags(java.nio.
>>> >>> > ByteBuffer,byte[],int,int)
>>> >>> > 344     method org.apache.hadoop.hbase.io.TagCompressionContext.
>>> >>> > uncompressTags(java.io.InputStream,byte[],int,int) is not
>>> applicable
>>> >>> > 345       (argument mismatch; java.nio.ByteBuffer cannot be
>>> converted
>>> >>> to
>>> >>> > java.io.InputStream)
>>> >>> > 346     method org.apache.hadoop.hbase.io.TagCompressionContext.
>>> >>> > uncompressTags(org.apache.hadoop.hbase.nio.ByteBuff,byte[],int,int)
>>> is
>>> >>> > not applicable
>>> >>> > 347       (argument mismatch; java.nio.ByteBuffer cannot be
>>> converted
>>> >>> to
>>> >>> > org.apache.hadoop.hbase.nio.ByteBuff)
>>> >>> >
>>> >>> > ....
>>> >>> >
>>> >>> > St.Ack
>>> >>> >
>>> >>> > On Tue, Oct 11, 2016 at 5:12 PM, Apekshit Sharma <
>>> appy@cloudera.com>
>>> >>> > wrote:
>>> >>> >
>>> >>> >> @stack, it compiled for me.
>>> >>> >>
>>> >>> >> Also tried few commands, and have to say, it's well designed from
>>> user
>>> >>> >> commands perspective.
>>> >>> >>
>>> >>> >> On Tue, Oct 11, 2016 at 5:08 PM, Stack <st...@duboce.net> wrote:
>>> >>> >>
>>> >>> >> > On Tue, Oct 11, 2016 at 1:50 PM, Vladimir Rodionov <
>>> >>> >> vladrodionov@gmail.com
>>> >>> >> > >
>>> >>> >> > wrote:
>>> >>> >> >
>>> >>> >> > > Michael,
>>> >>> >> > >
>>> >>> >> > > Its in HBASE-7912
>>> >>> >> > >
>>> >>> >> > > This is tip of git log:
>>> >>> >> > >
>>> >>> >> > > commit a072f6f49a26a7259ff2aaef6cb56d85eb592482
>>> >>> >> > > Author: Frank Welsch <fw...@jps.net>
>>> >>> >> > > Date:   Fri Sep 23 18:00:42 2016 -0400
>>> >>> >> > >
>>> >>> >> > >     HBASE-16574 Book updates for backup and restore
>>> >>> >> > >
>>> >>> >> > > commit b14e2ab1c24e65ff88dd4c579acf83cb4ed0605e
>>> >>> >> > > Author: tedyu <yu...@gmail.com>
>>> >>> >> > > Date:   Wed Oct 5 16:29:40 2016 -0700
>>> >>> >> > >
>>> >>> >> > >     HBASE-16727 Backup refactoring: remove MR dependencies
>>> from
>>> >>> >> HMaster
>>> >>> >> > > (Vladimir Rodionov)
>>> >>> >> > >
>>> >>> >> > >
>>> >>> >> > Thanks. I have that. I tried it and it doesn't compile for me.
>>> Does
>>> >>> it
>>> >>> >> > compile for you?
>>> >>> >> > Thanks,
>>> >>> >> > M
>>> >>> >> >
>>> >>> >> >
>>> >>> >> >
>>> >>> >> > > -Vlad
>>> >>> >> > >
>>> >>> >> > > On Tue, Oct 11, 2016 at 1:44 PM, Stack <st...@duboce.net>
>>> wrote:
>>> >>> >> > >
>>> >>> >> > > > Which branch do I check out to try it? HBASE-7912 is not
>>> it. I
>>> >>> don't
>>> >>> >> > see
>>> >>> >> > > an
>>> >>> >> > > > HBASE-16727...
>>> >>> >> > > > Thanks,
>>> >>> >> > > > M
>>> >>> >> > > >
>>> >>> >> > > >
>>> >>> >> > > > On Tue, Oct 11, 2016 at 11:41 AM, Vladimir Rodionov <
>>> >>> >> > > > vladrodionov@gmail.com>
>>> >>> >> > > > wrote:
>>> >>> >> > > >
>>> >>> >> > > > > The last patch is on review board:
>>> >>> >> > > > > https://reviews.apache.org/r/52748
>>> >>> >> > > > >
>>> >>> >> > > > > On Mon, Oct 10, 2016 at 3:52 PM, Vladimir Rodionov <
>>> >>> >> > > > vladrodionov@gmail.com
>>> >>> >> > > > > >
>>> >>> >> > > > > wrote:
>>> >>> >> > > > >
>>> >>> >> > > > > > >> How hard to put in an hbase-backup module?
>>> hbase-server
>>> >>> is
>>> >>> >> fat
>>> >>> >> > > > enough
>>> >>> >> > > > > > >> already. Could be done as a follow-up.
>>> >>> >> > > > > >
>>> >>> >> > > > > > https://issues.apache.org/jira/browse/HBASE-16727?
>>> >>> >> > > > > > focusedCommentId=15531237&page=com.atlassian.jira.
>>> >>> >> > > > > > plugin.system.issuetabpanels:c
>>> >>> omment-tabpanel#comment-15531237
>>> >>> >> > > > > >
>>> >>> >> > > > > > Can we do merge first? Then we can discuss separate
>>> module.
>>> >>> >> > > > > >
>>> >>> >> > > > > > -Vlad
>>> >>> >> > > > > >
>>> >>> >> > > > > > On Mon, Oct 10, 2016 at 3:44 PM, Ted Yu <
>>> >>> yuzhihong@gmail.com>
>>> >>> >> > wrote:
>>> >>> >> > > > > >
>>> >>> >> > > > > >> Looks like the first quote was cut off.
>>> >>> >> > > > > >> The original sentence was:
>>> >>> >> > > > > >>
>>> >>> >> > > > > >> bq. no mapreduce job launched from master or region
>>> server.
>>> >>> >> > > > > >>
>>> >>> >> > > > > >> mapreduce job is launched from the node where command
>>> line
>>> >>> >> tool is
>>> >>> >> > > > run.
>>> >>> >> > > > > >>
>>> >>> >> > > > > >> On Mon, Oct 10, 2016 at 3:38 PM, Stack <
>>> stack@duboce.net>
>>> >>> >> wrote:
>>> >>> >> > > > > >>
>>> >>> >> > > > > >> > bq. launched from master or region server.
>>> >>> >> > > > > >> >
>>> >>> >> > > > > >> > What does this mean please? Has to be run from
>>> Master or
>>> >>> >> > > > RegionServer?
>>> >>> >> > > > > >> Can
>>> >>> >> > > > > >> > it be run from another node altogether?
>>> >>> >> > > > > >> >
>>> >>> >> > > > > >> > On Mon, Oct 10, 2016 at 1:44 PM, Vladimir Rodionov <
>>> >>> >> > > > > >> vladrodionov@gmail.com
>>> >>> >> > > > > >> > >
>>> >>> >> > > > > >> > wrote:
>>> >>> >> > > > > >> >
>>> >>> >> > > > > >> > > >> mapreduce dependency has been moved to client
>>> side
>>> >>> - no
>>> >>> >> > > > mapreduce
>>> >>> >> > > > > >> job
>>> >>> >> > > > > >> > >
>>> >>> >> > > > > >> > > 1. We have no code in the client module anymore,
>>> due to
>>> >>> >> > > dependency
>>> >>> >> > > > > on
>>> >>> >> > > > > >> > > internal server API (HFile and WAL access).
>>> >>> >> > > > > >> > > 2. Backup/ restore are client - driven operations,
>>> but
>>> >>> all
>>> >>> >> the
>>> >>> >> > > > code
>>> >>> >> > > > > >> > resides
>>> >>> >> > > > > >> > > in the server module
>>> >>> >> > > > > >> > >
>>> >>> >> > > > > >> >
>>> >>> >> > > > > >> > How hard to put in an hbase-backup module?
>>> hbase-server
>>> >>> is
>>> >>> >> fat
>>> >>> >> > > > enough
>>> >>> >> > > > > >> > already. Could be done as a follow-up.
>>> >>> >> > > > > >> >
>>> >>> >> > > > > >> > Thanks,
>>> >>> >> > > > > >> > St.Ack
>>> >>> >> > > > > >> >
>>> >>> >> > > > > >> >
>>> >>> >> > > > > >> >
>>> >>> >> > > > > >> > > 3. No MR in Master, no procedure - driven
>>> execution.
>>> >>> >> > > > > >> > > 4. Old good MR from command-line.
>>> >>> >> > > > > >> > > 5. Security was simplified and now only super-user
>>> is
>>> >>> >> allowed
>>> >>> >> > to
>>> >>> >> > > > run
>>> >>> >> > > > > >> > > backup/restores.
>>> >>> >> > > > > >> > > 6. HBase Backup API was gone due to 1. Now only
>>> >>> >> command-line
>>> >>> >> > > > access
>>> >>> >> > > > > to
>>> >>> >> > > > > >> > > backup tools.
>>> >>> >> > > > > >> > >
>>> >>> >> > > > > >> > > These consequences of refactoring has been
>>> discussed in
>>> >>> >> > > > HBASE-16727.
>>> >>> >> > > > > >> > >
>>> >>> >> > > > > >> > > -Vlad
>>> >>> >> > > > > >> > >
>>> >>> >> > > > > >> > >
>>> >>> >> > > > > >> > >
>>> >>> >> > > > > >> > >
>>> >>> >> > > > > >> > > On Mon, Oct 10, 2016 at 1:31 PM, Ted Yu <
>>> >>> >> yuzhihong@gmail.com>
>>> >>> >> > > > > wrote:
>>> >>> >> > > > > >> > >
>>> >>> >> > > > > >> > > > Reviving this thread.
>>> >>> >> > > > > >> > > >
>>> >>> >> > > > > >> > > > The following has taken place:
>>> >>> >> > > > > >> > > >
>>> >>> >> > > > > >> > > > mapreduce dependency has been moved to client
>>> side -
>>> >>> no
>>> >>> >> > > > mapreduce
>>> >>> >> > > > > >> job
>>> >>> >> > > > > >> > > > launched from master or region server.
>>> >>> >> > > > > >> > > > document patch (HBASE-16574) has been integrated.
>>> >>> >> > > > > >> > > > Updated mega patch has been attached to
>>> HBASE-14123:
>>> >>> this
>>> >>> >> > > covers
>>> >>> >> > > > > the
>>> >>> >> > > > > >> > > > refactor in #1 above and the protobuf 3 merge.
>>> >>> >> > > > > >> > > >
>>> >>> >> > > > > >> > > > If community has more feedback on the merge
>>> >>> proposal, I
>>> >>> >> > would
>>> >>> >> > > > love
>>> >>> >> > > > > >> to
>>> >>> >> > > > > >> > > hear
>>> >>> >> > > > > >> > > > it.
>>> >>> >> > > > > >> > > >
>>> >>> >> > > > > >> > > > Thanks
>>> >>> >> > > > > >> > > >
>>> >>> >> > > > > >> > > > On Thu, Sep 22, 2016 at 10:31 AM, Sean Busbey <
>>> >>> >> > > > > busbey@cloudera.com>
>>> >>> >> > > > > >> > > wrote:
>>> >>> >> > > > > >> > > >
>>> >>> >> > > > > >> > > > > I'd like to see the docs proposed on
>>> HBASE-16574
>>> >>> >> > integrated
>>> >>> >> > > > into
>>> >>> >> > > > > >> our
>>> >>> >> > > > > >> > > > > project's documentation prior to merge.
>>> >>> >> > > > > >> > > > >
>>> >>> >> > > > > >> > > > > On Thu, Sep 22, 2016 at 9:02 AM, Ted Yu <
>>> >>> >> > > yuzhihong@gmail.com>
>>> >>> >> > > > > >> wrote:
>>> >>> >> > > > > >> > > > > > This feature can be marked experimental due
>>> to
>>> >>> some
>>> >>> >> > > > > limitations
>>> >>> >> > > > > >> > such
>>> >>> >> > > > > >> > > as
>>> >>> >> > > > > >> > > > > > security.
>>> >>> >> > > > > >> > > > > >
>>> >>> >> > > > > >> > > > > > Your previous round of comments have been
>>> >>> addressed.
>>> >>> >> > > > > >> > > > > > Command line tool has gone through:
>>> >>> >> > > > > >> > > > > >
>>> >>> >> > > > > >> > > > > > HBASE-16620 Fix backup command-line tool
>>> >>> usability
>>> >>> >> > issues
>>> >>> >> > > > > >> > > > > > HBASE-16655 hbase backup describe with
>>> incorrect
>>> >>> >> backup
>>> >>> >> > id
>>> >>> >> > > > > >> results
>>> >>> >> > > > > >> > in
>>> >>> >> > > > > >> > > > NPE
>>> >>> >> > > > > >> > > > > >
>>> >>> >> > > > > >> > > > > > The updated doc has been attached to
>>> HBASE-16574.
>>> >>> >> > > > > >> > > > > >
>>> >>> >> > > > > >> > > > > > Cheers
>>> >>> >> > > > > >> > > > > >
>>> >>> >> > > > > >> > > > > > On Thu, Sep 22, 2016 at 8:53 AM, Stack <
>>> >>> >> > stack@duboce.net>
>>> >>> >> > > > > >> wrote:
>>> >>> >> > > > > >> > > > > >
>>> >>> >> > > > > >> > > > > >> On Wed, Sep 21, 2016 at 7:43 AM, Ted Yu <
>>> >>> >> > > > yuzhihong@gmail.com
>>> >>> >> > > > > >
>>> >>> >> > > > > >> > > wrote:
>>> >>> >> > > > > >> > > > > >>
>>> >>> >> > > > > >> > > > > >> > Are there more (review) comments ?
>>> >>> >> > > > > >> > > > > >> >
>>> >>> >> > > > > >> > > > > >> >
>>> >>> >> > > > > >> > > > > >> Are outstanding comments addressed?
>>> >>> >> > > > > >> > > > > >>
>>> >>> >> > > > > >> > > > > >> I don't see answer to my 'is this
>>> >>> experimental/will
>>> >>> >> it
>>> >>> >> > be
>>> >>> >> > > > > >> marked
>>> >>> >> > > > > >> > > > > >> experimental' question.
>>> >>> >> > > > > >> > > > > >>
>>> >>> >> > > > > >> > > > > >> I ran into some issues trying to use the
>>> >>> feature and
>>> >>> >> > > > > suggested
>>> >>> >> > > > > >> > that
>>> >>> >> > > > > >> > > a
>>> >>> >> > > > > >> > > > > >> feature likes this needs polish else it'll
>>> just
>>> >>> rot,
>>> >>> >> > > > unused.
>>> >>> >> > > > > >> Has
>>> >>> >> > > > > >> > > > polish
>>> >>> >> > > > > >> > > > > >> been applied? All ready for another 'user'
>>> test?
>>> >>> >> > Suggest
>>> >>> >> > > > that
>>> >>> >> > > > > >> you
>>> >>> >> > > > > >> > > > update
>>> >>> >> > > > > >> > > > > >> here going forward for the benefit of those
>>> >>> trying
>>> >>> >> to
>>> >>> >> > > > follow
>>> >>> >> > > > > >> along
>>> >>> >> > > > > >> > > and
>>> >>> >> > > > > >> > > > > who
>>> >>> >> > > > > >> > > > > >> are not watching JIRA change fly-by.
>>> >>> >> > > > > >> > > > > >>
>>> >>> >> > > > > >> > > > > >> It looks like doc got a revision -- I have
>>> to
>>> >>> check
>>> >>> >> --
>>> >>> >> > to
>>> >>> >> > > > > take
>>> >>> >> > > > > >> on
>>> >>> >> > > > > >> > > > > >> suggestion made above but again, suggest,
>>> that
>>> >>> this
>>> >>> >> > > thread
>>> >>> >> > > > > gets
>>> >>> >> > > > > >> > > > updated.
>>> >>> >> > > > > >> > > > > >>
>>> >>> >> > > > > >> > > > > >> Thanks,
>>> >>> >> > > > > >> > > > > >> St.Ack
>>> >>> >> > > > > >> > > > > >>
>>> >>> >> > > > > >> > > > > >>
>>> >>> >> > > > > >> > > > > >>
>>> >>> >> > > > > >> > > > > >> > Thanks
>>> >>> >> > > > > >> > > > > >> >
>>> >>> >> > > > > >> > > > > >> > On Tue, Sep 20, 2016 at 10:02 AM, Devaraj
>>> Das
>>> >>> <
>>> >>> >> > > > > >> > > ddas@hortonworks.com
>>> >>> >> > > > > >> > > > >
>>> >>> >> > > > > >> > > > > >> > wrote:
>>> >>> >> > > > > >> > > > > >> >
>>> >>> >> > > > > >> > > > > >> > > Just reviving this thread. Thanks Sean,
>>> >>> Stack,
>>> >>> >> > Dima,
>>> >>> >> > > > and
>>> >>> >> > > > > >> > others
>>> >>> >> > > > > >> > > > for
>>> >>> >> > > > > >> > > > > the
>>> >>> >> > > > > >> > > > > >> > > thorough reviews and testing. Thanks
>>> Ted and
>>> >>> >> Vlad
>>> >>> >> > for
>>> >>> >> > > > > >> taking
>>> >>> >> > > > > >> > > care
>>> >>> >> > > > > >> > > > of
>>> >>> >> > > > > >> > > > > >> the
>>> >>> >> > > > > >> > > > > >> > > feedback. Are we all good to do the
>>> merge
>>> >>> now?
>>> >>> >> > Rather
>>> >>> >> > > > do
>>> >>> >> > > > > >> > sooner
>>> >>> >> > > > > >> > > > than
>>> >>> >> > > > > >> > > > > >> > later.
>>> >>> >> > > > > >> > > > > >> > > ______________________________
>>> __________
>>> >>> >> > > > > >> > > > > >> > > From: saint.ack@gmail.com <
>>> >>> saint.ack@gmail.com>
>>> >>> >> on
>>> >>> >> > > > > behalf
>>> >>> >> > > > > >> of
>>> >>> >> > > > > >> > > > Stack
>>> >>> >> > > > > >> > > > > <
>>> >>> >> > > > > >> > > > > >> > > stack@duboce.net>
>>> >>> >> > > > > >> > > > > >> > > Sent: Monday, September 12, 2016 1:18 PM
>>> >>> >> > > > > >> > > > > >> > > To: HBase Dev List
>>> >>> >> > > > > >> > > > > >> > > Subject: Re: [DISCUSSION] Merge Backup /
>>> >>> >> Restore -
>>> >>> >> > > > Branch
>>> >>> >> > > > > >> > > > HBASE-7912
>>> >>> >> > > > > >> > > > > >> > >
>>> >>> >> > > > > >> > > > > >> > > On Mon, Sep 12, 2016 at 12:19 PM, Ted
>>> Yu <
>>> >>> >> > > > > >> yuzhihong@gmail.com
>>> >>> >> > > > > >> > >
>>> >>> >> > > > > >> > > > > wrote:
>>> >>> >> > > > > >> > > > > >> > >
>>> >>> >> > > > > >> > > > > >> > > > Mega patch (rev 18) is on HBASE-14123.
>>> >>> >> > > > > >> > > > > >> > > >
>>> >>> >> > > > > >> > > > > >> > > > Please comment on HBASE-14123 on how
>>> you
>>> >>> want
>>> >>> >> to
>>> >>> >> > > > > review.
>>> >>> >> > > > > >> > > > > >> > > >
>>> >>> >> > > > > >> > > > > >> > >
>>> >>> >> > > > > >> > > > > >> > >
>>> >>> >> > > > > >> > > > > >> > > Yeah. That was my lost tab. Last rb was
>>> 6
>>> >>> months
>>> >>> >> > ago.
>>> >>> >> > > > > >> Suggest
>>> >>> >> > > > > >> > > > > updating
>>> >>> >> > > > > >> > > > > >> > it.
>>> >>> >> > > > > >> > > > > >> > > RB is pretty good for review. Patch is
>>> only
>>> >>> >> 1.5M so
>>> >>> >> > > > > should
>>> >>> >> > > > > >> be
>>> >>> >> > > > > >> > > > fine.
>>> >>> >> > > > > >> > > > > >> > >
>>> >>> >> > > > > >> > > > > >> > > St.Ack
>>> >>> >> > > > > >> > > > > >> > >
>>> >>> >> > > > > >> > > > > >> > >
>>> >>> >> > > > > >> > > > > >> > > >
>>> >>> >> > > > > >> > > > > >> > > > Thanks
>>> >>> >> > > > > >> > > > > >> > > >
>>> >>> >> > > > > >> > > > > >> > > > On Mon, Sep 12, 2016 at 12:15 PM,
>>> Stack <
>>> >>> >> > > > > >> stack@duboce.net>
>>> >>> >> > > > > >> > > > wrote:
>>> >>> >> > > > > >> > > > > >> > > >
>>> >>> >> > > > > >> > > > > >> > > > > On review of the 'patch', do I just
>>> >>> compare
>>> >>> >> the
>>> >>> >> > > > > branch
>>> >>> >> > > > > >> to
>>> >>> >> > > > > >> > > > > master or
>>> >>> >> > > > > >> > > > > >> > is
>>> >>> >> > > > > >> > > > > >> > > > > there a megapatch posted somewhere
>>> (I
>>> >>> think
>>> >>> >> I
>>> >>> >> > saw
>>> >>> >> > > > one
>>> >>> >> > > > > >> but
>>> >>> >> > > > > >> > it
>>> >>> >> > > > > >> > > > > seemed
>>> >>> >> > > > > >> > > > > >> > > stale
>>> >>> >> > > > > >> > > > > >> > > > > and then I 'lost' the tab). Sorry
>>> for
>>> >>> dumb
>>> >>> >> > > > question.
>>> >>> >> > > > > >> > > > > >> > > > > St.Ack
>>> >>> >> > > > > >> > > > > >> > > > >
>>> >>> >> > > > > >> > > > > >> > > > > On Mon, Sep 12, 2016 at 12:01 PM,
>>> Stack
>>> >>> <
>>> >>> >> > > > > >> stack@duboce.net
>>> >>> >> > > > > >> > >
>>> >>> >> > > > > >> > > > > wrote:
>>> >>> >> > > > > >> > > > > >> > > > >
>>> >>> >> > > > > >> > > > > >> > > > > > Late to the game. A few comments
>>> after
>>> >>> >> > > rereading
>>> >>> >> > > > > this
>>> >>> >> > > > > >> > > thread
>>> >>> >> > > > > >> > > > > as a
>>> >>> >> > > > > >> > > > > >> > > > 'user'.
>>> >>> >> > > > > >> > > > > >> > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > > + Before merge, a user-facing
>>> feature
>>> >>> like
>>> >>> >> > this
>>> >>> >> > > > > >> should
>>> >>> >> > > > > >> > > work
>>> >>> >> > > > > >> > > > > (If
>>> >>> >> > > > > >> > > > > >> > this
>>> >>> >> > > > > >> > > > > >> > > is
>>> >>> >> > > > > >> > > > > >> > > > > "higher-bar
>>> >>> >> > > > > >> > > > > >> > > > > > for new features", bring it on --
>>> >>> smile).
>>> >>> >> > > > > >> > > > > >> > > > > > + As a user, I tried the branch
>>> with
>>> >>> tools
>>> >>> >> > > after
>>> >>> >> > > > > >> > reviewing
>>> >>> >> > > > > >> > > > the
>>> >>> >> > > > > >> > > > > >> > > > > just-posted
>>> >>> >> > > > > >> > > > > >> > > > > > doc. I had an 'interesting'
>>> experience
>>> >>> >> (left
>>> >>> >> > > > > >> comments up
>>> >>> >> > > > > >> > > on
>>> >>> >> > > > > >> > > > > >> > issue). I
>>> >>> >> > > > > >> > > > > >> > > > > think
>>> >>> >> > > > > >> > > > > >> > > > > > the tooling/doc. important to get
>>> >>> right.
>>> >>> >> If
>>> >>> >> > it
>>> >>> >> > > > > breaks
>>> >>> >> > > > > >> > > easily
>>> >>> >> > > > > >> > > > > or
>>> >>> >> > > > > >> > > > > >> is
>>> >>> >> > > > > >> > > > > >> > > > > > inconsistent (or lacks 'polish'),
>>> >>> >> operators
>>> >>> >> > > will
>>> >>> >> > > > > >> judge
>>> >>> >> > > > > >> > the
>>> >>> >> > > > > >> > > > > whole
>>> >>> >> > > > > >> > > > > >> > > > > > backup/restore tooling chain as
>>> not
>>> >>> >> > trustworthy
>>> >>> >> > > > and
>>> >>> >> > > > > >> > > abandon
>>> >>> >> > > > > >> > > > > it.
>>> >>> >> > > > > >> > > > > >> > Lets
>>> >>> >> > > > > >> > > > > >> > > > not
>>> >>> >> > > > > >> > > > > >> > > > > > have this happen to this feature.
>>> >>> >> > > > > >> > > > > >> > > > > > + Matteo's suggestion (with a
>>> helpful
>>> >>> >> starter
>>> >>> >> > > > list)
>>> >>> >> > > > > >> that
>>> >>> >> > > > > >> > > > there
>>> >>> >> > > > > >> > > > > >> > needs
>>> >>> >> > > > > >> > > > > >> > > to
>>> >>> >> > > > > >> > > > > >> > > > > be
>>> >>> >> > > > > >> > > > > >> > > > > > explicit qualification on what is
>>> >>> actually
>>> >>> >> > > being
>>> >>> >> > > > > >> > delivered
>>> >>> >> > > > > >> > > > --
>>> >>> >> > > > > >> > > > > >> > > > including a
>>> >>> >> > > > > >> > > > > >> > > > > > listing of limitations (some look
>>> >>> serious
>>> >>> >> > such
>>> >>> >> > > as
>>> >>> >> > > > > >> data
>>> >>> >> > > > > >> > > bleed
>>> >>> >> > > > > >> > > > > from
>>> >>> >> > > > > >> > > > > >> > > other
>>> >>> >> > > > > >> > > > > >> > > > > > regions in WALs, but maybe I don't
>>> >>> care
>>> >>> >> for
>>> >>> >> > my
>>> >>> >> > > > use
>>> >>> >> > > > > >> > > case...)
>>> >>> >> > > > > >> > > > --
>>> >>> >> > > > > >> > > > > >> > needs
>>> >>> >> > > > > >> > > > > >> > > to
>>> >>> >> > > > > >> > > > > >> > > > > > accompany the merge. Lets fold
>>> them
>>> >>> into
>>> >>> >> the
>>> >>> >> > > user
>>> >>> >> > > > > >> doc.
>>> >>> >> > > > > >> > in
>>> >>> >> > > > > >> > > > the
>>> >>> >> > > > > >> > > > > >> > > technical
>>> >>> >> > > > > >> > > > > >> > > > > > overview area as suggested so user
>>> >>> >> > expectations
>>> >>> >> > > > are
>>> >>> >> > > > > >> > > properly
>>> >>> >> > > > > >> > > > > >> > managed
>>> >>> >> > > > > >> > > > > >> > > > > > (otherwise, they expect the world
>>> and
>>> >>> will
>>> >>> >> > just
>>> >>> >> > > > > give
>>> >>> >> > > > > >> up
>>> >>> >> > > > > >> > > when
>>> >>> >> > > > > >> > > > > we
>>> >>> >> > > > > >> > > > > >> > fall
>>> >>> >> > > > > >> > > > > >> > > > > > short). Vladimir did a list of
>>> what
>>> >>> is in
>>> >>> >> > each
>>> >>> >> > > of
>>> >>> >> > > > > the
>>> >>> >> > > > > >> > > phases
>>> >>> >> > > > > >> > > > > >> above
>>> >>> >> > > > > >> > > > > >> > > > which
>>> >>> >> > > > > >> > > > > >> > > > > > would serve as a good start.
>>> >>> >> > > > > >> > > > > >> > > > > > + Is this feature 'experimental'
>>> >>> (Matteo
>>> >>> >> asks
>>> >>> >> > > > > above).
>>> >>> >> > > > > >> > I'd
>>> >>> >> > > > > >> > > > > prefer
>>> >>> >> > > > > >> > > > > >> it
>>> >>> >> > > > > >> > > > > >> > > is
>>> >>> >> > > > > >> > > > > >> > > > > > not. If it is, it should be
>>> labelled
>>> >>> all
>>> >>> >> over
>>> >>> >> > > > that
>>> >>> >> > > > > >> it is
>>> >>> >> > > > > >> > > > so. I
>>> >>> >> > > > > >> > > > > >> see
>>> >>> >> > > > > >> > > > > >> > > > > current
>>> >>> >> > > > > >> > > > > >> > > > > > state called out as a '...
>>> technical
>>> >>> >> preview
>>> >>> >> > > > > >> feature'.
>>> >>> >> > > > > >> > > Does
>>> >>> >> > > > > >> > > > > this
>>> >>> >> > > > > >> > > > > >> > mean
>>> >>> >> > > > > >> > > > > >> > > > > > not-for-users?
>>> >>> >> > > > > >> > > > > >> > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > > St.Ack
>>> >>> >> > > > > >> > > > > >> > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > > On Mon, Sep 12, 2016 at 8:03 AM,
>>> Ted
>>> >>> Yu <
>>> >>> >> > > > > >> > > > yuzhihong@gmail.com>
>>> >>> >> > > > > >> > > > > >> > wrote:
>>> >>> >> > > > > >> > > > > >> > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> Sean:
>>> >>> >> > > > > >> > > > > >> > > > > >> Do you have more comments ?
>>> >>> >> > > > > >> > > > > >> > > > > >>
>>> >>> >> > > > > >> > > > > >> > > > > >> Cheers
>>> >>> >> > > > > >> > > > > >> > > > > >>
>>> >>> >> > > > > >> > > > > >> > > > > >> On Fri, Sep 9, 2016 at 1:42 PM,
>>> >>> Vladimir
>>> >>> >> > > > Rodionov
>>> >>> >> > > > > <
>>> >>> >> > > > > >> > > > > >> > > > > vladrodionov@gmail.com
>>> >>> >> > > > > >> > > > > >> > > > > >> >
>>> >>> >> > > > > >> > > > > >> > > > > >> wrote:
>>> >>> >> > > > > >> > > > > >> > > > > >>
>>> >>> >> > > > > >> > > > > >> > > > > >> > Sean,
>>> >>> >> > > > > >> > > > > >> > > > > >> >
>>> >>> >> > > > > >> > > > > >> > > > > >> > Backup/Restore can fail due to
>>> >>> various
>>> >>> >> > > > reasons:
>>> >>> >> > > > > >> > network
>>> >>> >> > > > > >> > > > > outage
>>> >>> >> > > > > >> > > > > >> > > > > (cluster
>>> >>> >> > > > > >> > > > > >> > > > > >> > wide), various time-outs in
>>> HBase
>>> >>> and
>>> >>> >> HDFS
>>> >>> >> > > > > layer,
>>> >>> >> > > > > >> M/R
>>> >>> >> > > > > >> > > > > failure
>>> >>> >> > > > > >> > > > > >> > due
>>> >>> >> > > > > >> > > > > >> > > to
>>> >>> >> > > > > >> > > > > >> > > > > >> "HDFS
>>> >>> >> > > > > >> > > > > >> > > > > >> > exceeded quota", user error
>>> (manual
>>> >>> >> > deletion
>>> >>> >> > > > of
>>> >>> >> > > > > >> data)
>>> >>> >> > > > > >> > > and
>>> >>> >> > > > > >> > > > > so
>>> >>> >> > > > > >> > > > > >> on
>>> >>> >> > > > > >> > > > > >> > so
>>> >>> >> > > > > >> > > > > >> > > > on.
>>> >>> >> > > > > >> > > > > >> > > > > >> That
>>> >>> >> > > > > >> > > > > >> > > > > >> > is impossible to enumerate all
>>> >>> possible
>>> >>> >> > > types
>>> >>> >> > > > of
>>> >>> >> > > > > >> > > failures
>>> >>> >> > > > > >> > > > > in a
>>> >>> >> > > > > >> > > > > >> > > > > >> distributed
>>> >>> >> > > > > >> > > > > >> > > > > >> > system - that is not our
>>> goal/task.
>>> >>> >> > > > > >> > > > > >> > > > > >> >
>>> >>> >> > > > > >> > > > > >> > > > > >> > We focus completely on backup
>>> >>> system
>>> >>> >> table
>>> >>> >> > > > > >> > consistency
>>> >>> >> > > > > >> > > > in a
>>> >>> >> > > > > >> > > > > >> > > presence
>>> >>> >> > > > > >> > > > > >> > > > > of
>>> >>> >> > > > > >> > > > > >> > > > > >> any
>>> >>> >> > > > > >> > > > > >> > > > > >> > type of failure. That is what I
>>> >>> call
>>> >>> >> > > > "tolerance
>>> >>> >> > > > > to
>>> >>> >> > > > > >> > > > > failures".
>>> >>> >> > > > > >> > > > > >> > > > > >> >
>>> >>> >> > > > > >> > > > > >> > > > > >> > On a failure:
>>> >>> >> > > > > >> > > > > >> > > > > >> >
>>> >>> >> > > > > >> > > > > >> > > > > >> > BACKUP. All backup system
>>> >>> information
>>> >>> >> > (prior
>>> >>> >> > > > to
>>> >>> >> > > > > >> > backup)
>>> >>> >> > > > > >> > > > > will
>>> >>> >> > > > > >> > > > > >> be
>>> >>> >> > > > > >> > > > > >> > > > > restored
>>> >>> >> > > > > >> > > > > >> > > > > >> > and all temporary data,
>>> related to
>>> >>> a
>>> >>> >> > failed
>>> >>> >> > > > > >> session,
>>> >>> >> > > > > >> > in
>>> >>> >> > > > > >> > > > > HDFS
>>> >>> >> > > > > >> > > > > >> > will
>>> >>> >> > > > > >> > > > > >> > > be
>>> >>> >> > > > > >> > > > > >> > > > > >> > deleted
>>> >>> >> > > > > >> > > > > >> > > > > >> > RESTORE. We do not care about
>>> >>> system
>>> >>> >> data,
>>> >>> >> > > > > because
>>> >>> >> > > > > >> > > > restore
>>> >>> >> > > > > >> > > > > >> does
>>> >>> >> > > > > >> > > > > >> > > not
>>> >>> >> > > > > >> > > > > >> > > > > >> change
>>> >>> >> > > > > >> > > > > >> > > > > >> > it. Temporary data in HDFS
>>> will be
>>> >>> >> cleaned
>>> >>> >> > > up
>>> >>> >> > > > > and
>>> >>> >> > > > > >> > table
>>> >>> >> > > > > >> > > > > will
>>> >>> >> > > > > >> > > > > >> be
>>> >>> >> > > > > >> > > > > >> > > in a
>>> >>> >> > > > > >> > > > > >> > > > > >> state
>>> >>> >> > > > > >> > > > > >> > > > > >> > back to where it was before
>>> >>> operation
>>> >>> >> > > started.
>>> >>> >> > > > > >> > > > > >> > > > > >> >
>>> >>> >> > > > > >> > > > > >> > > > > >> > This is what user should
>>> expect in
>>> >>> case
>>> >>> >> > of a
>>> >>> >> > > > > >> failure.
>>> >>> >> > > > > >> > > > > >> > > > > >> >
>>> >>> >> > > > > >> > > > > >> > > > > >> > -Vlad
>>> >>> >> > > > > >> > > > > >> > > > > >> >
>>> >>> >> > > > > >> > > > > >> > > > > >> >
>>> >>> >> > > > > >> > > > > >> > > > > >> > -Vlad
>>> >>> >> > > > > >> > > > > >> > > > > >> >
>>> >>> >> > > > > >> > > > > >> > > > > >> > On Fri, Sep 9, 2016 at 12:56
>>> PM,
>>> >>> Sean
>>> >>> >> > > Busbey <
>>> >>> >> > > > > >> > > > > >> busbey@apache.org
>>> >>> >> > > > > >> > > > > >> > >
>>> >>> >> > > > > >> > > > > >> > > > > wrote:
>>> >>> >> > > > > >> > > > > >> > > > > >> >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > Failing in a consistent way,
>>> with
>>> >>> >> docs
>>> >>> >> > > that
>>> >>> >> > > > > >> explain
>>> >>> >> > > > > >> > > the
>>> >>> >> > > > > >> > > > > >> > various
>>> >>> >> > > > > >> > > > > >> > > > > >> > > expected failures would be
>>> >>> >> sufficient.
>>> >>> >> > > > > >> > > > > >> > > > > >> > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > On Fri, Sep 9, 2016 at 12:16
>>> PM,
>>> >>> >> > Vladimir
>>> >>> >> > > > > >> Rodionov
>>> >>> >> > > > > >> > > > > >> > > > > >> > > <vl...@gmail.com>
>>> wrote:
>>> >>> >> > > > > >> > > > > >> > > > > >> > > > Do not worry Sean, doc is
>>> >>> coming
>>> >>> >> today
>>> >>> >> > > as
>>> >>> >> > > > a
>>> >>> >> > > > > >> > preview
>>> >>> >> > > > > >> > > > and
>>> >>> >> > > > > >> > > > > >> our
>>> >>> >> > > > > >> > > > > >> > > > writer
>>> >>> >> > > > > >> > > > > >> > > > > >> > Frank
>>> >>> >> > > > > >> > > > > >> > > > > >> > > > will be working on a
>>> putting
>>> >>> it
>>> >>> >> into
>>> >>> >> > > > Apache
>>> >>> >> > > > > >> > repo.
>>> >>> >> > > > > >> > > > > >> Timeline
>>> >>> >> > > > > >> > > > > >> > > > > depends
>>> >>> >> > > > > >> > > > > >> > > > > >> on
>>> >>> >> > > > > >> > > > > >> > > > > >> > > > Franks schedule but I hope
>>> we
>>> >>> will
>>> >>> >> get
>>> >>> >> > > it
>>> >>> >> > > > > >> rather
>>> >>> >> > > > > >> > > > sooner
>>> >>> >> > > > > >> > > > > >> than
>>> >>> >> > > > > >> > > > > >> > > > > later.
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > > As for failure testing, we
>>> are
>>> >>> >> > focusing
>>> >>> >> > > > only
>>> >>> >> > > > > >> on a
>>> >>> >> > > > > >> > > > > >> consistent
>>> >>> >> > > > > >> > > > > >> > > > state
>>> >>> >> > > > > >> > > > > >> > > > > >> of
>>> >>> >> > > > > >> > > > > >> > > > > >> > > > backup system data in a
>>> >>> presence of
>>> >>> >> > any
>>> >>> >> > > > type
>>> >>> >> > > > > >> of
>>> >>> >> > > > > >> > > > > failures,
>>> >>> >> > > > > >> > > > > >> We
>>> >>> >> > > > > >> > > > > >> > > are
>>> >>> >> > > > > >> > > > > >> > > > > not
>>> >>> >> > > > > >> > > > > >> > > > > >> > > going
>>> >>> >> > > > > >> > > > > >> > > > > >> > > > to implement  anything more
>>> >>> >> "fancy",
>>> >>> >> > > than
>>> >>> >> > > > > >> that.
>>> >>> >> > > > > >> > We
>>> >>> >> > > > > >> > > > > allow
>>> >>> >> > > > > >> > > > > >> > both:
>>> >>> >> > > > > >> > > > > >> > > > > >> backup
>>> >>> >> > > > > >> > > > > >> > > > > >> > and
>>> >>> >> > > > > >> > > > > >> > > > > >> > > > restore to fail. What we
>>> do not
>>> >>> >> allow
>>> >>> >> > is
>>> >>> >> > > > to
>>> >>> >> > > > > >> have
>>> >>> >> > > > > >> > > > system
>>> >>> >> > > > > >> > > > > >> data
>>> >>> >> > > > > >> > > > > >> > > > > >> corrupted.
>>> >>> >> > > > > >> > > > > >> > > > > >> > > > Will it suffice for you?
>>> Do you
>>> >>> >> have
>>> >>> >> > any
>>> >>> >> > > > > other
>>> >>> >> > > > > >> > > > > concerns,
>>> >>> >> > > > > >> > > > > >> you
>>> >>> >> > > > > >> > > > > >> > > > want
>>> >>> >> > > > > >> > > > > >> > > > > >> us to
>>> >>> >> > > > > >> > > > > >> > > > > >> > > > address?
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > > -Vlad
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > > On Fri, Sep 9, 2016 at
>>> 10:56
>>> >>> AM,
>>> >>> >> Sean
>>> >>> >> > > > > Busbey <
>>> >>> >> > > > > >> > > > > >> > > busbey@apache.org
>>> >>> >> > > > > >> > > > > >> > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > wrote:
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> "docs will come to Apache
>>> >>> soon"
>>> >>> >> does
>>> >>> >> > > not
>>> >>> >> > > > > >> address
>>> >>> >> > > > > >> > > my
>>> >>> >> > > > > >> > > > > >> concern
>>> >>> >> > > > > >> > > > > >> > > > > around
>>> >>> >> > > > > >> > > > > >> > > > > >> > docs
>>> >>> >> > > > > >> > > > > >> > > > > >> > > at
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> all, unless said docs have
>>> >>> already
>>> >>> >> > made
>>> >>> >> > > > it
>>> >>> >> > > > > >> into
>>> >>> >> > > > > >> > > the
>>> >>> >> > > > > >> > > > > >> project
>>> >>> >> > > > > >> > > > > >> > > > > repo. I
>>> >>> >> > > > > >> > > > > >> > > > > >> > > don't
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> want third party
>>> resources for
>>> >>> >> using
>>> >>> >> > a
>>> >>> >> > > > > major
>>> >>> >> > > > > >> and
>>> >>> >> > > > > >> > > > > >> important
>>> >>> >> > > > > >> > > > > >> > > > > feature
>>> >>> >> > > > > >> > > > > >> > > > > >> of
>>> >>> >> > > > > >> > > > > >> > > > > >> > > the
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> project, I want us to
>>> provide
>>> >>> end
>>> >>> >> > users
>>> >>> >> > > > > with
>>> >>> >> > > > > >> > what
>>> >>> >> > > > > >> > > > they
>>> >>> >> > > > > >> > > > > >> need
>>> >>> >> > > > > >> > > > > >> > > to
>>> >>> >> > > > > >> > > > > >> > > > > get
>>> >>> >> > > > > >> > > > > >> > > > > >> the
>>> >>> >> > > > > >> > > > > >> > > > > >> > > job
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> done.
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >>
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> I see some calls for
>>> patience
>>> >>> on
>>> >>> >> the
>>> >>> >> > > > > failure
>>> >>> >> > > > > >> > > > testing,
>>> >>> >> > > > > >> > > > > but
>>> >>> >> > > > > >> > > > > >> > the
>>> >>> >> > > > > >> > > > > >> > > > > >> appeal
>>> >>> >> > > > > >> > > > > >> > > > > >> > to
>>> >>> >> > > > > >> > > > > >> > > > > >> > > us
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> having done a bad job of
>>> >>> requiring
>>> >>> >> > > proper
>>> >>> >> > > > > >> tests
>>> >>> >> > > > > >> > of
>>> >>> >> > > > > >> > > > > >> previous
>>> >>> >> > > > > >> > > > > >> > > > > >> features
>>> >>> >> > > > > >> > > > > >> > > > > >> > > just
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> makes me more concerned
>>> about
>>> >>> not
>>> >>> >> > > getting
>>> >>> >> > > > > >> them
>>> >>> >> > > > > >> > > > here. I
>>> >>> >> > > > > >> > > > > >> > don't
>>> >>> >> > > > > >> > > > > >> > > > want
>>> >>> >> > > > > >> > > > > >> > > > > >> to
>>> >>> >> > > > > >> > > > > >> > > > > >> > set
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> yet another bad example
>>> that
>>> >>> will
>>> >>> >> > then
>>> >>> >> > > be
>>> >>> >> > > > > >> > pointed
>>> >>> >> > > > > >> > > to
>>> >>> >> > > > > >> > > > > in
>>> >>> >> > > > > >> > > > > >> the
>>> >>> >> > > > > >> > > > > >> > > > > future.
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >>
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> On Sep 8, 2016 10:50, "Ted
>>> >>> Yu" <
>>> >>> >> > > > > >> > > yuzhihong@gmail.com
>>> >>> >> > > > > >> > > > >
>>> >>> >> > > > > >> > > > > >> > wrote:
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >>
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > Is there any concern
>>> which
>>> >>> is
>>> >>> >> not
>>> >>> >> > > > > >> addressed ?
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > Do we need another Vote
>>> >>> thread ?
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > Thanks
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > On Thu, Sep 8, 2016 at
>>> 9:21
>>> >>> AM,
>>> >>> >> > > Andrew
>>> >>> >> > > > > >> > Purtell <
>>> >>> >> > > > > >> > > > > >> > > > > >> apurtell@apache.org
>>> >>> >> > > > > >> > > > > >> > > > > >> > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > wrote:
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > Vlad,
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > I apologize for using
>>> the
>>> >>> term
>>> >>> >> > > > > >> 'half-baked'
>>> >>> >> > > > > >> > > in a
>>> >>> >> > > > > >> > > > > way
>>> >>> >> > > > > >> > > > > >> > that
>>> >>> >> > > > > >> > > > > >> > > > > could
>>> >>> >> > > > > >> > > > > >> > > > > >> > > seem a
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > description of
>>> >>> HBASE-7912. I
>>> >>> >> > meant
>>> >>> >> > > > that
>>> >>> >> > > > > >> as a
>>> >>> >> > > > > >> > > > > general
>>> >>> >> > > > > >> > > > > >> > > > > >> hypothetical.
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > On Wed, Sep 7, 2016 at
>>> >>> 9:36
>>> >>> >> AM,
>>> >>> >> > > > > Vladimir
>>> >>> >> > > > > >> > > > Rodionov
>>> >>> >> > > > > >> > > > > <
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > vladrodionov@gmail.com>
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > wrote:
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >> I'm not sure that
>>> >>> "There
>>> >>> >> is
>>> >>> >> > > > > already
>>> >>> >> > > > > >> > lots
>>> >>> >> > > > > >> > > of
>>> >>> >> > > > > >> > > > > >> > > half-baked
>>> >>> >> > > > > >> > > > > >> > > > > >> code
>>> >>> >> > > > > >> > > > > >> > > > > >> > in
>>> >>> >> > > > > >> > > > > >> > > > > >> > > the
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > branch,
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > so what's the harm
>>> in
>>> >>> adding
>>> >>> >> > > more?"
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > I meant - not
>>> >>> production -
>>> >>> >> > ready
>>> >>> >> > > > yet.
>>> >>> >> > > > > >> This
>>> >>> >> > > > > >> > > is
>>> >>> >> > > > > >> > > > > 2.0
>>> >>> >> > > > > >> > > > > >> > > > > development
>>> >>> >> > > > > >> > > > > >> > > > > >> > > branch
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > and,
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > hence many features
>>> are
>>> >>> in
>>> >>> >> > works,
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > not being tested
>>> well
>>> >>> etc.
>>> >>> >> I do
>>> >>> >> > > not
>>> >>> >> > > > > >> > consider
>>> >>> >> > > > > >> > > > > backup
>>> >>> >> > > > > >> > > > > >> > as
>>> >>> >> > > > > >> > > > > >> > > > half
>>> >>> >> > > > > >> > > > > >> > > > > >> > baked
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > feature -
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > it has passed our
>>> >>> internal
>>> >>> >> QA
>>> >>> >> > and
>>> >>> >> > > > has
>>> >>> >> > > > > >> very
>>> >>> >> > > > > >> > > > good
>>> >>> >> > > > > >> > > > > >> doc,
>>> >>> >> > > > > >> > > > > >> > > > which
>>> >>> >> > > > > >> > > > > >> > > > > we
>>> >>> >> > > > > >> > > > > >> > > > > >> > will
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > provide
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > to Apache shortly.
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > -Vlad
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > On Wed, Sep 7, 2016
>>> at
>>> >>> 9:13
>>> >>> >> AM,
>>> >>> >> > > > > Andrew
>>> >>> >> > > > > >> > > > Purtell <
>>> >>> >> > > > > >> > > > > >> > > > > >> > > apurtell@apache.org>
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > We shouldn't admit
>>> >>> half
>>> >>> >> baked
>>> >>> >> > > > > changes
>>> >>> >> > > > > >> > that
>>> >>> >> > > > > >> > > > > won't
>>> >>> >> > > > > >> > > > > >> be
>>> >>> >> > > > > >> > > > > >> > > > > >> finished.
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> However
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > in
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > this case the crew
>>> >>> >> working on
>>> >>> >> > > > this
>>> >>> >> > > > > >> > feature
>>> >>> >> > > > > >> > > > are
>>> >>> >> > > > > >> > > > > >> long
>>> >>> >> > > > > >> > > > > >> > > > > timers
>>> >>> >> > > > > >> > > > > >> > > > > >> and
>>> >>> >> > > > > >> > > > > >> > > > > >> > > less
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > likely
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > than just about
>>> >>> anyone to
>>> >>> >> > leave
>>> >>> >> > > > > >> > something
>>> >>> >> > > > > >> > > > in a
>>> >>> >> > > > > >> > > > > >> half
>>> >>> >> > > > > >> > > > > >> > > > baked
>>> >>> >> > > > > >> > > > > >> > > > > >> > > state. Of
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > course
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > there is no
>>> guarantee
>>> >>> how
>>> >>> >> > > > anything
>>> >>> >> > > > > >> will
>>> >>> >> > > > > >> > > turn
>>> >>> >> > > > > >> > > > > out,
>>> >>> >> > > > > >> > > > > >> > > but I
>>> >>> >> > > > > >> > > > > >> > > > > am
>>> >>> >> > > > > >> > > > > >> > > > > >> > > willing
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> to
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > take
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > a little on faith
>>> if
>>> >>> they
>>> >>> >> > feel
>>> >>> >> > > > > their
>>> >>> >> > > > > >> > best
>>> >>> >> > > > > >> > > > path
>>> >>> >> > > > > >> > > > > >> > > forward
>>> >>> >> > > > > >> > > > > >> > > > > now
>>> >>> >> > > > > >> > > > > >> > > > > >> is
>>> >>> >> > > > > >> > > > > >> > > > > >> > to
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > merge
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > to
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > trunk. I only
>>> wish I
>>> >>> had
>>> >>> >> > > > bandwidth
>>> >>> >> > > > > to
>>> >>> >> > > > > >> > have
>>> >>> >> > > > > >> > > > > done
>>> >>> >> > > > > >> > > > > >> > some
>>> >>> >> > > > > >> > > > > >> > > > real
>>> >>> >> > > > > >> > > > > >> > > > > >> > > kicking
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> of
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > the
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > tires by now.
>>> Maybe
>>> >>> this
>>> >>> >> > week.
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > (Yes, I'm using
>>> some
>>> >>> of
>>> >>> >> that
>>> >>> >> > > time
>>> >>> >> > > > > for
>>> >>> >> > > > > >> > this
>>> >>> >> > > > > >> > > > > email
>>> >>> >> > > > > >> > > > > >> > :-)
>>> >>> >> > > > > >> > > > > >> > > > but
>>> >>> >> > > > > >> > > > > >> > > > > I
>>> >>> >> > > > > >> > > > > >> > > > > >> > type
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > fast.)
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > That said, I would
>>> >>> like to
>>> >>> >> > > > agitate
>>> >>> >> > > > > >> for
>>> >>> >> > > > > >> > > > making
>>> >>> >> > > > > >> > > > > 2.0
>>> >>> >> > > > > >> > > > > >> > > more
>>> >>> >> > > > > >> > > > > >> > > > > real
>>> >>> >> > > > > >> > > > > >> > > > > >> > and
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> spend
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > some
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > time on it now
>>> that
>>> >>> I'm
>>> >>> >> > winding
>>> >>> >> > > > > down
>>> >>> >> > > > > >> > with
>>> >>> >> > > > > >> > > > > 0.98. I
>>> >>> >> > > > > >> > > > > >> > > think
>>> >>> >> > > > > >> > > > > >> > > > > >> that
>>> >>> >> > > > > >> > > > > >> > > > > >> > > means
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branching for 2.0
>>> real
>>> >>> >> soon
>>> >>> >> > now
>>> >>> >> > > > and
>>> >>> >> > > > > >> even
>>> >>> >> > > > > >> > > > > evicting
>>> >>> >> > > > > >> > > > > >> > > > things
>>> >>> >> > > > > >> > > > > >> > > > > >> from
>>> >>> >> > > > > >> > > > > >> > > > > >> > > 2.0
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > branch
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > that aren't
>>> finished
>>> >>> or
>>> >>> >> > stable,
>>> >>> >> > > > > >> leaving
>>> >>> >> > > > > >> > > them
>>> >>> >> > > > > >> > > > > only
>>> >>> >> > > > > >> > > > > >> > > once
>>> >>> >> > > > > >> > > > > >> > > > > >> again
>>> >>> >> > > > > >> > > > > >> > > > > >> > in
>>> >>> >> > > > > >> > > > > >> > > > > >> > > the
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > master
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branch. Or, maybe
>>> just
>>> >>> >> > evicting
>>> >>> >> > > > > them.
>>> >>> >> > > > > >> > > Let's
>>> >>> >> > > > > >> > > > > take
>>> >>> >> > > > > >> > > > > >> it
>>> >>> >> > > > > >> > > > > >> > > > case
>>> >>> >> > > > > >> > > > > >> > > > > by
>>> >>> >> > > > > >> > > > > >> > > > > >> > > case.
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > I think this
>>> feature
>>> >>> can
>>> >>> >> come
>>> >>> >> > > in
>>> >>> >> > > > > >> > > relatively
>>> >>> >> > > > > >> > > > > >> safely.
>>> >>> >> > > > > >> > > > > >> > > As
>>> >>> >> > > > > >> > > > > >> > > > > >> added
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > insurance,
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > let's admit the
>>> >>> >> possibility
>>> >>> >> > it
>>> >>> >> > > > > could
>>> >>> >> > > > > >> be
>>> >>> >> > > > > >> > > > > reverted
>>> >>> >> > > > > >> > > > > >> on
>>> >>> >> > > > > >> > > > > >> > > the
>>> >>> >> > > > > >> > > > > >> > > > > 2.0
>>> >>> >> > > > > >> > > > > >> > > > > >> > > branch
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> if
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > folks
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > working on
>>> >>> stabilizing 2.0
>>> >>> >> > > decide
>>> >>> >> > > > > to
>>> >>> >> > > > > >> > evict
>>> >>> >> > > > > >> > > > it
>>> >>> >> > > > > >> > > > > >> > because
>>> >>> >> > > > > >> > > > > >> > > > it
>>> >>> >> > > > > >> > > > > >> > > > > is
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > unfinished
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > or
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > unstable, because
>>> that
>>> >>> >> > > certainly
>>> >>> >> > > > > can
>>> >>> >> > > > > >> > > > happen. I
>>> >>> >> > > > > >> > > > > >> > would
>>> >>> >> > > > > >> > > > > >> > > > > >> expect if
>>> >>> >> > > > > >> > > > > >> > > > > >> > > talk
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > like
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > that starts, we'd
>>> get
>>> >>> help
>>> >>> >> > > > > finishing
>>> >>> >> > > > > >> or
>>> >>> >> > > > > >> > > > > >> stabilizing
>>> >>> >> > > > > >> > > > > >> > > > > what's
>>> >>> >> > > > > >> > > > > >> > > > > >> > under
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > discussion
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > for revert. Or,
>>> we'd
>>> >>> have
>>> >>> >> a
>>> >>> >> > > > revert.
>>> >>> >> > > > > >> > Either
>>> >>> >> > > > > >> > > > way
>>> >>> >> > > > > >> > > > > >> the
>>> >>> >> > > > > >> > > > > >> > > > > outcome
>>> >>> >> > > > > >> > > > > >> > > > > >> is
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > acceptable.
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > On Wed, Sep 7,
>>> 2016 at
>>> >>> >> 8:56
>>> >>> >> > AM,
>>> >>> >> > > > > Dima
>>> >>> >> > > > > >> > > Spivak
>>> >>> >> > > > > >> > > > <
>>> >>> >> > > > > >> > > > > >> > > > > >> > > dimaspivak@apache.org
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > I'm not sure
>>> that
>>> >>> >> "There is
>>> >>> >> > > > > already
>>> >>> >> > > > > >> > lots
>>> >>> >> > > > > >> > > > of
>>> >>> >> > > > > >> > > > > >> > > > half-baked
>>> >>> >> > > > > >> > > > > >> > > > > >> code
>>> >>> >> > > > > >> > > > > >> > > > > >> > in
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> the
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branch,
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > so what's the
>>> harm
>>> >>> in
>>> >>> >> > adding
>>> >>> >> > > > > more?"
>>> >>> >> > > > > >> > is a
>>> >>> >> > > > > >> > > > > good
>>> >>> >> > > > > >> > > > > >> > code
>>> >>> >> > > > > >> > > > > >> > > > > commit
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > philosophy
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > for
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > a
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > fault-tolerant
>>> >>> >> distributed
>>> >>> >> > > data
>>> >>> >> > > > > >> store.
>>> >>> >> > > > > >> > > ;)
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > More seriously,
>>> a
>>> >>> lack
>>> >>> >> of
>>> >>> >> > > test
>>> >>> >> > > > > >> > coverage
>>> >>> >> > > > > >> > > > for
>>> >>> >> > > > > >> > > > > >> > > existing
>>> >>> >> > > > > >> > > > > >> > > > > >> > features
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > shouldn't
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > be
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > used as
>>> >>> justification
>>> >>> >> for
>>> >>> >> > > > > >> introducing
>>> >>> >> > > > > >> > > new
>>> >>> >> > > > > >> > > > > >> > features
>>> >>> >> > > > > >> > > > > >> > > > with
>>> >>> >> > > > > >> > > > > >> > > > > >> the
>>> >>> >> > > > > >> > > > > >> > > > > >> > > same
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > shortcomings.
>>> >>> >> Ultimately,
>>> >>> >> > > it's
>>> >>> >> > > > > the
>>> >>> >> > > > > >> end
>>> >>> >> > > > > >> > > > user
>>> >>> >> > > > > >> > > > > who
>>> >>> >> > > > > >> > > > > >> > > will
>>> >>> >> > > > > >> > > > > >> > > > > feel
>>> >>> >> > > > > >> > > > > >> > > > > >> > the
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> pain,
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > so
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > shouldn't we do
>>> >>> >> everything
>>> >>> >> > we
>>> >>> >> > > > can
>>> >>> >> > > > > >> to
>>> >>> >> > > > > >> > > > > mitigate
>>> >>> >> > > > > >> > > > > >> > that?
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > -Dima
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > On Wed, Sep 7,
>>> 2016
>>> >>> at
>>> >>> >> 8:46
>>> >>> >> > > AM,
>>> >>> >> > > > > >> > Vladimir
>>> >>> >> > > > > >> > > > > >> > Rodionov <
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>> >>> vladrodionov@gmail.com>
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > wrote:
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Sean,
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have docs
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Agree. We
>>> have a
>>> >>> doc
>>> >>> >> and
>>> >>> >> > > > backup
>>> >>> >> > > > > >> is
>>> >>> >> > > > > >> > the
>>> >>> >> > > > > >> > > > > most
>>> >>> >> > > > > >> > > > > >> > > > > documented
>>> >>> >> > > > > >> > > > > >> > > > > >> > > feature
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > :),
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > we
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > will
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > release it
>>> >>> shortly to
>>> >>> >> > > Apache.
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have
>>> sunny-day
>>> >>> >> > > correctness
>>> >>> >> > > > > >> tests
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Feature has
>>> >>> close to
>>> >>> >> 60
>>> >>> >> > > test
>>> >>> >> > > > > >> cases,
>>> >>> >> > > > > >> > > > which
>>> >>> >> > > > > >> > > > > >> run
>>> >>> >> > > > > >> > > > > >> > > for
>>> >>> >> > > > > >> > > > > >> > > > > >> approx
>>> >>> >> > > > > >> > > > > >> > > > > >> > 30
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> min.
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > We
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > can
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > add more, if
>>> >>> >> community do
>>> >>> >> > > not
>>> >>> >> > > > > >> mind
>>> >>> >> > > > > >> > :)
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have
>>> >>> >> > > > > >> > correctness-in-face-of-failure
>>> >>> >> > > > > >> > > > > tests
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Any examples
>>> of
>>> >>> these
>>> >>> >> > tests
>>> >>> >> > > > in
>>> >>> >> > > > > >> > > existing
>>> >>> >> > > > > >> > > > > >> > features?
>>> >>> >> > > > > >> > > > > >> > > > In
>>> >>> >> > > > > >> > > > > >> > > > > >> > works,
>>> >>> >> > > > > >> > > > > >> > > > > >> > > we
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > have a
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > clear
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > understanding
>>> of
>>> >>> what
>>> >>> >> > > should
>>> >>> >> > > > be
>>> >>> >> > > > > >> done
>>> >>> >> > > > > >> > > by
>>> >>> >> > > > > >> > > > > the
>>> >>> >> > > > > >> > > > > >> > time
>>> >>> >> > > > > >> > > > > >> > > of
>>> >>> >> > > > > >> > > > > >> > > > > 2.0
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> release.
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > That is very
>>> close
>>> >>> >> goal
>>> >>> >> > for
>>> >>> >> > > > us,
>>> >>> >> > > > > >> to
>>> >>> >> > > > > >> > > > verify
>>> >>> >> > > > > >> > > > > IT
>>> >>> >> > > > > >> > > > > >> > > monkey
>>> >>> >> > > > > >> > > > > >> > > > > for
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> existing
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > code.
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * don't rely
>>> on
>>> >>> things
>>> >>> >> > > > outside
>>> >>> >> > > > > of
>>> >>> >> > > > > >> > > HBase
>>> >>> >> > > > > >> > > > > for
>>> >>> >> > > > > >> > > > > >> > > normal
>>> >>> >> > > > > >> > > > > >> > > > > >> > operation
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > (okay
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > for
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > advanced
>>> >>> operation)
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > We do not.
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Enormous time
>>> has
>>> >>> been
>>> >>> >> > > spent
>>> >>> >> > > > > >> already
>>> >>> >> > > > > >> > > on
>>> >>> >> > > > > >> > > > > the
>>> >>> >> > > > > >> > > > > >> > > > > development
>>> >>> >> > > > > >> > > > > >> > > > > >> > and
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > testing
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > the
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > feature, it
>>> has
>>> >>> passed
>>> >>> >> > our
>>> >>> >> > > > > >> internal
>>> >>> >> > > > > >> > > > tests
>>> >>> >> > > > > >> > > > > and
>>> >>> >> > > > > >> > > > > >> > > many
>>> >>> >> > > > > >> > > > > >> > > > > >> rounds
>>> >>> >> > > > > >> > > > > >> > > > > >> > of
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> code
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > reviews
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > by HBase
>>> >>> committers.
>>> >>> >> We
>>> >>> >> > do
>>> >>> >> > > > not
>>> >>> >> > > > > >> mind
>>> >>> >> > > > > >> > if
>>> >>> >> > > > > >> > > > > >> someone
>>> >>> >> > > > > >> > > > > >> > > from
>>> >>> >> > > > > >> > > > > >> > > > > >> HBase
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > community
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > (outside of
>>> HW)
>>> >>> will
>>> >>> >> > review
>>> >>> >> > > > the
>>> >>> >> > > > > >> > code,
>>> >>> >> > > > > >> > > > but
>>> >>> >> > > > > >> > > > > it
>>> >>> >> > > > > >> > > > > >> > will
>>> >>> >> > > > > >> > > > > >> > > > > >> probably
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> takes
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > forever
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > to
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > wait for
>>> >>> volunteer?,
>>> >>> >> the
>>> >>> >> > > > > feature
>>> >>> >> > > > > >> is
>>> >>> >> > > > > >> > > > quite
>>> >>> >> > > > > >> > > > > >> large
>>> >>> >> > > > > >> > > > > >> > > > (1MB+
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> cumulative
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > patch)
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > 2.0 branch is
>>> >>> full of
>>> >>> >> > half
>>> >>> >> > > > > baked
>>> >>> >> > > > > >> > > > features,
>>> >>> >> > > > > >> > > > > >> most
>>> >>> >> > > > > >> > > > > >> > > of
>>> >>> >> > > > > >> > > > > >> > > > > them
>>> >>> >> > > > > >> > > > > >> > > > > >> > are
>>> >>> >> > > > > >> > > > > >> > > > > >> > > in
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > active
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > development,
>>> >>> >> therefore I
>>> >>> >> > am
>>> >>> >> > > > not
>>> >>> >> > > > > >> > > > following
>>> >>> >> > > > > >> > > > > you
>>> >>> >> > > > > >> > > > > >> > > here,
>>> >>> >> > > > > >> > > > > >> > > > > >> Sean?
>>> >>> >> > > > > >> > > > > >> > > > > >> > > Why
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > HBASE-7912
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > is
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > not good
>>> enough
>>> >>> yet
>>> >>> >> to be
>>> >>> >> > > > > >> integrated
>>> >>> >> > > > > >> > > > into
>>> >>> >> > > > > >> > > > > 2.0
>>> >>> >> > > > > >> > > > > >> > > > branch?
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > -Vlad
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > On Wed, Sep 7,
>>> >>> 2016 at
>>> >>> >> > 8:23
>>> >>> >> > > > AM,
>>> >>> >> > > > > >> Sean
>>> >>> >> > > > > >> > > > > Busbey <
>>> >>> >> > > > > >> > > > > >> > > > > >> > > busbey@apache.org
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > On Tue, Sep
>>> 6,
>>> >>> 2016
>>> >>> >> at
>>> >>> >> > > > 10:36
>>> >>> >> > > > > >> PM,
>>> >>> >> > > > > >> > > Josh
>>> >>> >> > > > > >> > > > > >> Elser <
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > josh.elser@gmail.com>
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > wrote:
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > So, the
>>> >>> answer to
>>> >>> >> > > Sean's
>>> >>> >> > > > > >> > original
>>> >>> >> > > > > >> > > > > >> question
>>> >>> >> > > > > >> > > > > >> > is
>>> >>> >> > > > > >> > > > > >> > > > "as
>>> >>> >> > > > > >> > > > > >> > > > > >> > > robust as
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > snapshots
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > presently
>>> >>> are"?
>>> >>> >> > > > > >> (independence of
>>> >>> >> > > > > >> > > > > >> > > backup/restore
>>> >>> >> > > > > >> > > > > >> > > > > >> > failure
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > tolerance
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > from
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > snapshot
>>> >>> failure
>>> >>> >> > > > tolerance)
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > Is this
>>> just a
>>> >>> >> > question
>>> >>> >> > > > WRT
>>> >>> >> > > > > >> > > context
>>> >>> >> > > > > >> > > > of
>>> >>> >> > > > > >> > > > > >> the
>>> >>> >> > > > > >> > > > > >> > > > > change,
>>> >>> >> > > > > >> > > > > >> > > > > >> or
>>> >>> >> > > > > >> > > > > >> > > > > >> > > is it
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > means
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > for a
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > veto
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > from you,
>>> >>> Sean?
>>> >>> >> Just
>>> >>> >> > > > trying
>>> >>> >> > > > > >> to
>>> >>> >> > > > > >> > > make
>>> >>> >> > > > > >> > > > > sure
>>> >>> >> > > > > >> > > > > >> > I'm
>>> >>> >> > > > > >> > > > > >> > > > > >> following
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> along
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > adequately.
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > I'd say ATM
>>> I'm
>>> >>> -0,
>>> >>> >> > > > bordering
>>> >>> >> > > > > >> on
>>> >>> >> > > > > >> > -1
>>> >>> >> > > > > >> > > > but
>>> >>> >> > > > > >> > > > > not
>>> >>> >> > > > > >> > > > > >> > for
>>> >>> >> > > > > >> > > > > >> > > > > >> reasons
>>> >>> >> > > > > >> > > > > >> > > > > >> > I
>>> >>> >> > > > > >> > > > > >> > > > > >> > > can
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > articulate
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > well.
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Here's an
>>> >>> attempt.
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > We've been
>>> >>> trying to
>>> >>> >> > > move,
>>> >>> >> > > > > as a
>>> >>> >> > > > > >> > > > > community,
>>> >>> >> > > > > >> > > > > >> > > > towards
>>> >>> >> > > > > >> > > > > >> > > > > >> > > minimizing
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > risk
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > to
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > downstream
>>> >>> folks by
>>> >>> >> > > getting
>>> >>> >> > > > > >> > > "complete
>>> >>> >> > > > > >> > > > > >> enough
>>> >>> >> > > > > >> > > > > >> > > for
>>> >>> >> > > > > >> > > > > >> > > > > use"
>>> >>> >> > > > > >> > > > > >> > > > > >> > > gates
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> in
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > place
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > before we
>>> >>> introduce
>>> >>> >> new
>>> >>> >> > > > > >> features.
>>> >>> >> > > > > >> > > This
>>> >>> >> > > > > >> > > > > was
>>> >>> >> > > > > >> > > > > >> > > > spurred
>>> >>> >> > > > > >> > > > > >> > > > > >> by a
>>> >>> >> > > > > >> > > > > >> > > > > >> > > some
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > features
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > getting in
>>> >>> >> half-baked
>>> >>> >> > and
>>> >>> >> > > > > never
>>> >>> >> > > > > >> > > making
>>> >>> >> > > > > >> > > > > it
>>> >>> >> > > > > >> > > > > >> to
>>> >>> >> > > > > >> > > > > >> > > "can
>>> >>> >> > > > > >> > > > > >> > > > > >> really
>>> >>> >> > > > > >> > > > > >> > > > > >> > > use"
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > status
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > (I'm
>>> thinking of
>>> >>> >> > > > distributed
>>> >>> >> > > > > >> log
>>> >>> >> > > > > >> > > > replay
>>> >>> >> > > > > >> > > > > and
>>> >>> >> > > > > >> > > > > >> > the
>>> >>> >> > > > > >> > > > > >> > > > > >> zk-less
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > assignment
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > stuff, I
>>> don't
>>> >>> >> recall
>>> >>> >> > if
>>> >>> >> > > > > there
>>> >>> >> > > > > >> was
>>> >>> >> > > > > >> > > > > more).
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The gates,
>>> >>> >> generally,
>>> >>> >> > > > > included
>>> >>> >> > > > > >> > > things
>>> >>> >> > > > > >> > > > > like:
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have docs
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have
>>> sunny-day
>>> >>> >> > > > correctness
>>> >>> >> > > > > >> tests
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have
>>> >>> >> > > > > >> > > correctness-in-face-of-failure
>>> >>> >> > > > > >> > > > > tests
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't
>>> rely on
>>> >>> >> things
>>> >>> >> > > > > outside
>>> >>> >> > > > > >> of
>>> >>> >> > > > > >> > > > HBase
>>> >>> >> > > > > >> > > > > for
>>> >>> >> > > > > >> > > > > >> > > > normal
>>> >>> >> > > > > >> > > > > >> > > > > >> > > operation
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > (okay
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > for
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > advanced
>>> >>> operation)
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > As an
>>> example,
>>> >>> we
>>> >>> >> kept
>>> >>> >> > > the
>>> >>> >> > > > > MOB
>>> >>> >> > > > > >> > work
>>> >>> >> > > > > >> > > > off
>>> >>> >> > > > > >> > > > > in
>>> >>> >> > > > > >> > > > > >> a
>>> >>> >> > > > > >> > > > > >> > > > branch
>>> >>> >> > > > > >> > > > > >> > > > > >> and
>>> >>> >> > > > > >> > > > > >> > > > > >> > > out
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> of
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > master
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > until it
>>> could
>>> >>> pass
>>> >>> >> > these
>>> >>> >> > > > > >> > criteria.
>>> >>> >> > > > > >> > > > The
>>> >>> >> > > > > >> > > > > big
>>> >>> >> > > > > >> > > > > >> > > > > exemption
>>> >>> >> > > > > >> > > > > >> > > > > >> > > we've
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> had
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > to
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > this was the
>>> >>> >> > hbase-spark
>>> >>> >> > > > > >> > > integration,
>>> >>> >> > > > > >> > > > > where
>>> >>> >> > > > > >> > > > > >> > we
>>> >>> >> > > > > >> > > > > >> > > > all
>>> >>> >> > > > > >> > > > > >> > > > > >> > agreed
>>> >>> >> > > > > >> > > > > >> > > > > >> > > it
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > could
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > land in
>>> master
>>> >>> >> because
>>> >>> >> > it
>>> >>> >> > > > was
>>> >>> >> > > > > >> very
>>> >>> >> > > > > >> > > > well
>>> >>> >> > > > > >> > > > > >> > > isolated
>>> >>> >> > > > > >> > > > > >> > > > > (the
>>> >>> >> > > > > >> > > > > >> > > > > >> > > slide
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > away
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > from
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > including
>>> docs
>>> >>> as a
>>> >>> >> > > > > first-class
>>> >>> >> > > > > >> > part
>>> >>> >> > > > > >> > > > of
>>> >>> >> > > > > >> > > > > >> > > building
>>> >>> >> > > > > >> > > > > >> > > > up
>>> >>> >> > > > > >> > > > > >> > > > > >> that
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > integration
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > has led me
>>> to
>>> >>> doubt
>>> >>> >> the
>>> >>> >> > > > > wisdom
>>> >>> >> > > > > >> of
>>> >>> >> > > > > >> > > this
>>> >>> >> > > > > >> > > > > >> > > decision).
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > We've also
>>> been
>>> >>> >> > treating
>>> >>> >> > > > > >> inclusion
>>> >>> >> > > > > >> > > in
>>> >>> >> > > > > >> > > > a
>>> >>> >> > > > > >> > > > > >> > > "probably
>>> >>> >> > > > > >> > > > > >> > > > > >> will
>>> >>> >> > > > > >> > > > > >> > > > > >> > be
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > released
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > to
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > downstream"
>>> >>> branches
>>> >>> >> > as a
>>> >>> >> > > > > >> higher
>>> >>> >> > > > > >> > > bar,
>>> >>> >> > > > > >> > > > > >> > requiring
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't
>>> >>> moderately
>>> >>> >> > impact
>>> >>> >> > > > > >> > > performance
>>> >>> >> > > > > >> > > > > when
>>> >>> >> > > > > >> > > > > >> > the
>>> >>> >> > > > > >> > > > > >> > > > > >> feature
>>> >>> >> > > > > >> > > > > >> > > > > >> > > isn't
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> in
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > use
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't
>>> severely
>>> >>> >> impact
>>> >>> >> > > > > >> > performance
>>> >>> >> > > > > >> > > > when
>>> >>> >> > > > > >> > > > > >> the
>>> >>> >> > > > > >> > > > > >> > > > > feature
>>> >>> >> > > > > >> > > > > >> > > > > >> is
>>> >>> >> > > > > >> > > > > >> > > > > >> > in
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> use
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * either
>>> >>> >> default-to-on
>>> >>> >> > or
>>> >>> >> > > > > show
>>> >>> >> > > > > >> > > enough
>>> >>> >> > > > > >> > > > > >> demand
>>> >>> >> > > > > >> > > > > >> > to
>>> >>> >> > > > > >> > > > > >> > > > > >> believe
>>> >>> >> > > > > >> > > > > >> > > > > >> > a
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > non-trivial
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > number of
>>> folks
>>> >>> will
>>> >>> >> > turn
>>> >>> >> > > > the
>>> >>> >> > > > > >> > > feature
>>> >>> >> > > > > >> > > > on
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The above
>>> has
>>> >>> kept
>>> >>> >> MOB
>>> >>> >> > > and
>>> >>> >> > > > > >> > > hbase-spark
>>> >>> >> > > > > >> > > > > >> > > > integration
>>> >>> >> > > > > >> > > > > >> > > > > >> out
>>> >>> >> > > > > >> > > > > >> > > > > >> > of
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > branch-1,
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > presumably
>>> while
>>> >>> >> > they've
>>> >>> >> > > > > >> "gotten
>>> >>> >> > > > > >> > > more
>>> >>> >> > > > > >> > > > > >> stable"
>>> >>> >> > > > > >> > > > > >> > > in
>>> >>> >> > > > > >> > > > > >> > > > > >> master
>>> >>> >> > > > > >> > > > > >> > > > > >> > > from
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > the
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > odd
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > vendor
>>> >>> inclusion.
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Are we
>>> going to
>>> >>> >> have a
>>> >>> >> > > 2.0
>>> >>> >> > > > > >> release
>>> >>> >> > > > > >> > > > > before
>>> >>> >> > > > > >> > > > > >> the
>>> >>> >> > > > > >> > > > > >> > > end
>>> >>> >> > > > > >> > > > > >> > > > > of
>>> >>> >> > > > > >> > > > > >> > > > > >> the
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> year?
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > We're
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > coming up
>>> on 1.5
>>> >>> >> years
>>> >>> >> > > > since
>>> >>> >> > > > > >> the
>>> >>> >> > > > > >> > > > > release of
>>> >>> >> > > > > >> > > > > >> > > > version
>>> >>> >> > > > > >> > > > > >> > > > > >> 1.0;
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> seems
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > like
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > it's about
>>> time,
>>> >>> >> > though I
>>> >>> >> > > > > >> haven't
>>> >>> >> > > > > >> > > seen
>>> >>> >> > > > > >> > > > > any
>>> >>> >> > > > > >> > > > > >> > > > concrete
>>> >>> >> > > > > >> > > > > >> > > > > >> > plans
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> this
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > year.
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Presuming
>>> we are
>>> >>> >> going
>>> >>> >> > to
>>> >>> >> > > > > have
>>> >>> >> > > > > >> one
>>> >>> >> > > > > >> > > by
>>> >>> >> > > > > >> > > > > the
>>> >>> >> > > > > >> > > > > >> end
>>> >>> >> > > > > >> > > > > >> > > of
>>> >>> >> > > > > >> > > > > >> > > > > the
>>> >>> >> > > > > >> > > > > >> > > > > >> > > year, it
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > seems a
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > bit close to
>>> >>> still
>>> >>> >> be
>>> >>> >> > > > adding
>>> >>> >> > > > > in
>>> >>> >> > > > > >> > > > > "features
>>> >>> >> > > > > >> > > > > >> > that
>>> >>> >> > > > > >> > > > > >> > > > need
>>> >>> >> > > > > >> > > > > >> > > > > >> > > maturing"
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > on
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > the
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > branch.
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The lack of
>>> a
>>> >>> >> concrete
>>> >>> >> > > plan
>>> >>> >> > > > > for
>>> >>> >> > > > > >> > 2.0
>>> >>> >> > > > > >> > > > > keeps
>>> >>> >> > > > > >> > > > > >> me
>>> >>> >> > > > > >> > > > > >> > > from
>>> >>> >> > > > > >> > > > > >> > > > > >> > > considering
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > these
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > things
>>> blocker
>>> >>> at
>>> >>> >> the
>>> >>> >> > > > moment.
>>> >>> >> > > > > >> But
>>> >>> >> > > > > >> > I
>>> >>> >> > > > > >> > > > know
>>> >>> >> > > > > >> > > > > >> > first
>>> >>> >> > > > > >> > > > > >> > > > hand
>>> >>> >> > > > > >> > > > > >> > > > > >> how
>>> >>> >> > > > > >> > > > > >> > > > > >> > > much
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > trouble
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > folks have
>>> had
>>> >>> with
>>> >>> >> > other
>>> >>> >> > > > > >> features
>>> >>> >> > > > > >> > > > that
>>> >>> >> > > > > >> > > > > >> have
>>> >>> >> > > > > >> > > > > >> > > gone
>>> >>> >> > > > > >> > > > > >> > > > > >> into
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > downstream
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > facing
>>> releases
>>> >>> >> without
>>> >>> >> > > > > >> robustness
>>> >>> >> > > > > >> > > > > checks
>>> >>> >> > > > > >> > > > > >> > (i.e.
>>> >>> >> > > > > >> > > > > >> > > > > >> > > replication),
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > and
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > I'm
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > concerned
>>> about
>>> >>> what
>>> >>> >> > > we're
>>> >>> >> > > > > >> setting
>>> >>> >> > > > > >> > > up
>>> >>> >> > > > > >> > > > if
>>> >>> >> > > > > >> > > > > >> 2.0
>>> >>> >> > > > > >> > > > > >> > > goes
>>> >>> >> > > > > >> > > > > >> > > > > out
>>> >>> >> > > > > >> > > > > >> > > > > >> > with
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> this
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > feature in
>>> its
>>> >>> >> current
>>> >>> >> > > > state.
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > --
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > Best regards,
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >    - Andy
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > Problems worthy of
>>> >>> attack
>>> >>> >> > prove
>>> >>> >> > > > > their
>>> >>> >> > > > > >> > > worth
>>> >>> >> > > > > >> > > > by
>>> >>> >> > > > > >> > > > > >> > > hitting
>>> >>> >> > > > > >> > > > > >> > > > > >> back. -
>>> >>> >> > > > > >> > > > > >> > > > > >> > > Piet
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > Hein
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > (via Tom White)
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > --
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > Best regards,
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >    - Andy
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > Problems worthy of
>>> attack
>>> >>> >> prove
>>> >>> >> > > their
>>> >>> >> > > > > >> worth
>>> >>> >> > > > > >> > by
>>> >>> >> > > > > >> > > > > >> hitting
>>> >>> >> > > > > >> > > > > >> > > > back.
>>> >>> >> > > > > >> > > > > >> > > > > -
>>> >>> >> > > > > >> > > > > >> > > > > >> > Piet
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> Hein
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > (via Tom White)
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
>>> >>> >> > > > > >> > > > > >> > > > > >> > > >>
>>> >>> >> > > > > >> > > > > >> > > > > >> > >
>>> >>> >> > > > > >> > > > > >> > > > > >> >
>>> >>> >> > > > > >> > > > > >> > > > > >>
>>> >>> >> > > > > >> > > > > >> > > > > >
>>> >>> >> > > > > >> > > > > >> > > > > >
>>> >>> >> > > > > >> > > > > >> > > > >
>>> >>> >> > > > > >> > > > > >> > > >
>>> >>> >> > > > > >> > > > > >> > >
>>> >>> >> > > > > >> > > > > >> >
>>> >>> >> > > > > >> > > > > >>
>>> >>> >> > > > > >> > > > >
>>> >>> >> > > > > >> > > > >
>>> >>> >> > > > > >> > > > >
>>> >>> >> > > > > >> > > > > --
>>> >>> >> > > > > >> > > > > busbey
>>> >>> >> > > > > >> > > > >
>>> >>> >> > > > > >> > > >
>>> >>> >> > > > > >> > >
>>> >>> >> > > > > >> >
>>> >>> >> > > > > >>
>>> >>> >> > > > > >
>>> >>> >> > > > > >
>>> >>> >> > > > >
>>> >>> >> > > >
>>> >>> >> > >
>>> >>> >> >
>>> >>> >>
>>> >>> >>
>>> >>> >>
>>> >>> >> --
>>> >>> >>
>>> >>> >> -- Appy
>>> >>> >>
>>> >>> >
>>> >>> >
>>> >>>
>>> >>
>>> >>
>>> >
>>>
>>
>>
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Stack <st...@duboce.net>]

Fails again when I do this:

$ mvn clean install -DskipTests assembly:single
....


[INFO]
------------------------------------------------------------------------
[INFO] Building Apache HBase - Server 2.0.0-SNAPSHOT
[INFO]
------------------------------------------------------------------------
Downloading:
file:/home/stack/hbase.git/hbase-server/src/main/site/resources/repo/org/apache/hadoop/hadoop-distcp/2.7.1/hadoop-distcp-2.7.1.pom
Exception in thread "pool-1-thread-1"
---------------------------------------------------
constituent[0]: file:/usr/share/maven/lib/aether-spi.jar
constituent[1]: file:/usr/share/maven/lib/maven-settings-builder-3.x.jar
constituent[2]: file:/usr/share/maven/lib/maven-artifact-3.x.jar
constituent[3]: file:/usr/share/maven/lib/commons-httpclient.jar
constituent[4]: file:/usr/share/maven/lib/wagon-provider-api.jar
constituent[5]: file:/usr/share/maven/lib/maven-core-3.x.jar
constituent[6]: file:/usr/share/maven/lib/aether-impl.jar
constituent[7]: file:/usr/share/maven/lib/sisu-inject-bean.jar
constituent[8]: file:/usr/share/maven/lib/sisu-inject-plexus.jar
constituent[9]: file:/usr/share/maven/lib/wagon-file.jar
constituent[10]: file:/usr/share/maven/lib/guava.jar
constituent[11]: file:/usr/share/maven/lib/maven-compat-3.x.jar
constituent[12]: file:/usr/share/maven/lib/maven-model-3.x.jar
constituent[13]: file:/usr/share/maven/lib/plexus-interpolation.jar
constituent[14]: file:/usr/share/maven/lib/commons-codec.jar
constituent[15]: file:/usr/share/maven/lib/aether-util.jar
constituent[16]: file:/usr/share/maven/lib/aether-api.jar
constituent[17]: file:/usr/share/maven/lib/maven-model-builder-3.x.jar
constituent[18]: file:/usr/share/maven/lib/maven-repository-metadata-3.x.jar
constituent[19]: file:/usr/share/maven/lib/maven-settings-3.x.jar
constituent[20]: file:/usr/share/maven/lib/plexus-component-annotations.jar
constituent[21]: file:/usr/share/maven/lib/commons-cli.jar
constituent[22]: file:/usr/share/maven/lib/maven-embedder-3.x.jar
constituent[23]: file:/usr/share/maven/lib/maven-plugin-api-3.x.jar
constituent[24]: file:/usr/share/maven/lib/wagon-http-shaded.jar
constituent[25]: file:/usr/share/maven/lib/commons-logging.jar
constituent[26]: file:/usr/share/maven/lib/plexus-sec-dispatcher.jar
constituent[27]: file:/usr/share/maven/lib/aether-connector-wagon.jar
constituent[28]: file:/usr/share/maven/lib/plexus-utils.jar
constituent[29]: file:/usr/share/maven/lib/sisu-guice.jar
constituent[30]: file:/usr/share/maven/lib/plexus-cipher.jar
constituent[31]: file:/usr/share/maven/lib/maven-aether-provider-3.x.jar
---------------------------------------------------
java.lang.NoClassDefFoundError: org/apache/commons/lang/StringUtils
at
org.apache.maven.wagon.providers.file.FileWagon.resolveDestinationPath(FileWagon.java:206)
at
org.apache.maven.wagon.providers.file.FileWagon.resourceExists(FileWagon.java:265)
at
org.sonatype.aether.connector.wagon.WagonRepositoryConnector$GetTask.run(WagonRepositoryConnector.java:577)
at
org.sonatype.aether.util.concurrency.RunnableErrorForwarder$1.run(RunnableErrorForwarder.java:60)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException:
org.apache.commons.lang.StringUtils
at
org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy.loadClass(SelfFirstStrategy.java:50)
at
org.codehaus.plexus.classworlds.realm.ClassRealm.unsynchronizedLoadClass(ClassRealm.java:259)
at
org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:235)
at
org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:227)
... 7 more
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/commons/lang/StringUtils
at
org.apache.maven.wagon.providers.file.FileWagon.resolveDestinationPath(FileWagon.java:206)
at
org.apache.maven.wagon.providers.file.FileWagon.resourceExists(FileWagon.java:265)
at
org.sonatype.aether.connector.wagon.WagonRepositoryConnector$GetTask.run(WagonRepositoryConnector.java:577)
at
org.sonatype.aether.util.concurrency.RunnableErrorForwarder$1.run(RunnableErrorForwarder.java:60)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException:
org.apache.commons.lang.StringUtils
at
org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy.loadClass(SelfFirstStrategy.java:50)
at
org.codehaus.plexus.classworlds.realm.ClassRealm.unsynchronizedLoadClass(ClassRealm.java:259)
at
org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:235)
at
org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:227)



Does it work for you lot? This is branch checkout.

I want assembly to work so I can distribute the build around a cluster.

Thanks,
St.Ack.



On Wed, Oct 12, 2016 at 1:52 PM, Stack <st...@duboce.net> wrote:

> I did a super-clean and recheck out. Now it works. Sorry for the noise.
> St.Ack
>
> On Wed, Oct 12, 2016 at 1:10 PM, Vladimir Rodionov <vladrodionov@gmail.com
> > wrote:
>
>> checked out HBASE-7912
>>
>> ran:
>>
>> mvn clean install -DskipTests
>>
>> successfully.
>>
>> -Vlad
>>
>> On Wed, Oct 12, 2016 at 1:01 PM, Vladimir Rodionov <
>> vladrodionov@gmail.com>
>> wrote:
>>
>> > I usually use:
>> >
>> > mvn clean install -DskipTests
>> >
>> > and it works.
>> >
>> > On Wed, Oct 12, 2016 at 1:01 PM, Vladimir Rodionov <
>> vladrodionov@gmail.com
>> > > wrote:
>> >
>> >> Michael,
>> >>
>> >> you can try master + latest patch on HBASE-14123 (v29). No need to use
>> >> HBASE-7912 branch. I will double check HBASE-7912.
>> >>
>> >> -Vlad
>> >>
>> >> On Wed, Oct 12, 2016 at 12:31 PM, Stack <st...@duboce.net> wrote:
>> >>
>> >>> More info:
>> >>>
>> >>> stack@ve0524:~/hbase.git$ git checkout origin/HBASE-7912 -b 7912v2
>> >>> Branch 7912v2 set up to track remote branch HBASE-7912 from origin.
>> >>> Switched to a new branch '7912v2'
>> >>> stack@ve0524:~/hbase.git$ java -version
>> >>> java version "1.8.0_101"
>> >>> Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
>> >>> Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode)
>> >>> stack@ve0524:~/hbase.git$ mvn clean install -DskipTests &>
>> /tmp/out.txt
>> >>>
>> >>> ...
>> >>>
>> >>> St.Ack
>> >>>
>> >>>
>> >>> On Wed, Oct 12, 2016 at 12:29 PM, Stack <st...@duboce.net> wrote:
>> >>>
>> >>> > Interesting. When I try it fails w/ below:
>> >>> >
>> >>> > [INFO] 26 warnings
>> >>> > 322 [INFO] ------------------------------
>> >>> -------------------------------
>> >>> > 323 [INFO] ------------------------------
>> >>> -------------------------------
>> >>> > 324 [ERROR] COMPILATION ERROR :
>> >>> > 325 [INFO] ------------------------------
>> >>> -------------------------------
>> >>> > 326 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.
>> java:[48,8]
>> >>> > org.apache.hadoop.hbase.io.encoding.rowindexV2.RowIndexCodecV2 is
>> not
>> >>> > abstract and does not override abstract method createSeeker(org.ap
>> >>> >  ache.hadoop.hbase.CellComparator,org.apache.hadoop.hbase.io.
>> >>> encoding.HFileBlockDecodingContext)
>> >>> > in org.apache.hadoop.hbase.io.encoding.DataBlockEncoder
>> >>> > 327 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.j
>> >>> ava:[143,3]
>> >>> > method does not override or implement a method from a supertype
>> >>> > 328 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.j
>> >>> ava:[147,29]
>> >>> > incompatible types: java.nio.ByteBuffer cannot be converted to
>> >>> > org.apache.hadoop.hbase.nio.ByteBuff
>> >>> > 329 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.j
>> >>> ava:[148,33]
>> >>> > cannot find symbol
>> >>> > 330   symbol:   method getKeyDeepCopy()
>> >>> > 331   location: variable seeker of type org.apache.hadoop.hbase.io.
>> >>> > encoding.DataBlockEncoder.EncodedSeeker
>> >>> > 332 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.j
>> >>> ava:[153,3]
>> >>> > method does not override or implement a method from a supertype
>> >>> > 333 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>> >>> > apache/hadoop/hbase/io/encoding/rowindexV1/RowIndexCodecV1.
>> java:[45,8]
>> >>> > org.apache.hadoop.hbase.io.encoding.rowindexV1.RowIndexCodecV1 is
>> not
>> >>> > abstract and does not override abstract method createSeeker(org.ap
>> >>> >  ache.hadoop.hbase.CellComparator,org.apache.hadoop.hbase.io.
>> >>> encoding.HFileBlockDecodingContext)
>> >>> > in org.apache.hadoop.hbase.io.encoding.DataBlockEncoder
>> >>> > 334 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>> >>> > apache/hadoop/hbase/io/encoding/rowindexV1/RowIndexCodecV1.j
>> >>> ava:[145,3]
>> >>> > method does not override or implement a method from a supertype
>> >>> > 335 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>> >>> > apache/hadoop/hbase/io/encoding/rowindexV1/RowIndexCodecV1.j
>> >>> ava:[158,3]
>> >>> > method does not override or implement a method from a supertype
>> >>> > 336 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>> >>> java:[46,8]
>> >>> > org.apache.hadoop.hbase.io.encoding.rowindexV2.RowIndexSeekerV2 is
>> not
>> >>> > abstract and does not override abstract method compareKey(org.ap
>> >>> >  ache.hadoop.hbase.CellComparator,org.apache.hadoop.hbase.Cell) in
>> >>> > org.apache.hadoop.hbase.io.encoding.DataBlockEncoder.EncodedSeeker
>> >>> > 337 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>> >>> java:[79,3]
>> >>> > method does not override or implement a method from a supertype
>> >>> > 338 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>> >>> java:[117,3]
>> >>> > method does not override or implement a method from a supertype
>> >>> > 339 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>> >>> java:[190,3]
>> >>> > method does not override or implement a method from a supertype
>> >>> > 340 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>> >>> java:[214,3]
>> >>> > method does not override or implement a method from a supertype
>> >>> > 341 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>> >>> java:[349,3]
>> >>> > method does not override or implement a method from a supertype
>> >>> > 342 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>> >>> java:[355,3]
>> >>> > method does not override or implement a method from a supertype
>> >>> > 343 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>> >>> java:[421,36]
>> >>> > no suitable method found for uncompressTags(java.nio.
>> >>> > ByteBuffer,byte[],int,int)
>> >>> > 344     method org.apache.hadoop.hbase.io.TagCompressionContext.
>> >>> > uncompressTags(java.io.InputStream,byte[],int,int) is not
>> applicable
>> >>> > 345       (argument mismatch; java.nio.ByteBuffer cannot be
>> converted
>> >>> to
>> >>> > java.io.InputStream)
>> >>> > 346     method org.apache.hadoop.hbase.io.TagCompressionContext.
>> >>> > uncompressTags(org.apache.hadoop.hbase.nio.ByteBuff,byte[],int,int)
>> is
>> >>> > not applicable
>> >>> > 347       (argument mismatch; java.nio.ByteBuffer cannot be
>> converted
>> >>> to
>> >>> > org.apache.hadoop.hbase.nio.ByteBuff)
>> >>> >
>> >>> > ....
>> >>> >
>> >>> > St.Ack
>> >>> >
>> >>> > On Tue, Oct 11, 2016 at 5:12 PM, Apekshit Sharma <appy@cloudera.com
>> >
>> >>> > wrote:
>> >>> >
>> >>> >> @stack, it compiled for me.
>> >>> >>
>> >>> >> Also tried few commands, and have to say, it's well designed from
>> user
>> >>> >> commands perspective.
>> >>> >>
>> >>> >> On Tue, Oct 11, 2016 at 5:08 PM, Stack <st...@duboce.net> wrote:
>> >>> >>
>> >>> >> > On Tue, Oct 11, 2016 at 1:50 PM, Vladimir Rodionov <
>> >>> >> vladrodionov@gmail.com
>> >>> >> > >
>> >>> >> > wrote:
>> >>> >> >
>> >>> >> > > Michael,
>> >>> >> > >
>> >>> >> > > Its in HBASE-7912
>> >>> >> > >
>> >>> >> > > This is tip of git log:
>> >>> >> > >
>> >>> >> > > commit a072f6f49a26a7259ff2aaef6cb56d85eb592482
>> >>> >> > > Author: Frank Welsch <fw...@jps.net>
>> >>> >> > > Date:   Fri Sep 23 18:00:42 2016 -0400
>> >>> >> > >
>> >>> >> > >     HBASE-16574 Book updates for backup and restore
>> >>> >> > >
>> >>> >> > > commit b14e2ab1c24e65ff88dd4c579acf83cb4ed0605e
>> >>> >> > > Author: tedyu <yu...@gmail.com>
>> >>> >> > > Date:   Wed Oct 5 16:29:40 2016 -0700
>> >>> >> > >
>> >>> >> > >     HBASE-16727 Backup refactoring: remove MR dependencies from
>> >>> >> HMaster
>> >>> >> > > (Vladimir Rodionov)
>> >>> >> > >
>> >>> >> > >
>> >>> >> > Thanks. I have that. I tried it and it doesn't compile for me.
>> Does
>> >>> it
>> >>> >> > compile for you?
>> >>> >> > Thanks,
>> >>> >> > M
>> >>> >> >
>> >>> >> >
>> >>> >> >
>> >>> >> > > -Vlad
>> >>> >> > >
>> >>> >> > > On Tue, Oct 11, 2016 at 1:44 PM, Stack <st...@duboce.net>
>> wrote:
>> >>> >> > >
>> >>> >> > > > Which branch do I check out to try it? HBASE-7912 is not it.
>> I
>> >>> don't
>> >>> >> > see
>> >>> >> > > an
>> >>> >> > > > HBASE-16727...
>> >>> >> > > > Thanks,
>> >>> >> > > > M
>> >>> >> > > >
>> >>> >> > > >
>> >>> >> > > > On Tue, Oct 11, 2016 at 11:41 AM, Vladimir Rodionov <
>> >>> >> > > > vladrodionov@gmail.com>
>> >>> >> > > > wrote:
>> >>> >> > > >
>> >>> >> > > > > The last patch is on review board:
>> >>> >> > > > > https://reviews.apache.org/r/52748
>> >>> >> > > > >
>> >>> >> > > > > On Mon, Oct 10, 2016 at 3:52 PM, Vladimir Rodionov <
>> >>> >> > > > vladrodionov@gmail.com
>> >>> >> > > > > >
>> >>> >> > > > > wrote:
>> >>> >> > > > >
>> >>> >> > > > > > >> How hard to put in an hbase-backup module?
>> hbase-server
>> >>> is
>> >>> >> fat
>> >>> >> > > > enough
>> >>> >> > > > > > >> already. Could be done as a follow-up.
>> >>> >> > > > > >
>> >>> >> > > > > > https://issues.apache.org/jira/browse/HBASE-16727?
>> >>> >> > > > > > focusedCommentId=15531237&page=com.atlassian.jira.
>> >>> >> > > > > > plugin.system.issuetabpanels:c
>> >>> omment-tabpanel#comment-15531237
>> >>> >> > > > > >
>> >>> >> > > > > > Can we do merge first? Then we can discuss separate
>> module.
>> >>> >> > > > > >
>> >>> >> > > > > > -Vlad
>> >>> >> > > > > >
>> >>> >> > > > > > On Mon, Oct 10, 2016 at 3:44 PM, Ted Yu <
>> >>> yuzhihong@gmail.com>
>> >>> >> > wrote:
>> >>> >> > > > > >
>> >>> >> > > > > >> Looks like the first quote was cut off.
>> >>> >> > > > > >> The original sentence was:
>> >>> >> > > > > >>
>> >>> >> > > > > >> bq. no mapreduce job launched from master or region
>> server.
>> >>> >> > > > > >>
>> >>> >> > > > > >> mapreduce job is launched from the node where command
>> line
>> >>> >> tool is
>> >>> >> > > > run.
>> >>> >> > > > > >>
>> >>> >> > > > > >> On Mon, Oct 10, 2016 at 3:38 PM, Stack <
>> stack@duboce.net>
>> >>> >> wrote:
>> >>> >> > > > > >>
>> >>> >> > > > > >> > bq. launched from master or region server.
>> >>> >> > > > > >> >
>> >>> >> > > > > >> > What does this mean please? Has to be run from Master
>> or
>> >>> >> > > > RegionServer?
>> >>> >> > > > > >> Can
>> >>> >> > > > > >> > it be run from another node altogether?
>> >>> >> > > > > >> >
>> >>> >> > > > > >> > On Mon, Oct 10, 2016 at 1:44 PM, Vladimir Rodionov <
>> >>> >> > > > > >> vladrodionov@gmail.com
>> >>> >> > > > > >> > >
>> >>> >> > > > > >> > wrote:
>> >>> >> > > > > >> >
>> >>> >> > > > > >> > > >> mapreduce dependency has been moved to client
>> side
>> >>> - no
>> >>> >> > > > mapreduce
>> >>> >> > > > > >> job
>> >>> >> > > > > >> > >
>> >>> >> > > > > >> > > 1. We have no code in the client module anymore,
>> due to
>> >>> >> > > dependency
>> >>> >> > > > > on
>> >>> >> > > > > >> > > internal server API (HFile and WAL access).
>> >>> >> > > > > >> > > 2. Backup/ restore are client - driven operations,
>> but
>> >>> all
>> >>> >> the
>> >>> >> > > > code
>> >>> >> > > > > >> > resides
>> >>> >> > > > > >> > > in the server module
>> >>> >> > > > > >> > >
>> >>> >> > > > > >> >
>> >>> >> > > > > >> > How hard to put in an hbase-backup module?
>> hbase-server
>> >>> is
>> >>> >> fat
>> >>> >> > > > enough
>> >>> >> > > > > >> > already. Could be done as a follow-up.
>> >>> >> > > > > >> >
>> >>> >> > > > > >> > Thanks,
>> >>> >> > > > > >> > St.Ack
>> >>> >> > > > > >> >
>> >>> >> > > > > >> >
>> >>> >> > > > > >> >
>> >>> >> > > > > >> > > 3. No MR in Master, no procedure - driven execution.
>> >>> >> > > > > >> > > 4. Old good MR from command-line.
>> >>> >> > > > > >> > > 5. Security was simplified and now only super-user
>> is
>> >>> >> allowed
>> >>> >> > to
>> >>> >> > > > run
>> >>> >> > > > > >> > > backup/restores.
>> >>> >> > > > > >> > > 6. HBase Backup API was gone due to 1. Now only
>> >>> >> command-line
>> >>> >> > > > access
>> >>> >> > > > > to
>> >>> >> > > > > >> > > backup tools.
>> >>> >> > > > > >> > >
>> >>> >> > > > > >> > > These consequences of refactoring has been
>> discussed in
>> >>> >> > > > HBASE-16727.
>> >>> >> > > > > >> > >
>> >>> >> > > > > >> > > -Vlad
>> >>> >> > > > > >> > >
>> >>> >> > > > > >> > >
>> >>> >> > > > > >> > >
>> >>> >> > > > > >> > >
>> >>> >> > > > > >> > > On Mon, Oct 10, 2016 at 1:31 PM, Ted Yu <
>> >>> >> yuzhihong@gmail.com>
>> >>> >> > > > > wrote:
>> >>> >> > > > > >> > >
>> >>> >> > > > > >> > > > Reviving this thread.
>> >>> >> > > > > >> > > >
>> >>> >> > > > > >> > > > The following has taken place:
>> >>> >> > > > > >> > > >
>> >>> >> > > > > >> > > > mapreduce dependency has been moved to client
>> side -
>> >>> no
>> >>> >> > > > mapreduce
>> >>> >> > > > > >> job
>> >>> >> > > > > >> > > > launched from master or region server.
>> >>> >> > > > > >> > > > document patch (HBASE-16574) has been integrated.
>> >>> >> > > > > >> > > > Updated mega patch has been attached to
>> HBASE-14123:
>> >>> this
>> >>> >> > > covers
>> >>> >> > > > > the
>> >>> >> > > > > >> > > > refactor in #1 above and the protobuf 3 merge.
>> >>> >> > > > > >> > > >
>> >>> >> > > > > >> > > > If community has more feedback on the merge
>> >>> proposal, I
>> >>> >> > would
>> >>> >> > > > love
>> >>> >> > > > > >> to
>> >>> >> > > > > >> > > hear
>> >>> >> > > > > >> > > > it.
>> >>> >> > > > > >> > > >
>> >>> >> > > > > >> > > > Thanks
>> >>> >> > > > > >> > > >
>> >>> >> > > > > >> > > > On Thu, Sep 22, 2016 at 10:31 AM, Sean Busbey <
>> >>> >> > > > > busbey@cloudera.com>
>> >>> >> > > > > >> > > wrote:
>> >>> >> > > > > >> > > >
>> >>> >> > > > > >> > > > > I'd like to see the docs proposed on HBASE-16574
>> >>> >> > integrated
>> >>> >> > > > into
>> >>> >> > > > > >> our
>> >>> >> > > > > >> > > > > project's documentation prior to merge.
>> >>> >> > > > > >> > > > >
>> >>> >> > > > > >> > > > > On Thu, Sep 22, 2016 at 9:02 AM, Ted Yu <
>> >>> >> > > yuzhihong@gmail.com>
>> >>> >> > > > > >> wrote:
>> >>> >> > > > > >> > > > > > This feature can be marked experimental due to
>> >>> some
>> >>> >> > > > > limitations
>> >>> >> > > > > >> > such
>> >>> >> > > > > >> > > as
>> >>> >> > > > > >> > > > > > security.
>> >>> >> > > > > >> > > > > >
>> >>> >> > > > > >> > > > > > Your previous round of comments have been
>> >>> addressed.
>> >>> >> > > > > >> > > > > > Command line tool has gone through:
>> >>> >> > > > > >> > > > > >
>> >>> >> > > > > >> > > > > > HBASE-16620 Fix backup command-line tool
>> >>> usability
>> >>> >> > issues
>> >>> >> > > > > >> > > > > > HBASE-16655 hbase backup describe with
>> incorrect
>> >>> >> backup
>> >>> >> > id
>> >>> >> > > > > >> results
>> >>> >> > > > > >> > in
>> >>> >> > > > > >> > > > NPE
>> >>> >> > > > > >> > > > > >
>> >>> >> > > > > >> > > > > > The updated doc has been attached to
>> HBASE-16574.
>> >>> >> > > > > >> > > > > >
>> >>> >> > > > > >> > > > > > Cheers
>> >>> >> > > > > >> > > > > >
>> >>> >> > > > > >> > > > > > On Thu, Sep 22, 2016 at 8:53 AM, Stack <
>> >>> >> > stack@duboce.net>
>> >>> >> > > > > >> wrote:
>> >>> >> > > > > >> > > > > >
>> >>> >> > > > > >> > > > > >> On Wed, Sep 21, 2016 at 7:43 AM, Ted Yu <
>> >>> >> > > > yuzhihong@gmail.com
>> >>> >> > > > > >
>> >>> >> > > > > >> > > wrote:
>> >>> >> > > > > >> > > > > >>
>> >>> >> > > > > >> > > > > >> > Are there more (review) comments ?
>> >>> >> > > > > >> > > > > >> >
>> >>> >> > > > > >> > > > > >> >
>> >>> >> > > > > >> > > > > >> Are outstanding comments addressed?
>> >>> >> > > > > >> > > > > >>
>> >>> >> > > > > >> > > > > >> I don't see answer to my 'is this
>> >>> experimental/will
>> >>> >> it
>> >>> >> > be
>> >>> >> > > > > >> marked
>> >>> >> > > > > >> > > > > >> experimental' question.
>> >>> >> > > > > >> > > > > >>
>> >>> >> > > > > >> > > > > >> I ran into some issues trying to use the
>> >>> feature and
>> >>> >> > > > > suggested
>> >>> >> > > > > >> > that
>> >>> >> > > > > >> > > a
>> >>> >> > > > > >> > > > > >> feature likes this needs polish else it'll
>> just
>> >>> rot,
>> >>> >> > > > unused.
>> >>> >> > > > > >> Has
>> >>> >> > > > > >> > > > polish
>> >>> >> > > > > >> > > > > >> been applied? All ready for another 'user'
>> test?
>> >>> >> > Suggest
>> >>> >> > > > that
>> >>> >> > > > > >> you
>> >>> >> > > > > >> > > > update
>> >>> >> > > > > >> > > > > >> here going forward for the benefit of those
>> >>> trying
>> >>> >> to
>> >>> >> > > > follow
>> >>> >> > > > > >> along
>> >>> >> > > > > >> > > and
>> >>> >> > > > > >> > > > > who
>> >>> >> > > > > >> > > > > >> are not watching JIRA change fly-by.
>> >>> >> > > > > >> > > > > >>
>> >>> >> > > > > >> > > > > >> It looks like doc got a revision -- I have to
>> >>> check
>> >>> >> --
>> >>> >> > to
>> >>> >> > > > > take
>> >>> >> > > > > >> on
>> >>> >> > > > > >> > > > > >> suggestion made above but again, suggest,
>> that
>> >>> this
>> >>> >> > > thread
>> >>> >> > > > > gets
>> >>> >> > > > > >> > > > updated.
>> >>> >> > > > > >> > > > > >>
>> >>> >> > > > > >> > > > > >> Thanks,
>> >>> >> > > > > >> > > > > >> St.Ack
>> >>> >> > > > > >> > > > > >>
>> >>> >> > > > > >> > > > > >>
>> >>> >> > > > > >> > > > > >>
>> >>> >> > > > > >> > > > > >> > Thanks
>> >>> >> > > > > >> > > > > >> >
>> >>> >> > > > > >> > > > > >> > On Tue, Sep 20, 2016 at 10:02 AM, Devaraj
>> Das
>> >>> <
>> >>> >> > > > > >> > > ddas@hortonworks.com
>> >>> >> > > > > >> > > > >
>> >>> >> > > > > >> > > > > >> > wrote:
>> >>> >> > > > > >> > > > > >> >
>> >>> >> > > > > >> > > > > >> > > Just reviving this thread. Thanks Sean,
>> >>> Stack,
>> >>> >> > Dima,
>> >>> >> > > > and
>> >>> >> > > > > >> > others
>> >>> >> > > > > >> > > > for
>> >>> >> > > > > >> > > > > the
>> >>> >> > > > > >> > > > > >> > > thorough reviews and testing. Thanks Ted
>> and
>> >>> >> Vlad
>> >>> >> > for
>> >>> >> > > > > >> taking
>> >>> >> > > > > >> > > care
>> >>> >> > > > > >> > > > of
>> >>> >> > > > > >> > > > > >> the
>> >>> >> > > > > >> > > > > >> > > feedback. Are we all good to do the merge
>> >>> now?
>> >>> >> > Rather
>> >>> >> > > > do
>> >>> >> > > > > >> > sooner
>> >>> >> > > > > >> > > > than
>> >>> >> > > > > >> > > > > >> > later.
>> >>> >> > > > > >> > > > > >> > > ________________________________________
>> >>> >> > > > > >> > > > > >> > > From: saint.ack@gmail.com <
>> >>> saint.ack@gmail.com>
>> >>> >> on
>> >>> >> > > > > behalf
>> >>> >> > > > > >> of
>> >>> >> > > > > >> > > > Stack
>> >>> >> > > > > >> > > > > <
>> >>> >> > > > > >> > > > > >> > > stack@duboce.net>
>> >>> >> > > > > >> > > > > >> > > Sent: Monday, September 12, 2016 1:18 PM
>> >>> >> > > > > >> > > > > >> > > To: HBase Dev List
>> >>> >> > > > > >> > > > > >> > > Subject: Re: [DISCUSSION] Merge Backup /
>> >>> >> Restore -
>> >>> >> > > > Branch
>> >>> >> > > > > >> > > > HBASE-7912
>> >>> >> > > > > >> > > > > >> > >
>> >>> >> > > > > >> > > > > >> > > On Mon, Sep 12, 2016 at 12:19 PM, Ted Yu
>> <
>> >>> >> > > > > >> yuzhihong@gmail.com
>> >>> >> > > > > >> > >
>> >>> >> > > > > >> > > > > wrote:
>> >>> >> > > > > >> > > > > >> > >
>> >>> >> > > > > >> > > > > >> > > > Mega patch (rev 18) is on HBASE-14123.
>> >>> >> > > > > >> > > > > >> > > >
>> >>> >> > > > > >> > > > > >> > > > Please comment on HBASE-14123 on how
>> you
>> >>> want
>> >>> >> to
>> >>> >> > > > > review.
>> >>> >> > > > > >> > > > > >> > > >
>> >>> >> > > > > >> > > > > >> > >
>> >>> >> > > > > >> > > > > >> > >
>> >>> >> > > > > >> > > > > >> > > Yeah. That was my lost tab. Last rb was 6
>> >>> months
>> >>> >> > ago.
>> >>> >> > > > > >> Suggest
>> >>> >> > > > > >> > > > > updating
>> >>> >> > > > > >> > > > > >> > it.
>> >>> >> > > > > >> > > > > >> > > RB is pretty good for review. Patch is
>> only
>> >>> >> 1.5M so
>> >>> >> > > > > should
>> >>> >> > > > > >> be
>> >>> >> > > > > >> > > > fine.
>> >>> >> > > > > >> > > > > >> > >
>> >>> >> > > > > >> > > > > >> > > St.Ack
>> >>> >> > > > > >> > > > > >> > >
>> >>> >> > > > > >> > > > > >> > >
>> >>> >> > > > > >> > > > > >> > > >
>> >>> >> > > > > >> > > > > >> > > > Thanks
>> >>> >> > > > > >> > > > > >> > > >
>> >>> >> > > > > >> > > > > >> > > > On Mon, Sep 12, 2016 at 12:15 PM,
>> Stack <
>> >>> >> > > > > >> stack@duboce.net>
>> >>> >> > > > > >> > > > wrote:
>> >>> >> > > > > >> > > > > >> > > >
>> >>> >> > > > > >> > > > > >> > > > > On review of the 'patch', do I just
>> >>> compare
>> >>> >> the
>> >>> >> > > > > branch
>> >>> >> > > > > >> to
>> >>> >> > > > > >> > > > > master or
>> >>> >> > > > > >> > > > > >> > is
>> >>> >> > > > > >> > > > > >> > > > > there a megapatch posted somewhere (I
>> >>> think
>> >>> >> I
>> >>> >> > saw
>> >>> >> > > > one
>> >>> >> > > > > >> but
>> >>> >> > > > > >> > it
>> >>> >> > > > > >> > > > > seemed
>> >>> >> > > > > >> > > > > >> > > stale
>> >>> >> > > > > >> > > > > >> > > > > and then I 'lost' the tab). Sorry for
>> >>> dumb
>> >>> >> > > > question.
>> >>> >> > > > > >> > > > > >> > > > > St.Ack
>> >>> >> > > > > >> > > > > >> > > > >
>> >>> >> > > > > >> > > > > >> > > > > On Mon, Sep 12, 2016 at 12:01 PM,
>> Stack
>> >>> <
>> >>> >> > > > > >> stack@duboce.net
>> >>> >> > > > > >> > >
>> >>> >> > > > > >> > > > > wrote:
>> >>> >> > > > > >> > > > > >> > > > >
>> >>> >> > > > > >> > > > > >> > > > > > Late to the game. A few comments
>> after
>> >>> >> > > rereading
>> >>> >> > > > > this
>> >>> >> > > > > >> > > thread
>> >>> >> > > > > >> > > > > as a
>> >>> >> > > > > >> > > > > >> > > > 'user'.
>> >>> >> > > > > >> > > > > >> > > > > >
>> >>> >> > > > > >> > > > > >> > > > > > + Before merge, a user-facing
>> feature
>> >>> like
>> >>> >> > this
>> >>> >> > > > > >> should
>> >>> >> > > > > >> > > work
>> >>> >> > > > > >> > > > > (If
>> >>> >> > > > > >> > > > > >> > this
>> >>> >> > > > > >> > > > > >> > > is
>> >>> >> > > > > >> > > > > >> > > > > "higher-bar
>> >>> >> > > > > >> > > > > >> > > > > > for new features", bring it on --
>> >>> smile).
>> >>> >> > > > > >> > > > > >> > > > > > + As a user, I tried the branch
>> with
>> >>> tools
>> >>> >> > > after
>> >>> >> > > > > >> > reviewing
>> >>> >> > > > > >> > > > the
>> >>> >> > > > > >> > > > > >> > > > > just-posted
>> >>> >> > > > > >> > > > > >> > > > > > doc. I had an 'interesting'
>> experience
>> >>> >> (left
>> >>> >> > > > > >> comments up
>> >>> >> > > > > >> > > on
>> >>> >> > > > > >> > > > > >> > issue). I
>> >>> >> > > > > >> > > > > >> > > > > think
>> >>> >> > > > > >> > > > > >> > > > > > the tooling/doc. important to get
>> >>> right.
>> >>> >> If
>> >>> >> > it
>> >>> >> > > > > breaks
>> >>> >> > > > > >> > > easily
>> >>> >> > > > > >> > > > > or
>> >>> >> > > > > >> > > > > >> is
>> >>> >> > > > > >> > > > > >> > > > > > inconsistent (or lacks 'polish'),
>> >>> >> operators
>> >>> >> > > will
>> >>> >> > > > > >> judge
>> >>> >> > > > > >> > the
>> >>> >> > > > > >> > > > > whole
>> >>> >> > > > > >> > > > > >> > > > > > backup/restore tooling chain as not
>> >>> >> > trustworthy
>> >>> >> > > > and
>> >>> >> > > > > >> > > abandon
>> >>> >> > > > > >> > > > > it.
>> >>> >> > > > > >> > > > > >> > Lets
>> >>> >> > > > > >> > > > > >> > > > not
>> >>> >> > > > > >> > > > > >> > > > > > have this happen to this feature.
>> >>> >> > > > > >> > > > > >> > > > > > + Matteo's suggestion (with a
>> helpful
>> >>> >> starter
>> >>> >> > > > list)
>> >>> >> > > > > >> that
>> >>> >> > > > > >> > > > there
>> >>> >> > > > > >> > > > > >> > needs
>> >>> >> > > > > >> > > > > >> > > to
>> >>> >> > > > > >> > > > > >> > > > > be
>> >>> >> > > > > >> > > > > >> > > > > > explicit qualification on what is
>> >>> actually
>> >>> >> > > being
>> >>> >> > > > > >> > delivered
>> >>> >> > > > > >> > > > --
>> >>> >> > > > > >> > > > > >> > > > including a
>> >>> >> > > > > >> > > > > >> > > > > > listing of limitations (some look
>> >>> serious
>> >>> >> > such
>> >>> >> > > as
>> >>> >> > > > > >> data
>> >>> >> > > > > >> > > bleed
>> >>> >> > > > > >> > > > > from
>> >>> >> > > > > >> > > > > >> > > other
>> >>> >> > > > > >> > > > > >> > > > > > regions in WALs, but maybe I don't
>> >>> care
>> >>> >> for
>> >>> >> > my
>> >>> >> > > > use
>> >>> >> > > > > >> > > case...)
>> >>> >> > > > > >> > > > --
>> >>> >> > > > > >> > > > > >> > needs
>> >>> >> > > > > >> > > > > >> > > to
>> >>> >> > > > > >> > > > > >> > > > > > accompany the merge. Lets fold them
>> >>> into
>> >>> >> the
>> >>> >> > > user
>> >>> >> > > > > >> doc.
>> >>> >> > > > > >> > in
>> >>> >> > > > > >> > > > the
>> >>> >> > > > > >> > > > > >> > > technical
>> >>> >> > > > > >> > > > > >> > > > > > overview area as suggested so user
>> >>> >> > expectations
>> >>> >> > > > are
>> >>> >> > > > > >> > > properly
>> >>> >> > > > > >> > > > > >> > managed
>> >>> >> > > > > >> > > > > >> > > > > > (otherwise, they expect the world
>> and
>> >>> will
>> >>> >> > just
>> >>> >> > > > > give
>> >>> >> > > > > >> up
>> >>> >> > > > > >> > > when
>> >>> >> > > > > >> > > > > we
>> >>> >> > > > > >> > > > > >> > fall
>> >>> >> > > > > >> > > > > >> > > > > > short). Vladimir did a list of what
>> >>> is in
>> >>> >> > each
>> >>> >> > > of
>> >>> >> > > > > the
>> >>> >> > > > > >> > > phases
>> >>> >> > > > > >> > > > > >> above
>> >>> >> > > > > >> > > > > >> > > > which
>> >>> >> > > > > >> > > > > >> > > > > > would serve as a good start.
>> >>> >> > > > > >> > > > > >> > > > > > + Is this feature 'experimental'
>> >>> (Matteo
>> >>> >> asks
>> >>> >> > > > > above).
>> >>> >> > > > > >> > I'd
>> >>> >> > > > > >> > > > > prefer
>> >>> >> > > > > >> > > > > >> it
>> >>> >> > > > > >> > > > > >> > > is
>> >>> >> > > > > >> > > > > >> > > > > > not. If it is, it should be
>> labelled
>> >>> all
>> >>> >> over
>> >>> >> > > > that
>> >>> >> > > > > >> it is
>> >>> >> > > > > >> > > > so. I
>> >>> >> > > > > >> > > > > >> see
>> >>> >> > > > > >> > > > > >> > > > > current
>> >>> >> > > > > >> > > > > >> > > > > > state called out as a '...
>> technical
>> >>> >> preview
>> >>> >> > > > > >> feature'.
>> >>> >> > > > > >> > > Does
>> >>> >> > > > > >> > > > > this
>> >>> >> > > > > >> > > > > >> > mean
>> >>> >> > > > > >> > > > > >> > > > > > not-for-users?
>> >>> >> > > > > >> > > > > >> > > > > >
>> >>> >> > > > > >> > > > > >> > > > > > St.Ack
>> >>> >> > > > > >> > > > > >> > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >
>> >>> >> > > > > >> > > > > >> > > > > > On Mon, Sep 12, 2016 at 8:03 AM,
>> Ted
>> >>> Yu <
>> >>> >> > > > > >> > > > yuzhihong@gmail.com>
>> >>> >> > > > > >> > > > > >> > wrote:
>> >>> >> > > > > >> > > > > >> > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> Sean:
>> >>> >> > > > > >> > > > > >> > > > > >> Do you have more comments ?
>> >>> >> > > > > >> > > > > >> > > > > >>
>> >>> >> > > > > >> > > > > >> > > > > >> Cheers
>> >>> >> > > > > >> > > > > >> > > > > >>
>> >>> >> > > > > >> > > > > >> > > > > >> On Fri, Sep 9, 2016 at 1:42 PM,
>> >>> Vladimir
>> >>> >> > > > Rodionov
>> >>> >> > > > > <
>> >>> >> > > > > >> > > > > >> > > > > vladrodionov@gmail.com
>> >>> >> > > > > >> > > > > >> > > > > >> >
>> >>> >> > > > > >> > > > > >> > > > > >> wrote:
>> >>> >> > > > > >> > > > > >> > > > > >>
>> >>> >> > > > > >> > > > > >> > > > > >> > Sean,
>> >>> >> > > > > >> > > > > >> > > > > >> >
>> >>> >> > > > > >> > > > > >> > > > > >> > Backup/Restore can fail due to
>> >>> various
>> >>> >> > > > reasons:
>> >>> >> > > > > >> > network
>> >>> >> > > > > >> > > > > outage
>> >>> >> > > > > >> > > > > >> > > > > (cluster
>> >>> >> > > > > >> > > > > >> > > > > >> > wide), various time-outs in
>> HBase
>> >>> and
>> >>> >> HDFS
>> >>> >> > > > > layer,
>> >>> >> > > > > >> M/R
>> >>> >> > > > > >> > > > > failure
>> >>> >> > > > > >> > > > > >> > due
>> >>> >> > > > > >> > > > > >> > > to
>> >>> >> > > > > >> > > > > >> > > > > >> "HDFS
>> >>> >> > > > > >> > > > > >> > > > > >> > exceeded quota", user error
>> (manual
>> >>> >> > deletion
>> >>> >> > > > of
>> >>> >> > > > > >> data)
>> >>> >> > > > > >> > > and
>> >>> >> > > > > >> > > > > so
>> >>> >> > > > > >> > > > > >> on
>> >>> >> > > > > >> > > > > >> > so
>> >>> >> > > > > >> > > > > >> > > > on.
>> >>> >> > > > > >> > > > > >> > > > > >> That
>> >>> >> > > > > >> > > > > >> > > > > >> > is impossible to enumerate all
>> >>> possible
>> >>> >> > > types
>> >>> >> > > > of
>> >>> >> > > > > >> > > failures
>> >>> >> > > > > >> > > > > in a
>> >>> >> > > > > >> > > > > >> > > > > >> distributed
>> >>> >> > > > > >> > > > > >> > > > > >> > system - that is not our
>> goal/task.
>> >>> >> > > > > >> > > > > >> > > > > >> >
>> >>> >> > > > > >> > > > > >> > > > > >> > We focus completely on backup
>> >>> system
>> >>> >> table
>> >>> >> > > > > >> > consistency
>> >>> >> > > > > >> > > > in a
>> >>> >> > > > > >> > > > > >> > > presence
>> >>> >> > > > > >> > > > > >> > > > > of
>> >>> >> > > > > >> > > > > >> > > > > >> any
>> >>> >> > > > > >> > > > > >> > > > > >> > type of failure. That is what I
>> >>> call
>> >>> >> > > > "tolerance
>> >>> >> > > > > to
>> >>> >> > > > > >> > > > > failures".
>> >>> >> > > > > >> > > > > >> > > > > >> >
>> >>> >> > > > > >> > > > > >> > > > > >> > On a failure:
>> >>> >> > > > > >> > > > > >> > > > > >> >
>> >>> >> > > > > >> > > > > >> > > > > >> > BACKUP. All backup system
>> >>> information
>> >>> >> > (prior
>> >>> >> > > > to
>> >>> >> > > > > >> > backup)
>> >>> >> > > > > >> > > > > will
>> >>> >> > > > > >> > > > > >> be
>> >>> >> > > > > >> > > > > >> > > > > restored
>> >>> >> > > > > >> > > > > >> > > > > >> > and all temporary data, related
>> to
>> >>> a
>> >>> >> > failed
>> >>> >> > > > > >> session,
>> >>> >> > > > > >> > in
>> >>> >> > > > > >> > > > > HDFS
>> >>> >> > > > > >> > > > > >> > will
>> >>> >> > > > > >> > > > > >> > > be
>> >>> >> > > > > >> > > > > >> > > > > >> > deleted
>> >>> >> > > > > >> > > > > >> > > > > >> > RESTORE. We do not care about
>> >>> system
>> >>> >> data,
>> >>> >> > > > > because
>> >>> >> > > > > >> > > > restore
>> >>> >> > > > > >> > > > > >> does
>> >>> >> > > > > >> > > > > >> > > not
>> >>> >> > > > > >> > > > > >> > > > > >> change
>> >>> >> > > > > >> > > > > >> > > > > >> > it. Temporary data in HDFS will
>> be
>> >>> >> cleaned
>> >>> >> > > up
>> >>> >> > > > > and
>> >>> >> > > > > >> > table
>> >>> >> > > > > >> > > > > will
>> >>> >> > > > > >> > > > > >> be
>> >>> >> > > > > >> > > > > >> > > in a
>> >>> >> > > > > >> > > > > >> > > > > >> state
>> >>> >> > > > > >> > > > > >> > > > > >> > back to where it was before
>> >>> operation
>> >>> >> > > started.
>> >>> >> > > > > >> > > > > >> > > > > >> >
>> >>> >> > > > > >> > > > > >> > > > > >> > This is what user should expect
>> in
>> >>> case
>> >>> >> > of a
>> >>> >> > > > > >> failure.
>> >>> >> > > > > >> > > > > >> > > > > >> >
>> >>> >> > > > > >> > > > > >> > > > > >> > -Vlad
>> >>> >> > > > > >> > > > > >> > > > > >> >
>> >>> >> > > > > >> > > > > >> > > > > >> >
>> >>> >> > > > > >> > > > > >> > > > > >> > -Vlad
>> >>> >> > > > > >> > > > > >> > > > > >> >
>> >>> >> > > > > >> > > > > >> > > > > >> > On Fri, Sep 9, 2016 at 12:56 PM,
>> >>> Sean
>> >>> >> > > Busbey <
>> >>> >> > > > > >> > > > > >> busbey@apache.org
>> >>> >> > > > > >> > > > > >> > >
>> >>> >> > > > > >> > > > > >> > > > > wrote:
>> >>> >> > > > > >> > > > > >> > > > > >> >
>> >>> >> > > > > >> > > > > >> > > > > >> > > Failing in a consistent way,
>> with
>> >>> >> docs
>> >>> >> > > that
>> >>> >> > > > > >> explain
>> >>> >> > > > > >> > > the
>> >>> >> > > > > >> > > > > >> > various
>> >>> >> > > > > >> > > > > >> > > > > >> > > expected failures would be
>> >>> >> sufficient.
>> >>> >> > > > > >> > > > > >> > > > > >> > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > On Fri, Sep 9, 2016 at 12:16
>> PM,
>> >>> >> > Vladimir
>> >>> >> > > > > >> Rodionov
>> >>> >> > > > > >> > > > > >> > > > > >> > > <vl...@gmail.com>
>> wrote:
>> >>> >> > > > > >> > > > > >> > > > > >> > > > Do not worry Sean, doc is
>> >>> coming
>> >>> >> today
>> >>> >> > > as
>> >>> >> > > > a
>> >>> >> > > > > >> > preview
>> >>> >> > > > > >> > > > and
>> >>> >> > > > > >> > > > > >> our
>> >>> >> > > > > >> > > > > >> > > > writer
>> >>> >> > > > > >> > > > > >> > > > > >> > Frank
>> >>> >> > > > > >> > > > > >> > > > > >> > > > will be working on a putting
>> >>> it
>> >>> >> into
>> >>> >> > > > Apache
>> >>> >> > > > > >> > repo.
>> >>> >> > > > > >> > > > > >> Timeline
>> >>> >> > > > > >> > > > > >> > > > > depends
>> >>> >> > > > > >> > > > > >> > > > > >> on
>> >>> >> > > > > >> > > > > >> > > > > >> > > > Franks schedule but I hope
>> we
>> >>> will
>> >>> >> get
>> >>> >> > > it
>> >>> >> > > > > >> rather
>> >>> >> > > > > >> > > > sooner
>> >>> >> > > > > >> > > > > >> than
>> >>> >> > > > > >> > > > > >> > > > > later.
>> >>> >> > > > > >> > > > > >> > > > > >> > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > > As for failure testing, we
>> are
>> >>> >> > focusing
>> >>> >> > > > only
>> >>> >> > > > > >> on a
>> >>> >> > > > > >> > > > > >> consistent
>> >>> >> > > > > >> > > > > >> > > > state
>> >>> >> > > > > >> > > > > >> > > > > >> of
>> >>> >> > > > > >> > > > > >> > > > > >> > > > backup system data in a
>> >>> presence of
>> >>> >> > any
>> >>> >> > > > type
>> >>> >> > > > > >> of
>> >>> >> > > > > >> > > > > failures,
>> >>> >> > > > > >> > > > > >> We
>> >>> >> > > > > >> > > > > >> > > are
>> >>> >> > > > > >> > > > > >> > > > > not
>> >>> >> > > > > >> > > > > >> > > > > >> > > going
>> >>> >> > > > > >> > > > > >> > > > > >> > > > to implement  anything more
>> >>> >> "fancy",
>> >>> >> > > than
>> >>> >> > > > > >> that.
>> >>> >> > > > > >> > We
>> >>> >> > > > > >> > > > > allow
>> >>> >> > > > > >> > > > > >> > both:
>> >>> >> > > > > >> > > > > >> > > > > >> backup
>> >>> >> > > > > >> > > > > >> > > > > >> > and
>> >>> >> > > > > >> > > > > >> > > > > >> > > > restore to fail. What we do
>> not
>> >>> >> allow
>> >>> >> > is
>> >>> >> > > > to
>> >>> >> > > > > >> have
>> >>> >> > > > > >> > > > system
>> >>> >> > > > > >> > > > > >> data
>> >>> >> > > > > >> > > > > >> > > > > >> corrupted.
>> >>> >> > > > > >> > > > > >> > > > > >> > > > Will it suffice for you? Do
>> you
>> >>> >> have
>> >>> >> > any
>> >>> >> > > > > other
>> >>> >> > > > > >> > > > > concerns,
>> >>> >> > > > > >> > > > > >> you
>> >>> >> > > > > >> > > > > >> > > > want
>> >>> >> > > > > >> > > > > >> > > > > >> us to
>> >>> >> > > > > >> > > > > >> > > > > >> > > > address?
>> >>> >> > > > > >> > > > > >> > > > > >> > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > > -Vlad
>> >>> >> > > > > >> > > > > >> > > > > >> > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > > On Fri, Sep 9, 2016 at 10:56
>> >>> AM,
>> >>> >> Sean
>> >>> >> > > > > Busbey <
>> >>> >> > > > > >> > > > > >> > > busbey@apache.org
>> >>> >> > > > > >> > > > > >> > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > wrote:
>> >>> >> > > > > >> > > > > >> > > > > >> > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> "docs will come to Apache
>> >>> soon"
>> >>> >> does
>> >>> >> > > not
>> >>> >> > > > > >> address
>> >>> >> > > > > >> > > my
>> >>> >> > > > > >> > > > > >> concern
>> >>> >> > > > > >> > > > > >> > > > > around
>> >>> >> > > > > >> > > > > >> > > > > >> > docs
>> >>> >> > > > > >> > > > > >> > > > > >> > > at
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> all, unless said docs have
>> >>> already
>> >>> >> > made
>> >>> >> > > > it
>> >>> >> > > > > >> into
>> >>> >> > > > > >> > > the
>> >>> >> > > > > >> > > > > >> project
>> >>> >> > > > > >> > > > > >> > > > > repo. I
>> >>> >> > > > > >> > > > > >> > > > > >> > > don't
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> want third party resources
>> for
>> >>> >> using
>> >>> >> > a
>> >>> >> > > > > major
>> >>> >> > > > > >> and
>> >>> >> > > > > >> > > > > >> important
>> >>> >> > > > > >> > > > > >> > > > > feature
>> >>> >> > > > > >> > > > > >> > > > > >> of
>> >>> >> > > > > >> > > > > >> > > > > >> > > the
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> project, I want us to
>> provide
>> >>> end
>> >>> >> > users
>> >>> >> > > > > with
>> >>> >> > > > > >> > what
>> >>> >> > > > > >> > > > they
>> >>> >> > > > > >> > > > > >> need
>> >>> >> > > > > >> > > > > >> > > to
>> >>> >> > > > > >> > > > > >> > > > > get
>> >>> >> > > > > >> > > > > >> > > > > >> the
>> >>> >> > > > > >> > > > > >> > > > > >> > > job
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> done.
>> >>> >> > > > > >> > > > > >> > > > > >> > > >>
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> I see some calls for
>> patience
>> >>> on
>> >>> >> the
>> >>> >> > > > > failure
>> >>> >> > > > > >> > > > testing,
>> >>> >> > > > > >> > > > > but
>> >>> >> > > > > >> > > > > >> > the
>> >>> >> > > > > >> > > > > >> > > > > >> appeal
>> >>> >> > > > > >> > > > > >> > > > > >> > to
>> >>> >> > > > > >> > > > > >> > > > > >> > > us
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> having done a bad job of
>> >>> requiring
>> >>> >> > > proper
>> >>> >> > > > > >> tests
>> >>> >> > > > > >> > of
>> >>> >> > > > > >> > > > > >> previous
>> >>> >> > > > > >> > > > > >> > > > > >> features
>> >>> >> > > > > >> > > > > >> > > > > >> > > just
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> makes me more concerned
>> about
>> >>> not
>> >>> >> > > getting
>> >>> >> > > > > >> them
>> >>> >> > > > > >> > > > here. I
>> >>> >> > > > > >> > > > > >> > don't
>> >>> >> > > > > >> > > > > >> > > > want
>> >>> >> > > > > >> > > > > >> > > > > >> to
>> >>> >> > > > > >> > > > > >> > > > > >> > set
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> yet another bad example
>> that
>> >>> will
>> >>> >> > then
>> >>> >> > > be
>> >>> >> > > > > >> > pointed
>> >>> >> > > > > >> > > to
>> >>> >> > > > > >> > > > > in
>> >>> >> > > > > >> > > > > >> the
>> >>> >> > > > > >> > > > > >> > > > > future.
>> >>> >> > > > > >> > > > > >> > > > > >> > > >>
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> On Sep 8, 2016 10:50, "Ted
>> >>> Yu" <
>> >>> >> > > > > >> > > yuzhihong@gmail.com
>> >>> >> > > > > >> > > > >
>> >>> >> > > > > >> > > > > >> > wrote:
>> >>> >> > > > > >> > > > > >> > > > > >> > > >>
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > Is there any concern
>> which
>> >>> is
>> >>> >> not
>> >>> >> > > > > >> addressed ?
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > Do we need another Vote
>> >>> thread ?
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > Thanks
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > On Thu, Sep 8, 2016 at
>> 9:21
>> >>> AM,
>> >>> >> > > Andrew
>> >>> >> > > > > >> > Purtell <
>> >>> >> > > > > >> > > > > >> > > > > >> apurtell@apache.org
>> >>> >> > > > > >> > > > > >> > > > > >> > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > wrote:
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > Vlad,
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > I apologize for using
>> the
>> >>> term
>> >>> >> > > > > >> 'half-baked'
>> >>> >> > > > > >> > > in a
>> >>> >> > > > > >> > > > > way
>> >>> >> > > > > >> > > > > >> > that
>> >>> >> > > > > >> > > > > >> > > > > could
>> >>> >> > > > > >> > > > > >> > > > > >> > > seem a
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > description of
>> >>> HBASE-7912. I
>> >>> >> > meant
>> >>> >> > > > that
>> >>> >> > > > > >> as a
>> >>> >> > > > > >> > > > > general
>> >>> >> > > > > >> > > > > >> > > > > >> hypothetical.
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > On Wed, Sep 7, 2016 at
>> >>> 9:36
>> >>> >> AM,
>> >>> >> > > > > Vladimir
>> >>> >> > > > > >> > > > Rodionov
>> >>> >> > > > > >> > > > > <
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > vladrodionov@gmail.com>
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > wrote:
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >> I'm not sure that
>> >>> "There
>> >>> >> is
>> >>> >> > > > > already
>> >>> >> > > > > >> > lots
>> >>> >> > > > > >> > > of
>> >>> >> > > > > >> > > > > >> > > half-baked
>> >>> >> > > > > >> > > > > >> > > > > >> code
>> >>> >> > > > > >> > > > > >> > > > > >> > in
>> >>> >> > > > > >> > > > > >> > > > > >> > > the
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > branch,
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > so what's the harm in
>> >>> adding
>> >>> >> > > more?"
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > I meant - not
>> >>> production -
>> >>> >> > ready
>> >>> >> > > > yet.
>> >>> >> > > > > >> This
>> >>> >> > > > > >> > > is
>> >>> >> > > > > >> > > > > 2.0
>> >>> >> > > > > >> > > > > >> > > > > development
>> >>> >> > > > > >> > > > > >> > > > > >> > > branch
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > and,
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > hence many features
>> are
>> >>> in
>> >>> >> > works,
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > not being tested well
>> >>> etc.
>> >>> >> I do
>> >>> >> > > not
>> >>> >> > > > > >> > consider
>> >>> >> > > > > >> > > > > backup
>> >>> >> > > > > >> > > > > >> > as
>> >>> >> > > > > >> > > > > >> > > > half
>> >>> >> > > > > >> > > > > >> > > > > >> > baked
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > feature -
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > it has passed our
>> >>> internal
>> >>> >> QA
>> >>> >> > and
>> >>> >> > > > has
>> >>> >> > > > > >> very
>> >>> >> > > > > >> > > > good
>> >>> >> > > > > >> > > > > >> doc,
>> >>> >> > > > > >> > > > > >> > > > which
>> >>> >> > > > > >> > > > > >> > > > > we
>> >>> >> > > > > >> > > > > >> > > > > >> > will
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > provide
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > to Apache shortly.
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > -Vlad
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > On Wed, Sep 7, 2016
>> at
>> >>> 9:13
>> >>> >> AM,
>> >>> >> > > > > Andrew
>> >>> >> > > > > >> > > > Purtell <
>> >>> >> > > > > >> > > > > >> > > > > >> > > apurtell@apache.org>
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > We shouldn't admit
>> >>> half
>> >>> >> baked
>> >>> >> > > > > changes
>> >>> >> > > > > >> > that
>> >>> >> > > > > >> > > > > won't
>> >>> >> > > > > >> > > > > >> be
>> >>> >> > > > > >> > > > > >> > > > > >> finished.
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> However
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > in
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > this case the crew
>> >>> >> working on
>> >>> >> > > > this
>> >>> >> > > > > >> > feature
>> >>> >> > > > > >> > > > are
>> >>> >> > > > > >> > > > > >> long
>> >>> >> > > > > >> > > > > >> > > > > timers
>> >>> >> > > > > >> > > > > >> > > > > >> and
>> >>> >> > > > > >> > > > > >> > > > > >> > > less
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > likely
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > than just about
>> >>> anyone to
>> >>> >> > leave
>> >>> >> > > > > >> > something
>> >>> >> > > > > >> > > > in a
>> >>> >> > > > > >> > > > > >> half
>> >>> >> > > > > >> > > > > >> > > > baked
>> >>> >> > > > > >> > > > > >> > > > > >> > > state. Of
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > course
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > there is no
>> guarantee
>> >>> how
>> >>> >> > > > anything
>> >>> >> > > > > >> will
>> >>> >> > > > > >> > > turn
>> >>> >> > > > > >> > > > > out,
>> >>> >> > > > > >> > > > > >> > > but I
>> >>> >> > > > > >> > > > > >> > > > > am
>> >>> >> > > > > >> > > > > >> > > > > >> > > willing
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> to
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > take
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > a little on faith
>> if
>> >>> they
>> >>> >> > feel
>> >>> >> > > > > their
>> >>> >> > > > > >> > best
>> >>> >> > > > > >> > > > path
>> >>> >> > > > > >> > > > > >> > > forward
>> >>> >> > > > > >> > > > > >> > > > > now
>> >>> >> > > > > >> > > > > >> > > > > >> is
>> >>> >> > > > > >> > > > > >> > > > > >> > to
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > merge
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > to
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > trunk. I only wish
>> I
>> >>> had
>> >>> >> > > > bandwidth
>> >>> >> > > > > to
>> >>> >> > > > > >> > have
>> >>> >> > > > > >> > > > > done
>> >>> >> > > > > >> > > > > >> > some
>> >>> >> > > > > >> > > > > >> > > > real
>> >>> >> > > > > >> > > > > >> > > > > >> > > kicking
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> of
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > the
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > tires by now. Maybe
>> >>> this
>> >>> >> > week.
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > (Yes, I'm using
>> some
>> >>> of
>> >>> >> that
>> >>> >> > > time
>> >>> >> > > > > for
>> >>> >> > > > > >> > this
>> >>> >> > > > > >> > > > > email
>> >>> >> > > > > >> > > > > >> > :-)
>> >>> >> > > > > >> > > > > >> > > > but
>> >>> >> > > > > >> > > > > >> > > > > I
>> >>> >> > > > > >> > > > > >> > > > > >> > type
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > fast.)
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > That said, I would
>> >>> like to
>> >>> >> > > > agitate
>> >>> >> > > > > >> for
>> >>> >> > > > > >> > > > making
>> >>> >> > > > > >> > > > > 2.0
>> >>> >> > > > > >> > > > > >> > > more
>> >>> >> > > > > >> > > > > >> > > > > real
>> >>> >> > > > > >> > > > > >> > > > > >> > and
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> spend
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > some
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > time on it now that
>> >>> I'm
>> >>> >> > winding
>> >>> >> > > > > down
>> >>> >> > > > > >> > with
>> >>> >> > > > > >> > > > > 0.98. I
>> >>> >> > > > > >> > > > > >> > > think
>> >>> >> > > > > >> > > > > >> > > > > >> that
>> >>> >> > > > > >> > > > > >> > > > > >> > > means
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branching for 2.0
>> real
>> >>> >> soon
>> >>> >> > now
>> >>> >> > > > and
>> >>> >> > > > > >> even
>> >>> >> > > > > >> > > > > evicting
>> >>> >> > > > > >> > > > > >> > > > things
>> >>> >> > > > > >> > > > > >> > > > > >> from
>> >>> >> > > > > >> > > > > >> > > > > >> > > 2.0
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > branch
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > that aren't
>> finished
>> >>> or
>> >>> >> > stable,
>> >>> >> > > > > >> leaving
>> >>> >> > > > > >> > > them
>> >>> >> > > > > >> > > > > only
>> >>> >> > > > > >> > > > > >> > > once
>> >>> >> > > > > >> > > > > >> > > > > >> again
>> >>> >> > > > > >> > > > > >> > > > > >> > in
>> >>> >> > > > > >> > > > > >> > > > > >> > > the
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > master
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branch. Or, maybe
>> just
>> >>> >> > evicting
>> >>> >> > > > > them.
>> >>> >> > > > > >> > > Let's
>> >>> >> > > > > >> > > > > take
>> >>> >> > > > > >> > > > > >> it
>> >>> >> > > > > >> > > > > >> > > > case
>> >>> >> > > > > >> > > > > >> > > > > by
>> >>> >> > > > > >> > > > > >> > > > > >> > > case.
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > I think this
>> feature
>> >>> can
>> >>> >> come
>> >>> >> > > in
>> >>> >> > > > > >> > > relatively
>> >>> >> > > > > >> > > > > >> safely.
>> >>> >> > > > > >> > > > > >> > > As
>> >>> >> > > > > >> > > > > >> > > > > >> added
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > insurance,
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > let's admit the
>> >>> >> possibility
>> >>> >> > it
>> >>> >> > > > > could
>> >>> >> > > > > >> be
>> >>> >> > > > > >> > > > > reverted
>> >>> >> > > > > >> > > > > >> on
>> >>> >> > > > > >> > > > > >> > > the
>> >>> >> > > > > >> > > > > >> > > > > 2.0
>> >>> >> > > > > >> > > > > >> > > > > >> > > branch
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> if
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > folks
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > working on
>> >>> stabilizing 2.0
>> >>> >> > > decide
>> >>> >> > > > > to
>> >>> >> > > > > >> > evict
>> >>> >> > > > > >> > > > it
>> >>> >> > > > > >> > > > > >> > because
>> >>> >> > > > > >> > > > > >> > > > it
>> >>> >> > > > > >> > > > > >> > > > > is
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > unfinished
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > or
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > unstable, because
>> that
>> >>> >> > > certainly
>> >>> >> > > > > can
>> >>> >> > > > > >> > > > happen. I
>> >>> >> > > > > >> > > > > >> > would
>> >>> >> > > > > >> > > > > >> > > > > >> expect if
>> >>> >> > > > > >> > > > > >> > > > > >> > > talk
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > like
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > that starts, we'd
>> get
>> >>> help
>> >>> >> > > > > finishing
>> >>> >> > > > > >> or
>> >>> >> > > > > >> > > > > >> stabilizing
>> >>> >> > > > > >> > > > > >> > > > > what's
>> >>> >> > > > > >> > > > > >> > > > > >> > under
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > discussion
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > for revert. Or,
>> we'd
>> >>> have
>> >>> >> a
>> >>> >> > > > revert.
>> >>> >> > > > > >> > Either
>> >>> >> > > > > >> > > > way
>> >>> >> > > > > >> > > > > >> the
>> >>> >> > > > > >> > > > > >> > > > > outcome
>> >>> >> > > > > >> > > > > >> > > > > >> is
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > acceptable.
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > On Wed, Sep 7,
>> 2016 at
>> >>> >> 8:56
>> >>> >> > AM,
>> >>> >> > > > > Dima
>> >>> >> > > > > >> > > Spivak
>> >>> >> > > > > >> > > > <
>> >>> >> > > > > >> > > > > >> > > > > >> > > dimaspivak@apache.org
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > I'm not sure that
>> >>> >> "There is
>> >>> >> > > > > already
>> >>> >> > > > > >> > lots
>> >>> >> > > > > >> > > > of
>> >>> >> > > > > >> > > > > >> > > > half-baked
>> >>> >> > > > > >> > > > > >> > > > > >> code
>> >>> >> > > > > >> > > > > >> > > > > >> > in
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> the
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branch,
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > so what's the
>> harm
>> >>> in
>> >>> >> > adding
>> >>> >> > > > > more?"
>> >>> >> > > > > >> > is a
>> >>> >> > > > > >> > > > > good
>> >>> >> > > > > >> > > > > >> > code
>> >>> >> > > > > >> > > > > >> > > > > commit
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > philosophy
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > for
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > a
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > fault-tolerant
>> >>> >> distributed
>> >>> >> > > data
>> >>> >> > > > > >> store.
>> >>> >> > > > > >> > > ;)
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > More seriously, a
>> >>> lack
>> >>> >> of
>> >>> >> > > test
>> >>> >> > > > > >> > coverage
>> >>> >> > > > > >> > > > for
>> >>> >> > > > > >> > > > > >> > > existing
>> >>> >> > > > > >> > > > > >> > > > > >> > features
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > shouldn't
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > be
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > used as
>> >>> justification
>> >>> >> for
>> >>> >> > > > > >> introducing
>> >>> >> > > > > >> > > new
>> >>> >> > > > > >> > > > > >> > features
>> >>> >> > > > > >> > > > > >> > > > with
>> >>> >> > > > > >> > > > > >> > > > > >> the
>> >>> >> > > > > >> > > > > >> > > > > >> > > same
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > shortcomings.
>> >>> >> Ultimately,
>> >>> >> > > it's
>> >>> >> > > > > the
>> >>> >> > > > > >> end
>> >>> >> > > > > >> > > > user
>> >>> >> > > > > >> > > > > who
>> >>> >> > > > > >> > > > > >> > > will
>> >>> >> > > > > >> > > > > >> > > > > feel
>> >>> >> > > > > >> > > > > >> > > > > >> > the
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> pain,
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > so
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > shouldn't we do
>> >>> >> everything
>> >>> >> > we
>> >>> >> > > > can
>> >>> >> > > > > >> to
>> >>> >> > > > > >> > > > > mitigate
>> >>> >> > > > > >> > > > > >> > that?
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > -Dima
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > On Wed, Sep 7,
>> 2016
>> >>> at
>> >>> >> 8:46
>> >>> >> > > AM,
>> >>> >> > > > > >> > Vladimir
>> >>> >> > > > > >> > > > > >> > Rodionov <
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>> >>> vladrodionov@gmail.com>
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > wrote:
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Sean,
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have docs
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Agree. We have
>> a
>> >>> doc
>> >>> >> and
>> >>> >> > > > backup
>> >>> >> > > > > >> is
>> >>> >> > > > > >> > the
>> >>> >> > > > > >> > > > > most
>> >>> >> > > > > >> > > > > >> > > > > documented
>> >>> >> > > > > >> > > > > >> > > > > >> > > feature
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > :),
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > we
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > will
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > release it
>> >>> shortly to
>> >>> >> > > Apache.
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have
>> sunny-day
>> >>> >> > > correctness
>> >>> >> > > > > >> tests
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Feature has
>> >>> close to
>> >>> >> 60
>> >>> >> > > test
>> >>> >> > > > > >> cases,
>> >>> >> > > > > >> > > > which
>> >>> >> > > > > >> > > > > >> run
>> >>> >> > > > > >> > > > > >> > > for
>> >>> >> > > > > >> > > > > >> > > > > >> approx
>> >>> >> > > > > >> > > > > >> > > > > >> > 30
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> min.
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > We
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > can
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > add more, if
>> >>> >> community do
>> >>> >> > > not
>> >>> >> > > > > >> mind
>> >>> >> > > > > >> > :)
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have
>> >>> >> > > > > >> > correctness-in-face-of-failure
>> >>> >> > > > > >> > > > > tests
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Any examples of
>> >>> these
>> >>> >> > tests
>> >>> >> > > > in
>> >>> >> > > > > >> > > existing
>> >>> >> > > > > >> > > > > >> > features?
>> >>> >> > > > > >> > > > > >> > > > In
>> >>> >> > > > > >> > > > > >> > > > > >> > works,
>> >>> >> > > > > >> > > > > >> > > > > >> > > we
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > have a
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > clear
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > understanding
>> of
>> >>> what
>> >>> >> > > should
>> >>> >> > > > be
>> >>> >> > > > > >> done
>> >>> >> > > > > >> > > by
>> >>> >> > > > > >> > > > > the
>> >>> >> > > > > >> > > > > >> > time
>> >>> >> > > > > >> > > > > >> > > of
>> >>> >> > > > > >> > > > > >> > > > > 2.0
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> release.
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > That is very
>> close
>> >>> >> goal
>> >>> >> > for
>> >>> >> > > > us,
>> >>> >> > > > > >> to
>> >>> >> > > > > >> > > > verify
>> >>> >> > > > > >> > > > > IT
>> >>> >> > > > > >> > > > > >> > > monkey
>> >>> >> > > > > >> > > > > >> > > > > for
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> existing
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > code.
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * don't rely on
>> >>> things
>> >>> >> > > > outside
>> >>> >> > > > > of
>> >>> >> > > > > >> > > HBase
>> >>> >> > > > > >> > > > > for
>> >>> >> > > > > >> > > > > >> > > normal
>> >>> >> > > > > >> > > > > >> > > > > >> > operation
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > (okay
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > for
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > advanced
>> >>> operation)
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > We do not.
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Enormous time
>> has
>> >>> been
>> >>> >> > > spent
>> >>> >> > > > > >> already
>> >>> >> > > > > >> > > on
>> >>> >> > > > > >> > > > > the
>> >>> >> > > > > >> > > > > >> > > > > development
>> >>> >> > > > > >> > > > > >> > > > > >> > and
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > testing
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > the
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > feature, it has
>> >>> passed
>> >>> >> > our
>> >>> >> > > > > >> internal
>> >>> >> > > > > >> > > > tests
>> >>> >> > > > > >> > > > > and
>> >>> >> > > > > >> > > > > >> > > many
>> >>> >> > > > > >> > > > > >> > > > > >> rounds
>> >>> >> > > > > >> > > > > >> > > > > >> > of
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> code
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > reviews
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > by HBase
>> >>> committers.
>> >>> >> We
>> >>> >> > do
>> >>> >> > > > not
>> >>> >> > > > > >> mind
>> >>> >> > > > > >> > if
>> >>> >> > > > > >> > > > > >> someone
>> >>> >> > > > > >> > > > > >> > > from
>> >>> >> > > > > >> > > > > >> > > > > >> HBase
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > community
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > (outside of HW)
>> >>> will
>> >>> >> > review
>> >>> >> > > > the
>> >>> >> > > > > >> > code,
>> >>> >> > > > > >> > > > but
>> >>> >> > > > > >> > > > > it
>> >>> >> > > > > >> > > > > >> > will
>> >>> >> > > > > >> > > > > >> > > > > >> probably
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> takes
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > forever
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > to
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > wait for
>> >>> volunteer?,
>> >>> >> the
>> >>> >> > > > > feature
>> >>> >> > > > > >> is
>> >>> >> > > > > >> > > > quite
>> >>> >> > > > > >> > > > > >> large
>> >>> >> > > > > >> > > > > >> > > > (1MB+
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> cumulative
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > patch)
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > 2.0 branch is
>> >>> full of
>> >>> >> > half
>> >>> >> > > > > baked
>> >>> >> > > > > >> > > > features,
>> >>> >> > > > > >> > > > > >> most
>> >>> >> > > > > >> > > > > >> > > of
>> >>> >> > > > > >> > > > > >> > > > > them
>> >>> >> > > > > >> > > > > >> > > > > >> > are
>> >>> >> > > > > >> > > > > >> > > > > >> > > in
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > active
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > development,
>> >>> >> therefore I
>> >>> >> > am
>> >>> >> > > > not
>> >>> >> > > > > >> > > > following
>> >>> >> > > > > >> > > > > you
>> >>> >> > > > > >> > > > > >> > > here,
>> >>> >> > > > > >> > > > > >> > > > > >> Sean?
>> >>> >> > > > > >> > > > > >> > > > > >> > > Why
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > HBASE-7912
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > is
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > not good enough
>> >>> yet
>> >>> >> to be
>> >>> >> > > > > >> integrated
>> >>> >> > > > > >> > > > into
>> >>> >> > > > > >> > > > > 2.0
>> >>> >> > > > > >> > > > > >> > > > branch?
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > -Vlad
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > On Wed, Sep 7,
>> >>> 2016 at
>> >>> >> > 8:23
>> >>> >> > > > AM,
>> >>> >> > > > > >> Sean
>> >>> >> > > > > >> > > > > Busbey <
>> >>> >> > > > > >> > > > > >> > > > > >> > > busbey@apache.org
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > On Tue, Sep
>> 6,
>> >>> 2016
>> >>> >> at
>> >>> >> > > > 10:36
>> >>> >> > > > > >> PM,
>> >>> >> > > > > >> > > Josh
>> >>> >> > > > > >> > > > > >> Elser <
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > josh.elser@gmail.com>
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > wrote:
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > So, the
>> >>> answer to
>> >>> >> > > Sean's
>> >>> >> > > > > >> > original
>> >>> >> > > > > >> > > > > >> question
>> >>> >> > > > > >> > > > > >> > is
>> >>> >> > > > > >> > > > > >> > > > "as
>> >>> >> > > > > >> > > > > >> > > > > >> > > robust as
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > snapshots
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > presently
>> >>> are"?
>> >>> >> > > > > >> (independence of
>> >>> >> > > > > >> > > > > >> > > backup/restore
>> >>> >> > > > > >> > > > > >> > > > > >> > failure
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > tolerance
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > from
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > snapshot
>> >>> failure
>> >>> >> > > > tolerance)
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > Is this
>> just a
>> >>> >> > question
>> >>> >> > > > WRT
>> >>> >> > > > > >> > > context
>> >>> >> > > > > >> > > > of
>> >>> >> > > > > >> > > > > >> the
>> >>> >> > > > > >> > > > > >> > > > > change,
>> >>> >> > > > > >> > > > > >> > > > > >> or
>> >>> >> > > > > >> > > > > >> > > > > >> > > is it
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > means
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > for a
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > veto
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > from you,
>> >>> Sean?
>> >>> >> Just
>> >>> >> > > > trying
>> >>> >> > > > > >> to
>> >>> >> > > > > >> > > make
>> >>> >> > > > > >> > > > > sure
>> >>> >> > > > > >> > > > > >> > I'm
>> >>> >> > > > > >> > > > > >> > > > > >> following
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> along
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > adequately.
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > I'd say ATM
>> I'm
>> >>> -0,
>> >>> >> > > > bordering
>> >>> >> > > > > >> on
>> >>> >> > > > > >> > -1
>> >>> >> > > > > >> > > > but
>> >>> >> > > > > >> > > > > not
>> >>> >> > > > > >> > > > > >> > for
>> >>> >> > > > > >> > > > > >> > > > > >> reasons
>> >>> >> > > > > >> > > > > >> > > > > >> > I
>> >>> >> > > > > >> > > > > >> > > > > >> > > can
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > articulate
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > well.
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Here's an
>> >>> attempt.
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > We've been
>> >>> trying to
>> >>> >> > > move,
>> >>> >> > > > > as a
>> >>> >> > > > > >> > > > > community,
>> >>> >> > > > > >> > > > > >> > > > towards
>> >>> >> > > > > >> > > > > >> > > > > >> > > minimizing
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > risk
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > to
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > downstream
>> >>> folks by
>> >>> >> > > getting
>> >>> >> > > > > >> > > "complete
>> >>> >> > > > > >> > > > > >> enough
>> >>> >> > > > > >> > > > > >> > > for
>> >>> >> > > > > >> > > > > >> > > > > use"
>> >>> >> > > > > >> > > > > >> > > > > >> > > gates
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> in
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > place
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > before we
>> >>> introduce
>> >>> >> new
>> >>> >> > > > > >> features.
>> >>> >> > > > > >> > > This
>> >>> >> > > > > >> > > > > was
>> >>> >> > > > > >> > > > > >> > > > spurred
>> >>> >> > > > > >> > > > > >> > > > > >> by a
>> >>> >> > > > > >> > > > > >> > > > > >> > > some
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > features
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > getting in
>> >>> >> half-baked
>> >>> >> > and
>> >>> >> > > > > never
>> >>> >> > > > > >> > > making
>> >>> >> > > > > >> > > > > it
>> >>> >> > > > > >> > > > > >> to
>> >>> >> > > > > >> > > > > >> > > "can
>> >>> >> > > > > >> > > > > >> > > > > >> really
>> >>> >> > > > > >> > > > > >> > > > > >> > > use"
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > status
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > (I'm
>> thinking of
>> >>> >> > > > distributed
>> >>> >> > > > > >> log
>> >>> >> > > > > >> > > > replay
>> >>> >> > > > > >> > > > > and
>> >>> >> > > > > >> > > > > >> > the
>> >>> >> > > > > >> > > > > >> > > > > >> zk-less
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > assignment
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > stuff, I
>> don't
>> >>> >> recall
>> >>> >> > if
>> >>> >> > > > > there
>> >>> >> > > > > >> was
>> >>> >> > > > > >> > > > > more).
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The gates,
>> >>> >> generally,
>> >>> >> > > > > included
>> >>> >> > > > > >> > > things
>> >>> >> > > > > >> > > > > like:
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have docs
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have
>> sunny-day
>> >>> >> > > > correctness
>> >>> >> > > > > >> tests
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have
>> >>> >> > > > > >> > > correctness-in-face-of-failure
>> >>> >> > > > > >> > > > > tests
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't rely
>> on
>> >>> >> things
>> >>> >> > > > > outside
>> >>> >> > > > > >> of
>> >>> >> > > > > >> > > > HBase
>> >>> >> > > > > >> > > > > for
>> >>> >> > > > > >> > > > > >> > > > normal
>> >>> >> > > > > >> > > > > >> > > > > >> > > operation
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > (okay
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > for
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > advanced
>> >>> operation)
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > As an
>> example,
>> >>> we
>> >>> >> kept
>> >>> >> > > the
>> >>> >> > > > > MOB
>> >>> >> > > > > >> > work
>> >>> >> > > > > >> > > > off
>> >>> >> > > > > >> > > > > in
>> >>> >> > > > > >> > > > > >> a
>> >>> >> > > > > >> > > > > >> > > > branch
>> >>> >> > > > > >> > > > > >> > > > > >> and
>> >>> >> > > > > >> > > > > >> > > > > >> > > out
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> of
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > master
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > until it
>> could
>> >>> pass
>> >>> >> > these
>> >>> >> > > > > >> > criteria.
>> >>> >> > > > > >> > > > The
>> >>> >> > > > > >> > > > > big
>> >>> >> > > > > >> > > > > >> > > > > exemption
>> >>> >> > > > > >> > > > > >> > > > > >> > > we've
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> had
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > to
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > this was the
>> >>> >> > hbase-spark
>> >>> >> > > > > >> > > integration,
>> >>> >> > > > > >> > > > > where
>> >>> >> > > > > >> > > > > >> > we
>> >>> >> > > > > >> > > > > >> > > > all
>> >>> >> > > > > >> > > > > >> > > > > >> > agreed
>> >>> >> > > > > >> > > > > >> > > > > >> > > it
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > could
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > land in
>> master
>> >>> >> because
>> >>> >> > it
>> >>> >> > > > was
>> >>> >> > > > > >> very
>> >>> >> > > > > >> > > > well
>> >>> >> > > > > >> > > > > >> > > isolated
>> >>> >> > > > > >> > > > > >> > > > > (the
>> >>> >> > > > > >> > > > > >> > > > > >> > > slide
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > away
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > from
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > including
>> docs
>> >>> as a
>> >>> >> > > > > first-class
>> >>> >> > > > > >> > part
>> >>> >> > > > > >> > > > of
>> >>> >> > > > > >> > > > > >> > > building
>> >>> >> > > > > >> > > > > >> > > > up
>> >>> >> > > > > >> > > > > >> > > > > >> that
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > integration
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > has led me to
>> >>> doubt
>> >>> >> the
>> >>> >> > > > > wisdom
>> >>> >> > > > > >> of
>> >>> >> > > > > >> > > this
>> >>> >> > > > > >> > > > > >> > > decision).
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > We've also
>> been
>> >>> >> > treating
>> >>> >> > > > > >> inclusion
>> >>> >> > > > > >> > > in
>> >>> >> > > > > >> > > > a
>> >>> >> > > > > >> > > > > >> > > "probably
>> >>> >> > > > > >> > > > > >> > > > > >> will
>> >>> >> > > > > >> > > > > >> > > > > >> > be
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > released
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > to
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > downstream"
>> >>> branches
>> >>> >> > as a
>> >>> >> > > > > >> higher
>> >>> >> > > > > >> > > bar,
>> >>> >> > > > > >> > > > > >> > requiring
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't
>> >>> moderately
>> >>> >> > impact
>> >>> >> > > > > >> > > performance
>> >>> >> > > > > >> > > > > when
>> >>> >> > > > > >> > > > > >> > the
>> >>> >> > > > > >> > > > > >> > > > > >> feature
>> >>> >> > > > > >> > > > > >> > > > > >> > > isn't
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> in
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > use
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't
>> severely
>> >>> >> impact
>> >>> >> > > > > >> > performance
>> >>> >> > > > > >> > > > when
>> >>> >> > > > > >> > > > > >> the
>> >>> >> > > > > >> > > > > >> > > > > feature
>> >>> >> > > > > >> > > > > >> > > > > >> is
>> >>> >> > > > > >> > > > > >> > > > > >> > in
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> use
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * either
>> >>> >> default-to-on
>> >>> >> > or
>> >>> >> > > > > show
>> >>> >> > > > > >> > > enough
>> >>> >> > > > > >> > > > > >> demand
>> >>> >> > > > > >> > > > > >> > to
>> >>> >> > > > > >> > > > > >> > > > > >> believe
>> >>> >> > > > > >> > > > > >> > > > > >> > a
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > non-trivial
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > number of
>> folks
>> >>> will
>> >>> >> > turn
>> >>> >> > > > the
>> >>> >> > > > > >> > > feature
>> >>> >> > > > > >> > > > on
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The above has
>> >>> kept
>> >>> >> MOB
>> >>> >> > > and
>> >>> >> > > > > >> > > hbase-spark
>> >>> >> > > > > >> > > > > >> > > > integration
>> >>> >> > > > > >> > > > > >> > > > > >> out
>> >>> >> > > > > >> > > > > >> > > > > >> > of
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > branch-1,
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > presumably
>> while
>> >>> >> > they've
>> >>> >> > > > > >> "gotten
>> >>> >> > > > > >> > > more
>> >>> >> > > > > >> > > > > >> stable"
>> >>> >> > > > > >> > > > > >> > > in
>> >>> >> > > > > >> > > > > >> > > > > >> master
>> >>> >> > > > > >> > > > > >> > > > > >> > > from
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > the
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > odd
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > vendor
>> >>> inclusion.
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Are we going
>> to
>> >>> >> have a
>> >>> >> > > 2.0
>> >>> >> > > > > >> release
>> >>> >> > > > > >> > > > > before
>> >>> >> > > > > >> > > > > >> the
>> >>> >> > > > > >> > > > > >> > > end
>> >>> >> > > > > >> > > > > >> > > > > of
>> >>> >> > > > > >> > > > > >> > > > > >> the
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> year?
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > We're
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > coming up on
>> 1.5
>> >>> >> years
>> >>> >> > > > since
>> >>> >> > > > > >> the
>> >>> >> > > > > >> > > > > release of
>> >>> >> > > > > >> > > > > >> > > > version
>> >>> >> > > > > >> > > > > >> > > > > >> 1.0;
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> seems
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > like
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > it's about
>> time,
>> >>> >> > though I
>> >>> >> > > > > >> haven't
>> >>> >> > > > > >> > > seen
>> >>> >> > > > > >> > > > > any
>> >>> >> > > > > >> > > > > >> > > > concrete
>> >>> >> > > > > >> > > > > >> > > > > >> > plans
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> this
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > year.
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Presuming we
>> are
>> >>> >> going
>> >>> >> > to
>> >>> >> > > > > have
>> >>> >> > > > > >> one
>> >>> >> > > > > >> > > by
>> >>> >> > > > > >> > > > > the
>> >>> >> > > > > >> > > > > >> end
>> >>> >> > > > > >> > > > > >> > > of
>> >>> >> > > > > >> > > > > >> > > > > the
>> >>> >> > > > > >> > > > > >> > > > > >> > > year, it
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > seems a
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > bit close to
>> >>> still
>> >>> >> be
>> >>> >> > > > adding
>> >>> >> > > > > in
>> >>> >> > > > > >> > > > > "features
>> >>> >> > > > > >> > > > > >> > that
>> >>> >> > > > > >> > > > > >> > > > need
>> >>> >> > > > > >> > > > > >> > > > > >> > > maturing"
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > on
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > the
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > branch.
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The lack of a
>> >>> >> concrete
>> >>> >> > > plan
>> >>> >> > > > > for
>> >>> >> > > > > >> > 2.0
>> >>> >> > > > > >> > > > > keeps
>> >>> >> > > > > >> > > > > >> me
>> >>> >> > > > > >> > > > > >> > > from
>> >>> >> > > > > >> > > > > >> > > > > >> > > considering
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > these
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > things
>> blocker
>> >>> at
>> >>> >> the
>> >>> >> > > > moment.
>> >>> >> > > > > >> But
>> >>> >> > > > > >> > I
>> >>> >> > > > > >> > > > know
>> >>> >> > > > > >> > > > > >> > first
>> >>> >> > > > > >> > > > > >> > > > hand
>> >>> >> > > > > >> > > > > >> > > > > >> how
>> >>> >> > > > > >> > > > > >> > > > > >> > > much
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > trouble
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > folks have
>> had
>> >>> with
>> >>> >> > other
>> >>> >> > > > > >> features
>> >>> >> > > > > >> > > > that
>> >>> >> > > > > >> > > > > >> have
>> >>> >> > > > > >> > > > > >> > > gone
>> >>> >> > > > > >> > > > > >> > > > > >> into
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > downstream
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > facing
>> releases
>> >>> >> without
>> >>> >> > > > > >> robustness
>> >>> >> > > > > >> > > > > checks
>> >>> >> > > > > >> > > > > >> > (i.e.
>> >>> >> > > > > >> > > > > >> > > > > >> > > replication),
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > and
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > I'm
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > concerned
>> about
>> >>> what
>> >>> >> > > we're
>> >>> >> > > > > >> setting
>> >>> >> > > > > >> > > up
>> >>> >> > > > > >> > > > if
>> >>> >> > > > > >> > > > > >> 2.0
>> >>> >> > > > > >> > > > > >> > > goes
>> >>> >> > > > > >> > > > > >> > > > > out
>> >>> >> > > > > >> > > > > >> > > > > >> > with
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> this
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > feature in
>> its
>> >>> >> current
>> >>> >> > > > state.
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > --
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > Best regards,
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >    - Andy
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > Problems worthy of
>> >>> attack
>> >>> >> > prove
>> >>> >> > > > > their
>> >>> >> > > > > >> > > worth
>> >>> >> > > > > >> > > > by
>> >>> >> > > > > >> > > > > >> > > hitting
>> >>> >> > > > > >> > > > > >> > > > > >> back. -
>> >>> >> > > > > >> > > > > >> > > > > >> > > Piet
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > Hein
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > (via Tom White)
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > --
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > Best regards,
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >    - Andy
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > Problems worthy of
>> attack
>> >>> >> prove
>> >>> >> > > their
>> >>> >> > > > > >> worth
>> >>> >> > > > > >> > by
>> >>> >> > > > > >> > > > > >> hitting
>> >>> >> > > > > >> > > > > >> > > > back.
>> >>> >> > > > > >> > > > > >> > > > > -
>> >>> >> > > > > >> > > > > >> > > > > >> > Piet
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> Hein
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > (via Tom White)
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
>> >>> >> > > > > >> > > > > >> > > > > >> > > >>
>> >>> >> > > > > >> > > > > >> > > > > >> > >
>> >>> >> > > > > >> > > > > >> > > > > >> >
>> >>> >> > > > > >> > > > > >> > > > > >>
>> >>> >> > > > > >> > > > > >> > > > > >
>> >>> >> > > > > >> > > > > >> > > > > >
>> >>> >> > > > > >> > > > > >> > > > >
>> >>> >> > > > > >> > > > > >> > > >
>> >>> >> > > > > >> > > > > >> > >
>> >>> >> > > > > >> > > > > >> >
>> >>> >> > > > > >> > > > > >>
>> >>> >> > > > > >> > > > >
>> >>> >> > > > > >> > > > >
>> >>> >> > > > > >> > > > >
>> >>> >> > > > > >> > > > > --
>> >>> >> > > > > >> > > > > busbey
>> >>> >> > > > > >> > > > >
>> >>> >> > > > > >> > > >
>> >>> >> > > > > >> > >
>> >>> >> > > > > >> >
>> >>> >> > > > > >>
>> >>> >> > > > > >
>> >>> >> > > > > >
>> >>> >> > > > >
>> >>> >> > > >
>> >>> >> > >
>> >>> >> >
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >> --
>> >>> >>
>> >>> >> -- Appy
>> >>> >>
>> >>> >
>> >>> >
>> >>>
>> >>
>> >>
>> >
>>
>
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Stack <st...@duboce.net>]

I did a super-clean and recheck out. Now it works. Sorry for the noise.
St.Ack

On Wed, Oct 12, 2016 at 1:10 PM, Vladimir Rodionov <vl...@gmail.com>
wrote:

> checked out HBASE-7912
>
> ran:
>
> mvn clean install -DskipTests
>
> successfully.
>
> -Vlad
>
> On Wed, Oct 12, 2016 at 1:01 PM, Vladimir Rodionov <vladrodionov@gmail.com
> >
> wrote:
>
> > I usually use:
> >
> > mvn clean install -DskipTests
> >
> > and it works.
> >
> > On Wed, Oct 12, 2016 at 1:01 PM, Vladimir Rodionov <
> vladrodionov@gmail.com
> > > wrote:
> >
> >> Michael,
> >>
> >> you can try master + latest patch on HBASE-14123 (v29). No need to use
> >> HBASE-7912 branch. I will double check HBASE-7912.
> >>
> >> -Vlad
> >>
> >> On Wed, Oct 12, 2016 at 12:31 PM, Stack <st...@duboce.net> wrote:
> >>
> >>> More info:
> >>>
> >>> stack@ve0524:~/hbase.git$ git checkout origin/HBASE-7912 -b 7912v2
> >>> Branch 7912v2 set up to track remote branch HBASE-7912 from origin.
> >>> Switched to a new branch '7912v2'
> >>> stack@ve0524:~/hbase.git$ java -version
> >>> java version "1.8.0_101"
> >>> Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
> >>> Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode)
> >>> stack@ve0524:~/hbase.git$ mvn clean install -DskipTests &>
> /tmp/out.txt
> >>>
> >>> ...
> >>>
> >>> St.Ack
> >>>
> >>>
> >>> On Wed, Oct 12, 2016 at 12:29 PM, Stack <st...@duboce.net> wrote:
> >>>
> >>> > Interesting. When I try it fails w/ below:
> >>> >
> >>> > [INFO] 26 warnings
> >>> > 322 [INFO] ------------------------------
> >>> -------------------------------
> >>> > 323 [INFO] ------------------------------
> >>> -------------------------------
> >>> > 324 [ERROR] COMPILATION ERROR :
> >>> > 325 [INFO] ------------------------------
> >>> -------------------------------
> >>> > 326 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/
> RowIndexCodecV2.java:[48,8]
> >>> > org.apache.hadoop.hbase.io.encoding.rowindexV2.RowIndexCodecV2 is
> not
> >>> > abstract and does not override abstract method createSeeker(org.ap
> >>> >  ache.hadoop.hbase.CellComparator,org.apache.hadoop.hbase.io.
> >>> encoding.HFileBlockDecodingContext)
> >>> > in org.apache.hadoop.hbase.io.encoding.DataBlockEncoder
> >>> > 327 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.j
> >>> ava:[143,3]
> >>> > method does not override or implement a method from a supertype
> >>> > 328 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.j
> >>> ava:[147,29]
> >>> > incompatible types: java.nio.ByteBuffer cannot be converted to
> >>> > org.apache.hadoop.hbase.nio.ByteBuff
> >>> > 329 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.j
> >>> ava:[148,33]
> >>> > cannot find symbol
> >>> > 330   symbol:   method getKeyDeepCopy()
> >>> > 331   location: variable seeker of type org.apache.hadoop.hbase.io.
> >>> > encoding.DataBlockEncoder.EncodedSeeker
> >>> > 332 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.j
> >>> ava:[153,3]
> >>> > method does not override or implement a method from a supertype
> >>> > 333 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> >>> > apache/hadoop/hbase/io/encoding/rowindexV1/
> RowIndexCodecV1.java:[45,8]
> >>> > org.apache.hadoop.hbase.io.encoding.rowindexV1.RowIndexCodecV1 is
> not
> >>> > abstract and does not override abstract method createSeeker(org.ap
> >>> >  ache.hadoop.hbase.CellComparator,org.apache.hadoop.hbase.io.
> >>> encoding.HFileBlockDecodingContext)
> >>> > in org.apache.hadoop.hbase.io.encoding.DataBlockEncoder
> >>> > 334 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> >>> > apache/hadoop/hbase/io/encoding/rowindexV1/RowIndexCodecV1.j
> >>> ava:[145,3]
> >>> > method does not override or implement a method from a supertype
> >>> > 335 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> >>> > apache/hadoop/hbase/io/encoding/rowindexV1/RowIndexCodecV1.j
> >>> ava:[158,3]
> >>> > method does not override or implement a method from a supertype
> >>> > 336 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
> >>> java:[46,8]
> >>> > org.apache.hadoop.hbase.io.encoding.rowindexV2.RowIndexSeekerV2 is
> not
> >>> > abstract and does not override abstract method compareKey(org.ap
> >>> >  ache.hadoop.hbase.CellComparator,org.apache.hadoop.hbase.Cell) in
> >>> > org.apache.hadoop.hbase.io.encoding.DataBlockEncoder.EncodedSeeker
> >>> > 337 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
> >>> java:[79,3]
> >>> > method does not override or implement a method from a supertype
> >>> > 338 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
> >>> java:[117,3]
> >>> > method does not override or implement a method from a supertype
> >>> > 339 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
> >>> java:[190,3]
> >>> > method does not override or implement a method from a supertype
> >>> > 340 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
> >>> java:[214,3]
> >>> > method does not override or implement a method from a supertype
> >>> > 341 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
> >>> java:[349,3]
> >>> > method does not override or implement a method from a supertype
> >>> > 342 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
> >>> java:[355,3]
> >>> > method does not override or implement a method from a supertype
> >>> > 343 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> >>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
> >>> java:[421,36]
> >>> > no suitable method found for uncompressTags(java.nio.
> >>> > ByteBuffer,byte[],int,int)
> >>> > 344     method org.apache.hadoop.hbase.io.TagCompressionContext.
> >>> > uncompressTags(java.io.InputStream,byte[],int,int) is not applicable
> >>> > 345       (argument mismatch; java.nio.ByteBuffer cannot be converted
> >>> to
> >>> > java.io.InputStream)
> >>> > 346     method org.apache.hadoop.hbase.io.TagCompressionContext.
> >>> > uncompressTags(org.apache.hadoop.hbase.nio.ByteBuff,byte[],int,int)
> is
> >>> > not applicable
> >>> > 347       (argument mismatch; java.nio.ByteBuffer cannot be converted
> >>> to
> >>> > org.apache.hadoop.hbase.nio.ByteBuff)
> >>> >
> >>> > ....
> >>> >
> >>> > St.Ack
> >>> >
> >>> > On Tue, Oct 11, 2016 at 5:12 PM, Apekshit Sharma <ap...@cloudera.com>
> >>> > wrote:
> >>> >
> >>> >> @stack, it compiled for me.
> >>> >>
> >>> >> Also tried few commands, and have to say, it's well designed from
> user
> >>> >> commands perspective.
> >>> >>
> >>> >> On Tue, Oct 11, 2016 at 5:08 PM, Stack <st...@duboce.net> wrote:
> >>> >>
> >>> >> > On Tue, Oct 11, 2016 at 1:50 PM, Vladimir Rodionov <
> >>> >> vladrodionov@gmail.com
> >>> >> > >
> >>> >> > wrote:
> >>> >> >
> >>> >> > > Michael,
> >>> >> > >
> >>> >> > > Its in HBASE-7912
> >>> >> > >
> >>> >> > > This is tip of git log:
> >>> >> > >
> >>> >> > > commit a072f6f49a26a7259ff2aaef6cb56d85eb592482
> >>> >> > > Author: Frank Welsch <fw...@jps.net>
> >>> >> > > Date:   Fri Sep 23 18:00:42 2016 -0400
> >>> >> > >
> >>> >> > >     HBASE-16574 Book updates for backup and restore
> >>> >> > >
> >>> >> > > commit b14e2ab1c24e65ff88dd4c579acf83cb4ed0605e
> >>> >> > > Author: tedyu <yu...@gmail.com>
> >>> >> > > Date:   Wed Oct 5 16:29:40 2016 -0700
> >>> >> > >
> >>> >> > >     HBASE-16727 Backup refactoring: remove MR dependencies from
> >>> >> HMaster
> >>> >> > > (Vladimir Rodionov)
> >>> >> > >
> >>> >> > >
> >>> >> > Thanks. I have that. I tried it and it doesn't compile for me.
> Does
> >>> it
> >>> >> > compile for you?
> >>> >> > Thanks,
> >>> >> > M
> >>> >> >
> >>> >> >
> >>> >> >
> >>> >> > > -Vlad
> >>> >> > >
> >>> >> > > On Tue, Oct 11, 2016 at 1:44 PM, Stack <st...@duboce.net>
> wrote:
> >>> >> > >
> >>> >> > > > Which branch do I check out to try it? HBASE-7912 is not it. I
> >>> don't
> >>> >> > see
> >>> >> > > an
> >>> >> > > > HBASE-16727...
> >>> >> > > > Thanks,
> >>> >> > > > M
> >>> >> > > >
> >>> >> > > >
> >>> >> > > > On Tue, Oct 11, 2016 at 11:41 AM, Vladimir Rodionov <
> >>> >> > > > vladrodionov@gmail.com>
> >>> >> > > > wrote:
> >>> >> > > >
> >>> >> > > > > The last patch is on review board:
> >>> >> > > > > https://reviews.apache.org/r/52748
> >>> >> > > > >
> >>> >> > > > > On Mon, Oct 10, 2016 at 3:52 PM, Vladimir Rodionov <
> >>> >> > > > vladrodionov@gmail.com
> >>> >> > > > > >
> >>> >> > > > > wrote:
> >>> >> > > > >
> >>> >> > > > > > >> How hard to put in an hbase-backup module? hbase-server
> >>> is
> >>> >> fat
> >>> >> > > > enough
> >>> >> > > > > > >> already. Could be done as a follow-up.
> >>> >> > > > > >
> >>> >> > > > > > https://issues.apache.org/jira/browse/HBASE-16727?
> >>> >> > > > > > focusedCommentId=15531237&page=com.atlassian.jira.
> >>> >> > > > > > plugin.system.issuetabpanels:c
> >>> omment-tabpanel#comment-15531237
> >>> >> > > > > >
> >>> >> > > > > > Can we do merge first? Then we can discuss separate
> module.
> >>> >> > > > > >
> >>> >> > > > > > -Vlad
> >>> >> > > > > >
> >>> >> > > > > > On Mon, Oct 10, 2016 at 3:44 PM, Ted Yu <
> >>> yuzhihong@gmail.com>
> >>> >> > wrote:
> >>> >> > > > > >
> >>> >> > > > > >> Looks like the first quote was cut off.
> >>> >> > > > > >> The original sentence was:
> >>> >> > > > > >>
> >>> >> > > > > >> bq. no mapreduce job launched from master or region
> server.
> >>> >> > > > > >>
> >>> >> > > > > >> mapreduce job is launched from the node where command
> line
> >>> >> tool is
> >>> >> > > > run.
> >>> >> > > > > >>
> >>> >> > > > > >> On Mon, Oct 10, 2016 at 3:38 PM, Stack <stack@duboce.net
> >
> >>> >> wrote:
> >>> >> > > > > >>
> >>> >> > > > > >> > bq. launched from master or region server.
> >>> >> > > > > >> >
> >>> >> > > > > >> > What does this mean please? Has to be run from Master
> or
> >>> >> > > > RegionServer?
> >>> >> > > > > >> Can
> >>> >> > > > > >> > it be run from another node altogether?
> >>> >> > > > > >> >
> >>> >> > > > > >> > On Mon, Oct 10, 2016 at 1:44 PM, Vladimir Rodionov <
> >>> >> > > > > >> vladrodionov@gmail.com
> >>> >> > > > > >> > >
> >>> >> > > > > >> > wrote:
> >>> >> > > > > >> >
> >>> >> > > > > >> > > >> mapreduce dependency has been moved to client side
> >>> - no
> >>> >> > > > mapreduce
> >>> >> > > > > >> job
> >>> >> > > > > >> > >
> >>> >> > > > > >> > > 1. We have no code in the client module anymore, due
> to
> >>> >> > > dependency
> >>> >> > > > > on
> >>> >> > > > > >> > > internal server API (HFile and WAL access).
> >>> >> > > > > >> > > 2. Backup/ restore are client - driven operations,
> but
> >>> all
> >>> >> the
> >>> >> > > > code
> >>> >> > > > > >> > resides
> >>> >> > > > > >> > > in the server module
> >>> >> > > > > >> > >
> >>> >> > > > > >> >
> >>> >> > > > > >> > How hard to put in an hbase-backup module? hbase-server
> >>> is
> >>> >> fat
> >>> >> > > > enough
> >>> >> > > > > >> > already. Could be done as a follow-up.
> >>> >> > > > > >> >
> >>> >> > > > > >> > Thanks,
> >>> >> > > > > >> > St.Ack
> >>> >> > > > > >> >
> >>> >> > > > > >> >
> >>> >> > > > > >> >
> >>> >> > > > > >> > > 3. No MR in Master, no procedure - driven execution.
> >>> >> > > > > >> > > 4. Old good MR from command-line.
> >>> >> > > > > >> > > 5. Security was simplified and now only super-user is
> >>> >> allowed
> >>> >> > to
> >>> >> > > > run
> >>> >> > > > > >> > > backup/restores.
> >>> >> > > > > >> > > 6. HBase Backup API was gone due to 1. Now only
> >>> >> command-line
> >>> >> > > > access
> >>> >> > > > > to
> >>> >> > > > > >> > > backup tools.
> >>> >> > > > > >> > >
> >>> >> > > > > >> > > These consequences of refactoring has been discussed
> in
> >>> >> > > > HBASE-16727.
> >>> >> > > > > >> > >
> >>> >> > > > > >> > > -Vlad
> >>> >> > > > > >> > >
> >>> >> > > > > >> > >
> >>> >> > > > > >> > >
> >>> >> > > > > >> > >
> >>> >> > > > > >> > > On Mon, Oct 10, 2016 at 1:31 PM, Ted Yu <
> >>> >> yuzhihong@gmail.com>
> >>> >> > > > > wrote:
> >>> >> > > > > >> > >
> >>> >> > > > > >> > > > Reviving this thread.
> >>> >> > > > > >> > > >
> >>> >> > > > > >> > > > The following has taken place:
> >>> >> > > > > >> > > >
> >>> >> > > > > >> > > > mapreduce dependency has been moved to client side
> -
> >>> no
> >>> >> > > > mapreduce
> >>> >> > > > > >> job
> >>> >> > > > > >> > > > launched from master or region server.
> >>> >> > > > > >> > > > document patch (HBASE-16574) has been integrated.
> >>> >> > > > > >> > > > Updated mega patch has been attached to
> HBASE-14123:
> >>> this
> >>> >> > > covers
> >>> >> > > > > the
> >>> >> > > > > >> > > > refactor in #1 above and the protobuf 3 merge.
> >>> >> > > > > >> > > >
> >>> >> > > > > >> > > > If community has more feedback on the merge
> >>> proposal, I
> >>> >> > would
> >>> >> > > > love
> >>> >> > > > > >> to
> >>> >> > > > > >> > > hear
> >>> >> > > > > >> > > > it.
> >>> >> > > > > >> > > >
> >>> >> > > > > >> > > > Thanks
> >>> >> > > > > >> > > >
> >>> >> > > > > >> > > > On Thu, Sep 22, 2016 at 10:31 AM, Sean Busbey <
> >>> >> > > > > busbey@cloudera.com>
> >>> >> > > > > >> > > wrote:
> >>> >> > > > > >> > > >
> >>> >> > > > > >> > > > > I'd like to see the docs proposed on HBASE-16574
> >>> >> > integrated
> >>> >> > > > into
> >>> >> > > > > >> our
> >>> >> > > > > >> > > > > project's documentation prior to merge.
> >>> >> > > > > >> > > > >
> >>> >> > > > > >> > > > > On Thu, Sep 22, 2016 at 9:02 AM, Ted Yu <
> >>> >> > > yuzhihong@gmail.com>
> >>> >> > > > > >> wrote:
> >>> >> > > > > >> > > > > > This feature can be marked experimental due to
> >>> some
> >>> >> > > > > limitations
> >>> >> > > > > >> > such
> >>> >> > > > > >> > > as
> >>> >> > > > > >> > > > > > security.
> >>> >> > > > > >> > > > > >
> >>> >> > > > > >> > > > > > Your previous round of comments have been
> >>> addressed.
> >>> >> > > > > >> > > > > > Command line tool has gone through:
> >>> >> > > > > >> > > > > >
> >>> >> > > > > >> > > > > > HBASE-16620 Fix backup command-line tool
> >>> usability
> >>> >> > issues
> >>> >> > > > > >> > > > > > HBASE-16655 hbase backup describe with
> incorrect
> >>> >> backup
> >>> >> > id
> >>> >> > > > > >> results
> >>> >> > > > > >> > in
> >>> >> > > > > >> > > > NPE
> >>> >> > > > > >> > > > > >
> >>> >> > > > > >> > > > > > The updated doc has been attached to
> HBASE-16574.
> >>> >> > > > > >> > > > > >
> >>> >> > > > > >> > > > > > Cheers
> >>> >> > > > > >> > > > > >
> >>> >> > > > > >> > > > > > On Thu, Sep 22, 2016 at 8:53 AM, Stack <
> >>> >> > stack@duboce.net>
> >>> >> > > > > >> wrote:
> >>> >> > > > > >> > > > > >
> >>> >> > > > > >> > > > > >> On Wed, Sep 21, 2016 at 7:43 AM, Ted Yu <
> >>> >> > > > yuzhihong@gmail.com
> >>> >> > > > > >
> >>> >> > > > > >> > > wrote:
> >>> >> > > > > >> > > > > >>
> >>> >> > > > > >> > > > > >> > Are there more (review) comments ?
> >>> >> > > > > >> > > > > >> >
> >>> >> > > > > >> > > > > >> >
> >>> >> > > > > >> > > > > >> Are outstanding comments addressed?
> >>> >> > > > > >> > > > > >>
> >>> >> > > > > >> > > > > >> I don't see answer to my 'is this
> >>> experimental/will
> >>> >> it
> >>> >> > be
> >>> >> > > > > >> marked
> >>> >> > > > > >> > > > > >> experimental' question.
> >>> >> > > > > >> > > > > >>
> >>> >> > > > > >> > > > > >> I ran into some issues trying to use the
> >>> feature and
> >>> >> > > > > suggested
> >>> >> > > > > >> > that
> >>> >> > > > > >> > > a
> >>> >> > > > > >> > > > > >> feature likes this needs polish else it'll
> just
> >>> rot,
> >>> >> > > > unused.
> >>> >> > > > > >> Has
> >>> >> > > > > >> > > > polish
> >>> >> > > > > >> > > > > >> been applied? All ready for another 'user'
> test?
> >>> >> > Suggest
> >>> >> > > > that
> >>> >> > > > > >> you
> >>> >> > > > > >> > > > update
> >>> >> > > > > >> > > > > >> here going forward for the benefit of those
> >>> trying
> >>> >> to
> >>> >> > > > follow
> >>> >> > > > > >> along
> >>> >> > > > > >> > > and
> >>> >> > > > > >> > > > > who
> >>> >> > > > > >> > > > > >> are not watching JIRA change fly-by.
> >>> >> > > > > >> > > > > >>
> >>> >> > > > > >> > > > > >> It looks like doc got a revision -- I have to
> >>> check
> >>> >> --
> >>> >> > to
> >>> >> > > > > take
> >>> >> > > > > >> on
> >>> >> > > > > >> > > > > >> suggestion made above but again, suggest, that
> >>> this
> >>> >> > > thread
> >>> >> > > > > gets
> >>> >> > > > > >> > > > updated.
> >>> >> > > > > >> > > > > >>
> >>> >> > > > > >> > > > > >> Thanks,
> >>> >> > > > > >> > > > > >> St.Ack
> >>> >> > > > > >> > > > > >>
> >>> >> > > > > >> > > > > >>
> >>> >> > > > > >> > > > > >>
> >>> >> > > > > >> > > > > >> > Thanks
> >>> >> > > > > >> > > > > >> >
> >>> >> > > > > >> > > > > >> > On Tue, Sep 20, 2016 at 10:02 AM, Devaraj
> Das
> >>> <
> >>> >> > > > > >> > > ddas@hortonworks.com
> >>> >> > > > > >> > > > >
> >>> >> > > > > >> > > > > >> > wrote:
> >>> >> > > > > >> > > > > >> >
> >>> >> > > > > >> > > > > >> > > Just reviving this thread. Thanks Sean,
> >>> Stack,
> >>> >> > Dima,
> >>> >> > > > and
> >>> >> > > > > >> > others
> >>> >> > > > > >> > > > for
> >>> >> > > > > >> > > > > the
> >>> >> > > > > >> > > > > >> > > thorough reviews and testing. Thanks Ted
> and
> >>> >> Vlad
> >>> >> > for
> >>> >> > > > > >> taking
> >>> >> > > > > >> > > care
> >>> >> > > > > >> > > > of
> >>> >> > > > > >> > > > > >> the
> >>> >> > > > > >> > > > > >> > > feedback. Are we all good to do the merge
> >>> now?
> >>> >> > Rather
> >>> >> > > > do
> >>> >> > > > > >> > sooner
> >>> >> > > > > >> > > > than
> >>> >> > > > > >> > > > > >> > later.
> >>> >> > > > > >> > > > > >> > > ________________________________________
> >>> >> > > > > >> > > > > >> > > From: saint.ack@gmail.com <
> >>> saint.ack@gmail.com>
> >>> >> on
> >>> >> > > > > behalf
> >>> >> > > > > >> of
> >>> >> > > > > >> > > > Stack
> >>> >> > > > > >> > > > > <
> >>> >> > > > > >> > > > > >> > > stack@duboce.net>
> >>> >> > > > > >> > > > > >> > > Sent: Monday, September 12, 2016 1:18 PM
> >>> >> > > > > >> > > > > >> > > To: HBase Dev List
> >>> >> > > > > >> > > > > >> > > Subject: Re: [DISCUSSION] Merge Backup /
> >>> >> Restore -
> >>> >> > > > Branch
> >>> >> > > > > >> > > > HBASE-7912
> >>> >> > > > > >> > > > > >> > >
> >>> >> > > > > >> > > > > >> > > On Mon, Sep 12, 2016 at 12:19 PM, Ted Yu <
> >>> >> > > > > >> yuzhihong@gmail.com
> >>> >> > > > > >> > >
> >>> >> > > > > >> > > > > wrote:
> >>> >> > > > > >> > > > > >> > >
> >>> >> > > > > >> > > > > >> > > > Mega patch (rev 18) is on HBASE-14123.
> >>> >> > > > > >> > > > > >> > > >
> >>> >> > > > > >> > > > > >> > > > Please comment on HBASE-14123 on how you
> >>> want
> >>> >> to
> >>> >> > > > > review.
> >>> >> > > > > >> > > > > >> > > >
> >>> >> > > > > >> > > > > >> > >
> >>> >> > > > > >> > > > > >> > >
> >>> >> > > > > >> > > > > >> > > Yeah. That was my lost tab. Last rb was 6
> >>> months
> >>> >> > ago.
> >>> >> > > > > >> Suggest
> >>> >> > > > > >> > > > > updating
> >>> >> > > > > >> > > > > >> > it.
> >>> >> > > > > >> > > > > >> > > RB is pretty good for review. Patch is
> only
> >>> >> 1.5M so
> >>> >> > > > > should
> >>> >> > > > > >> be
> >>> >> > > > > >> > > > fine.
> >>> >> > > > > >> > > > > >> > >
> >>> >> > > > > >> > > > > >> > > St.Ack
> >>> >> > > > > >> > > > > >> > >
> >>> >> > > > > >> > > > > >> > >
> >>> >> > > > > >> > > > > >> > > >
> >>> >> > > > > >> > > > > >> > > > Thanks
> >>> >> > > > > >> > > > > >> > > >
> >>> >> > > > > >> > > > > >> > > > On Mon, Sep 12, 2016 at 12:15 PM, Stack
> <
> >>> >> > > > > >> stack@duboce.net>
> >>> >> > > > > >> > > > wrote:
> >>> >> > > > > >> > > > > >> > > >
> >>> >> > > > > >> > > > > >> > > > > On review of the 'patch', do I just
> >>> compare
> >>> >> the
> >>> >> > > > > branch
> >>> >> > > > > >> to
> >>> >> > > > > >> > > > > master or
> >>> >> > > > > >> > > > > >> > is
> >>> >> > > > > >> > > > > >> > > > > there a megapatch posted somewhere (I
> >>> think
> >>> >> I
> >>> >> > saw
> >>> >> > > > one
> >>> >> > > > > >> but
> >>> >> > > > > >> > it
> >>> >> > > > > >> > > > > seemed
> >>> >> > > > > >> > > > > >> > > stale
> >>> >> > > > > >> > > > > >> > > > > and then I 'lost' the tab). Sorry for
> >>> dumb
> >>> >> > > > question.
> >>> >> > > > > >> > > > > >> > > > > St.Ack
> >>> >> > > > > >> > > > > >> > > > >
> >>> >> > > > > >> > > > > >> > > > > On Mon, Sep 12, 2016 at 12:01 PM,
> Stack
> >>> <
> >>> >> > > > > >> stack@duboce.net
> >>> >> > > > > >> > >
> >>> >> > > > > >> > > > > wrote:
> >>> >> > > > > >> > > > > >> > > > >
> >>> >> > > > > >> > > > > >> > > > > > Late to the game. A few comments
> after
> >>> >> > > rereading
> >>> >> > > > > this
> >>> >> > > > > >> > > thread
> >>> >> > > > > >> > > > > as a
> >>> >> > > > > >> > > > > >> > > > 'user'.
> >>> >> > > > > >> > > > > >> > > > > >
> >>> >> > > > > >> > > > > >> > > > > > + Before merge, a user-facing
> feature
> >>> like
> >>> >> > this
> >>> >> > > > > >> should
> >>> >> > > > > >> > > work
> >>> >> > > > > >> > > > > (If
> >>> >> > > > > >> > > > > >> > this
> >>> >> > > > > >> > > > > >> > > is
> >>> >> > > > > >> > > > > >> > > > > "higher-bar
> >>> >> > > > > >> > > > > >> > > > > > for new features", bring it on --
> >>> smile).
> >>> >> > > > > >> > > > > >> > > > > > + As a user, I tried the branch with
> >>> tools
> >>> >> > > after
> >>> >> > > > > >> > reviewing
> >>> >> > > > > >> > > > the
> >>> >> > > > > >> > > > > >> > > > > just-posted
> >>> >> > > > > >> > > > > >> > > > > > doc. I had an 'interesting'
> experience
> >>> >> (left
> >>> >> > > > > >> comments up
> >>> >> > > > > >> > > on
> >>> >> > > > > >> > > > > >> > issue). I
> >>> >> > > > > >> > > > > >> > > > > think
> >>> >> > > > > >> > > > > >> > > > > > the tooling/doc. important to get
> >>> right.
> >>> >> If
> >>> >> > it
> >>> >> > > > > breaks
> >>> >> > > > > >> > > easily
> >>> >> > > > > >> > > > > or
> >>> >> > > > > >> > > > > >> is
> >>> >> > > > > >> > > > > >> > > > > > inconsistent (or lacks 'polish'),
> >>> >> operators
> >>> >> > > will
> >>> >> > > > > >> judge
> >>> >> > > > > >> > the
> >>> >> > > > > >> > > > > whole
> >>> >> > > > > >> > > > > >> > > > > > backup/restore tooling chain as not
> >>> >> > trustworthy
> >>> >> > > > and
> >>> >> > > > > >> > > abandon
> >>> >> > > > > >> > > > > it.
> >>> >> > > > > >> > > > > >> > Lets
> >>> >> > > > > >> > > > > >> > > > not
> >>> >> > > > > >> > > > > >> > > > > > have this happen to this feature.
> >>> >> > > > > >> > > > > >> > > > > > + Matteo's suggestion (with a
> helpful
> >>> >> starter
> >>> >> > > > list)
> >>> >> > > > > >> that
> >>> >> > > > > >> > > > there
> >>> >> > > > > >> > > > > >> > needs
> >>> >> > > > > >> > > > > >> > > to
> >>> >> > > > > >> > > > > >> > > > > be
> >>> >> > > > > >> > > > > >> > > > > > explicit qualification on what is
> >>> actually
> >>> >> > > being
> >>> >> > > > > >> > delivered
> >>> >> > > > > >> > > > --
> >>> >> > > > > >> > > > > >> > > > including a
> >>> >> > > > > >> > > > > >> > > > > > listing of limitations (some look
> >>> serious
> >>> >> > such
> >>> >> > > as
> >>> >> > > > > >> data
> >>> >> > > > > >> > > bleed
> >>> >> > > > > >> > > > > from
> >>> >> > > > > >> > > > > >> > > other
> >>> >> > > > > >> > > > > >> > > > > > regions in WALs, but maybe I don't
> >>> care
> >>> >> for
> >>> >> > my
> >>> >> > > > use
> >>> >> > > > > >> > > case...)
> >>> >> > > > > >> > > > --
> >>> >> > > > > >> > > > > >> > needs
> >>> >> > > > > >> > > > > >> > > to
> >>> >> > > > > >> > > > > >> > > > > > accompany the merge. Lets fold them
> >>> into
> >>> >> the
> >>> >> > > user
> >>> >> > > > > >> doc.
> >>> >> > > > > >> > in
> >>> >> > > > > >> > > > the
> >>> >> > > > > >> > > > > >> > > technical
> >>> >> > > > > >> > > > > >> > > > > > overview area as suggested so user
> >>> >> > expectations
> >>> >> > > > are
> >>> >> > > > > >> > > properly
> >>> >> > > > > >> > > > > >> > managed
> >>> >> > > > > >> > > > > >> > > > > > (otherwise, they expect the world
> and
> >>> will
> >>> >> > just
> >>> >> > > > > give
> >>> >> > > > > >> up
> >>> >> > > > > >> > > when
> >>> >> > > > > >> > > > > we
> >>> >> > > > > >> > > > > >> > fall
> >>> >> > > > > >> > > > > >> > > > > > short). Vladimir did a list of what
> >>> is in
> >>> >> > each
> >>> >> > > of
> >>> >> > > > > the
> >>> >> > > > > >> > > phases
> >>> >> > > > > >> > > > > >> above
> >>> >> > > > > >> > > > > >> > > > which
> >>> >> > > > > >> > > > > >> > > > > > would serve as a good start.
> >>> >> > > > > >> > > > > >> > > > > > + Is this feature 'experimental'
> >>> (Matteo
> >>> >> asks
> >>> >> > > > > above).
> >>> >> > > > > >> > I'd
> >>> >> > > > > >> > > > > prefer
> >>> >> > > > > >> > > > > >> it
> >>> >> > > > > >> > > > > >> > > is
> >>> >> > > > > >> > > > > >> > > > > > not. If it is, it should be labelled
> >>> all
> >>> >> over
> >>> >> > > > that
> >>> >> > > > > >> it is
> >>> >> > > > > >> > > > so. I
> >>> >> > > > > >> > > > > >> see
> >>> >> > > > > >> > > > > >> > > > > current
> >>> >> > > > > >> > > > > >> > > > > > state called out as a '... technical
> >>> >> preview
> >>> >> > > > > >> feature'.
> >>> >> > > > > >> > > Does
> >>> >> > > > > >> > > > > this
> >>> >> > > > > >> > > > > >> > mean
> >>> >> > > > > >> > > > > >> > > > > > not-for-users?
> >>> >> > > > > >> > > > > >> > > > > >
> >>> >> > > > > >> > > > > >> > > > > > St.Ack
> >>> >> > > > > >> > > > > >> > > > > >
> >>> >> > > > > >> > > > > >> > > > > >
> >>> >> > > > > >> > > > > >> > > > > >
> >>> >> > > > > >> > > > > >> > > > > >
> >>> >> > > > > >> > > > > >> > > > > >
> >>> >> > > > > >> > > > > >> > > > > >
> >>> >> > > > > >> > > > > >> > > > > >
> >>> >> > > > > >> > > > > >> > > > > >
> >>> >> > > > > >> > > > > >> > > > > >
> >>> >> > > > > >> > > > > >> > > > > >
> >>> >> > > > > >> > > > > >> > > > > >
> >>> >> > > > > >> > > > > >> > > > > > On Mon, Sep 12, 2016 at 8:03 AM, Ted
> >>> Yu <
> >>> >> > > > > >> > > > yuzhihong@gmail.com>
> >>> >> > > > > >> > > > > >> > wrote:
> >>> >> > > > > >> > > > > >> > > > > >
> >>> >> > > > > >> > > > > >> > > > > >> Sean:
> >>> >> > > > > >> > > > > >> > > > > >> Do you have more comments ?
> >>> >> > > > > >> > > > > >> > > > > >>
> >>> >> > > > > >> > > > > >> > > > > >> Cheers
> >>> >> > > > > >> > > > > >> > > > > >>
> >>> >> > > > > >> > > > > >> > > > > >> On Fri, Sep 9, 2016 at 1:42 PM,
> >>> Vladimir
> >>> >> > > > Rodionov
> >>> >> > > > > <
> >>> >> > > > > >> > > > > >> > > > > vladrodionov@gmail.com
> >>> >> > > > > >> > > > > >> > > > > >> >
> >>> >> > > > > >> > > > > >> > > > > >> wrote:
> >>> >> > > > > >> > > > > >> > > > > >>
> >>> >> > > > > >> > > > > >> > > > > >> > Sean,
> >>> >> > > > > >> > > > > >> > > > > >> >
> >>> >> > > > > >> > > > > >> > > > > >> > Backup/Restore can fail due to
> >>> various
> >>> >> > > > reasons:
> >>> >> > > > > >> > network
> >>> >> > > > > >> > > > > outage
> >>> >> > > > > >> > > > > >> > > > > (cluster
> >>> >> > > > > >> > > > > >> > > > > >> > wide), various time-outs in HBase
> >>> and
> >>> >> HDFS
> >>> >> > > > > layer,
> >>> >> > > > > >> M/R
> >>> >> > > > > >> > > > > failure
> >>> >> > > > > >> > > > > >> > due
> >>> >> > > > > >> > > > > >> > > to
> >>> >> > > > > >> > > > > >> > > > > >> "HDFS
> >>> >> > > > > >> > > > > >> > > > > >> > exceeded quota", user error
> (manual
> >>> >> > deletion
> >>> >> > > > of
> >>> >> > > > > >> data)
> >>> >> > > > > >> > > and
> >>> >> > > > > >> > > > > so
> >>> >> > > > > >> > > > > >> on
> >>> >> > > > > >> > > > > >> > so
> >>> >> > > > > >> > > > > >> > > > on.
> >>> >> > > > > >> > > > > >> > > > > >> That
> >>> >> > > > > >> > > > > >> > > > > >> > is impossible to enumerate all
> >>> possible
> >>> >> > > types
> >>> >> > > > of
> >>> >> > > > > >> > > failures
> >>> >> > > > > >> > > > > in a
> >>> >> > > > > >> > > > > >> > > > > >> distributed
> >>> >> > > > > >> > > > > >> > > > > >> > system - that is not our
> goal/task.
> >>> >> > > > > >> > > > > >> > > > > >> >
> >>> >> > > > > >> > > > > >> > > > > >> > We focus completely on backup
> >>> system
> >>> >> table
> >>> >> > > > > >> > consistency
> >>> >> > > > > >> > > > in a
> >>> >> > > > > >> > > > > >> > > presence
> >>> >> > > > > >> > > > > >> > > > > of
> >>> >> > > > > >> > > > > >> > > > > >> any
> >>> >> > > > > >> > > > > >> > > > > >> > type of failure. That is what I
> >>> call
> >>> >> > > > "tolerance
> >>> >> > > > > to
> >>> >> > > > > >> > > > > failures".
> >>> >> > > > > >> > > > > >> > > > > >> >
> >>> >> > > > > >> > > > > >> > > > > >> > On a failure:
> >>> >> > > > > >> > > > > >> > > > > >> >
> >>> >> > > > > >> > > > > >> > > > > >> > BACKUP. All backup system
> >>> information
> >>> >> > (prior
> >>> >> > > > to
> >>> >> > > > > >> > backup)
> >>> >> > > > > >> > > > > will
> >>> >> > > > > >> > > > > >> be
> >>> >> > > > > >> > > > > >> > > > > restored
> >>> >> > > > > >> > > > > >> > > > > >> > and all temporary data, related
> to
> >>> a
> >>> >> > failed
> >>> >> > > > > >> session,
> >>> >> > > > > >> > in
> >>> >> > > > > >> > > > > HDFS
> >>> >> > > > > >> > > > > >> > will
> >>> >> > > > > >> > > > > >> > > be
> >>> >> > > > > >> > > > > >> > > > > >> > deleted
> >>> >> > > > > >> > > > > >> > > > > >> > RESTORE. We do not care about
> >>> system
> >>> >> data,
> >>> >> > > > > because
> >>> >> > > > > >> > > > restore
> >>> >> > > > > >> > > > > >> does
> >>> >> > > > > >> > > > > >> > > not
> >>> >> > > > > >> > > > > >> > > > > >> change
> >>> >> > > > > >> > > > > >> > > > > >> > it. Temporary data in HDFS will
> be
> >>> >> cleaned
> >>> >> > > up
> >>> >> > > > > and
> >>> >> > > > > >> > table
> >>> >> > > > > >> > > > > will
> >>> >> > > > > >> > > > > >> be
> >>> >> > > > > >> > > > > >> > > in a
> >>> >> > > > > >> > > > > >> > > > > >> state
> >>> >> > > > > >> > > > > >> > > > > >> > back to where it was before
> >>> operation
> >>> >> > > started.
> >>> >> > > > > >> > > > > >> > > > > >> >
> >>> >> > > > > >> > > > > >> > > > > >> > This is what user should expect
> in
> >>> case
> >>> >> > of a
> >>> >> > > > > >> failure.
> >>> >> > > > > >> > > > > >> > > > > >> >
> >>> >> > > > > >> > > > > >> > > > > >> > -Vlad
> >>> >> > > > > >> > > > > >> > > > > >> >
> >>> >> > > > > >> > > > > >> > > > > >> >
> >>> >> > > > > >> > > > > >> > > > > >> > -Vlad
> >>> >> > > > > >> > > > > >> > > > > >> >
> >>> >> > > > > >> > > > > >> > > > > >> > On Fri, Sep 9, 2016 at 12:56 PM,
> >>> Sean
> >>> >> > > Busbey <
> >>> >> > > > > >> > > > > >> busbey@apache.org
> >>> >> > > > > >> > > > > >> > >
> >>> >> > > > > >> > > > > >> > > > > wrote:
> >>> >> > > > > >> > > > > >> > > > > >> >
> >>> >> > > > > >> > > > > >> > > > > >> > > Failing in a consistent way,
> with
> >>> >> docs
> >>> >> > > that
> >>> >> > > > > >> explain
> >>> >> > > > > >> > > the
> >>> >> > > > > >> > > > > >> > various
> >>> >> > > > > >> > > > > >> > > > > >> > > expected failures would be
> >>> >> sufficient.
> >>> >> > > > > >> > > > > >> > > > > >> > >
> >>> >> > > > > >> > > > > >> > > > > >> > > On Fri, Sep 9, 2016 at 12:16
> PM,
> >>> >> > Vladimir
> >>> >> > > > > >> Rodionov
> >>> >> > > > > >> > > > > >> > > > > >> > > <vl...@gmail.com>
> wrote:
> >>> >> > > > > >> > > > > >> > > > > >> > > > Do not worry Sean, doc is
> >>> coming
> >>> >> today
> >>> >> > > as
> >>> >> > > > a
> >>> >> > > > > >> > preview
> >>> >> > > > > >> > > > and
> >>> >> > > > > >> > > > > >> our
> >>> >> > > > > >> > > > > >> > > > writer
> >>> >> > > > > >> > > > > >> > > > > >> > Frank
> >>> >> > > > > >> > > > > >> > > > > >> > > > will be working on a putting
> >>> it
> >>> >> into
> >>> >> > > > Apache
> >>> >> > > > > >> > repo.
> >>> >> > > > > >> > > > > >> Timeline
> >>> >> > > > > >> > > > > >> > > > > depends
> >>> >> > > > > >> > > > > >> > > > > >> on
> >>> >> > > > > >> > > > > >> > > > > >> > > > Franks schedule but I hope we
> >>> will
> >>> >> get
> >>> >> > > it
> >>> >> > > > > >> rather
> >>> >> > > > > >> > > > sooner
> >>> >> > > > > >> > > > > >> than
> >>> >> > > > > >> > > > > >> > > > > later.
> >>> >> > > > > >> > > > > >> > > > > >> > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > > As for failure testing, we
> are
> >>> >> > focusing
> >>> >> > > > only
> >>> >> > > > > >> on a
> >>> >> > > > > >> > > > > >> consistent
> >>> >> > > > > >> > > > > >> > > > state
> >>> >> > > > > >> > > > > >> > > > > >> of
> >>> >> > > > > >> > > > > >> > > > > >> > > > backup system data in a
> >>> presence of
> >>> >> > any
> >>> >> > > > type
> >>> >> > > > > >> of
> >>> >> > > > > >> > > > > failures,
> >>> >> > > > > >> > > > > >> We
> >>> >> > > > > >> > > > > >> > > are
> >>> >> > > > > >> > > > > >> > > > > not
> >>> >> > > > > >> > > > > >> > > > > >> > > going
> >>> >> > > > > >> > > > > >> > > > > >> > > > to implement  anything more
> >>> >> "fancy",
> >>> >> > > than
> >>> >> > > > > >> that.
> >>> >> > > > > >> > We
> >>> >> > > > > >> > > > > allow
> >>> >> > > > > >> > > > > >> > both:
> >>> >> > > > > >> > > > > >> > > > > >> backup
> >>> >> > > > > >> > > > > >> > > > > >> > and
> >>> >> > > > > >> > > > > >> > > > > >> > > > restore to fail. What we do
> not
> >>> >> allow
> >>> >> > is
> >>> >> > > > to
> >>> >> > > > > >> have
> >>> >> > > > > >> > > > system
> >>> >> > > > > >> > > > > >> data
> >>> >> > > > > >> > > > > >> > > > > >> corrupted.
> >>> >> > > > > >> > > > > >> > > > > >> > > > Will it suffice for you? Do
> you
> >>> >> have
> >>> >> > any
> >>> >> > > > > other
> >>> >> > > > > >> > > > > concerns,
> >>> >> > > > > >> > > > > >> you
> >>> >> > > > > >> > > > > >> > > > want
> >>> >> > > > > >> > > > > >> > > > > >> us to
> >>> >> > > > > >> > > > > >> > > > > >> > > > address?
> >>> >> > > > > >> > > > > >> > > > > >> > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > > -Vlad
> >>> >> > > > > >> > > > > >> > > > > >> > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > > On Fri, Sep 9, 2016 at 10:56
> >>> AM,
> >>> >> Sean
> >>> >> > > > > Busbey <
> >>> >> > > > > >> > > > > >> > > busbey@apache.org
> >>> >> > > > > >> > > > > >> > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > wrote:
> >>> >> > > > > >> > > > > >> > > > > >> > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> "docs will come to Apache
> >>> soon"
> >>> >> does
> >>> >> > > not
> >>> >> > > > > >> address
> >>> >> > > > > >> > > my
> >>> >> > > > > >> > > > > >> concern
> >>> >> > > > > >> > > > > >> > > > > around
> >>> >> > > > > >> > > > > >> > > > > >> > docs
> >>> >> > > > > >> > > > > >> > > > > >> > > at
> >>> >> > > > > >> > > > > >> > > > > >> > > >> all, unless said docs have
> >>> already
> >>> >> > made
> >>> >> > > > it
> >>> >> > > > > >> into
> >>> >> > > > > >> > > the
> >>> >> > > > > >> > > > > >> project
> >>> >> > > > > >> > > > > >> > > > > repo. I
> >>> >> > > > > >> > > > > >> > > > > >> > > don't
> >>> >> > > > > >> > > > > >> > > > > >> > > >> want third party resources
> for
> >>> >> using
> >>> >> > a
> >>> >> > > > > major
> >>> >> > > > > >> and
> >>> >> > > > > >> > > > > >> important
> >>> >> > > > > >> > > > > >> > > > > feature
> >>> >> > > > > >> > > > > >> > > > > >> of
> >>> >> > > > > >> > > > > >> > > > > >> > > the
> >>> >> > > > > >> > > > > >> > > > > >> > > >> project, I want us to
> provide
> >>> end
> >>> >> > users
> >>> >> > > > > with
> >>> >> > > > > >> > what
> >>> >> > > > > >> > > > they
> >>> >> > > > > >> > > > > >> need
> >>> >> > > > > >> > > > > >> > > to
> >>> >> > > > > >> > > > > >> > > > > get
> >>> >> > > > > >> > > > > >> > > > > >> the
> >>> >> > > > > >> > > > > >> > > > > >> > > job
> >>> >> > > > > >> > > > > >> > > > > >> > > >> done.
> >>> >> > > > > >> > > > > >> > > > > >> > > >>
> >>> >> > > > > >> > > > > >> > > > > >> > > >> I see some calls for
> patience
> >>> on
> >>> >> the
> >>> >> > > > > failure
> >>> >> > > > > >> > > > testing,
> >>> >> > > > > >> > > > > but
> >>> >> > > > > >> > > > > >> > the
> >>> >> > > > > >> > > > > >> > > > > >> appeal
> >>> >> > > > > >> > > > > >> > > > > >> > to
> >>> >> > > > > >> > > > > >> > > > > >> > > us
> >>> >> > > > > >> > > > > >> > > > > >> > > >> having done a bad job of
> >>> requiring
> >>> >> > > proper
> >>> >> > > > > >> tests
> >>> >> > > > > >> > of
> >>> >> > > > > >> > > > > >> previous
> >>> >> > > > > >> > > > > >> > > > > >> features
> >>> >> > > > > >> > > > > >> > > > > >> > > just
> >>> >> > > > > >> > > > > >> > > > > >> > > >> makes me more concerned
> about
> >>> not
> >>> >> > > getting
> >>> >> > > > > >> them
> >>> >> > > > > >> > > > here. I
> >>> >> > > > > >> > > > > >> > don't
> >>> >> > > > > >> > > > > >> > > > want
> >>> >> > > > > >> > > > > >> > > > > >> to
> >>> >> > > > > >> > > > > >> > > > > >> > set
> >>> >> > > > > >> > > > > >> > > > > >> > > >> yet another bad example that
> >>> will
> >>> >> > then
> >>> >> > > be
> >>> >> > > > > >> > pointed
> >>> >> > > > > >> > > to
> >>> >> > > > > >> > > > > in
> >>> >> > > > > >> > > > > >> the
> >>> >> > > > > >> > > > > >> > > > > future.
> >>> >> > > > > >> > > > > >> > > > > >> > > >>
> >>> >> > > > > >> > > > > >> > > > > >> > > >> On Sep 8, 2016 10:50, "Ted
> >>> Yu" <
> >>> >> > > > > >> > > yuzhihong@gmail.com
> >>> >> > > > > >> > > > >
> >>> >> > > > > >> > > > > >> > wrote:
> >>> >> > > > > >> > > > > >> > > > > >> > > >>
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > Is there any concern which
> >>> is
> >>> >> not
> >>> >> > > > > >> addressed ?
> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > Do we need another Vote
> >>> thread ?
> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > Thanks
> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > On Thu, Sep 8, 2016 at
> 9:21
> >>> AM,
> >>> >> > > Andrew
> >>> >> > > > > >> > Purtell <
> >>> >> > > > > >> > > > > >> > > > > >> apurtell@apache.org
> >>> >> > > > > >> > > > > >> > > > > >> > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > wrote:
> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > Vlad,
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > I apologize for using
> the
> >>> term
> >>> >> > > > > >> 'half-baked'
> >>> >> > > > > >> > > in a
> >>> >> > > > > >> > > > > way
> >>> >> > > > > >> > > > > >> > that
> >>> >> > > > > >> > > > > >> > > > > could
> >>> >> > > > > >> > > > > >> > > > > >> > > seem a
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > description of
> >>> HBASE-7912. I
> >>> >> > meant
> >>> >> > > > that
> >>> >> > > > > >> as a
> >>> >> > > > > >> > > > > general
> >>> >> > > > > >> > > > > >> > > > > >> hypothetical.
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > On Wed, Sep 7, 2016 at
> >>> 9:36
> >>> >> AM,
> >>> >> > > > > Vladimir
> >>> >> > > > > >> > > > Rodionov
> >>> >> > > > > >> > > > > <
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > vladrodionov@gmail.com>
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > wrote:
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >> I'm not sure that
> >>> "There
> >>> >> is
> >>> >> > > > > already
> >>> >> > > > > >> > lots
> >>> >> > > > > >> > > of
> >>> >> > > > > >> > > > > >> > > half-baked
> >>> >> > > > > >> > > > > >> > > > > >> code
> >>> >> > > > > >> > > > > >> > > > > >> > in
> >>> >> > > > > >> > > > > >> > > > > >> > > the
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > branch,
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > so what's the harm in
> >>> adding
> >>> >> > > more?"
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > I meant - not
> >>> production -
> >>> >> > ready
> >>> >> > > > yet.
> >>> >> > > > > >> This
> >>> >> > > > > >> > > is
> >>> >> > > > > >> > > > > 2.0
> >>> >> > > > > >> > > > > >> > > > > development
> >>> >> > > > > >> > > > > >> > > > > >> > > branch
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > and,
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > hence many features
> are
> >>> in
> >>> >> > works,
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > not being tested well
> >>> etc.
> >>> >> I do
> >>> >> > > not
> >>> >> > > > > >> > consider
> >>> >> > > > > >> > > > > backup
> >>> >> > > > > >> > > > > >> > as
> >>> >> > > > > >> > > > > >> > > > half
> >>> >> > > > > >> > > > > >> > > > > >> > baked
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > feature -
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > it has passed our
> >>> internal
> >>> >> QA
> >>> >> > and
> >>> >> > > > has
> >>> >> > > > > >> very
> >>> >> > > > > >> > > > good
> >>> >> > > > > >> > > > > >> doc,
> >>> >> > > > > >> > > > > >> > > > which
> >>> >> > > > > >> > > > > >> > > > > we
> >>> >> > > > > >> > > > > >> > > > > >> > will
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > provide
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > to Apache shortly.
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > -Vlad
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > On Wed, Sep 7, 2016 at
> >>> 9:13
> >>> >> AM,
> >>> >> > > > > Andrew
> >>> >> > > > > >> > > > Purtell <
> >>> >> > > > > >> > > > > >> > > > > >> > > apurtell@apache.org>
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > We shouldn't admit
> >>> half
> >>> >> baked
> >>> >> > > > > changes
> >>> >> > > > > >> > that
> >>> >> > > > > >> > > > > won't
> >>> >> > > > > >> > > > > >> be
> >>> >> > > > > >> > > > > >> > > > > >> finished.
> >>> >> > > > > >> > > > > >> > > > > >> > > >> However
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > in
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > this case the crew
> >>> >> working on
> >>> >> > > > this
> >>> >> > > > > >> > feature
> >>> >> > > > > >> > > > are
> >>> >> > > > > >> > > > > >> long
> >>> >> > > > > >> > > > > >> > > > > timers
> >>> >> > > > > >> > > > > >> > > > > >> and
> >>> >> > > > > >> > > > > >> > > > > >> > > less
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > likely
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > than just about
> >>> anyone to
> >>> >> > leave
> >>> >> > > > > >> > something
> >>> >> > > > > >> > > > in a
> >>> >> > > > > >> > > > > >> half
> >>> >> > > > > >> > > > > >> > > > baked
> >>> >> > > > > >> > > > > >> > > > > >> > > state. Of
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > course
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > there is no
> guarantee
> >>> how
> >>> >> > > > anything
> >>> >> > > > > >> will
> >>> >> > > > > >> > > turn
> >>> >> > > > > >> > > > > out,
> >>> >> > > > > >> > > > > >> > > but I
> >>> >> > > > > >> > > > > >> > > > > am
> >>> >> > > > > >> > > > > >> > > > > >> > > willing
> >>> >> > > > > >> > > > > >> > > > > >> > > >> to
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > take
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > a little on faith if
> >>> they
> >>> >> > feel
> >>> >> > > > > their
> >>> >> > > > > >> > best
> >>> >> > > > > >> > > > path
> >>> >> > > > > >> > > > > >> > > forward
> >>> >> > > > > >> > > > > >> > > > > now
> >>> >> > > > > >> > > > > >> > > > > >> is
> >>> >> > > > > >> > > > > >> > > > > >> > to
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > merge
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > to
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > trunk. I only wish I
> >>> had
> >>> >> > > > bandwidth
> >>> >> > > > > to
> >>> >> > > > > >> > have
> >>> >> > > > > >> > > > > done
> >>> >> > > > > >> > > > > >> > some
> >>> >> > > > > >> > > > > >> > > > real
> >>> >> > > > > >> > > > > >> > > > > >> > > kicking
> >>> >> > > > > >> > > > > >> > > > > >> > > >> of
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > the
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > tires by now. Maybe
> >>> this
> >>> >> > week.
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > (Yes, I'm using some
> >>> of
> >>> >> that
> >>> >> > > time
> >>> >> > > > > for
> >>> >> > > > > >> > this
> >>> >> > > > > >> > > > > email
> >>> >> > > > > >> > > > > >> > :-)
> >>> >> > > > > >> > > > > >> > > > but
> >>> >> > > > > >> > > > > >> > > > > I
> >>> >> > > > > >> > > > > >> > > > > >> > type
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > fast.)
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > That said, I would
> >>> like to
> >>> >> > > > agitate
> >>> >> > > > > >> for
> >>> >> > > > > >> > > > making
> >>> >> > > > > >> > > > > 2.0
> >>> >> > > > > >> > > > > >> > > more
> >>> >> > > > > >> > > > > >> > > > > real
> >>> >> > > > > >> > > > > >> > > > > >> > and
> >>> >> > > > > >> > > > > >> > > > > >> > > >> spend
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > some
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > time on it now that
> >>> I'm
> >>> >> > winding
> >>> >> > > > > down
> >>> >> > > > > >> > with
> >>> >> > > > > >> > > > > 0.98. I
> >>> >> > > > > >> > > > > >> > > think
> >>> >> > > > > >> > > > > >> > > > > >> that
> >>> >> > > > > >> > > > > >> > > > > >> > > means
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branching for 2.0
> real
> >>> >> soon
> >>> >> > now
> >>> >> > > > and
> >>> >> > > > > >> even
> >>> >> > > > > >> > > > > evicting
> >>> >> > > > > >> > > > > >> > > > things
> >>> >> > > > > >> > > > > >> > > > > >> from
> >>> >> > > > > >> > > > > >> > > > > >> > > 2.0
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > branch
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > that aren't finished
> >>> or
> >>> >> > stable,
> >>> >> > > > > >> leaving
> >>> >> > > > > >> > > them
> >>> >> > > > > >> > > > > only
> >>> >> > > > > >> > > > > >> > > once
> >>> >> > > > > >> > > > > >> > > > > >> again
> >>> >> > > > > >> > > > > >> > > > > >> > in
> >>> >> > > > > >> > > > > >> > > > > >> > > the
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > master
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branch. Or, maybe
> just
> >>> >> > evicting
> >>> >> > > > > them.
> >>> >> > > > > >> > > Let's
> >>> >> > > > > >> > > > > take
> >>> >> > > > > >> > > > > >> it
> >>> >> > > > > >> > > > > >> > > > case
> >>> >> > > > > >> > > > > >> > > > > by
> >>> >> > > > > >> > > > > >> > > > > >> > > case.
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > I think this feature
> >>> can
> >>> >> come
> >>> >> > > in
> >>> >> > > > > >> > > relatively
> >>> >> > > > > >> > > > > >> safely.
> >>> >> > > > > >> > > > > >> > > As
> >>> >> > > > > >> > > > > >> > > > > >> added
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > insurance,
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > let's admit the
> >>> >> possibility
> >>> >> > it
> >>> >> > > > > could
> >>> >> > > > > >> be
> >>> >> > > > > >> > > > > reverted
> >>> >> > > > > >> > > > > >> on
> >>> >> > > > > >> > > > > >> > > the
> >>> >> > > > > >> > > > > >> > > > > 2.0
> >>> >> > > > > >> > > > > >> > > > > >> > > branch
> >>> >> > > > > >> > > > > >> > > > > >> > > >> if
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > folks
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > working on
> >>> stabilizing 2.0
> >>> >> > > decide
> >>> >> > > > > to
> >>> >> > > > > >> > evict
> >>> >> > > > > >> > > > it
> >>> >> > > > > >> > > > > >> > because
> >>> >> > > > > >> > > > > >> > > > it
> >>> >> > > > > >> > > > > >> > > > > is
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > unfinished
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > or
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > unstable, because
> that
> >>> >> > > certainly
> >>> >> > > > > can
> >>> >> > > > > >> > > > happen. I
> >>> >> > > > > >> > > > > >> > would
> >>> >> > > > > >> > > > > >> > > > > >> expect if
> >>> >> > > > > >> > > > > >> > > > > >> > > talk
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > like
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > that starts, we'd
> get
> >>> help
> >>> >> > > > > finishing
> >>> >> > > > > >> or
> >>> >> > > > > >> > > > > >> stabilizing
> >>> >> > > > > >> > > > > >> > > > > what's
> >>> >> > > > > >> > > > > >> > > > > >> > under
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > discussion
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > for revert. Or, we'd
> >>> have
> >>> >> a
> >>> >> > > > revert.
> >>> >> > > > > >> > Either
> >>> >> > > > > >> > > > way
> >>> >> > > > > >> > > > > >> the
> >>> >> > > > > >> > > > > >> > > > > outcome
> >>> >> > > > > >> > > > > >> > > > > >> is
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > acceptable.
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > On Wed, Sep 7, 2016
> at
> >>> >> 8:56
> >>> >> > AM,
> >>> >> > > > > Dima
> >>> >> > > > > >> > > Spivak
> >>> >> > > > > >> > > > <
> >>> >> > > > > >> > > > > >> > > > > >> > > dimaspivak@apache.org
> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > I'm not sure that
> >>> >> "There is
> >>> >> > > > > already
> >>> >> > > > > >> > lots
> >>> >> > > > > >> > > > of
> >>> >> > > > > >> > > > > >> > > > half-baked
> >>> >> > > > > >> > > > > >> > > > > >> code
> >>> >> > > > > >> > > > > >> > > > > >> > in
> >>> >> > > > > >> > > > > >> > > > > >> > > >> the
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branch,
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > so what's the harm
> >>> in
> >>> >> > adding
> >>> >> > > > > more?"
> >>> >> > > > > >> > is a
> >>> >> > > > > >> > > > > good
> >>> >> > > > > >> > > > > >> > code
> >>> >> > > > > >> > > > > >> > > > > commit
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > philosophy
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > for
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > a
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > fault-tolerant
> >>> >> distributed
> >>> >> > > data
> >>> >> > > > > >> store.
> >>> >> > > > > >> > > ;)
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > More seriously, a
> >>> lack
> >>> >> of
> >>> >> > > test
> >>> >> > > > > >> > coverage
> >>> >> > > > > >> > > > for
> >>> >> > > > > >> > > > > >> > > existing
> >>> >> > > > > >> > > > > >> > > > > >> > features
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > shouldn't
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > be
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > used as
> >>> justification
> >>> >> for
> >>> >> > > > > >> introducing
> >>> >> > > > > >> > > new
> >>> >> > > > > >> > > > > >> > features
> >>> >> > > > > >> > > > > >> > > > with
> >>> >> > > > > >> > > > > >> > > > > >> the
> >>> >> > > > > >> > > > > >> > > > > >> > > same
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > shortcomings.
> >>> >> Ultimately,
> >>> >> > > it's
> >>> >> > > > > the
> >>> >> > > > > >> end
> >>> >> > > > > >> > > > user
> >>> >> > > > > >> > > > > who
> >>> >> > > > > >> > > > > >> > > will
> >>> >> > > > > >> > > > > >> > > > > feel
> >>> >> > > > > >> > > > > >> > > > > >> > the
> >>> >> > > > > >> > > > > >> > > > > >> > > >> pain,
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > so
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > shouldn't we do
> >>> >> everything
> >>> >> > we
> >>> >> > > > can
> >>> >> > > > > >> to
> >>> >> > > > > >> > > > > mitigate
> >>> >> > > > > >> > > > > >> > that?
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > -Dima
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > On Wed, Sep 7,
> 2016
> >>> at
> >>> >> 8:46
> >>> >> > > AM,
> >>> >> > > > > >> > Vladimir
> >>> >> > > > > >> > > > > >> > Rodionov <
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >>> vladrodionov@gmail.com>
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > wrote:
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Sean,
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have docs
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Agree. We have a
> >>> doc
> >>> >> and
> >>> >> > > > backup
> >>> >> > > > > >> is
> >>> >> > > > > >> > the
> >>> >> > > > > >> > > > > most
> >>> >> > > > > >> > > > > >> > > > > documented
> >>> >> > > > > >> > > > > >> > > > > >> > > feature
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > :),
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > we
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > will
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > release it
> >>> shortly to
> >>> >> > > Apache.
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have sunny-day
> >>> >> > > correctness
> >>> >> > > > > >> tests
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Feature has
> >>> close to
> >>> >> 60
> >>> >> > > test
> >>> >> > > > > >> cases,
> >>> >> > > > > >> > > > which
> >>> >> > > > > >> > > > > >> run
> >>> >> > > > > >> > > > > >> > > for
> >>> >> > > > > >> > > > > >> > > > > >> approx
> >>> >> > > > > >> > > > > >> > > > > >> > 30
> >>> >> > > > > >> > > > > >> > > > > >> > > >> min.
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > We
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > can
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > add more, if
> >>> >> community do
> >>> >> > > not
> >>> >> > > > > >> mind
> >>> >> > > > > >> > :)
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have
> >>> >> > > > > >> > correctness-in-face-of-failure
> >>> >> > > > > >> > > > > tests
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Any examples of
> >>> these
> >>> >> > tests
> >>> >> > > > in
> >>> >> > > > > >> > > existing
> >>> >> > > > > >> > > > > >> > features?
> >>> >> > > > > >> > > > > >> > > > In
> >>> >> > > > > >> > > > > >> > > > > >> > works,
> >>> >> > > > > >> > > > > >> > > > > >> > > we
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > have a
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > clear
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > understanding of
> >>> what
> >>> >> > > should
> >>> >> > > > be
> >>> >> > > > > >> done
> >>> >> > > > > >> > > by
> >>> >> > > > > >> > > > > the
> >>> >> > > > > >> > > > > >> > time
> >>> >> > > > > >> > > > > >> > > of
> >>> >> > > > > >> > > > > >> > > > > 2.0
> >>> >> > > > > >> > > > > >> > > > > >> > > >> release.
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > That is very
> close
> >>> >> goal
> >>> >> > for
> >>> >> > > > us,
> >>> >> > > > > >> to
> >>> >> > > > > >> > > > verify
> >>> >> > > > > >> > > > > IT
> >>> >> > > > > >> > > > > >> > > monkey
> >>> >> > > > > >> > > > > >> > > > > for
> >>> >> > > > > >> > > > > >> > > > > >> > > >> existing
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > code.
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * don't rely on
> >>> things
> >>> >> > > > outside
> >>> >> > > > > of
> >>> >> > > > > >> > > HBase
> >>> >> > > > > >> > > > > for
> >>> >> > > > > >> > > > > >> > > normal
> >>> >> > > > > >> > > > > >> > > > > >> > operation
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > (okay
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > for
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > advanced
> >>> operation)
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > We do not.
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Enormous time
> has
> >>> been
> >>> >> > > spent
> >>> >> > > > > >> already
> >>> >> > > > > >> > > on
> >>> >> > > > > >> > > > > the
> >>> >> > > > > >> > > > > >> > > > > development
> >>> >> > > > > >> > > > > >> > > > > >> > and
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > testing
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > the
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > feature, it has
> >>> passed
> >>> >> > our
> >>> >> > > > > >> internal
> >>> >> > > > > >> > > > tests
> >>> >> > > > > >> > > > > and
> >>> >> > > > > >> > > > > >> > > many
> >>> >> > > > > >> > > > > >> > > > > >> rounds
> >>> >> > > > > >> > > > > >> > > > > >> > of
> >>> >> > > > > >> > > > > >> > > > > >> > > >> code
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > reviews
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > by HBase
> >>> committers.
> >>> >> We
> >>> >> > do
> >>> >> > > > not
> >>> >> > > > > >> mind
> >>> >> > > > > >> > if
> >>> >> > > > > >> > > > > >> someone
> >>> >> > > > > >> > > > > >> > > from
> >>> >> > > > > >> > > > > >> > > > > >> HBase
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > community
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > (outside of HW)
> >>> will
> >>> >> > review
> >>> >> > > > the
> >>> >> > > > > >> > code,
> >>> >> > > > > >> > > > but
> >>> >> > > > > >> > > > > it
> >>> >> > > > > >> > > > > >> > will
> >>> >> > > > > >> > > > > >> > > > > >> probably
> >>> >> > > > > >> > > > > >> > > > > >> > > >> takes
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > forever
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > to
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > wait for
> >>> volunteer?,
> >>> >> the
> >>> >> > > > > feature
> >>> >> > > > > >> is
> >>> >> > > > > >> > > > quite
> >>> >> > > > > >> > > > > >> large
> >>> >> > > > > >> > > > > >> > > > (1MB+
> >>> >> > > > > >> > > > > >> > > > > >> > > >> cumulative
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > patch)
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > 2.0 branch is
> >>> full of
> >>> >> > half
> >>> >> > > > > baked
> >>> >> > > > > >> > > > features,
> >>> >> > > > > >> > > > > >> most
> >>> >> > > > > >> > > > > >> > > of
> >>> >> > > > > >> > > > > >> > > > > them
> >>> >> > > > > >> > > > > >> > > > > >> > are
> >>> >> > > > > >> > > > > >> > > > > >> > > in
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > active
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > development,
> >>> >> therefore I
> >>> >> > am
> >>> >> > > > not
> >>> >> > > > > >> > > > following
> >>> >> > > > > >> > > > > you
> >>> >> > > > > >> > > > > >> > > here,
> >>> >> > > > > >> > > > > >> > > > > >> Sean?
> >>> >> > > > > >> > > > > >> > > > > >> > > Why
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > HBASE-7912
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > is
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > not good enough
> >>> yet
> >>> >> to be
> >>> >> > > > > >> integrated
> >>> >> > > > > >> > > > into
> >>> >> > > > > >> > > > > 2.0
> >>> >> > > > > >> > > > > >> > > > branch?
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > -Vlad
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > On Wed, Sep 7,
> >>> 2016 at
> >>> >> > 8:23
> >>> >> > > > AM,
> >>> >> > > > > >> Sean
> >>> >> > > > > >> > > > > Busbey <
> >>> >> > > > > >> > > > > >> > > > > >> > > busbey@apache.org
> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > On Tue, Sep 6,
> >>> 2016
> >>> >> at
> >>> >> > > > 10:36
> >>> >> > > > > >> PM,
> >>> >> > > > > >> > > Josh
> >>> >> > > > > >> > > > > >> Elser <
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > josh.elser@gmail.com>
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > wrote:
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > So, the
> >>> answer to
> >>> >> > > Sean's
> >>> >> > > > > >> > original
> >>> >> > > > > >> > > > > >> question
> >>> >> > > > > >> > > > > >> > is
> >>> >> > > > > >> > > > > >> > > > "as
> >>> >> > > > > >> > > > > >> > > > > >> > > robust as
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > snapshots
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > presently
> >>> are"?
> >>> >> > > > > >> (independence of
> >>> >> > > > > >> > > > > >> > > backup/restore
> >>> >> > > > > >> > > > > >> > > > > >> > failure
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > tolerance
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > from
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > snapshot
> >>> failure
> >>> >> > > > tolerance)
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > Is this
> just a
> >>> >> > question
> >>> >> > > > WRT
> >>> >> > > > > >> > > context
> >>> >> > > > > >> > > > of
> >>> >> > > > > >> > > > > >> the
> >>> >> > > > > >> > > > > >> > > > > change,
> >>> >> > > > > >> > > > > >> > > > > >> or
> >>> >> > > > > >> > > > > >> > > > > >> > > is it
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > means
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > for a
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > veto
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > from you,
> >>> Sean?
> >>> >> Just
> >>> >> > > > trying
> >>> >> > > > > >> to
> >>> >> > > > > >> > > make
> >>> >> > > > > >> > > > > sure
> >>> >> > > > > >> > > > > >> > I'm
> >>> >> > > > > >> > > > > >> > > > > >> following
> >>> >> > > > > >> > > > > >> > > > > >> > > >> along
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > adequately.
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > I'd say ATM
> I'm
> >>> -0,
> >>> >> > > > bordering
> >>> >> > > > > >> on
> >>> >> > > > > >> > -1
> >>> >> > > > > >> > > > but
> >>> >> > > > > >> > > > > not
> >>> >> > > > > >> > > > > >> > for
> >>> >> > > > > >> > > > > >> > > > > >> reasons
> >>> >> > > > > >> > > > > >> > > > > >> > I
> >>> >> > > > > >> > > > > >> > > > > >> > > can
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > articulate
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > well.
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Here's an
> >>> attempt.
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > We've been
> >>> trying to
> >>> >> > > move,
> >>> >> > > > > as a
> >>> >> > > > > >> > > > > community,
> >>> >> > > > > >> > > > > >> > > > towards
> >>> >> > > > > >> > > > > >> > > > > >> > > minimizing
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > risk
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > to
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > downstream
> >>> folks by
> >>> >> > > getting
> >>> >> > > > > >> > > "complete
> >>> >> > > > > >> > > > > >> enough
> >>> >> > > > > >> > > > > >> > > for
> >>> >> > > > > >> > > > > >> > > > > use"
> >>> >> > > > > >> > > > > >> > > > > >> > > gates
> >>> >> > > > > >> > > > > >> > > > > >> > > >> in
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > place
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > before we
> >>> introduce
> >>> >> new
> >>> >> > > > > >> features.
> >>> >> > > > > >> > > This
> >>> >> > > > > >> > > > > was
> >>> >> > > > > >> > > > > >> > > > spurred
> >>> >> > > > > >> > > > > >> > > > > >> by a
> >>> >> > > > > >> > > > > >> > > > > >> > > some
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > features
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > getting in
> >>> >> half-baked
> >>> >> > and
> >>> >> > > > > never
> >>> >> > > > > >> > > making
> >>> >> > > > > >> > > > > it
> >>> >> > > > > >> > > > > >> to
> >>> >> > > > > >> > > > > >> > > "can
> >>> >> > > > > >> > > > > >> > > > > >> really
> >>> >> > > > > >> > > > > >> > > > > >> > > use"
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > status
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > (I'm thinking
> of
> >>> >> > > > distributed
> >>> >> > > > > >> log
> >>> >> > > > > >> > > > replay
> >>> >> > > > > >> > > > > and
> >>> >> > > > > >> > > > > >> > the
> >>> >> > > > > >> > > > > >> > > > > >> zk-less
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > assignment
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > stuff, I don't
> >>> >> recall
> >>> >> > if
> >>> >> > > > > there
> >>> >> > > > > >> was
> >>> >> > > > > >> > > > > more).
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The gates,
> >>> >> generally,
> >>> >> > > > > included
> >>> >> > > > > >> > > things
> >>> >> > > > > >> > > > > like:
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have docs
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have
> sunny-day
> >>> >> > > > correctness
> >>> >> > > > > >> tests
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have
> >>> >> > > > > >> > > correctness-in-face-of-failure
> >>> >> > > > > >> > > > > tests
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't rely
> on
> >>> >> things
> >>> >> > > > > outside
> >>> >> > > > > >> of
> >>> >> > > > > >> > > > HBase
> >>> >> > > > > >> > > > > for
> >>> >> > > > > >> > > > > >> > > > normal
> >>> >> > > > > >> > > > > >> > > > > >> > > operation
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > (okay
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > for
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > advanced
> >>> operation)
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > As an example,
> >>> we
> >>> >> kept
> >>> >> > > the
> >>> >> > > > > MOB
> >>> >> > > > > >> > work
> >>> >> > > > > >> > > > off
> >>> >> > > > > >> > > > > in
> >>> >> > > > > >> > > > > >> a
> >>> >> > > > > >> > > > > >> > > > branch
> >>> >> > > > > >> > > > > >> > > > > >> and
> >>> >> > > > > >> > > > > >> > > > > >> > > out
> >>> >> > > > > >> > > > > >> > > > > >> > > >> of
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > master
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > until it could
> >>> pass
> >>> >> > these
> >>> >> > > > > >> > criteria.
> >>> >> > > > > >> > > > The
> >>> >> > > > > >> > > > > big
> >>> >> > > > > >> > > > > >> > > > > exemption
> >>> >> > > > > >> > > > > >> > > > > >> > > we've
> >>> >> > > > > >> > > > > >> > > > > >> > > >> had
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > to
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > this was the
> >>> >> > hbase-spark
> >>> >> > > > > >> > > integration,
> >>> >> > > > > >> > > > > where
> >>> >> > > > > >> > > > > >> > we
> >>> >> > > > > >> > > > > >> > > > all
> >>> >> > > > > >> > > > > >> > > > > >> > agreed
> >>> >> > > > > >> > > > > >> > > > > >> > > it
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > could
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > land in master
> >>> >> because
> >>> >> > it
> >>> >> > > > was
> >>> >> > > > > >> very
> >>> >> > > > > >> > > > well
> >>> >> > > > > >> > > > > >> > > isolated
> >>> >> > > > > >> > > > > >> > > > > (the
> >>> >> > > > > >> > > > > >> > > > > >> > > slide
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > away
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > from
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > including docs
> >>> as a
> >>> >> > > > > first-class
> >>> >> > > > > >> > part
> >>> >> > > > > >> > > > of
> >>> >> > > > > >> > > > > >> > > building
> >>> >> > > > > >> > > > > >> > > > up
> >>> >> > > > > >> > > > > >> > > > > >> that
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > integration
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > has led me to
> >>> doubt
> >>> >> the
> >>> >> > > > > wisdom
> >>> >> > > > > >> of
> >>> >> > > > > >> > > this
> >>> >> > > > > >> > > > > >> > > decision).
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > We've also
> been
> >>> >> > treating
> >>> >> > > > > >> inclusion
> >>> >> > > > > >> > > in
> >>> >> > > > > >> > > > a
> >>> >> > > > > >> > > > > >> > > "probably
> >>> >> > > > > >> > > > > >> > > > > >> will
> >>> >> > > > > >> > > > > >> > > > > >> > be
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > released
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > to
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > downstream"
> >>> branches
> >>> >> > as a
> >>> >> > > > > >> higher
> >>> >> > > > > >> > > bar,
> >>> >> > > > > >> > > > > >> > requiring
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't
> >>> moderately
> >>> >> > impact
> >>> >> > > > > >> > > performance
> >>> >> > > > > >> > > > > when
> >>> >> > > > > >> > > > > >> > the
> >>> >> > > > > >> > > > > >> > > > > >> feature
> >>> >> > > > > >> > > > > >> > > > > >> > > isn't
> >>> >> > > > > >> > > > > >> > > > > >> > > >> in
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > use
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't
> severely
> >>> >> impact
> >>> >> > > > > >> > performance
> >>> >> > > > > >> > > > when
> >>> >> > > > > >> > > > > >> the
> >>> >> > > > > >> > > > > >> > > > > feature
> >>> >> > > > > >> > > > > >> > > > > >> is
> >>> >> > > > > >> > > > > >> > > > > >> > in
> >>> >> > > > > >> > > > > >> > > > > >> > > >> use
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * either
> >>> >> default-to-on
> >>> >> > or
> >>> >> > > > > show
> >>> >> > > > > >> > > enough
> >>> >> > > > > >> > > > > >> demand
> >>> >> > > > > >> > > > > >> > to
> >>> >> > > > > >> > > > > >> > > > > >> believe
> >>> >> > > > > >> > > > > >> > > > > >> > a
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > non-trivial
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > number of
> folks
> >>> will
> >>> >> > turn
> >>> >> > > > the
> >>> >> > > > > >> > > feature
> >>> >> > > > > >> > > > on
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The above has
> >>> kept
> >>> >> MOB
> >>> >> > > and
> >>> >> > > > > >> > > hbase-spark
> >>> >> > > > > >> > > > > >> > > > integration
> >>> >> > > > > >> > > > > >> > > > > >> out
> >>> >> > > > > >> > > > > >> > > > > >> > of
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > branch-1,
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > presumably
> while
> >>> >> > they've
> >>> >> > > > > >> "gotten
> >>> >> > > > > >> > > more
> >>> >> > > > > >> > > > > >> stable"
> >>> >> > > > > >> > > > > >> > > in
> >>> >> > > > > >> > > > > >> > > > > >> master
> >>> >> > > > > >> > > > > >> > > > > >> > > from
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > the
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > odd
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > vendor
> >>> inclusion.
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Are we going
> to
> >>> >> have a
> >>> >> > > 2.0
> >>> >> > > > > >> release
> >>> >> > > > > >> > > > > before
> >>> >> > > > > >> > > > > >> the
> >>> >> > > > > >> > > > > >> > > end
> >>> >> > > > > >> > > > > >> > > > > of
> >>> >> > > > > >> > > > > >> > > > > >> the
> >>> >> > > > > >> > > > > >> > > > > >> > > >> year?
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > We're
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > coming up on
> 1.5
> >>> >> years
> >>> >> > > > since
> >>> >> > > > > >> the
> >>> >> > > > > >> > > > > release of
> >>> >> > > > > >> > > > > >> > > > version
> >>> >> > > > > >> > > > > >> > > > > >> 1.0;
> >>> >> > > > > >> > > > > >> > > > > >> > > >> seems
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > like
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > it's about
> time,
> >>> >> > though I
> >>> >> > > > > >> haven't
> >>> >> > > > > >> > > seen
> >>> >> > > > > >> > > > > any
> >>> >> > > > > >> > > > > >> > > > concrete
> >>> >> > > > > >> > > > > >> > > > > >> > plans
> >>> >> > > > > >> > > > > >> > > > > >> > > >> this
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > year.
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Presuming we
> are
> >>> >> going
> >>> >> > to
> >>> >> > > > > have
> >>> >> > > > > >> one
> >>> >> > > > > >> > > by
> >>> >> > > > > >> > > > > the
> >>> >> > > > > >> > > > > >> end
> >>> >> > > > > >> > > > > >> > > of
> >>> >> > > > > >> > > > > >> > > > > the
> >>> >> > > > > >> > > > > >> > > > > >> > > year, it
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > seems a
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > bit close to
> >>> still
> >>> >> be
> >>> >> > > > adding
> >>> >> > > > > in
> >>> >> > > > > >> > > > > "features
> >>> >> > > > > >> > > > > >> > that
> >>> >> > > > > >> > > > > >> > > > need
> >>> >> > > > > >> > > > > >> > > > > >> > > maturing"
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > on
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > the
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > branch.
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The lack of a
> >>> >> concrete
> >>> >> > > plan
> >>> >> > > > > for
> >>> >> > > > > >> > 2.0
> >>> >> > > > > >> > > > > keeps
> >>> >> > > > > >> > > > > >> me
> >>> >> > > > > >> > > > > >> > > from
> >>> >> > > > > >> > > > > >> > > > > >> > > considering
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > these
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > things blocker
> >>> at
> >>> >> the
> >>> >> > > > moment.
> >>> >> > > > > >> But
> >>> >> > > > > >> > I
> >>> >> > > > > >> > > > know
> >>> >> > > > > >> > > > > >> > first
> >>> >> > > > > >> > > > > >> > > > hand
> >>> >> > > > > >> > > > > >> > > > > >> how
> >>> >> > > > > >> > > > > >> > > > > >> > > much
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > trouble
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > folks have had
> >>> with
> >>> >> > other
> >>> >> > > > > >> features
> >>> >> > > > > >> > > > that
> >>> >> > > > > >> > > > > >> have
> >>> >> > > > > >> > > > > >> > > gone
> >>> >> > > > > >> > > > > >> > > > > >> into
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > downstream
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > facing
> releases
> >>> >> without
> >>> >> > > > > >> robustness
> >>> >> > > > > >> > > > > checks
> >>> >> > > > > >> > > > > >> > (i.e.
> >>> >> > > > > >> > > > > >> > > > > >> > > replication),
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > and
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > I'm
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > concerned
> about
> >>> what
> >>> >> > > we're
> >>> >> > > > > >> setting
> >>> >> > > > > >> > > up
> >>> >> > > > > >> > > > if
> >>> >> > > > > >> > > > > >> 2.0
> >>> >> > > > > >> > > > > >> > > goes
> >>> >> > > > > >> > > > > >> > > > > out
> >>> >> > > > > >> > > > > >> > > > > >> > with
> >>> >> > > > > >> > > > > >> > > > > >> > > >> this
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > feature in its
> >>> >> current
> >>> >> > > > state.
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > --
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > Best regards,
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >    - Andy
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > Problems worthy of
> >>> attack
> >>> >> > prove
> >>> >> > > > > their
> >>> >> > > > > >> > > worth
> >>> >> > > > > >> > > > by
> >>> >> > > > > >> > > > > >> > > hitting
> >>> >> > > > > >> > > > > >> > > > > >> back. -
> >>> >> > > > > >> > > > > >> > > > > >> > > Piet
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > Hein
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > (via Tom White)
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > --
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > Best regards,
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >    - Andy
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > Problems worthy of
> attack
> >>> >> prove
> >>> >> > > their
> >>> >> > > > > >> worth
> >>> >> > > > > >> > by
> >>> >> > > > > >> > > > > >> hitting
> >>> >> > > > > >> > > > > >> > > > back.
> >>> >> > > > > >> > > > > >> > > > > -
> >>> >> > > > > >> > > > > >> > > > > >> > Piet
> >>> >> > > > > >> > > > > >> > > > > >> > > >> Hein
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > > (via Tom White)
> >>> >> > > > > >> > > > > >> > > > > >> > > >> > >
> >>> >> > > > > >> > > > > >> > > > > >> > > >> >
> >>> >> > > > > >> > > > > >> > > > > >> > > >>
> >>> >> > > > > >> > > > > >> > > > > >> > >
> >>> >> > > > > >> > > > > >> > > > > >> >
> >>> >> > > > > >> > > > > >> > > > > >>
> >>> >> > > > > >> > > > > >> > > > > >
> >>> >> > > > > >> > > > > >> > > > > >
> >>> >> > > > > >> > > > > >> > > > >
> >>> >> > > > > >> > > > > >> > > >
> >>> >> > > > > >> > > > > >> > >
> >>> >> > > > > >> > > > > >> >
> >>> >> > > > > >> > > > > >>
> >>> >> > > > > >> > > > >
> >>> >> > > > > >> > > > >
> >>> >> > > > > >> > > > >
> >>> >> > > > > >> > > > > --
> >>> >> > > > > >> > > > > busbey
> >>> >> > > > > >> > > > >
> >>> >> > > > > >> > > >
> >>> >> > > > > >> > >
> >>> >> > > > > >> >
> >>> >> > > > > >>
> >>> >> > > > > >
> >>> >> > > > > >
> >>> >> > > > >
> >>> >> > > >
> >>> >> > >
> >>> >> >
> >>> >>
> >>> >>
> >>> >>
> >>> >> --
> >>> >>
> >>> >> -- Appy
> >>> >>
> >>> >
> >>> >
> >>>
> >>
> >>
> >
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Vladimir Rodionov <vl...@gmail.com>]

checked out HBASE-7912

ran:

mvn clean install -DskipTests

successfully.

-Vlad

On Wed, Oct 12, 2016 at 1:01 PM, Vladimir Rodionov <vl...@gmail.com>
wrote:

> I usually use:
>
> mvn clean install -DskipTests
>
> and it works.
>
> On Wed, Oct 12, 2016 at 1:01 PM, Vladimir Rodionov <vladrodionov@gmail.com
> > wrote:
>
>> Michael,
>>
>> you can try master + latest patch on HBASE-14123 (v29). No need to use
>> HBASE-7912 branch. I will double check HBASE-7912.
>>
>> -Vlad
>>
>> On Wed, Oct 12, 2016 at 12:31 PM, Stack <st...@duboce.net> wrote:
>>
>>> More info:
>>>
>>> stack@ve0524:~/hbase.git$ git checkout origin/HBASE-7912 -b 7912v2
>>> Branch 7912v2 set up to track remote branch HBASE-7912 from origin.
>>> Switched to a new branch '7912v2'
>>> stack@ve0524:~/hbase.git$ java -version
>>> java version "1.8.0_101"
>>> Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
>>> Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode)
>>> stack@ve0524:~/hbase.git$ mvn clean install -DskipTests &> /tmp/out.txt
>>>
>>> ...
>>>
>>> St.Ack
>>>
>>>
>>> On Wed, Oct 12, 2016 at 12:29 PM, Stack <st...@duboce.net> wrote:
>>>
>>> > Interesting. When I try it fails w/ below:
>>> >
>>> > [INFO] 26 warnings
>>> > 322 [INFO] ------------------------------
>>> -------------------------------
>>> > 323 [INFO] ------------------------------
>>> -------------------------------
>>> > 324 [ERROR] COMPILATION ERROR :
>>> > 325 [INFO] ------------------------------
>>> -------------------------------
>>> > 326 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.java:[48,8]
>>> > org.apache.hadoop.hbase.io.encoding.rowindexV2.RowIndexCodecV2 is not
>>> > abstract and does not override abstract method createSeeker(org.ap
>>> >  ache.hadoop.hbase.CellComparator,org.apache.hadoop.hbase.io.
>>> encoding.HFileBlockDecodingContext)
>>> > in org.apache.hadoop.hbase.io.encoding.DataBlockEncoder
>>> > 327 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.j
>>> ava:[143,3]
>>> > method does not override or implement a method from a supertype
>>> > 328 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.j
>>> ava:[147,29]
>>> > incompatible types: java.nio.ByteBuffer cannot be converted to
>>> > org.apache.hadoop.hbase.nio.ByteBuff
>>> > 329 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.j
>>> ava:[148,33]
>>> > cannot find symbol
>>> > 330   symbol:   method getKeyDeepCopy()
>>> > 331   location: variable seeker of type org.apache.hadoop.hbase.io.
>>> > encoding.DataBlockEncoder.EncodedSeeker
>>> > 332 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.j
>>> ava:[153,3]
>>> > method does not override or implement a method from a supertype
>>> > 333 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> > apache/hadoop/hbase/io/encoding/rowindexV1/RowIndexCodecV1.java:[45,8]
>>> > org.apache.hadoop.hbase.io.encoding.rowindexV1.RowIndexCodecV1 is not
>>> > abstract and does not override abstract method createSeeker(org.ap
>>> >  ache.hadoop.hbase.CellComparator,org.apache.hadoop.hbase.io.
>>> encoding.HFileBlockDecodingContext)
>>> > in org.apache.hadoop.hbase.io.encoding.DataBlockEncoder
>>> > 334 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> > apache/hadoop/hbase/io/encoding/rowindexV1/RowIndexCodecV1.j
>>> ava:[145,3]
>>> > method does not override or implement a method from a supertype
>>> > 335 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> > apache/hadoop/hbase/io/encoding/rowindexV1/RowIndexCodecV1.j
>>> ava:[158,3]
>>> > method does not override or implement a method from a supertype
>>> > 336 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>>> java:[46,8]
>>> > org.apache.hadoop.hbase.io.encoding.rowindexV2.RowIndexSeekerV2 is not
>>> > abstract and does not override abstract method compareKey(org.ap
>>> >  ache.hadoop.hbase.CellComparator,org.apache.hadoop.hbase.Cell) in
>>> > org.apache.hadoop.hbase.io.encoding.DataBlockEncoder.EncodedSeeker
>>> > 337 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>>> java:[79,3]
>>> > method does not override or implement a method from a supertype
>>> > 338 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>>> java:[117,3]
>>> > method does not override or implement a method from a supertype
>>> > 339 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>>> java:[190,3]
>>> > method does not override or implement a method from a supertype
>>> > 340 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>>> java:[214,3]
>>> > method does not override or implement a method from a supertype
>>> > 341 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>>> java:[349,3]
>>> > method does not override or implement a method from a supertype
>>> > 342 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>>> java:[355,3]
>>> > method does not override or implement a method from a supertype
>>> > 343 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>>> java:[421,36]
>>> > no suitable method found for uncompressTags(java.nio.
>>> > ByteBuffer,byte[],int,int)
>>> > 344     method org.apache.hadoop.hbase.io.TagCompressionContext.
>>> > uncompressTags(java.io.InputStream,byte[],int,int) is not applicable
>>> > 345       (argument mismatch; java.nio.ByteBuffer cannot be converted
>>> to
>>> > java.io.InputStream)
>>> > 346     method org.apache.hadoop.hbase.io.TagCompressionContext.
>>> > uncompressTags(org.apache.hadoop.hbase.nio.ByteBuff,byte[],int,int) is
>>> > not applicable
>>> > 347       (argument mismatch; java.nio.ByteBuffer cannot be converted
>>> to
>>> > org.apache.hadoop.hbase.nio.ByteBuff)
>>> >
>>> > ....
>>> >
>>> > St.Ack
>>> >
>>> > On Tue, Oct 11, 2016 at 5:12 PM, Apekshit Sharma <ap...@cloudera.com>
>>> > wrote:
>>> >
>>> >> @stack, it compiled for me.
>>> >>
>>> >> Also tried few commands, and have to say, it's well designed from user
>>> >> commands perspective.
>>> >>
>>> >> On Tue, Oct 11, 2016 at 5:08 PM, Stack <st...@duboce.net> wrote:
>>> >>
>>> >> > On Tue, Oct 11, 2016 at 1:50 PM, Vladimir Rodionov <
>>> >> vladrodionov@gmail.com
>>> >> > >
>>> >> > wrote:
>>> >> >
>>> >> > > Michael,
>>> >> > >
>>> >> > > Its in HBASE-7912
>>> >> > >
>>> >> > > This is tip of git log:
>>> >> > >
>>> >> > > commit a072f6f49a26a7259ff2aaef6cb56d85eb592482
>>> >> > > Author: Frank Welsch <fw...@jps.net>
>>> >> > > Date:   Fri Sep 23 18:00:42 2016 -0400
>>> >> > >
>>> >> > >     HBASE-16574 Book updates for backup and restore
>>> >> > >
>>> >> > > commit b14e2ab1c24e65ff88dd4c579acf83cb4ed0605e
>>> >> > > Author: tedyu <yu...@gmail.com>
>>> >> > > Date:   Wed Oct 5 16:29:40 2016 -0700
>>> >> > >
>>> >> > >     HBASE-16727 Backup refactoring: remove MR dependencies from
>>> >> HMaster
>>> >> > > (Vladimir Rodionov)
>>> >> > >
>>> >> > >
>>> >> > Thanks. I have that. I tried it and it doesn't compile for me. Does
>>> it
>>> >> > compile for you?
>>> >> > Thanks,
>>> >> > M
>>> >> >
>>> >> >
>>> >> >
>>> >> > > -Vlad
>>> >> > >
>>> >> > > On Tue, Oct 11, 2016 at 1:44 PM, Stack <st...@duboce.net> wrote:
>>> >> > >
>>> >> > > > Which branch do I check out to try it? HBASE-7912 is not it. I
>>> don't
>>> >> > see
>>> >> > > an
>>> >> > > > HBASE-16727...
>>> >> > > > Thanks,
>>> >> > > > M
>>> >> > > >
>>> >> > > >
>>> >> > > > On Tue, Oct 11, 2016 at 11:41 AM, Vladimir Rodionov <
>>> >> > > > vladrodionov@gmail.com>
>>> >> > > > wrote:
>>> >> > > >
>>> >> > > > > The last patch is on review board:
>>> >> > > > > https://reviews.apache.org/r/52748
>>> >> > > > >
>>> >> > > > > On Mon, Oct 10, 2016 at 3:52 PM, Vladimir Rodionov <
>>> >> > > > vladrodionov@gmail.com
>>> >> > > > > >
>>> >> > > > > wrote:
>>> >> > > > >
>>> >> > > > > > >> How hard to put in an hbase-backup module? hbase-server
>>> is
>>> >> fat
>>> >> > > > enough
>>> >> > > > > > >> already. Could be done as a follow-up.
>>> >> > > > > >
>>> >> > > > > > https://issues.apache.org/jira/browse/HBASE-16727?
>>> >> > > > > > focusedCommentId=15531237&page=com.atlassian.jira.
>>> >> > > > > > plugin.system.issuetabpanels:c
>>> omment-tabpanel#comment-15531237
>>> >> > > > > >
>>> >> > > > > > Can we do merge first? Then we can discuss separate module.
>>> >> > > > > >
>>> >> > > > > > -Vlad
>>> >> > > > > >
>>> >> > > > > > On Mon, Oct 10, 2016 at 3:44 PM, Ted Yu <
>>> yuzhihong@gmail.com>
>>> >> > wrote:
>>> >> > > > > >
>>> >> > > > > >> Looks like the first quote was cut off.
>>> >> > > > > >> The original sentence was:
>>> >> > > > > >>
>>> >> > > > > >> bq. no mapreduce job launched from master or region server.
>>> >> > > > > >>
>>> >> > > > > >> mapreduce job is launched from the node where command line
>>> >> tool is
>>> >> > > > run.
>>> >> > > > > >>
>>> >> > > > > >> On Mon, Oct 10, 2016 at 3:38 PM, Stack <st...@duboce.net>
>>> >> wrote:
>>> >> > > > > >>
>>> >> > > > > >> > bq. launched from master or region server.
>>> >> > > > > >> >
>>> >> > > > > >> > What does this mean please? Has to be run from Master or
>>> >> > > > RegionServer?
>>> >> > > > > >> Can
>>> >> > > > > >> > it be run from another node altogether?
>>> >> > > > > >> >
>>> >> > > > > >> > On Mon, Oct 10, 2016 at 1:44 PM, Vladimir Rodionov <
>>> >> > > > > >> vladrodionov@gmail.com
>>> >> > > > > >> > >
>>> >> > > > > >> > wrote:
>>> >> > > > > >> >
>>> >> > > > > >> > > >> mapreduce dependency has been moved to client side
>>> - no
>>> >> > > > mapreduce
>>> >> > > > > >> job
>>> >> > > > > >> > >
>>> >> > > > > >> > > 1. We have no code in the client module anymore, due to
>>> >> > > dependency
>>> >> > > > > on
>>> >> > > > > >> > > internal server API (HFile and WAL access).
>>> >> > > > > >> > > 2. Backup/ restore are client - driven operations, but
>>> all
>>> >> the
>>> >> > > > code
>>> >> > > > > >> > resides
>>> >> > > > > >> > > in the server module
>>> >> > > > > >> > >
>>> >> > > > > >> >
>>> >> > > > > >> > How hard to put in an hbase-backup module? hbase-server
>>> is
>>> >> fat
>>> >> > > > enough
>>> >> > > > > >> > already. Could be done as a follow-up.
>>> >> > > > > >> >
>>> >> > > > > >> > Thanks,
>>> >> > > > > >> > St.Ack
>>> >> > > > > >> >
>>> >> > > > > >> >
>>> >> > > > > >> >
>>> >> > > > > >> > > 3. No MR in Master, no procedure - driven execution.
>>> >> > > > > >> > > 4. Old good MR from command-line.
>>> >> > > > > >> > > 5. Security was simplified and now only super-user is
>>> >> allowed
>>> >> > to
>>> >> > > > run
>>> >> > > > > >> > > backup/restores.
>>> >> > > > > >> > > 6. HBase Backup API was gone due to 1. Now only
>>> >> command-line
>>> >> > > > access
>>> >> > > > > to
>>> >> > > > > >> > > backup tools.
>>> >> > > > > >> > >
>>> >> > > > > >> > > These consequences of refactoring has been discussed in
>>> >> > > > HBASE-16727.
>>> >> > > > > >> > >
>>> >> > > > > >> > > -Vlad
>>> >> > > > > >> > >
>>> >> > > > > >> > >
>>> >> > > > > >> > >
>>> >> > > > > >> > >
>>> >> > > > > >> > > On Mon, Oct 10, 2016 at 1:31 PM, Ted Yu <
>>> >> yuzhihong@gmail.com>
>>> >> > > > > wrote:
>>> >> > > > > >> > >
>>> >> > > > > >> > > > Reviving this thread.
>>> >> > > > > >> > > >
>>> >> > > > > >> > > > The following has taken place:
>>> >> > > > > >> > > >
>>> >> > > > > >> > > > mapreduce dependency has been moved to client side -
>>> no
>>> >> > > > mapreduce
>>> >> > > > > >> job
>>> >> > > > > >> > > > launched from master or region server.
>>> >> > > > > >> > > > document patch (HBASE-16574) has been integrated.
>>> >> > > > > >> > > > Updated mega patch has been attached to HBASE-14123:
>>> this
>>> >> > > covers
>>> >> > > > > the
>>> >> > > > > >> > > > refactor in #1 above and the protobuf 3 merge.
>>> >> > > > > >> > > >
>>> >> > > > > >> > > > If community has more feedback on the merge
>>> proposal, I
>>> >> > would
>>> >> > > > love
>>> >> > > > > >> to
>>> >> > > > > >> > > hear
>>> >> > > > > >> > > > it.
>>> >> > > > > >> > > >
>>> >> > > > > >> > > > Thanks
>>> >> > > > > >> > > >
>>> >> > > > > >> > > > On Thu, Sep 22, 2016 at 10:31 AM, Sean Busbey <
>>> >> > > > > busbey@cloudera.com>
>>> >> > > > > >> > > wrote:
>>> >> > > > > >> > > >
>>> >> > > > > >> > > > > I'd like to see the docs proposed on HBASE-16574
>>> >> > integrated
>>> >> > > > into
>>> >> > > > > >> our
>>> >> > > > > >> > > > > project's documentation prior to merge.
>>> >> > > > > >> > > > >
>>> >> > > > > >> > > > > On Thu, Sep 22, 2016 at 9:02 AM, Ted Yu <
>>> >> > > yuzhihong@gmail.com>
>>> >> > > > > >> wrote:
>>> >> > > > > >> > > > > > This feature can be marked experimental due to
>>> some
>>> >> > > > > limitations
>>> >> > > > > >> > such
>>> >> > > > > >> > > as
>>> >> > > > > >> > > > > > security.
>>> >> > > > > >> > > > > >
>>> >> > > > > >> > > > > > Your previous round of comments have been
>>> addressed.
>>> >> > > > > >> > > > > > Command line tool has gone through:
>>> >> > > > > >> > > > > >
>>> >> > > > > >> > > > > > HBASE-16620 Fix backup command-line tool
>>> usability
>>> >> > issues
>>> >> > > > > >> > > > > > HBASE-16655 hbase backup describe with incorrect
>>> >> backup
>>> >> > id
>>> >> > > > > >> results
>>> >> > > > > >> > in
>>> >> > > > > >> > > > NPE
>>> >> > > > > >> > > > > >
>>> >> > > > > >> > > > > > The updated doc has been attached to HBASE-16574.
>>> >> > > > > >> > > > > >
>>> >> > > > > >> > > > > > Cheers
>>> >> > > > > >> > > > > >
>>> >> > > > > >> > > > > > On Thu, Sep 22, 2016 at 8:53 AM, Stack <
>>> >> > stack@duboce.net>
>>> >> > > > > >> wrote:
>>> >> > > > > >> > > > > >
>>> >> > > > > >> > > > > >> On Wed, Sep 21, 2016 at 7:43 AM, Ted Yu <
>>> >> > > > yuzhihong@gmail.com
>>> >> > > > > >
>>> >> > > > > >> > > wrote:
>>> >> > > > > >> > > > > >>
>>> >> > > > > >> > > > > >> > Are there more (review) comments ?
>>> >> > > > > >> > > > > >> >
>>> >> > > > > >> > > > > >> >
>>> >> > > > > >> > > > > >> Are outstanding comments addressed?
>>> >> > > > > >> > > > > >>
>>> >> > > > > >> > > > > >> I don't see answer to my 'is this
>>> experimental/will
>>> >> it
>>> >> > be
>>> >> > > > > >> marked
>>> >> > > > > >> > > > > >> experimental' question.
>>> >> > > > > >> > > > > >>
>>> >> > > > > >> > > > > >> I ran into some issues trying to use the
>>> feature and
>>> >> > > > > suggested
>>> >> > > > > >> > that
>>> >> > > > > >> > > a
>>> >> > > > > >> > > > > >> feature likes this needs polish else it'll just
>>> rot,
>>> >> > > > unused.
>>> >> > > > > >> Has
>>> >> > > > > >> > > > polish
>>> >> > > > > >> > > > > >> been applied? All ready for another 'user' test?
>>> >> > Suggest
>>> >> > > > that
>>> >> > > > > >> you
>>> >> > > > > >> > > > update
>>> >> > > > > >> > > > > >> here going forward for the benefit of those
>>> trying
>>> >> to
>>> >> > > > follow
>>> >> > > > > >> along
>>> >> > > > > >> > > and
>>> >> > > > > >> > > > > who
>>> >> > > > > >> > > > > >> are not watching JIRA change fly-by.
>>> >> > > > > >> > > > > >>
>>> >> > > > > >> > > > > >> It looks like doc got a revision -- I have to
>>> check
>>> >> --
>>> >> > to
>>> >> > > > > take
>>> >> > > > > >> on
>>> >> > > > > >> > > > > >> suggestion made above but again, suggest, that
>>> this
>>> >> > > thread
>>> >> > > > > gets
>>> >> > > > > >> > > > updated.
>>> >> > > > > >> > > > > >>
>>> >> > > > > >> > > > > >> Thanks,
>>> >> > > > > >> > > > > >> St.Ack
>>> >> > > > > >> > > > > >>
>>> >> > > > > >> > > > > >>
>>> >> > > > > >> > > > > >>
>>> >> > > > > >> > > > > >> > Thanks
>>> >> > > > > >> > > > > >> >
>>> >> > > > > >> > > > > >> > On Tue, Sep 20, 2016 at 10:02 AM, Devaraj Das
>>> <
>>> >> > > > > >> > > ddas@hortonworks.com
>>> >> > > > > >> > > > >
>>> >> > > > > >> > > > > >> > wrote:
>>> >> > > > > >> > > > > >> >
>>> >> > > > > >> > > > > >> > > Just reviving this thread. Thanks Sean,
>>> Stack,
>>> >> > Dima,
>>> >> > > > and
>>> >> > > > > >> > others
>>> >> > > > > >> > > > for
>>> >> > > > > >> > > > > the
>>> >> > > > > >> > > > > >> > > thorough reviews and testing. Thanks Ted and
>>> >> Vlad
>>> >> > for
>>> >> > > > > >> taking
>>> >> > > > > >> > > care
>>> >> > > > > >> > > > of
>>> >> > > > > >> > > > > >> the
>>> >> > > > > >> > > > > >> > > feedback. Are we all good to do the merge
>>> now?
>>> >> > Rather
>>> >> > > > do
>>> >> > > > > >> > sooner
>>> >> > > > > >> > > > than
>>> >> > > > > >> > > > > >> > later.
>>> >> > > > > >> > > > > >> > > ________________________________________
>>> >> > > > > >> > > > > >> > > From: saint.ack@gmail.com <
>>> saint.ack@gmail.com>
>>> >> on
>>> >> > > > > behalf
>>> >> > > > > >> of
>>> >> > > > > >> > > > Stack
>>> >> > > > > >> > > > > <
>>> >> > > > > >> > > > > >> > > stack@duboce.net>
>>> >> > > > > >> > > > > >> > > Sent: Monday, September 12, 2016 1:18 PM
>>> >> > > > > >> > > > > >> > > To: HBase Dev List
>>> >> > > > > >> > > > > >> > > Subject: Re: [DISCUSSION] Merge Backup /
>>> >> Restore -
>>> >> > > > Branch
>>> >> > > > > >> > > > HBASE-7912
>>> >> > > > > >> > > > > >> > >
>>> >> > > > > >> > > > > >> > > On Mon, Sep 12, 2016 at 12:19 PM, Ted Yu <
>>> >> > > > > >> yuzhihong@gmail.com
>>> >> > > > > >> > >
>>> >> > > > > >> > > > > wrote:
>>> >> > > > > >> > > > > >> > >
>>> >> > > > > >> > > > > >> > > > Mega patch (rev 18) is on HBASE-14123.
>>> >> > > > > >> > > > > >> > > >
>>> >> > > > > >> > > > > >> > > > Please comment on HBASE-14123 on how you
>>> want
>>> >> to
>>> >> > > > > review.
>>> >> > > > > >> > > > > >> > > >
>>> >> > > > > >> > > > > >> > >
>>> >> > > > > >> > > > > >> > >
>>> >> > > > > >> > > > > >> > > Yeah. That was my lost tab. Last rb was 6
>>> months
>>> >> > ago.
>>> >> > > > > >> Suggest
>>> >> > > > > >> > > > > updating
>>> >> > > > > >> > > > > >> > it.
>>> >> > > > > >> > > > > >> > > RB is pretty good for review. Patch is only
>>> >> 1.5M so
>>> >> > > > > should
>>> >> > > > > >> be
>>> >> > > > > >> > > > fine.
>>> >> > > > > >> > > > > >> > >
>>> >> > > > > >> > > > > >> > > St.Ack
>>> >> > > > > >> > > > > >> > >
>>> >> > > > > >> > > > > >> > >
>>> >> > > > > >> > > > > >> > > >
>>> >> > > > > >> > > > > >> > > > Thanks
>>> >> > > > > >> > > > > >> > > >
>>> >> > > > > >> > > > > >> > > > On Mon, Sep 12, 2016 at 12:15 PM, Stack <
>>> >> > > > > >> stack@duboce.net>
>>> >> > > > > >> > > > wrote:
>>> >> > > > > >> > > > > >> > > >
>>> >> > > > > >> > > > > >> > > > > On review of the 'patch', do I just
>>> compare
>>> >> the
>>> >> > > > > branch
>>> >> > > > > >> to
>>> >> > > > > >> > > > > master or
>>> >> > > > > >> > > > > >> > is
>>> >> > > > > >> > > > > >> > > > > there a megapatch posted somewhere (I
>>> think
>>> >> I
>>> >> > saw
>>> >> > > > one
>>> >> > > > > >> but
>>> >> > > > > >> > it
>>> >> > > > > >> > > > > seemed
>>> >> > > > > >> > > > > >> > > stale
>>> >> > > > > >> > > > > >> > > > > and then I 'lost' the tab). Sorry for
>>> dumb
>>> >> > > > question.
>>> >> > > > > >> > > > > >> > > > > St.Ack
>>> >> > > > > >> > > > > >> > > > >
>>> >> > > > > >> > > > > >> > > > > On Mon, Sep 12, 2016 at 12:01 PM, Stack
>>> <
>>> >> > > > > >> stack@duboce.net
>>> >> > > > > >> > >
>>> >> > > > > >> > > > > wrote:
>>> >> > > > > >> > > > > >> > > > >
>>> >> > > > > >> > > > > >> > > > > > Late to the game. A few comments after
>>> >> > > rereading
>>> >> > > > > this
>>> >> > > > > >> > > thread
>>> >> > > > > >> > > > > as a
>>> >> > > > > >> > > > > >> > > > 'user'.
>>> >> > > > > >> > > > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > > > + Before merge, a user-facing feature
>>> like
>>> >> > this
>>> >> > > > > >> should
>>> >> > > > > >> > > work
>>> >> > > > > >> > > > > (If
>>> >> > > > > >> > > > > >> > this
>>> >> > > > > >> > > > > >> > > is
>>> >> > > > > >> > > > > >> > > > > "higher-bar
>>> >> > > > > >> > > > > >> > > > > > for new features", bring it on --
>>> smile).
>>> >> > > > > >> > > > > >> > > > > > + As a user, I tried the branch with
>>> tools
>>> >> > > after
>>> >> > > > > >> > reviewing
>>> >> > > > > >> > > > the
>>> >> > > > > >> > > > > >> > > > > just-posted
>>> >> > > > > >> > > > > >> > > > > > doc. I had an 'interesting' experience
>>> >> (left
>>> >> > > > > >> comments up
>>> >> > > > > >> > > on
>>> >> > > > > >> > > > > >> > issue). I
>>> >> > > > > >> > > > > >> > > > > think
>>> >> > > > > >> > > > > >> > > > > > the tooling/doc. important to get
>>> right.
>>> >> If
>>> >> > it
>>> >> > > > > breaks
>>> >> > > > > >> > > easily
>>> >> > > > > >> > > > > or
>>> >> > > > > >> > > > > >> is
>>> >> > > > > >> > > > > >> > > > > > inconsistent (or lacks 'polish'),
>>> >> operators
>>> >> > > will
>>> >> > > > > >> judge
>>> >> > > > > >> > the
>>> >> > > > > >> > > > > whole
>>> >> > > > > >> > > > > >> > > > > > backup/restore tooling chain as not
>>> >> > trustworthy
>>> >> > > > and
>>> >> > > > > >> > > abandon
>>> >> > > > > >> > > > > it.
>>> >> > > > > >> > > > > >> > Lets
>>> >> > > > > >> > > > > >> > > > not
>>> >> > > > > >> > > > > >> > > > > > have this happen to this feature.
>>> >> > > > > >> > > > > >> > > > > > + Matteo's suggestion (with a helpful
>>> >> starter
>>> >> > > > list)
>>> >> > > > > >> that
>>> >> > > > > >> > > > there
>>> >> > > > > >> > > > > >> > needs
>>> >> > > > > >> > > > > >> > > to
>>> >> > > > > >> > > > > >> > > > > be
>>> >> > > > > >> > > > > >> > > > > > explicit qualification on what is
>>> actually
>>> >> > > being
>>> >> > > > > >> > delivered
>>> >> > > > > >> > > > --
>>> >> > > > > >> > > > > >> > > > including a
>>> >> > > > > >> > > > > >> > > > > > listing of limitations (some look
>>> serious
>>> >> > such
>>> >> > > as
>>> >> > > > > >> data
>>> >> > > > > >> > > bleed
>>> >> > > > > >> > > > > from
>>> >> > > > > >> > > > > >> > > other
>>> >> > > > > >> > > > > >> > > > > > regions in WALs, but maybe I don't
>>> care
>>> >> for
>>> >> > my
>>> >> > > > use
>>> >> > > > > >> > > case...)
>>> >> > > > > >> > > > --
>>> >> > > > > >> > > > > >> > needs
>>> >> > > > > >> > > > > >> > > to
>>> >> > > > > >> > > > > >> > > > > > accompany the merge. Lets fold them
>>> into
>>> >> the
>>> >> > > user
>>> >> > > > > >> doc.
>>> >> > > > > >> > in
>>> >> > > > > >> > > > the
>>> >> > > > > >> > > > > >> > > technical
>>> >> > > > > >> > > > > >> > > > > > overview area as suggested so user
>>> >> > expectations
>>> >> > > > are
>>> >> > > > > >> > > properly
>>> >> > > > > >> > > > > >> > managed
>>> >> > > > > >> > > > > >> > > > > > (otherwise, they expect the world and
>>> will
>>> >> > just
>>> >> > > > > give
>>> >> > > > > >> up
>>> >> > > > > >> > > when
>>> >> > > > > >> > > > > we
>>> >> > > > > >> > > > > >> > fall
>>> >> > > > > >> > > > > >> > > > > > short). Vladimir did a list of what
>>> is in
>>> >> > each
>>> >> > > of
>>> >> > > > > the
>>> >> > > > > >> > > phases
>>> >> > > > > >> > > > > >> above
>>> >> > > > > >> > > > > >> > > > which
>>> >> > > > > >> > > > > >> > > > > > would serve as a good start.
>>> >> > > > > >> > > > > >> > > > > > + Is this feature 'experimental'
>>> (Matteo
>>> >> asks
>>> >> > > > > above).
>>> >> > > > > >> > I'd
>>> >> > > > > >> > > > > prefer
>>> >> > > > > >> > > > > >> it
>>> >> > > > > >> > > > > >> > > is
>>> >> > > > > >> > > > > >> > > > > > not. If it is, it should be labelled
>>> all
>>> >> over
>>> >> > > > that
>>> >> > > > > >> it is
>>> >> > > > > >> > > > so. I
>>> >> > > > > >> > > > > >> see
>>> >> > > > > >> > > > > >> > > > > current
>>> >> > > > > >> > > > > >> > > > > > state called out as a '... technical
>>> >> preview
>>> >> > > > > >> feature'.
>>> >> > > > > >> > > Does
>>> >> > > > > >> > > > > this
>>> >> > > > > >> > > > > >> > mean
>>> >> > > > > >> > > > > >> > > > > > not-for-users?
>>> >> > > > > >> > > > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > > > St.Ack
>>> >> > > > > >> > > > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > > > On Mon, Sep 12, 2016 at 8:03 AM, Ted
>>> Yu <
>>> >> > > > > >> > > > yuzhihong@gmail.com>
>>> >> > > > > >> > > > > >> > wrote:
>>> >> > > > > >> > > > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > > >> Sean:
>>> >> > > > > >> > > > > >> > > > > >> Do you have more comments ?
>>> >> > > > > >> > > > > >> > > > > >>
>>> >> > > > > >> > > > > >> > > > > >> Cheers
>>> >> > > > > >> > > > > >> > > > > >>
>>> >> > > > > >> > > > > >> > > > > >> On Fri, Sep 9, 2016 at 1:42 PM,
>>> Vladimir
>>> >> > > > Rodionov
>>> >> > > > > <
>>> >> > > > > >> > > > > >> > > > > vladrodionov@gmail.com
>>> >> > > > > >> > > > > >> > > > > >> >
>>> >> > > > > >> > > > > >> > > > > >> wrote:
>>> >> > > > > >> > > > > >> > > > > >>
>>> >> > > > > >> > > > > >> > > > > >> > Sean,
>>> >> > > > > >> > > > > >> > > > > >> >
>>> >> > > > > >> > > > > >> > > > > >> > Backup/Restore can fail due to
>>> various
>>> >> > > > reasons:
>>> >> > > > > >> > network
>>> >> > > > > >> > > > > outage
>>> >> > > > > >> > > > > >> > > > > (cluster
>>> >> > > > > >> > > > > >> > > > > >> > wide), various time-outs in HBase
>>> and
>>> >> HDFS
>>> >> > > > > layer,
>>> >> > > > > >> M/R
>>> >> > > > > >> > > > > failure
>>> >> > > > > >> > > > > >> > due
>>> >> > > > > >> > > > > >> > > to
>>> >> > > > > >> > > > > >> > > > > >> "HDFS
>>> >> > > > > >> > > > > >> > > > > >> > exceeded quota", user error (manual
>>> >> > deletion
>>> >> > > > of
>>> >> > > > > >> data)
>>> >> > > > > >> > > and
>>> >> > > > > >> > > > > so
>>> >> > > > > >> > > > > >> on
>>> >> > > > > >> > > > > >> > so
>>> >> > > > > >> > > > > >> > > > on.
>>> >> > > > > >> > > > > >> > > > > >> That
>>> >> > > > > >> > > > > >> > > > > >> > is impossible to enumerate all
>>> possible
>>> >> > > types
>>> >> > > > of
>>> >> > > > > >> > > failures
>>> >> > > > > >> > > > > in a
>>> >> > > > > >> > > > > >> > > > > >> distributed
>>> >> > > > > >> > > > > >> > > > > >> > system - that is not our goal/task.
>>> >> > > > > >> > > > > >> > > > > >> >
>>> >> > > > > >> > > > > >> > > > > >> > We focus completely on backup
>>> system
>>> >> table
>>> >> > > > > >> > consistency
>>> >> > > > > >> > > > in a
>>> >> > > > > >> > > > > >> > > presence
>>> >> > > > > >> > > > > >> > > > > of
>>> >> > > > > >> > > > > >> > > > > >> any
>>> >> > > > > >> > > > > >> > > > > >> > type of failure. That is what I
>>> call
>>> >> > > > "tolerance
>>> >> > > > > to
>>> >> > > > > >> > > > > failures".
>>> >> > > > > >> > > > > >> > > > > >> >
>>> >> > > > > >> > > > > >> > > > > >> > On a failure:
>>> >> > > > > >> > > > > >> > > > > >> >
>>> >> > > > > >> > > > > >> > > > > >> > BACKUP. All backup system
>>> information
>>> >> > (prior
>>> >> > > > to
>>> >> > > > > >> > backup)
>>> >> > > > > >> > > > > will
>>> >> > > > > >> > > > > >> be
>>> >> > > > > >> > > > > >> > > > > restored
>>> >> > > > > >> > > > > >> > > > > >> > and all temporary data, related to
>>> a
>>> >> > failed
>>> >> > > > > >> session,
>>> >> > > > > >> > in
>>> >> > > > > >> > > > > HDFS
>>> >> > > > > >> > > > > >> > will
>>> >> > > > > >> > > > > >> > > be
>>> >> > > > > >> > > > > >> > > > > >> > deleted
>>> >> > > > > >> > > > > >> > > > > >> > RESTORE. We do not care about
>>> system
>>> >> data,
>>> >> > > > > because
>>> >> > > > > >> > > > restore
>>> >> > > > > >> > > > > >> does
>>> >> > > > > >> > > > > >> > > not
>>> >> > > > > >> > > > > >> > > > > >> change
>>> >> > > > > >> > > > > >> > > > > >> > it. Temporary data in HDFS will be
>>> >> cleaned
>>> >> > > up
>>> >> > > > > and
>>> >> > > > > >> > table
>>> >> > > > > >> > > > > will
>>> >> > > > > >> > > > > >> be
>>> >> > > > > >> > > > > >> > > in a
>>> >> > > > > >> > > > > >> > > > > >> state
>>> >> > > > > >> > > > > >> > > > > >> > back to where it was before
>>> operation
>>> >> > > started.
>>> >> > > > > >> > > > > >> > > > > >> >
>>> >> > > > > >> > > > > >> > > > > >> > This is what user should expect in
>>> case
>>> >> > of a
>>> >> > > > > >> failure.
>>> >> > > > > >> > > > > >> > > > > >> >
>>> >> > > > > >> > > > > >> > > > > >> > -Vlad
>>> >> > > > > >> > > > > >> > > > > >> >
>>> >> > > > > >> > > > > >> > > > > >> >
>>> >> > > > > >> > > > > >> > > > > >> > -Vlad
>>> >> > > > > >> > > > > >> > > > > >> >
>>> >> > > > > >> > > > > >> > > > > >> > On Fri, Sep 9, 2016 at 12:56 PM,
>>> Sean
>>> >> > > Busbey <
>>> >> > > > > >> > > > > >> busbey@apache.org
>>> >> > > > > >> > > > > >> > >
>>> >> > > > > >> > > > > >> > > > > wrote:
>>> >> > > > > >> > > > > >> > > > > >> >
>>> >> > > > > >> > > > > >> > > > > >> > > Failing in a consistent way, with
>>> >> docs
>>> >> > > that
>>> >> > > > > >> explain
>>> >> > > > > >> > > the
>>> >> > > > > >> > > > > >> > various
>>> >> > > > > >> > > > > >> > > > > >> > > expected failures would be
>>> >> sufficient.
>>> >> > > > > >> > > > > >> > > > > >> > >
>>> >> > > > > >> > > > > >> > > > > >> > > On Fri, Sep 9, 2016 at 12:16 PM,
>>> >> > Vladimir
>>> >> > > > > >> Rodionov
>>> >> > > > > >> > > > > >> > > > > >> > > <vl...@gmail.com> wrote:
>>> >> > > > > >> > > > > >> > > > > >> > > > Do not worry Sean, doc is
>>> coming
>>> >> today
>>> >> > > as
>>> >> > > > a
>>> >> > > > > >> > preview
>>> >> > > > > >> > > > and
>>> >> > > > > >> > > > > >> our
>>> >> > > > > >> > > > > >> > > > writer
>>> >> > > > > >> > > > > >> > > > > >> > Frank
>>> >> > > > > >> > > > > >> > > > > >> > > > will be working on a putting
>>> it
>>> >> into
>>> >> > > > Apache
>>> >> > > > > >> > repo.
>>> >> > > > > >> > > > > >> Timeline
>>> >> > > > > >> > > > > >> > > > > depends
>>> >> > > > > >> > > > > >> > > > > >> on
>>> >> > > > > >> > > > > >> > > > > >> > > > Franks schedule but I hope we
>>> will
>>> >> get
>>> >> > > it
>>> >> > > > > >> rather
>>> >> > > > > >> > > > sooner
>>> >> > > > > >> > > > > >> than
>>> >> > > > > >> > > > > >> > > > > later.
>>> >> > > > > >> > > > > >> > > > > >> > > >
>>> >> > > > > >> > > > > >> > > > > >> > > > As for failure testing, we are
>>> >> > focusing
>>> >> > > > only
>>> >> > > > > >> on a
>>> >> > > > > >> > > > > >> consistent
>>> >> > > > > >> > > > > >> > > > state
>>> >> > > > > >> > > > > >> > > > > >> of
>>> >> > > > > >> > > > > >> > > > > >> > > > backup system data in a
>>> presence of
>>> >> > any
>>> >> > > > type
>>> >> > > > > >> of
>>> >> > > > > >> > > > > failures,
>>> >> > > > > >> > > > > >> We
>>> >> > > > > >> > > > > >> > > are
>>> >> > > > > >> > > > > >> > > > > not
>>> >> > > > > >> > > > > >> > > > > >> > > going
>>> >> > > > > >> > > > > >> > > > > >> > > > to implement  anything more
>>> >> "fancy",
>>> >> > > than
>>> >> > > > > >> that.
>>> >> > > > > >> > We
>>> >> > > > > >> > > > > allow
>>> >> > > > > >> > > > > >> > both:
>>> >> > > > > >> > > > > >> > > > > >> backup
>>> >> > > > > >> > > > > >> > > > > >> > and
>>> >> > > > > >> > > > > >> > > > > >> > > > restore to fail. What we do not
>>> >> allow
>>> >> > is
>>> >> > > > to
>>> >> > > > > >> have
>>> >> > > > > >> > > > system
>>> >> > > > > >> > > > > >> data
>>> >> > > > > >> > > > > >> > > > > >> corrupted.
>>> >> > > > > >> > > > > >> > > > > >> > > > Will it suffice for you? Do you
>>> >> have
>>> >> > any
>>> >> > > > > other
>>> >> > > > > >> > > > > concerns,
>>> >> > > > > >> > > > > >> you
>>> >> > > > > >> > > > > >> > > > want
>>> >> > > > > >> > > > > >> > > > > >> us to
>>> >> > > > > >> > > > > >> > > > > >> > > > address?
>>> >> > > > > >> > > > > >> > > > > >> > > >
>>> >> > > > > >> > > > > >> > > > > >> > > > -Vlad
>>> >> > > > > >> > > > > >> > > > > >> > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >
>>> >> > > > > >> > > > > >> > > > > >> > > > On Fri, Sep 9, 2016 at 10:56
>>> AM,
>>> >> Sean
>>> >> > > > > Busbey <
>>> >> > > > > >> > > > > >> > > busbey@apache.org
>>> >> > > > > >> > > > > >> > > > >
>>> >> > > > > >> > > > > >> > > > > >> > wrote:
>>> >> > > > > >> > > > > >> > > > > >> > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> "docs will come to Apache
>>> soon"
>>> >> does
>>> >> > > not
>>> >> > > > > >> address
>>> >> > > > > >> > > my
>>> >> > > > > >> > > > > >> concern
>>> >> > > > > >> > > > > >> > > > > around
>>> >> > > > > >> > > > > >> > > > > >> > docs
>>> >> > > > > >> > > > > >> > > > > >> > > at
>>> >> > > > > >> > > > > >> > > > > >> > > >> all, unless said docs have
>>> already
>>> >> > made
>>> >> > > > it
>>> >> > > > > >> into
>>> >> > > > > >> > > the
>>> >> > > > > >> > > > > >> project
>>> >> > > > > >> > > > > >> > > > > repo. I
>>> >> > > > > >> > > > > >> > > > > >> > > don't
>>> >> > > > > >> > > > > >> > > > > >> > > >> want third party resources for
>>> >> using
>>> >> > a
>>> >> > > > > major
>>> >> > > > > >> and
>>> >> > > > > >> > > > > >> important
>>> >> > > > > >> > > > > >> > > > > feature
>>> >> > > > > >> > > > > >> > > > > >> of
>>> >> > > > > >> > > > > >> > > > > >> > > the
>>> >> > > > > >> > > > > >> > > > > >> > > >> project, I want us to provide
>>> end
>>> >> > users
>>> >> > > > > with
>>> >> > > > > >> > what
>>> >> > > > > >> > > > they
>>> >> > > > > >> > > > > >> need
>>> >> > > > > >> > > > > >> > > to
>>> >> > > > > >> > > > > >> > > > > get
>>> >> > > > > >> > > > > >> > > > > >> the
>>> >> > > > > >> > > > > >> > > > > >> > > job
>>> >> > > > > >> > > > > >> > > > > >> > > >> done.
>>> >> > > > > >> > > > > >> > > > > >> > > >>
>>> >> > > > > >> > > > > >> > > > > >> > > >> I see some calls for patience
>>> on
>>> >> the
>>> >> > > > > failure
>>> >> > > > > >> > > > testing,
>>> >> > > > > >> > > > > but
>>> >> > > > > >> > > > > >> > the
>>> >> > > > > >> > > > > >> > > > > >> appeal
>>> >> > > > > >> > > > > >> > > > > >> > to
>>> >> > > > > >> > > > > >> > > > > >> > > us
>>> >> > > > > >> > > > > >> > > > > >> > > >> having done a bad job of
>>> requiring
>>> >> > > proper
>>> >> > > > > >> tests
>>> >> > > > > >> > of
>>> >> > > > > >> > > > > >> previous
>>> >> > > > > >> > > > > >> > > > > >> features
>>> >> > > > > >> > > > > >> > > > > >> > > just
>>> >> > > > > >> > > > > >> > > > > >> > > >> makes me more concerned about
>>> not
>>> >> > > getting
>>> >> > > > > >> them
>>> >> > > > > >> > > > here. I
>>> >> > > > > >> > > > > >> > don't
>>> >> > > > > >> > > > > >> > > > want
>>> >> > > > > >> > > > > >> > > > > >> to
>>> >> > > > > >> > > > > >> > > > > >> > set
>>> >> > > > > >> > > > > >> > > > > >> > > >> yet another bad example that
>>> will
>>> >> > then
>>> >> > > be
>>> >> > > > > >> > pointed
>>> >> > > > > >> > > to
>>> >> > > > > >> > > > > in
>>> >> > > > > >> > > > > >> the
>>> >> > > > > >> > > > > >> > > > > future.
>>> >> > > > > >> > > > > >> > > > > >> > > >>
>>> >> > > > > >> > > > > >> > > > > >> > > >> On Sep 8, 2016 10:50, "Ted
>>> Yu" <
>>> >> > > > > >> > > yuzhihong@gmail.com
>>> >> > > > > >> > > > >
>>> >> > > > > >> > > > > >> > wrote:
>>> >> > > > > >> > > > > >> > > > > >> > > >>
>>> >> > > > > >> > > > > >> > > > > >> > > >> > Is there any concern which
>>> is
>>> >> not
>>> >> > > > > >> addressed ?
>>> >> > > > > >> > > > > >> > > > > >> > > >> >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > Do we need another Vote
>>> thread ?
>>> >> > > > > >> > > > > >> > > > > >> > > >> >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > Thanks
>>> >> > > > > >> > > > > >> > > > > >> > > >> >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > On Thu, Sep 8, 2016 at 9:21
>>> AM,
>>> >> > > Andrew
>>> >> > > > > >> > Purtell <
>>> >> > > > > >> > > > > >> > > > > >> apurtell@apache.org
>>> >> > > > > >> > > > > >> > > > > >> > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > wrote:
>>> >> > > > > >> > > > > >> > > > > >> > > >> >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > Vlad,
>>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > I apologize for using the
>>> term
>>> >> > > > > >> 'half-baked'
>>> >> > > > > >> > > in a
>>> >> > > > > >> > > > > way
>>> >> > > > > >> > > > > >> > that
>>> >> > > > > >> > > > > >> > > > > could
>>> >> > > > > >> > > > > >> > > > > >> > > seem a
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > description of
>>> HBASE-7912. I
>>> >> > meant
>>> >> > > > that
>>> >> > > > > >> as a
>>> >> > > > > >> > > > > general
>>> >> > > > > >> > > > > >> > > > > >> hypothetical.
>>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > On Wed, Sep 7, 2016 at
>>> 9:36
>>> >> AM,
>>> >> > > > > Vladimir
>>> >> > > > > >> > > > Rodionov
>>> >> > > > > >> > > > > <
>>> >> > > > > >> > > > > >> > > > > >> > > >> > vladrodionov@gmail.com>
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > wrote:
>>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >> I'm not sure that
>>> "There
>>> >> is
>>> >> > > > > already
>>> >> > > > > >> > lots
>>> >> > > > > >> > > of
>>> >> > > > > >> > > > > >> > > half-baked
>>> >> > > > > >> > > > > >> > > > > >> code
>>> >> > > > > >> > > > > >> > > > > >> > in
>>> >> > > > > >> > > > > >> > > > > >> > > the
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > branch,
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > so what's the harm in
>>> adding
>>> >> > > more?"
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > I meant - not
>>> production -
>>> >> > ready
>>> >> > > > yet.
>>> >> > > > > >> This
>>> >> > > > > >> > > is
>>> >> > > > > >> > > > > 2.0
>>> >> > > > > >> > > > > >> > > > > development
>>> >> > > > > >> > > > > >> > > > > >> > > branch
>>> >> > > > > >> > > > > >> > > > > >> > > >> > and,
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > hence many features are
>>> in
>>> >> > works,
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > not being tested well
>>> etc.
>>> >> I do
>>> >> > > not
>>> >> > > > > >> > consider
>>> >> > > > > >> > > > > backup
>>> >> > > > > >> > > > > >> > as
>>> >> > > > > >> > > > > >> > > > half
>>> >> > > > > >> > > > > >> > > > > >> > baked
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > feature -
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > it has passed our
>>> internal
>>> >> QA
>>> >> > and
>>> >> > > > has
>>> >> > > > > >> very
>>> >> > > > > >> > > > good
>>> >> > > > > >> > > > > >> doc,
>>> >> > > > > >> > > > > >> > > > which
>>> >> > > > > >> > > > > >> > > > > we
>>> >> > > > > >> > > > > >> > > > > >> > will
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > provide
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > to Apache shortly.
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > -Vlad
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > On Wed, Sep 7, 2016 at
>>> 9:13
>>> >> AM,
>>> >> > > > > Andrew
>>> >> > > > > >> > > > Purtell <
>>> >> > > > > >> > > > > >> > > > > >> > > apurtell@apache.org>
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > We shouldn't admit
>>> half
>>> >> baked
>>> >> > > > > changes
>>> >> > > > > >> > that
>>> >> > > > > >> > > > > won't
>>> >> > > > > >> > > > > >> be
>>> >> > > > > >> > > > > >> > > > > >> finished.
>>> >> > > > > >> > > > > >> > > > > >> > > >> However
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > in
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > this case the crew
>>> >> working on
>>> >> > > > this
>>> >> > > > > >> > feature
>>> >> > > > > >> > > > are
>>> >> > > > > >> > > > > >> long
>>> >> > > > > >> > > > > >> > > > > timers
>>> >> > > > > >> > > > > >> > > > > >> and
>>> >> > > > > >> > > > > >> > > > > >> > > less
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > likely
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > than just about
>>> anyone to
>>> >> > leave
>>> >> > > > > >> > something
>>> >> > > > > >> > > > in a
>>> >> > > > > >> > > > > >> half
>>> >> > > > > >> > > > > >> > > > baked
>>> >> > > > > >> > > > > >> > > > > >> > > state. Of
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > course
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > there is no guarantee
>>> how
>>> >> > > > anything
>>> >> > > > > >> will
>>> >> > > > > >> > > turn
>>> >> > > > > >> > > > > out,
>>> >> > > > > >> > > > > >> > > but I
>>> >> > > > > >> > > > > >> > > > > am
>>> >> > > > > >> > > > > >> > > > > >> > > willing
>>> >> > > > > >> > > > > >> > > > > >> > > >> to
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > take
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > a little on faith if
>>> they
>>> >> > feel
>>> >> > > > > their
>>> >> > > > > >> > best
>>> >> > > > > >> > > > path
>>> >> > > > > >> > > > > >> > > forward
>>> >> > > > > >> > > > > >> > > > > now
>>> >> > > > > >> > > > > >> > > > > >> is
>>> >> > > > > >> > > > > >> > > > > >> > to
>>> >> > > > > >> > > > > >> > > > > >> > > >> > merge
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > to
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > trunk. I only wish I
>>> had
>>> >> > > > bandwidth
>>> >> > > > > to
>>> >> > > > > >> > have
>>> >> > > > > >> > > > > done
>>> >> > > > > >> > > > > >> > some
>>> >> > > > > >> > > > > >> > > > real
>>> >> > > > > >> > > > > >> > > > > >> > > kicking
>>> >> > > > > >> > > > > >> > > > > >> > > >> of
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > the
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > tires by now. Maybe
>>> this
>>> >> > week.
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > (Yes, I'm using some
>>> of
>>> >> that
>>> >> > > time
>>> >> > > > > for
>>> >> > > > > >> > this
>>> >> > > > > >> > > > > email
>>> >> > > > > >> > > > > >> > :-)
>>> >> > > > > >> > > > > >> > > > but
>>> >> > > > > >> > > > > >> > > > > I
>>> >> > > > > >> > > > > >> > > > > >> > type
>>> >> > > > > >> > > > > >> > > > > >> > > >> > fast.)
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > That said, I would
>>> like to
>>> >> > > > agitate
>>> >> > > > > >> for
>>> >> > > > > >> > > > making
>>> >> > > > > >> > > > > 2.0
>>> >> > > > > >> > > > > >> > > more
>>> >> > > > > >> > > > > >> > > > > real
>>> >> > > > > >> > > > > >> > > > > >> > and
>>> >> > > > > >> > > > > >> > > > > >> > > >> spend
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > some
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > time on it now that
>>> I'm
>>> >> > winding
>>> >> > > > > down
>>> >> > > > > >> > with
>>> >> > > > > >> > > > > 0.98. I
>>> >> > > > > >> > > > > >> > > think
>>> >> > > > > >> > > > > >> > > > > >> that
>>> >> > > > > >> > > > > >> > > > > >> > > means
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branching for 2.0 real
>>> >> soon
>>> >> > now
>>> >> > > > and
>>> >> > > > > >> even
>>> >> > > > > >> > > > > evicting
>>> >> > > > > >> > > > > >> > > > things
>>> >> > > > > >> > > > > >> > > > > >> from
>>> >> > > > > >> > > > > >> > > > > >> > > 2.0
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > branch
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > that aren't finished
>>> or
>>> >> > stable,
>>> >> > > > > >> leaving
>>> >> > > > > >> > > them
>>> >> > > > > >> > > > > only
>>> >> > > > > >> > > > > >> > > once
>>> >> > > > > >> > > > > >> > > > > >> again
>>> >> > > > > >> > > > > >> > > > > >> > in
>>> >> > > > > >> > > > > >> > > > > >> > > the
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > master
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branch. Or, maybe just
>>> >> > evicting
>>> >> > > > > them.
>>> >> > > > > >> > > Let's
>>> >> > > > > >> > > > > take
>>> >> > > > > >> > > > > >> it
>>> >> > > > > >> > > > > >> > > > case
>>> >> > > > > >> > > > > >> > > > > by
>>> >> > > > > >> > > > > >> > > > > >> > > case.
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > I think this feature
>>> can
>>> >> come
>>> >> > > in
>>> >> > > > > >> > > relatively
>>> >> > > > > >> > > > > >> safely.
>>> >> > > > > >> > > > > >> > > As
>>> >> > > > > >> > > > > >> > > > > >> added
>>> >> > > > > >> > > > > >> > > > > >> > > >> > insurance,
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > let's admit the
>>> >> possibility
>>> >> > it
>>> >> > > > > could
>>> >> > > > > >> be
>>> >> > > > > >> > > > > reverted
>>> >> > > > > >> > > > > >> on
>>> >> > > > > >> > > > > >> > > the
>>> >> > > > > >> > > > > >> > > > > 2.0
>>> >> > > > > >> > > > > >> > > > > >> > > branch
>>> >> > > > > >> > > > > >> > > > > >> > > >> if
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > folks
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > working on
>>> stabilizing 2.0
>>> >> > > decide
>>> >> > > > > to
>>> >> > > > > >> > evict
>>> >> > > > > >> > > > it
>>> >> > > > > >> > > > > >> > because
>>> >> > > > > >> > > > > >> > > > it
>>> >> > > > > >> > > > > >> > > > > is
>>> >> > > > > >> > > > > >> > > > > >> > > >> > unfinished
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > or
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > unstable, because that
>>> >> > > certainly
>>> >> > > > > can
>>> >> > > > > >> > > > happen. I
>>> >> > > > > >> > > > > >> > would
>>> >> > > > > >> > > > > >> > > > > >> expect if
>>> >> > > > > >> > > > > >> > > > > >> > > talk
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > like
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > that starts, we'd get
>>> help
>>> >> > > > > finishing
>>> >> > > > > >> or
>>> >> > > > > >> > > > > >> stabilizing
>>> >> > > > > >> > > > > >> > > > > what's
>>> >> > > > > >> > > > > >> > > > > >> > under
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > discussion
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > for revert. Or, we'd
>>> have
>>> >> a
>>> >> > > > revert.
>>> >> > > > > >> > Either
>>> >> > > > > >> > > > way
>>> >> > > > > >> > > > > >> the
>>> >> > > > > >> > > > > >> > > > > outcome
>>> >> > > > > >> > > > > >> > > > > >> is
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > acceptable.
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > On Wed, Sep 7, 2016 at
>>> >> 8:56
>>> >> > AM,
>>> >> > > > > Dima
>>> >> > > > > >> > > Spivak
>>> >> > > > > >> > > > <
>>> >> > > > > >> > > > > >> > > > > >> > > dimaspivak@apache.org
>>> >> > > > > >> > > > > >> > > > > >> > > >> >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > I'm not sure that
>>> >> "There is
>>> >> > > > > already
>>> >> > > > > >> > lots
>>> >> > > > > >> > > > of
>>> >> > > > > >> > > > > >> > > > half-baked
>>> >> > > > > >> > > > > >> > > > > >> code
>>> >> > > > > >> > > > > >> > > > > >> > in
>>> >> > > > > >> > > > > >> > > > > >> > > >> the
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branch,
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > so what's the harm
>>> in
>>> >> > adding
>>> >> > > > > more?"
>>> >> > > > > >> > is a
>>> >> > > > > >> > > > > good
>>> >> > > > > >> > > > > >> > code
>>> >> > > > > >> > > > > >> > > > > commit
>>> >> > > > > >> > > > > >> > > > > >> > > >> > philosophy
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > for
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > a
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > fault-tolerant
>>> >> distributed
>>> >> > > data
>>> >> > > > > >> store.
>>> >> > > > > >> > > ;)
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > More seriously, a
>>> lack
>>> >> of
>>> >> > > test
>>> >> > > > > >> > coverage
>>> >> > > > > >> > > > for
>>> >> > > > > >> > > > > >> > > existing
>>> >> > > > > >> > > > > >> > > > > >> > features
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > shouldn't
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > be
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > used as
>>> justification
>>> >> for
>>> >> > > > > >> introducing
>>> >> > > > > >> > > new
>>> >> > > > > >> > > > > >> > features
>>> >> > > > > >> > > > > >> > > > with
>>> >> > > > > >> > > > > >> > > > > >> the
>>> >> > > > > >> > > > > >> > > > > >> > > same
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > shortcomings.
>>> >> Ultimately,
>>> >> > > it's
>>> >> > > > > the
>>> >> > > > > >> end
>>> >> > > > > >> > > > user
>>> >> > > > > >> > > > > who
>>> >> > > > > >> > > > > >> > > will
>>> >> > > > > >> > > > > >> > > > > feel
>>> >> > > > > >> > > > > >> > > > > >> > the
>>> >> > > > > >> > > > > >> > > > > >> > > >> pain,
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > so
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > shouldn't we do
>>> >> everything
>>> >> > we
>>> >> > > > can
>>> >> > > > > >> to
>>> >> > > > > >> > > > > mitigate
>>> >> > > > > >> > > > > >> > that?
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > -Dima
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > On Wed, Sep 7, 2016
>>> at
>>> >> 8:46
>>> >> > > AM,
>>> >> > > > > >> > Vladimir
>>> >> > > > > >> > > > > >> > Rodionov <
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>> vladrodionov@gmail.com>
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > wrote:
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Sean,
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have docs
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Agree. We have a
>>> doc
>>> >> and
>>> >> > > > backup
>>> >> > > > > >> is
>>> >> > > > > >> > the
>>> >> > > > > >> > > > > most
>>> >> > > > > >> > > > > >> > > > > documented
>>> >> > > > > >> > > > > >> > > > > >> > > feature
>>> >> > > > > >> > > > > >> > > > > >> > > >> > :),
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > we
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > will
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > release it
>>> shortly to
>>> >> > > Apache.
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have sunny-day
>>> >> > > correctness
>>> >> > > > > >> tests
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Feature has
>>> close to
>>> >> 60
>>> >> > > test
>>> >> > > > > >> cases,
>>> >> > > > > >> > > > which
>>> >> > > > > >> > > > > >> run
>>> >> > > > > >> > > > > >> > > for
>>> >> > > > > >> > > > > >> > > > > >> approx
>>> >> > > > > >> > > > > >> > > > > >> > 30
>>> >> > > > > >> > > > > >> > > > > >> > > >> min.
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > We
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > can
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > add more, if
>>> >> community do
>>> >> > > not
>>> >> > > > > >> mind
>>> >> > > > > >> > :)
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have
>>> >> > > > > >> > correctness-in-face-of-failure
>>> >> > > > > >> > > > > tests
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Any examples of
>>> these
>>> >> > tests
>>> >> > > > in
>>> >> > > > > >> > > existing
>>> >> > > > > >> > > > > >> > features?
>>> >> > > > > >> > > > > >> > > > In
>>> >> > > > > >> > > > > >> > > > > >> > works,
>>> >> > > > > >> > > > > >> > > > > >> > > we
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > have a
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > clear
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > understanding of
>>> what
>>> >> > > should
>>> >> > > > be
>>> >> > > > > >> done
>>> >> > > > > >> > > by
>>> >> > > > > >> > > > > the
>>> >> > > > > >> > > > > >> > time
>>> >> > > > > >> > > > > >> > > of
>>> >> > > > > >> > > > > >> > > > > 2.0
>>> >> > > > > >> > > > > >> > > > > >> > > >> release.
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > That is very close
>>> >> goal
>>> >> > for
>>> >> > > > us,
>>> >> > > > > >> to
>>> >> > > > > >> > > > verify
>>> >> > > > > >> > > > > IT
>>> >> > > > > >> > > > > >> > > monkey
>>> >> > > > > >> > > > > >> > > > > for
>>> >> > > > > >> > > > > >> > > > > >> > > >> existing
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > code.
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * don't rely on
>>> things
>>> >> > > > outside
>>> >> > > > > of
>>> >> > > > > >> > > HBase
>>> >> > > > > >> > > > > for
>>> >> > > > > >> > > > > >> > > normal
>>> >> > > > > >> > > > > >> > > > > >> > operation
>>> >> > > > > >> > > > > >> > > > > >> > > >> > (okay
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > for
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > advanced
>>> operation)
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > We do not.
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Enormous time has
>>> been
>>> >> > > spent
>>> >> > > > > >> already
>>> >> > > > > >> > > on
>>> >> > > > > >> > > > > the
>>> >> > > > > >> > > > > >> > > > > development
>>> >> > > > > >> > > > > >> > > > > >> > and
>>> >> > > > > >> > > > > >> > > > > >> > > >> > testing
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > the
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > feature, it has
>>> passed
>>> >> > our
>>> >> > > > > >> internal
>>> >> > > > > >> > > > tests
>>> >> > > > > >> > > > > and
>>> >> > > > > >> > > > > >> > > many
>>> >> > > > > >> > > > > >> > > > > >> rounds
>>> >> > > > > >> > > > > >> > > > > >> > of
>>> >> > > > > >> > > > > >> > > > > >> > > >> code
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > reviews
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > by HBase
>>> committers.
>>> >> We
>>> >> > do
>>> >> > > > not
>>> >> > > > > >> mind
>>> >> > > > > >> > if
>>> >> > > > > >> > > > > >> someone
>>> >> > > > > >> > > > > >> > > from
>>> >> > > > > >> > > > > >> > > > > >> HBase
>>> >> > > > > >> > > > > >> > > > > >> > > >> > community
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > (outside of HW)
>>> will
>>> >> > review
>>> >> > > > the
>>> >> > > > > >> > code,
>>> >> > > > > >> > > > but
>>> >> > > > > >> > > > > it
>>> >> > > > > >> > > > > >> > will
>>> >> > > > > >> > > > > >> > > > > >> probably
>>> >> > > > > >> > > > > >> > > > > >> > > >> takes
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > forever
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > to
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > wait for
>>> volunteer?,
>>> >> the
>>> >> > > > > feature
>>> >> > > > > >> is
>>> >> > > > > >> > > > quite
>>> >> > > > > >> > > > > >> large
>>> >> > > > > >> > > > > >> > > > (1MB+
>>> >> > > > > >> > > > > >> > > > > >> > > >> cumulative
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > patch)
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > 2.0 branch is
>>> full of
>>> >> > half
>>> >> > > > > baked
>>> >> > > > > >> > > > features,
>>> >> > > > > >> > > > > >> most
>>> >> > > > > >> > > > > >> > > of
>>> >> > > > > >> > > > > >> > > > > them
>>> >> > > > > >> > > > > >> > > > > >> > are
>>> >> > > > > >> > > > > >> > > > > >> > > in
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > active
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > development,
>>> >> therefore I
>>> >> > am
>>> >> > > > not
>>> >> > > > > >> > > > following
>>> >> > > > > >> > > > > you
>>> >> > > > > >> > > > > >> > > here,
>>> >> > > > > >> > > > > >> > > > > >> Sean?
>>> >> > > > > >> > > > > >> > > > > >> > > Why
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > HBASE-7912
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > is
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > not good enough
>>> yet
>>> >> to be
>>> >> > > > > >> integrated
>>> >> > > > > >> > > > into
>>> >> > > > > >> > > > > 2.0
>>> >> > > > > >> > > > > >> > > > branch?
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > -Vlad
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > On Wed, Sep 7,
>>> 2016 at
>>> >> > 8:23
>>> >> > > > AM,
>>> >> > > > > >> Sean
>>> >> > > > > >> > > > > Busbey <
>>> >> > > > > >> > > > > >> > > > > >> > > busbey@apache.org
>>> >> > > > > >> > > > > >> > > > > >> > > >> >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > On Tue, Sep 6,
>>> 2016
>>> >> at
>>> >> > > > 10:36
>>> >> > > > > >> PM,
>>> >> > > > > >> > > Josh
>>> >> > > > > >> > > > > >> Elser <
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > josh.elser@gmail.com>
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > wrote:
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > So, the
>>> answer to
>>> >> > > Sean's
>>> >> > > > > >> > original
>>> >> > > > > >> > > > > >> question
>>> >> > > > > >> > > > > >> > is
>>> >> > > > > >> > > > > >> > > > "as
>>> >> > > > > >> > > > > >> > > > > >> > > robust as
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > snapshots
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > presently
>>> are"?
>>> >> > > > > >> (independence of
>>> >> > > > > >> > > > > >> > > backup/restore
>>> >> > > > > >> > > > > >> > > > > >> > failure
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > tolerance
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > from
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > snapshot
>>> failure
>>> >> > > > tolerance)
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > Is this just a
>>> >> > question
>>> >> > > > WRT
>>> >> > > > > >> > > context
>>> >> > > > > >> > > > of
>>> >> > > > > >> > > > > >> the
>>> >> > > > > >> > > > > >> > > > > change,
>>> >> > > > > >> > > > > >> > > > > >> or
>>> >> > > > > >> > > > > >> > > > > >> > > is it
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > means
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > for a
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > veto
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > from you,
>>> Sean?
>>> >> Just
>>> >> > > > trying
>>> >> > > > > >> to
>>> >> > > > > >> > > make
>>> >> > > > > >> > > > > sure
>>> >> > > > > >> > > > > >> > I'm
>>> >> > > > > >> > > > > >> > > > > >> following
>>> >> > > > > >> > > > > >> > > > > >> > > >> along
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > adequately.
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > I'd say ATM I'm
>>> -0,
>>> >> > > > bordering
>>> >> > > > > >> on
>>> >> > > > > >> > -1
>>> >> > > > > >> > > > but
>>> >> > > > > >> > > > > not
>>> >> > > > > >> > > > > >> > for
>>> >> > > > > >> > > > > >> > > > > >> reasons
>>> >> > > > > >> > > > > >> > > > > >> > I
>>> >> > > > > >> > > > > >> > > > > >> > > can
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > articulate
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > well.
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Here's an
>>> attempt.
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > We've been
>>> trying to
>>> >> > > move,
>>> >> > > > > as a
>>> >> > > > > >> > > > > community,
>>> >> > > > > >> > > > > >> > > > towards
>>> >> > > > > >> > > > > >> > > > > >> > > minimizing
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > risk
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > to
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > downstream
>>> folks by
>>> >> > > getting
>>> >> > > > > >> > > "complete
>>> >> > > > > >> > > > > >> enough
>>> >> > > > > >> > > > > >> > > for
>>> >> > > > > >> > > > > >> > > > > use"
>>> >> > > > > >> > > > > >> > > > > >> > > gates
>>> >> > > > > >> > > > > >> > > > > >> > > >> in
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > place
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > before we
>>> introduce
>>> >> new
>>> >> > > > > >> features.
>>> >> > > > > >> > > This
>>> >> > > > > >> > > > > was
>>> >> > > > > >> > > > > >> > > > spurred
>>> >> > > > > >> > > > > >> > > > > >> by a
>>> >> > > > > >> > > > > >> > > > > >> > > some
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > features
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > getting in
>>> >> half-baked
>>> >> > and
>>> >> > > > > never
>>> >> > > > > >> > > making
>>> >> > > > > >> > > > > it
>>> >> > > > > >> > > > > >> to
>>> >> > > > > >> > > > > >> > > "can
>>> >> > > > > >> > > > > >> > > > > >> really
>>> >> > > > > >> > > > > >> > > > > >> > > use"
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > status
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > (I'm thinking of
>>> >> > > > distributed
>>> >> > > > > >> log
>>> >> > > > > >> > > > replay
>>> >> > > > > >> > > > > and
>>> >> > > > > >> > > > > >> > the
>>> >> > > > > >> > > > > >> > > > > >> zk-less
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > assignment
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > stuff, I don't
>>> >> recall
>>> >> > if
>>> >> > > > > there
>>> >> > > > > >> was
>>> >> > > > > >> > > > > more).
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The gates,
>>> >> generally,
>>> >> > > > > included
>>> >> > > > > >> > > things
>>> >> > > > > >> > > > > like:
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have docs
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have sunny-day
>>> >> > > > correctness
>>> >> > > > > >> tests
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have
>>> >> > > > > >> > > correctness-in-face-of-failure
>>> >> > > > > >> > > > > tests
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't rely on
>>> >> things
>>> >> > > > > outside
>>> >> > > > > >> of
>>> >> > > > > >> > > > HBase
>>> >> > > > > >> > > > > for
>>> >> > > > > >> > > > > >> > > > normal
>>> >> > > > > >> > > > > >> > > > > >> > > operation
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > (okay
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > for
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > advanced
>>> operation)
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > As an example,
>>> we
>>> >> kept
>>> >> > > the
>>> >> > > > > MOB
>>> >> > > > > >> > work
>>> >> > > > > >> > > > off
>>> >> > > > > >> > > > > in
>>> >> > > > > >> > > > > >> a
>>> >> > > > > >> > > > > >> > > > branch
>>> >> > > > > >> > > > > >> > > > > >> and
>>> >> > > > > >> > > > > >> > > > > >> > > out
>>> >> > > > > >> > > > > >> > > > > >> > > >> of
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > master
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > until it could
>>> pass
>>> >> > these
>>> >> > > > > >> > criteria.
>>> >> > > > > >> > > > The
>>> >> > > > > >> > > > > big
>>> >> > > > > >> > > > > >> > > > > exemption
>>> >> > > > > >> > > > > >> > > > > >> > > we've
>>> >> > > > > >> > > > > >> > > > > >> > > >> had
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > to
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > this was the
>>> >> > hbase-spark
>>> >> > > > > >> > > integration,
>>> >> > > > > >> > > > > where
>>> >> > > > > >> > > > > >> > we
>>> >> > > > > >> > > > > >> > > > all
>>> >> > > > > >> > > > > >> > > > > >> > agreed
>>> >> > > > > >> > > > > >> > > > > >> > > it
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > could
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > land in master
>>> >> because
>>> >> > it
>>> >> > > > was
>>> >> > > > > >> very
>>> >> > > > > >> > > > well
>>> >> > > > > >> > > > > >> > > isolated
>>> >> > > > > >> > > > > >> > > > > (the
>>> >> > > > > >> > > > > >> > > > > >> > > slide
>>> >> > > > > >> > > > > >> > > > > >> > > >> > away
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > from
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > including docs
>>> as a
>>> >> > > > > first-class
>>> >> > > > > >> > part
>>> >> > > > > >> > > > of
>>> >> > > > > >> > > > > >> > > building
>>> >> > > > > >> > > > > >> > > > up
>>> >> > > > > >> > > > > >> > > > > >> that
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > integration
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > has led me to
>>> doubt
>>> >> the
>>> >> > > > > wisdom
>>> >> > > > > >> of
>>> >> > > > > >> > > this
>>> >> > > > > >> > > > > >> > > decision).
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > We've also been
>>> >> > treating
>>> >> > > > > >> inclusion
>>> >> > > > > >> > > in
>>> >> > > > > >> > > > a
>>> >> > > > > >> > > > > >> > > "probably
>>> >> > > > > >> > > > > >> > > > > >> will
>>> >> > > > > >> > > > > >> > > > > >> > be
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > released
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > to
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > downstream"
>>> branches
>>> >> > as a
>>> >> > > > > >> higher
>>> >> > > > > >> > > bar,
>>> >> > > > > >> > > > > >> > requiring
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't
>>> moderately
>>> >> > impact
>>> >> > > > > >> > > performance
>>> >> > > > > >> > > > > when
>>> >> > > > > >> > > > > >> > the
>>> >> > > > > >> > > > > >> > > > > >> feature
>>> >> > > > > >> > > > > >> > > > > >> > > isn't
>>> >> > > > > >> > > > > >> > > > > >> > > >> in
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > use
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't severely
>>> >> impact
>>> >> > > > > >> > performance
>>> >> > > > > >> > > > when
>>> >> > > > > >> > > > > >> the
>>> >> > > > > >> > > > > >> > > > > feature
>>> >> > > > > >> > > > > >> > > > > >> is
>>> >> > > > > >> > > > > >> > > > > >> > in
>>> >> > > > > >> > > > > >> > > > > >> > > >> use
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * either
>>> >> default-to-on
>>> >> > or
>>> >> > > > > show
>>> >> > > > > >> > > enough
>>> >> > > > > >> > > > > >> demand
>>> >> > > > > >> > > > > >> > to
>>> >> > > > > >> > > > > >> > > > > >> believe
>>> >> > > > > >> > > > > >> > > > > >> > a
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > non-trivial
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > number of folks
>>> will
>>> >> > turn
>>> >> > > > the
>>> >> > > > > >> > > feature
>>> >> > > > > >> > > > on
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The above has
>>> kept
>>> >> MOB
>>> >> > > and
>>> >> > > > > >> > > hbase-spark
>>> >> > > > > >> > > > > >> > > > integration
>>> >> > > > > >> > > > > >> > > > > >> out
>>> >> > > > > >> > > > > >> > > > > >> > of
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > branch-1,
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > presumably while
>>> >> > they've
>>> >> > > > > >> "gotten
>>> >> > > > > >> > > more
>>> >> > > > > >> > > > > >> stable"
>>> >> > > > > >> > > > > >> > > in
>>> >> > > > > >> > > > > >> > > > > >> master
>>> >> > > > > >> > > > > >> > > > > >> > > from
>>> >> > > > > >> > > > > >> > > > > >> > > >> > the
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > odd
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > vendor
>>> inclusion.
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Are we going to
>>> >> have a
>>> >> > > 2.0
>>> >> > > > > >> release
>>> >> > > > > >> > > > > before
>>> >> > > > > >> > > > > >> the
>>> >> > > > > >> > > > > >> > > end
>>> >> > > > > >> > > > > >> > > > > of
>>> >> > > > > >> > > > > >> > > > > >> the
>>> >> > > > > >> > > > > >> > > > > >> > > >> year?
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > We're
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > coming up on 1.5
>>> >> years
>>> >> > > > since
>>> >> > > > > >> the
>>> >> > > > > >> > > > > release of
>>> >> > > > > >> > > > > >> > > > version
>>> >> > > > > >> > > > > >> > > > > >> 1.0;
>>> >> > > > > >> > > > > >> > > > > >> > > >> seems
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > like
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > it's about time,
>>> >> > though I
>>> >> > > > > >> haven't
>>> >> > > > > >> > > seen
>>> >> > > > > >> > > > > any
>>> >> > > > > >> > > > > >> > > > concrete
>>> >> > > > > >> > > > > >> > > > > >> > plans
>>> >> > > > > >> > > > > >> > > > > >> > > >> this
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > year.
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Presuming we are
>>> >> going
>>> >> > to
>>> >> > > > > have
>>> >> > > > > >> one
>>> >> > > > > >> > > by
>>> >> > > > > >> > > > > the
>>> >> > > > > >> > > > > >> end
>>> >> > > > > >> > > > > >> > > of
>>> >> > > > > >> > > > > >> > > > > the
>>> >> > > > > >> > > > > >> > > > > >> > > year, it
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > seems a
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > bit close to
>>> still
>>> >> be
>>> >> > > > adding
>>> >> > > > > in
>>> >> > > > > >> > > > > "features
>>> >> > > > > >> > > > > >> > that
>>> >> > > > > >> > > > > >> > > > need
>>> >> > > > > >> > > > > >> > > > > >> > > maturing"
>>> >> > > > > >> > > > > >> > > > > >> > > >> > on
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > the
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > branch.
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The lack of a
>>> >> concrete
>>> >> > > plan
>>> >> > > > > for
>>> >> > > > > >> > 2.0
>>> >> > > > > >> > > > > keeps
>>> >> > > > > >> > > > > >> me
>>> >> > > > > >> > > > > >> > > from
>>> >> > > > > >> > > > > >> > > > > >> > > considering
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > these
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > things blocker
>>> at
>>> >> the
>>> >> > > > moment.
>>> >> > > > > >> But
>>> >> > > > > >> > I
>>> >> > > > > >> > > > know
>>> >> > > > > >> > > > > >> > first
>>> >> > > > > >> > > > > >> > > > hand
>>> >> > > > > >> > > > > >> > > > > >> how
>>> >> > > > > >> > > > > >> > > > > >> > > much
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > trouble
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > folks have had
>>> with
>>> >> > other
>>> >> > > > > >> features
>>> >> > > > > >> > > > that
>>> >> > > > > >> > > > > >> have
>>> >> > > > > >> > > > > >> > > gone
>>> >> > > > > >> > > > > >> > > > > >> into
>>> >> > > > > >> > > > > >> > > > > >> > > >> > downstream
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > facing releases
>>> >> without
>>> >> > > > > >> robustness
>>> >> > > > > >> > > > > checks
>>> >> > > > > >> > > > > >> > (i.e.
>>> >> > > > > >> > > > > >> > > > > >> > > replication),
>>> >> > > > > >> > > > > >> > > > > >> > > >> > and
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > I'm
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > concerned about
>>> what
>>> >> > > we're
>>> >> > > > > >> setting
>>> >> > > > > >> > > up
>>> >> > > > > >> > > > if
>>> >> > > > > >> > > > > >> 2.0
>>> >> > > > > >> > > > > >> > > goes
>>> >> > > > > >> > > > > >> > > > > out
>>> >> > > > > >> > > > > >> > > > > >> > with
>>> >> > > > > >> > > > > >> > > > > >> > > >> this
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > feature in its
>>> >> current
>>> >> > > > state.
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > --
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > Best regards,
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >    - Andy
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > Problems worthy of
>>> attack
>>> >> > prove
>>> >> > > > > their
>>> >> > > > > >> > > worth
>>> >> > > > > >> > > > by
>>> >> > > > > >> > > > > >> > > hitting
>>> >> > > > > >> > > > > >> > > > > >> back. -
>>> >> > > > > >> > > > > >> > > > > >> > > Piet
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > Hein
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > (via Tom White)
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > --
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > Best regards,
>>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > >    - Andy
>>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > Problems worthy of attack
>>> >> prove
>>> >> > > their
>>> >> > > > > >> worth
>>> >> > > > > >> > by
>>> >> > > > > >> > > > > >> hitting
>>> >> > > > > >> > > > > >> > > > back.
>>> >> > > > > >> > > > > >> > > > > -
>>> >> > > > > >> > > > > >> > > > > >> > Piet
>>> >> > > > > >> > > > > >> > > > > >> > > >> Hein
>>> >> > > > > >> > > > > >> > > > > >> > > >> > > (via Tom White)
>>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>>> >> > > > > >> > > > > >> > > > > >> > > >> >
>>> >> > > > > >> > > > > >> > > > > >> > > >>
>>> >> > > > > >> > > > > >> > > > > >> > >
>>> >> > > > > >> > > > > >> > > > > >> >
>>> >> > > > > >> > > > > >> > > > > >>
>>> >> > > > > >> > > > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > > >
>>> >> > > > > >> > > > > >> > > > >
>>> >> > > > > >> > > > > >> > > >
>>> >> > > > > >> > > > > >> > >
>>> >> > > > > >> > > > > >> >
>>> >> > > > > >> > > > > >>
>>> >> > > > > >> > > > >
>>> >> > > > > >> > > > >
>>> >> > > > > >> > > > >
>>> >> > > > > >> > > > > --
>>> >> > > > > >> > > > > busbey
>>> >> > > > > >> > > > >
>>> >> > > > > >> > > >
>>> >> > > > > >> > >
>>> >> > > > > >> >
>>> >> > > > > >>
>>> >> > > > > >
>>> >> > > > > >
>>> >> > > > >
>>> >> > > >
>>> >> > >
>>> >> >
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >>
>>> >> -- Appy
>>> >>
>>> >
>>> >
>>>
>>
>>
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Vladimir Rodionov <vl...@gmail.com>]

I usually use:

mvn clean install -DskipTests

and it works.

On Wed, Oct 12, 2016 at 1:01 PM, Vladimir Rodionov <vl...@gmail.com>
wrote:

> Michael,
>
> you can try master + latest patch on HBASE-14123 (v29). No need to use
> HBASE-7912 branch. I will double check HBASE-7912.
>
> -Vlad
>
> On Wed, Oct 12, 2016 at 12:31 PM, Stack <st...@duboce.net> wrote:
>
>> More info:
>>
>> stack@ve0524:~/hbase.git$ git checkout origin/HBASE-7912 -b 7912v2
>> Branch 7912v2 set up to track remote branch HBASE-7912 from origin.
>> Switched to a new branch '7912v2'
>> stack@ve0524:~/hbase.git$ java -version
>> java version "1.8.0_101"
>> Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
>> Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode)
>> stack@ve0524:~/hbase.git$ mvn clean install -DskipTests &> /tmp/out.txt
>>
>> ...
>>
>> St.Ack
>>
>>
>> On Wed, Oct 12, 2016 at 12:29 PM, Stack <st...@duboce.net> wrote:
>>
>> > Interesting. When I try it fails w/ below:
>> >
>> > [INFO] 26 warnings
>> > 322 [INFO] ------------------------------------------------------------
>> -
>> > 323 [INFO] ------------------------------------------------------------
>> -
>> > 324 [ERROR] COMPILATION ERROR :
>> > 325 [INFO] ------------------------------------------------------------
>> -
>> > 326 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.java:[48,8]
>> > org.apache.hadoop.hbase.io.encoding.rowindexV2.RowIndexCodecV2 is not
>> > abstract and does not override abstract method createSeeker(org.ap
>> >  ache.hadoop.hbase.CellComparator,org.apache.hadoop.hbase.io.
>> encoding.HFileBlockDecodingContext)
>> > in org.apache.hadoop.hbase.io.encoding.DataBlockEncoder
>> > 327 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.java:[143,3]
>> > method does not override or implement a method from a supertype
>> > 328 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.
>> java:[147,29]
>> > incompatible types: java.nio.ByteBuffer cannot be converted to
>> > org.apache.hadoop.hbase.nio.ByteBuff
>> > 329 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.
>> java:[148,33]
>> > cannot find symbol
>> > 330   symbol:   method getKeyDeepCopy()
>> > 331   location: variable seeker of type org.apache.hadoop.hbase.io.
>> > encoding.DataBlockEncoder.EncodedSeeker
>> > 332 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.java:[153,3]
>> > method does not override or implement a method from a supertype
>> > 333 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>> > apache/hadoop/hbase/io/encoding/rowindexV1/RowIndexCodecV1.java:[45,8]
>> > org.apache.hadoop.hbase.io.encoding.rowindexV1.RowIndexCodecV1 is not
>> > abstract and does not override abstract method createSeeker(org.ap
>> >  ache.hadoop.hbase.CellComparator,org.apache.hadoop.hbase.io.
>> encoding.HFileBlockDecodingContext)
>> > in org.apache.hadoop.hbase.io.encoding.DataBlockEncoder
>> > 334 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>> > apache/hadoop/hbase/io/encoding/rowindexV1/RowIndexCodecV1.java:[145,3]
>> > method does not override or implement a method from a supertype
>> > 335 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>> > apache/hadoop/hbase/io/encoding/rowindexV1/RowIndexCodecV1.java:[158,3]
>> > method does not override or implement a method from a supertype
>> > 336 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.java:[46,8]
>> > org.apache.hadoop.hbase.io.encoding.rowindexV2.RowIndexSeekerV2 is not
>> > abstract and does not override abstract method compareKey(org.ap
>> >  ache.hadoop.hbase.CellComparator,org.apache.hadoop.hbase.Cell) in
>> > org.apache.hadoop.hbase.io.encoding.DataBlockEncoder.EncodedSeeker
>> > 337 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.java:[79,3]
>> > method does not override or implement a method from a supertype
>> > 338 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>> java:[117,3]
>> > method does not override or implement a method from a supertype
>> > 339 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>> java:[190,3]
>> > method does not override or implement a method from a supertype
>> > 340 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>> java:[214,3]
>> > method does not override or implement a method from a supertype
>> > 341 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>> java:[349,3]
>> > method does not override or implement a method from a supertype
>> > 342 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>> java:[355,3]
>> > method does not override or implement a method from a supertype
>> > 343 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
>> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.
>> java:[421,36]
>> > no suitable method found for uncompressTags(java.nio.
>> > ByteBuffer,byte[],int,int)
>> > 344     method org.apache.hadoop.hbase.io.TagCompressionContext.
>> > uncompressTags(java.io.InputStream,byte[],int,int) is not applicable
>> > 345       (argument mismatch; java.nio.ByteBuffer cannot be converted to
>> > java.io.InputStream)
>> > 346     method org.apache.hadoop.hbase.io.TagCompressionContext.
>> > uncompressTags(org.apache.hadoop.hbase.nio.ByteBuff,byte[],int,int) is
>> > not applicable
>> > 347       (argument mismatch; java.nio.ByteBuffer cannot be converted to
>> > org.apache.hadoop.hbase.nio.ByteBuff)
>> >
>> > ....
>> >
>> > St.Ack
>> >
>> > On Tue, Oct 11, 2016 at 5:12 PM, Apekshit Sharma <ap...@cloudera.com>
>> > wrote:
>> >
>> >> @stack, it compiled for me.
>> >>
>> >> Also tried few commands, and have to say, it's well designed from user
>> >> commands perspective.
>> >>
>> >> On Tue, Oct 11, 2016 at 5:08 PM, Stack <st...@duboce.net> wrote:
>> >>
>> >> > On Tue, Oct 11, 2016 at 1:50 PM, Vladimir Rodionov <
>> >> vladrodionov@gmail.com
>> >> > >
>> >> > wrote:
>> >> >
>> >> > > Michael,
>> >> > >
>> >> > > Its in HBASE-7912
>> >> > >
>> >> > > This is tip of git log:
>> >> > >
>> >> > > commit a072f6f49a26a7259ff2aaef6cb56d85eb592482
>> >> > > Author: Frank Welsch <fw...@jps.net>
>> >> > > Date:   Fri Sep 23 18:00:42 2016 -0400
>> >> > >
>> >> > >     HBASE-16574 Book updates for backup and restore
>> >> > >
>> >> > > commit b14e2ab1c24e65ff88dd4c579acf83cb4ed0605e
>> >> > > Author: tedyu <yu...@gmail.com>
>> >> > > Date:   Wed Oct 5 16:29:40 2016 -0700
>> >> > >
>> >> > >     HBASE-16727 Backup refactoring: remove MR dependencies from
>> >> HMaster
>> >> > > (Vladimir Rodionov)
>> >> > >
>> >> > >
>> >> > Thanks. I have that. I tried it and it doesn't compile for me. Does
>> it
>> >> > compile for you?
>> >> > Thanks,
>> >> > M
>> >> >
>> >> >
>> >> >
>> >> > > -Vlad
>> >> > >
>> >> > > On Tue, Oct 11, 2016 at 1:44 PM, Stack <st...@duboce.net> wrote:
>> >> > >
>> >> > > > Which branch do I check out to try it? HBASE-7912 is not it. I
>> don't
>> >> > see
>> >> > > an
>> >> > > > HBASE-16727...
>> >> > > > Thanks,
>> >> > > > M
>> >> > > >
>> >> > > >
>> >> > > > On Tue, Oct 11, 2016 at 11:41 AM, Vladimir Rodionov <
>> >> > > > vladrodionov@gmail.com>
>> >> > > > wrote:
>> >> > > >
>> >> > > > > The last patch is on review board:
>> >> > > > > https://reviews.apache.org/r/52748
>> >> > > > >
>> >> > > > > On Mon, Oct 10, 2016 at 3:52 PM, Vladimir Rodionov <
>> >> > > > vladrodionov@gmail.com
>> >> > > > > >
>> >> > > > > wrote:
>> >> > > > >
>> >> > > > > > >> How hard to put in an hbase-backup module? hbase-server is
>> >> fat
>> >> > > > enough
>> >> > > > > > >> already. Could be done as a follow-up.
>> >> > > > > >
>> >> > > > > > https://issues.apache.org/jira/browse/HBASE-16727?
>> >> > > > > > focusedCommentId=15531237&page=com.atlassian.jira.
>> >> > > > > > plugin.system.issuetabpanels:comment-tabpanel#comment-155312
>> 37
>> >> > > > > >
>> >> > > > > > Can we do merge first? Then we can discuss separate module.
>> >> > > > > >
>> >> > > > > > -Vlad
>> >> > > > > >
>> >> > > > > > On Mon, Oct 10, 2016 at 3:44 PM, Ted Yu <yuzhihong@gmail.com
>> >
>> >> > wrote:
>> >> > > > > >
>> >> > > > > >> Looks like the first quote was cut off.
>> >> > > > > >> The original sentence was:
>> >> > > > > >>
>> >> > > > > >> bq. no mapreduce job launched from master or region server.
>> >> > > > > >>
>> >> > > > > >> mapreduce job is launched from the node where command line
>> >> tool is
>> >> > > > run.
>> >> > > > > >>
>> >> > > > > >> On Mon, Oct 10, 2016 at 3:38 PM, Stack <st...@duboce.net>
>> >> wrote:
>> >> > > > > >>
>> >> > > > > >> > bq. launched from master or region server.
>> >> > > > > >> >
>> >> > > > > >> > What does this mean please? Has to be run from Master or
>> >> > > > RegionServer?
>> >> > > > > >> Can
>> >> > > > > >> > it be run from another node altogether?
>> >> > > > > >> >
>> >> > > > > >> > On Mon, Oct 10, 2016 at 1:44 PM, Vladimir Rodionov <
>> >> > > > > >> vladrodionov@gmail.com
>> >> > > > > >> > >
>> >> > > > > >> > wrote:
>> >> > > > > >> >
>> >> > > > > >> > > >> mapreduce dependency has been moved to client side -
>> no
>> >> > > > mapreduce
>> >> > > > > >> job
>> >> > > > > >> > >
>> >> > > > > >> > > 1. We have no code in the client module anymore, due to
>> >> > > dependency
>> >> > > > > on
>> >> > > > > >> > > internal server API (HFile and WAL access).
>> >> > > > > >> > > 2. Backup/ restore are client - driven operations, but
>> all
>> >> the
>> >> > > > code
>> >> > > > > >> > resides
>> >> > > > > >> > > in the server module
>> >> > > > > >> > >
>> >> > > > > >> >
>> >> > > > > >> > How hard to put in an hbase-backup module? hbase-server is
>> >> fat
>> >> > > > enough
>> >> > > > > >> > already. Could be done as a follow-up.
>> >> > > > > >> >
>> >> > > > > >> > Thanks,
>> >> > > > > >> > St.Ack
>> >> > > > > >> >
>> >> > > > > >> >
>> >> > > > > >> >
>> >> > > > > >> > > 3. No MR in Master, no procedure - driven execution.
>> >> > > > > >> > > 4. Old good MR from command-line.
>> >> > > > > >> > > 5. Security was simplified and now only super-user is
>> >> allowed
>> >> > to
>> >> > > > run
>> >> > > > > >> > > backup/restores.
>> >> > > > > >> > > 6. HBase Backup API was gone due to 1. Now only
>> >> command-line
>> >> > > > access
>> >> > > > > to
>> >> > > > > >> > > backup tools.
>> >> > > > > >> > >
>> >> > > > > >> > > These consequences of refactoring has been discussed in
>> >> > > > HBASE-16727.
>> >> > > > > >> > >
>> >> > > > > >> > > -Vlad
>> >> > > > > >> > >
>> >> > > > > >> > >
>> >> > > > > >> > >
>> >> > > > > >> > >
>> >> > > > > >> > > On Mon, Oct 10, 2016 at 1:31 PM, Ted Yu <
>> >> yuzhihong@gmail.com>
>> >> > > > > wrote:
>> >> > > > > >> > >
>> >> > > > > >> > > > Reviving this thread.
>> >> > > > > >> > > >
>> >> > > > > >> > > > The following has taken place:
>> >> > > > > >> > > >
>> >> > > > > >> > > > mapreduce dependency has been moved to client side -
>> no
>> >> > > > mapreduce
>> >> > > > > >> job
>> >> > > > > >> > > > launched from master or region server.
>> >> > > > > >> > > > document patch (HBASE-16574) has been integrated.
>> >> > > > > >> > > > Updated mega patch has been attached to HBASE-14123:
>> this
>> >> > > covers
>> >> > > > > the
>> >> > > > > >> > > > refactor in #1 above and the protobuf 3 merge.
>> >> > > > > >> > > >
>> >> > > > > >> > > > If community has more feedback on the merge proposal,
>> I
>> >> > would
>> >> > > > love
>> >> > > > > >> to
>> >> > > > > >> > > hear
>> >> > > > > >> > > > it.
>> >> > > > > >> > > >
>> >> > > > > >> > > > Thanks
>> >> > > > > >> > > >
>> >> > > > > >> > > > On Thu, Sep 22, 2016 at 10:31 AM, Sean Busbey <
>> >> > > > > busbey@cloudera.com>
>> >> > > > > >> > > wrote:
>> >> > > > > >> > > >
>> >> > > > > >> > > > > I'd like to see the docs proposed on HBASE-16574
>> >> > integrated
>> >> > > > into
>> >> > > > > >> our
>> >> > > > > >> > > > > project's documentation prior to merge.
>> >> > > > > >> > > > >
>> >> > > > > >> > > > > On Thu, Sep 22, 2016 at 9:02 AM, Ted Yu <
>> >> > > yuzhihong@gmail.com>
>> >> > > > > >> wrote:
>> >> > > > > >> > > > > > This feature can be marked experimental due to
>> some
>> >> > > > > limitations
>> >> > > > > >> > such
>> >> > > > > >> > > as
>> >> > > > > >> > > > > > security.
>> >> > > > > >> > > > > >
>> >> > > > > >> > > > > > Your previous round of comments have been
>> addressed.
>> >> > > > > >> > > > > > Command line tool has gone through:
>> >> > > > > >> > > > > >
>> >> > > > > >> > > > > > HBASE-16620 Fix backup command-line tool usability
>> >> > issues
>> >> > > > > >> > > > > > HBASE-16655 hbase backup describe with incorrect
>> >> backup
>> >> > id
>> >> > > > > >> results
>> >> > > > > >> > in
>> >> > > > > >> > > > NPE
>> >> > > > > >> > > > > >
>> >> > > > > >> > > > > > The updated doc has been attached to HBASE-16574.
>> >> > > > > >> > > > > >
>> >> > > > > >> > > > > > Cheers
>> >> > > > > >> > > > > >
>> >> > > > > >> > > > > > On Thu, Sep 22, 2016 at 8:53 AM, Stack <
>> >> > stack@duboce.net>
>> >> > > > > >> wrote:
>> >> > > > > >> > > > > >
>> >> > > > > >> > > > > >> On Wed, Sep 21, 2016 at 7:43 AM, Ted Yu <
>> >> > > > yuzhihong@gmail.com
>> >> > > > > >
>> >> > > > > >> > > wrote:
>> >> > > > > >> > > > > >>
>> >> > > > > >> > > > > >> > Are there more (review) comments ?
>> >> > > > > >> > > > > >> >
>> >> > > > > >> > > > > >> >
>> >> > > > > >> > > > > >> Are outstanding comments addressed?
>> >> > > > > >> > > > > >>
>> >> > > > > >> > > > > >> I don't see answer to my 'is this
>> experimental/will
>> >> it
>> >> > be
>> >> > > > > >> marked
>> >> > > > > >> > > > > >> experimental' question.
>> >> > > > > >> > > > > >>
>> >> > > > > >> > > > > >> I ran into some issues trying to use the feature
>> and
>> >> > > > > suggested
>> >> > > > > >> > that
>> >> > > > > >> > > a
>> >> > > > > >> > > > > >> feature likes this needs polish else it'll just
>> rot,
>> >> > > > unused.
>> >> > > > > >> Has
>> >> > > > > >> > > > polish
>> >> > > > > >> > > > > >> been applied? All ready for another 'user' test?
>> >> > Suggest
>> >> > > > that
>> >> > > > > >> you
>> >> > > > > >> > > > update
>> >> > > > > >> > > > > >> here going forward for the benefit of those
>> trying
>> >> to
>> >> > > > follow
>> >> > > > > >> along
>> >> > > > > >> > > and
>> >> > > > > >> > > > > who
>> >> > > > > >> > > > > >> are not watching JIRA change fly-by.
>> >> > > > > >> > > > > >>
>> >> > > > > >> > > > > >> It looks like doc got a revision -- I have to
>> check
>> >> --
>> >> > to
>> >> > > > > take
>> >> > > > > >> on
>> >> > > > > >> > > > > >> suggestion made above but again, suggest, that
>> this
>> >> > > thread
>> >> > > > > gets
>> >> > > > > >> > > > updated.
>> >> > > > > >> > > > > >>
>> >> > > > > >> > > > > >> Thanks,
>> >> > > > > >> > > > > >> St.Ack
>> >> > > > > >> > > > > >>
>> >> > > > > >> > > > > >>
>> >> > > > > >> > > > > >>
>> >> > > > > >> > > > > >> > Thanks
>> >> > > > > >> > > > > >> >
>> >> > > > > >> > > > > >> > On Tue, Sep 20, 2016 at 10:02 AM, Devaraj Das <
>> >> > > > > >> > > ddas@hortonworks.com
>> >> > > > > >> > > > >
>> >> > > > > >> > > > > >> > wrote:
>> >> > > > > >> > > > > >> >
>> >> > > > > >> > > > > >> > > Just reviving this thread. Thanks Sean,
>> Stack,
>> >> > Dima,
>> >> > > > and
>> >> > > > > >> > others
>> >> > > > > >> > > > for
>> >> > > > > >> > > > > the
>> >> > > > > >> > > > > >> > > thorough reviews and testing. Thanks Ted and
>> >> Vlad
>> >> > for
>> >> > > > > >> taking
>> >> > > > > >> > > care
>> >> > > > > >> > > > of
>> >> > > > > >> > > > > >> the
>> >> > > > > >> > > > > >> > > feedback. Are we all good to do the merge
>> now?
>> >> > Rather
>> >> > > > do
>> >> > > > > >> > sooner
>> >> > > > > >> > > > than
>> >> > > > > >> > > > > >> > later.
>> >> > > > > >> > > > > >> > > ________________________________________
>> >> > > > > >> > > > > >> > > From: saint.ack@gmail.com <
>> saint.ack@gmail.com>
>> >> on
>> >> > > > > behalf
>> >> > > > > >> of
>> >> > > > > >> > > > Stack
>> >> > > > > >> > > > > <
>> >> > > > > >> > > > > >> > > stack@duboce.net>
>> >> > > > > >> > > > > >> > > Sent: Monday, September 12, 2016 1:18 PM
>> >> > > > > >> > > > > >> > > To: HBase Dev List
>> >> > > > > >> > > > > >> > > Subject: Re: [DISCUSSION] Merge Backup /
>> >> Restore -
>> >> > > > Branch
>> >> > > > > >> > > > HBASE-7912
>> >> > > > > >> > > > > >> > >
>> >> > > > > >> > > > > >> > > On Mon, Sep 12, 2016 at 12:19 PM, Ted Yu <
>> >> > > > > >> yuzhihong@gmail.com
>> >> > > > > >> > >
>> >> > > > > >> > > > > wrote:
>> >> > > > > >> > > > > >> > >
>> >> > > > > >> > > > > >> > > > Mega patch (rev 18) is on HBASE-14123.
>> >> > > > > >> > > > > >> > > >
>> >> > > > > >> > > > > >> > > > Please comment on HBASE-14123 on how you
>> want
>> >> to
>> >> > > > > review.
>> >> > > > > >> > > > > >> > > >
>> >> > > > > >> > > > > >> > >
>> >> > > > > >> > > > > >> > >
>> >> > > > > >> > > > > >> > > Yeah. That was my lost tab. Last rb was 6
>> months
>> >> > ago.
>> >> > > > > >> Suggest
>> >> > > > > >> > > > > updating
>> >> > > > > >> > > > > >> > it.
>> >> > > > > >> > > > > >> > > RB is pretty good for review. Patch is only
>> >> 1.5M so
>> >> > > > > should
>> >> > > > > >> be
>> >> > > > > >> > > > fine.
>> >> > > > > >> > > > > >> > >
>> >> > > > > >> > > > > >> > > St.Ack
>> >> > > > > >> > > > > >> > >
>> >> > > > > >> > > > > >> > >
>> >> > > > > >> > > > > >> > > >
>> >> > > > > >> > > > > >> > > > Thanks
>> >> > > > > >> > > > > >> > > >
>> >> > > > > >> > > > > >> > > > On Mon, Sep 12, 2016 at 12:15 PM, Stack <
>> >> > > > > >> stack@duboce.net>
>> >> > > > > >> > > > wrote:
>> >> > > > > >> > > > > >> > > >
>> >> > > > > >> > > > > >> > > > > On review of the 'patch', do I just
>> compare
>> >> the
>> >> > > > > branch
>> >> > > > > >> to
>> >> > > > > >> > > > > master or
>> >> > > > > >> > > > > >> > is
>> >> > > > > >> > > > > >> > > > > there a megapatch posted somewhere (I
>> think
>> >> I
>> >> > saw
>> >> > > > one
>> >> > > > > >> but
>> >> > > > > >> > it
>> >> > > > > >> > > > > seemed
>> >> > > > > >> > > > > >> > > stale
>> >> > > > > >> > > > > >> > > > > and then I 'lost' the tab). Sorry for
>> dumb
>> >> > > > question.
>> >> > > > > >> > > > > >> > > > > St.Ack
>> >> > > > > >> > > > > >> > > > >
>> >> > > > > >> > > > > >> > > > > On Mon, Sep 12, 2016 at 12:01 PM, Stack <
>> >> > > > > >> stack@duboce.net
>> >> > > > > >> > >
>> >> > > > > >> > > > > wrote:
>> >> > > > > >> > > > > >> > > > >
>> >> > > > > >> > > > > >> > > > > > Late to the game. A few comments after
>> >> > > rereading
>> >> > > > > this
>> >> > > > > >> > > thread
>> >> > > > > >> > > > > as a
>> >> > > > > >> > > > > >> > > > 'user'.
>> >> > > > > >> > > > > >> > > > > >
>> >> > > > > >> > > > > >> > > > > > + Before merge, a user-facing feature
>> like
>> >> > this
>> >> > > > > >> should
>> >> > > > > >> > > work
>> >> > > > > >> > > > > (If
>> >> > > > > >> > > > > >> > this
>> >> > > > > >> > > > > >> > > is
>> >> > > > > >> > > > > >> > > > > "higher-bar
>> >> > > > > >> > > > > >> > > > > > for new features", bring it on --
>> smile).
>> >> > > > > >> > > > > >> > > > > > + As a user, I tried the branch with
>> tools
>> >> > > after
>> >> > > > > >> > reviewing
>> >> > > > > >> > > > the
>> >> > > > > >> > > > > >> > > > > just-posted
>> >> > > > > >> > > > > >> > > > > > doc. I had an 'interesting' experience
>> >> (left
>> >> > > > > >> comments up
>> >> > > > > >> > > on
>> >> > > > > >> > > > > >> > issue). I
>> >> > > > > >> > > > > >> > > > > think
>> >> > > > > >> > > > > >> > > > > > the tooling/doc. important to get
>> right.
>> >> If
>> >> > it
>> >> > > > > breaks
>> >> > > > > >> > > easily
>> >> > > > > >> > > > > or
>> >> > > > > >> > > > > >> is
>> >> > > > > >> > > > > >> > > > > > inconsistent (or lacks 'polish'),
>> >> operators
>> >> > > will
>> >> > > > > >> judge
>> >> > > > > >> > the
>> >> > > > > >> > > > > whole
>> >> > > > > >> > > > > >> > > > > > backup/restore tooling chain as not
>> >> > trustworthy
>> >> > > > and
>> >> > > > > >> > > abandon
>> >> > > > > >> > > > > it.
>> >> > > > > >> > > > > >> > Lets
>> >> > > > > >> > > > > >> > > > not
>> >> > > > > >> > > > > >> > > > > > have this happen to this feature.
>> >> > > > > >> > > > > >> > > > > > + Matteo's suggestion (with a helpful
>> >> starter
>> >> > > > list)
>> >> > > > > >> that
>> >> > > > > >> > > > there
>> >> > > > > >> > > > > >> > needs
>> >> > > > > >> > > > > >> > > to
>> >> > > > > >> > > > > >> > > > > be
>> >> > > > > >> > > > > >> > > > > > explicit qualification on what is
>> actually
>> >> > > being
>> >> > > > > >> > delivered
>> >> > > > > >> > > > --
>> >> > > > > >> > > > > >> > > > including a
>> >> > > > > >> > > > > >> > > > > > listing of limitations (some look
>> serious
>> >> > such
>> >> > > as
>> >> > > > > >> data
>> >> > > > > >> > > bleed
>> >> > > > > >> > > > > from
>> >> > > > > >> > > > > >> > > other
>> >> > > > > >> > > > > >> > > > > > regions in WALs, but maybe I don't care
>> >> for
>> >> > my
>> >> > > > use
>> >> > > > > >> > > case...)
>> >> > > > > >> > > > --
>> >> > > > > >> > > > > >> > needs
>> >> > > > > >> > > > > >> > > to
>> >> > > > > >> > > > > >> > > > > > accompany the merge. Lets fold them
>> into
>> >> the
>> >> > > user
>> >> > > > > >> doc.
>> >> > > > > >> > in
>> >> > > > > >> > > > the
>> >> > > > > >> > > > > >> > > technical
>> >> > > > > >> > > > > >> > > > > > overview area as suggested so user
>> >> > expectations
>> >> > > > are
>> >> > > > > >> > > properly
>> >> > > > > >> > > > > >> > managed
>> >> > > > > >> > > > > >> > > > > > (otherwise, they expect the world and
>> will
>> >> > just
>> >> > > > > give
>> >> > > > > >> up
>> >> > > > > >> > > when
>> >> > > > > >> > > > > we
>> >> > > > > >> > > > > >> > fall
>> >> > > > > >> > > > > >> > > > > > short). Vladimir did a list of what is
>> in
>> >> > each
>> >> > > of
>> >> > > > > the
>> >> > > > > >> > > phases
>> >> > > > > >> > > > > >> above
>> >> > > > > >> > > > > >> > > > which
>> >> > > > > >> > > > > >> > > > > > would serve as a good start.
>> >> > > > > >> > > > > >> > > > > > + Is this feature 'experimental'
>> (Matteo
>> >> asks
>> >> > > > > above).
>> >> > > > > >> > I'd
>> >> > > > > >> > > > > prefer
>> >> > > > > >> > > > > >> it
>> >> > > > > >> > > > > >> > > is
>> >> > > > > >> > > > > >> > > > > > not. If it is, it should be labelled
>> all
>> >> over
>> >> > > > that
>> >> > > > > >> it is
>> >> > > > > >> > > > so. I
>> >> > > > > >> > > > > >> see
>> >> > > > > >> > > > > >> > > > > current
>> >> > > > > >> > > > > >> > > > > > state called out as a '... technical
>> >> preview
>> >> > > > > >> feature'.
>> >> > > > > >> > > Does
>> >> > > > > >> > > > > this
>> >> > > > > >> > > > > >> > mean
>> >> > > > > >> > > > > >> > > > > > not-for-users?
>> >> > > > > >> > > > > >> > > > > >
>> >> > > > > >> > > > > >> > > > > > St.Ack
>> >> > > > > >> > > > > >> > > > > >
>> >> > > > > >> > > > > >> > > > > >
>> >> > > > > >> > > > > >> > > > > >
>> >> > > > > >> > > > > >> > > > > >
>> >> > > > > >> > > > > >> > > > > >
>> >> > > > > >> > > > > >> > > > > >
>> >> > > > > >> > > > > >> > > > > >
>> >> > > > > >> > > > > >> > > > > >
>> >> > > > > >> > > > > >> > > > > >
>> >> > > > > >> > > > > >> > > > > >
>> >> > > > > >> > > > > >> > > > > >
>> >> > > > > >> > > > > >> > > > > > On Mon, Sep 12, 2016 at 8:03 AM, Ted
>> Yu <
>> >> > > > > >> > > > yuzhihong@gmail.com>
>> >> > > > > >> > > > > >> > wrote:
>> >> > > > > >> > > > > >> > > > > >
>> >> > > > > >> > > > > >> > > > > >> Sean:
>> >> > > > > >> > > > > >> > > > > >> Do you have more comments ?
>> >> > > > > >> > > > > >> > > > > >>
>> >> > > > > >> > > > > >> > > > > >> Cheers
>> >> > > > > >> > > > > >> > > > > >>
>> >> > > > > >> > > > > >> > > > > >> On Fri, Sep 9, 2016 at 1:42 PM,
>> Vladimir
>> >> > > > Rodionov
>> >> > > > > <
>> >> > > > > >> > > > > >> > > > > vladrodionov@gmail.com
>> >> > > > > >> > > > > >> > > > > >> >
>> >> > > > > >> > > > > >> > > > > >> wrote:
>> >> > > > > >> > > > > >> > > > > >>
>> >> > > > > >> > > > > >> > > > > >> > Sean,
>> >> > > > > >> > > > > >> > > > > >> >
>> >> > > > > >> > > > > >> > > > > >> > Backup/Restore can fail due to
>> various
>> >> > > > reasons:
>> >> > > > > >> > network
>> >> > > > > >> > > > > outage
>> >> > > > > >> > > > > >> > > > > (cluster
>> >> > > > > >> > > > > >> > > > > >> > wide), various time-outs in HBase
>> and
>> >> HDFS
>> >> > > > > layer,
>> >> > > > > >> M/R
>> >> > > > > >> > > > > failure
>> >> > > > > >> > > > > >> > due
>> >> > > > > >> > > > > >> > > to
>> >> > > > > >> > > > > >> > > > > >> "HDFS
>> >> > > > > >> > > > > >> > > > > >> > exceeded quota", user error (manual
>> >> > deletion
>> >> > > > of
>> >> > > > > >> data)
>> >> > > > > >> > > and
>> >> > > > > >> > > > > so
>> >> > > > > >> > > > > >> on
>> >> > > > > >> > > > > >> > so
>> >> > > > > >> > > > > >> > > > on.
>> >> > > > > >> > > > > >> > > > > >> That
>> >> > > > > >> > > > > >> > > > > >> > is impossible to enumerate all
>> possible
>> >> > > types
>> >> > > > of
>> >> > > > > >> > > failures
>> >> > > > > >> > > > > in a
>> >> > > > > >> > > > > >> > > > > >> distributed
>> >> > > > > >> > > > > >> > > > > >> > system - that is not our goal/task.
>> >> > > > > >> > > > > >> > > > > >> >
>> >> > > > > >> > > > > >> > > > > >> > We focus completely on backup system
>> >> table
>> >> > > > > >> > consistency
>> >> > > > > >> > > > in a
>> >> > > > > >> > > > > >> > > presence
>> >> > > > > >> > > > > >> > > > > of
>> >> > > > > >> > > > > >> > > > > >> any
>> >> > > > > >> > > > > >> > > > > >> > type of failure. That is what I call
>> >> > > > "tolerance
>> >> > > > > to
>> >> > > > > >> > > > > failures".
>> >> > > > > >> > > > > >> > > > > >> >
>> >> > > > > >> > > > > >> > > > > >> > On a failure:
>> >> > > > > >> > > > > >> > > > > >> >
>> >> > > > > >> > > > > >> > > > > >> > BACKUP. All backup system
>> information
>> >> > (prior
>> >> > > > to
>> >> > > > > >> > backup)
>> >> > > > > >> > > > > will
>> >> > > > > >> > > > > >> be
>> >> > > > > >> > > > > >> > > > > restored
>> >> > > > > >> > > > > >> > > > > >> > and all temporary data, related to a
>> >> > failed
>> >> > > > > >> session,
>> >> > > > > >> > in
>> >> > > > > >> > > > > HDFS
>> >> > > > > >> > > > > >> > will
>> >> > > > > >> > > > > >> > > be
>> >> > > > > >> > > > > >> > > > > >> > deleted
>> >> > > > > >> > > > > >> > > > > >> > RESTORE. We do not care about system
>> >> data,
>> >> > > > > because
>> >> > > > > >> > > > restore
>> >> > > > > >> > > > > >> does
>> >> > > > > >> > > > > >> > > not
>> >> > > > > >> > > > > >> > > > > >> change
>> >> > > > > >> > > > > >> > > > > >> > it. Temporary data in HDFS will be
>> >> cleaned
>> >> > > up
>> >> > > > > and
>> >> > > > > >> > table
>> >> > > > > >> > > > > will
>> >> > > > > >> > > > > >> be
>> >> > > > > >> > > > > >> > > in a
>> >> > > > > >> > > > > >> > > > > >> state
>> >> > > > > >> > > > > >> > > > > >> > back to where it was before
>> operation
>> >> > > started.
>> >> > > > > >> > > > > >> > > > > >> >
>> >> > > > > >> > > > > >> > > > > >> > This is what user should expect in
>> case
>> >> > of a
>> >> > > > > >> failure.
>> >> > > > > >> > > > > >> > > > > >> >
>> >> > > > > >> > > > > >> > > > > >> > -Vlad
>> >> > > > > >> > > > > >> > > > > >> >
>> >> > > > > >> > > > > >> > > > > >> >
>> >> > > > > >> > > > > >> > > > > >> > -Vlad
>> >> > > > > >> > > > > >> > > > > >> >
>> >> > > > > >> > > > > >> > > > > >> > On Fri, Sep 9, 2016 at 12:56 PM,
>> Sean
>> >> > > Busbey <
>> >> > > > > >> > > > > >> busbey@apache.org
>> >> > > > > >> > > > > >> > >
>> >> > > > > >> > > > > >> > > > > wrote:
>> >> > > > > >> > > > > >> > > > > >> >
>> >> > > > > >> > > > > >> > > > > >> > > Failing in a consistent way, with
>> >> docs
>> >> > > that
>> >> > > > > >> explain
>> >> > > > > >> > > the
>> >> > > > > >> > > > > >> > various
>> >> > > > > >> > > > > >> > > > > >> > > expected failures would be
>> >> sufficient.
>> >> > > > > >> > > > > >> > > > > >> > >
>> >> > > > > >> > > > > >> > > > > >> > > On Fri, Sep 9, 2016 at 12:16 PM,
>> >> > Vladimir
>> >> > > > > >> Rodionov
>> >> > > > > >> > > > > >> > > > > >> > > <vl...@gmail.com> wrote:
>> >> > > > > >> > > > > >> > > > > >> > > > Do not worry Sean, doc is coming
>> >> today
>> >> > > as
>> >> > > > a
>> >> > > > > >> > preview
>> >> > > > > >> > > > and
>> >> > > > > >> > > > > >> our
>> >> > > > > >> > > > > >> > > > writer
>> >> > > > > >> > > > > >> > > > > >> > Frank
>> >> > > > > >> > > > > >> > > > > >> > > > will be working on a putting  it
>> >> into
>> >> > > > Apache
>> >> > > > > >> > repo.
>> >> > > > > >> > > > > >> Timeline
>> >> > > > > >> > > > > >> > > > > depends
>> >> > > > > >> > > > > >> > > > > >> on
>> >> > > > > >> > > > > >> > > > > >> > > > Franks schedule but I hope we
>> will
>> >> get
>> >> > > it
>> >> > > > > >> rather
>> >> > > > > >> > > > sooner
>> >> > > > > >> > > > > >> than
>> >> > > > > >> > > > > >> > > > > later.
>> >> > > > > >> > > > > >> > > > > >> > > >
>> >> > > > > >> > > > > >> > > > > >> > > > As for failure testing, we are
>> >> > focusing
>> >> > > > only
>> >> > > > > >> on a
>> >> > > > > >> > > > > >> consistent
>> >> > > > > >> > > > > >> > > > state
>> >> > > > > >> > > > > >> > > > > >> of
>> >> > > > > >> > > > > >> > > > > >> > > > backup system data in a
>> presence of
>> >> > any
>> >> > > > type
>> >> > > > > >> of
>> >> > > > > >> > > > > failures,
>> >> > > > > >> > > > > >> We
>> >> > > > > >> > > > > >> > > are
>> >> > > > > >> > > > > >> > > > > not
>> >> > > > > >> > > > > >> > > > > >> > > going
>> >> > > > > >> > > > > >> > > > > >> > > > to implement  anything more
>> >> "fancy",
>> >> > > than
>> >> > > > > >> that.
>> >> > > > > >> > We
>> >> > > > > >> > > > > allow
>> >> > > > > >> > > > > >> > both:
>> >> > > > > >> > > > > >> > > > > >> backup
>> >> > > > > >> > > > > >> > > > > >> > and
>> >> > > > > >> > > > > >> > > > > >> > > > restore to fail. What we do not
>> >> allow
>> >> > is
>> >> > > > to
>> >> > > > > >> have
>> >> > > > > >> > > > system
>> >> > > > > >> > > > > >> data
>> >> > > > > >> > > > > >> > > > > >> corrupted.
>> >> > > > > >> > > > > >> > > > > >> > > > Will it suffice for you? Do you
>> >> have
>> >> > any
>> >> > > > > other
>> >> > > > > >> > > > > concerns,
>> >> > > > > >> > > > > >> you
>> >> > > > > >> > > > > >> > > > want
>> >> > > > > >> > > > > >> > > > > >> us to
>> >> > > > > >> > > > > >> > > > > >> > > > address?
>> >> > > > > >> > > > > >> > > > > >> > > >
>> >> > > > > >> > > > > >> > > > > >> > > > -Vlad
>> >> > > > > >> > > > > >> > > > > >> > > >
>> >> > > > > >> > > > > >> > > > > >> > > >
>> >> > > > > >> > > > > >> > > > > >> > > > On Fri, Sep 9, 2016 at 10:56 AM,
>> >> Sean
>> >> > > > > Busbey <
>> >> > > > > >> > > > > >> > > busbey@apache.org
>> >> > > > > >> > > > > >> > > > >
>> >> > > > > >> > > > > >> > > > > >> > wrote:
>> >> > > > > >> > > > > >> > > > > >> > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> "docs will come to Apache soon"
>> >> does
>> >> > > not
>> >> > > > > >> address
>> >> > > > > >> > > my
>> >> > > > > >> > > > > >> concern
>> >> > > > > >> > > > > >> > > > > around
>> >> > > > > >> > > > > >> > > > > >> > docs
>> >> > > > > >> > > > > >> > > > > >> > > at
>> >> > > > > >> > > > > >> > > > > >> > > >> all, unless said docs have
>> already
>> >> > made
>> >> > > > it
>> >> > > > > >> into
>> >> > > > > >> > > the
>> >> > > > > >> > > > > >> project
>> >> > > > > >> > > > > >> > > > > repo. I
>> >> > > > > >> > > > > >> > > > > >> > > don't
>> >> > > > > >> > > > > >> > > > > >> > > >> want third party resources for
>> >> using
>> >> > a
>> >> > > > > major
>> >> > > > > >> and
>> >> > > > > >> > > > > >> important
>> >> > > > > >> > > > > >> > > > > feature
>> >> > > > > >> > > > > >> > > > > >> of
>> >> > > > > >> > > > > >> > > > > >> > > the
>> >> > > > > >> > > > > >> > > > > >> > > >> project, I want us to provide
>> end
>> >> > users
>> >> > > > > with
>> >> > > > > >> > what
>> >> > > > > >> > > > they
>> >> > > > > >> > > > > >> need
>> >> > > > > >> > > > > >> > > to
>> >> > > > > >> > > > > >> > > > > get
>> >> > > > > >> > > > > >> > > > > >> the
>> >> > > > > >> > > > > >> > > > > >> > > job
>> >> > > > > >> > > > > >> > > > > >> > > >> done.
>> >> > > > > >> > > > > >> > > > > >> > > >>
>> >> > > > > >> > > > > >> > > > > >> > > >> I see some calls for patience
>> on
>> >> the
>> >> > > > > failure
>> >> > > > > >> > > > testing,
>> >> > > > > >> > > > > but
>> >> > > > > >> > > > > >> > the
>> >> > > > > >> > > > > >> > > > > >> appeal
>> >> > > > > >> > > > > >> > > > > >> > to
>> >> > > > > >> > > > > >> > > > > >> > > us
>> >> > > > > >> > > > > >> > > > > >> > > >> having done a bad job of
>> requiring
>> >> > > proper
>> >> > > > > >> tests
>> >> > > > > >> > of
>> >> > > > > >> > > > > >> previous
>> >> > > > > >> > > > > >> > > > > >> features
>> >> > > > > >> > > > > >> > > > > >> > > just
>> >> > > > > >> > > > > >> > > > > >> > > >> makes me more concerned about
>> not
>> >> > > getting
>> >> > > > > >> them
>> >> > > > > >> > > > here. I
>> >> > > > > >> > > > > >> > don't
>> >> > > > > >> > > > > >> > > > want
>> >> > > > > >> > > > > >> > > > > >> to
>> >> > > > > >> > > > > >> > > > > >> > set
>> >> > > > > >> > > > > >> > > > > >> > > >> yet another bad example that
>> will
>> >> > then
>> >> > > be
>> >> > > > > >> > pointed
>> >> > > > > >> > > to
>> >> > > > > >> > > > > in
>> >> > > > > >> > > > > >> the
>> >> > > > > >> > > > > >> > > > > future.
>> >> > > > > >> > > > > >> > > > > >> > > >>
>> >> > > > > >> > > > > >> > > > > >> > > >> On Sep 8, 2016 10:50, "Ted Yu"
>> <
>> >> > > > > >> > > yuzhihong@gmail.com
>> >> > > > > >> > > > >
>> >> > > > > >> > > > > >> > wrote:
>> >> > > > > >> > > > > >> > > > > >> > > >>
>> >> > > > > >> > > > > >> > > > > >> > > >> > Is there any concern which is
>> >> not
>> >> > > > > >> addressed ?
>> >> > > > > >> > > > > >> > > > > >> > > >> >
>> >> > > > > >> > > > > >> > > > > >> > > >> > Do we need another Vote
>> thread ?
>> >> > > > > >> > > > > >> > > > > >> > > >> >
>> >> > > > > >> > > > > >> > > > > >> > > >> > Thanks
>> >> > > > > >> > > > > >> > > > > >> > > >> >
>> >> > > > > >> > > > > >> > > > > >> > > >> > On Thu, Sep 8, 2016 at 9:21
>> AM,
>> >> > > Andrew
>> >> > > > > >> > Purtell <
>> >> > > > > >> > > > > >> > > > > >> apurtell@apache.org
>> >> > > > > >> > > > > >> > > > > >> > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > wrote:
>> >> > > > > >> > > > > >> > > > > >> > > >> >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > Vlad,
>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > I apologize for using the
>> term
>> >> > > > > >> 'half-baked'
>> >> > > > > >> > > in a
>> >> > > > > >> > > > > way
>> >> > > > > >> > > > > >> > that
>> >> > > > > >> > > > > >> > > > > could
>> >> > > > > >> > > > > >> > > > > >> > > seem a
>> >> > > > > >> > > > > >> > > > > >> > > >> > > description of HBASE-7912.
>> I
>> >> > meant
>> >> > > > that
>> >> > > > > >> as a
>> >> > > > > >> > > > > general
>> >> > > > > >> > > > > >> > > > > >> hypothetical.
>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > On Wed, Sep 7, 2016 at 9:36
>> >> AM,
>> >> > > > > Vladimir
>> >> > > > > >> > > > Rodionov
>> >> > > > > >> > > > > <
>> >> > > > > >> > > > > >> > > > > >> > > >> > vladrodionov@gmail.com>
>> >> > > > > >> > > > > >> > > > > >> > > >> > > wrote:
>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >> I'm not sure that
>> "There
>> >> is
>> >> > > > > already
>> >> > > > > >> > lots
>> >> > > > > >> > > of
>> >> > > > > >> > > > > >> > > half-baked
>> >> > > > > >> > > > > >> > > > > >> code
>> >> > > > > >> > > > > >> > > > > >> > in
>> >> > > > > >> > > > > >> > > > > >> > > the
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > branch,
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > so what's the harm in
>> adding
>> >> > > more?"
>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > I meant - not production
>> -
>> >> > ready
>> >> > > > yet.
>> >> > > > > >> This
>> >> > > > > >> > > is
>> >> > > > > >> > > > > 2.0
>> >> > > > > >> > > > > >> > > > > development
>> >> > > > > >> > > > > >> > > > > >> > > branch
>> >> > > > > >> > > > > >> > > > > >> > > >> > and,
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > hence many features are
>> in
>> >> > works,
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > not being tested well
>> etc.
>> >> I do
>> >> > > not
>> >> > > > > >> > consider
>> >> > > > > >> > > > > backup
>> >> > > > > >> > > > > >> > as
>> >> > > > > >> > > > > >> > > > half
>> >> > > > > >> > > > > >> > > > > >> > baked
>> >> > > > > >> > > > > >> > > > > >> > > >> > > feature -
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > it has passed our
>> internal
>> >> QA
>> >> > and
>> >> > > > has
>> >> > > > > >> very
>> >> > > > > >> > > > good
>> >> > > > > >> > > > > >> doc,
>> >> > > > > >> > > > > >> > > > which
>> >> > > > > >> > > > > >> > > > > we
>> >> > > > > >> > > > > >> > > > > >> > will
>> >> > > > > >> > > > > >> > > > > >> > > >> > > provide
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > to Apache shortly.
>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > -Vlad
>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > On Wed, Sep 7, 2016 at
>> 9:13
>> >> AM,
>> >> > > > > Andrew
>> >> > > > > >> > > > Purtell <
>> >> > > > > >> > > > > >> > > > > >> > > apurtell@apache.org>
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > We shouldn't admit half
>> >> baked
>> >> > > > > changes
>> >> > > > > >> > that
>> >> > > > > >> > > > > won't
>> >> > > > > >> > > > > >> be
>> >> > > > > >> > > > > >> > > > > >> finished.
>> >> > > > > >> > > > > >> > > > > >> > > >> However
>> >> > > > > >> > > > > >> > > > > >> > > >> > > in
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > this case the crew
>> >> working on
>> >> > > > this
>> >> > > > > >> > feature
>> >> > > > > >> > > > are
>> >> > > > > >> > > > > >> long
>> >> > > > > >> > > > > >> > > > > timers
>> >> > > > > >> > > > > >> > > > > >> and
>> >> > > > > >> > > > > >> > > > > >> > > less
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > likely
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > than just about anyone
>> to
>> >> > leave
>> >> > > > > >> > something
>> >> > > > > >> > > > in a
>> >> > > > > >> > > > > >> half
>> >> > > > > >> > > > > >> > > > baked
>> >> > > > > >> > > > > >> > > > > >> > > state. Of
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > course
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > there is no guarantee
>> how
>> >> > > > anything
>> >> > > > > >> will
>> >> > > > > >> > > turn
>> >> > > > > >> > > > > out,
>> >> > > > > >> > > > > >> > > but I
>> >> > > > > >> > > > > >> > > > > am
>> >> > > > > >> > > > > >> > > > > >> > > willing
>> >> > > > > >> > > > > >> > > > > >> > > >> to
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > take
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > a little on faith if
>> they
>> >> > feel
>> >> > > > > their
>> >> > > > > >> > best
>> >> > > > > >> > > > path
>> >> > > > > >> > > > > >> > > forward
>> >> > > > > >> > > > > >> > > > > now
>> >> > > > > >> > > > > >> > > > > >> is
>> >> > > > > >> > > > > >> > > > > >> > to
>> >> > > > > >> > > > > >> > > > > >> > > >> > merge
>> >> > > > > >> > > > > >> > > > > >> > > >> > > to
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > trunk. I only wish I
>> had
>> >> > > > bandwidth
>> >> > > > > to
>> >> > > > > >> > have
>> >> > > > > >> > > > > done
>> >> > > > > >> > > > > >> > some
>> >> > > > > >> > > > > >> > > > real
>> >> > > > > >> > > > > >> > > > > >> > > kicking
>> >> > > > > >> > > > > >> > > > > >> > > >> of
>> >> > > > > >> > > > > >> > > > > >> > > >> > > the
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > tires by now. Maybe
>> this
>> >> > week.
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > (Yes, I'm using some of
>> >> that
>> >> > > time
>> >> > > > > for
>> >> > > > > >> > this
>> >> > > > > >> > > > > email
>> >> > > > > >> > > > > >> > :-)
>> >> > > > > >> > > > > >> > > > but
>> >> > > > > >> > > > > >> > > > > I
>> >> > > > > >> > > > > >> > > > > >> > type
>> >> > > > > >> > > > > >> > > > > >> > > >> > fast.)
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > That said, I would
>> like to
>> >> > > > agitate
>> >> > > > > >> for
>> >> > > > > >> > > > making
>> >> > > > > >> > > > > 2.0
>> >> > > > > >> > > > > >> > > more
>> >> > > > > >> > > > > >> > > > > real
>> >> > > > > >> > > > > >> > > > > >> > and
>> >> > > > > >> > > > > >> > > > > >> > > >> spend
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > some
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > time on it now that I'm
>> >> > winding
>> >> > > > > down
>> >> > > > > >> > with
>> >> > > > > >> > > > > 0.98. I
>> >> > > > > >> > > > > >> > > think
>> >> > > > > >> > > > > >> > > > > >> that
>> >> > > > > >> > > > > >> > > > > >> > > means
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branching for 2.0 real
>> >> soon
>> >> > now
>> >> > > > and
>> >> > > > > >> even
>> >> > > > > >> > > > > evicting
>> >> > > > > >> > > > > >> > > > things
>> >> > > > > >> > > > > >> > > > > >> from
>> >> > > > > >> > > > > >> > > > > >> > > 2.0
>> >> > > > > >> > > > > >> > > > > >> > > >> > > branch
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > that aren't finished or
>> >> > stable,
>> >> > > > > >> leaving
>> >> > > > > >> > > them
>> >> > > > > >> > > > > only
>> >> > > > > >> > > > > >> > > once
>> >> > > > > >> > > > > >> > > > > >> again
>> >> > > > > >> > > > > >> > > > > >> > in
>> >> > > > > >> > > > > >> > > > > >> > > the
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > master
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branch. Or, maybe just
>> >> > evicting
>> >> > > > > them.
>> >> > > > > >> > > Let's
>> >> > > > > >> > > > > take
>> >> > > > > >> > > > > >> it
>> >> > > > > >> > > > > >> > > > case
>> >> > > > > >> > > > > >> > > > > by
>> >> > > > > >> > > > > >> > > > > >> > > case.
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > I think this feature
>> can
>> >> come
>> >> > > in
>> >> > > > > >> > > relatively
>> >> > > > > >> > > > > >> safely.
>> >> > > > > >> > > > > >> > > As
>> >> > > > > >> > > > > >> > > > > >> added
>> >> > > > > >> > > > > >> > > > > >> > > >> > insurance,
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > let's admit the
>> >> possibility
>> >> > it
>> >> > > > > could
>> >> > > > > >> be
>> >> > > > > >> > > > > reverted
>> >> > > > > >> > > > > >> on
>> >> > > > > >> > > > > >> > > the
>> >> > > > > >> > > > > >> > > > > 2.0
>> >> > > > > >> > > > > >> > > > > >> > > branch
>> >> > > > > >> > > > > >> > > > > >> > > >> if
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > folks
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > working on stabilizing
>> 2.0
>> >> > > decide
>> >> > > > > to
>> >> > > > > >> > evict
>> >> > > > > >> > > > it
>> >> > > > > >> > > > > >> > because
>> >> > > > > >> > > > > >> > > > it
>> >> > > > > >> > > > > >> > > > > is
>> >> > > > > >> > > > > >> > > > > >> > > >> > unfinished
>> >> > > > > >> > > > > >> > > > > >> > > >> > > or
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > unstable, because that
>> >> > > certainly
>> >> > > > > can
>> >> > > > > >> > > > happen. I
>> >> > > > > >> > > > > >> > would
>> >> > > > > >> > > > > >> > > > > >> expect if
>> >> > > > > >> > > > > >> > > > > >> > > talk
>> >> > > > > >> > > > > >> > > > > >> > > >> > > like
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > that starts, we'd get
>> help
>> >> > > > > finishing
>> >> > > > > >> or
>> >> > > > > >> > > > > >> stabilizing
>> >> > > > > >> > > > > >> > > > > what's
>> >> > > > > >> > > > > >> > > > > >> > under
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > discussion
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > for revert. Or, we'd
>> have
>> >> a
>> >> > > > revert.
>> >> > > > > >> > Either
>> >> > > > > >> > > > way
>> >> > > > > >> > > > > >> the
>> >> > > > > >> > > > > >> > > > > outcome
>> >> > > > > >> > > > > >> > > > > >> is
>> >> > > > > >> > > > > >> > > > > >> > > >> > > acceptable.
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > On Wed, Sep 7, 2016 at
>> >> 8:56
>> >> > AM,
>> >> > > > > Dima
>> >> > > > > >> > > Spivak
>> >> > > > > >> > > > <
>> >> > > > > >> > > > > >> > > > > >> > > dimaspivak@apache.org
>> >> > > > > >> > > > > >> > > > > >> > > >> >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > I'm not sure that
>> >> "There is
>> >> > > > > already
>> >> > > > > >> > lots
>> >> > > > > >> > > > of
>> >> > > > > >> > > > > >> > > > half-baked
>> >> > > > > >> > > > > >> > > > > >> code
>> >> > > > > >> > > > > >> > > > > >> > in
>> >> > > > > >> > > > > >> > > > > >> > > >> the
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branch,
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > so what's the harm in
>> >> > adding
>> >> > > > > more?"
>> >> > > > > >> > is a
>> >> > > > > >> > > > > good
>> >> > > > > >> > > > > >> > code
>> >> > > > > >> > > > > >> > > > > commit
>> >> > > > > >> > > > > >> > > > > >> > > >> > philosophy
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > for
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > a
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > fault-tolerant
>> >> distributed
>> >> > > data
>> >> > > > > >> store.
>> >> > > > > >> > > ;)
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > More seriously, a
>> lack
>> >> of
>> >> > > test
>> >> > > > > >> > coverage
>> >> > > > > >> > > > for
>> >> > > > > >> > > > > >> > > existing
>> >> > > > > >> > > > > >> > > > > >> > features
>> >> > > > > >> > > > > >> > > > > >> > > >> > > shouldn't
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > be
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > used as justification
>> >> for
>> >> > > > > >> introducing
>> >> > > > > >> > > new
>> >> > > > > >> > > > > >> > features
>> >> > > > > >> > > > > >> > > > with
>> >> > > > > >> > > > > >> > > > > >> the
>> >> > > > > >> > > > > >> > > > > >> > > same
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > shortcomings.
>> >> Ultimately,
>> >> > > it's
>> >> > > > > the
>> >> > > > > >> end
>> >> > > > > >> > > > user
>> >> > > > > >> > > > > who
>> >> > > > > >> > > > > >> > > will
>> >> > > > > >> > > > > >> > > > > feel
>> >> > > > > >> > > > > >> > > > > >> > the
>> >> > > > > >> > > > > >> > > > > >> > > >> pain,
>> >> > > > > >> > > > > >> > > > > >> > > >> > > so
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > shouldn't we do
>> >> everything
>> >> > we
>> >> > > > can
>> >> > > > > >> to
>> >> > > > > >> > > > > mitigate
>> >> > > > > >> > > > > >> > that?
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > -Dima
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > On Wed, Sep 7, 2016
>> at
>> >> 8:46
>> >> > > AM,
>> >> > > > > >> > Vladimir
>> >> > > > > >> > > > > >> > Rodionov <
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > vladrodionov@gmail.com
>> >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > wrote:
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Sean,
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have docs
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Agree. We have a
>> doc
>> >> and
>> >> > > > backup
>> >> > > > > >> is
>> >> > > > > >> > the
>> >> > > > > >> > > > > most
>> >> > > > > >> > > > > >> > > > > documented
>> >> > > > > >> > > > > >> > > > > >> > > feature
>> >> > > > > >> > > > > >> > > > > >> > > >> > :),
>> >> > > > > >> > > > > >> > > > > >> > > >> > > we
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > will
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > release it shortly
>> to
>> >> > > Apache.
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have sunny-day
>> >> > > correctness
>> >> > > > > >> tests
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Feature has  close
>> to
>> >> 60
>> >> > > test
>> >> > > > > >> cases,
>> >> > > > > >> > > > which
>> >> > > > > >> > > > > >> run
>> >> > > > > >> > > > > >> > > for
>> >> > > > > >> > > > > >> > > > > >> approx
>> >> > > > > >> > > > > >> > > > > >> > 30
>> >> > > > > >> > > > > >> > > > > >> > > >> min.
>> >> > > > > >> > > > > >> > > > > >> > > >> > > We
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > can
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > add more, if
>> >> community do
>> >> > > not
>> >> > > > > >> mind
>> >> > > > > >> > :)
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have
>> >> > > > > >> > correctness-in-face-of-failure
>> >> > > > > >> > > > > tests
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Any examples of
>> these
>> >> > tests
>> >> > > > in
>> >> > > > > >> > > existing
>> >> > > > > >> > > > > >> > features?
>> >> > > > > >> > > > > >> > > > In
>> >> > > > > >> > > > > >> > > > > >> > works,
>> >> > > > > >> > > > > >> > > > > >> > > we
>> >> > > > > >> > > > > >> > > > > >> > > >> > > have a
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > clear
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > understanding of
>> what
>> >> > > should
>> >> > > > be
>> >> > > > > >> done
>> >> > > > > >> > > by
>> >> > > > > >> > > > > the
>> >> > > > > >> > > > > >> > time
>> >> > > > > >> > > > > >> > > of
>> >> > > > > >> > > > > >> > > > > 2.0
>> >> > > > > >> > > > > >> > > > > >> > > >> release.
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > That is very close
>> >> goal
>> >> > for
>> >> > > > us,
>> >> > > > > >> to
>> >> > > > > >> > > > verify
>> >> > > > > >> > > > > IT
>> >> > > > > >> > > > > >> > > monkey
>> >> > > > > >> > > > > >> > > > > for
>> >> > > > > >> > > > > >> > > > > >> > > >> existing
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > code.
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * don't rely on
>> things
>> >> > > > outside
>> >> > > > > of
>> >> > > > > >> > > HBase
>> >> > > > > >> > > > > for
>> >> > > > > >> > > > > >> > > normal
>> >> > > > > >> > > > > >> > > > > >> > operation
>> >> > > > > >> > > > > >> > > > > >> > > >> > (okay
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > for
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > advanced operation)
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > We do not.
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Enormous time has
>> been
>> >> > > spent
>> >> > > > > >> already
>> >> > > > > >> > > on
>> >> > > > > >> > > > > the
>> >> > > > > >> > > > > >> > > > > development
>> >> > > > > >> > > > > >> > > > > >> > and
>> >> > > > > >> > > > > >> > > > > >> > > >> > testing
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > the
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > feature, it has
>> passed
>> >> > our
>> >> > > > > >> internal
>> >> > > > > >> > > > tests
>> >> > > > > >> > > > > and
>> >> > > > > >> > > > > >> > > many
>> >> > > > > >> > > > > >> > > > > >> rounds
>> >> > > > > >> > > > > >> > > > > >> > of
>> >> > > > > >> > > > > >> > > > > >> > > >> code
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > reviews
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > by HBase
>> committers.
>> >> We
>> >> > do
>> >> > > > not
>> >> > > > > >> mind
>> >> > > > > >> > if
>> >> > > > > >> > > > > >> someone
>> >> > > > > >> > > > > >> > > from
>> >> > > > > >> > > > > >> > > > > >> HBase
>> >> > > > > >> > > > > >> > > > > >> > > >> > community
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > (outside of HW)
>> will
>> >> > review
>> >> > > > the
>> >> > > > > >> > code,
>> >> > > > > >> > > > but
>> >> > > > > >> > > > > it
>> >> > > > > >> > > > > >> > will
>> >> > > > > >> > > > > >> > > > > >> probably
>> >> > > > > >> > > > > >> > > > > >> > > >> takes
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > forever
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > to
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > wait for
>> volunteer?,
>> >> the
>> >> > > > > feature
>> >> > > > > >> is
>> >> > > > > >> > > > quite
>> >> > > > > >> > > > > >> large
>> >> > > > > >> > > > > >> > > > (1MB+
>> >> > > > > >> > > > > >> > > > > >> > > >> cumulative
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > patch)
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > 2.0 branch is full
>> of
>> >> > half
>> >> > > > > baked
>> >> > > > > >> > > > features,
>> >> > > > > >> > > > > >> most
>> >> > > > > >> > > > > >> > > of
>> >> > > > > >> > > > > >> > > > > them
>> >> > > > > >> > > > > >> > > > > >> > are
>> >> > > > > >> > > > > >> > > > > >> > > in
>> >> > > > > >> > > > > >> > > > > >> > > >> > > active
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > development,
>> >> therefore I
>> >> > am
>> >> > > > not
>> >> > > > > >> > > > following
>> >> > > > > >> > > > > you
>> >> > > > > >> > > > > >> > > here,
>> >> > > > > >> > > > > >> > > > > >> Sean?
>> >> > > > > >> > > > > >> > > > > >> > > Why
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > HBASE-7912
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > is
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > not good enough yet
>> >> to be
>> >> > > > > >> integrated
>> >> > > > > >> > > > into
>> >> > > > > >> > > > > 2.0
>> >> > > > > >> > > > > >> > > > branch?
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > -Vlad
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > On Wed, Sep 7,
>> 2016 at
>> >> > 8:23
>> >> > > > AM,
>> >> > > > > >> Sean
>> >> > > > > >> > > > > Busbey <
>> >> > > > > >> > > > > >> > > > > >> > > busbey@apache.org
>> >> > > > > >> > > > > >> > > > > >> > > >> >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > On Tue, Sep 6,
>> 2016
>> >> at
>> >> > > > 10:36
>> >> > > > > >> PM,
>> >> > > > > >> > > Josh
>> >> > > > > >> > > > > >> Elser <
>> >> > > > > >> > > > > >> > > > > >> > > >> > > josh.elser@gmail.com>
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > wrote:
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > So, the answer
>> to
>> >> > > Sean's
>> >> > > > > >> > original
>> >> > > > > >> > > > > >> question
>> >> > > > > >> > > > > >> > is
>> >> > > > > >> > > > > >> > > > "as
>> >> > > > > >> > > > > >> > > > > >> > > robust as
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > snapshots
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > presently are"?
>> >> > > > > >> (independence of
>> >> > > > > >> > > > > >> > > backup/restore
>> >> > > > > >> > > > > >> > > > > >> > failure
>> >> > > > > >> > > > > >> > > > > >> > > >> > > tolerance
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > from
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > snapshot
>> failure
>> >> > > > tolerance)
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > Is this just a
>> >> > question
>> >> > > > WRT
>> >> > > > > >> > > context
>> >> > > > > >> > > > of
>> >> > > > > >> > > > > >> the
>> >> > > > > >> > > > > >> > > > > change,
>> >> > > > > >> > > > > >> > > > > >> or
>> >> > > > > >> > > > > >> > > > > >> > > is it
>> >> > > > > >> > > > > >> > > > > >> > > >> > > means
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > for a
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > veto
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > from you, Sean?
>> >> Just
>> >> > > > trying
>> >> > > > > >> to
>> >> > > > > >> > > make
>> >> > > > > >> > > > > sure
>> >> > > > > >> > > > > >> > I'm
>> >> > > > > >> > > > > >> > > > > >> following
>> >> > > > > >> > > > > >> > > > > >> > > >> along
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > adequately.
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > I'd say ATM I'm
>> -0,
>> >> > > > bordering
>> >> > > > > >> on
>> >> > > > > >> > -1
>> >> > > > > >> > > > but
>> >> > > > > >> > > > > not
>> >> > > > > >> > > > > >> > for
>> >> > > > > >> > > > > >> > > > > >> reasons
>> >> > > > > >> > > > > >> > > > > >> > I
>> >> > > > > >> > > > > >> > > > > >> > > can
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > articulate
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > well.
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Here's an
>> attempt.
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > We've been
>> trying to
>> >> > > move,
>> >> > > > > as a
>> >> > > > > >> > > > > community,
>> >> > > > > >> > > > > >> > > > towards
>> >> > > > > >> > > > > >> > > > > >> > > minimizing
>> >> > > > > >> > > > > >> > > > > >> > > >> > > risk
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > to
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > downstream folks
>> by
>> >> > > getting
>> >> > > > > >> > > "complete
>> >> > > > > >> > > > > >> enough
>> >> > > > > >> > > > > >> > > for
>> >> > > > > >> > > > > >> > > > > use"
>> >> > > > > >> > > > > >> > > > > >> > > gates
>> >> > > > > >> > > > > >> > > > > >> > > >> in
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > place
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > before we
>> introduce
>> >> new
>> >> > > > > >> features.
>> >> > > > > >> > > This
>> >> > > > > >> > > > > was
>> >> > > > > >> > > > > >> > > > spurred
>> >> > > > > >> > > > > >> > > > > >> by a
>> >> > > > > >> > > > > >> > > > > >> > > some
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > features
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > getting in
>> >> half-baked
>> >> > and
>> >> > > > > never
>> >> > > > > >> > > making
>> >> > > > > >> > > > > it
>> >> > > > > >> > > > > >> to
>> >> > > > > >> > > > > >> > > "can
>> >> > > > > >> > > > > >> > > > > >> really
>> >> > > > > >> > > > > >> > > > > >> > > use"
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > status
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > (I'm thinking of
>> >> > > > distributed
>> >> > > > > >> log
>> >> > > > > >> > > > replay
>> >> > > > > >> > > > > and
>> >> > > > > >> > > > > >> > the
>> >> > > > > >> > > > > >> > > > > >> zk-less
>> >> > > > > >> > > > > >> > > > > >> > > >> > > assignment
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > stuff, I don't
>> >> recall
>> >> > if
>> >> > > > > there
>> >> > > > > >> was
>> >> > > > > >> > > > > more).
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The gates,
>> >> generally,
>> >> > > > > included
>> >> > > > > >> > > things
>> >> > > > > >> > > > > like:
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have docs
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have sunny-day
>> >> > > > correctness
>> >> > > > > >> tests
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have
>> >> > > > > >> > > correctness-in-face-of-failure
>> >> > > > > >> > > > > tests
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't rely on
>> >> things
>> >> > > > > outside
>> >> > > > > >> of
>> >> > > > > >> > > > HBase
>> >> > > > > >> > > > > for
>> >> > > > > >> > > > > >> > > > normal
>> >> > > > > >> > > > > >> > > > > >> > > operation
>> >> > > > > >> > > > > >> > > > > >> > > >> > > (okay
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > for
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > advanced
>> operation)
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > As an example, we
>> >> kept
>> >> > > the
>> >> > > > > MOB
>> >> > > > > >> > work
>> >> > > > > >> > > > off
>> >> > > > > >> > > > > in
>> >> > > > > >> > > > > >> a
>> >> > > > > >> > > > > >> > > > branch
>> >> > > > > >> > > > > >> > > > > >> and
>> >> > > > > >> > > > > >> > > > > >> > > out
>> >> > > > > >> > > > > >> > > > > >> > > >> of
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > master
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > until it could
>> pass
>> >> > these
>> >> > > > > >> > criteria.
>> >> > > > > >> > > > The
>> >> > > > > >> > > > > big
>> >> > > > > >> > > > > >> > > > > exemption
>> >> > > > > >> > > > > >> > > > > >> > > we've
>> >> > > > > >> > > > > >> > > > > >> > > >> had
>> >> > > > > >> > > > > >> > > > > >> > > >> > > to
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > this was the
>> >> > hbase-spark
>> >> > > > > >> > > integration,
>> >> > > > > >> > > > > where
>> >> > > > > >> > > > > >> > we
>> >> > > > > >> > > > > >> > > > all
>> >> > > > > >> > > > > >> > > > > >> > agreed
>> >> > > > > >> > > > > >> > > > > >> > > it
>> >> > > > > >> > > > > >> > > > > >> > > >> > > could
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > land in master
>> >> because
>> >> > it
>> >> > > > was
>> >> > > > > >> very
>> >> > > > > >> > > > well
>> >> > > > > >> > > > > >> > > isolated
>> >> > > > > >> > > > > >> > > > > (the
>> >> > > > > >> > > > > >> > > > > >> > > slide
>> >> > > > > >> > > > > >> > > > > >> > > >> > away
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > from
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > including docs
>> as a
>> >> > > > > first-class
>> >> > > > > >> > part
>> >> > > > > >> > > > of
>> >> > > > > >> > > > > >> > > building
>> >> > > > > >> > > > > >> > > > up
>> >> > > > > >> > > > > >> > > > > >> that
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > integration
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > has led me to
>> doubt
>> >> the
>> >> > > > > wisdom
>> >> > > > > >> of
>> >> > > > > >> > > this
>> >> > > > > >> > > > > >> > > decision).
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > We've also been
>> >> > treating
>> >> > > > > >> inclusion
>> >> > > > > >> > > in
>> >> > > > > >> > > > a
>> >> > > > > >> > > > > >> > > "probably
>> >> > > > > >> > > > > >> > > > > >> will
>> >> > > > > >> > > > > >> > > > > >> > be
>> >> > > > > >> > > > > >> > > > > >> > > >> > > released
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > to
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > downstream"
>> branches
>> >> > as a
>> >> > > > > >> higher
>> >> > > > > >> > > bar,
>> >> > > > > >> > > > > >> > requiring
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't
>> moderately
>> >> > impact
>> >> > > > > >> > > performance
>> >> > > > > >> > > > > when
>> >> > > > > >> > > > > >> > the
>> >> > > > > >> > > > > >> > > > > >> feature
>> >> > > > > >> > > > > >> > > > > >> > > isn't
>> >> > > > > >> > > > > >> > > > > >> > > >> in
>> >> > > > > >> > > > > >> > > > > >> > > >> > > use
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't severely
>> >> impact
>> >> > > > > >> > performance
>> >> > > > > >> > > > when
>> >> > > > > >> > > > > >> the
>> >> > > > > >> > > > > >> > > > > feature
>> >> > > > > >> > > > > >> > > > > >> is
>> >> > > > > >> > > > > >> > > > > >> > in
>> >> > > > > >> > > > > >> > > > > >> > > >> use
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * either
>> >> default-to-on
>> >> > or
>> >> > > > > show
>> >> > > > > >> > > enough
>> >> > > > > >> > > > > >> demand
>> >> > > > > >> > > > > >> > to
>> >> > > > > >> > > > > >> > > > > >> believe
>> >> > > > > >> > > > > >> > > > > >> > a
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > non-trivial
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > number of folks
>> will
>> >> > turn
>> >> > > > the
>> >> > > > > >> > > feature
>> >> > > > > >> > > > on
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The above has
>> kept
>> >> MOB
>> >> > > and
>> >> > > > > >> > > hbase-spark
>> >> > > > > >> > > > > >> > > > integration
>> >> > > > > >> > > > > >> > > > > >> out
>> >> > > > > >> > > > > >> > > > > >> > of
>> >> > > > > >> > > > > >> > > > > >> > > >> > > branch-1,
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > presumably while
>> >> > they've
>> >> > > > > >> "gotten
>> >> > > > > >> > > more
>> >> > > > > >> > > > > >> stable"
>> >> > > > > >> > > > > >> > > in
>> >> > > > > >> > > > > >> > > > > >> master
>> >> > > > > >> > > > > >> > > > > >> > > from
>> >> > > > > >> > > > > >> > > > > >> > > >> > the
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > odd
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > vendor inclusion.
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Are we going to
>> >> have a
>> >> > > 2.0
>> >> > > > > >> release
>> >> > > > > >> > > > > before
>> >> > > > > >> > > > > >> the
>> >> > > > > >> > > > > >> > > end
>> >> > > > > >> > > > > >> > > > > of
>> >> > > > > >> > > > > >> > > > > >> the
>> >> > > > > >> > > > > >> > > > > >> > > >> year?
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > We're
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > coming up on 1.5
>> >> years
>> >> > > > since
>> >> > > > > >> the
>> >> > > > > >> > > > > release of
>> >> > > > > >> > > > > >> > > > version
>> >> > > > > >> > > > > >> > > > > >> 1.0;
>> >> > > > > >> > > > > >> > > > > >> > > >> seems
>> >> > > > > >> > > > > >> > > > > >> > > >> > > like
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > it's about time,
>> >> > though I
>> >> > > > > >> haven't
>> >> > > > > >> > > seen
>> >> > > > > >> > > > > any
>> >> > > > > >> > > > > >> > > > concrete
>> >> > > > > >> > > > > >> > > > > >> > plans
>> >> > > > > >> > > > > >> > > > > >> > > >> this
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > year.
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Presuming we are
>> >> going
>> >> > to
>> >> > > > > have
>> >> > > > > >> one
>> >> > > > > >> > > by
>> >> > > > > >> > > > > the
>> >> > > > > >> > > > > >> end
>> >> > > > > >> > > > > >> > > of
>> >> > > > > >> > > > > >> > > > > the
>> >> > > > > >> > > > > >> > > > > >> > > year, it
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > seems a
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > bit close to
>> still
>> >> be
>> >> > > > adding
>> >> > > > > in
>> >> > > > > >> > > > > "features
>> >> > > > > >> > > > > >> > that
>> >> > > > > >> > > > > >> > > > need
>> >> > > > > >> > > > > >> > > > > >> > > maturing"
>> >> > > > > >> > > > > >> > > > > >> > > >> > on
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > the
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > branch.
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The lack of a
>> >> concrete
>> >> > > plan
>> >> > > > > for
>> >> > > > > >> > 2.0
>> >> > > > > >> > > > > keeps
>> >> > > > > >> > > > > >> me
>> >> > > > > >> > > > > >> > > from
>> >> > > > > >> > > > > >> > > > > >> > > considering
>> >> > > > > >> > > > > >> > > > > >> > > >> > > these
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > things blocker at
>> >> the
>> >> > > > moment.
>> >> > > > > >> But
>> >> > > > > >> > I
>> >> > > > > >> > > > know
>> >> > > > > >> > > > > >> > first
>> >> > > > > >> > > > > >> > > > hand
>> >> > > > > >> > > > > >> > > > > >> how
>> >> > > > > >> > > > > >> > > > > >> > > much
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > trouble
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > folks have had
>> with
>> >> > other
>> >> > > > > >> features
>> >> > > > > >> > > > that
>> >> > > > > >> > > > > >> have
>> >> > > > > >> > > > > >> > > gone
>> >> > > > > >> > > > > >> > > > > >> into
>> >> > > > > >> > > > > >> > > > > >> > > >> > downstream
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > facing releases
>> >> without
>> >> > > > > >> robustness
>> >> > > > > >> > > > > checks
>> >> > > > > >> > > > > >> > (i.e.
>> >> > > > > >> > > > > >> > > > > >> > > replication),
>> >> > > > > >> > > > > >> > > > > >> > > >> > and
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > I'm
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > concerned about
>> what
>> >> > > we're
>> >> > > > > >> setting
>> >> > > > > >> > > up
>> >> > > > > >> > > > if
>> >> > > > > >> > > > > >> 2.0
>> >> > > > > >> > > > > >> > > goes
>> >> > > > > >> > > > > >> > > > > out
>> >> > > > > >> > > > > >> > > > > >> > with
>> >> > > > > >> > > > > >> > > > > >> > > >> this
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > feature in its
>> >> current
>> >> > > > state.
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > --
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > Best regards,
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >    - Andy
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > Problems worthy of
>> attack
>> >> > prove
>> >> > > > > their
>> >> > > > > >> > > worth
>> >> > > > > >> > > > by
>> >> > > > > >> > > > > >> > > hitting
>> >> > > > > >> > > > > >> > > > > >> back. -
>> >> > > > > >> > > > > >> > > > > >> > > Piet
>> >> > > > > >> > > > > >> > > > > >> > > >> > > Hein
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > > (via Tom White)
>> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > --
>> >> > > > > >> > > > > >> > > > > >> > > >> > > Best regards,
>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > >    - Andy
>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>> >> > > > > >> > > > > >> > > > > >> > > >> > > Problems worthy of attack
>> >> prove
>> >> > > their
>> >> > > > > >> worth
>> >> > > > > >> > by
>> >> > > > > >> > > > > >> hitting
>> >> > > > > >> > > > > >> > > > back.
>> >> > > > > >> > > > > >> > > > > -
>> >> > > > > >> > > > > >> > > > > >> > Piet
>> >> > > > > >> > > > > >> > > > > >> > > >> Hein
>> >> > > > > >> > > > > >> > > > > >> > > >> > > (via Tom White)
>> >> > > > > >> > > > > >> > > > > >> > > >> > >
>> >> > > > > >> > > > > >> > > > > >> > > >> >
>> >> > > > > >> > > > > >> > > > > >> > > >>
>> >> > > > > >> > > > > >> > > > > >> > >
>> >> > > > > >> > > > > >> > > > > >> >
>> >> > > > > >> > > > > >> > > > > >>
>> >> > > > > >> > > > > >> > > > > >
>> >> > > > > >> > > > > >> > > > > >
>> >> > > > > >> > > > > >> > > > >
>> >> > > > > >> > > > > >> > > >
>> >> > > > > >> > > > > >> > >
>> >> > > > > >> > > > > >> >
>> >> > > > > >> > > > > >>
>> >> > > > > >> > > > >
>> >> > > > > >> > > > >
>> >> > > > > >> > > > >
>> >> > > > > >> > > > > --
>> >> > > > > >> > > > > busbey
>> >> > > > > >> > > > >
>> >> > > > > >> > > >
>> >> > > > > >> > >
>> >> > > > > >> >
>> >> > > > > >>
>> >> > > > > >
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >>
>> >> -- Appy
>> >>
>> >
>> >
>>
>
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Vladimir Rodionov <vl...@gmail.com>]

Michael,

you can try master + latest patch on HBASE-14123 (v29). No need to use
HBASE-7912 branch. I will double check HBASE-7912.

-Vlad

On Wed, Oct 12, 2016 at 12:31 PM, Stack <st...@duboce.net> wrote:

> More info:
>
> stack@ve0524:~/hbase.git$ git checkout origin/HBASE-7912 -b 7912v2
> Branch 7912v2 set up to track remote branch HBASE-7912 from origin.
> Switched to a new branch '7912v2'
> stack@ve0524:~/hbase.git$ java -version
> java version "1.8.0_101"
> Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
> Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode)
> stack@ve0524:~/hbase.git$ mvn clean install -DskipTests &> /tmp/out.txt
>
> ...
>
> St.Ack
>
>
> On Wed, Oct 12, 2016 at 12:29 PM, Stack <st...@duboce.net> wrote:
>
> > Interesting. When I try it fails w/ below:
> >
> > [INFO] 26 warnings
> > 322 [INFO] -------------------------------------------------------------
> > 323 [INFO] -------------------------------------------------------------
> > 324 [ERROR] COMPILATION ERROR :
> > 325 [INFO] -------------------------------------------------------------
> > 326 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.java:[48,8]
> > org.apache.hadoop.hbase.io.encoding.rowindexV2.RowIndexCodecV2 is not
> > abstract and does not override abstract method createSeeker(org.ap
> >  ache.hadoop.hbase.CellComparator,org.apache.hadoop.hbase.io.encoding.
> HFileBlockDecodingContext)
> > in org.apache.hadoop.hbase.io.encoding.DataBlockEncoder
> > 327 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.java:[143,3]
> > method does not override or implement a method from a supertype
> > 328 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.java:[147,29]
> > incompatible types: java.nio.ByteBuffer cannot be converted to
> > org.apache.hadoop.hbase.nio.ByteBuff
> > 329 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.java:[148,33]
> > cannot find symbol
> > 330   symbol:   method getKeyDeepCopy()
> > 331   location: variable seeker of type org.apache.hadoop.hbase.io.
> > encoding.DataBlockEncoder.EncodedSeeker
> > 332 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.java:[153,3]
> > method does not override or implement a method from a supertype
> > 333 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> > apache/hadoop/hbase/io/encoding/rowindexV1/RowIndexCodecV1.java:[45,8]
> > org.apache.hadoop.hbase.io.encoding.rowindexV1.RowIndexCodecV1 is not
> > abstract and does not override abstract method createSeeker(org.ap
> >  ache.hadoop.hbase.CellComparator,org.apache.hadoop.hbase.io.encoding.
> HFileBlockDecodingContext)
> > in org.apache.hadoop.hbase.io.encoding.DataBlockEncoder
> > 334 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> > apache/hadoop/hbase/io/encoding/rowindexV1/RowIndexCodecV1.java:[145,3]
> > method does not override or implement a method from a supertype
> > 335 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> > apache/hadoop/hbase/io/encoding/rowindexV1/RowIndexCodecV1.java:[158,3]
> > method does not override or implement a method from a supertype
> > 336 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.java:[46,8]
> > org.apache.hadoop.hbase.io.encoding.rowindexV2.RowIndexSeekerV2 is not
> > abstract and does not override abstract method compareKey(org.ap
> >  ache.hadoop.hbase.CellComparator,org.apache.hadoop.hbase.Cell) in
> > org.apache.hadoop.hbase.io.encoding.DataBlockEncoder.EncodedSeeker
> > 337 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.java:[79,3]
> > method does not override or implement a method from a supertype
> > 338 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.java:[117,3]
> > method does not override or implement a method from a supertype
> > 339 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.java:[190,3]
> > method does not override or implement a method from a supertype
> > 340 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.java:[214,3]
> > method does not override or implement a method from a supertype
> > 341 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.java:[349,3]
> > method does not override or implement a method from a supertype
> > 342 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> > apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.java:[355,3]
> > method does not override or implement a method from a supertype
> > 343 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> > apache/hadoop/hbase/io/encoding/rowindexV2/
> RowIndexSeekerV2.java:[421,36]
> > no suitable method found for uncompressTags(java.nio.
> > ByteBuffer,byte[],int,int)
> > 344     method org.apache.hadoop.hbase.io.TagCompressionContext.
> > uncompressTags(java.io.InputStream,byte[],int,int) is not applicable
> > 345       (argument mismatch; java.nio.ByteBuffer cannot be converted to
> > java.io.InputStream)
> > 346     method org.apache.hadoop.hbase.io.TagCompressionContext.
> > uncompressTags(org.apache.hadoop.hbase.nio.ByteBuff,byte[],int,int) is
> > not applicable
> > 347       (argument mismatch; java.nio.ByteBuffer cannot be converted to
> > org.apache.hadoop.hbase.nio.ByteBuff)
> >
> > ....
> >
> > St.Ack
> >
> > On Tue, Oct 11, 2016 at 5:12 PM, Apekshit Sharma <ap...@cloudera.com>
> > wrote:
> >
> >> @stack, it compiled for me.
> >>
> >> Also tried few commands, and have to say, it's well designed from user
> >> commands perspective.
> >>
> >> On Tue, Oct 11, 2016 at 5:08 PM, Stack <st...@duboce.net> wrote:
> >>
> >> > On Tue, Oct 11, 2016 at 1:50 PM, Vladimir Rodionov <
> >> vladrodionov@gmail.com
> >> > >
> >> > wrote:
> >> >
> >> > > Michael,
> >> > >
> >> > > Its in HBASE-7912
> >> > >
> >> > > This is tip of git log:
> >> > >
> >> > > commit a072f6f49a26a7259ff2aaef6cb56d85eb592482
> >> > > Author: Frank Welsch <fw...@jps.net>
> >> > > Date:   Fri Sep 23 18:00:42 2016 -0400
> >> > >
> >> > >     HBASE-16574 Book updates for backup and restore
> >> > >
> >> > > commit b14e2ab1c24e65ff88dd4c579acf83cb4ed0605e
> >> > > Author: tedyu <yu...@gmail.com>
> >> > > Date:   Wed Oct 5 16:29:40 2016 -0700
> >> > >
> >> > >     HBASE-16727 Backup refactoring: remove MR dependencies from
> >> HMaster
> >> > > (Vladimir Rodionov)
> >> > >
> >> > >
> >> > Thanks. I have that. I tried it and it doesn't compile for me. Does it
> >> > compile for you?
> >> > Thanks,
> >> > M
> >> >
> >> >
> >> >
> >> > > -Vlad
> >> > >
> >> > > On Tue, Oct 11, 2016 at 1:44 PM, Stack <st...@duboce.net> wrote:
> >> > >
> >> > > > Which branch do I check out to try it? HBASE-7912 is not it. I
> don't
> >> > see
> >> > > an
> >> > > > HBASE-16727...
> >> > > > Thanks,
> >> > > > M
> >> > > >
> >> > > >
> >> > > > On Tue, Oct 11, 2016 at 11:41 AM, Vladimir Rodionov <
> >> > > > vladrodionov@gmail.com>
> >> > > > wrote:
> >> > > >
> >> > > > > The last patch is on review board:
> >> > > > > https://reviews.apache.org/r/52748
> >> > > > >
> >> > > > > On Mon, Oct 10, 2016 at 3:52 PM, Vladimir Rodionov <
> >> > > > vladrodionov@gmail.com
> >> > > > > >
> >> > > > > wrote:
> >> > > > >
> >> > > > > > >> How hard to put in an hbase-backup module? hbase-server is
> >> fat
> >> > > > enough
> >> > > > > > >> already. Could be done as a follow-up.
> >> > > > > >
> >> > > > > > https://issues.apache.org/jira/browse/HBASE-16727?
> >> > > > > > focusedCommentId=15531237&page=com.atlassian.jira.
> >> > > > > > plugin.system.issuetabpanels:comment-tabpanel#comment-
> 15531237
> >> > > > > >
> >> > > > > > Can we do merge first? Then we can discuss separate module.
> >> > > > > >
> >> > > > > > -Vlad
> >> > > > > >
> >> > > > > > On Mon, Oct 10, 2016 at 3:44 PM, Ted Yu <yu...@gmail.com>
> >> > wrote:
> >> > > > > >
> >> > > > > >> Looks like the first quote was cut off.
> >> > > > > >> The original sentence was:
> >> > > > > >>
> >> > > > > >> bq. no mapreduce job launched from master or region server.
> >> > > > > >>
> >> > > > > >> mapreduce job is launched from the node where command line
> >> tool is
> >> > > > run.
> >> > > > > >>
> >> > > > > >> On Mon, Oct 10, 2016 at 3:38 PM, Stack <st...@duboce.net>
> >> wrote:
> >> > > > > >>
> >> > > > > >> > bq. launched from master or region server.
> >> > > > > >> >
> >> > > > > >> > What does this mean please? Has to be run from Master or
> >> > > > RegionServer?
> >> > > > > >> Can
> >> > > > > >> > it be run from another node altogether?
> >> > > > > >> >
> >> > > > > >> > On Mon, Oct 10, 2016 at 1:44 PM, Vladimir Rodionov <
> >> > > > > >> vladrodionov@gmail.com
> >> > > > > >> > >
> >> > > > > >> > wrote:
> >> > > > > >> >
> >> > > > > >> > > >> mapreduce dependency has been moved to client side -
> no
> >> > > > mapreduce
> >> > > > > >> job
> >> > > > > >> > >
> >> > > > > >> > > 1. We have no code in the client module anymore, due to
> >> > > dependency
> >> > > > > on
> >> > > > > >> > > internal server API (HFile and WAL access).
> >> > > > > >> > > 2. Backup/ restore are client - driven operations, but
> all
> >> the
> >> > > > code
> >> > > > > >> > resides
> >> > > > > >> > > in the server module
> >> > > > > >> > >
> >> > > > > >> >
> >> > > > > >> > How hard to put in an hbase-backup module? hbase-server is
> >> fat
> >> > > > enough
> >> > > > > >> > already. Could be done as a follow-up.
> >> > > > > >> >
> >> > > > > >> > Thanks,
> >> > > > > >> > St.Ack
> >> > > > > >> >
> >> > > > > >> >
> >> > > > > >> >
> >> > > > > >> > > 3. No MR in Master, no procedure - driven execution.
> >> > > > > >> > > 4. Old good MR from command-line.
> >> > > > > >> > > 5. Security was simplified and now only super-user is
> >> allowed
> >> > to
> >> > > > run
> >> > > > > >> > > backup/restores.
> >> > > > > >> > > 6. HBase Backup API was gone due to 1. Now only
> >> command-line
> >> > > > access
> >> > > > > to
> >> > > > > >> > > backup tools.
> >> > > > > >> > >
> >> > > > > >> > > These consequences of refactoring has been discussed in
> >> > > > HBASE-16727.
> >> > > > > >> > >
> >> > > > > >> > > -Vlad
> >> > > > > >> > >
> >> > > > > >> > >
> >> > > > > >> > >
> >> > > > > >> > >
> >> > > > > >> > > On Mon, Oct 10, 2016 at 1:31 PM, Ted Yu <
> >> yuzhihong@gmail.com>
> >> > > > > wrote:
> >> > > > > >> > >
> >> > > > > >> > > > Reviving this thread.
> >> > > > > >> > > >
> >> > > > > >> > > > The following has taken place:
> >> > > > > >> > > >
> >> > > > > >> > > > mapreduce dependency has been moved to client side - no
> >> > > > mapreduce
> >> > > > > >> job
> >> > > > > >> > > > launched from master or region server.
> >> > > > > >> > > > document patch (HBASE-16574) has been integrated.
> >> > > > > >> > > > Updated mega patch has been attached to HBASE-14123:
> this
> >> > > covers
> >> > > > > the
> >> > > > > >> > > > refactor in #1 above and the protobuf 3 merge.
> >> > > > > >> > > >
> >> > > > > >> > > > If community has more feedback on the merge proposal, I
> >> > would
> >> > > > love
> >> > > > > >> to
> >> > > > > >> > > hear
> >> > > > > >> > > > it.
> >> > > > > >> > > >
> >> > > > > >> > > > Thanks
> >> > > > > >> > > >
> >> > > > > >> > > > On Thu, Sep 22, 2016 at 10:31 AM, Sean Busbey <
> >> > > > > busbey@cloudera.com>
> >> > > > > >> > > wrote:
> >> > > > > >> > > >
> >> > > > > >> > > > > I'd like to see the docs proposed on HBASE-16574
> >> > integrated
> >> > > > into
> >> > > > > >> our
> >> > > > > >> > > > > project's documentation prior to merge.
> >> > > > > >> > > > >
> >> > > > > >> > > > > On Thu, Sep 22, 2016 at 9:02 AM, Ted Yu <
> >> > > yuzhihong@gmail.com>
> >> > > > > >> wrote:
> >> > > > > >> > > > > > This feature can be marked experimental due to some
> >> > > > > limitations
> >> > > > > >> > such
> >> > > > > >> > > as
> >> > > > > >> > > > > > security.
> >> > > > > >> > > > > >
> >> > > > > >> > > > > > Your previous round of comments have been
> addressed.
> >> > > > > >> > > > > > Command line tool has gone through:
> >> > > > > >> > > > > >
> >> > > > > >> > > > > > HBASE-16620 Fix backup command-line tool usability
> >> > issues
> >> > > > > >> > > > > > HBASE-16655 hbase backup describe with incorrect
> >> backup
> >> > id
> >> > > > > >> results
> >> > > > > >> > in
> >> > > > > >> > > > NPE
> >> > > > > >> > > > > >
> >> > > > > >> > > > > > The updated doc has been attached to HBASE-16574.
> >> > > > > >> > > > > >
> >> > > > > >> > > > > > Cheers
> >> > > > > >> > > > > >
> >> > > > > >> > > > > > On Thu, Sep 22, 2016 at 8:53 AM, Stack <
> >> > stack@duboce.net>
> >> > > > > >> wrote:
> >> > > > > >> > > > > >
> >> > > > > >> > > > > >> On Wed, Sep 21, 2016 at 7:43 AM, Ted Yu <
> >> > > > yuzhihong@gmail.com
> >> > > > > >
> >> > > > > >> > > wrote:
> >> > > > > >> > > > > >>
> >> > > > > >> > > > > >> > Are there more (review) comments ?
> >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> Are outstanding comments addressed?
> >> > > > > >> > > > > >>
> >> > > > > >> > > > > >> I don't see answer to my 'is this
> experimental/will
> >> it
> >> > be
> >> > > > > >> marked
> >> > > > > >> > > > > >> experimental' question.
> >> > > > > >> > > > > >>
> >> > > > > >> > > > > >> I ran into some issues trying to use the feature
> and
> >> > > > > suggested
> >> > > > > >> > that
> >> > > > > >> > > a
> >> > > > > >> > > > > >> feature likes this needs polish else it'll just
> rot,
> >> > > > unused.
> >> > > > > >> Has
> >> > > > > >> > > > polish
> >> > > > > >> > > > > >> been applied? All ready for another 'user' test?
> >> > Suggest
> >> > > > that
> >> > > > > >> you
> >> > > > > >> > > > update
> >> > > > > >> > > > > >> here going forward for the benefit of those trying
> >> to
> >> > > > follow
> >> > > > > >> along
> >> > > > > >> > > and
> >> > > > > >> > > > > who
> >> > > > > >> > > > > >> are not watching JIRA change fly-by.
> >> > > > > >> > > > > >>
> >> > > > > >> > > > > >> It looks like doc got a revision -- I have to
> check
> >> --
> >> > to
> >> > > > > take
> >> > > > > >> on
> >> > > > > >> > > > > >> suggestion made above but again, suggest, that
> this
> >> > > thread
> >> > > > > gets
> >> > > > > >> > > > updated.
> >> > > > > >> > > > > >>
> >> > > > > >> > > > > >> Thanks,
> >> > > > > >> > > > > >> St.Ack
> >> > > > > >> > > > > >>
> >> > > > > >> > > > > >>
> >> > > > > >> > > > > >>
> >> > > > > >> > > > > >> > Thanks
> >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > On Tue, Sep 20, 2016 at 10:02 AM, Devaraj Das <
> >> > > > > >> > > ddas@hortonworks.com
> >> > > > > >> > > > >
> >> > > > > >> > > > > >> > wrote:
> >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > > Just reviving this thread. Thanks Sean, Stack,
> >> > Dima,
> >> > > > and
> >> > > > > >> > others
> >> > > > > >> > > > for
> >> > > > > >> > > > > the
> >> > > > > >> > > > > >> > > thorough reviews and testing. Thanks Ted and
> >> Vlad
> >> > for
> >> > > > > >> taking
> >> > > > > >> > > care
> >> > > > > >> > > > of
> >> > > > > >> > > > > >> the
> >> > > > > >> > > > > >> > > feedback. Are we all good to do the merge now?
> >> > Rather
> >> > > > do
> >> > > > > >> > sooner
> >> > > > > >> > > > than
> >> > > > > >> > > > > >> > later.
> >> > > > > >> > > > > >> > > ________________________________________
> >> > > > > >> > > > > >> > > From: saint.ack@gmail.com <
> saint.ack@gmail.com>
> >> on
> >> > > > > behalf
> >> > > > > >> of
> >> > > > > >> > > > Stack
> >> > > > > >> > > > > <
> >> > > > > >> > > > > >> > > stack@duboce.net>
> >> > > > > >> > > > > >> > > Sent: Monday, September 12, 2016 1:18 PM
> >> > > > > >> > > > > >> > > To: HBase Dev List
> >> > > > > >> > > > > >> > > Subject: Re: [DISCUSSION] Merge Backup /
> >> Restore -
> >> > > > Branch
> >> > > > > >> > > > HBASE-7912
> >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> > > On Mon, Sep 12, 2016 at 12:19 PM, Ted Yu <
> >> > > > > >> yuzhihong@gmail.com
> >> > > > > >> > >
> >> > > > > >> > > > > wrote:
> >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> > > > Mega patch (rev 18) is on HBASE-14123.
> >> > > > > >> > > > > >> > > >
> >> > > > > >> > > > > >> > > > Please comment on HBASE-14123 on how you
> want
> >> to
> >> > > > > review.
> >> > > > > >> > > > > >> > > >
> >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> > > Yeah. That was my lost tab. Last rb was 6
> months
> >> > ago.
> >> > > > > >> Suggest
> >> > > > > >> > > > > updating
> >> > > > > >> > > > > >> > it.
> >> > > > > >> > > > > >> > > RB is pretty good for review. Patch is only
> >> 1.5M so
> >> > > > > should
> >> > > > > >> be
> >> > > > > >> > > > fine.
> >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> > > St.Ack
> >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> > > >
> >> > > > > >> > > > > >> > > > Thanks
> >> > > > > >> > > > > >> > > >
> >> > > > > >> > > > > >> > > > On Mon, Sep 12, 2016 at 12:15 PM, Stack <
> >> > > > > >> stack@duboce.net>
> >> > > > > >> > > > wrote:
> >> > > > > >> > > > > >> > > >
> >> > > > > >> > > > > >> > > > > On review of the 'patch', do I just
> compare
> >> the
> >> > > > > branch
> >> > > > > >> to
> >> > > > > >> > > > > master or
> >> > > > > >> > > > > >> > is
> >> > > > > >> > > > > >> > > > > there a megapatch posted somewhere (I
> think
> >> I
> >> > saw
> >> > > > one
> >> > > > > >> but
> >> > > > > >> > it
> >> > > > > >> > > > > seemed
> >> > > > > >> > > > > >> > > stale
> >> > > > > >> > > > > >> > > > > and then I 'lost' the tab). Sorry for dumb
> >> > > > question.
> >> > > > > >> > > > > >> > > > > St.Ack
> >> > > > > >> > > > > >> > > > >
> >> > > > > >> > > > > >> > > > > On Mon, Sep 12, 2016 at 12:01 PM, Stack <
> >> > > > > >> stack@duboce.net
> >> > > > > >> > >
> >> > > > > >> > > > > wrote:
> >> > > > > >> > > > > >> > > > >
> >> > > > > >> > > > > >> > > > > > Late to the game. A few comments after
> >> > > rereading
> >> > > > > this
> >> > > > > >> > > thread
> >> > > > > >> > > > > as a
> >> > > > > >> > > > > >> > > > 'user'.
> >> > > > > >> > > > > >> > > > > >
> >> > > > > >> > > > > >> > > > > > + Before merge, a user-facing feature
> like
> >> > this
> >> > > > > >> should
> >> > > > > >> > > work
> >> > > > > >> > > > > (If
> >> > > > > >> > > > > >> > this
> >> > > > > >> > > > > >> > > is
> >> > > > > >> > > > > >> > > > > "higher-bar
> >> > > > > >> > > > > >> > > > > > for new features", bring it on --
> smile).
> >> > > > > >> > > > > >> > > > > > + As a user, I tried the branch with
> tools
> >> > > after
> >> > > > > >> > reviewing
> >> > > > > >> > > > the
> >> > > > > >> > > > > >> > > > > just-posted
> >> > > > > >> > > > > >> > > > > > doc. I had an 'interesting' experience
> >> (left
> >> > > > > >> comments up
> >> > > > > >> > > on
> >> > > > > >> > > > > >> > issue). I
> >> > > > > >> > > > > >> > > > > think
> >> > > > > >> > > > > >> > > > > > the tooling/doc. important to get right.
> >> If
> >> > it
> >> > > > > breaks
> >> > > > > >> > > easily
> >> > > > > >> > > > > or
> >> > > > > >> > > > > >> is
> >> > > > > >> > > > > >> > > > > > inconsistent (or lacks 'polish'),
> >> operators
> >> > > will
> >> > > > > >> judge
> >> > > > > >> > the
> >> > > > > >> > > > > whole
> >> > > > > >> > > > > >> > > > > > backup/restore tooling chain as not
> >> > trustworthy
> >> > > > and
> >> > > > > >> > > abandon
> >> > > > > >> > > > > it.
> >> > > > > >> > > > > >> > Lets
> >> > > > > >> > > > > >> > > > not
> >> > > > > >> > > > > >> > > > > > have this happen to this feature.
> >> > > > > >> > > > > >> > > > > > + Matteo's suggestion (with a helpful
> >> starter
> >> > > > list)
> >> > > > > >> that
> >> > > > > >> > > > there
> >> > > > > >> > > > > >> > needs
> >> > > > > >> > > > > >> > > to
> >> > > > > >> > > > > >> > > > > be
> >> > > > > >> > > > > >> > > > > > explicit qualification on what is
> actually
> >> > > being
> >> > > > > >> > delivered
> >> > > > > >> > > > --
> >> > > > > >> > > > > >> > > > including a
> >> > > > > >> > > > > >> > > > > > listing of limitations (some look
> serious
> >> > such
> >> > > as
> >> > > > > >> data
> >> > > > > >> > > bleed
> >> > > > > >> > > > > from
> >> > > > > >> > > > > >> > > other
> >> > > > > >> > > > > >> > > > > > regions in WALs, but maybe I don't care
> >> for
> >> > my
> >> > > > use
> >> > > > > >> > > case...)
> >> > > > > >> > > > --
> >> > > > > >> > > > > >> > needs
> >> > > > > >> > > > > >> > > to
> >> > > > > >> > > > > >> > > > > > accompany the merge. Lets fold them into
> >> the
> >> > > user
> >> > > > > >> doc.
> >> > > > > >> > in
> >> > > > > >> > > > the
> >> > > > > >> > > > > >> > > technical
> >> > > > > >> > > > > >> > > > > > overview area as suggested so user
> >> > expectations
> >> > > > are
> >> > > > > >> > > properly
> >> > > > > >> > > > > >> > managed
> >> > > > > >> > > > > >> > > > > > (otherwise, they expect the world and
> will
> >> > just
> >> > > > > give
> >> > > > > >> up
> >> > > > > >> > > when
> >> > > > > >> > > > > we
> >> > > > > >> > > > > >> > fall
> >> > > > > >> > > > > >> > > > > > short). Vladimir did a list of what is
> in
> >> > each
> >> > > of
> >> > > > > the
> >> > > > > >> > > phases
> >> > > > > >> > > > > >> above
> >> > > > > >> > > > > >> > > > which
> >> > > > > >> > > > > >> > > > > > would serve as a good start.
> >> > > > > >> > > > > >> > > > > > + Is this feature 'experimental' (Matteo
> >> asks
> >> > > > > above).
> >> > > > > >> > I'd
> >> > > > > >> > > > > prefer
> >> > > > > >> > > > > >> it
> >> > > > > >> > > > > >> > > is
> >> > > > > >> > > > > >> > > > > > not. If it is, it should be labelled all
> >> over
> >> > > > that
> >> > > > > >> it is
> >> > > > > >> > > > so. I
> >> > > > > >> > > > > >> see
> >> > > > > >> > > > > >> > > > > current
> >> > > > > >> > > > > >> > > > > > state called out as a '... technical
> >> preview
> >> > > > > >> feature'.
> >> > > > > >> > > Does
> >> > > > > >> > > > > this
> >> > > > > >> > > > > >> > mean
> >> > > > > >> > > > > >> > > > > > not-for-users?
> >> > > > > >> > > > > >> > > > > >
> >> > > > > >> > > > > >> > > > > > St.Ack
> >> > > > > >> > > > > >> > > > > >
> >> > > > > >> > > > > >> > > > > >
> >> > > > > >> > > > > >> > > > > >
> >> > > > > >> > > > > >> > > > > >
> >> > > > > >> > > > > >> > > > > >
> >> > > > > >> > > > > >> > > > > >
> >> > > > > >> > > > > >> > > > > >
> >> > > > > >> > > > > >> > > > > >
> >> > > > > >> > > > > >> > > > > >
> >> > > > > >> > > > > >> > > > > >
> >> > > > > >> > > > > >> > > > > >
> >> > > > > >> > > > > >> > > > > > On Mon, Sep 12, 2016 at 8:03 AM, Ted Yu
> <
> >> > > > > >> > > > yuzhihong@gmail.com>
> >> > > > > >> > > > > >> > wrote:
> >> > > > > >> > > > > >> > > > > >
> >> > > > > >> > > > > >> > > > > >> Sean:
> >> > > > > >> > > > > >> > > > > >> Do you have more comments ?
> >> > > > > >> > > > > >> > > > > >>
> >> > > > > >> > > > > >> > > > > >> Cheers
> >> > > > > >> > > > > >> > > > > >>
> >> > > > > >> > > > > >> > > > > >> On Fri, Sep 9, 2016 at 1:42 PM,
> Vladimir
> >> > > > Rodionov
> >> > > > > <
> >> > > > > >> > > > > >> > > > > vladrodionov@gmail.com
> >> > > > > >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > > > > >> wrote:
> >> > > > > >> > > > > >> > > > > >>
> >> > > > > >> > > > > >> > > > > >> > Sean,
> >> > > > > >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > > > > >> > Backup/Restore can fail due to
> various
> >> > > > reasons:
> >> > > > > >> > network
> >> > > > > >> > > > > outage
> >> > > > > >> > > > > >> > > > > (cluster
> >> > > > > >> > > > > >> > > > > >> > wide), various time-outs in HBase and
> >> HDFS
> >> > > > > layer,
> >> > > > > >> M/R
> >> > > > > >> > > > > failure
> >> > > > > >> > > > > >> > due
> >> > > > > >> > > > > >> > > to
> >> > > > > >> > > > > >> > > > > >> "HDFS
> >> > > > > >> > > > > >> > > > > >> > exceeded quota", user error (manual
> >> > deletion
> >> > > > of
> >> > > > > >> data)
> >> > > > > >> > > and
> >> > > > > >> > > > > so
> >> > > > > >> > > > > >> on
> >> > > > > >> > > > > >> > so
> >> > > > > >> > > > > >> > > > on.
> >> > > > > >> > > > > >> > > > > >> That
> >> > > > > >> > > > > >> > > > > >> > is impossible to enumerate all
> possible
> >> > > types
> >> > > > of
> >> > > > > >> > > failures
> >> > > > > >> > > > > in a
> >> > > > > >> > > > > >> > > > > >> distributed
> >> > > > > >> > > > > >> > > > > >> > system - that is not our goal/task.
> >> > > > > >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > > > > >> > We focus completely on backup system
> >> table
> >> > > > > >> > consistency
> >> > > > > >> > > > in a
> >> > > > > >> > > > > >> > > presence
> >> > > > > >> > > > > >> > > > > of
> >> > > > > >> > > > > >> > > > > >> any
> >> > > > > >> > > > > >> > > > > >> > type of failure. That is what I call
> >> > > > "tolerance
> >> > > > > to
> >> > > > > >> > > > > failures".
> >> > > > > >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > > > > >> > On a failure:
> >> > > > > >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > > > > >> > BACKUP. All backup system information
> >> > (prior
> >> > > > to
> >> > > > > >> > backup)
> >> > > > > >> > > > > will
> >> > > > > >> > > > > >> be
> >> > > > > >> > > > > >> > > > > restored
> >> > > > > >> > > > > >> > > > > >> > and all temporary data, related to a
> >> > failed
> >> > > > > >> session,
> >> > > > > >> > in
> >> > > > > >> > > > > HDFS
> >> > > > > >> > > > > >> > will
> >> > > > > >> > > > > >> > > be
> >> > > > > >> > > > > >> > > > > >> > deleted
> >> > > > > >> > > > > >> > > > > >> > RESTORE. We do not care about system
> >> data,
> >> > > > > because
> >> > > > > >> > > > restore
> >> > > > > >> > > > > >> does
> >> > > > > >> > > > > >> > > not
> >> > > > > >> > > > > >> > > > > >> change
> >> > > > > >> > > > > >> > > > > >> > it. Temporary data in HDFS will be
> >> cleaned
> >> > > up
> >> > > > > and
> >> > > > > >> > table
> >> > > > > >> > > > > will
> >> > > > > >> > > > > >> be
> >> > > > > >> > > > > >> > > in a
> >> > > > > >> > > > > >> > > > > >> state
> >> > > > > >> > > > > >> > > > > >> > back to where it was before operation
> >> > > started.
> >> > > > > >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > > > > >> > This is what user should expect in
> case
> >> > of a
> >> > > > > >> failure.
> >> > > > > >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > > > > >> > -Vlad
> >> > > > > >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > > > > >> > -Vlad
> >> > > > > >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > > > > >> > On Fri, Sep 9, 2016 at 12:56 PM, Sean
> >> > > Busbey <
> >> > > > > >> > > > > >> busbey@apache.org
> >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> > > > > wrote:
> >> > > > > >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > > > > >> > > Failing in a consistent way, with
> >> docs
> >> > > that
> >> > > > > >> explain
> >> > > > > >> > > the
> >> > > > > >> > > > > >> > various
> >> > > > > >> > > > > >> > > > > >> > > expected failures would be
> >> sufficient.
> >> > > > > >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> > > > > >> > > On Fri, Sep 9, 2016 at 12:16 PM,
> >> > Vladimir
> >> > > > > >> Rodionov
> >> > > > > >> > > > > >> > > > > >> > > <vl...@gmail.com> wrote:
> >> > > > > >> > > > > >> > > > > >> > > > Do not worry Sean, doc is coming
> >> today
> >> > > as
> >> > > > a
> >> > > > > >> > preview
> >> > > > > >> > > > and
> >> > > > > >> > > > > >> our
> >> > > > > >> > > > > >> > > > writer
> >> > > > > >> > > > > >> > > > > >> > Frank
> >> > > > > >> > > > > >> > > > > >> > > > will be working on a putting  it
> >> into
> >> > > > Apache
> >> > > > > >> > repo.
> >> > > > > >> > > > > >> Timeline
> >> > > > > >> > > > > >> > > > > depends
> >> > > > > >> > > > > >> > > > > >> on
> >> > > > > >> > > > > >> > > > > >> > > > Franks schedule but I hope we
> will
> >> get
> >> > > it
> >> > > > > >> rather
> >> > > > > >> > > > sooner
> >> > > > > >> > > > > >> than
> >> > > > > >> > > > > >> > > > > later.
> >> > > > > >> > > > > >> > > > > >> > > >
> >> > > > > >> > > > > >> > > > > >> > > > As for failure testing, we are
> >> > focusing
> >> > > > only
> >> > > > > >> on a
> >> > > > > >> > > > > >> consistent
> >> > > > > >> > > > > >> > > > state
> >> > > > > >> > > > > >> > > > > >> of
> >> > > > > >> > > > > >> > > > > >> > > > backup system data in a presence
> of
> >> > any
> >> > > > type
> >> > > > > >> of
> >> > > > > >> > > > > failures,
> >> > > > > >> > > > > >> We
> >> > > > > >> > > > > >> > > are
> >> > > > > >> > > > > >> > > > > not
> >> > > > > >> > > > > >> > > > > >> > > going
> >> > > > > >> > > > > >> > > > > >> > > > to implement  anything more
> >> "fancy",
> >> > > than
> >> > > > > >> that.
> >> > > > > >> > We
> >> > > > > >> > > > > allow
> >> > > > > >> > > > > >> > both:
> >> > > > > >> > > > > >> > > > > >> backup
> >> > > > > >> > > > > >> > > > > >> > and
> >> > > > > >> > > > > >> > > > > >> > > > restore to fail. What we do not
> >> allow
> >> > is
> >> > > > to
> >> > > > > >> have
> >> > > > > >> > > > system
> >> > > > > >> > > > > >> data
> >> > > > > >> > > > > >> > > > > >> corrupted.
> >> > > > > >> > > > > >> > > > > >> > > > Will it suffice for you? Do you
> >> have
> >> > any
> >> > > > > other
> >> > > > > >> > > > > concerns,
> >> > > > > >> > > > > >> you
> >> > > > > >> > > > > >> > > > want
> >> > > > > >> > > > > >> > > > > >> us to
> >> > > > > >> > > > > >> > > > > >> > > > address?
> >> > > > > >> > > > > >> > > > > >> > > >
> >> > > > > >> > > > > >> > > > > >> > > > -Vlad
> >> > > > > >> > > > > >> > > > > >> > > >
> >> > > > > >> > > > > >> > > > > >> > > >
> >> > > > > >> > > > > >> > > > > >> > > > On Fri, Sep 9, 2016 at 10:56 AM,
> >> Sean
> >> > > > > Busbey <
> >> > > > > >> > > > > >> > > busbey@apache.org
> >> > > > > >> > > > > >> > > > >
> >> > > > > >> > > > > >> > > > > >> > wrote:
> >> > > > > >> > > > > >> > > > > >> > > >
> >> > > > > >> > > > > >> > > > > >> > > >> "docs will come to Apache soon"
> >> does
> >> > > not
> >> > > > > >> address
> >> > > > > >> > > my
> >> > > > > >> > > > > >> concern
> >> > > > > >> > > > > >> > > > > around
> >> > > > > >> > > > > >> > > > > >> > docs
> >> > > > > >> > > > > >> > > > > >> > > at
> >> > > > > >> > > > > >> > > > > >> > > >> all, unless said docs have
> already
> >> > made
> >> > > > it
> >> > > > > >> into
> >> > > > > >> > > the
> >> > > > > >> > > > > >> project
> >> > > > > >> > > > > >> > > > > repo. I
> >> > > > > >> > > > > >> > > > > >> > > don't
> >> > > > > >> > > > > >> > > > > >> > > >> want third party resources for
> >> using
> >> > a
> >> > > > > major
> >> > > > > >> and
> >> > > > > >> > > > > >> important
> >> > > > > >> > > > > >> > > > > feature
> >> > > > > >> > > > > >> > > > > >> of
> >> > > > > >> > > > > >> > > > > >> > > the
> >> > > > > >> > > > > >> > > > > >> > > >> project, I want us to provide
> end
> >> > users
> >> > > > > with
> >> > > > > >> > what
> >> > > > > >> > > > they
> >> > > > > >> > > > > >> need
> >> > > > > >> > > > > >> > > to
> >> > > > > >> > > > > >> > > > > get
> >> > > > > >> > > > > >> > > > > >> the
> >> > > > > >> > > > > >> > > > > >> > > job
> >> > > > > >> > > > > >> > > > > >> > > >> done.
> >> > > > > >> > > > > >> > > > > >> > > >>
> >> > > > > >> > > > > >> > > > > >> > > >> I see some calls for patience on
> >> the
> >> > > > > failure
> >> > > > > >> > > > testing,
> >> > > > > >> > > > > but
> >> > > > > >> > > > > >> > the
> >> > > > > >> > > > > >> > > > > >> appeal
> >> > > > > >> > > > > >> > > > > >> > to
> >> > > > > >> > > > > >> > > > > >> > > us
> >> > > > > >> > > > > >> > > > > >> > > >> having done a bad job of
> requiring
> >> > > proper
> >> > > > > >> tests
> >> > > > > >> > of
> >> > > > > >> > > > > >> previous
> >> > > > > >> > > > > >> > > > > >> features
> >> > > > > >> > > > > >> > > > > >> > > just
> >> > > > > >> > > > > >> > > > > >> > > >> makes me more concerned about
> not
> >> > > getting
> >> > > > > >> them
> >> > > > > >> > > > here. I
> >> > > > > >> > > > > >> > don't
> >> > > > > >> > > > > >> > > > want
> >> > > > > >> > > > > >> > > > > >> to
> >> > > > > >> > > > > >> > > > > >> > set
> >> > > > > >> > > > > >> > > > > >> > > >> yet another bad example that
> will
> >> > then
> >> > > be
> >> > > > > >> > pointed
> >> > > > > >> > > to
> >> > > > > >> > > > > in
> >> > > > > >> > > > > >> the
> >> > > > > >> > > > > >> > > > > future.
> >> > > > > >> > > > > >> > > > > >> > > >>
> >> > > > > >> > > > > >> > > > > >> > > >> On Sep 8, 2016 10:50, "Ted Yu" <
> >> > > > > >> > > yuzhihong@gmail.com
> >> > > > > >> > > > >
> >> > > > > >> > > > > >> > wrote:
> >> > > > > >> > > > > >> > > > > >> > > >>
> >> > > > > >> > > > > >> > > > > >> > > >> > Is there any concern which is
> >> not
> >> > > > > >> addressed ?
> >> > > > > >> > > > > >> > > > > >> > > >> >
> >> > > > > >> > > > > >> > > > > >> > > >> > Do we need another Vote
> thread ?
> >> > > > > >> > > > > >> > > > > >> > > >> >
> >> > > > > >> > > > > >> > > > > >> > > >> > Thanks
> >> > > > > >> > > > > >> > > > > >> > > >> >
> >> > > > > >> > > > > >> > > > > >> > > >> > On Thu, Sep 8, 2016 at 9:21
> AM,
> >> > > Andrew
> >> > > > > >> > Purtell <
> >> > > > > >> > > > > >> > > > > >> apurtell@apache.org
> >> > > > > >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> > > > > >> > > >> > wrote:
> >> > > > > >> > > > > >> > > > > >> > > >> >
> >> > > > > >> > > > > >> > > > > >> > > >> > > Vlad,
> >> > > > > >> > > > > >> > > > > >> > > >> > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > I apologize for using the
> term
> >> > > > > >> 'half-baked'
> >> > > > > >> > > in a
> >> > > > > >> > > > > way
> >> > > > > >> > > > > >> > that
> >> > > > > >> > > > > >> > > > > could
> >> > > > > >> > > > > >> > > > > >> > > seem a
> >> > > > > >> > > > > >> > > > > >> > > >> > > description of HBASE-7912. I
> >> > meant
> >> > > > that
> >> > > > > >> as a
> >> > > > > >> > > > > general
> >> > > > > >> > > > > >> > > > > >> hypothetical.
> >> > > > > >> > > > > >> > > > > >> > > >> > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > On Wed, Sep 7, 2016 at 9:36
> >> AM,
> >> > > > > Vladimir
> >> > > > > >> > > > Rodionov
> >> > > > > >> > > > > <
> >> > > > > >> > > > > >> > > > > >> > > >> > vladrodionov@gmail.com>
> >> > > > > >> > > > > >> > > > > >> > > >> > > wrote:
> >> > > > > >> > > > > >> > > > > >> > > >> > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > >> I'm not sure that
> "There
> >> is
> >> > > > > already
> >> > > > > >> > lots
> >> > > > > >> > > of
> >> > > > > >> > > > > >> > > half-baked
> >> > > > > >> > > > > >> > > > > >> code
> >> > > > > >> > > > > >> > > > > >> > in
> >> > > > > >> > > > > >> > > > > >> > > the
> >> > > > > >> > > > > >> > > > > >> > > >> > > > branch,
> >> > > > > >> > > > > >> > > > > >> > > >> > > > so what's the harm in
> adding
> >> > > more?"
> >> > > > > >> > > > > >> > > > > >> > > >> > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > I meant - not production -
> >> > ready
> >> > > > yet.
> >> > > > > >> This
> >> > > > > >> > > is
> >> > > > > >> > > > > 2.0
> >> > > > > >> > > > > >> > > > > development
> >> > > > > >> > > > > >> > > > > >> > > branch
> >> > > > > >> > > > > >> > > > > >> > > >> > and,
> >> > > > > >> > > > > >> > > > > >> > > >> > > > hence many features are in
> >> > works,
> >> > > > > >> > > > > >> > > > > >> > > >> > > > not being tested well etc.
> >> I do
> >> > > not
> >> > > > > >> > consider
> >> > > > > >> > > > > backup
> >> > > > > >> > > > > >> > as
> >> > > > > >> > > > > >> > > > half
> >> > > > > >> > > > > >> > > > > >> > baked
> >> > > > > >> > > > > >> > > > > >> > > >> > > feature -
> >> > > > > >> > > > > >> > > > > >> > > >> > > > it has passed our internal
> >> QA
> >> > and
> >> > > > has
> >> > > > > >> very
> >> > > > > >> > > > good
> >> > > > > >> > > > > >> doc,
> >> > > > > >> > > > > >> > > > which
> >> > > > > >> > > > > >> > > > > we
> >> > > > > >> > > > > >> > > > > >> > will
> >> > > > > >> > > > > >> > > > > >> > > >> > > provide
> >> > > > > >> > > > > >> > > > > >> > > >> > > > to Apache shortly.
> >> > > > > >> > > > > >> > > > > >> > > >> > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > -Vlad
> >> > > > > >> > > > > >> > > > > >> > > >> > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > On Wed, Sep 7, 2016 at
> 9:13
> >> AM,
> >> > > > > Andrew
> >> > > > > >> > > > Purtell <
> >> > > > > >> > > > > >> > > > > >> > > apurtell@apache.org>
> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
> >> > > > > >> > > > > >> > > > > >> > > >> > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > We shouldn't admit half
> >> baked
> >> > > > > changes
> >> > > > > >> > that
> >> > > > > >> > > > > won't
> >> > > > > >> > > > > >> be
> >> > > > > >> > > > > >> > > > > >> finished.
> >> > > > > >> > > > > >> > > > > >> > > >> However
> >> > > > > >> > > > > >> > > > > >> > > >> > > in
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > this case the crew
> >> working on
> >> > > > this
> >> > > > > >> > feature
> >> > > > > >> > > > are
> >> > > > > >> > > > > >> long
> >> > > > > >> > > > > >> > > > > timers
> >> > > > > >> > > > > >> > > > > >> and
> >> > > > > >> > > > > >> > > > > >> > > less
> >> > > > > >> > > > > >> > > > > >> > > >> > > > likely
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > than just about anyone
> to
> >> > leave
> >> > > > > >> > something
> >> > > > > >> > > > in a
> >> > > > > >> > > > > >> half
> >> > > > > >> > > > > >> > > > baked
> >> > > > > >> > > > > >> > > > > >> > > state. Of
> >> > > > > >> > > > > >> > > > > >> > > >> > > > course
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > there is no guarantee
> how
> >> > > > anything
> >> > > > > >> will
> >> > > > > >> > > turn
> >> > > > > >> > > > > out,
> >> > > > > >> > > > > >> > > but I
> >> > > > > >> > > > > >> > > > > am
> >> > > > > >> > > > > >> > > > > >> > > willing
> >> > > > > >> > > > > >> > > > > >> > > >> to
> >> > > > > >> > > > > >> > > > > >> > > >> > > > take
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > a little on faith if
> they
> >> > feel
> >> > > > > their
> >> > > > > >> > best
> >> > > > > >> > > > path
> >> > > > > >> > > > > >> > > forward
> >> > > > > >> > > > > >> > > > > now
> >> > > > > >> > > > > >> > > > > >> is
> >> > > > > >> > > > > >> > > > > >> > to
> >> > > > > >> > > > > >> > > > > >> > > >> > merge
> >> > > > > >> > > > > >> > > > > >> > > >> > > to
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > trunk. I only wish I had
> >> > > > bandwidth
> >> > > > > to
> >> > > > > >> > have
> >> > > > > >> > > > > done
> >> > > > > >> > > > > >> > some
> >> > > > > >> > > > > >> > > > real
> >> > > > > >> > > > > >> > > > > >> > > kicking
> >> > > > > >> > > > > >> > > > > >> > > >> of
> >> > > > > >> > > > > >> > > > > >> > > >> > > the
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > tires by now. Maybe this
> >> > week.
> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > (Yes, I'm using some of
> >> that
> >> > > time
> >> > > > > for
> >> > > > > >> > this
> >> > > > > >> > > > > email
> >> > > > > >> > > > > >> > :-)
> >> > > > > >> > > > > >> > > > but
> >> > > > > >> > > > > >> > > > > I
> >> > > > > >> > > > > >> > > > > >> > type
> >> > > > > >> > > > > >> > > > > >> > > >> > fast.)
> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > That said, I would like
> to
> >> > > > agitate
> >> > > > > >> for
> >> > > > > >> > > > making
> >> > > > > >> > > > > 2.0
> >> > > > > >> > > > > >> > > more
> >> > > > > >> > > > > >> > > > > real
> >> > > > > >> > > > > >> > > > > >> > and
> >> > > > > >> > > > > >> > > > > >> > > >> spend
> >> > > > > >> > > > > >> > > > > >> > > >> > > > some
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > time on it now that I'm
> >> > winding
> >> > > > > down
> >> > > > > >> > with
> >> > > > > >> > > > > 0.98. I
> >> > > > > >> > > > > >> > > think
> >> > > > > >> > > > > >> > > > > >> that
> >> > > > > >> > > > > >> > > > > >> > > means
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branching for 2.0 real
> >> soon
> >> > now
> >> > > > and
> >> > > > > >> even
> >> > > > > >> > > > > evicting
> >> > > > > >> > > > > >> > > > things
> >> > > > > >> > > > > >> > > > > >> from
> >> > > > > >> > > > > >> > > > > >> > > 2.0
> >> > > > > >> > > > > >> > > > > >> > > >> > > branch
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > that aren't finished or
> >> > stable,
> >> > > > > >> leaving
> >> > > > > >> > > them
> >> > > > > >> > > > > only
> >> > > > > >> > > > > >> > > once
> >> > > > > >> > > > > >> > > > > >> again
> >> > > > > >> > > > > >> > > > > >> > in
> >> > > > > >> > > > > >> > > > > >> > > the
> >> > > > > >> > > > > >> > > > > >> > > >> > > > master
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branch. Or, maybe just
> >> > evicting
> >> > > > > them.
> >> > > > > >> > > Let's
> >> > > > > >> > > > > take
> >> > > > > >> > > > > >> it
> >> > > > > >> > > > > >> > > > case
> >> > > > > >> > > > > >> > > > > by
> >> > > > > >> > > > > >> > > > > >> > > case.
> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > I think this feature can
> >> come
> >> > > in
> >> > > > > >> > > relatively
> >> > > > > >> > > > > >> safely.
> >> > > > > >> > > > > >> > > As
> >> > > > > >> > > > > >> > > > > >> added
> >> > > > > >> > > > > >> > > > > >> > > >> > insurance,
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > let's admit the
> >> possibility
> >> > it
> >> > > > > could
> >> > > > > >> be
> >> > > > > >> > > > > reverted
> >> > > > > >> > > > > >> on
> >> > > > > >> > > > > >> > > the
> >> > > > > >> > > > > >> > > > > 2.0
> >> > > > > >> > > > > >> > > > > >> > > branch
> >> > > > > >> > > > > >> > > > > >> > > >> if
> >> > > > > >> > > > > >> > > > > >> > > >> > > > folks
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > working on stabilizing
> 2.0
> >> > > decide
> >> > > > > to
> >> > > > > >> > evict
> >> > > > > >> > > > it
> >> > > > > >> > > > > >> > because
> >> > > > > >> > > > > >> > > > it
> >> > > > > >> > > > > >> > > > > is
> >> > > > > >> > > > > >> > > > > >> > > >> > unfinished
> >> > > > > >> > > > > >> > > > > >> > > >> > > or
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > unstable, because that
> >> > > certainly
> >> > > > > can
> >> > > > > >> > > > happen. I
> >> > > > > >> > > > > >> > would
> >> > > > > >> > > > > >> > > > > >> expect if
> >> > > > > >> > > > > >> > > > > >> > > talk
> >> > > > > >> > > > > >> > > > > >> > > >> > > like
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > that starts, we'd get
> help
> >> > > > > finishing
> >> > > > > >> or
> >> > > > > >> > > > > >> stabilizing
> >> > > > > >> > > > > >> > > > > what's
> >> > > > > >> > > > > >> > > > > >> > under
> >> > > > > >> > > > > >> > > > > >> > > >> > > > discussion
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > for revert. Or, we'd
> have
> >> a
> >> > > > revert.
> >> > > > > >> > Either
> >> > > > > >> > > > way
> >> > > > > >> > > > > >> the
> >> > > > > >> > > > > >> > > > > outcome
> >> > > > > >> > > > > >> > > > > >> is
> >> > > > > >> > > > > >> > > > > >> > > >> > > acceptable.
> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > On Wed, Sep 7, 2016 at
> >> 8:56
> >> > AM,
> >> > > > > Dima
> >> > > > > >> > > Spivak
> >> > > > > >> > > > <
> >> > > > > >> > > > > >> > > > > >> > > dimaspivak@apache.org
> >> > > > > >> > > > > >> > > > > >> > > >> >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > I'm not sure that
> >> "There is
> >> > > > > already
> >> > > > > >> > lots
> >> > > > > >> > > > of
> >> > > > > >> > > > > >> > > > half-baked
> >> > > > > >> > > > > >> > > > > >> code
> >> > > > > >> > > > > >> > > > > >> > in
> >> > > > > >> > > > > >> > > > > >> > > >> the
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > branch,
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > so what's the harm in
> >> > adding
> >> > > > > more?"
> >> > > > > >> > is a
> >> > > > > >> > > > > good
> >> > > > > >> > > > > >> > code
> >> > > > > >> > > > > >> > > > > commit
> >> > > > > >> > > > > >> > > > > >> > > >> > philosophy
> >> > > > > >> > > > > >> > > > > >> > > >> > > > for
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > a
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > fault-tolerant
> >> distributed
> >> > > data
> >> > > > > >> store.
> >> > > > > >> > > ;)
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > More seriously, a lack
> >> of
> >> > > test
> >> > > > > >> > coverage
> >> > > > > >> > > > for
> >> > > > > >> > > > > >> > > existing
> >> > > > > >> > > > > >> > > > > >> > features
> >> > > > > >> > > > > >> > > > > >> > > >> > > shouldn't
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > be
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > used as justification
> >> for
> >> > > > > >> introducing
> >> > > > > >> > > new
> >> > > > > >> > > > > >> > features
> >> > > > > >> > > > > >> > > > with
> >> > > > > >> > > > > >> > > > > >> the
> >> > > > > >> > > > > >> > > > > >> > > same
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > shortcomings.
> >> Ultimately,
> >> > > it's
> >> > > > > the
> >> > > > > >> end
> >> > > > > >> > > > user
> >> > > > > >> > > > > who
> >> > > > > >> > > > > >> > > will
> >> > > > > >> > > > > >> > > > > feel
> >> > > > > >> > > > > >> > > > > >> > the
> >> > > > > >> > > > > >> > > > > >> > > >> pain,
> >> > > > > >> > > > > >> > > > > >> > > >> > > so
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > shouldn't we do
> >> everything
> >> > we
> >> > > > can
> >> > > > > >> to
> >> > > > > >> > > > > mitigate
> >> > > > > >> > > > > >> > that?
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > -Dima
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > On Wed, Sep 7, 2016 at
> >> 8:46
> >> > > AM,
> >> > > > > >> > Vladimir
> >> > > > > >> > > > > >> > Rodionov <
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > vladrodionov@gmail.com>
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > wrote:
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Sean,
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have docs
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Agree. We have a doc
> >> and
> >> > > > backup
> >> > > > > >> is
> >> > > > > >> > the
> >> > > > > >> > > > > most
> >> > > > > >> > > > > >> > > > > documented
> >> > > > > >> > > > > >> > > > > >> > > feature
> >> > > > > >> > > > > >> > > > > >> > > >> > :),
> >> > > > > >> > > > > >> > > > > >> > > >> > > we
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > will
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > release it shortly
> to
> >> > > Apache.
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have sunny-day
> >> > > correctness
> >> > > > > >> tests
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Feature has  close
> to
> >> 60
> >> > > test
> >> > > > > >> cases,
> >> > > > > >> > > > which
> >> > > > > >> > > > > >> run
> >> > > > > >> > > > > >> > > for
> >> > > > > >> > > > > >> > > > > >> approx
> >> > > > > >> > > > > >> > > > > >> > 30
> >> > > > > >> > > > > >> > > > > >> > > >> min.
> >> > > > > >> > > > > >> > > > > >> > > >> > > We
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > can
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > add more, if
> >> community do
> >> > > not
> >> > > > > >> mind
> >> > > > > >> > :)
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have
> >> > > > > >> > correctness-in-face-of-failure
> >> > > > > >> > > > > tests
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Any examples of
> these
> >> > tests
> >> > > > in
> >> > > > > >> > > existing
> >> > > > > >> > > > > >> > features?
> >> > > > > >> > > > > >> > > > In
> >> > > > > >> > > > > >> > > > > >> > works,
> >> > > > > >> > > > > >> > > > > >> > > we
> >> > > > > >> > > > > >> > > > > >> > > >> > > have a
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > clear
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > understanding of
> what
> >> > > should
> >> > > > be
> >> > > > > >> done
> >> > > > > >> > > by
> >> > > > > >> > > > > the
> >> > > > > >> > > > > >> > time
> >> > > > > >> > > > > >> > > of
> >> > > > > >> > > > > >> > > > > 2.0
> >> > > > > >> > > > > >> > > > > >> > > >> release.
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > That is very close
> >> goal
> >> > for
> >> > > > us,
> >> > > > > >> to
> >> > > > > >> > > > verify
> >> > > > > >> > > > > IT
> >> > > > > >> > > > > >> > > monkey
> >> > > > > >> > > > > >> > > > > for
> >> > > > > >> > > > > >> > > > > >> > > >> existing
> >> > > > > >> > > > > >> > > > > >> > > >> > > > code.
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * don't rely on
> things
> >> > > > outside
> >> > > > > of
> >> > > > > >> > > HBase
> >> > > > > >> > > > > for
> >> > > > > >> > > > > >> > > normal
> >> > > > > >> > > > > >> > > > > >> > operation
> >> > > > > >> > > > > >> > > > > >> > > >> > (okay
> >> > > > > >> > > > > >> > > > > >> > > >> > > > for
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > advanced operation)
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > We do not.
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Enormous time has
> been
> >> > > spent
> >> > > > > >> already
> >> > > > > >> > > on
> >> > > > > >> > > > > the
> >> > > > > >> > > > > >> > > > > development
> >> > > > > >> > > > > >> > > > > >> > and
> >> > > > > >> > > > > >> > > > > >> > > >> > testing
> >> > > > > >> > > > > >> > > > > >> > > >> > > > the
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > feature, it has
> passed
> >> > our
> >> > > > > >> internal
> >> > > > > >> > > > tests
> >> > > > > >> > > > > and
> >> > > > > >> > > > > >> > > many
> >> > > > > >> > > > > >> > > > > >> rounds
> >> > > > > >> > > > > >> > > > > >> > of
> >> > > > > >> > > > > >> > > > > >> > > >> code
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > reviews
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > by HBase committers.
> >> We
> >> > do
> >> > > > not
> >> > > > > >> mind
> >> > > > > >> > if
> >> > > > > >> > > > > >> someone
> >> > > > > >> > > > > >> > > from
> >> > > > > >> > > > > >> > > > > >> HBase
> >> > > > > >> > > > > >> > > > > >> > > >> > community
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > (outside of HW) will
> >> > review
> >> > > > the
> >> > > > > >> > code,
> >> > > > > >> > > > but
> >> > > > > >> > > > > it
> >> > > > > >> > > > > >> > will
> >> > > > > >> > > > > >> > > > > >> probably
> >> > > > > >> > > > > >> > > > > >> > > >> takes
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > forever
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > to
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > wait for volunteer?,
> >> the
> >> > > > > feature
> >> > > > > >> is
> >> > > > > >> > > > quite
> >> > > > > >> > > > > >> large
> >> > > > > >> > > > > >> > > > (1MB+
> >> > > > > >> > > > > >> > > > > >> > > >> cumulative
> >> > > > > >> > > > > >> > > > > >> > > >> > > > patch)
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > 2.0 branch is full
> of
> >> > half
> >> > > > > baked
> >> > > > > >> > > > features,
> >> > > > > >> > > > > >> most
> >> > > > > >> > > > > >> > > of
> >> > > > > >> > > > > >> > > > > them
> >> > > > > >> > > > > >> > > > > >> > are
> >> > > > > >> > > > > >> > > > > >> > > in
> >> > > > > >> > > > > >> > > > > >> > > >> > > active
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > development,
> >> therefore I
> >> > am
> >> > > > not
> >> > > > > >> > > > following
> >> > > > > >> > > > > you
> >> > > > > >> > > > > >> > > here,
> >> > > > > >> > > > > >> > > > > >> Sean?
> >> > > > > >> > > > > >> > > > > >> > > Why
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > HBASE-7912
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > is
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > not good enough yet
> >> to be
> >> > > > > >> integrated
> >> > > > > >> > > > into
> >> > > > > >> > > > > 2.0
> >> > > > > >> > > > > >> > > > branch?
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > -Vlad
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > On Wed, Sep 7, 2016
> at
> >> > 8:23
> >> > > > AM,
> >> > > > > >> Sean
> >> > > > > >> > > > > Busbey <
> >> > > > > >> > > > > >> > > > > >> > > busbey@apache.org
> >> > > > > >> > > > > >> > > > > >> > > >> >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > On Tue, Sep 6,
> 2016
> >> at
> >> > > > 10:36
> >> > > > > >> PM,
> >> > > > > >> > > Josh
> >> > > > > >> > > > > >> Elser <
> >> > > > > >> > > > > >> > > > > >> > > >> > > josh.elser@gmail.com>
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > wrote:
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > So, the answer
> to
> >> > > Sean's
> >> > > > > >> > original
> >> > > > > >> > > > > >> question
> >> > > > > >> > > > > >> > is
> >> > > > > >> > > > > >> > > > "as
> >> > > > > >> > > > > >> > > > > >> > > robust as
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > snapshots
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > presently are"?
> >> > > > > >> (independence of
> >> > > > > >> > > > > >> > > backup/restore
> >> > > > > >> > > > > >> > > > > >> > failure
> >> > > > > >> > > > > >> > > > > >> > > >> > > tolerance
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > from
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > snapshot failure
> >> > > > tolerance)
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > Is this just a
> >> > question
> >> > > > WRT
> >> > > > > >> > > context
> >> > > > > >> > > > of
> >> > > > > >> > > > > >> the
> >> > > > > >> > > > > >> > > > > change,
> >> > > > > >> > > > > >> > > > > >> or
> >> > > > > >> > > > > >> > > > > >> > > is it
> >> > > > > >> > > > > >> > > > > >> > > >> > > means
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > for a
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > veto
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > from you, Sean?
> >> Just
> >> > > > trying
> >> > > > > >> to
> >> > > > > >> > > make
> >> > > > > >> > > > > sure
> >> > > > > >> > > > > >> > I'm
> >> > > > > >> > > > > >> > > > > >> following
> >> > > > > >> > > > > >> > > > > >> > > >> along
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > adequately.
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > I'd say ATM I'm
> -0,
> >> > > > bordering
> >> > > > > >> on
> >> > > > > >> > -1
> >> > > > > >> > > > but
> >> > > > > >> > > > > not
> >> > > > > >> > > > > >> > for
> >> > > > > >> > > > > >> > > > > >> reasons
> >> > > > > >> > > > > >> > > > > >> > I
> >> > > > > >> > > > > >> > > > > >> > > can
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > articulate
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > well.
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Here's an attempt.
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > We've been trying
> to
> >> > > move,
> >> > > > > as a
> >> > > > > >> > > > > community,
> >> > > > > >> > > > > >> > > > towards
> >> > > > > >> > > > > >> > > > > >> > > minimizing
> >> > > > > >> > > > > >> > > > > >> > > >> > > risk
> >> > > > > >> > > > > >> > > > > >> > > >> > > > to
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > downstream folks
> by
> >> > > getting
> >> > > > > >> > > "complete
> >> > > > > >> > > > > >> enough
> >> > > > > >> > > > > >> > > for
> >> > > > > >> > > > > >> > > > > use"
> >> > > > > >> > > > > >> > > > > >> > > gates
> >> > > > > >> > > > > >> > > > > >> > > >> in
> >> > > > > >> > > > > >> > > > > >> > > >> > > > place
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > before we
> introduce
> >> new
> >> > > > > >> features.
> >> > > > > >> > > This
> >> > > > > >> > > > > was
> >> > > > > >> > > > > >> > > > spurred
> >> > > > > >> > > > > >> > > > > >> by a
> >> > > > > >> > > > > >> > > > > >> > > some
> >> > > > > >> > > > > >> > > > > >> > > >> > > > features
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > getting in
> >> half-baked
> >> > and
> >> > > > > never
> >> > > > > >> > > making
> >> > > > > >> > > > > it
> >> > > > > >> > > > > >> to
> >> > > > > >> > > > > >> > > "can
> >> > > > > >> > > > > >> > > > > >> really
> >> > > > > >> > > > > >> > > > > >> > > use"
> >> > > > > >> > > > > >> > > > > >> > > >> > > > status
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > (I'm thinking of
> >> > > > distributed
> >> > > > > >> log
> >> > > > > >> > > > replay
> >> > > > > >> > > > > and
> >> > > > > >> > > > > >> > the
> >> > > > > >> > > > > >> > > > > >> zk-less
> >> > > > > >> > > > > >> > > > > >> > > >> > > assignment
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > stuff, I don't
> >> recall
> >> > if
> >> > > > > there
> >> > > > > >> was
> >> > > > > >> > > > > more).
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The gates,
> >> generally,
> >> > > > > included
> >> > > > > >> > > things
> >> > > > > >> > > > > like:
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have docs
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have sunny-day
> >> > > > correctness
> >> > > > > >> tests
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have
> >> > > > > >> > > correctness-in-face-of-failure
> >> > > > > >> > > > > tests
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't rely on
> >> things
> >> > > > > outside
> >> > > > > >> of
> >> > > > > >> > > > HBase
> >> > > > > >> > > > > for
> >> > > > > >> > > > > >> > > > normal
> >> > > > > >> > > > > >> > > > > >> > > operation
> >> > > > > >> > > > > >> > > > > >> > > >> > > (okay
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > for
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > advanced
> operation)
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > As an example, we
> >> kept
> >> > > the
> >> > > > > MOB
> >> > > > > >> > work
> >> > > > > >> > > > off
> >> > > > > >> > > > > in
> >> > > > > >> > > > > >> a
> >> > > > > >> > > > > >> > > > branch
> >> > > > > >> > > > > >> > > > > >> and
> >> > > > > >> > > > > >> > > > > >> > > out
> >> > > > > >> > > > > >> > > > > >> > > >> of
> >> > > > > >> > > > > >> > > > > >> > > >> > > > master
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > until it could
> pass
> >> > these
> >> > > > > >> > criteria.
> >> > > > > >> > > > The
> >> > > > > >> > > > > big
> >> > > > > >> > > > > >> > > > > exemption
> >> > > > > >> > > > > >> > > > > >> > > we've
> >> > > > > >> > > > > >> > > > > >> > > >> had
> >> > > > > >> > > > > >> > > > > >> > > >> > > to
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > this was the
> >> > hbase-spark
> >> > > > > >> > > integration,
> >> > > > > >> > > > > where
> >> > > > > >> > > > > >> > we
> >> > > > > >> > > > > >> > > > all
> >> > > > > >> > > > > >> > > > > >> > agreed
> >> > > > > >> > > > > >> > > > > >> > > it
> >> > > > > >> > > > > >> > > > > >> > > >> > > could
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > land in master
> >> because
> >> > it
> >> > > > was
> >> > > > > >> very
> >> > > > > >> > > > well
> >> > > > > >> > > > > >> > > isolated
> >> > > > > >> > > > > >> > > > > (the
> >> > > > > >> > > > > >> > > > > >> > > slide
> >> > > > > >> > > > > >> > > > > >> > > >> > away
> >> > > > > >> > > > > >> > > > > >> > > >> > > > from
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > including docs as
> a
> >> > > > > first-class
> >> > > > > >> > part
> >> > > > > >> > > > of
> >> > > > > >> > > > > >> > > building
> >> > > > > >> > > > > >> > > > up
> >> > > > > >> > > > > >> > > > > >> that
> >> > > > > >> > > > > >> > > > > >> > > >> > > > integration
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > has led me to
> doubt
> >> the
> >> > > > > wisdom
> >> > > > > >> of
> >> > > > > >> > > this
> >> > > > > >> > > > > >> > > decision).
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > We've also been
> >> > treating
> >> > > > > >> inclusion
> >> > > > > >> > > in
> >> > > > > >> > > > a
> >> > > > > >> > > > > >> > > "probably
> >> > > > > >> > > > > >> > > > > >> will
> >> > > > > >> > > > > >> > > > > >> > be
> >> > > > > >> > > > > >> > > > > >> > > >> > > released
> >> > > > > >> > > > > >> > > > > >> > > >> > > > to
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > downstream"
> branches
> >> > as a
> >> > > > > >> higher
> >> > > > > >> > > bar,
> >> > > > > >> > > > > >> > requiring
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't moderately
> >> > impact
> >> > > > > >> > > performance
> >> > > > > >> > > > > when
> >> > > > > >> > > > > >> > the
> >> > > > > >> > > > > >> > > > > >> feature
> >> > > > > >> > > > > >> > > > > >> > > isn't
> >> > > > > >> > > > > >> > > > > >> > > >> in
> >> > > > > >> > > > > >> > > > > >> > > >> > > use
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't severely
> >> impact
> >> > > > > >> > performance
> >> > > > > >> > > > when
> >> > > > > >> > > > > >> the
> >> > > > > >> > > > > >> > > > > feature
> >> > > > > >> > > > > >> > > > > >> is
> >> > > > > >> > > > > >> > > > > >> > in
> >> > > > > >> > > > > >> > > > > >> > > >> use
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * either
> >> default-to-on
> >> > or
> >> > > > > show
> >> > > > > >> > > enough
> >> > > > > >> > > > > >> demand
> >> > > > > >> > > > > >> > to
> >> > > > > >> > > > > >> > > > > >> believe
> >> > > > > >> > > > > >> > > > > >> > a
> >> > > > > >> > > > > >> > > > > >> > > >> > > > non-trivial
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > number of folks
> will
> >> > turn
> >> > > > the
> >> > > > > >> > > feature
> >> > > > > >> > > > on
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The above has kept
> >> MOB
> >> > > and
> >> > > > > >> > > hbase-spark
> >> > > > > >> > > > > >> > > > integration
> >> > > > > >> > > > > >> > > > > >> out
> >> > > > > >> > > > > >> > > > > >> > of
> >> > > > > >> > > > > >> > > > > >> > > >> > > branch-1,
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > presumably while
> >> > they've
> >> > > > > >> "gotten
> >> > > > > >> > > more
> >> > > > > >> > > > > >> stable"
> >> > > > > >> > > > > >> > > in
> >> > > > > >> > > > > >> > > > > >> master
> >> > > > > >> > > > > >> > > > > >> > > from
> >> > > > > >> > > > > >> > > > > >> > > >> > the
> >> > > > > >> > > > > >> > > > > >> > > >> > > > odd
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > vendor inclusion.
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Are we going to
> >> have a
> >> > > 2.0
> >> > > > > >> release
> >> > > > > >> > > > > before
> >> > > > > >> > > > > >> the
> >> > > > > >> > > > > >> > > end
> >> > > > > >> > > > > >> > > > > of
> >> > > > > >> > > > > >> > > > > >> the
> >> > > > > >> > > > > >> > > > > >> > > >> year?
> >> > > > > >> > > > > >> > > > > >> > > >> > > > We're
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > coming up on 1.5
> >> years
> >> > > > since
> >> > > > > >> the
> >> > > > > >> > > > > release of
> >> > > > > >> > > > > >> > > > version
> >> > > > > >> > > > > >> > > > > >> 1.0;
> >> > > > > >> > > > > >> > > > > >> > > >> seems
> >> > > > > >> > > > > >> > > > > >> > > >> > > like
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > it's about time,
> >> > though I
> >> > > > > >> haven't
> >> > > > > >> > > seen
> >> > > > > >> > > > > any
> >> > > > > >> > > > > >> > > > concrete
> >> > > > > >> > > > > >> > > > > >> > plans
> >> > > > > >> > > > > >> > > > > >> > > >> this
> >> > > > > >> > > > > >> > > > > >> > > >> > > > year.
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Presuming we are
> >> going
> >> > to
> >> > > > > have
> >> > > > > >> one
> >> > > > > >> > > by
> >> > > > > >> > > > > the
> >> > > > > >> > > > > >> end
> >> > > > > >> > > > > >> > > of
> >> > > > > >> > > > > >> > > > > the
> >> > > > > >> > > > > >> > > > > >> > > year, it
> >> > > > > >> > > > > >> > > > > >> > > >> > > > seems a
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > bit close to still
> >> be
> >> > > > adding
> >> > > > > in
> >> > > > > >> > > > > "features
> >> > > > > >> > > > > >> > that
> >> > > > > >> > > > > >> > > > need
> >> > > > > >> > > > > >> > > > > >> > > maturing"
> >> > > > > >> > > > > >> > > > > >> > > >> > on
> >> > > > > >> > > > > >> > > > > >> > > >> > > > the
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > branch.
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The lack of a
> >> concrete
> >> > > plan
> >> > > > > for
> >> > > > > >> > 2.0
> >> > > > > >> > > > > keeps
> >> > > > > >> > > > > >> me
> >> > > > > >> > > > > >> > > from
> >> > > > > >> > > > > >> > > > > >> > > considering
> >> > > > > >> > > > > >> > > > > >> > > >> > > these
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > things blocker at
> >> the
> >> > > > moment.
> >> > > > > >> But
> >> > > > > >> > I
> >> > > > > >> > > > know
> >> > > > > >> > > > > >> > first
> >> > > > > >> > > > > >> > > > hand
> >> > > > > >> > > > > >> > > > > >> how
> >> > > > > >> > > > > >> > > > > >> > > much
> >> > > > > >> > > > > >> > > > > >> > > >> > > > trouble
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > folks have had
> with
> >> > other
> >> > > > > >> features
> >> > > > > >> > > > that
> >> > > > > >> > > > > >> have
> >> > > > > >> > > > > >> > > gone
> >> > > > > >> > > > > >> > > > > >> into
> >> > > > > >> > > > > >> > > > > >> > > >> > downstream
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > facing releases
> >> without
> >> > > > > >> robustness
> >> > > > > >> > > > > checks
> >> > > > > >> > > > > >> > (i.e.
> >> > > > > >> > > > > >> > > > > >> > > replication),
> >> > > > > >> > > > > >> > > > > >> > > >> > and
> >> > > > > >> > > > > >> > > > > >> > > >> > > > I'm
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > concerned about
> what
> >> > > we're
> >> > > > > >> setting
> >> > > > > >> > > up
> >> > > > > >> > > > if
> >> > > > > >> > > > > >> 2.0
> >> > > > > >> > > > > >> > > goes
> >> > > > > >> > > > > >> > > > > out
> >> > > > > >> > > > > >> > > > > >> > with
> >> > > > > >> > > > > >> > > > > >> > > >> this
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > feature in its
> >> current
> >> > > > state.
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > --
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > Best regards,
> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > >    - Andy
> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > Problems worthy of
> attack
> >> > prove
> >> > > > > their
> >> > > > > >> > > worth
> >> > > > > >> > > > by
> >> > > > > >> > > > > >> > > hitting
> >> > > > > >> > > > > >> > > > > >> back. -
> >> > > > > >> > > > > >> > > > > >> > > Piet
> >> > > > > >> > > > > >> > > > > >> > > >> > > Hein
> >> > > > > >> > > > > >> > > > > >> > > >> > > > > (via Tom White)
> >> > > > > >> > > > > >> > > > > >> > > >> > > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > >
> >> > > > > >> > > > > >> > > > > >> > > >> > >
> >> > > > > >> > > > > >> > > > > >> > > >> > >
> >> > > > > >> > > > > >> > > > > >> > > >> > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > --
> >> > > > > >> > > > > >> > > > > >> > > >> > > Best regards,
> >> > > > > >> > > > > >> > > > > >> > > >> > >
> >> > > > > >> > > > > >> > > > > >> > > >> > >    - Andy
> >> > > > > >> > > > > >> > > > > >> > > >> > >
> >> > > > > >> > > > > >> > > > > >> > > >> > > Problems worthy of attack
> >> prove
> >> > > their
> >> > > > > >> worth
> >> > > > > >> > by
> >> > > > > >> > > > > >> hitting
> >> > > > > >> > > > > >> > > > back.
> >> > > > > >> > > > > >> > > > > -
> >> > > > > >> > > > > >> > > > > >> > Piet
> >> > > > > >> > > > > >> > > > > >> > > >> Hein
> >> > > > > >> > > > > >> > > > > >> > > >> > > (via Tom White)
> >> > > > > >> > > > > >> > > > > >> > > >> > >
> >> > > > > >> > > > > >> > > > > >> > > >> >
> >> > > > > >> > > > > >> > > > > >> > > >>
> >> > > > > >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > > > > >>
> >> > > > > >> > > > > >> > > > > >
> >> > > > > >> > > > > >> > > > > >
> >> > > > > >> > > > > >> > > > >
> >> > > > > >> > > > > >> > > >
> >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >>
> >> > > > > >> > > > >
> >> > > > > >> > > > >
> >> > > > > >> > > > >
> >> > > > > >> > > > > --
> >> > > > > >> > > > > busbey
> >> > > > > >> > > > >
> >> > > > > >> > > >
> >> > > > > >> > >
> >> > > > > >> >
> >> > > > > >>
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >>
> >>
> >> --
> >>
> >> -- Appy
> >>
> >
> >
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Stack <st...@duboce.net>]

More info:

stack@ve0524:~/hbase.git$ git checkout origin/HBASE-7912 -b 7912v2
Branch 7912v2 set up to track remote branch HBASE-7912 from origin.
Switched to a new branch '7912v2'
stack@ve0524:~/hbase.git$ java -version
java version "1.8.0_101"
Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode)
stack@ve0524:~/hbase.git$ mvn clean install -DskipTests &> /tmp/out.txt

...

St.Ack


On Wed, Oct 12, 2016 at 12:29 PM, Stack <st...@duboce.net> wrote:

> Interesting. When I try it fails w/ below:
>
> [INFO] 26 warnings
> 322 [INFO] -------------------------------------------------------------
> 323 [INFO] -------------------------------------------------------------
> 324 [ERROR] COMPILATION ERROR :
> 325 [INFO] -------------------------------------------------------------
> 326 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.java:[48,8]
> org.apache.hadoop.hbase.io.encoding.rowindexV2.RowIndexCodecV2 is not
> abstract and does not override abstract method createSeeker(org.ap
>  ache.hadoop.hbase.CellComparator,org.apache.hadoop.hbase.io.encoding.HFileBlockDecodingContext)
> in org.apache.hadoop.hbase.io.encoding.DataBlockEncoder
> 327 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.java:[143,3]
> method does not override or implement a method from a supertype
> 328 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.java:[147,29]
> incompatible types: java.nio.ByteBuffer cannot be converted to
> org.apache.hadoop.hbase.nio.ByteBuff
> 329 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.java:[148,33]
> cannot find symbol
> 330   symbol:   method getKeyDeepCopy()
> 331   location: variable seeker of type org.apache.hadoop.hbase.io.
> encoding.DataBlockEncoder.EncodedSeeker
> 332 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.java:[153,3]
> method does not override or implement a method from a supertype
> 333 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> apache/hadoop/hbase/io/encoding/rowindexV1/RowIndexCodecV1.java:[45,8]
> org.apache.hadoop.hbase.io.encoding.rowindexV1.RowIndexCodecV1 is not
> abstract and does not override abstract method createSeeker(org.ap
>  ache.hadoop.hbase.CellComparator,org.apache.hadoop.hbase.io.encoding.HFileBlockDecodingContext)
> in org.apache.hadoop.hbase.io.encoding.DataBlockEncoder
> 334 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> apache/hadoop/hbase/io/encoding/rowindexV1/RowIndexCodecV1.java:[145,3]
> method does not override or implement a method from a supertype
> 335 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> apache/hadoop/hbase/io/encoding/rowindexV1/RowIndexCodecV1.java:[158,3]
> method does not override or implement a method from a supertype
> 336 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.java:[46,8]
> org.apache.hadoop.hbase.io.encoding.rowindexV2.RowIndexSeekerV2 is not
> abstract and does not override abstract method compareKey(org.ap
>  ache.hadoop.hbase.CellComparator,org.apache.hadoop.hbase.Cell) in
> org.apache.hadoop.hbase.io.encoding.DataBlockEncoder.EncodedSeeker
> 337 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.java:[79,3]
> method does not override or implement a method from a supertype
> 338 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.java:[117,3]
> method does not override or implement a method from a supertype
> 339 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.java:[190,3]
> method does not override or implement a method from a supertype
> 340 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.java:[214,3]
> method does not override or implement a method from a supertype
> 341 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.java:[349,3]
> method does not override or implement a method from a supertype
> 342 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.java:[355,3]
> method does not override or implement a method from a supertype
> 343 [ERROR] /home/stack/hbase.git/hbase-common/src/main/java/org/
> apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.java:[421,36]
> no suitable method found for uncompressTags(java.nio.
> ByteBuffer,byte[],int,int)
> 344     method org.apache.hadoop.hbase.io.TagCompressionContext.
> uncompressTags(java.io.InputStream,byte[],int,int) is not applicable
> 345       (argument mismatch; java.nio.ByteBuffer cannot be converted to
> java.io.InputStream)
> 346     method org.apache.hadoop.hbase.io.TagCompressionContext.
> uncompressTags(org.apache.hadoop.hbase.nio.ByteBuff,byte[],int,int) is
> not applicable
> 347       (argument mismatch; java.nio.ByteBuffer cannot be converted to
> org.apache.hadoop.hbase.nio.ByteBuff)
>
> ....
>
> St.Ack
>
> On Tue, Oct 11, 2016 at 5:12 PM, Apekshit Sharma <ap...@cloudera.com>
> wrote:
>
>> @stack, it compiled for me.
>>
>> Also tried few commands, and have to say, it's well designed from user
>> commands perspective.
>>
>> On Tue, Oct 11, 2016 at 5:08 PM, Stack <st...@duboce.net> wrote:
>>
>> > On Tue, Oct 11, 2016 at 1:50 PM, Vladimir Rodionov <
>> vladrodionov@gmail.com
>> > >
>> > wrote:
>> >
>> > > Michael,
>> > >
>> > > Its in HBASE-7912
>> > >
>> > > This is tip of git log:
>> > >
>> > > commit a072f6f49a26a7259ff2aaef6cb56d85eb592482
>> > > Author: Frank Welsch <fw...@jps.net>
>> > > Date:   Fri Sep 23 18:00:42 2016 -0400
>> > >
>> > >     HBASE-16574 Book updates for backup and restore
>> > >
>> > > commit b14e2ab1c24e65ff88dd4c579acf83cb4ed0605e
>> > > Author: tedyu <yu...@gmail.com>
>> > > Date:   Wed Oct 5 16:29:40 2016 -0700
>> > >
>> > >     HBASE-16727 Backup refactoring: remove MR dependencies from
>> HMaster
>> > > (Vladimir Rodionov)
>> > >
>> > >
>> > Thanks. I have that. I tried it and it doesn't compile for me. Does it
>> > compile for you?
>> > Thanks,
>> > M
>> >
>> >
>> >
>> > > -Vlad
>> > >
>> > > On Tue, Oct 11, 2016 at 1:44 PM, Stack <st...@duboce.net> wrote:
>> > >
>> > > > Which branch do I check out to try it? HBASE-7912 is not it. I don't
>> > see
>> > > an
>> > > > HBASE-16727...
>> > > > Thanks,
>> > > > M
>> > > >
>> > > >
>> > > > On Tue, Oct 11, 2016 at 11:41 AM, Vladimir Rodionov <
>> > > > vladrodionov@gmail.com>
>> > > > wrote:
>> > > >
>> > > > > The last patch is on review board:
>> > > > > https://reviews.apache.org/r/52748
>> > > > >
>> > > > > On Mon, Oct 10, 2016 at 3:52 PM, Vladimir Rodionov <
>> > > > vladrodionov@gmail.com
>> > > > > >
>> > > > > wrote:
>> > > > >
>> > > > > > >> How hard to put in an hbase-backup module? hbase-server is
>> fat
>> > > > enough
>> > > > > > >> already. Could be done as a follow-up.
>> > > > > >
>> > > > > > https://issues.apache.org/jira/browse/HBASE-16727?
>> > > > > > focusedCommentId=15531237&page=com.atlassian.jira.
>> > > > > > plugin.system.issuetabpanels:comment-tabpanel#comment-15531237
>> > > > > >
>> > > > > > Can we do merge first? Then we can discuss separate module.
>> > > > > >
>> > > > > > -Vlad
>> > > > > >
>> > > > > > On Mon, Oct 10, 2016 at 3:44 PM, Ted Yu <yu...@gmail.com>
>> > wrote:
>> > > > > >
>> > > > > >> Looks like the first quote was cut off.
>> > > > > >> The original sentence was:
>> > > > > >>
>> > > > > >> bq. no mapreduce job launched from master or region server.
>> > > > > >>
>> > > > > >> mapreduce job is launched from the node where command line
>> tool is
>> > > > run.
>> > > > > >>
>> > > > > >> On Mon, Oct 10, 2016 at 3:38 PM, Stack <st...@duboce.net>
>> wrote:
>> > > > > >>
>> > > > > >> > bq. launched from master or region server.
>> > > > > >> >
>> > > > > >> > What does this mean please? Has to be run from Master or
>> > > > RegionServer?
>> > > > > >> Can
>> > > > > >> > it be run from another node altogether?
>> > > > > >> >
>> > > > > >> > On Mon, Oct 10, 2016 at 1:44 PM, Vladimir Rodionov <
>> > > > > >> vladrodionov@gmail.com
>> > > > > >> > >
>> > > > > >> > wrote:
>> > > > > >> >
>> > > > > >> > > >> mapreduce dependency has been moved to client side - no
>> > > > mapreduce
>> > > > > >> job
>> > > > > >> > >
>> > > > > >> > > 1. We have no code in the client module anymore, due to
>> > > dependency
>> > > > > on
>> > > > > >> > > internal server API (HFile and WAL access).
>> > > > > >> > > 2. Backup/ restore are client - driven operations, but all
>> the
>> > > > code
>> > > > > >> > resides
>> > > > > >> > > in the server module
>> > > > > >> > >
>> > > > > >> >
>> > > > > >> > How hard to put in an hbase-backup module? hbase-server is
>> fat
>> > > > enough
>> > > > > >> > already. Could be done as a follow-up.
>> > > > > >> >
>> > > > > >> > Thanks,
>> > > > > >> > St.Ack
>> > > > > >> >
>> > > > > >> >
>> > > > > >> >
>> > > > > >> > > 3. No MR in Master, no procedure - driven execution.
>> > > > > >> > > 4. Old good MR from command-line.
>> > > > > >> > > 5. Security was simplified and now only super-user is
>> allowed
>> > to
>> > > > run
>> > > > > >> > > backup/restores.
>> > > > > >> > > 6. HBase Backup API was gone due to 1. Now only
>> command-line
>> > > > access
>> > > > > to
>> > > > > >> > > backup tools.
>> > > > > >> > >
>> > > > > >> > > These consequences of refactoring has been discussed in
>> > > > HBASE-16727.
>> > > > > >> > >
>> > > > > >> > > -Vlad
>> > > > > >> > >
>> > > > > >> > >
>> > > > > >> > >
>> > > > > >> > >
>> > > > > >> > > On Mon, Oct 10, 2016 at 1:31 PM, Ted Yu <
>> yuzhihong@gmail.com>
>> > > > > wrote:
>> > > > > >> > >
>> > > > > >> > > > Reviving this thread.
>> > > > > >> > > >
>> > > > > >> > > > The following has taken place:
>> > > > > >> > > >
>> > > > > >> > > > mapreduce dependency has been moved to client side - no
>> > > > mapreduce
>> > > > > >> job
>> > > > > >> > > > launched from master or region server.
>> > > > > >> > > > document patch (HBASE-16574) has been integrated.
>> > > > > >> > > > Updated mega patch has been attached to HBASE-14123: this
>> > > covers
>> > > > > the
>> > > > > >> > > > refactor in #1 above and the protobuf 3 merge.
>> > > > > >> > > >
>> > > > > >> > > > If community has more feedback on the merge proposal, I
>> > would
>> > > > love
>> > > > > >> to
>> > > > > >> > > hear
>> > > > > >> > > > it.
>> > > > > >> > > >
>> > > > > >> > > > Thanks
>> > > > > >> > > >
>> > > > > >> > > > On Thu, Sep 22, 2016 at 10:31 AM, Sean Busbey <
>> > > > > busbey@cloudera.com>
>> > > > > >> > > wrote:
>> > > > > >> > > >
>> > > > > >> > > > > I'd like to see the docs proposed on HBASE-16574
>> > integrated
>> > > > into
>> > > > > >> our
>> > > > > >> > > > > project's documentation prior to merge.
>> > > > > >> > > > >
>> > > > > >> > > > > On Thu, Sep 22, 2016 at 9:02 AM, Ted Yu <
>> > > yuzhihong@gmail.com>
>> > > > > >> wrote:
>> > > > > >> > > > > > This feature can be marked experimental due to some
>> > > > > limitations
>> > > > > >> > such
>> > > > > >> > > as
>> > > > > >> > > > > > security.
>> > > > > >> > > > > >
>> > > > > >> > > > > > Your previous round of comments have been addressed.
>> > > > > >> > > > > > Command line tool has gone through:
>> > > > > >> > > > > >
>> > > > > >> > > > > > HBASE-16620 Fix backup command-line tool usability
>> > issues
>> > > > > >> > > > > > HBASE-16655 hbase backup describe with incorrect
>> backup
>> > id
>> > > > > >> results
>> > > > > >> > in
>> > > > > >> > > > NPE
>> > > > > >> > > > > >
>> > > > > >> > > > > > The updated doc has been attached to HBASE-16574.
>> > > > > >> > > > > >
>> > > > > >> > > > > > Cheers
>> > > > > >> > > > > >
>> > > > > >> > > > > > On Thu, Sep 22, 2016 at 8:53 AM, Stack <
>> > stack@duboce.net>
>> > > > > >> wrote:
>> > > > > >> > > > > >
>> > > > > >> > > > > >> On Wed, Sep 21, 2016 at 7:43 AM, Ted Yu <
>> > > > yuzhihong@gmail.com
>> > > > > >
>> > > > > >> > > wrote:
>> > > > > >> > > > > >>
>> > > > > >> > > > > >> > Are there more (review) comments ?
>> > > > > >> > > > > >> >
>> > > > > >> > > > > >> >
>> > > > > >> > > > > >> Are outstanding comments addressed?
>> > > > > >> > > > > >>
>> > > > > >> > > > > >> I don't see answer to my 'is this experimental/will
>> it
>> > be
>> > > > > >> marked
>> > > > > >> > > > > >> experimental' question.
>> > > > > >> > > > > >>
>> > > > > >> > > > > >> I ran into some issues trying to use the feature and
>> > > > > suggested
>> > > > > >> > that
>> > > > > >> > > a
>> > > > > >> > > > > >> feature likes this needs polish else it'll just rot,
>> > > > unused.
>> > > > > >> Has
>> > > > > >> > > > polish
>> > > > > >> > > > > >> been applied? All ready for another 'user' test?
>> > Suggest
>> > > > that
>> > > > > >> you
>> > > > > >> > > > update
>> > > > > >> > > > > >> here going forward for the benefit of those trying
>> to
>> > > > follow
>> > > > > >> along
>> > > > > >> > > and
>> > > > > >> > > > > who
>> > > > > >> > > > > >> are not watching JIRA change fly-by.
>> > > > > >> > > > > >>
>> > > > > >> > > > > >> It looks like doc got a revision -- I have to check
>> --
>> > to
>> > > > > take
>> > > > > >> on
>> > > > > >> > > > > >> suggestion made above but again, suggest, that this
>> > > thread
>> > > > > gets
>> > > > > >> > > > updated.
>> > > > > >> > > > > >>
>> > > > > >> > > > > >> Thanks,
>> > > > > >> > > > > >> St.Ack
>> > > > > >> > > > > >>
>> > > > > >> > > > > >>
>> > > > > >> > > > > >>
>> > > > > >> > > > > >> > Thanks
>> > > > > >> > > > > >> >
>> > > > > >> > > > > >> > On Tue, Sep 20, 2016 at 10:02 AM, Devaraj Das <
>> > > > > >> > > ddas@hortonworks.com
>> > > > > >> > > > >
>> > > > > >> > > > > >> > wrote:
>> > > > > >> > > > > >> >
>> > > > > >> > > > > >> > > Just reviving this thread. Thanks Sean, Stack,
>> > Dima,
>> > > > and
>> > > > > >> > others
>> > > > > >> > > > for
>> > > > > >> > > > > the
>> > > > > >> > > > > >> > > thorough reviews and testing. Thanks Ted and
>> Vlad
>> > for
>> > > > > >> taking
>> > > > > >> > > care
>> > > > > >> > > > of
>> > > > > >> > > > > >> the
>> > > > > >> > > > > >> > > feedback. Are we all good to do the merge now?
>> > Rather
>> > > > do
>> > > > > >> > sooner
>> > > > > >> > > > than
>> > > > > >> > > > > >> > later.
>> > > > > >> > > > > >> > > ________________________________________
>> > > > > >> > > > > >> > > From: saint.ack@gmail.com <sa...@gmail.com>
>> on
>> > > > > behalf
>> > > > > >> of
>> > > > > >> > > > Stack
>> > > > > >> > > > > <
>> > > > > >> > > > > >> > > stack@duboce.net>
>> > > > > >> > > > > >> > > Sent: Monday, September 12, 2016 1:18 PM
>> > > > > >> > > > > >> > > To: HBase Dev List
>> > > > > >> > > > > >> > > Subject: Re: [DISCUSSION] Merge Backup /
>> Restore -
>> > > > Branch
>> > > > > >> > > > HBASE-7912
>> > > > > >> > > > > >> > >
>> > > > > >> > > > > >> > > On Mon, Sep 12, 2016 at 12:19 PM, Ted Yu <
>> > > > > >> yuzhihong@gmail.com
>> > > > > >> > >
>> > > > > >> > > > > wrote:
>> > > > > >> > > > > >> > >
>> > > > > >> > > > > >> > > > Mega patch (rev 18) is on HBASE-14123.
>> > > > > >> > > > > >> > > >
>> > > > > >> > > > > >> > > > Please comment on HBASE-14123 on how you want
>> to
>> > > > > review.
>> > > > > >> > > > > >> > > >
>> > > > > >> > > > > >> > >
>> > > > > >> > > > > >> > >
>> > > > > >> > > > > >> > > Yeah. That was my lost tab. Last rb was 6 months
>> > ago.
>> > > > > >> Suggest
>> > > > > >> > > > > updating
>> > > > > >> > > > > >> > it.
>> > > > > >> > > > > >> > > RB is pretty good for review. Patch is only
>> 1.5M so
>> > > > > should
>> > > > > >> be
>> > > > > >> > > > fine.
>> > > > > >> > > > > >> > >
>> > > > > >> > > > > >> > > St.Ack
>> > > > > >> > > > > >> > >
>> > > > > >> > > > > >> > >
>> > > > > >> > > > > >> > > >
>> > > > > >> > > > > >> > > > Thanks
>> > > > > >> > > > > >> > > >
>> > > > > >> > > > > >> > > > On Mon, Sep 12, 2016 at 12:15 PM, Stack <
>> > > > > >> stack@duboce.net>
>> > > > > >> > > > wrote:
>> > > > > >> > > > > >> > > >
>> > > > > >> > > > > >> > > > > On review of the 'patch', do I just compare
>> the
>> > > > > branch
>> > > > > >> to
>> > > > > >> > > > > master or
>> > > > > >> > > > > >> > is
>> > > > > >> > > > > >> > > > > there a megapatch posted somewhere (I think
>> I
>> > saw
>> > > > one
>> > > > > >> but
>> > > > > >> > it
>> > > > > >> > > > > seemed
>> > > > > >> > > > > >> > > stale
>> > > > > >> > > > > >> > > > > and then I 'lost' the tab). Sorry for dumb
>> > > > question.
>> > > > > >> > > > > >> > > > > St.Ack
>> > > > > >> > > > > >> > > > >
>> > > > > >> > > > > >> > > > > On Mon, Sep 12, 2016 at 12:01 PM, Stack <
>> > > > > >> stack@duboce.net
>> > > > > >> > >
>> > > > > >> > > > > wrote:
>> > > > > >> > > > > >> > > > >
>> > > > > >> > > > > >> > > > > > Late to the game. A few comments after
>> > > rereading
>> > > > > this
>> > > > > >> > > thread
>> > > > > >> > > > > as a
>> > > > > >> > > > > >> > > > 'user'.
>> > > > > >> > > > > >> > > > > >
>> > > > > >> > > > > >> > > > > > + Before merge, a user-facing feature like
>> > this
>> > > > > >> should
>> > > > > >> > > work
>> > > > > >> > > > > (If
>> > > > > >> > > > > >> > this
>> > > > > >> > > > > >> > > is
>> > > > > >> > > > > >> > > > > "higher-bar
>> > > > > >> > > > > >> > > > > > for new features", bring it on -- smile).
>> > > > > >> > > > > >> > > > > > + As a user, I tried the branch with tools
>> > > after
>> > > > > >> > reviewing
>> > > > > >> > > > the
>> > > > > >> > > > > >> > > > > just-posted
>> > > > > >> > > > > >> > > > > > doc. I had an 'interesting' experience
>> (left
>> > > > > >> comments up
>> > > > > >> > > on
>> > > > > >> > > > > >> > issue). I
>> > > > > >> > > > > >> > > > > think
>> > > > > >> > > > > >> > > > > > the tooling/doc. important to get right.
>> If
>> > it
>> > > > > breaks
>> > > > > >> > > easily
>> > > > > >> > > > > or
>> > > > > >> > > > > >> is
>> > > > > >> > > > > >> > > > > > inconsistent (or lacks 'polish'),
>> operators
>> > > will
>> > > > > >> judge
>> > > > > >> > the
>> > > > > >> > > > > whole
>> > > > > >> > > > > >> > > > > > backup/restore tooling chain as not
>> > trustworthy
>> > > > and
>> > > > > >> > > abandon
>> > > > > >> > > > > it.
>> > > > > >> > > > > >> > Lets
>> > > > > >> > > > > >> > > > not
>> > > > > >> > > > > >> > > > > > have this happen to this feature.
>> > > > > >> > > > > >> > > > > > + Matteo's suggestion (with a helpful
>> starter
>> > > > list)
>> > > > > >> that
>> > > > > >> > > > there
>> > > > > >> > > > > >> > needs
>> > > > > >> > > > > >> > > to
>> > > > > >> > > > > >> > > > > be
>> > > > > >> > > > > >> > > > > > explicit qualification on what is actually
>> > > being
>> > > > > >> > delivered
>> > > > > >> > > > --
>> > > > > >> > > > > >> > > > including a
>> > > > > >> > > > > >> > > > > > listing of limitations (some look serious
>> > such
>> > > as
>> > > > > >> data
>> > > > > >> > > bleed
>> > > > > >> > > > > from
>> > > > > >> > > > > >> > > other
>> > > > > >> > > > > >> > > > > > regions in WALs, but maybe I don't care
>> for
>> > my
>> > > > use
>> > > > > >> > > case...)
>> > > > > >> > > > --
>> > > > > >> > > > > >> > needs
>> > > > > >> > > > > >> > > to
>> > > > > >> > > > > >> > > > > > accompany the merge. Lets fold them into
>> the
>> > > user
>> > > > > >> doc.
>> > > > > >> > in
>> > > > > >> > > > the
>> > > > > >> > > > > >> > > technical
>> > > > > >> > > > > >> > > > > > overview area as suggested so user
>> > expectations
>> > > > are
>> > > > > >> > > properly
>> > > > > >> > > > > >> > managed
>> > > > > >> > > > > >> > > > > > (otherwise, they expect the world and will
>> > just
>> > > > > give
>> > > > > >> up
>> > > > > >> > > when
>> > > > > >> > > > > we
>> > > > > >> > > > > >> > fall
>> > > > > >> > > > > >> > > > > > short). Vladimir did a list of what is in
>> > each
>> > > of
>> > > > > the
>> > > > > >> > > phases
>> > > > > >> > > > > >> above
>> > > > > >> > > > > >> > > > which
>> > > > > >> > > > > >> > > > > > would serve as a good start.
>> > > > > >> > > > > >> > > > > > + Is this feature 'experimental' (Matteo
>> asks
>> > > > > above).
>> > > > > >> > I'd
>> > > > > >> > > > > prefer
>> > > > > >> > > > > >> it
>> > > > > >> > > > > >> > > is
>> > > > > >> > > > > >> > > > > > not. If it is, it should be labelled all
>> over
>> > > > that
>> > > > > >> it is
>> > > > > >> > > > so. I
>> > > > > >> > > > > >> see
>> > > > > >> > > > > >> > > > > current
>> > > > > >> > > > > >> > > > > > state called out as a '... technical
>> preview
>> > > > > >> feature'.
>> > > > > >> > > Does
>> > > > > >> > > > > this
>> > > > > >> > > > > >> > mean
>> > > > > >> > > > > >> > > > > > not-for-users?
>> > > > > >> > > > > >> > > > > >
>> > > > > >> > > > > >> > > > > > St.Ack
>> > > > > >> > > > > >> > > > > >
>> > > > > >> > > > > >> > > > > >
>> > > > > >> > > > > >> > > > > >
>> > > > > >> > > > > >> > > > > >
>> > > > > >> > > > > >> > > > > >
>> > > > > >> > > > > >> > > > > >
>> > > > > >> > > > > >> > > > > >
>> > > > > >> > > > > >> > > > > >
>> > > > > >> > > > > >> > > > > >
>> > > > > >> > > > > >> > > > > >
>> > > > > >> > > > > >> > > > > >
>> > > > > >> > > > > >> > > > > > On Mon, Sep 12, 2016 at 8:03 AM, Ted Yu <
>> > > > > >> > > > yuzhihong@gmail.com>
>> > > > > >> > > > > >> > wrote:
>> > > > > >> > > > > >> > > > > >
>> > > > > >> > > > > >> > > > > >> Sean:
>> > > > > >> > > > > >> > > > > >> Do you have more comments ?
>> > > > > >> > > > > >> > > > > >>
>> > > > > >> > > > > >> > > > > >> Cheers
>> > > > > >> > > > > >> > > > > >>
>> > > > > >> > > > > >> > > > > >> On Fri, Sep 9, 2016 at 1:42 PM, Vladimir
>> > > > Rodionov
>> > > > > <
>> > > > > >> > > > > >> > > > > vladrodionov@gmail.com
>> > > > > >> > > > > >> > > > > >> >
>> > > > > >> > > > > >> > > > > >> wrote:
>> > > > > >> > > > > >> > > > > >>
>> > > > > >> > > > > >> > > > > >> > Sean,
>> > > > > >> > > > > >> > > > > >> >
>> > > > > >> > > > > >> > > > > >> > Backup/Restore can fail due to various
>> > > > reasons:
>> > > > > >> > network
>> > > > > >> > > > > outage
>> > > > > >> > > > > >> > > > > (cluster
>> > > > > >> > > > > >> > > > > >> > wide), various time-outs in HBase and
>> HDFS
>> > > > > layer,
>> > > > > >> M/R
>> > > > > >> > > > > failure
>> > > > > >> > > > > >> > due
>> > > > > >> > > > > >> > > to
>> > > > > >> > > > > >> > > > > >> "HDFS
>> > > > > >> > > > > >> > > > > >> > exceeded quota", user error (manual
>> > deletion
>> > > > of
>> > > > > >> data)
>> > > > > >> > > and
>> > > > > >> > > > > so
>> > > > > >> > > > > >> on
>> > > > > >> > > > > >> > so
>> > > > > >> > > > > >> > > > on.
>> > > > > >> > > > > >> > > > > >> That
>> > > > > >> > > > > >> > > > > >> > is impossible to enumerate all possible
>> > > types
>> > > > of
>> > > > > >> > > failures
>> > > > > >> > > > > in a
>> > > > > >> > > > > >> > > > > >> distributed
>> > > > > >> > > > > >> > > > > >> > system - that is not our goal/task.
>> > > > > >> > > > > >> > > > > >> >
>> > > > > >> > > > > >> > > > > >> > We focus completely on backup system
>> table
>> > > > > >> > consistency
>> > > > > >> > > > in a
>> > > > > >> > > > > >> > > presence
>> > > > > >> > > > > >> > > > > of
>> > > > > >> > > > > >> > > > > >> any
>> > > > > >> > > > > >> > > > > >> > type of failure. That is what I call
>> > > > "tolerance
>> > > > > to
>> > > > > >> > > > > failures".
>> > > > > >> > > > > >> > > > > >> >
>> > > > > >> > > > > >> > > > > >> > On a failure:
>> > > > > >> > > > > >> > > > > >> >
>> > > > > >> > > > > >> > > > > >> > BACKUP. All backup system information
>> > (prior
>> > > > to
>> > > > > >> > backup)
>> > > > > >> > > > > will
>> > > > > >> > > > > >> be
>> > > > > >> > > > > >> > > > > restored
>> > > > > >> > > > > >> > > > > >> > and all temporary data, related to a
>> > failed
>> > > > > >> session,
>> > > > > >> > in
>> > > > > >> > > > > HDFS
>> > > > > >> > > > > >> > will
>> > > > > >> > > > > >> > > be
>> > > > > >> > > > > >> > > > > >> > deleted
>> > > > > >> > > > > >> > > > > >> > RESTORE. We do not care about system
>> data,
>> > > > > because
>> > > > > >> > > > restore
>> > > > > >> > > > > >> does
>> > > > > >> > > > > >> > > not
>> > > > > >> > > > > >> > > > > >> change
>> > > > > >> > > > > >> > > > > >> > it. Temporary data in HDFS will be
>> cleaned
>> > > up
>> > > > > and
>> > > > > >> > table
>> > > > > >> > > > > will
>> > > > > >> > > > > >> be
>> > > > > >> > > > > >> > > in a
>> > > > > >> > > > > >> > > > > >> state
>> > > > > >> > > > > >> > > > > >> > back to where it was before operation
>> > > started.
>> > > > > >> > > > > >> > > > > >> >
>> > > > > >> > > > > >> > > > > >> > This is what user should expect in case
>> > of a
>> > > > > >> failure.
>> > > > > >> > > > > >> > > > > >> >
>> > > > > >> > > > > >> > > > > >> > -Vlad
>> > > > > >> > > > > >> > > > > >> >
>> > > > > >> > > > > >> > > > > >> >
>> > > > > >> > > > > >> > > > > >> > -Vlad
>> > > > > >> > > > > >> > > > > >> >
>> > > > > >> > > > > >> > > > > >> > On Fri, Sep 9, 2016 at 12:56 PM, Sean
>> > > Busbey <
>> > > > > >> > > > > >> busbey@apache.org
>> > > > > >> > > > > >> > >
>> > > > > >> > > > > >> > > > > wrote:
>> > > > > >> > > > > >> > > > > >> >
>> > > > > >> > > > > >> > > > > >> > > Failing in a consistent way, with
>> docs
>> > > that
>> > > > > >> explain
>> > > > > >> > > the
>> > > > > >> > > > > >> > various
>> > > > > >> > > > > >> > > > > >> > > expected failures would be
>> sufficient.
>> > > > > >> > > > > >> > > > > >> > >
>> > > > > >> > > > > >> > > > > >> > > On Fri, Sep 9, 2016 at 12:16 PM,
>> > Vladimir
>> > > > > >> Rodionov
>> > > > > >> > > > > >> > > > > >> > > <vl...@gmail.com> wrote:
>> > > > > >> > > > > >> > > > > >> > > > Do not worry Sean, doc is coming
>> today
>> > > as
>> > > > a
>> > > > > >> > preview
>> > > > > >> > > > and
>> > > > > >> > > > > >> our
>> > > > > >> > > > > >> > > > writer
>> > > > > >> > > > > >> > > > > >> > Frank
>> > > > > >> > > > > >> > > > > >> > > > will be working on a putting  it
>> into
>> > > > Apache
>> > > > > >> > repo.
>> > > > > >> > > > > >> Timeline
>> > > > > >> > > > > >> > > > > depends
>> > > > > >> > > > > >> > > > > >> on
>> > > > > >> > > > > >> > > > > >> > > > Franks schedule but I hope we will
>> get
>> > > it
>> > > > > >> rather
>> > > > > >> > > > sooner
>> > > > > >> > > > > >> than
>> > > > > >> > > > > >> > > > > later.
>> > > > > >> > > > > >> > > > > >> > > >
>> > > > > >> > > > > >> > > > > >> > > > As for failure testing, we are
>> > focusing
>> > > > only
>> > > > > >> on a
>> > > > > >> > > > > >> consistent
>> > > > > >> > > > > >> > > > state
>> > > > > >> > > > > >> > > > > >> of
>> > > > > >> > > > > >> > > > > >> > > > backup system data in a presence of
>> > any
>> > > > type
>> > > > > >> of
>> > > > > >> > > > > failures,
>> > > > > >> > > > > >> We
>> > > > > >> > > > > >> > > are
>> > > > > >> > > > > >> > > > > not
>> > > > > >> > > > > >> > > > > >> > > going
>> > > > > >> > > > > >> > > > > >> > > > to implement  anything more
>> "fancy",
>> > > than
>> > > > > >> that.
>> > > > > >> > We
>> > > > > >> > > > > allow
>> > > > > >> > > > > >> > both:
>> > > > > >> > > > > >> > > > > >> backup
>> > > > > >> > > > > >> > > > > >> > and
>> > > > > >> > > > > >> > > > > >> > > > restore to fail. What we do not
>> allow
>> > is
>> > > > to
>> > > > > >> have
>> > > > > >> > > > system
>> > > > > >> > > > > >> data
>> > > > > >> > > > > >> > > > > >> corrupted.
>> > > > > >> > > > > >> > > > > >> > > > Will it suffice for you? Do you
>> have
>> > any
>> > > > > other
>> > > > > >> > > > > concerns,
>> > > > > >> > > > > >> you
>> > > > > >> > > > > >> > > > want
>> > > > > >> > > > > >> > > > > >> us to
>> > > > > >> > > > > >> > > > > >> > > > address?
>> > > > > >> > > > > >> > > > > >> > > >
>> > > > > >> > > > > >> > > > > >> > > > -Vlad
>> > > > > >> > > > > >> > > > > >> > > >
>> > > > > >> > > > > >> > > > > >> > > >
>> > > > > >> > > > > >> > > > > >> > > > On Fri, Sep 9, 2016 at 10:56 AM,
>> Sean
>> > > > > Busbey <
>> > > > > >> > > > > >> > > busbey@apache.org
>> > > > > >> > > > > >> > > > >
>> > > > > >> > > > > >> > > > > >> > wrote:
>> > > > > >> > > > > >> > > > > >> > > >
>> > > > > >> > > > > >> > > > > >> > > >> "docs will come to Apache soon"
>> does
>> > > not
>> > > > > >> address
>> > > > > >> > > my
>> > > > > >> > > > > >> concern
>> > > > > >> > > > > >> > > > > around
>> > > > > >> > > > > >> > > > > >> > docs
>> > > > > >> > > > > >> > > > > >> > > at
>> > > > > >> > > > > >> > > > > >> > > >> all, unless said docs have already
>> > made
>> > > > it
>> > > > > >> into
>> > > > > >> > > the
>> > > > > >> > > > > >> project
>> > > > > >> > > > > >> > > > > repo. I
>> > > > > >> > > > > >> > > > > >> > > don't
>> > > > > >> > > > > >> > > > > >> > > >> want third party resources for
>> using
>> > a
>> > > > > major
>> > > > > >> and
>> > > > > >> > > > > >> important
>> > > > > >> > > > > >> > > > > feature
>> > > > > >> > > > > >> > > > > >> of
>> > > > > >> > > > > >> > > > > >> > > the
>> > > > > >> > > > > >> > > > > >> > > >> project, I want us to provide end
>> > users
>> > > > > with
>> > > > > >> > what
>> > > > > >> > > > they
>> > > > > >> > > > > >> need
>> > > > > >> > > > > >> > > to
>> > > > > >> > > > > >> > > > > get
>> > > > > >> > > > > >> > > > > >> the
>> > > > > >> > > > > >> > > > > >> > > job
>> > > > > >> > > > > >> > > > > >> > > >> done.
>> > > > > >> > > > > >> > > > > >> > > >>
>> > > > > >> > > > > >> > > > > >> > > >> I see some calls for patience on
>> the
>> > > > > failure
>> > > > > >> > > > testing,
>> > > > > >> > > > > but
>> > > > > >> > > > > >> > the
>> > > > > >> > > > > >> > > > > >> appeal
>> > > > > >> > > > > >> > > > > >> > to
>> > > > > >> > > > > >> > > > > >> > > us
>> > > > > >> > > > > >> > > > > >> > > >> having done a bad job of requiring
>> > > proper
>> > > > > >> tests
>> > > > > >> > of
>> > > > > >> > > > > >> previous
>> > > > > >> > > > > >> > > > > >> features
>> > > > > >> > > > > >> > > > > >> > > just
>> > > > > >> > > > > >> > > > > >> > > >> makes me more concerned about not
>> > > getting
>> > > > > >> them
>> > > > > >> > > > here. I
>> > > > > >> > > > > >> > don't
>> > > > > >> > > > > >> > > > want
>> > > > > >> > > > > >> > > > > >> to
>> > > > > >> > > > > >> > > > > >> > set
>> > > > > >> > > > > >> > > > > >> > > >> yet another bad example that will
>> > then
>> > > be
>> > > > > >> > pointed
>> > > > > >> > > to
>> > > > > >> > > > > in
>> > > > > >> > > > > >> the
>> > > > > >> > > > > >> > > > > future.
>> > > > > >> > > > > >> > > > > >> > > >>
>> > > > > >> > > > > >> > > > > >> > > >> On Sep 8, 2016 10:50, "Ted Yu" <
>> > > > > >> > > yuzhihong@gmail.com
>> > > > > >> > > > >
>> > > > > >> > > > > >> > wrote:
>> > > > > >> > > > > >> > > > > >> > > >>
>> > > > > >> > > > > >> > > > > >> > > >> > Is there any concern which is
>> not
>> > > > > >> addressed ?
>> > > > > >> > > > > >> > > > > >> > > >> >
>> > > > > >> > > > > >> > > > > >> > > >> > Do we need another Vote thread ?
>> > > > > >> > > > > >> > > > > >> > > >> >
>> > > > > >> > > > > >> > > > > >> > > >> > Thanks
>> > > > > >> > > > > >> > > > > >> > > >> >
>> > > > > >> > > > > >> > > > > >> > > >> > On Thu, Sep 8, 2016 at 9:21 AM,
>> > > Andrew
>> > > > > >> > Purtell <
>> > > > > >> > > > > >> > > > > >> apurtell@apache.org
>> > > > > >> > > > > >> > > > > >> > >
>> > > > > >> > > > > >> > > > > >> > > >> > wrote:
>> > > > > >> > > > > >> > > > > >> > > >> >
>> > > > > >> > > > > >> > > > > >> > > >> > > Vlad,
>> > > > > >> > > > > >> > > > > >> > > >> > >
>> > > > > >> > > > > >> > > > > >> > > >> > > I apologize for using the term
>> > > > > >> 'half-baked'
>> > > > > >> > > in a
>> > > > > >> > > > > way
>> > > > > >> > > > > >> > that
>> > > > > >> > > > > >> > > > > could
>> > > > > >> > > > > >> > > > > >> > > seem a
>> > > > > >> > > > > >> > > > > >> > > >> > > description of HBASE-7912. I
>> > meant
>> > > > that
>> > > > > >> as a
>> > > > > >> > > > > general
>> > > > > >> > > > > >> > > > > >> hypothetical.
>> > > > > >> > > > > >> > > > > >> > > >> > >
>> > > > > >> > > > > >> > > > > >> > > >> > > On Wed, Sep 7, 2016 at 9:36
>> AM,
>> > > > > Vladimir
>> > > > > >> > > > Rodionov
>> > > > > >> > > > > <
>> > > > > >> > > > > >> > > > > >> > > >> > vladrodionov@gmail.com>
>> > > > > >> > > > > >> > > > > >> > > >> > > wrote:
>> > > > > >> > > > > >> > > > > >> > > >> > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > >> I'm not sure that "There
>> is
>> > > > > already
>> > > > > >> > lots
>> > > > > >> > > of
>> > > > > >> > > > > >> > > half-baked
>> > > > > >> > > > > >> > > > > >> code
>> > > > > >> > > > > >> > > > > >> > in
>> > > > > >> > > > > >> > > > > >> > > the
>> > > > > >> > > > > >> > > > > >> > > >> > > > branch,
>> > > > > >> > > > > >> > > > > >> > > >> > > > so what's the harm in adding
>> > > more?"
>> > > > > >> > > > > >> > > > > >> > > >> > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > I meant - not production -
>> > ready
>> > > > yet.
>> > > > > >> This
>> > > > > >> > > is
>> > > > > >> > > > > 2.0
>> > > > > >> > > > > >> > > > > development
>> > > > > >> > > > > >> > > > > >> > > branch
>> > > > > >> > > > > >> > > > > >> > > >> > and,
>> > > > > >> > > > > >> > > > > >> > > >> > > > hence many features are in
>> > works,
>> > > > > >> > > > > >> > > > > >> > > >> > > > not being tested well etc.
>> I do
>> > > not
>> > > > > >> > consider
>> > > > > >> > > > > backup
>> > > > > >> > > > > >> > as
>> > > > > >> > > > > >> > > > half
>> > > > > >> > > > > >> > > > > >> > baked
>> > > > > >> > > > > >> > > > > >> > > >> > > feature -
>> > > > > >> > > > > >> > > > > >> > > >> > > > it has passed our internal
>> QA
>> > and
>> > > > has
>> > > > > >> very
>> > > > > >> > > > good
>> > > > > >> > > > > >> doc,
>> > > > > >> > > > > >> > > > which
>> > > > > >> > > > > >> > > > > we
>> > > > > >> > > > > >> > > > > >> > will
>> > > > > >> > > > > >> > > > > >> > > >> > > provide
>> > > > > >> > > > > >> > > > > >> > > >> > > > to Apache shortly.
>> > > > > >> > > > > >> > > > > >> > > >> > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > -Vlad
>> > > > > >> > > > > >> > > > > >> > > >> > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > On Wed, Sep 7, 2016 at 9:13
>> AM,
>> > > > > Andrew
>> > > > > >> > > > Purtell <
>> > > > > >> > > > > >> > > > > >> > > apurtell@apache.org>
>> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
>> > > > > >> > > > > >> > > > > >> > > >> > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > We shouldn't admit half
>> baked
>> > > > > changes
>> > > > > >> > that
>> > > > > >> > > > > won't
>> > > > > >> > > > > >> be
>> > > > > >> > > > > >> > > > > >> finished.
>> > > > > >> > > > > >> > > > > >> > > >> However
>> > > > > >> > > > > >> > > > > >> > > >> > > in
>> > > > > >> > > > > >> > > > > >> > > >> > > > > this case the crew
>> working on
>> > > > this
>> > > > > >> > feature
>> > > > > >> > > > are
>> > > > > >> > > > > >> long
>> > > > > >> > > > > >> > > > > timers
>> > > > > >> > > > > >> > > > > >> and
>> > > > > >> > > > > >> > > > > >> > > less
>> > > > > >> > > > > >> > > > > >> > > >> > > > likely
>> > > > > >> > > > > >> > > > > >> > > >> > > > > than just about anyone to
>> > leave
>> > > > > >> > something
>> > > > > >> > > > in a
>> > > > > >> > > > > >> half
>> > > > > >> > > > > >> > > > baked
>> > > > > >> > > > > >> > > > > >> > > state. Of
>> > > > > >> > > > > >> > > > > >> > > >> > > > course
>> > > > > >> > > > > >> > > > > >> > > >> > > > > there is no guarantee how
>> > > > anything
>> > > > > >> will
>> > > > > >> > > turn
>> > > > > >> > > > > out,
>> > > > > >> > > > > >> > > but I
>> > > > > >> > > > > >> > > > > am
>> > > > > >> > > > > >> > > > > >> > > willing
>> > > > > >> > > > > >> > > > > >> > > >> to
>> > > > > >> > > > > >> > > > > >> > > >> > > > take
>> > > > > >> > > > > >> > > > > >> > > >> > > > > a little on faith if they
>> > feel
>> > > > > their
>> > > > > >> > best
>> > > > > >> > > > path
>> > > > > >> > > > > >> > > forward
>> > > > > >> > > > > >> > > > > now
>> > > > > >> > > > > >> > > > > >> is
>> > > > > >> > > > > >> > > > > >> > to
>> > > > > >> > > > > >> > > > > >> > > >> > merge
>> > > > > >> > > > > >> > > > > >> > > >> > > to
>> > > > > >> > > > > >> > > > > >> > > >> > > > > trunk. I only wish I had
>> > > > bandwidth
>> > > > > to
>> > > > > >> > have
>> > > > > >> > > > > done
>> > > > > >> > > > > >> > some
>> > > > > >> > > > > >> > > > real
>> > > > > >> > > > > >> > > > > >> > > kicking
>> > > > > >> > > > > >> > > > > >> > > >> of
>> > > > > >> > > > > >> > > > > >> > > >> > > the
>> > > > > >> > > > > >> > > > > >> > > >> > > > > tires by now. Maybe this
>> > week.
>> > > > > >> > > > > >> > > > > >> > > >> > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > (Yes, I'm using some of
>> that
>> > > time
>> > > > > for
>> > > > > >> > this
>> > > > > >> > > > > email
>> > > > > >> > > > > >> > :-)
>> > > > > >> > > > > >> > > > but
>> > > > > >> > > > > >> > > > > I
>> > > > > >> > > > > >> > > > > >> > type
>> > > > > >> > > > > >> > > > > >> > > >> > fast.)
>> > > > > >> > > > > >> > > > > >> > > >> > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > That said, I would like to
>> > > > agitate
>> > > > > >> for
>> > > > > >> > > > making
>> > > > > >> > > > > 2.0
>> > > > > >> > > > > >> > > more
>> > > > > >> > > > > >> > > > > real
>> > > > > >> > > > > >> > > > > >> > and
>> > > > > >> > > > > >> > > > > >> > > >> spend
>> > > > > >> > > > > >> > > > > >> > > >> > > > some
>> > > > > >> > > > > >> > > > > >> > > >> > > > > time on it now that I'm
>> > winding
>> > > > > down
>> > > > > >> > with
>> > > > > >> > > > > 0.98. I
>> > > > > >> > > > > >> > > think
>> > > > > >> > > > > >> > > > > >> that
>> > > > > >> > > > > >> > > > > >> > > means
>> > > > > >> > > > > >> > > > > >> > > >> > > > > branching for 2.0 real
>> soon
>> > now
>> > > > and
>> > > > > >> even
>> > > > > >> > > > > evicting
>> > > > > >> > > > > >> > > > things
>> > > > > >> > > > > >> > > > > >> from
>> > > > > >> > > > > >> > > > > >> > > 2.0
>> > > > > >> > > > > >> > > > > >> > > >> > > branch
>> > > > > >> > > > > >> > > > > >> > > >> > > > > that aren't finished or
>> > stable,
>> > > > > >> leaving
>> > > > > >> > > them
>> > > > > >> > > > > only
>> > > > > >> > > > > >> > > once
>> > > > > >> > > > > >> > > > > >> again
>> > > > > >> > > > > >> > > > > >> > in
>> > > > > >> > > > > >> > > > > >> > > the
>> > > > > >> > > > > >> > > > > >> > > >> > > > master
>> > > > > >> > > > > >> > > > > >> > > >> > > > > branch. Or, maybe just
>> > evicting
>> > > > > them.
>> > > > > >> > > Let's
>> > > > > >> > > > > take
>> > > > > >> > > > > >> it
>> > > > > >> > > > > >> > > > case
>> > > > > >> > > > > >> > > > > by
>> > > > > >> > > > > >> > > > > >> > > case.
>> > > > > >> > > > > >> > > > > >> > > >> > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > I think this feature can
>> come
>> > > in
>> > > > > >> > > relatively
>> > > > > >> > > > > >> safely.
>> > > > > >> > > > > >> > > As
>> > > > > >> > > > > >> > > > > >> added
>> > > > > >> > > > > >> > > > > >> > > >> > insurance,
>> > > > > >> > > > > >> > > > > >> > > >> > > > > let's admit the
>> possibility
>> > it
>> > > > > could
>> > > > > >> be
>> > > > > >> > > > > reverted
>> > > > > >> > > > > >> on
>> > > > > >> > > > > >> > > the
>> > > > > >> > > > > >> > > > > 2.0
>> > > > > >> > > > > >> > > > > >> > > branch
>> > > > > >> > > > > >> > > > > >> > > >> if
>> > > > > >> > > > > >> > > > > >> > > >> > > > folks
>> > > > > >> > > > > >> > > > > >> > > >> > > > > working on stabilizing 2.0
>> > > decide
>> > > > > to
>> > > > > >> > evict
>> > > > > >> > > > it
>> > > > > >> > > > > >> > because
>> > > > > >> > > > > >> > > > it
>> > > > > >> > > > > >> > > > > is
>> > > > > >> > > > > >> > > > > >> > > >> > unfinished
>> > > > > >> > > > > >> > > > > >> > > >> > > or
>> > > > > >> > > > > >> > > > > >> > > >> > > > > unstable, because that
>> > > certainly
>> > > > > can
>> > > > > >> > > > happen. I
>> > > > > >> > > > > >> > would
>> > > > > >> > > > > >> > > > > >> expect if
>> > > > > >> > > > > >> > > > > >> > > talk
>> > > > > >> > > > > >> > > > > >> > > >> > > like
>> > > > > >> > > > > >> > > > > >> > > >> > > > > that starts, we'd get help
>> > > > > finishing
>> > > > > >> or
>> > > > > >> > > > > >> stabilizing
>> > > > > >> > > > > >> > > > > what's
>> > > > > >> > > > > >> > > > > >> > under
>> > > > > >> > > > > >> > > > > >> > > >> > > > discussion
>> > > > > >> > > > > >> > > > > >> > > >> > > > > for revert. Or, we'd have
>> a
>> > > > revert.
>> > > > > >> > Either
>> > > > > >> > > > way
>> > > > > >> > > > > >> the
>> > > > > >> > > > > >> > > > > outcome
>> > > > > >> > > > > >> > > > > >> is
>> > > > > >> > > > > >> > > > > >> > > >> > > acceptable.
>> > > > > >> > > > > >> > > > > >> > > >> > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > On Wed, Sep 7, 2016 at
>> 8:56
>> > AM,
>> > > > > Dima
>> > > > > >> > > Spivak
>> > > > > >> > > > <
>> > > > > >> > > > > >> > > > > >> > > dimaspivak@apache.org
>> > > > > >> > > > > >> > > > > >> > > >> >
>> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
>> > > > > >> > > > > >> > > > > >> > > >> > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > I'm not sure that
>> "There is
>> > > > > already
>> > > > > >> > lots
>> > > > > >> > > > of
>> > > > > >> > > > > >> > > > half-baked
>> > > > > >> > > > > >> > > > > >> code
>> > > > > >> > > > > >> > > > > >> > in
>> > > > > >> > > > > >> > > > > >> > > >> the
>> > > > > >> > > > > >> > > > > >> > > >> > > > > branch,
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > so what's the harm in
>> > adding
>> > > > > more?"
>> > > > > >> > is a
>> > > > > >> > > > > good
>> > > > > >> > > > > >> > code
>> > > > > >> > > > > >> > > > > commit
>> > > > > >> > > > > >> > > > > >> > > >> > philosophy
>> > > > > >> > > > > >> > > > > >> > > >> > > > for
>> > > > > >> > > > > >> > > > > >> > > >> > > > > a
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > fault-tolerant
>> distributed
>> > > data
>> > > > > >> store.
>> > > > > >> > > ;)
>> > > > > >> > > > > >> > > > > >> > > >> > > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > More seriously, a lack
>> of
>> > > test
>> > > > > >> > coverage
>> > > > > >> > > > for
>> > > > > >> > > > > >> > > existing
>> > > > > >> > > > > >> > > > > >> > features
>> > > > > >> > > > > >> > > > > >> > > >> > > shouldn't
>> > > > > >> > > > > >> > > > > >> > > >> > > > > be
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > used as justification
>> for
>> > > > > >> introducing
>> > > > > >> > > new
>> > > > > >> > > > > >> > features
>> > > > > >> > > > > >> > > > with
>> > > > > >> > > > > >> > > > > >> the
>> > > > > >> > > > > >> > > > > >> > > same
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > shortcomings.
>> Ultimately,
>> > > it's
>> > > > > the
>> > > > > >> end
>> > > > > >> > > > user
>> > > > > >> > > > > who
>> > > > > >> > > > > >> > > will
>> > > > > >> > > > > >> > > > > feel
>> > > > > >> > > > > >> > > > > >> > the
>> > > > > >> > > > > >> > > > > >> > > >> pain,
>> > > > > >> > > > > >> > > > > >> > > >> > > so
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > shouldn't we do
>> everything
>> > we
>> > > > can
>> > > > > >> to
>> > > > > >> > > > > mitigate
>> > > > > >> > > > > >> > that?
>> > > > > >> > > > > >> > > > > >> > > >> > > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > -Dima
>> > > > > >> > > > > >> > > > > >> > > >> > > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > On Wed, Sep 7, 2016 at
>> 8:46
>> > > AM,
>> > > > > >> > Vladimir
>> > > > > >> > > > > >> > Rodionov <
>> > > > > >> > > > > >> > > > > >> > > >> > > > > vladrodionov@gmail.com>
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > wrote:
>> > > > > >> > > > > >> > > > > >> > > >> > > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Sean,
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have docs
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Agree. We have a doc
>> and
>> > > > backup
>> > > > > >> is
>> > > > > >> > the
>> > > > > >> > > > > most
>> > > > > >> > > > > >> > > > > documented
>> > > > > >> > > > > >> > > > > >> > > feature
>> > > > > >> > > > > >> > > > > >> > > >> > :),
>> > > > > >> > > > > >> > > > > >> > > >> > > we
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > will
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > release it shortly to
>> > > Apache.
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have sunny-day
>> > > correctness
>> > > > > >> tests
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Feature has  close to
>> 60
>> > > test
>> > > > > >> cases,
>> > > > > >> > > > which
>> > > > > >> > > > > >> run
>> > > > > >> > > > > >> > > for
>> > > > > >> > > > > >> > > > > >> approx
>> > > > > >> > > > > >> > > > > >> > 30
>> > > > > >> > > > > >> > > > > >> > > >> min.
>> > > > > >> > > > > >> > > > > >> > > >> > > We
>> > > > > >> > > > > >> > > > > >> > > >> > > > > can
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > add more, if
>> community do
>> > > not
>> > > > > >> mind
>> > > > > >> > :)
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have
>> > > > > >> > correctness-in-face-of-failure
>> > > > > >> > > > > tests
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Any examples of these
>> > tests
>> > > > in
>> > > > > >> > > existing
>> > > > > >> > > > > >> > features?
>> > > > > >> > > > > >> > > > In
>> > > > > >> > > > > >> > > > > >> > works,
>> > > > > >> > > > > >> > > > > >> > > we
>> > > > > >> > > > > >> > > > > >> > > >> > > have a
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > clear
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > understanding of what
>> > > should
>> > > > be
>> > > > > >> done
>> > > > > >> > > by
>> > > > > >> > > > > the
>> > > > > >> > > > > >> > time
>> > > > > >> > > > > >> > > of
>> > > > > >> > > > > >> > > > > 2.0
>> > > > > >> > > > > >> > > > > >> > > >> release.
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > That is very close
>> goal
>> > for
>> > > > us,
>> > > > > >> to
>> > > > > >> > > > verify
>> > > > > >> > > > > IT
>> > > > > >> > > > > >> > > monkey
>> > > > > >> > > > > >> > > > > for
>> > > > > >> > > > > >> > > > > >> > > >> existing
>> > > > > >> > > > > >> > > > > >> > > >> > > > code.
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * don't rely on things
>> > > > outside
>> > > > > of
>> > > > > >> > > HBase
>> > > > > >> > > > > for
>> > > > > >> > > > > >> > > normal
>> > > > > >> > > > > >> > > > > >> > operation
>> > > > > >> > > > > >> > > > > >> > > >> > (okay
>> > > > > >> > > > > >> > > > > >> > > >> > > > for
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > advanced operation)
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > We do not.
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Enormous time has been
>> > > spent
>> > > > > >> already
>> > > > > >> > > on
>> > > > > >> > > > > the
>> > > > > >> > > > > >> > > > > development
>> > > > > >> > > > > >> > > > > >> > and
>> > > > > >> > > > > >> > > > > >> > > >> > testing
>> > > > > >> > > > > >> > > > > >> > > >> > > > the
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > feature, it has passed
>> > our
>> > > > > >> internal
>> > > > > >> > > > tests
>> > > > > >> > > > > and
>> > > > > >> > > > > >> > > many
>> > > > > >> > > > > >> > > > > >> rounds
>> > > > > >> > > > > >> > > > > >> > of
>> > > > > >> > > > > >> > > > > >> > > >> code
>> > > > > >> > > > > >> > > > > >> > > >> > > > > reviews
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > by HBase committers.
>> We
>> > do
>> > > > not
>> > > > > >> mind
>> > > > > >> > if
>> > > > > >> > > > > >> someone
>> > > > > >> > > > > >> > > from
>> > > > > >> > > > > >> > > > > >> HBase
>> > > > > >> > > > > >> > > > > >> > > >> > community
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > (outside of HW) will
>> > review
>> > > > the
>> > > > > >> > code,
>> > > > > >> > > > but
>> > > > > >> > > > > it
>> > > > > >> > > > > >> > will
>> > > > > >> > > > > >> > > > > >> probably
>> > > > > >> > > > > >> > > > > >> > > >> takes
>> > > > > >> > > > > >> > > > > >> > > >> > > > > forever
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > to
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > wait for volunteer?,
>> the
>> > > > > feature
>> > > > > >> is
>> > > > > >> > > > quite
>> > > > > >> > > > > >> large
>> > > > > >> > > > > >> > > > (1MB+
>> > > > > >> > > > > >> > > > > >> > > >> cumulative
>> > > > > >> > > > > >> > > > > >> > > >> > > > patch)
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > 2.0 branch is full of
>> > half
>> > > > > baked
>> > > > > >> > > > features,
>> > > > > >> > > > > >> most
>> > > > > >> > > > > >> > > of
>> > > > > >> > > > > >> > > > > them
>> > > > > >> > > > > >> > > > > >> > are
>> > > > > >> > > > > >> > > > > >> > > in
>> > > > > >> > > > > >> > > > > >> > > >> > > active
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > development,
>> therefore I
>> > am
>> > > > not
>> > > > > >> > > > following
>> > > > > >> > > > > you
>> > > > > >> > > > > >> > > here,
>> > > > > >> > > > > >> > > > > >> Sean?
>> > > > > >> > > > > >> > > > > >> > > Why
>> > > > > >> > > > > >> > > > > >> > > >> > > > > HBASE-7912
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > is
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > not good enough yet
>> to be
>> > > > > >> integrated
>> > > > > >> > > > into
>> > > > > >> > > > > 2.0
>> > > > > >> > > > > >> > > > branch?
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > -Vlad
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > On Wed, Sep 7, 2016 at
>> > 8:23
>> > > > AM,
>> > > > > >> Sean
>> > > > > >> > > > > Busbey <
>> > > > > >> > > > > >> > > > > >> > > busbey@apache.org
>> > > > > >> > > > > >> > > > > >> > > >> >
>> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > On Tue, Sep 6, 2016
>> at
>> > > > 10:36
>> > > > > >> PM,
>> > > > > >> > > Josh
>> > > > > >> > > > > >> Elser <
>> > > > > >> > > > > >> > > > > >> > > >> > > josh.elser@gmail.com>
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > wrote:
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > So, the answer to
>> > > Sean's
>> > > > > >> > original
>> > > > > >> > > > > >> question
>> > > > > >> > > > > >> > is
>> > > > > >> > > > > >> > > > "as
>> > > > > >> > > > > >> > > > > >> > > robust as
>> > > > > >> > > > > >> > > > > >> > > >> > > > > snapshots
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > presently are"?
>> > > > > >> (independence of
>> > > > > >> > > > > >> > > backup/restore
>> > > > > >> > > > > >> > > > > >> > failure
>> > > > > >> > > > > >> > > > > >> > > >> > > tolerance
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > from
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > snapshot failure
>> > > > tolerance)
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > Is this just a
>> > question
>> > > > WRT
>> > > > > >> > > context
>> > > > > >> > > > of
>> > > > > >> > > > > >> the
>> > > > > >> > > > > >> > > > > change,
>> > > > > >> > > > > >> > > > > >> or
>> > > > > >> > > > > >> > > > > >> > > is it
>> > > > > >> > > > > >> > > > > >> > > >> > > means
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > for a
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > veto
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > from you, Sean?
>> Just
>> > > > trying
>> > > > > >> to
>> > > > > >> > > make
>> > > > > >> > > > > sure
>> > > > > >> > > > > >> > I'm
>> > > > > >> > > > > >> > > > > >> following
>> > > > > >> > > > > >> > > > > >> > > >> along
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > adequately.
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > I'd say ATM I'm -0,
>> > > > bordering
>> > > > > >> on
>> > > > > >> > -1
>> > > > > >> > > > but
>> > > > > >> > > > > not
>> > > > > >> > > > > >> > for
>> > > > > >> > > > > >> > > > > >> reasons
>> > > > > >> > > > > >> > > > > >> > I
>> > > > > >> > > > > >> > > > > >> > > can
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > articulate
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > well.
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Here's an attempt.
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > We've been trying to
>> > > move,
>> > > > > as a
>> > > > > >> > > > > community,
>> > > > > >> > > > > >> > > > towards
>> > > > > >> > > > > >> > > > > >> > > minimizing
>> > > > > >> > > > > >> > > > > >> > > >> > > risk
>> > > > > >> > > > > >> > > > > >> > > >> > > > to
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > downstream folks by
>> > > getting
>> > > > > >> > > "complete
>> > > > > >> > > > > >> enough
>> > > > > >> > > > > >> > > for
>> > > > > >> > > > > >> > > > > use"
>> > > > > >> > > > > >> > > > > >> > > gates
>> > > > > >> > > > > >> > > > > >> > > >> in
>> > > > > >> > > > > >> > > > > >> > > >> > > > place
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > before we introduce
>> new
>> > > > > >> features.
>> > > > > >> > > This
>> > > > > >> > > > > was
>> > > > > >> > > > > >> > > > spurred
>> > > > > >> > > > > >> > > > > >> by a
>> > > > > >> > > > > >> > > > > >> > > some
>> > > > > >> > > > > >> > > > > >> > > >> > > > features
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > getting in
>> half-baked
>> > and
>> > > > > never
>> > > > > >> > > making
>> > > > > >> > > > > it
>> > > > > >> > > > > >> to
>> > > > > >> > > > > >> > > "can
>> > > > > >> > > > > >> > > > > >> really
>> > > > > >> > > > > >> > > > > >> > > use"
>> > > > > >> > > > > >> > > > > >> > > >> > > > status
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > (I'm thinking of
>> > > > distributed
>> > > > > >> log
>> > > > > >> > > > replay
>> > > > > >> > > > > and
>> > > > > >> > > > > >> > the
>> > > > > >> > > > > >> > > > > >> zk-less
>> > > > > >> > > > > >> > > > > >> > > >> > > assignment
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > stuff, I don't
>> recall
>> > if
>> > > > > there
>> > > > > >> was
>> > > > > >> > > > > more).
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The gates,
>> generally,
>> > > > > included
>> > > > > >> > > things
>> > > > > >> > > > > like:
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have docs
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have sunny-day
>> > > > correctness
>> > > > > >> tests
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have
>> > > > > >> > > correctness-in-face-of-failure
>> > > > > >> > > > > tests
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't rely on
>> things
>> > > > > outside
>> > > > > >> of
>> > > > > >> > > > HBase
>> > > > > >> > > > > for
>> > > > > >> > > > > >> > > > normal
>> > > > > >> > > > > >> > > > > >> > > operation
>> > > > > >> > > > > >> > > > > >> > > >> > > (okay
>> > > > > >> > > > > >> > > > > >> > > >> > > > > for
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > advanced operation)
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > As an example, we
>> kept
>> > > the
>> > > > > MOB
>> > > > > >> > work
>> > > > > >> > > > off
>> > > > > >> > > > > in
>> > > > > >> > > > > >> a
>> > > > > >> > > > > >> > > > branch
>> > > > > >> > > > > >> > > > > >> and
>> > > > > >> > > > > >> > > > > >> > > out
>> > > > > >> > > > > >> > > > > >> > > >> of
>> > > > > >> > > > > >> > > > > >> > > >> > > > master
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > until it could pass
>> > these
>> > > > > >> > criteria.
>> > > > > >> > > > The
>> > > > > >> > > > > big
>> > > > > >> > > > > >> > > > > exemption
>> > > > > >> > > > > >> > > > > >> > > we've
>> > > > > >> > > > > >> > > > > >> > > >> had
>> > > > > >> > > > > >> > > > > >> > > >> > > to
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > this was the
>> > hbase-spark
>> > > > > >> > > integration,
>> > > > > >> > > > > where
>> > > > > >> > > > > >> > we
>> > > > > >> > > > > >> > > > all
>> > > > > >> > > > > >> > > > > >> > agreed
>> > > > > >> > > > > >> > > > > >> > > it
>> > > > > >> > > > > >> > > > > >> > > >> > > could
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > land in master
>> because
>> > it
>> > > > was
>> > > > > >> very
>> > > > > >> > > > well
>> > > > > >> > > > > >> > > isolated
>> > > > > >> > > > > >> > > > > (the
>> > > > > >> > > > > >> > > > > >> > > slide
>> > > > > >> > > > > >> > > > > >> > > >> > away
>> > > > > >> > > > > >> > > > > >> > > >> > > > from
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > including docs as a
>> > > > > first-class
>> > > > > >> > part
>> > > > > >> > > > of
>> > > > > >> > > > > >> > > building
>> > > > > >> > > > > >> > > > up
>> > > > > >> > > > > >> > > > > >> that
>> > > > > >> > > > > >> > > > > >> > > >> > > > integration
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > has led me to doubt
>> the
>> > > > > wisdom
>> > > > > >> of
>> > > > > >> > > this
>> > > > > >> > > > > >> > > decision).
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > We've also been
>> > treating
>> > > > > >> inclusion
>> > > > > >> > > in
>> > > > > >> > > > a
>> > > > > >> > > > > >> > > "probably
>> > > > > >> > > > > >> > > > > >> will
>> > > > > >> > > > > >> > > > > >> > be
>> > > > > >> > > > > >> > > > > >> > > >> > > released
>> > > > > >> > > > > >> > > > > >> > > >> > > > to
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > downstream" branches
>> > as a
>> > > > > >> higher
>> > > > > >> > > bar,
>> > > > > >> > > > > >> > requiring
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't moderately
>> > impact
>> > > > > >> > > performance
>> > > > > >> > > > > when
>> > > > > >> > > > > >> > the
>> > > > > >> > > > > >> > > > > >> feature
>> > > > > >> > > > > >> > > > > >> > > isn't
>> > > > > >> > > > > >> > > > > >> > > >> in
>> > > > > >> > > > > >> > > > > >> > > >> > > use
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't severely
>> impact
>> > > > > >> > performance
>> > > > > >> > > > when
>> > > > > >> > > > > >> the
>> > > > > >> > > > > >> > > > > feature
>> > > > > >> > > > > >> > > > > >> is
>> > > > > >> > > > > >> > > > > >> > in
>> > > > > >> > > > > >> > > > > >> > > >> use
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * either
>> default-to-on
>> > or
>> > > > > show
>> > > > > >> > > enough
>> > > > > >> > > > > >> demand
>> > > > > >> > > > > >> > to
>> > > > > >> > > > > >> > > > > >> believe
>> > > > > >> > > > > >> > > > > >> > a
>> > > > > >> > > > > >> > > > > >> > > >> > > > non-trivial
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > number of folks will
>> > turn
>> > > > the
>> > > > > >> > > feature
>> > > > > >> > > > on
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The above has kept
>> MOB
>> > > and
>> > > > > >> > > hbase-spark
>> > > > > >> > > > > >> > > > integration
>> > > > > >> > > > > >> > > > > >> out
>> > > > > >> > > > > >> > > > > >> > of
>> > > > > >> > > > > >> > > > > >> > > >> > > branch-1,
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > presumably while
>> > they've
>> > > > > >> "gotten
>> > > > > >> > > more
>> > > > > >> > > > > >> stable"
>> > > > > >> > > > > >> > > in
>> > > > > >> > > > > >> > > > > >> master
>> > > > > >> > > > > >> > > > > >> > > from
>> > > > > >> > > > > >> > > > > >> > > >> > the
>> > > > > >> > > > > >> > > > > >> > > >> > > > odd
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > vendor inclusion.
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Are we going to
>> have a
>> > > 2.0
>> > > > > >> release
>> > > > > >> > > > > before
>> > > > > >> > > > > >> the
>> > > > > >> > > > > >> > > end
>> > > > > >> > > > > >> > > > > of
>> > > > > >> > > > > >> > > > > >> the
>> > > > > >> > > > > >> > > > > >> > > >> year?
>> > > > > >> > > > > >> > > > > >> > > >> > > > We're
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > coming up on 1.5
>> years
>> > > > since
>> > > > > >> the
>> > > > > >> > > > > release of
>> > > > > >> > > > > >> > > > version
>> > > > > >> > > > > >> > > > > >> 1.0;
>> > > > > >> > > > > >> > > > > >> > > >> seems
>> > > > > >> > > > > >> > > > > >> > > >> > > like
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > it's about time,
>> > though I
>> > > > > >> haven't
>> > > > > >> > > seen
>> > > > > >> > > > > any
>> > > > > >> > > > > >> > > > concrete
>> > > > > >> > > > > >> > > > > >> > plans
>> > > > > >> > > > > >> > > > > >> > > >> this
>> > > > > >> > > > > >> > > > > >> > > >> > > > year.
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Presuming we are
>> going
>> > to
>> > > > > have
>> > > > > >> one
>> > > > > >> > > by
>> > > > > >> > > > > the
>> > > > > >> > > > > >> end
>> > > > > >> > > > > >> > > of
>> > > > > >> > > > > >> > > > > the
>> > > > > >> > > > > >> > > > > >> > > year, it
>> > > > > >> > > > > >> > > > > >> > > >> > > > seems a
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > bit close to still
>> be
>> > > > adding
>> > > > > in
>> > > > > >> > > > > "features
>> > > > > >> > > > > >> > that
>> > > > > >> > > > > >> > > > need
>> > > > > >> > > > > >> > > > > >> > > maturing"
>> > > > > >> > > > > >> > > > > >> > > >> > on
>> > > > > >> > > > > >> > > > > >> > > >> > > > the
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > branch.
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The lack of a
>> concrete
>> > > plan
>> > > > > for
>> > > > > >> > 2.0
>> > > > > >> > > > > keeps
>> > > > > >> > > > > >> me
>> > > > > >> > > > > >> > > from
>> > > > > >> > > > > >> > > > > >> > > considering
>> > > > > >> > > > > >> > > > > >> > > >> > > these
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > things blocker at
>> the
>> > > > moment.
>> > > > > >> But
>> > > > > >> > I
>> > > > > >> > > > know
>> > > > > >> > > > > >> > first
>> > > > > >> > > > > >> > > > hand
>> > > > > >> > > > > >> > > > > >> how
>> > > > > >> > > > > >> > > > > >> > > much
>> > > > > >> > > > > >> > > > > >> > > >> > > > trouble
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > folks have had with
>> > other
>> > > > > >> features
>> > > > > >> > > > that
>> > > > > >> > > > > >> have
>> > > > > >> > > > > >> > > gone
>> > > > > >> > > > > >> > > > > >> into
>> > > > > >> > > > > >> > > > > >> > > >> > downstream
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > facing releases
>> without
>> > > > > >> robustness
>> > > > > >> > > > > checks
>> > > > > >> > > > > >> > (i.e.
>> > > > > >> > > > > >> > > > > >> > > replication),
>> > > > > >> > > > > >> > > > > >> > > >> > and
>> > > > > >> > > > > >> > > > > >> > > >> > > > I'm
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > concerned about what
>> > > we're
>> > > > > >> setting
>> > > > > >> > > up
>> > > > > >> > > > if
>> > > > > >> > > > > >> 2.0
>> > > > > >> > > > > >> > > goes
>> > > > > >> > > > > >> > > > > out
>> > > > > >> > > > > >> > > > > >> > with
>> > > > > >> > > > > >> > > > > >> > > >> this
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > feature in its
>> current
>> > > > state.
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > --
>> > > > > >> > > > > >> > > > > >> > > >> > > > > Best regards,
>> > > > > >> > > > > >> > > > > >> > > >> > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > >    - Andy
>> > > > > >> > > > > >> > > > > >> > > >> > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > > > Problems worthy of attack
>> > prove
>> > > > > their
>> > > > > >> > > worth
>> > > > > >> > > > by
>> > > > > >> > > > > >> > > hitting
>> > > > > >> > > > > >> > > > > >> back. -
>> > > > > >> > > > > >> > > > > >> > > Piet
>> > > > > >> > > > > >> > > > > >> > > >> > > Hein
>> > > > > >> > > > > >> > > > > >> > > >> > > > > (via Tom White)
>> > > > > >> > > > > >> > > > > >> > > >> > > > >
>> > > > > >> > > > > >> > > > > >> > > >> > > >
>> > > > > >> > > > > >> > > > > >> > > >> > >
>> > > > > >> > > > > >> > > > > >> > > >> > >
>> > > > > >> > > > > >> > > > > >> > > >> > >
>> > > > > >> > > > > >> > > > > >> > > >> > > --
>> > > > > >> > > > > >> > > > > >> > > >> > > Best regards,
>> > > > > >> > > > > >> > > > > >> > > >> > >
>> > > > > >> > > > > >> > > > > >> > > >> > >    - Andy
>> > > > > >> > > > > >> > > > > >> > > >> > >
>> > > > > >> > > > > >> > > > > >> > > >> > > Problems worthy of attack
>> prove
>> > > their
>> > > > > >> worth
>> > > > > >> > by
>> > > > > >> > > > > >> hitting
>> > > > > >> > > > > >> > > > back.
>> > > > > >> > > > > >> > > > > -
>> > > > > >> > > > > >> > > > > >> > Piet
>> > > > > >> > > > > >> > > > > >> > > >> Hein
>> > > > > >> > > > > >> > > > > >> > > >> > > (via Tom White)
>> > > > > >> > > > > >> > > > > >> > > >> > >
>> > > > > >> > > > > >> > > > > >> > > >> >
>> > > > > >> > > > > >> > > > > >> > > >>
>> > > > > >> > > > > >> > > > > >> > >
>> > > > > >> > > > > >> > > > > >> >
>> > > > > >> > > > > >> > > > > >>
>> > > > > >> > > > > >> > > > > >
>> > > > > >> > > > > >> > > > > >
>> > > > > >> > > > > >> > > > >
>> > > > > >> > > > > >> > > >
>> > > > > >> > > > > >> > >
>> > > > > >> > > > > >> >
>> > > > > >> > > > > >>
>> > > > > >> > > > >
>> > > > > >> > > > >
>> > > > > >> > > > >
>> > > > > >> > > > > --
>> > > > > >> > > > > busbey
>> > > > > >> > > > >
>> > > > > >> > > >
>> > > > > >> > >
>> > > > > >> >
>> > > > > >>
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>>
>>
>> --
>>
>> -- Appy
>>
>
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Stack <st...@duboce.net>]

Interesting. When I try it fails w/ below:

[INFO] 26 warnings
322 [INFO] -------------------------------------------------------------
323 [INFO] -------------------------------------------------------------
324 [ERROR] COMPILATION ERROR :
325 [INFO] -------------------------------------------------------------
326 [ERROR]
/home/stack/hbase.git/hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.java:[48,8]
org.apache.hadoop.hbase.io.encoding.rowindexV2.RowIndexCodecV2 is not
abstract and does not override abstract method createSeeker(org.ap
 ache.hadoop.hbase.CellComparator,org.apache.hadoop.hbase.io.encoding.HFileBlockDecodingContext)
in org.apache.hadoop.hbase.io.encoding.DataBlockEncoder
327 [ERROR]
/home/stack/hbase.git/hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.java:[143,3]
method does not override or implement a method from a supertype
328 [ERROR]
/home/stack/hbase.git/hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.java:[147,29]
incompatible types: java.nio.ByteBuffer cannot be converted to
org.apache.hadoop.hbase.nio.ByteBuff
329 [ERROR]
/home/stack/hbase.git/hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.java:[148,33]
cannot find symbol
330   symbol:   method getKeyDeepCopy()
331   location: variable seeker of type
org.apache.hadoop.hbase.io.encoding.DataBlockEncoder.EncodedSeeker
332 [ERROR]
/home/stack/hbase.git/hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexCodecV2.java:[153,3]
method does not override or implement a method from a supertype
333 [ERROR]
/home/stack/hbase.git/hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/rowindexV1/RowIndexCodecV1.java:[45,8]
org.apache.hadoop.hbase.io.encoding.rowindexV1.RowIndexCodecV1 is not
abstract and does not override abstract method createSeeker(org.ap
 ache.hadoop.hbase.CellComparator,org.apache.hadoop.hbase.io.encoding.HFileBlockDecodingContext)
in org.apache.hadoop.hbase.io.encoding.DataBlockEncoder
334 [ERROR]
/home/stack/hbase.git/hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/rowindexV1/RowIndexCodecV1.java:[145,3]
method does not override or implement a method from a supertype
335 [ERROR]
/home/stack/hbase.git/hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/rowindexV1/RowIndexCodecV1.java:[158,3]
method does not override or implement a method from a supertype
336 [ERROR]
/home/stack/hbase.git/hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.java:[46,8]
org.apache.hadoop.hbase.io.encoding.rowindexV2.RowIndexSeekerV2 is not
abstract and does not override abstract method compareKey(org.ap
 ache.hadoop.hbase.CellComparator,org.apache.hadoop.hbase.Cell) in
org.apache.hadoop.hbase.io.encoding.DataBlockEncoder.EncodedSeeker
337 [ERROR]
/home/stack/hbase.git/hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.java:[79,3]
method does not override or implement a method from a supertype
338 [ERROR]
/home/stack/hbase.git/hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.java:[117,3]
method does not override or implement a method from a supertype
339 [ERROR]
/home/stack/hbase.git/hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.java:[190,3]
method does not override or implement a method from a supertype
340 [ERROR]
/home/stack/hbase.git/hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.java:[214,3]
method does not override or implement a method from a supertype
341 [ERROR]
/home/stack/hbase.git/hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.java:[349,3]
method does not override or implement a method from a supertype
342 [ERROR]
/home/stack/hbase.git/hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.java:[355,3]
method does not override or implement a method from a supertype
343 [ERROR]
/home/stack/hbase.git/hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/rowindexV2/RowIndexSeekerV2.java:[421,36]
no suitable method found for
uncompressTags(java.nio.ByteBuffer,byte[],int,int)
344     method
org.apache.hadoop.hbase.io.TagCompressionContext.uncompressTags(java.io.InputStream,byte[],int,int)
is not applicable
345       (argument mismatch; java.nio.ByteBuffer cannot be converted to
java.io.InputStream)
346     method
org.apache.hadoop.hbase.io.TagCompressionContext.uncompressTags(org.apache.hadoop.hbase.nio.ByteBuff,byte[],int,int)
is not applicable
347       (argument mismatch; java.nio.ByteBuffer cannot be converted to
org.apache.hadoop.hbase.nio.ByteBuff)

....

St.Ack

On Tue, Oct 11, 2016 at 5:12 PM, Apekshit Sharma <ap...@cloudera.com> wrote:

> @stack, it compiled for me.
>
> Also tried few commands, and have to say, it's well designed from user
> commands perspective.
>
> On Tue, Oct 11, 2016 at 5:08 PM, Stack <st...@duboce.net> wrote:
>
> > On Tue, Oct 11, 2016 at 1:50 PM, Vladimir Rodionov <
> vladrodionov@gmail.com
> > >
> > wrote:
> >
> > > Michael,
> > >
> > > Its in HBASE-7912
> > >
> > > This is tip of git log:
> > >
> > > commit a072f6f49a26a7259ff2aaef6cb56d85eb592482
> > > Author: Frank Welsch <fw...@jps.net>
> > > Date:   Fri Sep 23 18:00:42 2016 -0400
> > >
> > >     HBASE-16574 Book updates for backup and restore
> > >
> > > commit b14e2ab1c24e65ff88dd4c579acf83cb4ed0605e
> > > Author: tedyu <yu...@gmail.com>
> > > Date:   Wed Oct 5 16:29:40 2016 -0700
> > >
> > >     HBASE-16727 Backup refactoring: remove MR dependencies from HMaster
> > > (Vladimir Rodionov)
> > >
> > >
> > Thanks. I have that. I tried it and it doesn't compile for me. Does it
> > compile for you?
> > Thanks,
> > M
> >
> >
> >
> > > -Vlad
> > >
> > > On Tue, Oct 11, 2016 at 1:44 PM, Stack <st...@duboce.net> wrote:
> > >
> > > > Which branch do I check out to try it? HBASE-7912 is not it. I don't
> > see
> > > an
> > > > HBASE-16727...
> > > > Thanks,
> > > > M
> > > >
> > > >
> > > > On Tue, Oct 11, 2016 at 11:41 AM, Vladimir Rodionov <
> > > > vladrodionov@gmail.com>
> > > > wrote:
> > > >
> > > > > The last patch is on review board:
> > > > > https://reviews.apache.org/r/52748
> > > > >
> > > > > On Mon, Oct 10, 2016 at 3:52 PM, Vladimir Rodionov <
> > > > vladrodionov@gmail.com
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > >> How hard to put in an hbase-backup module? hbase-server is fat
> > > > enough
> > > > > > >> already. Could be done as a follow-up.
> > > > > >
> > > > > > https://issues.apache.org/jira/browse/HBASE-16727?
> > > > > > focusedCommentId=15531237&page=com.atlassian.jira.
> > > > > > plugin.system.issuetabpanels:comment-tabpanel#comment-15531237
> > > > > >
> > > > > > Can we do merge first? Then we can discuss separate module.
> > > > > >
> > > > > > -Vlad
> > > > > >
> > > > > > On Mon, Oct 10, 2016 at 3:44 PM, Ted Yu <yu...@gmail.com>
> > wrote:
> > > > > >
> > > > > >> Looks like the first quote was cut off.
> > > > > >> The original sentence was:
> > > > > >>
> > > > > >> bq. no mapreduce job launched from master or region server.
> > > > > >>
> > > > > >> mapreduce job is launched from the node where command line tool
> is
> > > > run.
> > > > > >>
> > > > > >> On Mon, Oct 10, 2016 at 3:38 PM, Stack <st...@duboce.net>
> wrote:
> > > > > >>
> > > > > >> > bq. launched from master or region server.
> > > > > >> >
> > > > > >> > What does this mean please? Has to be run from Master or
> > > > RegionServer?
> > > > > >> Can
> > > > > >> > it be run from another node altogether?
> > > > > >> >
> > > > > >> > On Mon, Oct 10, 2016 at 1:44 PM, Vladimir Rodionov <
> > > > > >> vladrodionov@gmail.com
> > > > > >> > >
> > > > > >> > wrote:
> > > > > >> >
> > > > > >> > > >> mapreduce dependency has been moved to client side - no
> > > > mapreduce
> > > > > >> job
> > > > > >> > >
> > > > > >> > > 1. We have no code in the client module anymore, due to
> > > dependency
> > > > > on
> > > > > >> > > internal server API (HFile and WAL access).
> > > > > >> > > 2. Backup/ restore are client - driven operations, but all
> the
> > > > code
> > > > > >> > resides
> > > > > >> > > in the server module
> > > > > >> > >
> > > > > >> >
> > > > > >> > How hard to put in an hbase-backup module? hbase-server is fat
> > > > enough
> > > > > >> > already. Could be done as a follow-up.
> > > > > >> >
> > > > > >> > Thanks,
> > > > > >> > St.Ack
> > > > > >> >
> > > > > >> >
> > > > > >> >
> > > > > >> > > 3. No MR in Master, no procedure - driven execution.
> > > > > >> > > 4. Old good MR from command-line.
> > > > > >> > > 5. Security was simplified and now only super-user is
> allowed
> > to
> > > > run
> > > > > >> > > backup/restores.
> > > > > >> > > 6. HBase Backup API was gone due to 1. Now only command-line
> > > > access
> > > > > to
> > > > > >> > > backup tools.
> > > > > >> > >
> > > > > >> > > These consequences of refactoring has been discussed in
> > > > HBASE-16727.
> > > > > >> > >
> > > > > >> > > -Vlad
> > > > > >> > >
> > > > > >> > >
> > > > > >> > >
> > > > > >> > >
> > > > > >> > > On Mon, Oct 10, 2016 at 1:31 PM, Ted Yu <
> yuzhihong@gmail.com>
> > > > > wrote:
> > > > > >> > >
> > > > > >> > > > Reviving this thread.
> > > > > >> > > >
> > > > > >> > > > The following has taken place:
> > > > > >> > > >
> > > > > >> > > > mapreduce dependency has been moved to client side - no
> > > > mapreduce
> > > > > >> job
> > > > > >> > > > launched from master or region server.
> > > > > >> > > > document patch (HBASE-16574) has been integrated.
> > > > > >> > > > Updated mega patch has been attached to HBASE-14123: this
> > > covers
> > > > > the
> > > > > >> > > > refactor in #1 above and the protobuf 3 merge.
> > > > > >> > > >
> > > > > >> > > > If community has more feedback on the merge proposal, I
> > would
> > > > love
> > > > > >> to
> > > > > >> > > hear
> > > > > >> > > > it.
> > > > > >> > > >
> > > > > >> > > > Thanks
> > > > > >> > > >
> > > > > >> > > > On Thu, Sep 22, 2016 at 10:31 AM, Sean Busbey <
> > > > > busbey@cloudera.com>
> > > > > >> > > wrote:
> > > > > >> > > >
> > > > > >> > > > > I'd like to see the docs proposed on HBASE-16574
> > integrated
> > > > into
> > > > > >> our
> > > > > >> > > > > project's documentation prior to merge.
> > > > > >> > > > >
> > > > > >> > > > > On Thu, Sep 22, 2016 at 9:02 AM, Ted Yu <
> > > yuzhihong@gmail.com>
> > > > > >> wrote:
> > > > > >> > > > > > This feature can be marked experimental due to some
> > > > > limitations
> > > > > >> > such
> > > > > >> > > as
> > > > > >> > > > > > security.
> > > > > >> > > > > >
> > > > > >> > > > > > Your previous round of comments have been addressed.
> > > > > >> > > > > > Command line tool has gone through:
> > > > > >> > > > > >
> > > > > >> > > > > > HBASE-16620 Fix backup command-line tool usability
> > issues
> > > > > >> > > > > > HBASE-16655 hbase backup describe with incorrect
> backup
> > id
> > > > > >> results
> > > > > >> > in
> > > > > >> > > > NPE
> > > > > >> > > > > >
> > > > > >> > > > > > The updated doc has been attached to HBASE-16574.
> > > > > >> > > > > >
> > > > > >> > > > > > Cheers
> > > > > >> > > > > >
> > > > > >> > > > > > On Thu, Sep 22, 2016 at 8:53 AM, Stack <
> > stack@duboce.net>
> > > > > >> wrote:
> > > > > >> > > > > >
> > > > > >> > > > > >> On Wed, Sep 21, 2016 at 7:43 AM, Ted Yu <
> > > > yuzhihong@gmail.com
> > > > > >
> > > > > >> > > wrote:
> > > > > >> > > > > >>
> > > > > >> > > > > >> > Are there more (review) comments ?
> > > > > >> > > > > >> >
> > > > > >> > > > > >> >
> > > > > >> > > > > >> Are outstanding comments addressed?
> > > > > >> > > > > >>
> > > > > >> > > > > >> I don't see answer to my 'is this experimental/will
> it
> > be
> > > > > >> marked
> > > > > >> > > > > >> experimental' question.
> > > > > >> > > > > >>
> > > > > >> > > > > >> I ran into some issues trying to use the feature and
> > > > > suggested
> > > > > >> > that
> > > > > >> > > a
> > > > > >> > > > > >> feature likes this needs polish else it'll just rot,
> > > > unused.
> > > > > >> Has
> > > > > >> > > > polish
> > > > > >> > > > > >> been applied? All ready for another 'user' test?
> > Suggest
> > > > that
> > > > > >> you
> > > > > >> > > > update
> > > > > >> > > > > >> here going forward for the benefit of those trying to
> > > > follow
> > > > > >> along
> > > > > >> > > and
> > > > > >> > > > > who
> > > > > >> > > > > >> are not watching JIRA change fly-by.
> > > > > >> > > > > >>
> > > > > >> > > > > >> It looks like doc got a revision -- I have to check
> --
> > to
> > > > > take
> > > > > >> on
> > > > > >> > > > > >> suggestion made above but again, suggest, that this
> > > thread
> > > > > gets
> > > > > >> > > > updated.
> > > > > >> > > > > >>
> > > > > >> > > > > >> Thanks,
> > > > > >> > > > > >> St.Ack
> > > > > >> > > > > >>
> > > > > >> > > > > >>
> > > > > >> > > > > >>
> > > > > >> > > > > >> > Thanks
> > > > > >> > > > > >> >
> > > > > >> > > > > >> > On Tue, Sep 20, 2016 at 10:02 AM, Devaraj Das <
> > > > > >> > > ddas@hortonworks.com
> > > > > >> > > > >
> > > > > >> > > > > >> > wrote:
> > > > > >> > > > > >> >
> > > > > >> > > > > >> > > Just reviving this thread. Thanks Sean, Stack,
> > Dima,
> > > > and
> > > > > >> > others
> > > > > >> > > > for
> > > > > >> > > > > the
> > > > > >> > > > > >> > > thorough reviews and testing. Thanks Ted and Vlad
> > for
> > > > > >> taking
> > > > > >> > > care
> > > > > >> > > > of
> > > > > >> > > > > >> the
> > > > > >> > > > > >> > > feedback. Are we all good to do the merge now?
> > Rather
> > > > do
> > > > > >> > sooner
> > > > > >> > > > than
> > > > > >> > > > > >> > later.
> > > > > >> > > > > >> > > ________________________________________
> > > > > >> > > > > >> > > From: saint.ack@gmail.com <sa...@gmail.com>
> on
> > > > > behalf
> > > > > >> of
> > > > > >> > > > Stack
> > > > > >> > > > > <
> > > > > >> > > > > >> > > stack@duboce.net>
> > > > > >> > > > > >> > > Sent: Monday, September 12, 2016 1:18 PM
> > > > > >> > > > > >> > > To: HBase Dev List
> > > > > >> > > > > >> > > Subject: Re: [DISCUSSION] Merge Backup / Restore
> -
> > > > Branch
> > > > > >> > > > HBASE-7912
> > > > > >> > > > > >> > >
> > > > > >> > > > > >> > > On Mon, Sep 12, 2016 at 12:19 PM, Ted Yu <
> > > > > >> yuzhihong@gmail.com
> > > > > >> > >
> > > > > >> > > > > wrote:
> > > > > >> > > > > >> > >
> > > > > >> > > > > >> > > > Mega patch (rev 18) is on HBASE-14123.
> > > > > >> > > > > >> > > >
> > > > > >> > > > > >> > > > Please comment on HBASE-14123 on how you want
> to
> > > > > review.
> > > > > >> > > > > >> > > >
> > > > > >> > > > > >> > >
> > > > > >> > > > > >> > >
> > > > > >> > > > > >> > > Yeah. That was my lost tab. Last rb was 6 months
> > ago.
> > > > > >> Suggest
> > > > > >> > > > > updating
> > > > > >> > > > > >> > it.
> > > > > >> > > > > >> > > RB is pretty good for review. Patch is only 1.5M
> so
> > > > > should
> > > > > >> be
> > > > > >> > > > fine.
> > > > > >> > > > > >> > >
> > > > > >> > > > > >> > > St.Ack
> > > > > >> > > > > >> > >
> > > > > >> > > > > >> > >
> > > > > >> > > > > >> > > >
> > > > > >> > > > > >> > > > Thanks
> > > > > >> > > > > >> > > >
> > > > > >> > > > > >> > > > On Mon, Sep 12, 2016 at 12:15 PM, Stack <
> > > > > >> stack@duboce.net>
> > > > > >> > > > wrote:
> > > > > >> > > > > >> > > >
> > > > > >> > > > > >> > > > > On review of the 'patch', do I just compare
> the
> > > > > branch
> > > > > >> to
> > > > > >> > > > > master or
> > > > > >> > > > > >> > is
> > > > > >> > > > > >> > > > > there a megapatch posted somewhere (I think I
> > saw
> > > > one
> > > > > >> but
> > > > > >> > it
> > > > > >> > > > > seemed
> > > > > >> > > > > >> > > stale
> > > > > >> > > > > >> > > > > and then I 'lost' the tab). Sorry for dumb
> > > > question.
> > > > > >> > > > > >> > > > > St.Ack
> > > > > >> > > > > >> > > > >
> > > > > >> > > > > >> > > > > On Mon, Sep 12, 2016 at 12:01 PM, Stack <
> > > > > >> stack@duboce.net
> > > > > >> > >
> > > > > >> > > > > wrote:
> > > > > >> > > > > >> > > > >
> > > > > >> > > > > >> > > > > > Late to the game. A few comments after
> > > rereading
> > > > > this
> > > > > >> > > thread
> > > > > >> > > > > as a
> > > > > >> > > > > >> > > > 'user'.
> > > > > >> > > > > >> > > > > >
> > > > > >> > > > > >> > > > > > + Before merge, a user-facing feature like
> > this
> > > > > >> should
> > > > > >> > > work
> > > > > >> > > > > (If
> > > > > >> > > > > >> > this
> > > > > >> > > > > >> > > is
> > > > > >> > > > > >> > > > > "higher-bar
> > > > > >> > > > > >> > > > > > for new features", bring it on -- smile).
> > > > > >> > > > > >> > > > > > + As a user, I tried the branch with tools
> > > after
> > > > > >> > reviewing
> > > > > >> > > > the
> > > > > >> > > > > >> > > > > just-posted
> > > > > >> > > > > >> > > > > > doc. I had an 'interesting' experience
> (left
> > > > > >> comments up
> > > > > >> > > on
> > > > > >> > > > > >> > issue). I
> > > > > >> > > > > >> > > > > think
> > > > > >> > > > > >> > > > > > the tooling/doc. important to get right. If
> > it
> > > > > breaks
> > > > > >> > > easily
> > > > > >> > > > > or
> > > > > >> > > > > >> is
> > > > > >> > > > > >> > > > > > inconsistent (or lacks 'polish'), operators
> > > will
> > > > > >> judge
> > > > > >> > the
> > > > > >> > > > > whole
> > > > > >> > > > > >> > > > > > backup/restore tooling chain as not
> > trustworthy
> > > > and
> > > > > >> > > abandon
> > > > > >> > > > > it.
> > > > > >> > > > > >> > Lets
> > > > > >> > > > > >> > > > not
> > > > > >> > > > > >> > > > > > have this happen to this feature.
> > > > > >> > > > > >> > > > > > + Matteo's suggestion (with a helpful
> starter
> > > > list)
> > > > > >> that
> > > > > >> > > > there
> > > > > >> > > > > >> > needs
> > > > > >> > > > > >> > > to
> > > > > >> > > > > >> > > > > be
> > > > > >> > > > > >> > > > > > explicit qualification on what is actually
> > > being
> > > > > >> > delivered
> > > > > >> > > > --
> > > > > >> > > > > >> > > > including a
> > > > > >> > > > > >> > > > > > listing of limitations (some look serious
> > such
> > > as
> > > > > >> data
> > > > > >> > > bleed
> > > > > >> > > > > from
> > > > > >> > > > > >> > > other
> > > > > >> > > > > >> > > > > > regions in WALs, but maybe I don't care for
> > my
> > > > use
> > > > > >> > > case...)
> > > > > >> > > > --
> > > > > >> > > > > >> > needs
> > > > > >> > > > > >> > > to
> > > > > >> > > > > >> > > > > > accompany the merge. Lets fold them into
> the
> > > user
> > > > > >> doc.
> > > > > >> > in
> > > > > >> > > > the
> > > > > >> > > > > >> > > technical
> > > > > >> > > > > >> > > > > > overview area as suggested so user
> > expectations
> > > > are
> > > > > >> > > properly
> > > > > >> > > > > >> > managed
> > > > > >> > > > > >> > > > > > (otherwise, they expect the world and will
> > just
> > > > > give
> > > > > >> up
> > > > > >> > > when
> > > > > >> > > > > we
> > > > > >> > > > > >> > fall
> > > > > >> > > > > >> > > > > > short). Vladimir did a list of what is in
> > each
> > > of
> > > > > the
> > > > > >> > > phases
> > > > > >> > > > > >> above
> > > > > >> > > > > >> > > > which
> > > > > >> > > > > >> > > > > > would serve as a good start.
> > > > > >> > > > > >> > > > > > + Is this feature 'experimental' (Matteo
> asks
> > > > > above).
> > > > > >> > I'd
> > > > > >> > > > > prefer
> > > > > >> > > > > >> it
> > > > > >> > > > > >> > > is
> > > > > >> > > > > >> > > > > > not. If it is, it should be labelled all
> over
> > > > that
> > > > > >> it is
> > > > > >> > > > so. I
> > > > > >> > > > > >> see
> > > > > >> > > > > >> > > > > current
> > > > > >> > > > > >> > > > > > state called out as a '... technical
> preview
> > > > > >> feature'.
> > > > > >> > > Does
> > > > > >> > > > > this
> > > > > >> > > > > >> > mean
> > > > > >> > > > > >> > > > > > not-for-users?
> > > > > >> > > > > >> > > > > >
> > > > > >> > > > > >> > > > > > St.Ack
> > > > > >> > > > > >> > > > > >
> > > > > >> > > > > >> > > > > >
> > > > > >> > > > > >> > > > > >
> > > > > >> > > > > >> > > > > >
> > > > > >> > > > > >> > > > > >
> > > > > >> > > > > >> > > > > >
> > > > > >> > > > > >> > > > > >
> > > > > >> > > > > >> > > > > >
> > > > > >> > > > > >> > > > > >
> > > > > >> > > > > >> > > > > >
> > > > > >> > > > > >> > > > > >
> > > > > >> > > > > >> > > > > > On Mon, Sep 12, 2016 at 8:03 AM, Ted Yu <
> > > > > >> > > > yuzhihong@gmail.com>
> > > > > >> > > > > >> > wrote:
> > > > > >> > > > > >> > > > > >
> > > > > >> > > > > >> > > > > >> Sean:
> > > > > >> > > > > >> > > > > >> Do you have more comments ?
> > > > > >> > > > > >> > > > > >>
> > > > > >> > > > > >> > > > > >> Cheers
> > > > > >> > > > > >> > > > > >>
> > > > > >> > > > > >> > > > > >> On Fri, Sep 9, 2016 at 1:42 PM, Vladimir
> > > > Rodionov
> > > > > <
> > > > > >> > > > > >> > > > > vladrodionov@gmail.com
> > > > > >> > > > > >> > > > > >> >
> > > > > >> > > > > >> > > > > >> wrote:
> > > > > >> > > > > >> > > > > >>
> > > > > >> > > > > >> > > > > >> > Sean,
> > > > > >> > > > > >> > > > > >> >
> > > > > >> > > > > >> > > > > >> > Backup/Restore can fail due to various
> > > > reasons:
> > > > > >> > network
> > > > > >> > > > > outage
> > > > > >> > > > > >> > > > > (cluster
> > > > > >> > > > > >> > > > > >> > wide), various time-outs in HBase and
> HDFS
> > > > > layer,
> > > > > >> M/R
> > > > > >> > > > > failure
> > > > > >> > > > > >> > due
> > > > > >> > > > > >> > > to
> > > > > >> > > > > >> > > > > >> "HDFS
> > > > > >> > > > > >> > > > > >> > exceeded quota", user error (manual
> > deletion
> > > > of
> > > > > >> data)
> > > > > >> > > and
> > > > > >> > > > > so
> > > > > >> > > > > >> on
> > > > > >> > > > > >> > so
> > > > > >> > > > > >> > > > on.
> > > > > >> > > > > >> > > > > >> That
> > > > > >> > > > > >> > > > > >> > is impossible to enumerate all possible
> > > types
> > > > of
> > > > > >> > > failures
> > > > > >> > > > > in a
> > > > > >> > > > > >> > > > > >> distributed
> > > > > >> > > > > >> > > > > >> > system - that is not our goal/task.
> > > > > >> > > > > >> > > > > >> >
> > > > > >> > > > > >> > > > > >> > We focus completely on backup system
> table
> > > > > >> > consistency
> > > > > >> > > > in a
> > > > > >> > > > > >> > > presence
> > > > > >> > > > > >> > > > > of
> > > > > >> > > > > >> > > > > >> any
> > > > > >> > > > > >> > > > > >> > type of failure. That is what I call
> > > > "tolerance
> > > > > to
> > > > > >> > > > > failures".
> > > > > >> > > > > >> > > > > >> >
> > > > > >> > > > > >> > > > > >> > On a failure:
> > > > > >> > > > > >> > > > > >> >
> > > > > >> > > > > >> > > > > >> > BACKUP. All backup system information
> > (prior
> > > > to
> > > > > >> > backup)
> > > > > >> > > > > will
> > > > > >> > > > > >> be
> > > > > >> > > > > >> > > > > restored
> > > > > >> > > > > >> > > > > >> > and all temporary data, related to a
> > failed
> > > > > >> session,
> > > > > >> > in
> > > > > >> > > > > HDFS
> > > > > >> > > > > >> > will
> > > > > >> > > > > >> > > be
> > > > > >> > > > > >> > > > > >> > deleted
> > > > > >> > > > > >> > > > > >> > RESTORE. We do not care about system
> data,
> > > > > because
> > > > > >> > > > restore
> > > > > >> > > > > >> does
> > > > > >> > > > > >> > > not
> > > > > >> > > > > >> > > > > >> change
> > > > > >> > > > > >> > > > > >> > it. Temporary data in HDFS will be
> cleaned
> > > up
> > > > > and
> > > > > >> > table
> > > > > >> > > > > will
> > > > > >> > > > > >> be
> > > > > >> > > > > >> > > in a
> > > > > >> > > > > >> > > > > >> state
> > > > > >> > > > > >> > > > > >> > back to where it was before operation
> > > started.
> > > > > >> > > > > >> > > > > >> >
> > > > > >> > > > > >> > > > > >> > This is what user should expect in case
> > of a
> > > > > >> failure.
> > > > > >> > > > > >> > > > > >> >
> > > > > >> > > > > >> > > > > >> > -Vlad
> > > > > >> > > > > >> > > > > >> >
> > > > > >> > > > > >> > > > > >> >
> > > > > >> > > > > >> > > > > >> > -Vlad
> > > > > >> > > > > >> > > > > >> >
> > > > > >> > > > > >> > > > > >> > On Fri, Sep 9, 2016 at 12:56 PM, Sean
> > > Busbey <
> > > > > >> > > > > >> busbey@apache.org
> > > > > >> > > > > >> > >
> > > > > >> > > > > >> > > > > wrote:
> > > > > >> > > > > >> > > > > >> >
> > > > > >> > > > > >> > > > > >> > > Failing in a consistent way, with docs
> > > that
> > > > > >> explain
> > > > > >> > > the
> > > > > >> > > > > >> > various
> > > > > >> > > > > >> > > > > >> > > expected failures would be sufficient.
> > > > > >> > > > > >> > > > > >> > >
> > > > > >> > > > > >> > > > > >> > > On Fri, Sep 9, 2016 at 12:16 PM,
> > Vladimir
> > > > > >> Rodionov
> > > > > >> > > > > >> > > > > >> > > <vl...@gmail.com> wrote:
> > > > > >> > > > > >> > > > > >> > > > Do not worry Sean, doc is coming
> today
> > > as
> > > > a
> > > > > >> > preview
> > > > > >> > > > and
> > > > > >> > > > > >> our
> > > > > >> > > > > >> > > > writer
> > > > > >> > > > > >> > > > > >> > Frank
> > > > > >> > > > > >> > > > > >> > > > will be working on a putting  it
> into
> > > > Apache
> > > > > >> > repo.
> > > > > >> > > > > >> Timeline
> > > > > >> > > > > >> > > > > depends
> > > > > >> > > > > >> > > > > >> on
> > > > > >> > > > > >> > > > > >> > > > Franks schedule but I hope we will
> get
> > > it
> > > > > >> rather
> > > > > >> > > > sooner
> > > > > >> > > > > >> than
> > > > > >> > > > > >> > > > > later.
> > > > > >> > > > > >> > > > > >> > > >
> > > > > >> > > > > >> > > > > >> > > > As for failure testing, we are
> > focusing
> > > > only
> > > > > >> on a
> > > > > >> > > > > >> consistent
> > > > > >> > > > > >> > > > state
> > > > > >> > > > > >> > > > > >> of
> > > > > >> > > > > >> > > > > >> > > > backup system data in a presence of
> > any
> > > > type
> > > > > >> of
> > > > > >> > > > > failures,
> > > > > >> > > > > >> We
> > > > > >> > > > > >> > > are
> > > > > >> > > > > >> > > > > not
> > > > > >> > > > > >> > > > > >> > > going
> > > > > >> > > > > >> > > > > >> > > > to implement  anything more "fancy",
> > > than
> > > > > >> that.
> > > > > >> > We
> > > > > >> > > > > allow
> > > > > >> > > > > >> > both:
> > > > > >> > > > > >> > > > > >> backup
> > > > > >> > > > > >> > > > > >> > and
> > > > > >> > > > > >> > > > > >> > > > restore to fail. What we do not
> allow
> > is
> > > > to
> > > > > >> have
> > > > > >> > > > system
> > > > > >> > > > > >> data
> > > > > >> > > > > >> > > > > >> corrupted.
> > > > > >> > > > > >> > > > > >> > > > Will it suffice for you? Do you have
> > any
> > > > > other
> > > > > >> > > > > concerns,
> > > > > >> > > > > >> you
> > > > > >> > > > > >> > > > want
> > > > > >> > > > > >> > > > > >> us to
> > > > > >> > > > > >> > > > > >> > > > address?
> > > > > >> > > > > >> > > > > >> > > >
> > > > > >> > > > > >> > > > > >> > > > -Vlad
> > > > > >> > > > > >> > > > > >> > > >
> > > > > >> > > > > >> > > > > >> > > >
> > > > > >> > > > > >> > > > > >> > > > On Fri, Sep 9, 2016 at 10:56 AM,
> Sean
> > > > > Busbey <
> > > > > >> > > > > >> > > busbey@apache.org
> > > > > >> > > > > >> > > > >
> > > > > >> > > > > >> > > > > >> > wrote:
> > > > > >> > > > > >> > > > > >> > > >
> > > > > >> > > > > >> > > > > >> > > >> "docs will come to Apache soon"
> does
> > > not
> > > > > >> address
> > > > > >> > > my
> > > > > >> > > > > >> concern
> > > > > >> > > > > >> > > > > around
> > > > > >> > > > > >> > > > > >> > docs
> > > > > >> > > > > >> > > > > >> > > at
> > > > > >> > > > > >> > > > > >> > > >> all, unless said docs have already
> > made
> > > > it
> > > > > >> into
> > > > > >> > > the
> > > > > >> > > > > >> project
> > > > > >> > > > > >> > > > > repo. I
> > > > > >> > > > > >> > > > > >> > > don't
> > > > > >> > > > > >> > > > > >> > > >> want third party resources for
> using
> > a
> > > > > major
> > > > > >> and
> > > > > >> > > > > >> important
> > > > > >> > > > > >> > > > > feature
> > > > > >> > > > > >> > > > > >> of
> > > > > >> > > > > >> > > > > >> > > the
> > > > > >> > > > > >> > > > > >> > > >> project, I want us to provide end
> > users
> > > > > with
> > > > > >> > what
> > > > > >> > > > they
> > > > > >> > > > > >> need
> > > > > >> > > > > >> > > to
> > > > > >> > > > > >> > > > > get
> > > > > >> > > > > >> > > > > >> the
> > > > > >> > > > > >> > > > > >> > > job
> > > > > >> > > > > >> > > > > >> > > >> done.
> > > > > >> > > > > >> > > > > >> > > >>
> > > > > >> > > > > >> > > > > >> > > >> I see some calls for patience on
> the
> > > > > failure
> > > > > >> > > > testing,
> > > > > >> > > > > but
> > > > > >> > > > > >> > the
> > > > > >> > > > > >> > > > > >> appeal
> > > > > >> > > > > >> > > > > >> > to
> > > > > >> > > > > >> > > > > >> > > us
> > > > > >> > > > > >> > > > > >> > > >> having done a bad job of requiring
> > > proper
> > > > > >> tests
> > > > > >> > of
> > > > > >> > > > > >> previous
> > > > > >> > > > > >> > > > > >> features
> > > > > >> > > > > >> > > > > >> > > just
> > > > > >> > > > > >> > > > > >> > > >> makes me more concerned about not
> > > getting
> > > > > >> them
> > > > > >> > > > here. I
> > > > > >> > > > > >> > don't
> > > > > >> > > > > >> > > > want
> > > > > >> > > > > >> > > > > >> to
> > > > > >> > > > > >> > > > > >> > set
> > > > > >> > > > > >> > > > > >> > > >> yet another bad example that will
> > then
> > > be
> > > > > >> > pointed
> > > > > >> > > to
> > > > > >> > > > > in
> > > > > >> > > > > >> the
> > > > > >> > > > > >> > > > > future.
> > > > > >> > > > > >> > > > > >> > > >>
> > > > > >> > > > > >> > > > > >> > > >> On Sep 8, 2016 10:50, "Ted Yu" <
> > > > > >> > > yuzhihong@gmail.com
> > > > > >> > > > >
> > > > > >> > > > > >> > wrote:
> > > > > >> > > > > >> > > > > >> > > >>
> > > > > >> > > > > >> > > > > >> > > >> > Is there any concern which is not
> > > > > >> addressed ?
> > > > > >> > > > > >> > > > > >> > > >> >
> > > > > >> > > > > >> > > > > >> > > >> > Do we need another Vote thread ?
> > > > > >> > > > > >> > > > > >> > > >> >
> > > > > >> > > > > >> > > > > >> > > >> > Thanks
> > > > > >> > > > > >> > > > > >> > > >> >
> > > > > >> > > > > >> > > > > >> > > >> > On Thu, Sep 8, 2016 at 9:21 AM,
> > > Andrew
> > > > > >> > Purtell <
> > > > > >> > > > > >> > > > > >> apurtell@apache.org
> > > > > >> > > > > >> > > > > >> > >
> > > > > >> > > > > >> > > > > >> > > >> > wrote:
> > > > > >> > > > > >> > > > > >> > > >> >
> > > > > >> > > > > >> > > > > >> > > >> > > Vlad,
> > > > > >> > > > > >> > > > > >> > > >> > >
> > > > > >> > > > > >> > > > > >> > > >> > > I apologize for using the term
> > > > > >> 'half-baked'
> > > > > >> > > in a
> > > > > >> > > > > way
> > > > > >> > > > > >> > that
> > > > > >> > > > > >> > > > > could
> > > > > >> > > > > >> > > > > >> > > seem a
> > > > > >> > > > > >> > > > > >> > > >> > > description of HBASE-7912. I
> > meant
> > > > that
> > > > > >> as a
> > > > > >> > > > > general
> > > > > >> > > > > >> > > > > >> hypothetical.
> > > > > >> > > > > >> > > > > >> > > >> > >
> > > > > >> > > > > >> > > > > >> > > >> > > On Wed, Sep 7, 2016 at 9:36 AM,
> > > > > Vladimir
> > > > > >> > > > Rodionov
> > > > > >> > > > > <
> > > > > >> > > > > >> > > > > >> > > >> > vladrodionov@gmail.com>
> > > > > >> > > > > >> > > > > >> > > >> > > wrote:
> > > > > >> > > > > >> > > > > >> > > >> > >
> > > > > >> > > > > >> > > > > >> > > >> > > > >> I'm not sure that "There
> is
> > > > > already
> > > > > >> > lots
> > > > > >> > > of
> > > > > >> > > > > >> > > half-baked
> > > > > >> > > > > >> > > > > >> code
> > > > > >> > > > > >> > > > > >> > in
> > > > > >> > > > > >> > > > > >> > > the
> > > > > >> > > > > >> > > > > >> > > >> > > > branch,
> > > > > >> > > > > >> > > > > >> > > >> > > > so what's the harm in adding
> > > more?"
> > > > > >> > > > > >> > > > > >> > > >> > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > I meant - not production -
> > ready
> > > > yet.
> > > > > >> This
> > > > > >> > > is
> > > > > >> > > > > 2.0
> > > > > >> > > > > >> > > > > development
> > > > > >> > > > > >> > > > > >> > > branch
> > > > > >> > > > > >> > > > > >> > > >> > and,
> > > > > >> > > > > >> > > > > >> > > >> > > > hence many features are in
> > works,
> > > > > >> > > > > >> > > > > >> > > >> > > > not being tested well etc. I
> do
> > > not
> > > > > >> > consider
> > > > > >> > > > > backup
> > > > > >> > > > > >> > as
> > > > > >> > > > > >> > > > half
> > > > > >> > > > > >> > > > > >> > baked
> > > > > >> > > > > >> > > > > >> > > >> > > feature -
> > > > > >> > > > > >> > > > > >> > > >> > > > it has passed our internal QA
> > and
> > > > has
> > > > > >> very
> > > > > >> > > > good
> > > > > >> > > > > >> doc,
> > > > > >> > > > > >> > > > which
> > > > > >> > > > > >> > > > > we
> > > > > >> > > > > >> > > > > >> > will
> > > > > >> > > > > >> > > > > >> > > >> > > provide
> > > > > >> > > > > >> > > > > >> > > >> > > > to Apache shortly.
> > > > > >> > > > > >> > > > > >> > > >> > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > -Vlad
> > > > > >> > > > > >> > > > > >> > > >> > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > On Wed, Sep 7, 2016 at 9:13
> AM,
> > > > > Andrew
> > > > > >> > > > Purtell <
> > > > > >> > > > > >> > > > > >> > > apurtell@apache.org>
> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
> > > > > >> > > > > >> > > > > >> > > >> > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > We shouldn't admit half
> baked
> > > > > changes
> > > > > >> > that
> > > > > >> > > > > won't
> > > > > >> > > > > >> be
> > > > > >> > > > > >> > > > > >> finished.
> > > > > >> > > > > >> > > > > >> > > >> However
> > > > > >> > > > > >> > > > > >> > > >> > > in
> > > > > >> > > > > >> > > > > >> > > >> > > > > this case the crew working
> on
> > > > this
> > > > > >> > feature
> > > > > >> > > > are
> > > > > >> > > > > >> long
> > > > > >> > > > > >> > > > > timers
> > > > > >> > > > > >> > > > > >> and
> > > > > >> > > > > >> > > > > >> > > less
> > > > > >> > > > > >> > > > > >> > > >> > > > likely
> > > > > >> > > > > >> > > > > >> > > >> > > > > than just about anyone to
> > leave
> > > > > >> > something
> > > > > >> > > > in a
> > > > > >> > > > > >> half
> > > > > >> > > > > >> > > > baked
> > > > > >> > > > > >> > > > > >> > > state. Of
> > > > > >> > > > > >> > > > > >> > > >> > > > course
> > > > > >> > > > > >> > > > > >> > > >> > > > > there is no guarantee how
> > > > anything
> > > > > >> will
> > > > > >> > > turn
> > > > > >> > > > > out,
> > > > > >> > > > > >> > > but I
> > > > > >> > > > > >> > > > > am
> > > > > >> > > > > >> > > > > >> > > willing
> > > > > >> > > > > >> > > > > >> > > >> to
> > > > > >> > > > > >> > > > > >> > > >> > > > take
> > > > > >> > > > > >> > > > > >> > > >> > > > > a little on faith if they
> > feel
> > > > > their
> > > > > >> > best
> > > > > >> > > > path
> > > > > >> > > > > >> > > forward
> > > > > >> > > > > >> > > > > now
> > > > > >> > > > > >> > > > > >> is
> > > > > >> > > > > >> > > > > >> > to
> > > > > >> > > > > >> > > > > >> > > >> > merge
> > > > > >> > > > > >> > > > > >> > > >> > > to
> > > > > >> > > > > >> > > > > >> > > >> > > > > trunk. I only wish I had
> > > > bandwidth
> > > > > to
> > > > > >> > have
> > > > > >> > > > > done
> > > > > >> > > > > >> > some
> > > > > >> > > > > >> > > > real
> > > > > >> > > > > >> > > > > >> > > kicking
> > > > > >> > > > > >> > > > > >> > > >> of
> > > > > >> > > > > >> > > > > >> > > >> > > the
> > > > > >> > > > > >> > > > > >> > > >> > > > > tires by now. Maybe this
> > week.
> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > (Yes, I'm using some of
> that
> > > time
> > > > > for
> > > > > >> > this
> > > > > >> > > > > email
> > > > > >> > > > > >> > :-)
> > > > > >> > > > > >> > > > but
> > > > > >> > > > > >> > > > > I
> > > > > >> > > > > >> > > > > >> > type
> > > > > >> > > > > >> > > > > >> > > >> > fast.)
> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > That said, I would like to
> > > > agitate
> > > > > >> for
> > > > > >> > > > making
> > > > > >> > > > > 2.0
> > > > > >> > > > > >> > > more
> > > > > >> > > > > >> > > > > real
> > > > > >> > > > > >> > > > > >> > and
> > > > > >> > > > > >> > > > > >> > > >> spend
> > > > > >> > > > > >> > > > > >> > > >> > > > some
> > > > > >> > > > > >> > > > > >> > > >> > > > > time on it now that I'm
> > winding
> > > > > down
> > > > > >> > with
> > > > > >> > > > > 0.98. I
> > > > > >> > > > > >> > > think
> > > > > >> > > > > >> > > > > >> that
> > > > > >> > > > > >> > > > > >> > > means
> > > > > >> > > > > >> > > > > >> > > >> > > > > branching for 2.0 real soon
> > now
> > > > and
> > > > > >> even
> > > > > >> > > > > evicting
> > > > > >> > > > > >> > > > things
> > > > > >> > > > > >> > > > > >> from
> > > > > >> > > > > >> > > > > >> > > 2.0
> > > > > >> > > > > >> > > > > >> > > >> > > branch
> > > > > >> > > > > >> > > > > >> > > >> > > > > that aren't finished or
> > stable,
> > > > > >> leaving
> > > > > >> > > them
> > > > > >> > > > > only
> > > > > >> > > > > >> > > once
> > > > > >> > > > > >> > > > > >> again
> > > > > >> > > > > >> > > > > >> > in
> > > > > >> > > > > >> > > > > >> > > the
> > > > > >> > > > > >> > > > > >> > > >> > > > master
> > > > > >> > > > > >> > > > > >> > > >> > > > > branch. Or, maybe just
> > evicting
> > > > > them.
> > > > > >> > > Let's
> > > > > >> > > > > take
> > > > > >> > > > > >> it
> > > > > >> > > > > >> > > > case
> > > > > >> > > > > >> > > > > by
> > > > > >> > > > > >> > > > > >> > > case.
> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > I think this feature can
> come
> > > in
> > > > > >> > > relatively
> > > > > >> > > > > >> safely.
> > > > > >> > > > > >> > > As
> > > > > >> > > > > >> > > > > >> added
> > > > > >> > > > > >> > > > > >> > > >> > insurance,
> > > > > >> > > > > >> > > > > >> > > >> > > > > let's admit the possibility
> > it
> > > > > could
> > > > > >> be
> > > > > >> > > > > reverted
> > > > > >> > > > > >> on
> > > > > >> > > > > >> > > the
> > > > > >> > > > > >> > > > > 2.0
> > > > > >> > > > > >> > > > > >> > > branch
> > > > > >> > > > > >> > > > > >> > > >> if
> > > > > >> > > > > >> > > > > >> > > >> > > > folks
> > > > > >> > > > > >> > > > > >> > > >> > > > > working on stabilizing 2.0
> > > decide
> > > > > to
> > > > > >> > evict
> > > > > >> > > > it
> > > > > >> > > > > >> > because
> > > > > >> > > > > >> > > > it
> > > > > >> > > > > >> > > > > is
> > > > > >> > > > > >> > > > > >> > > >> > unfinished
> > > > > >> > > > > >> > > > > >> > > >> > > or
> > > > > >> > > > > >> > > > > >> > > >> > > > > unstable, because that
> > > certainly
> > > > > can
> > > > > >> > > > happen. I
> > > > > >> > > > > >> > would
> > > > > >> > > > > >> > > > > >> expect if
> > > > > >> > > > > >> > > > > >> > > talk
> > > > > >> > > > > >> > > > > >> > > >> > > like
> > > > > >> > > > > >> > > > > >> > > >> > > > > that starts, we'd get help
> > > > > finishing
> > > > > >> or
> > > > > >> > > > > >> stabilizing
> > > > > >> > > > > >> > > > > what's
> > > > > >> > > > > >> > > > > >> > under
> > > > > >> > > > > >> > > > > >> > > >> > > > discussion
> > > > > >> > > > > >> > > > > >> > > >> > > > > for revert. Or, we'd have a
> > > > revert.
> > > > > >> > Either
> > > > > >> > > > way
> > > > > >> > > > > >> the
> > > > > >> > > > > >> > > > > outcome
> > > > > >> > > > > >> > > > > >> is
> > > > > >> > > > > >> > > > > >> > > >> > > acceptable.
> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > On Wed, Sep 7, 2016 at 8:56
> > AM,
> > > > > Dima
> > > > > >> > > Spivak
> > > > > >> > > > <
> > > > > >> > > > > >> > > > > >> > > dimaspivak@apache.org
> > > > > >> > > > > >> > > > > >> > > >> >
> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > > I'm not sure that "There
> is
> > > > > already
> > > > > >> > lots
> > > > > >> > > > of
> > > > > >> > > > > >> > > > half-baked
> > > > > >> > > > > >> > > > > >> code
> > > > > >> > > > > >> > > > > >> > in
> > > > > >> > > > > >> > > > > >> > > >> the
> > > > > >> > > > > >> > > > > >> > > >> > > > > branch,
> > > > > >> > > > > >> > > > > >> > > >> > > > > > so what's the harm in
> > adding
> > > > > more?"
> > > > > >> > is a
> > > > > >> > > > > good
> > > > > >> > > > > >> > code
> > > > > >> > > > > >> > > > > commit
> > > > > >> > > > > >> > > > > >> > > >> > philosophy
> > > > > >> > > > > >> > > > > >> > > >> > > > for
> > > > > >> > > > > >> > > > > >> > > >> > > > > a
> > > > > >> > > > > >> > > > > >> > > >> > > > > > fault-tolerant
> distributed
> > > data
> > > > > >> store.
> > > > > >> > > ;)
> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > > More seriously, a lack of
> > > test
> > > > > >> > coverage
> > > > > >> > > > for
> > > > > >> > > > > >> > > existing
> > > > > >> > > > > >> > > > > >> > features
> > > > > >> > > > > >> > > > > >> > > >> > > shouldn't
> > > > > >> > > > > >> > > > > >> > > >> > > > > be
> > > > > >> > > > > >> > > > > >> > > >> > > > > > used as justification for
> > > > > >> introducing
> > > > > >> > > new
> > > > > >> > > > > >> > features
> > > > > >> > > > > >> > > > with
> > > > > >> > > > > >> > > > > >> the
> > > > > >> > > > > >> > > > > >> > > same
> > > > > >> > > > > >> > > > > >> > > >> > > > > > shortcomings. Ultimately,
> > > it's
> > > > > the
> > > > > >> end
> > > > > >> > > > user
> > > > > >> > > > > who
> > > > > >> > > > > >> > > will
> > > > > >> > > > > >> > > > > feel
> > > > > >> > > > > >> > > > > >> > the
> > > > > >> > > > > >> > > > > >> > > >> pain,
> > > > > >> > > > > >> > > > > >> > > >> > > so
> > > > > >> > > > > >> > > > > >> > > >> > > > > > shouldn't we do
> everything
> > we
> > > > can
> > > > > >> to
> > > > > >> > > > > mitigate
> > > > > >> > > > > >> > that?
> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > > -Dima
> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > > On Wed, Sep 7, 2016 at
> 8:46
> > > AM,
> > > > > >> > Vladimir
> > > > > >> > > > > >> > Rodionov <
> > > > > >> > > > > >> > > > > >> > > >> > > > > vladrodionov@gmail.com>
> > > > > >> > > > > >> > > > > >> > > >> > > > > > wrote:
> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Sean,
> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have docs
> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Agree. We have a doc
> and
> > > > backup
> > > > > >> is
> > > > > >> > the
> > > > > >> > > > > most
> > > > > >> > > > > >> > > > > documented
> > > > > >> > > > > >> > > > > >> > > feature
> > > > > >> > > > > >> > > > > >> > > >> > :),
> > > > > >> > > > > >> > > > > >> > > >> > > we
> > > > > >> > > > > >> > > > > >> > > >> > > > > > will
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > release it shortly to
> > > Apache.
> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have sunny-day
> > > correctness
> > > > > >> tests
> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Feature has  close to
> 60
> > > test
> > > > > >> cases,
> > > > > >> > > > which
> > > > > >> > > > > >> run
> > > > > >> > > > > >> > > for
> > > > > >> > > > > >> > > > > >> approx
> > > > > >> > > > > >> > > > > >> > 30
> > > > > >> > > > > >> > > > > >> > > >> min.
> > > > > >> > > > > >> > > > > >> > > >> > > We
> > > > > >> > > > > >> > > > > >> > > >> > > > > can
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > add more, if community
> do
> > > not
> > > > > >> mind
> > > > > >> > :)
> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * have
> > > > > >> > correctness-in-face-of-failure
> > > > > >> > > > > tests
> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Any examples of these
> > tests
> > > > in
> > > > > >> > > existing
> > > > > >> > > > > >> > features?
> > > > > >> > > > > >> > > > In
> > > > > >> > > > > >> > > > > >> > works,
> > > > > >> > > > > >> > > > > >> > > we
> > > > > >> > > > > >> > > > > >> > > >> > > have a
> > > > > >> > > > > >> > > > > >> > > >> > > > > > clear
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > understanding of what
> > > should
> > > > be
> > > > > >> done
> > > > > >> > > by
> > > > > >> > > > > the
> > > > > >> > > > > >> > time
> > > > > >> > > > > >> > > of
> > > > > >> > > > > >> > > > > 2.0
> > > > > >> > > > > >> > > > > >> > > >> release.
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > That is very close goal
> > for
> > > > us,
> > > > > >> to
> > > > > >> > > > verify
> > > > > >> > > > > IT
> > > > > >> > > > > >> > > monkey
> > > > > >> > > > > >> > > > > for
> > > > > >> > > > > >> > > > > >> > > >> existing
> > > > > >> > > > > >> > > > > >> > > >> > > > code.
> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > * don't rely on things
> > > > outside
> > > > > of
> > > > > >> > > HBase
> > > > > >> > > > > for
> > > > > >> > > > > >> > > normal
> > > > > >> > > > > >> > > > > >> > operation
> > > > > >> > > > > >> > > > > >> > > >> > (okay
> > > > > >> > > > > >> > > > > >> > > >> > > > for
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > advanced operation)
> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > We do not.
> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > Enormous time has been
> > > spent
> > > > > >> already
> > > > > >> > > on
> > > > > >> > > > > the
> > > > > >> > > > > >> > > > > development
> > > > > >> > > > > >> > > > > >> > and
> > > > > >> > > > > >> > > > > >> > > >> > testing
> > > > > >> > > > > >> > > > > >> > > >> > > > the
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > feature, it has passed
> > our
> > > > > >> internal
> > > > > >> > > > tests
> > > > > >> > > > > and
> > > > > >> > > > > >> > > many
> > > > > >> > > > > >> > > > > >> rounds
> > > > > >> > > > > >> > > > > >> > of
> > > > > >> > > > > >> > > > > >> > > >> code
> > > > > >> > > > > >> > > > > >> > > >> > > > > reviews
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > by HBase committers. We
> > do
> > > > not
> > > > > >> mind
> > > > > >> > if
> > > > > >> > > > > >> someone
> > > > > >> > > > > >> > > from
> > > > > >> > > > > >> > > > > >> HBase
> > > > > >> > > > > >> > > > > >> > > >> > community
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > (outside of HW) will
> > review
> > > > the
> > > > > >> > code,
> > > > > >> > > > but
> > > > > >> > > > > it
> > > > > >> > > > > >> > will
> > > > > >> > > > > >> > > > > >> probably
> > > > > >> > > > > >> > > > > >> > > >> takes
> > > > > >> > > > > >> > > > > >> > > >> > > > > forever
> > > > > >> > > > > >> > > > > >> > > >> > > > > > to
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > wait for volunteer?,
> the
> > > > > feature
> > > > > >> is
> > > > > >> > > > quite
> > > > > >> > > > > >> large
> > > > > >> > > > > >> > > > (1MB+
> > > > > >> > > > > >> > > > > >> > > >> cumulative
> > > > > >> > > > > >> > > > > >> > > >> > > > patch)
> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > 2.0 branch is full of
> > half
> > > > > baked
> > > > > >> > > > features,
> > > > > >> > > > > >> most
> > > > > >> > > > > >> > > of
> > > > > >> > > > > >> > > > > them
> > > > > >> > > > > >> > > > > >> > are
> > > > > >> > > > > >> > > > > >> > > in
> > > > > >> > > > > >> > > > > >> > > >> > > active
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > development, therefore
> I
> > am
> > > > not
> > > > > >> > > > following
> > > > > >> > > > > you
> > > > > >> > > > > >> > > here,
> > > > > >> > > > > >> > > > > >> Sean?
> > > > > >> > > > > >> > > > > >> > > Why
> > > > > >> > > > > >> > > > > >> > > >> > > > > HBASE-7912
> > > > > >> > > > > >> > > > > >> > > >> > > > > > is
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > not good enough yet to
> be
> > > > > >> integrated
> > > > > >> > > > into
> > > > > >> > > > > 2.0
> > > > > >> > > > > >> > > > branch?
> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > -Vlad
> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > On Wed, Sep 7, 2016 at
> > 8:23
> > > > AM,
> > > > > >> Sean
> > > > > >> > > > > Busbey <
> > > > > >> > > > > >> > > > > >> > > busbey@apache.org
> > > > > >> > > > > >> > > > > >> > > >> >
> > > > > >> > > > > >> > > > > >> > > >> > > > wrote:
> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > On Tue, Sep 6, 2016
> at
> > > > 10:36
> > > > > >> PM,
> > > > > >> > > Josh
> > > > > >> > > > > >> Elser <
> > > > > >> > > > > >> > > > > >> > > >> > > josh.elser@gmail.com>
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > wrote:
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > So, the answer to
> > > Sean's
> > > > > >> > original
> > > > > >> > > > > >> question
> > > > > >> > > > > >> > is
> > > > > >> > > > > >> > > > "as
> > > > > >> > > > > >> > > > > >> > > robust as
> > > > > >> > > > > >> > > > > >> > > >> > > > > snapshots
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > presently are"?
> > > > > >> (independence of
> > > > > >> > > > > >> > > backup/restore
> > > > > >> > > > > >> > > > > >> > failure
> > > > > >> > > > > >> > > > > >> > > >> > > tolerance
> > > > > >> > > > > >> > > > > >> > > >> > > > > > from
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > snapshot failure
> > > > tolerance)
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > Is this just a
> > question
> > > > WRT
> > > > > >> > > context
> > > > > >> > > > of
> > > > > >> > > > > >> the
> > > > > >> > > > > >> > > > > change,
> > > > > >> > > > > >> > > > > >> or
> > > > > >> > > > > >> > > > > >> > > is it
> > > > > >> > > > > >> > > > > >> > > >> > > means
> > > > > >> > > > > >> > > > > >> > > >> > > > > > for a
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > veto
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > > from you, Sean?
> Just
> > > > trying
> > > > > >> to
> > > > > >> > > make
> > > > > >> > > > > sure
> > > > > >> > > > > >> > I'm
> > > > > >> > > > > >> > > > > >> following
> > > > > >> > > > > >> > > > > >> > > >> along
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > adequately.
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > I'd say ATM I'm -0,
> > > > bordering
> > > > > >> on
> > > > > >> > -1
> > > > > >> > > > but
> > > > > >> > > > > not
> > > > > >> > > > > >> > for
> > > > > >> > > > > >> > > > > >> reasons
> > > > > >> > > > > >> > > > > >> > I
> > > > > >> > > > > >> > > > > >> > > can
> > > > > >> > > > > >> > > > > >> > > >> > > > > > articulate
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > well.
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Here's an attempt.
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > We've been trying to
> > > move,
> > > > > as a
> > > > > >> > > > > community,
> > > > > >> > > > > >> > > > towards
> > > > > >> > > > > >> > > > > >> > > minimizing
> > > > > >> > > > > >> > > > > >> > > >> > > risk
> > > > > >> > > > > >> > > > > >> > > >> > > > to
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > downstream folks by
> > > getting
> > > > > >> > > "complete
> > > > > >> > > > > >> enough
> > > > > >> > > > > >> > > for
> > > > > >> > > > > >> > > > > use"
> > > > > >> > > > > >> > > > > >> > > gates
> > > > > >> > > > > >> > > > > >> > > >> in
> > > > > >> > > > > >> > > > > >> > > >> > > > place
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > before we introduce
> new
> > > > > >> features.
> > > > > >> > > This
> > > > > >> > > > > was
> > > > > >> > > > > >> > > > spurred
> > > > > >> > > > > >> > > > > >> by a
> > > > > >> > > > > >> > > > > >> > > some
> > > > > >> > > > > >> > > > > >> > > >> > > > features
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > getting in half-baked
> > and
> > > > > never
> > > > > >> > > making
> > > > > >> > > > > it
> > > > > >> > > > > >> to
> > > > > >> > > > > >> > > "can
> > > > > >> > > > > >> > > > > >> really
> > > > > >> > > > > >> > > > > >> > > use"
> > > > > >> > > > > >> > > > > >> > > >> > > > status
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > (I'm thinking of
> > > > distributed
> > > > > >> log
> > > > > >> > > > replay
> > > > > >> > > > > and
> > > > > >> > > > > >> > the
> > > > > >> > > > > >> > > > > >> zk-less
> > > > > >> > > > > >> > > > > >> > > >> > > assignment
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > stuff, I don't recall
> > if
> > > > > there
> > > > > >> was
> > > > > >> > > > > more).
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The gates, generally,
> > > > > included
> > > > > >> > > things
> > > > > >> > > > > like:
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have docs
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have sunny-day
> > > > correctness
> > > > > >> tests
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have
> > > > > >> > > correctness-in-face-of-failure
> > > > > >> > > > > tests
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't rely on
> things
> > > > > outside
> > > > > >> of
> > > > > >> > > > HBase
> > > > > >> > > > > for
> > > > > >> > > > > >> > > > normal
> > > > > >> > > > > >> > > > > >> > > operation
> > > > > >> > > > > >> > > > > >> > > >> > > (okay
> > > > > >> > > > > >> > > > > >> > > >> > > > > for
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > advanced operation)
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > As an example, we
> kept
> > > the
> > > > > MOB
> > > > > >> > work
> > > > > >> > > > off
> > > > > >> > > > > in
> > > > > >> > > > > >> a
> > > > > >> > > > > >> > > > branch
> > > > > >> > > > > >> > > > > >> and
> > > > > >> > > > > >> > > > > >> > > out
> > > > > >> > > > > >> > > > > >> > > >> of
> > > > > >> > > > > >> > > > > >> > > >> > > > master
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > until it could pass
> > these
> > > > > >> > criteria.
> > > > > >> > > > The
> > > > > >> > > > > big
> > > > > >> > > > > >> > > > > exemption
> > > > > >> > > > > >> > > > > >> > > we've
> > > > > >> > > > > >> > > > > >> > > >> had
> > > > > >> > > > > >> > > > > >> > > >> > > to
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > this was the
> > hbase-spark
> > > > > >> > > integration,
> > > > > >> > > > > where
> > > > > >> > > > > >> > we
> > > > > >> > > > > >> > > > all
> > > > > >> > > > > >> > > > > >> > agreed
> > > > > >> > > > > >> > > > > >> > > it
> > > > > >> > > > > >> > > > > >> > > >> > > could
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > land in master
> because
> > it
> > > > was
> > > > > >> very
> > > > > >> > > > well
> > > > > >> > > > > >> > > isolated
> > > > > >> > > > > >> > > > > (the
> > > > > >> > > > > >> > > > > >> > > slide
> > > > > >> > > > > >> > > > > >> > > >> > away
> > > > > >> > > > > >> > > > > >> > > >> > > > from
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > including docs as a
> > > > > first-class
> > > > > >> > part
> > > > > >> > > > of
> > > > > >> > > > > >> > > building
> > > > > >> > > > > >> > > > up
> > > > > >> > > > > >> > > > > >> that
> > > > > >> > > > > >> > > > > >> > > >> > > > integration
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > has led me to doubt
> the
> > > > > wisdom
> > > > > >> of
> > > > > >> > > this
> > > > > >> > > > > >> > > decision).
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > We've also been
> > treating
> > > > > >> inclusion
> > > > > >> > > in
> > > > > >> > > > a
> > > > > >> > > > > >> > > "probably
> > > > > >> > > > > >> > > > > >> will
> > > > > >> > > > > >> > > > > >> > be
> > > > > >> > > > > >> > > > > >> > > >> > > released
> > > > > >> > > > > >> > > > > >> > > >> > > > to
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > downstream" branches
> > as a
> > > > > >> higher
> > > > > >> > > bar,
> > > > > >> > > > > >> > requiring
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't moderately
> > impact
> > > > > >> > > performance
> > > > > >> > > > > when
> > > > > >> > > > > >> > the
> > > > > >> > > > > >> > > > > >> feature
> > > > > >> > > > > >> > > > > >> > > isn't
> > > > > >> > > > > >> > > > > >> > > >> in
> > > > > >> > > > > >> > > > > >> > > >> > > use
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't severely
> impact
> > > > > >> > performance
> > > > > >> > > > when
> > > > > >> > > > > >> the
> > > > > >> > > > > >> > > > > feature
> > > > > >> > > > > >> > > > > >> is
> > > > > >> > > > > >> > > > > >> > in
> > > > > >> > > > > >> > > > > >> > > >> use
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > * either
> default-to-on
> > or
> > > > > show
> > > > > >> > > enough
> > > > > >> > > > > >> demand
> > > > > >> > > > > >> > to
> > > > > >> > > > > >> > > > > >> believe
> > > > > >> > > > > >> > > > > >> > a
> > > > > >> > > > > >> > > > > >> > > >> > > > non-trivial
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > number of folks will
> > turn
> > > > the
> > > > > >> > > feature
> > > > > >> > > > on
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The above has kept
> MOB
> > > and
> > > > > >> > > hbase-spark
> > > > > >> > > > > >> > > > integration
> > > > > >> > > > > >> > > > > >> out
> > > > > >> > > > > >> > > > > >> > of
> > > > > >> > > > > >> > > > > >> > > >> > > branch-1,
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > presumably while
> > they've
> > > > > >> "gotten
> > > > > >> > > more
> > > > > >> > > > > >> stable"
> > > > > >> > > > > >> > > in
> > > > > >> > > > > >> > > > > >> master
> > > > > >> > > > > >> > > > > >> > > from
> > > > > >> > > > > >> > > > > >> > > >> > the
> > > > > >> > > > > >> > > > > >> > > >> > > > odd
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > vendor inclusion.
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Are we going to have
> a
> > > 2.0
> > > > > >> release
> > > > > >> > > > > before
> > > > > >> > > > > >> the
> > > > > >> > > > > >> > > end
> > > > > >> > > > > >> > > > > of
> > > > > >> > > > > >> > > > > >> the
> > > > > >> > > > > >> > > > > >> > > >> year?
> > > > > >> > > > > >> > > > > >> > > >> > > > We're
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > coming up on 1.5
> years
> > > > since
> > > > > >> the
> > > > > >> > > > > release of
> > > > > >> > > > > >> > > > version
> > > > > >> > > > > >> > > > > >> 1.0;
> > > > > >> > > > > >> > > > > >> > > >> seems
> > > > > >> > > > > >> > > > > >> > > >> > > like
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > it's about time,
> > though I
> > > > > >> haven't
> > > > > >> > > seen
> > > > > >> > > > > any
> > > > > >> > > > > >> > > > concrete
> > > > > >> > > > > >> > > > > >> > plans
> > > > > >> > > > > >> > > > > >> > > >> this
> > > > > >> > > > > >> > > > > >> > > >> > > > year.
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > Presuming we are
> going
> > to
> > > > > have
> > > > > >> one
> > > > > >> > > by
> > > > > >> > > > > the
> > > > > >> > > > > >> end
> > > > > >> > > > > >> > > of
> > > > > >> > > > > >> > > > > the
> > > > > >> > > > > >> > > > > >> > > year, it
> > > > > >> > > > > >> > > > > >> > > >> > > > seems a
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > bit close to still be
> > > > adding
> > > > > in
> > > > > >> > > > > "features
> > > > > >> > > > > >> > that
> > > > > >> > > > > >> > > > need
> > > > > >> > > > > >> > > > > >> > > maturing"
> > > > > >> > > > > >> > > > > >> > > >> > on
> > > > > >> > > > > >> > > > > >> > > >> > > > the
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > branch.
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > The lack of a
> concrete
> > > plan
> > > > > for
> > > > > >> > 2.0
> > > > > >> > > > > keeps
> > > > > >> > > > > >> me
> > > > > >> > > > > >> > > from
> > > > > >> > > > > >> > > > > >> > > considering
> > > > > >> > > > > >> > > > > >> > > >> > > these
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > things blocker at the
> > > > moment.
> > > > > >> But
> > > > > >> > I
> > > > > >> > > > know
> > > > > >> > > > > >> > first
> > > > > >> > > > > >> > > > hand
> > > > > >> > > > > >> > > > > >> how
> > > > > >> > > > > >> > > > > >> > > much
> > > > > >> > > > > >> > > > > >> > > >> > > > trouble
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > folks have had with
> > other
> > > > > >> features
> > > > > >> > > > that
> > > > > >> > > > > >> have
> > > > > >> > > > > >> > > gone
> > > > > >> > > > > >> > > > > >> into
> > > > > >> > > > > >> > > > > >> > > >> > downstream
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > facing releases
> without
> > > > > >> robustness
> > > > > >> > > > > checks
> > > > > >> > > > > >> > (i.e.
> > > > > >> > > > > >> > > > > >> > > replication),
> > > > > >> > > > > >> > > > > >> > > >> > and
> > > > > >> > > > > >> > > > > >> > > >> > > > I'm
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > concerned about what
> > > we're
> > > > > >> setting
> > > > > >> > > up
> > > > > >> > > > if
> > > > > >> > > > > >> 2.0
> > > > > >> > > > > >> > > goes
> > > > > >> > > > > >> > > > > out
> > > > > >> > > > > >> > > > > >> > with
> > > > > >> > > > > >> > > > > >> > > >> this
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > > feature in its
> current
> > > > state.
> > > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > --
> > > > > >> > > > > >> > > > > >> > > >> > > > > Best regards,
> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > >    - Andy
> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > > > Problems worthy of attack
> > prove
> > > > > their
> > > > > >> > > worth
> > > > > >> > > > by
> > > > > >> > > > > >> > > hitting
> > > > > >> > > > > >> > > > > >> back. -
> > > > > >> > > > > >> > > > > >> > > Piet
> > > > > >> > > > > >> > > > > >> > > >> > > Hein
> > > > > >> > > > > >> > > > > >> > > >> > > > > (via Tom White)
> > > > > >> > > > > >> > > > > >> > > >> > > > >
> > > > > >> > > > > >> > > > > >> > > >> > > >
> > > > > >> > > > > >> > > > > >> > > >> > >
> > > > > >> > > > > >> > > > > >> > > >> > >
> > > > > >> > > > > >> > > > > >> > > >> > >
> > > > > >> > > > > >> > > > > >> > > >> > > --
> > > > > >> > > > > >> > > > > >> > > >> > > Best regards,
> > > > > >> > > > > >> > > > > >> > > >> > >
> > > > > >> > > > > >> > > > > >> > > >> > >    - Andy
> > > > > >> > > > > >> > > > > >> > > >> > >
> > > > > >> > > > > >> > > > > >> > > >> > > Problems worthy of attack prove
> > > their
> > > > > >> worth
> > > > > >> > by
> > > > > >> > > > > >> hitting
> > > > > >> > > > > >> > > > back.
> > > > > >> > > > > >> > > > > -
> > > > > >> > > > > >> > > > > >> > Piet
> > > > > >> > > > > >> > > > > >> > > >> Hein
> > > > > >> > > > > >> > > > > >> > > >> > > (via Tom White)
> > > > > >> > > > > >> > > > > >> > > >> > >
> > > > > >> > > > > >> > > > > >> > > >> >
> > > > > >> > > > > >> > > > > >> > > >>
> > > > > >> > > > > >> > > > > >> > >
> > > > > >> > > > > >> > > > > >> >
> > > > > >> > > > > >> > > > > >>
> > > > > >> > > > > >> > > > > >
> > > > > >> > > > > >> > > > > >
> > > > > >> > > > > >> > > > >
> > > > > >> > > > > >> > > >
> > > > > >> > > > > >> > >
> > > > > >> > > > > >> >
> > > > > >> > > > > >>
> > > > > >> > > > >
> > > > > >> > > > >
> > > > > >> > > > >
> > > > > >> > > > > --
> > > > > >> > > > > busbey
> > > > > >> > > > >
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
>
>
> --
>
> -- Appy
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Vladimir Rodionov <vl...@gmail.com>]

Or you can apply 14123-master.v29.full.txt (HBASE-14123) to current master.

-Vlad

On Tue, Oct 11, 2016 at 1:50 PM, Vladimir Rodionov <vl...@gmail.com>
wrote:

> Michael,
>
> Its in HBASE-7912
>
> This is tip of git log:
>
> commit a072f6f49a26a7259ff2aaef6cb56d85eb592482
> Author: Frank Welsch <fw...@jps.net>
> Date:   Fri Sep 23 18:00:42 2016 -0400
>
>     HBASE-16574 Book updates for backup and restore
>
> commit b14e2ab1c24e65ff88dd4c579acf83cb4ed0605e
> Author: tedyu <yu...@gmail.com>
> Date:   Wed Oct 5 16:29:40 2016 -0700
>
>     HBASE-16727 Backup refactoring: remove MR dependencies from HMaster
> (Vladimir Rodionov)
>
> -Vlad
>
> On Tue, Oct 11, 2016 at 1:44 PM, Stack <st...@duboce.net> wrote:
>
>> Which branch do I check out to try it? HBASE-7912 is not it. I don't see
>> an
>> HBASE-16727...
>> Thanks,
>> M
>>
>>
>> On Tue, Oct 11, 2016 at 11:41 AM, Vladimir Rodionov <
>> vladrodionov@gmail.com>
>> wrote:
>>
>> > The last patch is on review board:
>> > https://reviews.apache.org/r/52748
>> >
>> > On Mon, Oct 10, 2016 at 3:52 PM, Vladimir Rodionov <
>> vladrodionov@gmail.com
>> > >
>> > wrote:
>> >
>> > > >> How hard to put in an hbase-backup module? hbase-server is fat
>> enough
>> > > >> already. Could be done as a follow-up.
>> > >
>> > > https://issues.apache.org/jira/browse/HBASE-16727?
>> > > focusedCommentId=15531237&page=com.atlassian.jira.
>> > > plugin.system.issuetabpanels:comment-tabpanel#comment-15531237
>> > >
>> > > Can we do merge first? Then we can discuss separate module.
>> > >
>> > > -Vlad
>> > >
>> > > On Mon, Oct 10, 2016 at 3:44 PM, Ted Yu <yu...@gmail.com> wrote:
>> > >
>> > >> Looks like the first quote was cut off.
>> > >> The original sentence was:
>> > >>
>> > >> bq. no mapreduce job launched from master or region server.
>> > >>
>> > >> mapreduce job is launched from the node where command line tool is
>> run.
>> > >>
>> > >> On Mon, Oct 10, 2016 at 3:38 PM, Stack <st...@duboce.net> wrote:
>> > >>
>> > >> > bq. launched from master or region server.
>> > >> >
>> > >> > What does this mean please? Has to be run from Master or
>> RegionServer?
>> > >> Can
>> > >> > it be run from another node altogether?
>> > >> >
>> > >> > On Mon, Oct 10, 2016 at 1:44 PM, Vladimir Rodionov <
>> > >> vladrodionov@gmail.com
>> > >> > >
>> > >> > wrote:
>> > >> >
>> > >> > > >> mapreduce dependency has been moved to client side - no
>> mapreduce
>> > >> job
>> > >> > >
>> > >> > > 1. We have no code in the client module anymore, due to
>> dependency
>> > on
>> > >> > > internal server API (HFile and WAL access).
>> > >> > > 2. Backup/ restore are client - driven operations, but all the
>> code
>> > >> > resides
>> > >> > > in the server module
>> > >> > >
>> > >> >
>> > >> > How hard to put in an hbase-backup module? hbase-server is fat
>> enough
>> > >> > already. Could be done as a follow-up.
>> > >> >
>> > >> > Thanks,
>> > >> > St.Ack
>> > >> >
>> > >> >
>> > >> >
>> > >> > > 3. No MR in Master, no procedure - driven execution.
>> > >> > > 4. Old good MR from command-line.
>> > >> > > 5. Security was simplified and now only super-user is allowed to
>> run
>> > >> > > backup/restores.
>> > >> > > 6. HBase Backup API was gone due to 1. Now only command-line
>> access
>> > to
>> > >> > > backup tools.
>> > >> > >
>> > >> > > These consequences of refactoring has been discussed in
>> HBASE-16727.
>> > >> > >
>> > >> > > -Vlad
>> > >> > >
>> > >> > >
>> > >> > >
>> > >> > >
>> > >> > > On Mon, Oct 10, 2016 at 1:31 PM, Ted Yu <yu...@gmail.com>
>> > wrote:
>> > >> > >
>> > >> > > > Reviving this thread.
>> > >> > > >
>> > >> > > > The following has taken place:
>> > >> > > >
>> > >> > > > mapreduce dependency has been moved to client side - no
>> mapreduce
>> > >> job
>> > >> > > > launched from master or region server.
>> > >> > > > document patch (HBASE-16574) has been integrated.
>> > >> > > > Updated mega patch has been attached to HBASE-14123: this
>> covers
>> > the
>> > >> > > > refactor in #1 above and the protobuf 3 merge.
>> > >> > > >
>> > >> > > > If community has more feedback on the merge proposal, I would
>> love
>> > >> to
>> > >> > > hear
>> > >> > > > it.
>> > >> > > >
>> > >> > > > Thanks
>> > >> > > >
>> > >> > > > On Thu, Sep 22, 2016 at 10:31 AM, Sean Busbey <
>> > busbey@cloudera.com>
>> > >> > > wrote:
>> > >> > > >
>> > >> > > > > I'd like to see the docs proposed on HBASE-16574 integrated
>> into
>> > >> our
>> > >> > > > > project's documentation prior to merge.
>> > >> > > > >
>> > >> > > > > On Thu, Sep 22, 2016 at 9:02 AM, Ted Yu <yuzhihong@gmail.com
>> >
>> > >> wrote:
>> > >> > > > > > This feature can be marked experimental due to some
>> > limitations
>> > >> > such
>> > >> > > as
>> > >> > > > > > security.
>> > >> > > > > >
>> > >> > > > > > Your previous round of comments have been addressed.
>> > >> > > > > > Command line tool has gone through:
>> > >> > > > > >
>> > >> > > > > > HBASE-16620 Fix backup command-line tool usability issues
>> > >> > > > > > HBASE-16655 hbase backup describe with incorrect backup id
>> > >> results
>> > >> > in
>> > >> > > > NPE
>> > >> > > > > >
>> > >> > > > > > The updated doc has been attached to HBASE-16574.
>> > >> > > > > >
>> > >> > > > > > Cheers
>> > >> > > > > >
>> > >> > > > > > On Thu, Sep 22, 2016 at 8:53 AM, Stack <st...@duboce.net>
>> > >> wrote:
>> > >> > > > > >
>> > >> > > > > >> On Wed, Sep 21, 2016 at 7:43 AM, Ted Yu <
>> yuzhihong@gmail.com
>> > >
>> > >> > > wrote:
>> > >> > > > > >>
>> > >> > > > > >> > Are there more (review) comments ?
>> > >> > > > > >> >
>> > >> > > > > >> >
>> > >> > > > > >> Are outstanding comments addressed?
>> > >> > > > > >>
>> > >> > > > > >> I don't see answer to my 'is this experimental/will it be
>> > >> marked
>> > >> > > > > >> experimental' question.
>> > >> > > > > >>
>> > >> > > > > >> I ran into some issues trying to use the feature and
>> > suggested
>> > >> > that
>> > >> > > a
>> > >> > > > > >> feature likes this needs polish else it'll just rot,
>> unused.
>> > >> Has
>> > >> > > > polish
>> > >> > > > > >> been applied? All ready for another 'user' test? Suggest
>> that
>> > >> you
>> > >> > > > update
>> > >> > > > > >> here going forward for the benefit of those trying to
>> follow
>> > >> along
>> > >> > > and
>> > >> > > > > who
>> > >> > > > > >> are not watching JIRA change fly-by.
>> > >> > > > > >>
>> > >> > > > > >> It looks like doc got a revision -- I have to check -- to
>> > take
>> > >> on
>> > >> > > > > >> suggestion made above but again, suggest, that this thread
>> > gets
>> > >> > > > updated.
>> > >> > > > > >>
>> > >> > > > > >> Thanks,
>> > >> > > > > >> St.Ack
>> > >> > > > > >>
>> > >> > > > > >>
>> > >> > > > > >>
>> > >> > > > > >> > Thanks
>> > >> > > > > >> >
>> > >> > > > > >> > On Tue, Sep 20, 2016 at 10:02 AM, Devaraj Das <
>> > >> > > ddas@hortonworks.com
>> > >> > > > >
>> > >> > > > > >> > wrote:
>> > >> > > > > >> >
>> > >> > > > > >> > > Just reviving this thread. Thanks Sean, Stack, Dima,
>> and
>> > >> > others
>> > >> > > > for
>> > >> > > > > the
>> > >> > > > > >> > > thorough reviews and testing. Thanks Ted and Vlad for
>> > >> taking
>> > >> > > care
>> > >> > > > of
>> > >> > > > > >> the
>> > >> > > > > >> > > feedback. Are we all good to do the merge now? Rather
>> do
>> > >> > sooner
>> > >> > > > than
>> > >> > > > > >> > later.
>> > >> > > > > >> > > ________________________________________
>> > >> > > > > >> > > From: saint.ack@gmail.com <sa...@gmail.com> on
>> > behalf
>> > >> of
>> > >> > > > Stack
>> > >> > > > > <
>> > >> > > > > >> > > stack@duboce.net>
>> > >> > > > > >> > > Sent: Monday, September 12, 2016 1:18 PM
>> > >> > > > > >> > > To: HBase Dev List
>> > >> > > > > >> > > Subject: Re: [DISCUSSION] Merge Backup / Restore -
>> Branch
>> > >> > > > HBASE-7912
>> > >> > > > > >> > >
>> > >> > > > > >> > > On Mon, Sep 12, 2016 at 12:19 PM, Ted Yu <
>> > >> yuzhihong@gmail.com
>> > >> > >
>> > >> > > > > wrote:
>> > >> > > > > >> > >
>> > >> > > > > >> > > > Mega patch (rev 18) is on HBASE-14123.
>> > >> > > > > >> > > >
>> > >> > > > > >> > > > Please comment on HBASE-14123 on how you want to
>> > review.
>> > >> > > > > >> > > >
>> > >> > > > > >> > >
>> > >> > > > > >> > >
>> > >> > > > > >> > > Yeah. That was my lost tab. Last rb was 6 months ago.
>> > >> Suggest
>> > >> > > > > updating
>> > >> > > > > >> > it.
>> > >> > > > > >> > > RB is pretty good for review. Patch is only 1.5M so
>> > should
>> > >> be
>> > >> > > > fine.
>> > >> > > > > >> > >
>> > >> > > > > >> > > St.Ack
>> > >> > > > > >> > >
>> > >> > > > > >> > >
>> > >> > > > > >> > > >
>> > >> > > > > >> > > > Thanks
>> > >> > > > > >> > > >
>> > >> > > > > >> > > > On Mon, Sep 12, 2016 at 12:15 PM, Stack <
>> > >> stack@duboce.net>
>> > >> > > > wrote:
>> > >> > > > > >> > > >
>> > >> > > > > >> > > > > On review of the 'patch', do I just compare the
>> > branch
>> > >> to
>> > >> > > > > master or
>> > >> > > > > >> > is
>> > >> > > > > >> > > > > there a megapatch posted somewhere (I think I saw
>> one
>> > >> but
>> > >> > it
>> > >> > > > > seemed
>> > >> > > > > >> > > stale
>> > >> > > > > >> > > > > and then I 'lost' the tab). Sorry for dumb
>> question.
>> > >> > > > > >> > > > > St.Ack
>> > >> > > > > >> > > > >
>> > >> > > > > >> > > > > On Mon, Sep 12, 2016 at 12:01 PM, Stack <
>> > >> stack@duboce.net
>> > >> > >
>> > >> > > > > wrote:
>> > >> > > > > >> > > > >
>> > >> > > > > >> > > > > > Late to the game. A few comments after rereading
>> > this
>> > >> > > thread
>> > >> > > > > as a
>> > >> > > > > >> > > > 'user'.
>> > >> > > > > >> > > > > >
>> > >> > > > > >> > > > > > + Before merge, a user-facing feature like this
>> > >> should
>> > >> > > work
>> > >> > > > > (If
>> > >> > > > > >> > this
>> > >> > > > > >> > > is
>> > >> > > > > >> > > > > "higher-bar
>> > >> > > > > >> > > > > > for new features", bring it on -- smile).
>> > >> > > > > >> > > > > > + As a user, I tried the branch with tools after
>> > >> > reviewing
>> > >> > > > the
>> > >> > > > > >> > > > > just-posted
>> > >> > > > > >> > > > > > doc. I had an 'interesting' experience (left
>> > >> comments up
>> > >> > > on
>> > >> > > > > >> > issue). I
>> > >> > > > > >> > > > > think
>> > >> > > > > >> > > > > > the tooling/doc. important to get right. If it
>> > breaks
>> > >> > > easily
>> > >> > > > > or
>> > >> > > > > >> is
>> > >> > > > > >> > > > > > inconsistent (or lacks 'polish'), operators will
>> > >> judge
>> > >> > the
>> > >> > > > > whole
>> > >> > > > > >> > > > > > backup/restore tooling chain as not trustworthy
>> and
>> > >> > > abandon
>> > >> > > > > it.
>> > >> > > > > >> > Lets
>> > >> > > > > >> > > > not
>> > >> > > > > >> > > > > > have this happen to this feature.
>> > >> > > > > >> > > > > > + Matteo's suggestion (with a helpful starter
>> list)
>> > >> that
>> > >> > > > there
>> > >> > > > > >> > needs
>> > >> > > > > >> > > to
>> > >> > > > > >> > > > > be
>> > >> > > > > >> > > > > > explicit qualification on what is actually being
>> > >> > delivered
>> > >> > > > --
>> > >> > > > > >> > > > including a
>> > >> > > > > >> > > > > > listing of limitations (some look serious such
>> as
>> > >> data
>> > >> > > bleed
>> > >> > > > > from
>> > >> > > > > >> > > other
>> > >> > > > > >> > > > > > regions in WALs, but maybe I don't care for my
>> use
>> > >> > > case...)
>> > >> > > > --
>> > >> > > > > >> > needs
>> > >> > > > > >> > > to
>> > >> > > > > >> > > > > > accompany the merge. Lets fold them into the
>> user
>> > >> doc.
>> > >> > in
>> > >> > > > the
>> > >> > > > > >> > > technical
>> > >> > > > > >> > > > > > overview area as suggested so user expectations
>> are
>> > >> > > properly
>> > >> > > > > >> > managed
>> > >> > > > > >> > > > > > (otherwise, they expect the world and will just
>> > give
>> > >> up
>> > >> > > when
>> > >> > > > > we
>> > >> > > > > >> > fall
>> > >> > > > > >> > > > > > short). Vladimir did a list of what is in each
>> of
>> > the
>> > >> > > phases
>> > >> > > > > >> above
>> > >> > > > > >> > > > which
>> > >> > > > > >> > > > > > would serve as a good start.
>> > >> > > > > >> > > > > > + Is this feature 'experimental' (Matteo asks
>> > above).
>> > >> > I'd
>> > >> > > > > prefer
>> > >> > > > > >> it
>> > >> > > > > >> > > is
>> > >> > > > > >> > > > > > not. If it is, it should be labelled all over
>> that
>> > >> it is
>> > >> > > > so. I
>> > >> > > > > >> see
>> > >> > > > > >> > > > > current
>> > >> > > > > >> > > > > > state called out as a '... technical preview
>> > >> feature'.
>> > >> > > Does
>> > >> > > > > this
>> > >> > > > > >> > mean
>> > >> > > > > >> > > > > > not-for-users?
>> > >> > > > > >> > > > > >
>> > >> > > > > >> > > > > > St.Ack
>> > >> > > > > >> > > > > >
>> > >> > > > > >> > > > > >
>> > >> > > > > >> > > > > >
>> > >> > > > > >> > > > > >
>> > >> > > > > >> > > > > >
>> > >> > > > > >> > > > > >
>> > >> > > > > >> > > > > >
>> > >> > > > > >> > > > > >
>> > >> > > > > >> > > > > >
>> > >> > > > > >> > > > > >
>> > >> > > > > >> > > > > >
>> > >> > > > > >> > > > > > On Mon, Sep 12, 2016 at 8:03 AM, Ted Yu <
>> > >> > > > yuzhihong@gmail.com>
>> > >> > > > > >> > wrote:
>> > >> > > > > >> > > > > >
>> > >> > > > > >> > > > > >> Sean:
>> > >> > > > > >> > > > > >> Do you have more comments ?
>> > >> > > > > >> > > > > >>
>> > >> > > > > >> > > > > >> Cheers
>> > >> > > > > >> > > > > >>
>> > >> > > > > >> > > > > >> On Fri, Sep 9, 2016 at 1:42 PM, Vladimir
>> Rodionov
>> > <
>> > >> > > > > >> > > > > vladrodionov@gmail.com
>> > >> > > > > >> > > > > >> >
>> > >> > > > > >> > > > > >> wrote:
>> > >> > > > > >> > > > > >>
>> > >> > > > > >> > > > > >> > Sean,
>> > >> > > > > >> > > > > >> >
>> > >> > > > > >> > > > > >> > Backup/Restore can fail due to various
>> reasons:
>> > >> > network
>> > >> > > > > outage
>> > >> > > > > >> > > > > (cluster
>> > >> > > > > >> > > > > >> > wide), various time-outs in HBase and HDFS
>> > layer,
>> > >> M/R
>> > >> > > > > failure
>> > >> > > > > >> > due
>> > >> > > > > >> > > to
>> > >> > > > > >> > > > > >> "HDFS
>> > >> > > > > >> > > > > >> > exceeded quota", user error (manual deletion
>> of
>> > >> data)
>> > >> > > and
>> > >> > > > > so
>> > >> > > > > >> on
>> > >> > > > > >> > so
>> > >> > > > > >> > > > on.
>> > >> > > > > >> > > > > >> That
>> > >> > > > > >> > > > > >> > is impossible to enumerate all possible
>> types of
>> > >> > > failures
>> > >> > > > > in a
>> > >> > > > > >> > > > > >> distributed
>> > >> > > > > >> > > > > >> > system - that is not our goal/task.
>> > >> > > > > >> > > > > >> >
>> > >> > > > > >> > > > > >> > We focus completely on backup system table
>> > >> > consistency
>> > >> > > > in a
>> > >> > > > > >> > > presence
>> > >> > > > > >> > > > > of
>> > >> > > > > >> > > > > >> any
>> > >> > > > > >> > > > > >> > type of failure. That is what I call
>> "tolerance
>> > to
>> > >> > > > > failures".
>> > >> > > > > >> > > > > >> >
>> > >> > > > > >> > > > > >> > On a failure:
>> > >> > > > > >> > > > > >> >
>> > >> > > > > >> > > > > >> > BACKUP. All backup system information (prior
>> to
>> > >> > backup)
>> > >> > > > > will
>> > >> > > > > >> be
>> > >> > > > > >> > > > > restored
>> > >> > > > > >> > > > > >> > and all temporary data, related to a failed
>> > >> session,
>> > >> > in
>> > >> > > > > HDFS
>> > >> > > > > >> > will
>> > >> > > > > >> > > be
>> > >> > > > > >> > > > > >> > deleted
>> > >> > > > > >> > > > > >> > RESTORE. We do not care about system data,
>> > because
>> > >> > > > restore
>> > >> > > > > >> does
>> > >> > > > > >> > > not
>> > >> > > > > >> > > > > >> change
>> > >> > > > > >> > > > > >> > it. Temporary data in HDFS will be cleaned up
>> > and
>> > >> > table
>> > >> > > > > will
>> > >> > > > > >> be
>> > >> > > > > >> > > in a
>> > >> > > > > >> > > > > >> state
>> > >> > > > > >> > > > > >> > back to where it was before operation
>> started.
>> > >> > > > > >> > > > > >> >
>> > >> > > > > >> > > > > >> > This is what user should expect in case of a
>> > >> failure.
>> > >> > > > > >> > > > > >> >
>> > >> > > > > >> > > > > >> > -Vlad
>> > >> > > > > >> > > > > >> >
>> > >> > > > > >> > > > > >> >
>> > >> > > > > >> > > > > >> > -Vlad
>> > >> > > > > >> > > > > >> >
>> > >> > > > > >> > > > > >> > On Fri, Sep 9, 2016 at 12:56 PM, Sean Busbey
>> <
>> > >> > > > > >> busbey@apache.org
>> > >> > > > > >> > >
>> > >> > > > > >> > > > > wrote:
>> > >> > > > > >> > > > > >> >
>> > >> > > > > >> > > > > >> > > Failing in a consistent way, with docs that
>> > >> explain
>> > >> > > the
>> > >> > > > > >> > various
>> > >> > > > > >> > > > > >> > > expected failures would be sufficient.
>> > >> > > > > >> > > > > >> > >
>> > >> > > > > >> > > > > >> > > On Fri, Sep 9, 2016 at 12:16 PM, Vladimir
>> > >> Rodionov
>> > >> > > > > >> > > > > >> > > <vl...@gmail.com> wrote:
>> > >> > > > > >> > > > > >> > > > Do not worry Sean, doc is coming today
>> as a
>> > >> > preview
>> > >> > > > and
>> > >> > > > > >> our
>> > >> > > > > >> > > > writer
>> > >> > > > > >> > > > > >> > Frank
>> > >> > > > > >> > > > > >> > > > will be working on a putting  it into
>> Apache
>> > >> > repo.
>> > >> > > > > >> Timeline
>> > >> > > > > >> > > > > depends
>> > >> > > > > >> > > > > >> on
>> > >> > > > > >> > > > > >> > > > Franks schedule but I hope we will get it
>> > >> rather
>> > >> > > > sooner
>> > >> > > > > >> than
>> > >> > > > > >> > > > > later.
>> > >> > > > > >> > > > > >> > > >
>> > >> > > > > >> > > > > >> > > > As for failure testing, we are focusing
>> only
>> > >> on a
>> > >> > > > > >> consistent
>> > >> > > > > >> > > > state
>> > >> > > > > >> > > > > >> of
>> > >> > > > > >> > > > > >> > > > backup system data in a presence of any
>> type
>> > >> of
>> > >> > > > > failures,
>> > >> > > > > >> We
>> > >> > > > > >> > > are
>> > >> > > > > >> > > > > not
>> > >> > > > > >> > > > > >> > > going
>> > >> > > > > >> > > > > >> > > > to implement  anything more "fancy", than
>> > >> that.
>> > >> > We
>> > >> > > > > allow
>> > >> > > > > >> > both:
>> > >> > > > > >> > > > > >> backup
>> > >> > > > > >> > > > > >> > and
>> > >> > > > > >> > > > > >> > > > restore to fail. What we do not allow is
>> to
>> > >> have
>> > >> > > > system
>> > >> > > > > >> data
>> > >> > > > > >> > > > > >> corrupted.
>> > >> > > > > >> > > > > >> > > > Will it suffice for you? Do you have any
>> > other
>> > >> > > > > concerns,
>> > >> > > > > >> you
>> > >> > > > > >> > > > want
>> > >> > > > > >> > > > > >> us to
>> > >> > > > > >> > > > > >> > > > address?
>> > >> > > > > >> > > > > >> > > >
>> > >> > > > > >> > > > > >> > > > -Vlad
>> > >> > > > > >> > > > > >> > > >
>> > >> > > > > >> > > > > >> > > >
>> > >> > > > > >> > > > > >> > > > On Fri, Sep 9, 2016 at 10:56 AM, Sean
>> > Busbey <
>> > >> > > > > >> > > busbey@apache.org
>> > >> > > > > >> > > > >
>> > >> > > > > >> > > > > >> > wrote:
>> > >> > > > > >> > > > > >> > > >
>> > >> > > > > >> > > > > >> > > >> "docs will come to Apache soon" does not
>> > >> address
>> > >> > > my
>> > >> > > > > >> concern
>> > >> > > > > >> > > > > around
>> > >> > > > > >> > > > > >> > docs
>> > >> > > > > >> > > > > >> > > at
>> > >> > > > > >> > > > > >> > > >> all, unless said docs have already made
>> it
>> > >> into
>> > >> > > the
>> > >> > > > > >> project
>> > >> > > > > >> > > > > repo. I
>> > >> > > > > >> > > > > >> > > don't
>> > >> > > > > >> > > > > >> > > >> want third party resources for using a
>> > major
>> > >> and
>> > >> > > > > >> important
>> > >> > > > > >> > > > > feature
>> > >> > > > > >> > > > > >> of
>> > >> > > > > >> > > > > >> > > the
>> > >> > > > > >> > > > > >> > > >> project, I want us to provide end users
>> > with
>> > >> > what
>> > >> > > > they
>> > >> > > > > >> need
>> > >> > > > > >> > > to
>> > >> > > > > >> > > > > get
>> > >> > > > > >> > > > > >> the
>> > >> > > > > >> > > > > >> > > job
>> > >> > > > > >> > > > > >> > > >> done.
>> > >> > > > > >> > > > > >> > > >>
>> > >> > > > > >> > > > > >> > > >> I see some calls for patience on the
>> > failure
>> > >> > > > testing,
>> > >> > > > > but
>> > >> > > > > >> > the
>> > >> > > > > >> > > > > >> appeal
>> > >> > > > > >> > > > > >> > to
>> > >> > > > > >> > > > > >> > > us
>> > >> > > > > >> > > > > >> > > >> having done a bad job of requiring
>> proper
>> > >> tests
>> > >> > of
>> > >> > > > > >> previous
>> > >> > > > > >> > > > > >> features
>> > >> > > > > >> > > > > >> > > just
>> > >> > > > > >> > > > > >> > > >> makes me more concerned about not
>> getting
>> > >> them
>> > >> > > > here. I
>> > >> > > > > >> > don't
>> > >> > > > > >> > > > want
>> > >> > > > > >> > > > > >> to
>> > >> > > > > >> > > > > >> > set
>> > >> > > > > >> > > > > >> > > >> yet another bad example that will then
>> be
>> > >> > pointed
>> > >> > > to
>> > >> > > > > in
>> > >> > > > > >> the
>> > >> > > > > >> > > > > future.
>> > >> > > > > >> > > > > >> > > >>
>> > >> > > > > >> > > > > >> > > >> On Sep 8, 2016 10:50, "Ted Yu" <
>> > >> > > yuzhihong@gmail.com
>> > >> > > > >
>> > >> > > > > >> > wrote:
>> > >> > > > > >> > > > > >> > > >>
>> > >> > > > > >> > > > > >> > > >> > Is there any concern which is not
>> > >> addressed ?
>> > >> > > > > >> > > > > >> > > >> >
>> > >> > > > > >> > > > > >> > > >> > Do we need another Vote thread ?
>> > >> > > > > >> > > > > >> > > >> >
>> > >> > > > > >> > > > > >> > > >> > Thanks
>> > >> > > > > >> > > > > >> > > >> >
>> > >> > > > > >> > > > > >> > > >> > On Thu, Sep 8, 2016 at 9:21 AM, Andrew
>> > >> > Purtell <
>> > >> > > > > >> > > > > >> apurtell@apache.org
>> > >> > > > > >> > > > > >> > >
>> > >> > > > > >> > > > > >> > > >> > wrote:
>> > >> > > > > >> > > > > >> > > >> >
>> > >> > > > > >> > > > > >> > > >> > > Vlad,
>> > >> > > > > >> > > > > >> > > >> > >
>> > >> > > > > >> > > > > >> > > >> > > I apologize for using the term
>> > >> 'half-baked'
>> > >> > > in a
>> > >> > > > > way
>> > >> > > > > >> > that
>> > >> > > > > >> > > > > could
>> > >> > > > > >> > > > > >> > > seem a
>> > >> > > > > >> > > > > >> > > >> > > description of HBASE-7912. I meant
>> that
>> > >> as a
>> > >> > > > > general
>> > >> > > > > >> > > > > >> hypothetical.
>> > >> > > > > >> > > > > >> > > >> > >
>> > >> > > > > >> > > > > >> > > >> > > On Wed, Sep 7, 2016 at 9:36 AM,
>> > Vladimir
>> > >> > > > Rodionov
>> > >> > > > > <
>> > >> > > > > >> > > > > >> > > >> > vladrodionov@gmail.com>
>> > >> > > > > >> > > > > >> > > >> > > wrote:
>> > >> > > > > >> > > > > >> > > >> > >
>> > >> > > > > >> > > > > >> > > >> > > > >> I'm not sure that "There is
>> > already
>> > >> > lots
>> > >> > > of
>> > >> > > > > >> > > half-baked
>> > >> > > > > >> > > > > >> code
>> > >> > > > > >> > > > > >> > in
>> > >> > > > > >> > > > > >> > > the
>> > >> > > > > >> > > > > >> > > >> > > > branch,
>> > >> > > > > >> > > > > >> > > >> > > > so what's the harm in adding
>> more?"
>> > >> > > > > >> > > > > >> > > >> > > >
>> > >> > > > > >> > > > > >> > > >> > > > I meant - not production - ready
>> yet.
>> > >> This
>> > >> > > is
>> > >> > > > > 2.0
>> > >> > > > > >> > > > > development
>> > >> > > > > >> > > > > >> > > branch
>> > >> > > > > >> > > > > >> > > >> > and,
>> > >> > > > > >> > > > > >> > > >> > > > hence many features are in works,
>> > >> > > > > >> > > > > >> > > >> > > > not being tested well etc. I do
>> not
>> > >> > consider
>> > >> > > > > backup
>> > >> > > > > >> > as
>> > >> > > > > >> > > > half
>> > >> > > > > >> > > > > >> > baked
>> > >> > > > > >> > > > > >> > > >> > > feature -
>> > >> > > > > >> > > > > >> > > >> > > > it has passed our internal QA and
>> has
>> > >> very
>> > >> > > > good
>> > >> > > > > >> doc,
>> > >> > > > > >> > > > which
>> > >> > > > > >> > > > > we
>> > >> > > > > >> > > > > >> > will
>> > >> > > > > >> > > > > >> > > >> > > provide
>> > >> > > > > >> > > > > >> > > >> > > > to Apache shortly.
>> > >> > > > > >> > > > > >> > > >> > > >
>> > >> > > > > >> > > > > >> > > >> > > > -Vlad
>> > >> > > > > >> > > > > >> > > >> > > >
>> > >> > > > > >> > > > > >> > > >> > > > On Wed, Sep 7, 2016 at 9:13 AM,
>> > Andrew
>> > >> > > > Purtell <
>> > >> > > > > >> > > > > >> > > apurtell@apache.org>
>> > >> > > > > >> > > > > >> > > >> > > > wrote:
>> > >> > > > > >> > > > > >> > > >> > > >
>> > >> > > > > >> > > > > >> > > >> > > > > We shouldn't admit half baked
>> > changes
>> > >> > that
>> > >> > > > > won't
>> > >> > > > > >> be
>> > >> > > > > >> > > > > >> finished.
>> > >> > > > > >> > > > > >> > > >> However
>> > >> > > > > >> > > > > >> > > >> > > in
>> > >> > > > > >> > > > > >> > > >> > > > > this case the crew working on
>> this
>> > >> > feature
>> > >> > > > are
>> > >> > > > > >> long
>> > >> > > > > >> > > > > timers
>> > >> > > > > >> > > > > >> and
>> > >> > > > > >> > > > > >> > > less
>> > >> > > > > >> > > > > >> > > >> > > > likely
>> > >> > > > > >> > > > > >> > > >> > > > > than just about anyone to leave
>> > >> > something
>> > >> > > > in a
>> > >> > > > > >> half
>> > >> > > > > >> > > > baked
>> > >> > > > > >> > > > > >> > > state. Of
>> > >> > > > > >> > > > > >> > > >> > > > course
>> > >> > > > > >> > > > > >> > > >> > > > > there is no guarantee how
>> anything
>> > >> will
>> > >> > > turn
>> > >> > > > > out,
>> > >> > > > > >> > > but I
>> > >> > > > > >> > > > > am
>> > >> > > > > >> > > > > >> > > willing
>> > >> > > > > >> > > > > >> > > >> to
>> > >> > > > > >> > > > > >> > > >> > > > take
>> > >> > > > > >> > > > > >> > > >> > > > > a little on faith if they feel
>> > their
>> > >> > best
>> > >> > > > path
>> > >> > > > > >> > > forward
>> > >> > > > > >> > > > > now
>> > >> > > > > >> > > > > >> is
>> > >> > > > > >> > > > > >> > to
>> > >> > > > > >> > > > > >> > > >> > merge
>> > >> > > > > >> > > > > >> > > >> > > to
>> > >> > > > > >> > > > > >> > > >> > > > > trunk. I only wish I had
>> bandwidth
>> > to
>> > >> > have
>> > >> > > > > done
>> > >> > > > > >> > some
>> > >> > > > > >> > > > real
>> > >> > > > > >> > > > > >> > > kicking
>> > >> > > > > >> > > > > >> > > >> of
>> > >> > > > > >> > > > > >> > > >> > > the
>> > >> > > > > >> > > > > >> > > >> > > > > tires by now. Maybe this week.
>> > >> > > > > >> > > > > >> > > >> > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > (Yes, I'm using some of that
>> time
>> > for
>> > >> > this
>> > >> > > > > email
>> > >> > > > > >> > :-)
>> > >> > > > > >> > > > but
>> > >> > > > > >> > > > > I
>> > >> > > > > >> > > > > >> > type
>> > >> > > > > >> > > > > >> > > >> > fast.)
>> > >> > > > > >> > > > > >> > > >> > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > That said, I would like to
>> agitate
>> > >> for
>> > >> > > > making
>> > >> > > > > 2.0
>> > >> > > > > >> > > more
>> > >> > > > > >> > > > > real
>> > >> > > > > >> > > > > >> > and
>> > >> > > > > >> > > > > >> > > >> spend
>> > >> > > > > >> > > > > >> > > >> > > > some
>> > >> > > > > >> > > > > >> > > >> > > > > time on it now that I'm winding
>> > down
>> > >> > with
>> > >> > > > > 0.98. I
>> > >> > > > > >> > > think
>> > >> > > > > >> > > > > >> that
>> > >> > > > > >> > > > > >> > > means
>> > >> > > > > >> > > > > >> > > >> > > > > branching for 2.0 real soon now
>> and
>> > >> even
>> > >> > > > > evicting
>> > >> > > > > >> > > > things
>> > >> > > > > >> > > > > >> from
>> > >> > > > > >> > > > > >> > > 2.0
>> > >> > > > > >> > > > > >> > > >> > > branch
>> > >> > > > > >> > > > > >> > > >> > > > > that aren't finished or stable,
>> > >> leaving
>> > >> > > them
>> > >> > > > > only
>> > >> > > > > >> > > once
>> > >> > > > > >> > > > > >> again
>> > >> > > > > >> > > > > >> > in
>> > >> > > > > >> > > > > >> > > the
>> > >> > > > > >> > > > > >> > > >> > > > master
>> > >> > > > > >> > > > > >> > > >> > > > > branch. Or, maybe just evicting
>> > them.
>> > >> > > Let's
>> > >> > > > > take
>> > >> > > > > >> it
>> > >> > > > > >> > > > case
>> > >> > > > > >> > > > > by
>> > >> > > > > >> > > > > >> > > case.
>> > >> > > > > >> > > > > >> > > >> > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > I think this feature can come in
>> > >> > > relatively
>> > >> > > > > >> safely.
>> > >> > > > > >> > > As
>> > >> > > > > >> > > > > >> added
>> > >> > > > > >> > > > > >> > > >> > insurance,
>> > >> > > > > >> > > > > >> > > >> > > > > let's admit the possibility it
>> > could
>> > >> be
>> > >> > > > > reverted
>> > >> > > > > >> on
>> > >> > > > > >> > > the
>> > >> > > > > >> > > > > 2.0
>> > >> > > > > >> > > > > >> > > branch
>> > >> > > > > >> > > > > >> > > >> if
>> > >> > > > > >> > > > > >> > > >> > > > folks
>> > >> > > > > >> > > > > >> > > >> > > > > working on stabilizing 2.0
>> decide
>> > to
>> > >> > evict
>> > >> > > > it
>> > >> > > > > >> > because
>> > >> > > > > >> > > > it
>> > >> > > > > >> > > > > is
>> > >> > > > > >> > > > > >> > > >> > unfinished
>> > >> > > > > >> > > > > >> > > >> > > or
>> > >> > > > > >> > > > > >> > > >> > > > > unstable, because that certainly
>> > can
>> > >> > > > happen. I
>> > >> > > > > >> > would
>> > >> > > > > >> > > > > >> expect if
>> > >> > > > > >> > > > > >> > > talk
>> > >> > > > > >> > > > > >> > > >> > > like
>> > >> > > > > >> > > > > >> > > >> > > > > that starts, we'd get help
>> > finishing
>> > >> or
>> > >> > > > > >> stabilizing
>> > >> > > > > >> > > > > what's
>> > >> > > > > >> > > > > >> > under
>> > >> > > > > >> > > > > >> > > >> > > > discussion
>> > >> > > > > >> > > > > >> > > >> > > > > for revert. Or, we'd have a
>> revert.
>> > >> > Either
>> > >> > > > way
>> > >> > > > > >> the
>> > >> > > > > >> > > > > outcome
>> > >> > > > > >> > > > > >> is
>> > >> > > > > >> > > > > >> > > >> > > acceptable.
>> > >> > > > > >> > > > > >> > > >> > > > >
>> > >> > > > > >> > > > > >> > > >> > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > On Wed, Sep 7, 2016 at 8:56 AM,
>> > Dima
>> > >> > > Spivak
>> > >> > > > <
>> > >> > > > > >> > > > > >> > > dimaspivak@apache.org
>> > >> > > > > >> > > > > >> > > >> >
>> > >> > > > > >> > > > > >> > > >> > > > wrote:
>> > >> > > > > >> > > > > >> > > >> > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > > I'm not sure that "There is
>> > already
>> > >> > lots
>> > >> > > > of
>> > >> > > > > >> > > > half-baked
>> > >> > > > > >> > > > > >> code
>> > >> > > > > >> > > > > >> > in
>> > >> > > > > >> > > > > >> > > >> the
>> > >> > > > > >> > > > > >> > > >> > > > > branch,
>> > >> > > > > >> > > > > >> > > >> > > > > > so what's the harm in adding
>> > more?"
>> > >> > is a
>> > >> > > > > good
>> > >> > > > > >> > code
>> > >> > > > > >> > > > > commit
>> > >> > > > > >> > > > > >> > > >> > philosophy
>> > >> > > > > >> > > > > >> > > >> > > > for
>> > >> > > > > >> > > > > >> > > >> > > > > a
>> > >> > > > > >> > > > > >> > > >> > > > > > fault-tolerant distributed
>> data
>> > >> store.
>> > >> > > ;)
>> > >> > > > > >> > > > > >> > > >> > > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > > More seriously, a lack of test
>> > >> > coverage
>> > >> > > > for
>> > >> > > > > >> > > existing
>> > >> > > > > >> > > > > >> > features
>> > >> > > > > >> > > > > >> > > >> > > shouldn't
>> > >> > > > > >> > > > > >> > > >> > > > > be
>> > >> > > > > >> > > > > >> > > >> > > > > > used as justification for
>> > >> introducing
>> > >> > > new
>> > >> > > > > >> > features
>> > >> > > > > >> > > > with
>> > >> > > > > >> > > > > >> the
>> > >> > > > > >> > > > > >> > > same
>> > >> > > > > >> > > > > >> > > >> > > > > > shortcomings. Ultimately, it's
>> > the
>> > >> end
>> > >> > > > user
>> > >> > > > > who
>> > >> > > > > >> > > will
>> > >> > > > > >> > > > > feel
>> > >> > > > > >> > > > > >> > the
>> > >> > > > > >> > > > > >> > > >> pain,
>> > >> > > > > >> > > > > >> > > >> > > so
>> > >> > > > > >> > > > > >> > > >> > > > > > shouldn't we do everything we
>> can
>> > >> to
>> > >> > > > > mitigate
>> > >> > > > > >> > that?
>> > >> > > > > >> > > > > >> > > >> > > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > > -Dima
>> > >> > > > > >> > > > > >> > > >> > > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > > On Wed, Sep 7, 2016 at 8:46
>> AM,
>> > >> > Vladimir
>> > >> > > > > >> > Rodionov <
>> > >> > > > > >> > > > > >> > > >> > > > > vladrodionov@gmail.com>
>> > >> > > > > >> > > > > >> > > >> > > > > > wrote:
>> > >> > > > > >> > > > > >> > > >> > > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > > > Sean,
>> > >> > > > > >> > > > > >> > > >> > > > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > > > * have docs
>> > >> > > > > >> > > > > >> > > >> > > > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > > > Agree. We have a doc and
>> backup
>> > >> is
>> > >> > the
>> > >> > > > > most
>> > >> > > > > >> > > > > documented
>> > >> > > > > >> > > > > >> > > feature
>> > >> > > > > >> > > > > >> > > >> > :),
>> > >> > > > > >> > > > > >> > > >> > > we
>> > >> > > > > >> > > > > >> > > >> > > > > > will
>> > >> > > > > >> > > > > >> > > >> > > > > > > release it shortly to
>> Apache.
>> > >> > > > > >> > > > > >> > > >> > > > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > > > * have sunny-day correctness
>> > >> tests
>> > >> > > > > >> > > > > >> > > >> > > > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > > > Feature has  close to 60
>> test
>> > >> cases,
>> > >> > > > which
>> > >> > > > > >> run
>> > >> > > > > >> > > for
>> > >> > > > > >> > > > > >> approx
>> > >> > > > > >> > > > > >> > 30
>> > >> > > > > >> > > > > >> > > >> min.
>> > >> > > > > >> > > > > >> > > >> > > We
>> > >> > > > > >> > > > > >> > > >> > > > > can
>> > >> > > > > >> > > > > >> > > >> > > > > > > add more, if community do
>> not
>> > >> mind
>> > >> > :)
>> > >> > > > > >> > > > > >> > > >> > > > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > > > * have
>> > >> > correctness-in-face-of-failure
>> > >> > > > > tests
>> > >> > > > > >> > > > > >> > > >> > > > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > > > Any examples of these tests
>> in
>> > >> > > existing
>> > >> > > > > >> > features?
>> > >> > > > > >> > > > In
>> > >> > > > > >> > > > > >> > works,
>> > >> > > > > >> > > > > >> > > we
>> > >> > > > > >> > > > > >> > > >> > > have a
>> > >> > > > > >> > > > > >> > > >> > > > > > clear
>> > >> > > > > >> > > > > >> > > >> > > > > > > understanding of what
>> should be
>> > >> done
>> > >> > > by
>> > >> > > > > the
>> > >> > > > > >> > time
>> > >> > > > > >> > > of
>> > >> > > > > >> > > > > 2.0
>> > >> > > > > >> > > > > >> > > >> release.
>> > >> > > > > >> > > > > >> > > >> > > > > > > That is very close goal for
>> us,
>> > >> to
>> > >> > > > verify
>> > >> > > > > IT
>> > >> > > > > >> > > monkey
>> > >> > > > > >> > > > > for
>> > >> > > > > >> > > > > >> > > >> existing
>> > >> > > > > >> > > > > >> > > >> > > > code.
>> > >> > > > > >> > > > > >> > > >> > > > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > > > * don't rely on things
>> outside
>> > of
>> > >> > > HBase
>> > >> > > > > for
>> > >> > > > > >> > > normal
>> > >> > > > > >> > > > > >> > operation
>> > >> > > > > >> > > > > >> > > >> > (okay
>> > >> > > > > >> > > > > >> > > >> > > > for
>> > >> > > > > >> > > > > >> > > >> > > > > > > advanced operation)
>> > >> > > > > >> > > > > >> > > >> > > > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > > > We do not.
>> > >> > > > > >> > > > > >> > > >> > > > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > > > Enormous time has been spent
>> > >> already
>> > >> > > on
>> > >> > > > > the
>> > >> > > > > >> > > > > development
>> > >> > > > > >> > > > > >> > and
>> > >> > > > > >> > > > > >> > > >> > testing
>> > >> > > > > >> > > > > >> > > >> > > > the
>> > >> > > > > >> > > > > >> > > >> > > > > > > feature, it has passed our
>> > >> internal
>> > >> > > > tests
>> > >> > > > > and
>> > >> > > > > >> > > many
>> > >> > > > > >> > > > > >> rounds
>> > >> > > > > >> > > > > >> > of
>> > >> > > > > >> > > > > >> > > >> code
>> > >> > > > > >> > > > > >> > > >> > > > > reviews
>> > >> > > > > >> > > > > >> > > >> > > > > > > by HBase committers. We do
>> not
>> > >> mind
>> > >> > if
>> > >> > > > > >> someone
>> > >> > > > > >> > > from
>> > >> > > > > >> > > > > >> HBase
>> > >> > > > > >> > > > > >> > > >> > community
>> > >> > > > > >> > > > > >> > > >> > > > > > > (outside of HW) will review
>> the
>> > >> > code,
>> > >> > > > but
>> > >> > > > > it
>> > >> > > > > >> > will
>> > >> > > > > >> > > > > >> probably
>> > >> > > > > >> > > > > >> > > >> takes
>> > >> > > > > >> > > > > >> > > >> > > > > forever
>> > >> > > > > >> > > > > >> > > >> > > > > > to
>> > >> > > > > >> > > > > >> > > >> > > > > > > wait for volunteer?, the
>> > feature
>> > >> is
>> > >> > > > quite
>> > >> > > > > >> large
>> > >> > > > > >> > > > (1MB+
>> > >> > > > > >> > > > > >> > > >> cumulative
>> > >> > > > > >> > > > > >> > > >> > > > patch)
>> > >> > > > > >> > > > > >> > > >> > > > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > > > 2.0 branch is full of half
>> > baked
>> > >> > > > features,
>> > >> > > > > >> most
>> > >> > > > > >> > > of
>> > >> > > > > >> > > > > them
>> > >> > > > > >> > > > > >> > are
>> > >> > > > > >> > > > > >> > > in
>> > >> > > > > >> > > > > >> > > >> > > active
>> > >> > > > > >> > > > > >> > > >> > > > > > > development, therefore I am
>> not
>> > >> > > > following
>> > >> > > > > you
>> > >> > > > > >> > > here,
>> > >> > > > > >> > > > > >> Sean?
>> > >> > > > > >> > > > > >> > > Why
>> > >> > > > > >> > > > > >> > > >> > > > > HBASE-7912
>> > >> > > > > >> > > > > >> > > >> > > > > > is
>> > >> > > > > >> > > > > >> > > >> > > > > > > not good enough yet to be
>> > >> integrated
>> > >> > > > into
>> > >> > > > > 2.0
>> > >> > > > > >> > > > branch?
>> > >> > > > > >> > > > > >> > > >> > > > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > > > -Vlad
>> > >> > > > > >> > > > > >> > > >> > > > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > > > On Wed, Sep 7, 2016 at 8:23
>> AM,
>> > >> Sean
>> > >> > > > > Busbey <
>> > >> > > > > >> > > > > >> > > busbey@apache.org
>> > >> > > > > >> > > > > >> > > >> >
>> > >> > > > > >> > > > > >> > > >> > > > wrote:
>> > >> > > > > >> > > > > >> > > >> > > > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > > > > On Tue, Sep 6, 2016 at
>> 10:36
>> > >> PM,
>> > >> > > Josh
>> > >> > > > > >> Elser <
>> > >> > > > > >> > > > > >> > > >> > > josh.elser@gmail.com>
>> > >> > > > > >> > > > > >> > > >> > > > > > > wrote:
>> > >> > > > > >> > > > > >> > > >> > > > > > > > > So, the answer to Sean's
>> > >> > original
>> > >> > > > > >> question
>> > >> > > > > >> > is
>> > >> > > > > >> > > > "as
>> > >> > > > > >> > > > > >> > > robust as
>> > >> > > > > >> > > > > >> > > >> > > > > snapshots
>> > >> > > > > >> > > > > >> > > >> > > > > > > > > presently are"?
>> > >> (independence of
>> > >> > > > > >> > > backup/restore
>> > >> > > > > >> > > > > >> > failure
>> > >> > > > > >> > > > > >> > > >> > > tolerance
>> > >> > > > > >> > > > > >> > > >> > > > > > from
>> > >> > > > > >> > > > > >> > > >> > > > > > > > > snapshot failure
>> tolerance)
>> > >> > > > > >> > > > > >> > > >> > > > > > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > > > > > Is this just a question
>> WRT
>> > >> > > context
>> > >> > > > of
>> > >> > > > > >> the
>> > >> > > > > >> > > > > change,
>> > >> > > > > >> > > > > >> or
>> > >> > > > > >> > > > > >> > > is it
>> > >> > > > > >> > > > > >> > > >> > > means
>> > >> > > > > >> > > > > >> > > >> > > > > > for a
>> > >> > > > > >> > > > > >> > > >> > > > > > > > veto
>> > >> > > > > >> > > > > >> > > >> > > > > > > > > from you, Sean? Just
>> trying
>> > >> to
>> > >> > > make
>> > >> > > > > sure
>> > >> > > > > >> > I'm
>> > >> > > > > >> > > > > >> following
>> > >> > > > > >> > > > > >> > > >> along
>> > >> > > > > >> > > > > >> > > >> > > > > > > adequately.
>> > >> > > > > >> > > > > >> > > >> > > > > > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > > > > I'd say ATM I'm -0,
>> bordering
>> > >> on
>> > >> > -1
>> > >> > > > but
>> > >> > > > > not
>> > >> > > > > >> > for
>> > >> > > > > >> > > > > >> reasons
>> > >> > > > > >> > > > > >> > I
>> > >> > > > > >> > > > > >> > > can
>> > >> > > > > >> > > > > >> > > >> > > > > > articulate
>> > >> > > > > >> > > > > >> > > >> > > > > > > > well.
>> > >> > > > > >> > > > > >> > > >> > > > > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > > > > Here's an attempt.
>> > >> > > > > >> > > > > >> > > >> > > > > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > > > > We've been trying to move,
>> > as a
>> > >> > > > > community,
>> > >> > > > > >> > > > towards
>> > >> > > > > >> > > > > >> > > minimizing
>> > >> > > > > >> > > > > >> > > >> > > risk
>> > >> > > > > >> > > > > >> > > >> > > > to
>> > >> > > > > >> > > > > >> > > >> > > > > > > > downstream folks by
>> getting
>> > >> > > "complete
>> > >> > > > > >> enough
>> > >> > > > > >> > > for
>> > >> > > > > >> > > > > use"
>> > >> > > > > >> > > > > >> > > gates
>> > >> > > > > >> > > > > >> > > >> in
>> > >> > > > > >> > > > > >> > > >> > > > place
>> > >> > > > > >> > > > > >> > > >> > > > > > > > before we introduce new
>> > >> features.
>> > >> > > This
>> > >> > > > > was
>> > >> > > > > >> > > > spurred
>> > >> > > > > >> > > > > >> by a
>> > >> > > > > >> > > > > >> > > some
>> > >> > > > > >> > > > > >> > > >> > > > features
>> > >> > > > > >> > > > > >> > > >> > > > > > > > getting in half-baked and
>> > never
>> > >> > > making
>> > >> > > > > it
>> > >> > > > > >> to
>> > >> > > > > >> > > "can
>> > >> > > > > >> > > > > >> really
>> > >> > > > > >> > > > > >> > > use"
>> > >> > > > > >> > > > > >> > > >> > > > status
>> > >> > > > > >> > > > > >> > > >> > > > > > > > (I'm thinking of
>> distributed
>> > >> log
>> > >> > > > replay
>> > >> > > > > and
>> > >> > > > > >> > the
>> > >> > > > > >> > > > > >> zk-less
>> > >> > > > > >> > > > > >> > > >> > > assignment
>> > >> > > > > >> > > > > >> > > >> > > > > > > > stuff, I don't recall if
>> > there
>> > >> was
>> > >> > > > > more).
>> > >> > > > > >> > > > > >> > > >> > > > > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > > > > The gates, generally,
>> > included
>> > >> > > things
>> > >> > > > > like:
>> > >> > > > > >> > > > > >> > > >> > > > > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > > > > * have docs
>> > >> > > > > >> > > > > >> > > >> > > > > > > > * have sunny-day
>> correctness
>> > >> tests
>> > >> > > > > >> > > > > >> > > >> > > > > > > > * have
>> > >> > > correctness-in-face-of-failure
>> > >> > > > > tests
>> > >> > > > > >> > > > > >> > > >> > > > > > > > * don't rely on things
>> > outside
>> > >> of
>> > >> > > > HBase
>> > >> > > > > for
>> > >> > > > > >> > > > normal
>> > >> > > > > >> > > > > >> > > operation
>> > >> > > > > >> > > > > >> > > >> > > (okay
>> > >> > > > > >> > > > > >> > > >> > > > > for
>> > >> > > > > >> > > > > >> > > >> > > > > > > > advanced operation)
>> > >> > > > > >> > > > > >> > > >> > > > > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > > > > As an example, we kept the
>> > MOB
>> > >> > work
>> > >> > > > off
>> > >> > > > > in
>> > >> > > > > >> a
>> > >> > > > > >> > > > branch
>> > >> > > > > >> > > > > >> and
>> > >> > > > > >> > > > > >> > > out
>> > >> > > > > >> > > > > >> > > >> of
>> > >> > > > > >> > > > > >> > > >> > > > master
>> > >> > > > > >> > > > > >> > > >> > > > > > > > until it could pass these
>> > >> > criteria.
>> > >> > > > The
>> > >> > > > > big
>> > >> > > > > >> > > > > exemption
>> > >> > > > > >> > > > > >> > > we've
>> > >> > > > > >> > > > > >> > > >> had
>> > >> > > > > >> > > > > >> > > >> > > to
>> > >> > > > > >> > > > > >> > > >> > > > > > > > this was the hbase-spark
>> > >> > > integration,
>> > >> > > > > where
>> > >> > > > > >> > we
>> > >> > > > > >> > > > all
>> > >> > > > > >> > > > > >> > agreed
>> > >> > > > > >> > > > > >> > > it
>> > >> > > > > >> > > > > >> > > >> > > could
>> > >> > > > > >> > > > > >> > > >> > > > > > > > land in master because it
>> was
>> > >> very
>> > >> > > > well
>> > >> > > > > >> > > isolated
>> > >> > > > > >> > > > > (the
>> > >> > > > > >> > > > > >> > > slide
>> > >> > > > > >> > > > > >> > > >> > away
>> > >> > > > > >> > > > > >> > > >> > > > from
>> > >> > > > > >> > > > > >> > > >> > > > > > > > including docs as a
>> > first-class
>> > >> > part
>> > >> > > > of
>> > >> > > > > >> > > building
>> > >> > > > > >> > > > up
>> > >> > > > > >> > > > > >> that
>> > >> > > > > >> > > > > >> > > >> > > > integration
>> > >> > > > > >> > > > > >> > > >> > > > > > > > has led me to doubt the
>> > wisdom
>> > >> of
>> > >> > > this
>> > >> > > > > >> > > decision).
>> > >> > > > > >> > > > > >> > > >> > > > > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > > > > We've also been treating
>> > >> inclusion
>> > >> > > in
>> > >> > > > a
>> > >> > > > > >> > > "probably
>> > >> > > > > >> > > > > >> will
>> > >> > > > > >> > > > > >> > be
>> > >> > > > > >> > > > > >> > > >> > > released
>> > >> > > > > >> > > > > >> > > >> > > > to
>> > >> > > > > >> > > > > >> > > >> > > > > > > > downstream" branches as a
>> > >> higher
>> > >> > > bar,
>> > >> > > > > >> > requiring
>> > >> > > > > >> > > > > >> > > >> > > > > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > > > > * don't moderately impact
>> > >> > > performance
>> > >> > > > > when
>> > >> > > > > >> > the
>> > >> > > > > >> > > > > >> feature
>> > >> > > > > >> > > > > >> > > isn't
>> > >> > > > > >> > > > > >> > > >> in
>> > >> > > > > >> > > > > >> > > >> > > use
>> > >> > > > > >> > > > > >> > > >> > > > > > > > * don't severely impact
>> > >> > performance
>> > >> > > > when
>> > >> > > > > >> the
>> > >> > > > > >> > > > > feature
>> > >> > > > > >> > > > > >> is
>> > >> > > > > >> > > > > >> > in
>> > >> > > > > >> > > > > >> > > >> use
>> > >> > > > > >> > > > > >> > > >> > > > > > > > * either default-to-on or
>> > show
>> > >> > > enough
>> > >> > > > > >> demand
>> > >> > > > > >> > to
>> > >> > > > > >> > > > > >> believe
>> > >> > > > > >> > > > > >> > a
>> > >> > > > > >> > > > > >> > > >> > > > non-trivial
>> > >> > > > > >> > > > > >> > > >> > > > > > > > number of folks will turn
>> the
>> > >> > > feature
>> > >> > > > on
>> > >> > > > > >> > > > > >> > > >> > > > > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > > > > The above has kept MOB and
>> > >> > > hbase-spark
>> > >> > > > > >> > > > integration
>> > >> > > > > >> > > > > >> out
>> > >> > > > > >> > > > > >> > of
>> > >> > > > > >> > > > > >> > > >> > > branch-1,
>> > >> > > > > >> > > > > >> > > >> > > > > > > > presumably while they've
>> > >> "gotten
>> > >> > > more
>> > >> > > > > >> stable"
>> > >> > > > > >> > > in
>> > >> > > > > >> > > > > >> master
>> > >> > > > > >> > > > > >> > > from
>> > >> > > > > >> > > > > >> > > >> > the
>> > >> > > > > >> > > > > >> > > >> > > > odd
>> > >> > > > > >> > > > > >> > > >> > > > > > > > vendor inclusion.
>> > >> > > > > >> > > > > >> > > >> > > > > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > > > > Are we going to have a 2.0
>> > >> release
>> > >> > > > > before
>> > >> > > > > >> the
>> > >> > > > > >> > > end
>> > >> > > > > >> > > > > of
>> > >> > > > > >> > > > > >> the
>> > >> > > > > >> > > > > >> > > >> year?
>> > >> > > > > >> > > > > >> > > >> > > > We're
>> > >> > > > > >> > > > > >> > > >> > > > > > > > coming up on 1.5 years
>> since
>> > >> the
>> > >> > > > > release of
>> > >> > > > > >> > > > version
>> > >> > > > > >> > > > > >> 1.0;
>> > >> > > > > >> > > > > >> > > >> seems
>> > >> > > > > >> > > > > >> > > >> > > like
>> > >> > > > > >> > > > > >> > > >> > > > > > > > it's about time, though I
>> > >> haven't
>> > >> > > seen
>> > >> > > > > any
>> > >> > > > > >> > > > concrete
>> > >> > > > > >> > > > > >> > plans
>> > >> > > > > >> > > > > >> > > >> this
>> > >> > > > > >> > > > > >> > > >> > > > year.
>> > >> > > > > >> > > > > >> > > >> > > > > > > > Presuming we are going to
>> > have
>> > >> one
>> > >> > > by
>> > >> > > > > the
>> > >> > > > > >> end
>> > >> > > > > >> > > of
>> > >> > > > > >> > > > > the
>> > >> > > > > >> > > > > >> > > year, it
>> > >> > > > > >> > > > > >> > > >> > > > seems a
>> > >> > > > > >> > > > > >> > > >> > > > > > > > bit close to still be
>> adding
>> > in
>> > >> > > > > "features
>> > >> > > > > >> > that
>> > >> > > > > >> > > > need
>> > >> > > > > >> > > > > >> > > maturing"
>> > >> > > > > >> > > > > >> > > >> > on
>> > >> > > > > >> > > > > >> > > >> > > > the
>> > >> > > > > >> > > > > >> > > >> > > > > > > > branch.
>> > >> > > > > >> > > > > >> > > >> > > > > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > > > > The lack of a concrete
>> plan
>> > for
>> > >> > 2.0
>> > >> > > > > keeps
>> > >> > > > > >> me
>> > >> > > > > >> > > from
>> > >> > > > > >> > > > > >> > > considering
>> > >> > > > > >> > > > > >> > > >> > > these
>> > >> > > > > >> > > > > >> > > >> > > > > > > > things blocker at the
>> moment.
>> > >> But
>> > >> > I
>> > >> > > > know
>> > >> > > > > >> > first
>> > >> > > > > >> > > > hand
>> > >> > > > > >> > > > > >> how
>> > >> > > > > >> > > > > >> > > much
>> > >> > > > > >> > > > > >> > > >> > > > trouble
>> > >> > > > > >> > > > > >> > > >> > > > > > > > folks have had with other
>> > >> features
>> > >> > > > that
>> > >> > > > > >> have
>> > >> > > > > >> > > gone
>> > >> > > > > >> > > > > >> into
>> > >> > > > > >> > > > > >> > > >> > downstream
>> > >> > > > > >> > > > > >> > > >> > > > > > > > facing releases without
>> > >> robustness
>> > >> > > > > checks
>> > >> > > > > >> > (i.e.
>> > >> > > > > >> > > > > >> > > replication),
>> > >> > > > > >> > > > > >> > > >> > and
>> > >> > > > > >> > > > > >> > > >> > > > I'm
>> > >> > > > > >> > > > > >> > > >> > > > > > > > concerned about what we're
>> > >> setting
>> > >> > > up
>> > >> > > > if
>> > >> > > > > >> 2.0
>> > >> > > > > >> > > goes
>> > >> > > > > >> > > > > out
>> > >> > > > > >> > > > > >> > with
>> > >> > > > > >> > > > > >> > > >> this
>> > >> > > > > >> > > > > >> > > >> > > > > > > > feature in its current
>> state.
>> > >> > > > > >> > > > > >> > > >> > > > > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > >
>> > >> > > > > >> > > > > >> > > >> > > > >
>> > >> > > > > >> > > > > >> > > >> > > > >
>> > >> > > > > >> > > > > >> > > >> > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > --
>> > >> > > > > >> > > > > >> > > >> > > > > Best regards,
>> > >> > > > > >> > > > > >> > > >> > > > >
>> > >> > > > > >> > > > > >> > > >> > > > >    - Andy
>> > >> > > > > >> > > > > >> > > >> > > > >
>> > >> > > > > >> > > > > >> > > >> > > > > Problems worthy of attack prove
>> > their
>> > >> > > worth
>> > >> > > > by
>> > >> > > > > >> > > hitting
>> > >> > > > > >> > > > > >> back. -
>> > >> > > > > >> > > > > >> > > Piet
>> > >> > > > > >> > > > > >> > > >> > > Hein
>> > >> > > > > >> > > > > >> > > >> > > > > (via Tom White)
>> > >> > > > > >> > > > > >> > > >> > > > >
>> > >> > > > > >> > > > > >> > > >> > > >
>> > >> > > > > >> > > > > >> > > >> > >
>> > >> > > > > >> > > > > >> > > >> > >
>> > >> > > > > >> > > > > >> > > >> > >
>> > >> > > > > >> > > > > >> > > >> > > --
>> > >> > > > > >> > > > > >> > > >> > > Best regards,
>> > >> > > > > >> > > > > >> > > >> > >
>> > >> > > > > >> > > > > >> > > >> > >    - Andy
>> > >> > > > > >> > > > > >> > > >> > >
>> > >> > > > > >> > > > > >> > > >> > > Problems worthy of attack prove
>> their
>> > >> worth
>> > >> > by
>> > >> > > > > >> hitting
>> > >> > > > > >> > > > back.
>> > >> > > > > >> > > > > -
>> > >> > > > > >> > > > > >> > Piet
>> > >> > > > > >> > > > > >> > > >> Hein
>> > >> > > > > >> > > > > >> > > >> > > (via Tom White)
>> > >> > > > > >> > > > > >> > > >> > >
>> > >> > > > > >> > > > > >> > > >> >
>> > >> > > > > >> > > > > >> > > >>
>> > >> > > > > >> > > > > >> > >
>> > >> > > > > >> > > > > >> >
>> > >> > > > > >> > > > > >>
>> > >> > > > > >> > > > > >
>> > >> > > > > >> > > > > >
>> > >> > > > > >> > > > >
>> > >> > > > > >> > > >
>> > >> > > > > >> > >
>> > >> > > > > >> >
>> > >> > > > > >>
>> > >> > > > >
>> > >> > > > >
>> > >> > > > >
>> > >> > > > > --
>> > >> > > > > busbey
>> > >> > > > >
>> > >> > > >
>> > >> > >
>> > >> >
>> > >>
>> > >
>> > >
>> >
>>
>
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Apekshit Sharma <ap...@cloudera.com>]

@stack, it compiled for me.

Also tried few commands, and have to say, it's well designed from user
commands perspective.

On Tue, Oct 11, 2016 at 5:08 PM, Stack <st...@duboce.net> wrote:

> On Tue, Oct 11, 2016 at 1:50 PM, Vladimir Rodionov <vladrodionov@gmail.com
> >
> wrote:
>
> > Michael,
> >
> > Its in HBASE-7912
> >
> > This is tip of git log:
> >
> > commit a072f6f49a26a7259ff2aaef6cb56d85eb592482
> > Author: Frank Welsch <fw...@jps.net>
> > Date:   Fri Sep 23 18:00:42 2016 -0400
> >
> >     HBASE-16574 Book updates for backup and restore
> >
> > commit b14e2ab1c24e65ff88dd4c579acf83cb4ed0605e
> > Author: tedyu <yu...@gmail.com>
> > Date:   Wed Oct 5 16:29:40 2016 -0700
> >
> >     HBASE-16727 Backup refactoring: remove MR dependencies from HMaster
> > (Vladimir Rodionov)
> >
> >
> Thanks. I have that. I tried it and it doesn't compile for me. Does it
> compile for you?
> Thanks,
> M
>
>
>
> > -Vlad
> >
> > On Tue, Oct 11, 2016 at 1:44 PM, Stack <st...@duboce.net> wrote:
> >
> > > Which branch do I check out to try it? HBASE-7912 is not it. I don't
> see
> > an
> > > HBASE-16727...
> > > Thanks,
> > > M
> > >
> > >
> > > On Tue, Oct 11, 2016 at 11:41 AM, Vladimir Rodionov <
> > > vladrodionov@gmail.com>
> > > wrote:
> > >
> > > > The last patch is on review board:
> > > > https://reviews.apache.org/r/52748
> > > >
> > > > On Mon, Oct 10, 2016 at 3:52 PM, Vladimir Rodionov <
> > > vladrodionov@gmail.com
> > > > >
> > > > wrote:
> > > >
> > > > > >> How hard to put in an hbase-backup module? hbase-server is fat
> > > enough
> > > > > >> already. Could be done as a follow-up.
> > > > >
> > > > > https://issues.apache.org/jira/browse/HBASE-16727?
> > > > > focusedCommentId=15531237&page=com.atlassian.jira.
> > > > > plugin.system.issuetabpanels:comment-tabpanel#comment-15531237
> > > > >
> > > > > Can we do merge first? Then we can discuss separate module.
> > > > >
> > > > > -Vlad
> > > > >
> > > > > On Mon, Oct 10, 2016 at 3:44 PM, Ted Yu <yu...@gmail.com>
> wrote:
> > > > >
> > > > >> Looks like the first quote was cut off.
> > > > >> The original sentence was:
> > > > >>
> > > > >> bq. no mapreduce job launched from master or region server.
> > > > >>
> > > > >> mapreduce job is launched from the node where command line tool is
> > > run.
> > > > >>
> > > > >> On Mon, Oct 10, 2016 at 3:38 PM, Stack <st...@duboce.net> wrote:
> > > > >>
> > > > >> > bq. launched from master or region server.
> > > > >> >
> > > > >> > What does this mean please? Has to be run from Master or
> > > RegionServer?
> > > > >> Can
> > > > >> > it be run from another node altogether?
> > > > >> >
> > > > >> > On Mon, Oct 10, 2016 at 1:44 PM, Vladimir Rodionov <
> > > > >> vladrodionov@gmail.com
> > > > >> > >
> > > > >> > wrote:
> > > > >> >
> > > > >> > > >> mapreduce dependency has been moved to client side - no
> > > mapreduce
> > > > >> job
> > > > >> > >
> > > > >> > > 1. We have no code in the client module anymore, due to
> > dependency
> > > > on
> > > > >> > > internal server API (HFile and WAL access).
> > > > >> > > 2. Backup/ restore are client - driven operations, but all the
> > > code
> > > > >> > resides
> > > > >> > > in the server module
> > > > >> > >
> > > > >> >
> > > > >> > How hard to put in an hbase-backup module? hbase-server is fat
> > > enough
> > > > >> > already. Could be done as a follow-up.
> > > > >> >
> > > > >> > Thanks,
> > > > >> > St.Ack
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >> > > 3. No MR in Master, no procedure - driven execution.
> > > > >> > > 4. Old good MR from command-line.
> > > > >> > > 5. Security was simplified and now only super-user is allowed
> to
> > > run
> > > > >> > > backup/restores.
> > > > >> > > 6. HBase Backup API was gone due to 1. Now only command-line
> > > access
> > > > to
> > > > >> > > backup tools.
> > > > >> > >
> > > > >> > > These consequences of refactoring has been discussed in
> > > HBASE-16727.
> > > > >> > >
> > > > >> > > -Vlad
> > > > >> > >
> > > > >> > >
> > > > >> > >
> > > > >> > >
> > > > >> > > On Mon, Oct 10, 2016 at 1:31 PM, Ted Yu <yu...@gmail.com>
> > > > wrote:
> > > > >> > >
> > > > >> > > > Reviving this thread.
> > > > >> > > >
> > > > >> > > > The following has taken place:
> > > > >> > > >
> > > > >> > > > mapreduce dependency has been moved to client side - no
> > > mapreduce
> > > > >> job
> > > > >> > > > launched from master or region server.
> > > > >> > > > document patch (HBASE-16574) has been integrated.
> > > > >> > > > Updated mega patch has been attached to HBASE-14123: this
> > covers
> > > > the
> > > > >> > > > refactor in #1 above and the protobuf 3 merge.
> > > > >> > > >
> > > > >> > > > If community has more feedback on the merge proposal, I
> would
> > > love
> > > > >> to
> > > > >> > > hear
> > > > >> > > > it.
> > > > >> > > >
> > > > >> > > > Thanks
> > > > >> > > >
> > > > >> > > > On Thu, Sep 22, 2016 at 10:31 AM, Sean Busbey <
> > > > busbey@cloudera.com>
> > > > >> > > wrote:
> > > > >> > > >
> > > > >> > > > > I'd like to see the docs proposed on HBASE-16574
> integrated
> > > into
> > > > >> our
> > > > >> > > > > project's documentation prior to merge.
> > > > >> > > > >
> > > > >> > > > > On Thu, Sep 22, 2016 at 9:02 AM, Ted Yu <
> > yuzhihong@gmail.com>
> > > > >> wrote:
> > > > >> > > > > > This feature can be marked experimental due to some
> > > > limitations
> > > > >> > such
> > > > >> > > as
> > > > >> > > > > > security.
> > > > >> > > > > >
> > > > >> > > > > > Your previous round of comments have been addressed.
> > > > >> > > > > > Command line tool has gone through:
> > > > >> > > > > >
> > > > >> > > > > > HBASE-16620 Fix backup command-line tool usability
> issues
> > > > >> > > > > > HBASE-16655 hbase backup describe with incorrect backup
> id
> > > > >> results
> > > > >> > in
> > > > >> > > > NPE
> > > > >> > > > > >
> > > > >> > > > > > The updated doc has been attached to HBASE-16574.
> > > > >> > > > > >
> > > > >> > > > > > Cheers
> > > > >> > > > > >
> > > > >> > > > > > On Thu, Sep 22, 2016 at 8:53 AM, Stack <
> stack@duboce.net>
> > > > >> wrote:
> > > > >> > > > > >
> > > > >> > > > > >> On Wed, Sep 21, 2016 at 7:43 AM, Ted Yu <
> > > yuzhihong@gmail.com
> > > > >
> > > > >> > > wrote:
> > > > >> > > > > >>
> > > > >> > > > > >> > Are there more (review) comments ?
> > > > >> > > > > >> >
> > > > >> > > > > >> >
> > > > >> > > > > >> Are outstanding comments addressed?
> > > > >> > > > > >>
> > > > >> > > > > >> I don't see answer to my 'is this experimental/will it
> be
> > > > >> marked
> > > > >> > > > > >> experimental' question.
> > > > >> > > > > >>
> > > > >> > > > > >> I ran into some issues trying to use the feature and
> > > > suggested
> > > > >> > that
> > > > >> > > a
> > > > >> > > > > >> feature likes this needs polish else it'll just rot,
> > > unused.
> > > > >> Has
> > > > >> > > > polish
> > > > >> > > > > >> been applied? All ready for another 'user' test?
> Suggest
> > > that
> > > > >> you
> > > > >> > > > update
> > > > >> > > > > >> here going forward for the benefit of those trying to
> > > follow
> > > > >> along
> > > > >> > > and
> > > > >> > > > > who
> > > > >> > > > > >> are not watching JIRA change fly-by.
> > > > >> > > > > >>
> > > > >> > > > > >> It looks like doc got a revision -- I have to check --
> to
> > > > take
> > > > >> on
> > > > >> > > > > >> suggestion made above but again, suggest, that this
> > thread
> > > > gets
> > > > >> > > > updated.
> > > > >> > > > > >>
> > > > >> > > > > >> Thanks,
> > > > >> > > > > >> St.Ack
> > > > >> > > > > >>
> > > > >> > > > > >>
> > > > >> > > > > >>
> > > > >> > > > > >> > Thanks
> > > > >> > > > > >> >
> > > > >> > > > > >> > On Tue, Sep 20, 2016 at 10:02 AM, Devaraj Das <
> > > > >> > > ddas@hortonworks.com
> > > > >> > > > >
> > > > >> > > > > >> > wrote:
> > > > >> > > > > >> >
> > > > >> > > > > >> > > Just reviving this thread. Thanks Sean, Stack,
> Dima,
> > > and
> > > > >> > others
> > > > >> > > > for
> > > > >> > > > > the
> > > > >> > > > > >> > > thorough reviews and testing. Thanks Ted and Vlad
> for
> > > > >> taking
> > > > >> > > care
> > > > >> > > > of
> > > > >> > > > > >> the
> > > > >> > > > > >> > > feedback. Are we all good to do the merge now?
> Rather
> > > do
> > > > >> > sooner
> > > > >> > > > than
> > > > >> > > > > >> > later.
> > > > >> > > > > >> > > ________________________________________
> > > > >> > > > > >> > > From: saint.ack@gmail.com <sa...@gmail.com> on
> > > > behalf
> > > > >> of
> > > > >> > > > Stack
> > > > >> > > > > <
> > > > >> > > > > >> > > stack@duboce.net>
> > > > >> > > > > >> > > Sent: Monday, September 12, 2016 1:18 PM
> > > > >> > > > > >> > > To: HBase Dev List
> > > > >> > > > > >> > > Subject: Re: [DISCUSSION] Merge Backup / Restore -
> > > Branch
> > > > >> > > > HBASE-7912
> > > > >> > > > > >> > >
> > > > >> > > > > >> > > On Mon, Sep 12, 2016 at 12:19 PM, Ted Yu <
> > > > >> yuzhihong@gmail.com
> > > > >> > >
> > > > >> > > > > wrote:
> > > > >> > > > > >> > >
> > > > >> > > > > >> > > > Mega patch (rev 18) is on HBASE-14123.
> > > > >> > > > > >> > > >
> > > > >> > > > > >> > > > Please comment on HBASE-14123 on how you want to
> > > > review.
> > > > >> > > > > >> > > >
> > > > >> > > > > >> > >
> > > > >> > > > > >> > >
> > > > >> > > > > >> > > Yeah. That was my lost tab. Last rb was 6 months
> ago.
> > > > >> Suggest
> > > > >> > > > > updating
> > > > >> > > > > >> > it.
> > > > >> > > > > >> > > RB is pretty good for review. Patch is only 1.5M so
> > > > should
> > > > >> be
> > > > >> > > > fine.
> > > > >> > > > > >> > >
> > > > >> > > > > >> > > St.Ack
> > > > >> > > > > >> > >
> > > > >> > > > > >> > >
> > > > >> > > > > >> > > >
> > > > >> > > > > >> > > > Thanks
> > > > >> > > > > >> > > >
> > > > >> > > > > >> > > > On Mon, Sep 12, 2016 at 12:15 PM, Stack <
> > > > >> stack@duboce.net>
> > > > >> > > > wrote:
> > > > >> > > > > >> > > >
> > > > >> > > > > >> > > > > On review of the 'patch', do I just compare the
> > > > branch
> > > > >> to
> > > > >> > > > > master or
> > > > >> > > > > >> > is
> > > > >> > > > > >> > > > > there a megapatch posted somewhere (I think I
> saw
> > > one
> > > > >> but
> > > > >> > it
> > > > >> > > > > seemed
> > > > >> > > > > >> > > stale
> > > > >> > > > > >> > > > > and then I 'lost' the tab). Sorry for dumb
> > > question.
> > > > >> > > > > >> > > > > St.Ack
> > > > >> > > > > >> > > > >
> > > > >> > > > > >> > > > > On Mon, Sep 12, 2016 at 12:01 PM, Stack <
> > > > >> stack@duboce.net
> > > > >> > >
> > > > >> > > > > wrote:
> > > > >> > > > > >> > > > >
> > > > >> > > > > >> > > > > > Late to the game. A few comments after
> > rereading
> > > > this
> > > > >> > > thread
> > > > >> > > > > as a
> > > > >> > > > > >> > > > 'user'.
> > > > >> > > > > >> > > > > >
> > > > >> > > > > >> > > > > > + Before merge, a user-facing feature like
> this
> > > > >> should
> > > > >> > > work
> > > > >> > > > > (If
> > > > >> > > > > >> > this
> > > > >> > > > > >> > > is
> > > > >> > > > > >> > > > > "higher-bar
> > > > >> > > > > >> > > > > > for new features", bring it on -- smile).
> > > > >> > > > > >> > > > > > + As a user, I tried the branch with tools
> > after
> > > > >> > reviewing
> > > > >> > > > the
> > > > >> > > > > >> > > > > just-posted
> > > > >> > > > > >> > > > > > doc. I had an 'interesting' experience (left
> > > > >> comments up
> > > > >> > > on
> > > > >> > > > > >> > issue). I
> > > > >> > > > > >> > > > > think
> > > > >> > > > > >> > > > > > the tooling/doc. important to get right. If
> it
> > > > breaks
> > > > >> > > easily
> > > > >> > > > > or
> > > > >> > > > > >> is
> > > > >> > > > > >> > > > > > inconsistent (or lacks 'polish'), operators
> > will
> > > > >> judge
> > > > >> > the
> > > > >> > > > > whole
> > > > >> > > > > >> > > > > > backup/restore tooling chain as not
> trustworthy
> > > and
> > > > >> > > abandon
> > > > >> > > > > it.
> > > > >> > > > > >> > Lets
> > > > >> > > > > >> > > > not
> > > > >> > > > > >> > > > > > have this happen to this feature.
> > > > >> > > > > >> > > > > > + Matteo's suggestion (with a helpful starter
> > > list)
> > > > >> that
> > > > >> > > > there
> > > > >> > > > > >> > needs
> > > > >> > > > > >> > > to
> > > > >> > > > > >> > > > > be
> > > > >> > > > > >> > > > > > explicit qualification on what is actually
> > being
> > > > >> > delivered
> > > > >> > > > --
> > > > >> > > > > >> > > > including a
> > > > >> > > > > >> > > > > > listing of limitations (some look serious
> such
> > as
> > > > >> data
> > > > >> > > bleed
> > > > >> > > > > from
> > > > >> > > > > >> > > other
> > > > >> > > > > >> > > > > > regions in WALs, but maybe I don't care for
> my
> > > use
> > > > >> > > case...)
> > > > >> > > > --
> > > > >> > > > > >> > needs
> > > > >> > > > > >> > > to
> > > > >> > > > > >> > > > > > accompany the merge. Lets fold them into the
> > user
> > > > >> doc.
> > > > >> > in
> > > > >> > > > the
> > > > >> > > > > >> > > technical
> > > > >> > > > > >> > > > > > overview area as suggested so user
> expectations
> > > are
> > > > >> > > properly
> > > > >> > > > > >> > managed
> > > > >> > > > > >> > > > > > (otherwise, they expect the world and will
> just
> > > > give
> > > > >> up
> > > > >> > > when
> > > > >> > > > > we
> > > > >> > > > > >> > fall
> > > > >> > > > > >> > > > > > short). Vladimir did a list of what is in
> each
> > of
> > > > the
> > > > >> > > phases
> > > > >> > > > > >> above
> > > > >> > > > > >> > > > which
> > > > >> > > > > >> > > > > > would serve as a good start.
> > > > >> > > > > >> > > > > > + Is this feature 'experimental' (Matteo asks
> > > > above).
> > > > >> > I'd
> > > > >> > > > > prefer
> > > > >> > > > > >> it
> > > > >> > > > > >> > > is
> > > > >> > > > > >> > > > > > not. If it is, it should be labelled all over
> > > that
> > > > >> it is
> > > > >> > > > so. I
> > > > >> > > > > >> see
> > > > >> > > > > >> > > > > current
> > > > >> > > > > >> > > > > > state called out as a '... technical preview
> > > > >> feature'.
> > > > >> > > Does
> > > > >> > > > > this
> > > > >> > > > > >> > mean
> > > > >> > > > > >> > > > > > not-for-users?
> > > > >> > > > > >> > > > > >
> > > > >> > > > > >> > > > > > St.Ack
> > > > >> > > > > >> > > > > >
> > > > >> > > > > >> > > > > >
> > > > >> > > > > >> > > > > >
> > > > >> > > > > >> > > > > >
> > > > >> > > > > >> > > > > >
> > > > >> > > > > >> > > > > >
> > > > >> > > > > >> > > > > >
> > > > >> > > > > >> > > > > >
> > > > >> > > > > >> > > > > >
> > > > >> > > > > >> > > > > >
> > > > >> > > > > >> > > > > >
> > > > >> > > > > >> > > > > > On Mon, Sep 12, 2016 at 8:03 AM, Ted Yu <
> > > > >> > > > yuzhihong@gmail.com>
> > > > >> > > > > >> > wrote:
> > > > >> > > > > >> > > > > >
> > > > >> > > > > >> > > > > >> Sean:
> > > > >> > > > > >> > > > > >> Do you have more comments ?
> > > > >> > > > > >> > > > > >>
> > > > >> > > > > >> > > > > >> Cheers
> > > > >> > > > > >> > > > > >>
> > > > >> > > > > >> > > > > >> On Fri, Sep 9, 2016 at 1:42 PM, Vladimir
> > > Rodionov
> > > > <
> > > > >> > > > > >> > > > > vladrodionov@gmail.com
> > > > >> > > > > >> > > > > >> >
> > > > >> > > > > >> > > > > >> wrote:
> > > > >> > > > > >> > > > > >>
> > > > >> > > > > >> > > > > >> > Sean,
> > > > >> > > > > >> > > > > >> >
> > > > >> > > > > >> > > > > >> > Backup/Restore can fail due to various
> > > reasons:
> > > > >> > network
> > > > >> > > > > outage
> > > > >> > > > > >> > > > > (cluster
> > > > >> > > > > >> > > > > >> > wide), various time-outs in HBase and HDFS
> > > > layer,
> > > > >> M/R
> > > > >> > > > > failure
> > > > >> > > > > >> > due
> > > > >> > > > > >> > > to
> > > > >> > > > > >> > > > > >> "HDFS
> > > > >> > > > > >> > > > > >> > exceeded quota", user error (manual
> deletion
> > > of
> > > > >> data)
> > > > >> > > and
> > > > >> > > > > so
> > > > >> > > > > >> on
> > > > >> > > > > >> > so
> > > > >> > > > > >> > > > on.
> > > > >> > > > > >> > > > > >> That
> > > > >> > > > > >> > > > > >> > is impossible to enumerate all possible
> > types
> > > of
> > > > >> > > failures
> > > > >> > > > > in a
> > > > >> > > > > >> > > > > >> distributed
> > > > >> > > > > >> > > > > >> > system - that is not our goal/task.
> > > > >> > > > > >> > > > > >> >
> > > > >> > > > > >> > > > > >> > We focus completely on backup system table
> > > > >> > consistency
> > > > >> > > > in a
> > > > >> > > > > >> > > presence
> > > > >> > > > > >> > > > > of
> > > > >> > > > > >> > > > > >> any
> > > > >> > > > > >> > > > > >> > type of failure. That is what I call
> > > "tolerance
> > > > to
> > > > >> > > > > failures".
> > > > >> > > > > >> > > > > >> >
> > > > >> > > > > >> > > > > >> > On a failure:
> > > > >> > > > > >> > > > > >> >
> > > > >> > > > > >> > > > > >> > BACKUP. All backup system information
> (prior
> > > to
> > > > >> > backup)
> > > > >> > > > > will
> > > > >> > > > > >> be
> > > > >> > > > > >> > > > > restored
> > > > >> > > > > >> > > > > >> > and all temporary data, related to a
> failed
> > > > >> session,
> > > > >> > in
> > > > >> > > > > HDFS
> > > > >> > > > > >> > will
> > > > >> > > > > >> > > be
> > > > >> > > > > >> > > > > >> > deleted
> > > > >> > > > > >> > > > > >> > RESTORE. We do not care about system data,
> > > > because
> > > > >> > > > restore
> > > > >> > > > > >> does
> > > > >> > > > > >> > > not
> > > > >> > > > > >> > > > > >> change
> > > > >> > > > > >> > > > > >> > it. Temporary data in HDFS will be cleaned
> > up
> > > > and
> > > > >> > table
> > > > >> > > > > will
> > > > >> > > > > >> be
> > > > >> > > > > >> > > in a
> > > > >> > > > > >> > > > > >> state
> > > > >> > > > > >> > > > > >> > back to where it was before operation
> > started.
> > > > >> > > > > >> > > > > >> >
> > > > >> > > > > >> > > > > >> > This is what user should expect in case
> of a
> > > > >> failure.
> > > > >> > > > > >> > > > > >> >
> > > > >> > > > > >> > > > > >> > -Vlad
> > > > >> > > > > >> > > > > >> >
> > > > >> > > > > >> > > > > >> >
> > > > >> > > > > >> > > > > >> > -Vlad
> > > > >> > > > > >> > > > > >> >
> > > > >> > > > > >> > > > > >> > On Fri, Sep 9, 2016 at 12:56 PM, Sean
> > Busbey <
> > > > >> > > > > >> busbey@apache.org
> > > > >> > > > > >> > >
> > > > >> > > > > >> > > > > wrote:
> > > > >> > > > > >> > > > > >> >
> > > > >> > > > > >> > > > > >> > > Failing in a consistent way, with docs
> > that
> > > > >> explain
> > > > >> > > the
> > > > >> > > > > >> > various
> > > > >> > > > > >> > > > > >> > > expected failures would be sufficient.
> > > > >> > > > > >> > > > > >> > >
> > > > >> > > > > >> > > > > >> > > On Fri, Sep 9, 2016 at 12:16 PM,
> Vladimir
> > > > >> Rodionov
> > > > >> > > > > >> > > > > >> > > <vl...@gmail.com> wrote:
> > > > >> > > > > >> > > > > >> > > > Do not worry Sean, doc is coming today
> > as
> > > a
> > > > >> > preview
> > > > >> > > > and
> > > > >> > > > > >> our
> > > > >> > > > > >> > > > writer
> > > > >> > > > > >> > > > > >> > Frank
> > > > >> > > > > >> > > > > >> > > > will be working on a putting  it into
> > > Apache
> > > > >> > repo.
> > > > >> > > > > >> Timeline
> > > > >> > > > > >> > > > > depends
> > > > >> > > > > >> > > > > >> on
> > > > >> > > > > >> > > > > >> > > > Franks schedule but I hope we will get
> > it
> > > > >> rather
> > > > >> > > > sooner
> > > > >> > > > > >> than
> > > > >> > > > > >> > > > > later.
> > > > >> > > > > >> > > > > >> > > >
> > > > >> > > > > >> > > > > >> > > > As for failure testing, we are
> focusing
> > > only
> > > > >> on a
> > > > >> > > > > >> consistent
> > > > >> > > > > >> > > > state
> > > > >> > > > > >> > > > > >> of
> > > > >> > > > > >> > > > > >> > > > backup system data in a presence of
> any
> > > type
> > > > >> of
> > > > >> > > > > failures,
> > > > >> > > > > >> We
> > > > >> > > > > >> > > are
> > > > >> > > > > >> > > > > not
> > > > >> > > > > >> > > > > >> > > going
> > > > >> > > > > >> > > > > >> > > > to implement  anything more "fancy",
> > than
> > > > >> that.
> > > > >> > We
> > > > >> > > > > allow
> > > > >> > > > > >> > both:
> > > > >> > > > > >> > > > > >> backup
> > > > >> > > > > >> > > > > >> > and
> > > > >> > > > > >> > > > > >> > > > restore to fail. What we do not allow
> is
> > > to
> > > > >> have
> > > > >> > > > system
> > > > >> > > > > >> data
> > > > >> > > > > >> > > > > >> corrupted.
> > > > >> > > > > >> > > > > >> > > > Will it suffice for you? Do you have
> any
> > > > other
> > > > >> > > > > concerns,
> > > > >> > > > > >> you
> > > > >> > > > > >> > > > want
> > > > >> > > > > >> > > > > >> us to
> > > > >> > > > > >> > > > > >> > > > address?
> > > > >> > > > > >> > > > > >> > > >
> > > > >> > > > > >> > > > > >> > > > -Vlad
> > > > >> > > > > >> > > > > >> > > >
> > > > >> > > > > >> > > > > >> > > >
> > > > >> > > > > >> > > > > >> > > > On Fri, Sep 9, 2016 at 10:56 AM, Sean
> > > > Busbey <
> > > > >> > > > > >> > > busbey@apache.org
> > > > >> > > > > >> > > > >
> > > > >> > > > > >> > > > > >> > wrote:
> > > > >> > > > > >> > > > > >> > > >
> > > > >> > > > > >> > > > > >> > > >> "docs will come to Apache soon" does
> > not
> > > > >> address
> > > > >> > > my
> > > > >> > > > > >> concern
> > > > >> > > > > >> > > > > around
> > > > >> > > > > >> > > > > >> > docs
> > > > >> > > > > >> > > > > >> > > at
> > > > >> > > > > >> > > > > >> > > >> all, unless said docs have already
> made
> > > it
> > > > >> into
> > > > >> > > the
> > > > >> > > > > >> project
> > > > >> > > > > >> > > > > repo. I
> > > > >> > > > > >> > > > > >> > > don't
> > > > >> > > > > >> > > > > >> > > >> want third party resources for using
> a
> > > > major
> > > > >> and
> > > > >> > > > > >> important
> > > > >> > > > > >> > > > > feature
> > > > >> > > > > >> > > > > >> of
> > > > >> > > > > >> > > > > >> > > the
> > > > >> > > > > >> > > > > >> > > >> project, I want us to provide end
> users
> > > > with
> > > > >> > what
> > > > >> > > > they
> > > > >> > > > > >> need
> > > > >> > > > > >> > > to
> > > > >> > > > > >> > > > > get
> > > > >> > > > > >> > > > > >> the
> > > > >> > > > > >> > > > > >> > > job
> > > > >> > > > > >> > > > > >> > > >> done.
> > > > >> > > > > >> > > > > >> > > >>
> > > > >> > > > > >> > > > > >> > > >> I see some calls for patience on the
> > > > failure
> > > > >> > > > testing,
> > > > >> > > > > but
> > > > >> > > > > >> > the
> > > > >> > > > > >> > > > > >> appeal
> > > > >> > > > > >> > > > > >> > to
> > > > >> > > > > >> > > > > >> > > us
> > > > >> > > > > >> > > > > >> > > >> having done a bad job of requiring
> > proper
> > > > >> tests
> > > > >> > of
> > > > >> > > > > >> previous
> > > > >> > > > > >> > > > > >> features
> > > > >> > > > > >> > > > > >> > > just
> > > > >> > > > > >> > > > > >> > > >> makes me more concerned about not
> > getting
> > > > >> them
> > > > >> > > > here. I
> > > > >> > > > > >> > don't
> > > > >> > > > > >> > > > want
> > > > >> > > > > >> > > > > >> to
> > > > >> > > > > >> > > > > >> > set
> > > > >> > > > > >> > > > > >> > > >> yet another bad example that will
> then
> > be
> > > > >> > pointed
> > > > >> > > to
> > > > >> > > > > in
> > > > >> > > > > >> the
> > > > >> > > > > >> > > > > future.
> > > > >> > > > > >> > > > > >> > > >>
> > > > >> > > > > >> > > > > >> > > >> On Sep 8, 2016 10:50, "Ted Yu" <
> > > > >> > > yuzhihong@gmail.com
> > > > >> > > > >
> > > > >> > > > > >> > wrote:
> > > > >> > > > > >> > > > > >> > > >>
> > > > >> > > > > >> > > > > >> > > >> > Is there any concern which is not
> > > > >> addressed ?
> > > > >> > > > > >> > > > > >> > > >> >
> > > > >> > > > > >> > > > > >> > > >> > Do we need another Vote thread ?
> > > > >> > > > > >> > > > > >> > > >> >
> > > > >> > > > > >> > > > > >> > > >> > Thanks
> > > > >> > > > > >> > > > > >> > > >> >
> > > > >> > > > > >> > > > > >> > > >> > On Thu, Sep 8, 2016 at 9:21 AM,
> > Andrew
> > > > >> > Purtell <
> > > > >> > > > > >> > > > > >> apurtell@apache.org
> > > > >> > > > > >> > > > > >> > >
> > > > >> > > > > >> > > > > >> > > >> > wrote:
> > > > >> > > > > >> > > > > >> > > >> >
> > > > >> > > > > >> > > > > >> > > >> > > Vlad,
> > > > >> > > > > >> > > > > >> > > >> > >
> > > > >> > > > > >> > > > > >> > > >> > > I apologize for using the term
> > > > >> 'half-baked'
> > > > >> > > in a
> > > > >> > > > > way
> > > > >> > > > > >> > that
> > > > >> > > > > >> > > > > could
> > > > >> > > > > >> > > > > >> > > seem a
> > > > >> > > > > >> > > > > >> > > >> > > description of HBASE-7912. I
> meant
> > > that
> > > > >> as a
> > > > >> > > > > general
> > > > >> > > > > >> > > > > >> hypothetical.
> > > > >> > > > > >> > > > > >> > > >> > >
> > > > >> > > > > >> > > > > >> > > >> > > On Wed, Sep 7, 2016 at 9:36 AM,
> > > > Vladimir
> > > > >> > > > Rodionov
> > > > >> > > > > <
> > > > >> > > > > >> > > > > >> > > >> > vladrodionov@gmail.com>
> > > > >> > > > > >> > > > > >> > > >> > > wrote:
> > > > >> > > > > >> > > > > >> > > >> > >
> > > > >> > > > > >> > > > > >> > > >> > > > >> I'm not sure that "There is
> > > > already
> > > > >> > lots
> > > > >> > > of
> > > > >> > > > > >> > > half-baked
> > > > >> > > > > >> > > > > >> code
> > > > >> > > > > >> > > > > >> > in
> > > > >> > > > > >> > > > > >> > > the
> > > > >> > > > > >> > > > > >> > > >> > > > branch,
> > > > >> > > > > >> > > > > >> > > >> > > > so what's the harm in adding
> > more?"
> > > > >> > > > > >> > > > > >> > > >> > > >
> > > > >> > > > > >> > > > > >> > > >> > > > I meant - not production -
> ready
> > > yet.
> > > > >> This
> > > > >> > > is
> > > > >> > > > > 2.0
> > > > >> > > > > >> > > > > development
> > > > >> > > > > >> > > > > >> > > branch
> > > > >> > > > > >> > > > > >> > > >> > and,
> > > > >> > > > > >> > > > > >> > > >> > > > hence many features are in
> works,
> > > > >> > > > > >> > > > > >> > > >> > > > not being tested well etc. I do
> > not
> > > > >> > consider
> > > > >> > > > > backup
> > > > >> > > > > >> > as
> > > > >> > > > > >> > > > half
> > > > >> > > > > >> > > > > >> > baked
> > > > >> > > > > >> > > > > >> > > >> > > feature -
> > > > >> > > > > >> > > > > >> > > >> > > > it has passed our internal QA
> and
> > > has
> > > > >> very
> > > > >> > > > good
> > > > >> > > > > >> doc,
> > > > >> > > > > >> > > > which
> > > > >> > > > > >> > > > > we
> > > > >> > > > > >> > > > > >> > will
> > > > >> > > > > >> > > > > >> > > >> > > provide
> > > > >> > > > > >> > > > > >> > > >> > > > to Apache shortly.
> > > > >> > > > > >> > > > > >> > > >> > > >
> > > > >> > > > > >> > > > > >> > > >> > > > -Vlad
> > > > >> > > > > >> > > > > >> > > >> > > >
> > > > >> > > > > >> > > > > >> > > >> > > > On Wed, Sep 7, 2016 at 9:13 AM,
> > > > Andrew
> > > > >> > > > Purtell <
> > > > >> > > > > >> > > > > >> > > apurtell@apache.org>
> > > > >> > > > > >> > > > > >> > > >> > > > wrote:
> > > > >> > > > > >> > > > > >> > > >> > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > We shouldn't admit half baked
> > > > changes
> > > > >> > that
> > > > >> > > > > won't
> > > > >> > > > > >> be
> > > > >> > > > > >> > > > > >> finished.
> > > > >> > > > > >> > > > > >> > > >> However
> > > > >> > > > > >> > > > > >> > > >> > > in
> > > > >> > > > > >> > > > > >> > > >> > > > > this case the crew working on
> > > this
> > > > >> > feature
> > > > >> > > > are
> > > > >> > > > > >> long
> > > > >> > > > > >> > > > > timers
> > > > >> > > > > >> > > > > >> and
> > > > >> > > > > >> > > > > >> > > less
> > > > >> > > > > >> > > > > >> > > >> > > > likely
> > > > >> > > > > >> > > > > >> > > >> > > > > than just about anyone to
> leave
> > > > >> > something
> > > > >> > > > in a
> > > > >> > > > > >> half
> > > > >> > > > > >> > > > baked
> > > > >> > > > > >> > > > > >> > > state. Of
> > > > >> > > > > >> > > > > >> > > >> > > > course
> > > > >> > > > > >> > > > > >> > > >> > > > > there is no guarantee how
> > > anything
> > > > >> will
> > > > >> > > turn
> > > > >> > > > > out,
> > > > >> > > > > >> > > but I
> > > > >> > > > > >> > > > > am
> > > > >> > > > > >> > > > > >> > > willing
> > > > >> > > > > >> > > > > >> > > >> to
> > > > >> > > > > >> > > > > >> > > >> > > > take
> > > > >> > > > > >> > > > > >> > > >> > > > > a little on faith if they
> feel
> > > > their
> > > > >> > best
> > > > >> > > > path
> > > > >> > > > > >> > > forward
> > > > >> > > > > >> > > > > now
> > > > >> > > > > >> > > > > >> is
> > > > >> > > > > >> > > > > >> > to
> > > > >> > > > > >> > > > > >> > > >> > merge
> > > > >> > > > > >> > > > > >> > > >> > > to
> > > > >> > > > > >> > > > > >> > > >> > > > > trunk. I only wish I had
> > > bandwidth
> > > > to
> > > > >> > have
> > > > >> > > > > done
> > > > >> > > > > >> > some
> > > > >> > > > > >> > > > real
> > > > >> > > > > >> > > > > >> > > kicking
> > > > >> > > > > >> > > > > >> > > >> of
> > > > >> > > > > >> > > > > >> > > >> > > the
> > > > >> > > > > >> > > > > >> > > >> > > > > tires by now. Maybe this
> week.
> > > > >> > > > > >> > > > > >> > > >> > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > (Yes, I'm using some of that
> > time
> > > > for
> > > > >> > this
> > > > >> > > > > email
> > > > >> > > > > >> > :-)
> > > > >> > > > > >> > > > but
> > > > >> > > > > >> > > > > I
> > > > >> > > > > >> > > > > >> > type
> > > > >> > > > > >> > > > > >> > > >> > fast.)
> > > > >> > > > > >> > > > > >> > > >> > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > That said, I would like to
> > > agitate
> > > > >> for
> > > > >> > > > making
> > > > >> > > > > 2.0
> > > > >> > > > > >> > > more
> > > > >> > > > > >> > > > > real
> > > > >> > > > > >> > > > > >> > and
> > > > >> > > > > >> > > > > >> > > >> spend
> > > > >> > > > > >> > > > > >> > > >> > > > some
> > > > >> > > > > >> > > > > >> > > >> > > > > time on it now that I'm
> winding
> > > > down
> > > > >> > with
> > > > >> > > > > 0.98. I
> > > > >> > > > > >> > > think
> > > > >> > > > > >> > > > > >> that
> > > > >> > > > > >> > > > > >> > > means
> > > > >> > > > > >> > > > > >> > > >> > > > > branching for 2.0 real soon
> now
> > > and
> > > > >> even
> > > > >> > > > > evicting
> > > > >> > > > > >> > > > things
> > > > >> > > > > >> > > > > >> from
> > > > >> > > > > >> > > > > >> > > 2.0
> > > > >> > > > > >> > > > > >> > > >> > > branch
> > > > >> > > > > >> > > > > >> > > >> > > > > that aren't finished or
> stable,
> > > > >> leaving
> > > > >> > > them
> > > > >> > > > > only
> > > > >> > > > > >> > > once
> > > > >> > > > > >> > > > > >> again
> > > > >> > > > > >> > > > > >> > in
> > > > >> > > > > >> > > > > >> > > the
> > > > >> > > > > >> > > > > >> > > >> > > > master
> > > > >> > > > > >> > > > > >> > > >> > > > > branch. Or, maybe just
> evicting
> > > > them.
> > > > >> > > Let's
> > > > >> > > > > take
> > > > >> > > > > >> it
> > > > >> > > > > >> > > > case
> > > > >> > > > > >> > > > > by
> > > > >> > > > > >> > > > > >> > > case.
> > > > >> > > > > >> > > > > >> > > >> > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > I think this feature can come
> > in
> > > > >> > > relatively
> > > > >> > > > > >> safely.
> > > > >> > > > > >> > > As
> > > > >> > > > > >> > > > > >> added
> > > > >> > > > > >> > > > > >> > > >> > insurance,
> > > > >> > > > > >> > > > > >> > > >> > > > > let's admit the possibility
> it
> > > > could
> > > > >> be
> > > > >> > > > > reverted
> > > > >> > > > > >> on
> > > > >> > > > > >> > > the
> > > > >> > > > > >> > > > > 2.0
> > > > >> > > > > >> > > > > >> > > branch
> > > > >> > > > > >> > > > > >> > > >> if
> > > > >> > > > > >> > > > > >> > > >> > > > folks
> > > > >> > > > > >> > > > > >> > > >> > > > > working on stabilizing 2.0
> > decide
> > > > to
> > > > >> > evict
> > > > >> > > > it
> > > > >> > > > > >> > because
> > > > >> > > > > >> > > > it
> > > > >> > > > > >> > > > > is
> > > > >> > > > > >> > > > > >> > > >> > unfinished
> > > > >> > > > > >> > > > > >> > > >> > > or
> > > > >> > > > > >> > > > > >> > > >> > > > > unstable, because that
> > certainly
> > > > can
> > > > >> > > > happen. I
> > > > >> > > > > >> > would
> > > > >> > > > > >> > > > > >> expect if
> > > > >> > > > > >> > > > > >> > > talk
> > > > >> > > > > >> > > > > >> > > >> > > like
> > > > >> > > > > >> > > > > >> > > >> > > > > that starts, we'd get help
> > > > finishing
> > > > >> or
> > > > >> > > > > >> stabilizing
> > > > >> > > > > >> > > > > what's
> > > > >> > > > > >> > > > > >> > under
> > > > >> > > > > >> > > > > >> > > >> > > > discussion
> > > > >> > > > > >> > > > > >> > > >> > > > > for revert. Or, we'd have a
> > > revert.
> > > > >> > Either
> > > > >> > > > way
> > > > >> > > > > >> the
> > > > >> > > > > >> > > > > outcome
> > > > >> > > > > >> > > > > >> is
> > > > >> > > > > >> > > > > >> > > >> > > acceptable.
> > > > >> > > > > >> > > > > >> > > >> > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > On Wed, Sep 7, 2016 at 8:56
> AM,
> > > > Dima
> > > > >> > > Spivak
> > > > >> > > > <
> > > > >> > > > > >> > > > > >> > > dimaspivak@apache.org
> > > > >> > > > > >> > > > > >> > > >> >
> > > > >> > > > > >> > > > > >> > > >> > > > wrote:
> > > > >> > > > > >> > > > > >> > > >> > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > > I'm not sure that "There is
> > > > already
> > > > >> > lots
> > > > >> > > > of
> > > > >> > > > > >> > > > half-baked
> > > > >> > > > > >> > > > > >> code
> > > > >> > > > > >> > > > > >> > in
> > > > >> > > > > >> > > > > >> > > >> the
> > > > >> > > > > >> > > > > >> > > >> > > > > branch,
> > > > >> > > > > >> > > > > >> > > >> > > > > > so what's the harm in
> adding
> > > > more?"
> > > > >> > is a
> > > > >> > > > > good
> > > > >> > > > > >> > code
> > > > >> > > > > >> > > > > commit
> > > > >> > > > > >> > > > > >> > > >> > philosophy
> > > > >> > > > > >> > > > > >> > > >> > > > for
> > > > >> > > > > >> > > > > >> > > >> > > > > a
> > > > >> > > > > >> > > > > >> > > >> > > > > > fault-tolerant distributed
> > data
> > > > >> store.
> > > > >> > > ;)
> > > > >> > > > > >> > > > > >> > > >> > > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > > More seriously, a lack of
> > test
> > > > >> > coverage
> > > > >> > > > for
> > > > >> > > > > >> > > existing
> > > > >> > > > > >> > > > > >> > features
> > > > >> > > > > >> > > > > >> > > >> > > shouldn't
> > > > >> > > > > >> > > > > >> > > >> > > > > be
> > > > >> > > > > >> > > > > >> > > >> > > > > > used as justification for
> > > > >> introducing
> > > > >> > > new
> > > > >> > > > > >> > features
> > > > >> > > > > >> > > > with
> > > > >> > > > > >> > > > > >> the
> > > > >> > > > > >> > > > > >> > > same
> > > > >> > > > > >> > > > > >> > > >> > > > > > shortcomings. Ultimately,
> > it's
> > > > the
> > > > >> end
> > > > >> > > > user
> > > > >> > > > > who
> > > > >> > > > > >> > > will
> > > > >> > > > > >> > > > > feel
> > > > >> > > > > >> > > > > >> > the
> > > > >> > > > > >> > > > > >> > > >> pain,
> > > > >> > > > > >> > > > > >> > > >> > > so
> > > > >> > > > > >> > > > > >> > > >> > > > > > shouldn't we do everything
> we
> > > can
> > > > >> to
> > > > >> > > > > mitigate
> > > > >> > > > > >> > that?
> > > > >> > > > > >> > > > > >> > > >> > > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > > -Dima
> > > > >> > > > > >> > > > > >> > > >> > > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > > On Wed, Sep 7, 2016 at 8:46
> > AM,
> > > > >> > Vladimir
> > > > >> > > > > >> > Rodionov <
> > > > >> > > > > >> > > > > >> > > >> > > > > vladrodionov@gmail.com>
> > > > >> > > > > >> > > > > >> > > >> > > > > > wrote:
> > > > >> > > > > >> > > > > >> > > >> > > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > > > Sean,
> > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > > > * have docs
> > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > > > Agree. We have a doc and
> > > backup
> > > > >> is
> > > > >> > the
> > > > >> > > > > most
> > > > >> > > > > >> > > > > documented
> > > > >> > > > > >> > > > > >> > > feature
> > > > >> > > > > >> > > > > >> > > >> > :),
> > > > >> > > > > >> > > > > >> > > >> > > we
> > > > >> > > > > >> > > > > >> > > >> > > > > > will
> > > > >> > > > > >> > > > > >> > > >> > > > > > > release it shortly to
> > Apache.
> > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > > > * have sunny-day
> > correctness
> > > > >> tests
> > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > > > Feature has  close to 60
> > test
> > > > >> cases,
> > > > >> > > > which
> > > > >> > > > > >> run
> > > > >> > > > > >> > > for
> > > > >> > > > > >> > > > > >> approx
> > > > >> > > > > >> > > > > >> > 30
> > > > >> > > > > >> > > > > >> > > >> min.
> > > > >> > > > > >> > > > > >> > > >> > > We
> > > > >> > > > > >> > > > > >> > > >> > > > > can
> > > > >> > > > > >> > > > > >> > > >> > > > > > > add more, if community do
> > not
> > > > >> mind
> > > > >> > :)
> > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > > > * have
> > > > >> > correctness-in-face-of-failure
> > > > >> > > > > tests
> > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > > > Any examples of these
> tests
> > > in
> > > > >> > > existing
> > > > >> > > > > >> > features?
> > > > >> > > > > >> > > > In
> > > > >> > > > > >> > > > > >> > works,
> > > > >> > > > > >> > > > > >> > > we
> > > > >> > > > > >> > > > > >> > > >> > > have a
> > > > >> > > > > >> > > > > >> > > >> > > > > > clear
> > > > >> > > > > >> > > > > >> > > >> > > > > > > understanding of what
> > should
> > > be
> > > > >> done
> > > > >> > > by
> > > > >> > > > > the
> > > > >> > > > > >> > time
> > > > >> > > > > >> > > of
> > > > >> > > > > >> > > > > 2.0
> > > > >> > > > > >> > > > > >> > > >> release.
> > > > >> > > > > >> > > > > >> > > >> > > > > > > That is very close goal
> for
> > > us,
> > > > >> to
> > > > >> > > > verify
> > > > >> > > > > IT
> > > > >> > > > > >> > > monkey
> > > > >> > > > > >> > > > > for
> > > > >> > > > > >> > > > > >> > > >> existing
> > > > >> > > > > >> > > > > >> > > >> > > > code.
> > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > > > * don't rely on things
> > > outside
> > > > of
> > > > >> > > HBase
> > > > >> > > > > for
> > > > >> > > > > >> > > normal
> > > > >> > > > > >> > > > > >> > operation
> > > > >> > > > > >> > > > > >> > > >> > (okay
> > > > >> > > > > >> > > > > >> > > >> > > > for
> > > > >> > > > > >> > > > > >> > > >> > > > > > > advanced operation)
> > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > > > We do not.
> > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > > > Enormous time has been
> > spent
> > > > >> already
> > > > >> > > on
> > > > >> > > > > the
> > > > >> > > > > >> > > > > development
> > > > >> > > > > >> > > > > >> > and
> > > > >> > > > > >> > > > > >> > > >> > testing
> > > > >> > > > > >> > > > > >> > > >> > > > the
> > > > >> > > > > >> > > > > >> > > >> > > > > > > feature, it has passed
> our
> > > > >> internal
> > > > >> > > > tests
> > > > >> > > > > and
> > > > >> > > > > >> > > many
> > > > >> > > > > >> > > > > >> rounds
> > > > >> > > > > >> > > > > >> > of
> > > > >> > > > > >> > > > > >> > > >> code
> > > > >> > > > > >> > > > > >> > > >> > > > > reviews
> > > > >> > > > > >> > > > > >> > > >> > > > > > > by HBase committers. We
> do
> > > not
> > > > >> mind
> > > > >> > if
> > > > >> > > > > >> someone
> > > > >> > > > > >> > > from
> > > > >> > > > > >> > > > > >> HBase
> > > > >> > > > > >> > > > > >> > > >> > community
> > > > >> > > > > >> > > > > >> > > >> > > > > > > (outside of HW) will
> review
> > > the
> > > > >> > code,
> > > > >> > > > but
> > > > >> > > > > it
> > > > >> > > > > >> > will
> > > > >> > > > > >> > > > > >> probably
> > > > >> > > > > >> > > > > >> > > >> takes
> > > > >> > > > > >> > > > > >> > > >> > > > > forever
> > > > >> > > > > >> > > > > >> > > >> > > > > > to
> > > > >> > > > > >> > > > > >> > > >> > > > > > > wait for volunteer?, the
> > > > feature
> > > > >> is
> > > > >> > > > quite
> > > > >> > > > > >> large
> > > > >> > > > > >> > > > (1MB+
> > > > >> > > > > >> > > > > >> > > >> cumulative
> > > > >> > > > > >> > > > > >> > > >> > > > patch)
> > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > > > 2.0 branch is full of
> half
> > > > baked
> > > > >> > > > features,
> > > > >> > > > > >> most
> > > > >> > > > > >> > > of
> > > > >> > > > > >> > > > > them
> > > > >> > > > > >> > > > > >> > are
> > > > >> > > > > >> > > > > >> > > in
> > > > >> > > > > >> > > > > >> > > >> > > active
> > > > >> > > > > >> > > > > >> > > >> > > > > > > development, therefore I
> am
> > > not
> > > > >> > > > following
> > > > >> > > > > you
> > > > >> > > > > >> > > here,
> > > > >> > > > > >> > > > > >> Sean?
> > > > >> > > > > >> > > > > >> > > Why
> > > > >> > > > > >> > > > > >> > > >> > > > > HBASE-7912
> > > > >> > > > > >> > > > > >> > > >> > > > > > is
> > > > >> > > > > >> > > > > >> > > >> > > > > > > not good enough yet to be
> > > > >> integrated
> > > > >> > > > into
> > > > >> > > > > 2.0
> > > > >> > > > > >> > > > branch?
> > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > > > -Vlad
> > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > > > On Wed, Sep 7, 2016 at
> 8:23
> > > AM,
> > > > >> Sean
> > > > >> > > > > Busbey <
> > > > >> > > > > >> > > > > >> > > busbey@apache.org
> > > > >> > > > > >> > > > > >> > > >> >
> > > > >> > > > > >> > > > > >> > > >> > > > wrote:
> > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > On Tue, Sep 6, 2016 at
> > > 10:36
> > > > >> PM,
> > > > >> > > Josh
> > > > >> > > > > >> Elser <
> > > > >> > > > > >> > > > > >> > > >> > > josh.elser@gmail.com>
> > > > >> > > > > >> > > > > >> > > >> > > > > > > wrote:
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > > So, the answer to
> > Sean's
> > > > >> > original
> > > > >> > > > > >> question
> > > > >> > > > > >> > is
> > > > >> > > > > >> > > > "as
> > > > >> > > > > >> > > > > >> > > robust as
> > > > >> > > > > >> > > > > >> > > >> > > > > snapshots
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > > presently are"?
> > > > >> (independence of
> > > > >> > > > > >> > > backup/restore
> > > > >> > > > > >> > > > > >> > failure
> > > > >> > > > > >> > > > > >> > > >> > > tolerance
> > > > >> > > > > >> > > > > >> > > >> > > > > > from
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > > snapshot failure
> > > tolerance)
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > > Is this just a
> question
> > > WRT
> > > > >> > > context
> > > > >> > > > of
> > > > >> > > > > >> the
> > > > >> > > > > >> > > > > change,
> > > > >> > > > > >> > > > > >> or
> > > > >> > > > > >> > > > > >> > > is it
> > > > >> > > > > >> > > > > >> > > >> > > means
> > > > >> > > > > >> > > > > >> > > >> > > > > > for a
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > veto
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > > from you, Sean? Just
> > > trying
> > > > >> to
> > > > >> > > make
> > > > >> > > > > sure
> > > > >> > > > > >> > I'm
> > > > >> > > > > >> > > > > >> following
> > > > >> > > > > >> > > > > >> > > >> along
> > > > >> > > > > >> > > > > >> > > >> > > > > > > adequately.
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > I'd say ATM I'm -0,
> > > bordering
> > > > >> on
> > > > >> > -1
> > > > >> > > > but
> > > > >> > > > > not
> > > > >> > > > > >> > for
> > > > >> > > > > >> > > > > >> reasons
> > > > >> > > > > >> > > > > >> > I
> > > > >> > > > > >> > > > > >> > > can
> > > > >> > > > > >> > > > > >> > > >> > > > > > articulate
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > well.
> > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > Here's an attempt.
> > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > We've been trying to
> > move,
> > > > as a
> > > > >> > > > > community,
> > > > >> > > > > >> > > > towards
> > > > >> > > > > >> > > > > >> > > minimizing
> > > > >> > > > > >> > > > > >> > > >> > > risk
> > > > >> > > > > >> > > > > >> > > >> > > > to
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > downstream folks by
> > getting
> > > > >> > > "complete
> > > > >> > > > > >> enough
> > > > >> > > > > >> > > for
> > > > >> > > > > >> > > > > use"
> > > > >> > > > > >> > > > > >> > > gates
> > > > >> > > > > >> > > > > >> > > >> in
> > > > >> > > > > >> > > > > >> > > >> > > > place
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > before we introduce new
> > > > >> features.
> > > > >> > > This
> > > > >> > > > > was
> > > > >> > > > > >> > > > spurred
> > > > >> > > > > >> > > > > >> by a
> > > > >> > > > > >> > > > > >> > > some
> > > > >> > > > > >> > > > > >> > > >> > > > features
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > getting in half-baked
> and
> > > > never
> > > > >> > > making
> > > > >> > > > > it
> > > > >> > > > > >> to
> > > > >> > > > > >> > > "can
> > > > >> > > > > >> > > > > >> really
> > > > >> > > > > >> > > > > >> > > use"
> > > > >> > > > > >> > > > > >> > > >> > > > status
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > (I'm thinking of
> > > distributed
> > > > >> log
> > > > >> > > > replay
> > > > >> > > > > and
> > > > >> > > > > >> > the
> > > > >> > > > > >> > > > > >> zk-less
> > > > >> > > > > >> > > > > >> > > >> > > assignment
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > stuff, I don't recall
> if
> > > > there
> > > > >> was
> > > > >> > > > > more).
> > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > The gates, generally,
> > > > included
> > > > >> > > things
> > > > >> > > > > like:
> > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have docs
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have sunny-day
> > > correctness
> > > > >> tests
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > * have
> > > > >> > > correctness-in-face-of-failure
> > > > >> > > > > tests
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't rely on things
> > > > outside
> > > > >> of
> > > > >> > > > HBase
> > > > >> > > > > for
> > > > >> > > > > >> > > > normal
> > > > >> > > > > >> > > > > >> > > operation
> > > > >> > > > > >> > > > > >> > > >> > > (okay
> > > > >> > > > > >> > > > > >> > > >> > > > > for
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > advanced operation)
> > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > As an example, we kept
> > the
> > > > MOB
> > > > >> > work
> > > > >> > > > off
> > > > >> > > > > in
> > > > >> > > > > >> a
> > > > >> > > > > >> > > > branch
> > > > >> > > > > >> > > > > >> and
> > > > >> > > > > >> > > > > >> > > out
> > > > >> > > > > >> > > > > >> > > >> of
> > > > >> > > > > >> > > > > >> > > >> > > > master
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > until it could pass
> these
> > > > >> > criteria.
> > > > >> > > > The
> > > > >> > > > > big
> > > > >> > > > > >> > > > > exemption
> > > > >> > > > > >> > > > > >> > > we've
> > > > >> > > > > >> > > > > >> > > >> had
> > > > >> > > > > >> > > > > >> > > >> > > to
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > this was the
> hbase-spark
> > > > >> > > integration,
> > > > >> > > > > where
> > > > >> > > > > >> > we
> > > > >> > > > > >> > > > all
> > > > >> > > > > >> > > > > >> > agreed
> > > > >> > > > > >> > > > > >> > > it
> > > > >> > > > > >> > > > > >> > > >> > > could
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > land in master because
> it
> > > was
> > > > >> very
> > > > >> > > > well
> > > > >> > > > > >> > > isolated
> > > > >> > > > > >> > > > > (the
> > > > >> > > > > >> > > > > >> > > slide
> > > > >> > > > > >> > > > > >> > > >> > away
> > > > >> > > > > >> > > > > >> > > >> > > > from
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > including docs as a
> > > > first-class
> > > > >> > part
> > > > >> > > > of
> > > > >> > > > > >> > > building
> > > > >> > > > > >> > > > up
> > > > >> > > > > >> > > > > >> that
> > > > >> > > > > >> > > > > >> > > >> > > > integration
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > has led me to doubt the
> > > > wisdom
> > > > >> of
> > > > >> > > this
> > > > >> > > > > >> > > decision).
> > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > We've also been
> treating
> > > > >> inclusion
> > > > >> > > in
> > > > >> > > > a
> > > > >> > > > > >> > > "probably
> > > > >> > > > > >> > > > > >> will
> > > > >> > > > > >> > > > > >> > be
> > > > >> > > > > >> > > > > >> > > >> > > released
> > > > >> > > > > >> > > > > >> > > >> > > > to
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > downstream" branches
> as a
> > > > >> higher
> > > > >> > > bar,
> > > > >> > > > > >> > requiring
> > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't moderately
> impact
> > > > >> > > performance
> > > > >> > > > > when
> > > > >> > > > > >> > the
> > > > >> > > > > >> > > > > >> feature
> > > > >> > > > > >> > > > > >> > > isn't
> > > > >> > > > > >> > > > > >> > > >> in
> > > > >> > > > > >> > > > > >> > > >> > > use
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't severely impact
> > > > >> > performance
> > > > >> > > > when
> > > > >> > > > > >> the
> > > > >> > > > > >> > > > > feature
> > > > >> > > > > >> > > > > >> is
> > > > >> > > > > >> > > > > >> > in
> > > > >> > > > > >> > > > > >> > > >> use
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > * either default-to-on
> or
> > > > show
> > > > >> > > enough
> > > > >> > > > > >> demand
> > > > >> > > > > >> > to
> > > > >> > > > > >> > > > > >> believe
> > > > >> > > > > >> > > > > >> > a
> > > > >> > > > > >> > > > > >> > > >> > > > non-trivial
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > number of folks will
> turn
> > > the
> > > > >> > > feature
> > > > >> > > > on
> > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > The above has kept MOB
> > and
> > > > >> > > hbase-spark
> > > > >> > > > > >> > > > integration
> > > > >> > > > > >> > > > > >> out
> > > > >> > > > > >> > > > > >> > of
> > > > >> > > > > >> > > > > >> > > >> > > branch-1,
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > presumably while
> they've
> > > > >> "gotten
> > > > >> > > more
> > > > >> > > > > >> stable"
> > > > >> > > > > >> > > in
> > > > >> > > > > >> > > > > >> master
> > > > >> > > > > >> > > > > >> > > from
> > > > >> > > > > >> > > > > >> > > >> > the
> > > > >> > > > > >> > > > > >> > > >> > > > odd
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > vendor inclusion.
> > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > Are we going to have a
> > 2.0
> > > > >> release
> > > > >> > > > > before
> > > > >> > > > > >> the
> > > > >> > > > > >> > > end
> > > > >> > > > > >> > > > > of
> > > > >> > > > > >> > > > > >> the
> > > > >> > > > > >> > > > > >> > > >> year?
> > > > >> > > > > >> > > > > >> > > >> > > > We're
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > coming up on 1.5 years
> > > since
> > > > >> the
> > > > >> > > > > release of
> > > > >> > > > > >> > > > version
> > > > >> > > > > >> > > > > >> 1.0;
> > > > >> > > > > >> > > > > >> > > >> seems
> > > > >> > > > > >> > > > > >> > > >> > > like
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > it's about time,
> though I
> > > > >> haven't
> > > > >> > > seen
> > > > >> > > > > any
> > > > >> > > > > >> > > > concrete
> > > > >> > > > > >> > > > > >> > plans
> > > > >> > > > > >> > > > > >> > > >> this
> > > > >> > > > > >> > > > > >> > > >> > > > year.
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > Presuming we are going
> to
> > > > have
> > > > >> one
> > > > >> > > by
> > > > >> > > > > the
> > > > >> > > > > >> end
> > > > >> > > > > >> > > of
> > > > >> > > > > >> > > > > the
> > > > >> > > > > >> > > > > >> > > year, it
> > > > >> > > > > >> > > > > >> > > >> > > > seems a
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > bit close to still be
> > > adding
> > > > in
> > > > >> > > > > "features
> > > > >> > > > > >> > that
> > > > >> > > > > >> > > > need
> > > > >> > > > > >> > > > > >> > > maturing"
> > > > >> > > > > >> > > > > >> > > >> > on
> > > > >> > > > > >> > > > > >> > > >> > > > the
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > branch.
> > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > The lack of a concrete
> > plan
> > > > for
> > > > >> > 2.0
> > > > >> > > > > keeps
> > > > >> > > > > >> me
> > > > >> > > > > >> > > from
> > > > >> > > > > >> > > > > >> > > considering
> > > > >> > > > > >> > > > > >> > > >> > > these
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > things blocker at the
> > > moment.
> > > > >> But
> > > > >> > I
> > > > >> > > > know
> > > > >> > > > > >> > first
> > > > >> > > > > >> > > > hand
> > > > >> > > > > >> > > > > >> how
> > > > >> > > > > >> > > > > >> > > much
> > > > >> > > > > >> > > > > >> > > >> > > > trouble
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > folks have had with
> other
> > > > >> features
> > > > >> > > > that
> > > > >> > > > > >> have
> > > > >> > > > > >> > > gone
> > > > >> > > > > >> > > > > >> into
> > > > >> > > > > >> > > > > >> > > >> > downstream
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > facing releases without
> > > > >> robustness
> > > > >> > > > > checks
> > > > >> > > > > >> > (i.e.
> > > > >> > > > > >> > > > > >> > > replication),
> > > > >> > > > > >> > > > > >> > > >> > and
> > > > >> > > > > >> > > > > >> > > >> > > > I'm
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > concerned about what
> > we're
> > > > >> setting
> > > > >> > > up
> > > > >> > > > if
> > > > >> > > > > >> 2.0
> > > > >> > > > > >> > > goes
> > > > >> > > > > >> > > > > out
> > > > >> > > > > >> > > > > >> > with
> > > > >> > > > > >> > > > > >> > > >> this
> > > > >> > > > > >> > > > > >> > > >> > > > > > > > feature in its current
> > > state.
> > > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > --
> > > > >> > > > > >> > > > > >> > > >> > > > > Best regards,
> > > > >> > > > > >> > > > > >> > > >> > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > >    - Andy
> > > > >> > > > > >> > > > > >> > > >> > > > >
> > > > >> > > > > >> > > > > >> > > >> > > > > Problems worthy of attack
> prove
> > > > their
> > > > >> > > worth
> > > > >> > > > by
> > > > >> > > > > >> > > hitting
> > > > >> > > > > >> > > > > >> back. -
> > > > >> > > > > >> > > > > >> > > Piet
> > > > >> > > > > >> > > > > >> > > >> > > Hein
> > > > >> > > > > >> > > > > >> > > >> > > > > (via Tom White)
> > > > >> > > > > >> > > > > >> > > >> > > > >
> > > > >> > > > > >> > > > > >> > > >> > > >
> > > > >> > > > > >> > > > > >> > > >> > >
> > > > >> > > > > >> > > > > >> > > >> > >
> > > > >> > > > > >> > > > > >> > > >> > >
> > > > >> > > > > >> > > > > >> > > >> > > --
> > > > >> > > > > >> > > > > >> > > >> > > Best regards,
> > > > >> > > > > >> > > > > >> > > >> > >
> > > > >> > > > > >> > > > > >> > > >> > >    - Andy
> > > > >> > > > > >> > > > > >> > > >> > >
> > > > >> > > > > >> > > > > >> > > >> > > Problems worthy of attack prove
> > their
> > > > >> worth
> > > > >> > by
> > > > >> > > > > >> hitting
> > > > >> > > > > >> > > > back.
> > > > >> > > > > >> > > > > -
> > > > >> > > > > >> > > > > >> > Piet
> > > > >> > > > > >> > > > > >> > > >> Hein
> > > > >> > > > > >> > > > > >> > > >> > > (via Tom White)
> > > > >> > > > > >> > > > > >> > > >> > >
> > > > >> > > > > >> > > > > >> > > >> >
> > > > >> > > > > >> > > > > >> > > >>
> > > > >> > > > > >> > > > > >> > >
> > > > >> > > > > >> > > > > >> >
> > > > >> > > > > >> > > > > >>
> > > > >> > > > > >> > > > > >
> > > > >> > > > > >> > > > > >
> > > > >> > > > > >> > > > >
> > > > >> > > > > >> > > >
> > > > >> > > > > >> > >
> > > > >> > > > > >> >
> > > > >> > > > > >>
> > > > >> > > > >
> > > > >> > > > >
> > > > >> > > > >
> > > > >> > > > > --
> > > > >> > > > > busbey
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>



-- 

-- Appy

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Stack <st...@duboce.net>]

On Tue, Oct 11, 2016 at 1:50 PM, Vladimir Rodionov <vl...@gmail.com>
wrote:

> Michael,
>
> Its in HBASE-7912
>
> This is tip of git log:
>
> commit a072f6f49a26a7259ff2aaef6cb56d85eb592482
> Author: Frank Welsch <fw...@jps.net>
> Date:   Fri Sep 23 18:00:42 2016 -0400
>
>     HBASE-16574 Book updates for backup and restore
>
> commit b14e2ab1c24e65ff88dd4c579acf83cb4ed0605e
> Author: tedyu <yu...@gmail.com>
> Date:   Wed Oct 5 16:29:40 2016 -0700
>
>     HBASE-16727 Backup refactoring: remove MR dependencies from HMaster
> (Vladimir Rodionov)
>
>
Thanks. I have that. I tried it and it doesn't compile for me. Does it
compile for you?
Thanks,
M



> -Vlad
>
> On Tue, Oct 11, 2016 at 1:44 PM, Stack <st...@duboce.net> wrote:
>
> > Which branch do I check out to try it? HBASE-7912 is not it. I don't see
> an
> > HBASE-16727...
> > Thanks,
> > M
> >
> >
> > On Tue, Oct 11, 2016 at 11:41 AM, Vladimir Rodionov <
> > vladrodionov@gmail.com>
> > wrote:
> >
> > > The last patch is on review board:
> > > https://reviews.apache.org/r/52748
> > >
> > > On Mon, Oct 10, 2016 at 3:52 PM, Vladimir Rodionov <
> > vladrodionov@gmail.com
> > > >
> > > wrote:
> > >
> > > > >> How hard to put in an hbase-backup module? hbase-server is fat
> > enough
> > > > >> already. Could be done as a follow-up.
> > > >
> > > > https://issues.apache.org/jira/browse/HBASE-16727?
> > > > focusedCommentId=15531237&page=com.atlassian.jira.
> > > > plugin.system.issuetabpanels:comment-tabpanel#comment-15531237
> > > >
> > > > Can we do merge first? Then we can discuss separate module.
> > > >
> > > > -Vlad
> > > >
> > > > On Mon, Oct 10, 2016 at 3:44 PM, Ted Yu <yu...@gmail.com> wrote:
> > > >
> > > >> Looks like the first quote was cut off.
> > > >> The original sentence was:
> > > >>
> > > >> bq. no mapreduce job launched from master or region server.
> > > >>
> > > >> mapreduce job is launched from the node where command line tool is
> > run.
> > > >>
> > > >> On Mon, Oct 10, 2016 at 3:38 PM, Stack <st...@duboce.net> wrote:
> > > >>
> > > >> > bq. launched from master or region server.
> > > >> >
> > > >> > What does this mean please? Has to be run from Master or
> > RegionServer?
> > > >> Can
> > > >> > it be run from another node altogether?
> > > >> >
> > > >> > On Mon, Oct 10, 2016 at 1:44 PM, Vladimir Rodionov <
> > > >> vladrodionov@gmail.com
> > > >> > >
> > > >> > wrote:
> > > >> >
> > > >> > > >> mapreduce dependency has been moved to client side - no
> > mapreduce
> > > >> job
> > > >> > >
> > > >> > > 1. We have no code in the client module anymore, due to
> dependency
> > > on
> > > >> > > internal server API (HFile and WAL access).
> > > >> > > 2. Backup/ restore are client - driven operations, but all the
> > code
> > > >> > resides
> > > >> > > in the server module
> > > >> > >
> > > >> >
> > > >> > How hard to put in an hbase-backup module? hbase-server is fat
> > enough
> > > >> > already. Could be done as a follow-up.
> > > >> >
> > > >> > Thanks,
> > > >> > St.Ack
> > > >> >
> > > >> >
> > > >> >
> > > >> > > 3. No MR in Master, no procedure - driven execution.
> > > >> > > 4. Old good MR from command-line.
> > > >> > > 5. Security was simplified and now only super-user is allowed to
> > run
> > > >> > > backup/restores.
> > > >> > > 6. HBase Backup API was gone due to 1. Now only command-line
> > access
> > > to
> > > >> > > backup tools.
> > > >> > >
> > > >> > > These consequences of refactoring has been discussed in
> > HBASE-16727.
> > > >> > >
> > > >> > > -Vlad
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > > On Mon, Oct 10, 2016 at 1:31 PM, Ted Yu <yu...@gmail.com>
> > > wrote:
> > > >> > >
> > > >> > > > Reviving this thread.
> > > >> > > >
> > > >> > > > The following has taken place:
> > > >> > > >
> > > >> > > > mapreduce dependency has been moved to client side - no
> > mapreduce
> > > >> job
> > > >> > > > launched from master or region server.
> > > >> > > > document patch (HBASE-16574) has been integrated.
> > > >> > > > Updated mega patch has been attached to HBASE-14123: this
> covers
> > > the
> > > >> > > > refactor in #1 above and the protobuf 3 merge.
> > > >> > > >
> > > >> > > > If community has more feedback on the merge proposal, I would
> > love
> > > >> to
> > > >> > > hear
> > > >> > > > it.
> > > >> > > >
> > > >> > > > Thanks
> > > >> > > >
> > > >> > > > On Thu, Sep 22, 2016 at 10:31 AM, Sean Busbey <
> > > busbey@cloudera.com>
> > > >> > > wrote:
> > > >> > > >
> > > >> > > > > I'd like to see the docs proposed on HBASE-16574 integrated
> > into
> > > >> our
> > > >> > > > > project's documentation prior to merge.
> > > >> > > > >
> > > >> > > > > On Thu, Sep 22, 2016 at 9:02 AM, Ted Yu <
> yuzhihong@gmail.com>
> > > >> wrote:
> > > >> > > > > > This feature can be marked experimental due to some
> > > limitations
> > > >> > such
> > > >> > > as
> > > >> > > > > > security.
> > > >> > > > > >
> > > >> > > > > > Your previous round of comments have been addressed.
> > > >> > > > > > Command line tool has gone through:
> > > >> > > > > >
> > > >> > > > > > HBASE-16620 Fix backup command-line tool usability issues
> > > >> > > > > > HBASE-16655 hbase backup describe with incorrect backup id
> > > >> results
> > > >> > in
> > > >> > > > NPE
> > > >> > > > > >
> > > >> > > > > > The updated doc has been attached to HBASE-16574.
> > > >> > > > > >
> > > >> > > > > > Cheers
> > > >> > > > > >
> > > >> > > > > > On Thu, Sep 22, 2016 at 8:53 AM, Stack <st...@duboce.net>
> > > >> wrote:
> > > >> > > > > >
> > > >> > > > > >> On Wed, Sep 21, 2016 at 7:43 AM, Ted Yu <
> > yuzhihong@gmail.com
> > > >
> > > >> > > wrote:
> > > >> > > > > >>
> > > >> > > > > >> > Are there more (review) comments ?
> > > >> > > > > >> >
> > > >> > > > > >> >
> > > >> > > > > >> Are outstanding comments addressed?
> > > >> > > > > >>
> > > >> > > > > >> I don't see answer to my 'is this experimental/will it be
> > > >> marked
> > > >> > > > > >> experimental' question.
> > > >> > > > > >>
> > > >> > > > > >> I ran into some issues trying to use the feature and
> > > suggested
> > > >> > that
> > > >> > > a
> > > >> > > > > >> feature likes this needs polish else it'll just rot,
> > unused.
> > > >> Has
> > > >> > > > polish
> > > >> > > > > >> been applied? All ready for another 'user' test? Suggest
> > that
> > > >> you
> > > >> > > > update
> > > >> > > > > >> here going forward for the benefit of those trying to
> > follow
> > > >> along
> > > >> > > and
> > > >> > > > > who
> > > >> > > > > >> are not watching JIRA change fly-by.
> > > >> > > > > >>
> > > >> > > > > >> It looks like doc got a revision -- I have to check -- to
> > > take
> > > >> on
> > > >> > > > > >> suggestion made above but again, suggest, that this
> thread
> > > gets
> > > >> > > > updated.
> > > >> > > > > >>
> > > >> > > > > >> Thanks,
> > > >> > > > > >> St.Ack
> > > >> > > > > >>
> > > >> > > > > >>
> > > >> > > > > >>
> > > >> > > > > >> > Thanks
> > > >> > > > > >> >
> > > >> > > > > >> > On Tue, Sep 20, 2016 at 10:02 AM, Devaraj Das <
> > > >> > > ddas@hortonworks.com
> > > >> > > > >
> > > >> > > > > >> > wrote:
> > > >> > > > > >> >
> > > >> > > > > >> > > Just reviving this thread. Thanks Sean, Stack, Dima,
> > and
> > > >> > others
> > > >> > > > for
> > > >> > > > > the
> > > >> > > > > >> > > thorough reviews and testing. Thanks Ted and Vlad for
> > > >> taking
> > > >> > > care
> > > >> > > > of
> > > >> > > > > >> the
> > > >> > > > > >> > > feedback. Are we all good to do the merge now? Rather
> > do
> > > >> > sooner
> > > >> > > > than
> > > >> > > > > >> > later.
> > > >> > > > > >> > > ________________________________________
> > > >> > > > > >> > > From: saint.ack@gmail.com <sa...@gmail.com> on
> > > behalf
> > > >> of
> > > >> > > > Stack
> > > >> > > > > <
> > > >> > > > > >> > > stack@duboce.net>
> > > >> > > > > >> > > Sent: Monday, September 12, 2016 1:18 PM
> > > >> > > > > >> > > To: HBase Dev List
> > > >> > > > > >> > > Subject: Re: [DISCUSSION] Merge Backup / Restore -
> > Branch
> > > >> > > > HBASE-7912
> > > >> > > > > >> > >
> > > >> > > > > >> > > On Mon, Sep 12, 2016 at 12:19 PM, Ted Yu <
> > > >> yuzhihong@gmail.com
> > > >> > >
> > > >> > > > > wrote:
> > > >> > > > > >> > >
> > > >> > > > > >> > > > Mega patch (rev 18) is on HBASE-14123.
> > > >> > > > > >> > > >
> > > >> > > > > >> > > > Please comment on HBASE-14123 on how you want to
> > > review.
> > > >> > > > > >> > > >
> > > >> > > > > >> > >
> > > >> > > > > >> > >
> > > >> > > > > >> > > Yeah. That was my lost tab. Last rb was 6 months ago.
> > > >> Suggest
> > > >> > > > > updating
> > > >> > > > > >> > it.
> > > >> > > > > >> > > RB is pretty good for review. Patch is only 1.5M so
> > > should
> > > >> be
> > > >> > > > fine.
> > > >> > > > > >> > >
> > > >> > > > > >> > > St.Ack
> > > >> > > > > >> > >
> > > >> > > > > >> > >
> > > >> > > > > >> > > >
> > > >> > > > > >> > > > Thanks
> > > >> > > > > >> > > >
> > > >> > > > > >> > > > On Mon, Sep 12, 2016 at 12:15 PM, Stack <
> > > >> stack@duboce.net>
> > > >> > > > wrote:
> > > >> > > > > >> > > >
> > > >> > > > > >> > > > > On review of the 'patch', do I just compare the
> > > branch
> > > >> to
> > > >> > > > > master or
> > > >> > > > > >> > is
> > > >> > > > > >> > > > > there a megapatch posted somewhere (I think I saw
> > one
> > > >> but
> > > >> > it
> > > >> > > > > seemed
> > > >> > > > > >> > > stale
> > > >> > > > > >> > > > > and then I 'lost' the tab). Sorry for dumb
> > question.
> > > >> > > > > >> > > > > St.Ack
> > > >> > > > > >> > > > >
> > > >> > > > > >> > > > > On Mon, Sep 12, 2016 at 12:01 PM, Stack <
> > > >> stack@duboce.net
> > > >> > >
> > > >> > > > > wrote:
> > > >> > > > > >> > > > >
> > > >> > > > > >> > > > > > Late to the game. A few comments after
> rereading
> > > this
> > > >> > > thread
> > > >> > > > > as a
> > > >> > > > > >> > > > 'user'.
> > > >> > > > > >> > > > > >
> > > >> > > > > >> > > > > > + Before merge, a user-facing feature like this
> > > >> should
> > > >> > > work
> > > >> > > > > (If
> > > >> > > > > >> > this
> > > >> > > > > >> > > is
> > > >> > > > > >> > > > > "higher-bar
> > > >> > > > > >> > > > > > for new features", bring it on -- smile).
> > > >> > > > > >> > > > > > + As a user, I tried the branch with tools
> after
> > > >> > reviewing
> > > >> > > > the
> > > >> > > > > >> > > > > just-posted
> > > >> > > > > >> > > > > > doc. I had an 'interesting' experience (left
> > > >> comments up
> > > >> > > on
> > > >> > > > > >> > issue). I
> > > >> > > > > >> > > > > think
> > > >> > > > > >> > > > > > the tooling/doc. important to get right. If it
> > > breaks
> > > >> > > easily
> > > >> > > > > or
> > > >> > > > > >> is
> > > >> > > > > >> > > > > > inconsistent (or lacks 'polish'), operators
> will
> > > >> judge
> > > >> > the
> > > >> > > > > whole
> > > >> > > > > >> > > > > > backup/restore tooling chain as not trustworthy
> > and
> > > >> > > abandon
> > > >> > > > > it.
> > > >> > > > > >> > Lets
> > > >> > > > > >> > > > not
> > > >> > > > > >> > > > > > have this happen to this feature.
> > > >> > > > > >> > > > > > + Matteo's suggestion (with a helpful starter
> > list)
> > > >> that
> > > >> > > > there
> > > >> > > > > >> > needs
> > > >> > > > > >> > > to
> > > >> > > > > >> > > > > be
> > > >> > > > > >> > > > > > explicit qualification on what is actually
> being
> > > >> > delivered
> > > >> > > > --
> > > >> > > > > >> > > > including a
> > > >> > > > > >> > > > > > listing of limitations (some look serious such
> as
> > > >> data
> > > >> > > bleed
> > > >> > > > > from
> > > >> > > > > >> > > other
> > > >> > > > > >> > > > > > regions in WALs, but maybe I don't care for my
> > use
> > > >> > > case...)
> > > >> > > > --
> > > >> > > > > >> > needs
> > > >> > > > > >> > > to
> > > >> > > > > >> > > > > > accompany the merge. Lets fold them into the
> user
> > > >> doc.
> > > >> > in
> > > >> > > > the
> > > >> > > > > >> > > technical
> > > >> > > > > >> > > > > > overview area as suggested so user expectations
> > are
> > > >> > > properly
> > > >> > > > > >> > managed
> > > >> > > > > >> > > > > > (otherwise, they expect the world and will just
> > > give
> > > >> up
> > > >> > > when
> > > >> > > > > we
> > > >> > > > > >> > fall
> > > >> > > > > >> > > > > > short). Vladimir did a list of what is in each
> of
> > > the
> > > >> > > phases
> > > >> > > > > >> above
> > > >> > > > > >> > > > which
> > > >> > > > > >> > > > > > would serve as a good start.
> > > >> > > > > >> > > > > > + Is this feature 'experimental' (Matteo asks
> > > above).
> > > >> > I'd
> > > >> > > > > prefer
> > > >> > > > > >> it
> > > >> > > > > >> > > is
> > > >> > > > > >> > > > > > not. If it is, it should be labelled all over
> > that
> > > >> it is
> > > >> > > > so. I
> > > >> > > > > >> see
> > > >> > > > > >> > > > > current
> > > >> > > > > >> > > > > > state called out as a '... technical preview
> > > >> feature'.
> > > >> > > Does
> > > >> > > > > this
> > > >> > > > > >> > mean
> > > >> > > > > >> > > > > > not-for-users?
> > > >> > > > > >> > > > > >
> > > >> > > > > >> > > > > > St.Ack
> > > >> > > > > >> > > > > >
> > > >> > > > > >> > > > > >
> > > >> > > > > >> > > > > >
> > > >> > > > > >> > > > > >
> > > >> > > > > >> > > > > >
> > > >> > > > > >> > > > > >
> > > >> > > > > >> > > > > >
> > > >> > > > > >> > > > > >
> > > >> > > > > >> > > > > >
> > > >> > > > > >> > > > > >
> > > >> > > > > >> > > > > >
> > > >> > > > > >> > > > > > On Mon, Sep 12, 2016 at 8:03 AM, Ted Yu <
> > > >> > > > yuzhihong@gmail.com>
> > > >> > > > > >> > wrote:
> > > >> > > > > >> > > > > >
> > > >> > > > > >> > > > > >> Sean:
> > > >> > > > > >> > > > > >> Do you have more comments ?
> > > >> > > > > >> > > > > >>
> > > >> > > > > >> > > > > >> Cheers
> > > >> > > > > >> > > > > >>
> > > >> > > > > >> > > > > >> On Fri, Sep 9, 2016 at 1:42 PM, Vladimir
> > Rodionov
> > > <
> > > >> > > > > >> > > > > vladrodionov@gmail.com
> > > >> > > > > >> > > > > >> >
> > > >> > > > > >> > > > > >> wrote:
> > > >> > > > > >> > > > > >>
> > > >> > > > > >> > > > > >> > Sean,
> > > >> > > > > >> > > > > >> >
> > > >> > > > > >> > > > > >> > Backup/Restore can fail due to various
> > reasons:
> > > >> > network
> > > >> > > > > outage
> > > >> > > > > >> > > > > (cluster
> > > >> > > > > >> > > > > >> > wide), various time-outs in HBase and HDFS
> > > layer,
> > > >> M/R
> > > >> > > > > failure
> > > >> > > > > >> > due
> > > >> > > > > >> > > to
> > > >> > > > > >> > > > > >> "HDFS
> > > >> > > > > >> > > > > >> > exceeded quota", user error (manual deletion
> > of
> > > >> data)
> > > >> > > and
> > > >> > > > > so
> > > >> > > > > >> on
> > > >> > > > > >> > so
> > > >> > > > > >> > > > on.
> > > >> > > > > >> > > > > >> That
> > > >> > > > > >> > > > > >> > is impossible to enumerate all possible
> types
> > of
> > > >> > > failures
> > > >> > > > > in a
> > > >> > > > > >> > > > > >> distributed
> > > >> > > > > >> > > > > >> > system - that is not our goal/task.
> > > >> > > > > >> > > > > >> >
> > > >> > > > > >> > > > > >> > We focus completely on backup system table
> > > >> > consistency
> > > >> > > > in a
> > > >> > > > > >> > > presence
> > > >> > > > > >> > > > > of
> > > >> > > > > >> > > > > >> any
> > > >> > > > > >> > > > > >> > type of failure. That is what I call
> > "tolerance
> > > to
> > > >> > > > > failures".
> > > >> > > > > >> > > > > >> >
> > > >> > > > > >> > > > > >> > On a failure:
> > > >> > > > > >> > > > > >> >
> > > >> > > > > >> > > > > >> > BACKUP. All backup system information (prior
> > to
> > > >> > backup)
> > > >> > > > > will
> > > >> > > > > >> be
> > > >> > > > > >> > > > > restored
> > > >> > > > > >> > > > > >> > and all temporary data, related to a failed
> > > >> session,
> > > >> > in
> > > >> > > > > HDFS
> > > >> > > > > >> > will
> > > >> > > > > >> > > be
> > > >> > > > > >> > > > > >> > deleted
> > > >> > > > > >> > > > > >> > RESTORE. We do not care about system data,
> > > because
> > > >> > > > restore
> > > >> > > > > >> does
> > > >> > > > > >> > > not
> > > >> > > > > >> > > > > >> change
> > > >> > > > > >> > > > > >> > it. Temporary data in HDFS will be cleaned
> up
> > > and
> > > >> > table
> > > >> > > > > will
> > > >> > > > > >> be
> > > >> > > > > >> > > in a
> > > >> > > > > >> > > > > >> state
> > > >> > > > > >> > > > > >> > back to where it was before operation
> started.
> > > >> > > > > >> > > > > >> >
> > > >> > > > > >> > > > > >> > This is what user should expect in case of a
> > > >> failure.
> > > >> > > > > >> > > > > >> >
> > > >> > > > > >> > > > > >> > -Vlad
> > > >> > > > > >> > > > > >> >
> > > >> > > > > >> > > > > >> >
> > > >> > > > > >> > > > > >> > -Vlad
> > > >> > > > > >> > > > > >> >
> > > >> > > > > >> > > > > >> > On Fri, Sep 9, 2016 at 12:56 PM, Sean
> Busbey <
> > > >> > > > > >> busbey@apache.org
> > > >> > > > > >> > >
> > > >> > > > > >> > > > > wrote:
> > > >> > > > > >> > > > > >> >
> > > >> > > > > >> > > > > >> > > Failing in a consistent way, with docs
> that
> > > >> explain
> > > >> > > the
> > > >> > > > > >> > various
> > > >> > > > > >> > > > > >> > > expected failures would be sufficient.
> > > >> > > > > >> > > > > >> > >
> > > >> > > > > >> > > > > >> > > On Fri, Sep 9, 2016 at 12:16 PM, Vladimir
> > > >> Rodionov
> > > >> > > > > >> > > > > >> > > <vl...@gmail.com> wrote:
> > > >> > > > > >> > > > > >> > > > Do not worry Sean, doc is coming today
> as
> > a
> > > >> > preview
> > > >> > > > and
> > > >> > > > > >> our
> > > >> > > > > >> > > > writer
> > > >> > > > > >> > > > > >> > Frank
> > > >> > > > > >> > > > > >> > > > will be working on a putting  it into
> > Apache
> > > >> > repo.
> > > >> > > > > >> Timeline
> > > >> > > > > >> > > > > depends
> > > >> > > > > >> > > > > >> on
> > > >> > > > > >> > > > > >> > > > Franks schedule but I hope we will get
> it
> > > >> rather
> > > >> > > > sooner
> > > >> > > > > >> than
> > > >> > > > > >> > > > > later.
> > > >> > > > > >> > > > > >> > > >
> > > >> > > > > >> > > > > >> > > > As for failure testing, we are focusing
> > only
> > > >> on a
> > > >> > > > > >> consistent
> > > >> > > > > >> > > > state
> > > >> > > > > >> > > > > >> of
> > > >> > > > > >> > > > > >> > > > backup system data in a presence of any
> > type
> > > >> of
> > > >> > > > > failures,
> > > >> > > > > >> We
> > > >> > > > > >> > > are
> > > >> > > > > >> > > > > not
> > > >> > > > > >> > > > > >> > > going
> > > >> > > > > >> > > > > >> > > > to implement  anything more "fancy",
> than
> > > >> that.
> > > >> > We
> > > >> > > > > allow
> > > >> > > > > >> > both:
> > > >> > > > > >> > > > > >> backup
> > > >> > > > > >> > > > > >> > and
> > > >> > > > > >> > > > > >> > > > restore to fail. What we do not allow is
> > to
> > > >> have
> > > >> > > > system
> > > >> > > > > >> data
> > > >> > > > > >> > > > > >> corrupted.
> > > >> > > > > >> > > > > >> > > > Will it suffice for you? Do you have any
> > > other
> > > >> > > > > concerns,
> > > >> > > > > >> you
> > > >> > > > > >> > > > want
> > > >> > > > > >> > > > > >> us to
> > > >> > > > > >> > > > > >> > > > address?
> > > >> > > > > >> > > > > >> > > >
> > > >> > > > > >> > > > > >> > > > -Vlad
> > > >> > > > > >> > > > > >> > > >
> > > >> > > > > >> > > > > >> > > >
> > > >> > > > > >> > > > > >> > > > On Fri, Sep 9, 2016 at 10:56 AM, Sean
> > > Busbey <
> > > >> > > > > >> > > busbey@apache.org
> > > >> > > > > >> > > > >
> > > >> > > > > >> > > > > >> > wrote:
> > > >> > > > > >> > > > > >> > > >
> > > >> > > > > >> > > > > >> > > >> "docs will come to Apache soon" does
> not
> > > >> address
> > > >> > > my
> > > >> > > > > >> concern
> > > >> > > > > >> > > > > around
> > > >> > > > > >> > > > > >> > docs
> > > >> > > > > >> > > > > >> > > at
> > > >> > > > > >> > > > > >> > > >> all, unless said docs have already made
> > it
> > > >> into
> > > >> > > the
> > > >> > > > > >> project
> > > >> > > > > >> > > > > repo. I
> > > >> > > > > >> > > > > >> > > don't
> > > >> > > > > >> > > > > >> > > >> want third party resources for using a
> > > major
> > > >> and
> > > >> > > > > >> important
> > > >> > > > > >> > > > > feature
> > > >> > > > > >> > > > > >> of
> > > >> > > > > >> > > > > >> > > the
> > > >> > > > > >> > > > > >> > > >> project, I want us to provide end users
> > > with
> > > >> > what
> > > >> > > > they
> > > >> > > > > >> need
> > > >> > > > > >> > > to
> > > >> > > > > >> > > > > get
> > > >> > > > > >> > > > > >> the
> > > >> > > > > >> > > > > >> > > job
> > > >> > > > > >> > > > > >> > > >> done.
> > > >> > > > > >> > > > > >> > > >>
> > > >> > > > > >> > > > > >> > > >> I see some calls for patience on the
> > > failure
> > > >> > > > testing,
> > > >> > > > > but
> > > >> > > > > >> > the
> > > >> > > > > >> > > > > >> appeal
> > > >> > > > > >> > > > > >> > to
> > > >> > > > > >> > > > > >> > > us
> > > >> > > > > >> > > > > >> > > >> having done a bad job of requiring
> proper
> > > >> tests
> > > >> > of
> > > >> > > > > >> previous
> > > >> > > > > >> > > > > >> features
> > > >> > > > > >> > > > > >> > > just
> > > >> > > > > >> > > > > >> > > >> makes me more concerned about not
> getting
> > > >> them
> > > >> > > > here. I
> > > >> > > > > >> > don't
> > > >> > > > > >> > > > want
> > > >> > > > > >> > > > > >> to
> > > >> > > > > >> > > > > >> > set
> > > >> > > > > >> > > > > >> > > >> yet another bad example that will then
> be
> > > >> > pointed
> > > >> > > to
> > > >> > > > > in
> > > >> > > > > >> the
> > > >> > > > > >> > > > > future.
> > > >> > > > > >> > > > > >> > > >>
> > > >> > > > > >> > > > > >> > > >> On Sep 8, 2016 10:50, "Ted Yu" <
> > > >> > > yuzhihong@gmail.com
> > > >> > > > >
> > > >> > > > > >> > wrote:
> > > >> > > > > >> > > > > >> > > >>
> > > >> > > > > >> > > > > >> > > >> > Is there any concern which is not
> > > >> addressed ?
> > > >> > > > > >> > > > > >> > > >> >
> > > >> > > > > >> > > > > >> > > >> > Do we need another Vote thread ?
> > > >> > > > > >> > > > > >> > > >> >
> > > >> > > > > >> > > > > >> > > >> > Thanks
> > > >> > > > > >> > > > > >> > > >> >
> > > >> > > > > >> > > > > >> > > >> > On Thu, Sep 8, 2016 at 9:21 AM,
> Andrew
> > > >> > Purtell <
> > > >> > > > > >> > > > > >> apurtell@apache.org
> > > >> > > > > >> > > > > >> > >
> > > >> > > > > >> > > > > >> > > >> > wrote:
> > > >> > > > > >> > > > > >> > > >> >
> > > >> > > > > >> > > > > >> > > >> > > Vlad,
> > > >> > > > > >> > > > > >> > > >> > >
> > > >> > > > > >> > > > > >> > > >> > > I apologize for using the term
> > > >> 'half-baked'
> > > >> > > in a
> > > >> > > > > way
> > > >> > > > > >> > that
> > > >> > > > > >> > > > > could
> > > >> > > > > >> > > > > >> > > seem a
> > > >> > > > > >> > > > > >> > > >> > > description of HBASE-7912. I meant
> > that
> > > >> as a
> > > >> > > > > general
> > > >> > > > > >> > > > > >> hypothetical.
> > > >> > > > > >> > > > > >> > > >> > >
> > > >> > > > > >> > > > > >> > > >> > > On Wed, Sep 7, 2016 at 9:36 AM,
> > > Vladimir
> > > >> > > > Rodionov
> > > >> > > > > <
> > > >> > > > > >> > > > > >> > > >> > vladrodionov@gmail.com>
> > > >> > > > > >> > > > > >> > > >> > > wrote:
> > > >> > > > > >> > > > > >> > > >> > >
> > > >> > > > > >> > > > > >> > > >> > > > >> I'm not sure that "There is
> > > already
> > > >> > lots
> > > >> > > of
> > > >> > > > > >> > > half-baked
> > > >> > > > > >> > > > > >> code
> > > >> > > > > >> > > > > >> > in
> > > >> > > > > >> > > > > >> > > the
> > > >> > > > > >> > > > > >> > > >> > > > branch,
> > > >> > > > > >> > > > > >> > > >> > > > so what's the harm in adding
> more?"
> > > >> > > > > >> > > > > >> > > >> > > >
> > > >> > > > > >> > > > > >> > > >> > > > I meant - not production - ready
> > yet.
> > > >> This
> > > >> > > is
> > > >> > > > > 2.0
> > > >> > > > > >> > > > > development
> > > >> > > > > >> > > > > >> > > branch
> > > >> > > > > >> > > > > >> > > >> > and,
> > > >> > > > > >> > > > > >> > > >> > > > hence many features are in works,
> > > >> > > > > >> > > > > >> > > >> > > > not being tested well etc. I do
> not
> > > >> > consider
> > > >> > > > > backup
> > > >> > > > > >> > as
> > > >> > > > > >> > > > half
> > > >> > > > > >> > > > > >> > baked
> > > >> > > > > >> > > > > >> > > >> > > feature -
> > > >> > > > > >> > > > > >> > > >> > > > it has passed our internal QA and
> > has
> > > >> very
> > > >> > > > good
> > > >> > > > > >> doc,
> > > >> > > > > >> > > > which
> > > >> > > > > >> > > > > we
> > > >> > > > > >> > > > > >> > will
> > > >> > > > > >> > > > > >> > > >> > > provide
> > > >> > > > > >> > > > > >> > > >> > > > to Apache shortly.
> > > >> > > > > >> > > > > >> > > >> > > >
> > > >> > > > > >> > > > > >> > > >> > > > -Vlad
> > > >> > > > > >> > > > > >> > > >> > > >
> > > >> > > > > >> > > > > >> > > >> > > > On Wed, Sep 7, 2016 at 9:13 AM,
> > > Andrew
> > > >> > > > Purtell <
> > > >> > > > > >> > > > > >> > > apurtell@apache.org>
> > > >> > > > > >> > > > > >> > > >> > > > wrote:
> > > >> > > > > >> > > > > >> > > >> > > >
> > > >> > > > > >> > > > > >> > > >> > > > > We shouldn't admit half baked
> > > changes
> > > >> > that
> > > >> > > > > won't
> > > >> > > > > >> be
> > > >> > > > > >> > > > > >> finished.
> > > >> > > > > >> > > > > >> > > >> However
> > > >> > > > > >> > > > > >> > > >> > > in
> > > >> > > > > >> > > > > >> > > >> > > > > this case the crew working on
> > this
> > > >> > feature
> > > >> > > > are
> > > >> > > > > >> long
> > > >> > > > > >> > > > > timers
> > > >> > > > > >> > > > > >> and
> > > >> > > > > >> > > > > >> > > less
> > > >> > > > > >> > > > > >> > > >> > > > likely
> > > >> > > > > >> > > > > >> > > >> > > > > than just about anyone to leave
> > > >> > something
> > > >> > > > in a
> > > >> > > > > >> half
> > > >> > > > > >> > > > baked
> > > >> > > > > >> > > > > >> > > state. Of
> > > >> > > > > >> > > > > >> > > >> > > > course
> > > >> > > > > >> > > > > >> > > >> > > > > there is no guarantee how
> > anything
> > > >> will
> > > >> > > turn
> > > >> > > > > out,
> > > >> > > > > >> > > but I
> > > >> > > > > >> > > > > am
> > > >> > > > > >> > > > > >> > > willing
> > > >> > > > > >> > > > > >> > > >> to
> > > >> > > > > >> > > > > >> > > >> > > > take
> > > >> > > > > >> > > > > >> > > >> > > > > a little on faith if they feel
> > > their
> > > >> > best
> > > >> > > > path
> > > >> > > > > >> > > forward
> > > >> > > > > >> > > > > now
> > > >> > > > > >> > > > > >> is
> > > >> > > > > >> > > > > >> > to
> > > >> > > > > >> > > > > >> > > >> > merge
> > > >> > > > > >> > > > > >> > > >> > > to
> > > >> > > > > >> > > > > >> > > >> > > > > trunk. I only wish I had
> > bandwidth
> > > to
> > > >> > have
> > > >> > > > > done
> > > >> > > > > >> > some
> > > >> > > > > >> > > > real
> > > >> > > > > >> > > > > >> > > kicking
> > > >> > > > > >> > > > > >> > > >> of
> > > >> > > > > >> > > > > >> > > >> > > the
> > > >> > > > > >> > > > > >> > > >> > > > > tires by now. Maybe this week.
> > > >> > > > > >> > > > > >> > > >> > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > (Yes, I'm using some of that
> time
> > > for
> > > >> > this
> > > >> > > > > email
> > > >> > > > > >> > :-)
> > > >> > > > > >> > > > but
> > > >> > > > > >> > > > > I
> > > >> > > > > >> > > > > >> > type
> > > >> > > > > >> > > > > >> > > >> > fast.)
> > > >> > > > > >> > > > > >> > > >> > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > That said, I would like to
> > agitate
> > > >> for
> > > >> > > > making
> > > >> > > > > 2.0
> > > >> > > > > >> > > more
> > > >> > > > > >> > > > > real
> > > >> > > > > >> > > > > >> > and
> > > >> > > > > >> > > > > >> > > >> spend
> > > >> > > > > >> > > > > >> > > >> > > > some
> > > >> > > > > >> > > > > >> > > >> > > > > time on it now that I'm winding
> > > down
> > > >> > with
> > > >> > > > > 0.98. I
> > > >> > > > > >> > > think
> > > >> > > > > >> > > > > >> that
> > > >> > > > > >> > > > > >> > > means
> > > >> > > > > >> > > > > >> > > >> > > > > branching for 2.0 real soon now
> > and
> > > >> even
> > > >> > > > > evicting
> > > >> > > > > >> > > > things
> > > >> > > > > >> > > > > >> from
> > > >> > > > > >> > > > > >> > > 2.0
> > > >> > > > > >> > > > > >> > > >> > > branch
> > > >> > > > > >> > > > > >> > > >> > > > > that aren't finished or stable,
> > > >> leaving
> > > >> > > them
> > > >> > > > > only
> > > >> > > > > >> > > once
> > > >> > > > > >> > > > > >> again
> > > >> > > > > >> > > > > >> > in
> > > >> > > > > >> > > > > >> > > the
> > > >> > > > > >> > > > > >> > > >> > > > master
> > > >> > > > > >> > > > > >> > > >> > > > > branch. Or, maybe just evicting
> > > them.
> > > >> > > Let's
> > > >> > > > > take
> > > >> > > > > >> it
> > > >> > > > > >> > > > case
> > > >> > > > > >> > > > > by
> > > >> > > > > >> > > > > >> > > case.
> > > >> > > > > >> > > > > >> > > >> > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > I think this feature can come
> in
> > > >> > > relatively
> > > >> > > > > >> safely.
> > > >> > > > > >> > > As
> > > >> > > > > >> > > > > >> added
> > > >> > > > > >> > > > > >> > > >> > insurance,
> > > >> > > > > >> > > > > >> > > >> > > > > let's admit the possibility it
> > > could
> > > >> be
> > > >> > > > > reverted
> > > >> > > > > >> on
> > > >> > > > > >> > > the
> > > >> > > > > >> > > > > 2.0
> > > >> > > > > >> > > > > >> > > branch
> > > >> > > > > >> > > > > >> > > >> if
> > > >> > > > > >> > > > > >> > > >> > > > folks
> > > >> > > > > >> > > > > >> > > >> > > > > working on stabilizing 2.0
> decide
> > > to
> > > >> > evict
> > > >> > > > it
> > > >> > > > > >> > because
> > > >> > > > > >> > > > it
> > > >> > > > > >> > > > > is
> > > >> > > > > >> > > > > >> > > >> > unfinished
> > > >> > > > > >> > > > > >> > > >> > > or
> > > >> > > > > >> > > > > >> > > >> > > > > unstable, because that
> certainly
> > > can
> > > >> > > > happen. I
> > > >> > > > > >> > would
> > > >> > > > > >> > > > > >> expect if
> > > >> > > > > >> > > > > >> > > talk
> > > >> > > > > >> > > > > >> > > >> > > like
> > > >> > > > > >> > > > > >> > > >> > > > > that starts, we'd get help
> > > finishing
> > > >> or
> > > >> > > > > >> stabilizing
> > > >> > > > > >> > > > > what's
> > > >> > > > > >> > > > > >> > under
> > > >> > > > > >> > > > > >> > > >> > > > discussion
> > > >> > > > > >> > > > > >> > > >> > > > > for revert. Or, we'd have a
> > revert.
> > > >> > Either
> > > >> > > > way
> > > >> > > > > >> the
> > > >> > > > > >> > > > > outcome
> > > >> > > > > >> > > > > >> is
> > > >> > > > > >> > > > > >> > > >> > > acceptable.
> > > >> > > > > >> > > > > >> > > >> > > > >
> > > >> > > > > >> > > > > >> > > >> > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > On Wed, Sep 7, 2016 at 8:56 AM,
> > > Dima
> > > >> > > Spivak
> > > >> > > > <
> > > >> > > > > >> > > > > >> > > dimaspivak@apache.org
> > > >> > > > > >> > > > > >> > > >> >
> > > >> > > > > >> > > > > >> > > >> > > > wrote:
> > > >> > > > > >> > > > > >> > > >> > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > > I'm not sure that "There is
> > > already
> > > >> > lots
> > > >> > > > of
> > > >> > > > > >> > > > half-baked
> > > >> > > > > >> > > > > >> code
> > > >> > > > > >> > > > > >> > in
> > > >> > > > > >> > > > > >> > > >> the
> > > >> > > > > >> > > > > >> > > >> > > > > branch,
> > > >> > > > > >> > > > > >> > > >> > > > > > so what's the harm in adding
> > > more?"
> > > >> > is a
> > > >> > > > > good
> > > >> > > > > >> > code
> > > >> > > > > >> > > > > commit
> > > >> > > > > >> > > > > >> > > >> > philosophy
> > > >> > > > > >> > > > > >> > > >> > > > for
> > > >> > > > > >> > > > > >> > > >> > > > > a
> > > >> > > > > >> > > > > >> > > >> > > > > > fault-tolerant distributed
> data
> > > >> store.
> > > >> > > ;)
> > > >> > > > > >> > > > > >> > > >> > > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > > More seriously, a lack of
> test
> > > >> > coverage
> > > >> > > > for
> > > >> > > > > >> > > existing
> > > >> > > > > >> > > > > >> > features
> > > >> > > > > >> > > > > >> > > >> > > shouldn't
> > > >> > > > > >> > > > > >> > > >> > > > > be
> > > >> > > > > >> > > > > >> > > >> > > > > > used as justification for
> > > >> introducing
> > > >> > > new
> > > >> > > > > >> > features
> > > >> > > > > >> > > > with
> > > >> > > > > >> > > > > >> the
> > > >> > > > > >> > > > > >> > > same
> > > >> > > > > >> > > > > >> > > >> > > > > > shortcomings. Ultimately,
> it's
> > > the
> > > >> end
> > > >> > > > user
> > > >> > > > > who
> > > >> > > > > >> > > will
> > > >> > > > > >> > > > > feel
> > > >> > > > > >> > > > > >> > the
> > > >> > > > > >> > > > > >> > > >> pain,
> > > >> > > > > >> > > > > >> > > >> > > so
> > > >> > > > > >> > > > > >> > > >> > > > > > shouldn't we do everything we
> > can
> > > >> to
> > > >> > > > > mitigate
> > > >> > > > > >> > that?
> > > >> > > > > >> > > > > >> > > >> > > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > > -Dima
> > > >> > > > > >> > > > > >> > > >> > > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > > On Wed, Sep 7, 2016 at 8:46
> AM,
> > > >> > Vladimir
> > > >> > > > > >> > Rodionov <
> > > >> > > > > >> > > > > >> > > >> > > > > vladrodionov@gmail.com>
> > > >> > > > > >> > > > > >> > > >> > > > > > wrote:
> > > >> > > > > >> > > > > >> > > >> > > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > > > Sean,
> > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > > > * have docs
> > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > > > Agree. We have a doc and
> > backup
> > > >> is
> > > >> > the
> > > >> > > > > most
> > > >> > > > > >> > > > > documented
> > > >> > > > > >> > > > > >> > > feature
> > > >> > > > > >> > > > > >> > > >> > :),
> > > >> > > > > >> > > > > >> > > >> > > we
> > > >> > > > > >> > > > > >> > > >> > > > > > will
> > > >> > > > > >> > > > > >> > > >> > > > > > > release it shortly to
> Apache.
> > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > > > * have sunny-day
> correctness
> > > >> tests
> > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > > > Feature has  close to 60
> test
> > > >> cases,
> > > >> > > > which
> > > >> > > > > >> run
> > > >> > > > > >> > > for
> > > >> > > > > >> > > > > >> approx
> > > >> > > > > >> > > > > >> > 30
> > > >> > > > > >> > > > > >> > > >> min.
> > > >> > > > > >> > > > > >> > > >> > > We
> > > >> > > > > >> > > > > >> > > >> > > > > can
> > > >> > > > > >> > > > > >> > > >> > > > > > > add more, if community do
> not
> > > >> mind
> > > >> > :)
> > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > > > * have
> > > >> > correctness-in-face-of-failure
> > > >> > > > > tests
> > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > > > Any examples of these tests
> > in
> > > >> > > existing
> > > >> > > > > >> > features?
> > > >> > > > > >> > > > In
> > > >> > > > > >> > > > > >> > works,
> > > >> > > > > >> > > > > >> > > we
> > > >> > > > > >> > > > > >> > > >> > > have a
> > > >> > > > > >> > > > > >> > > >> > > > > > clear
> > > >> > > > > >> > > > > >> > > >> > > > > > > understanding of what
> should
> > be
> > > >> done
> > > >> > > by
> > > >> > > > > the
> > > >> > > > > >> > time
> > > >> > > > > >> > > of
> > > >> > > > > >> > > > > 2.0
> > > >> > > > > >> > > > > >> > > >> release.
> > > >> > > > > >> > > > > >> > > >> > > > > > > That is very close goal for
> > us,
> > > >> to
> > > >> > > > verify
> > > >> > > > > IT
> > > >> > > > > >> > > monkey
> > > >> > > > > >> > > > > for
> > > >> > > > > >> > > > > >> > > >> existing
> > > >> > > > > >> > > > > >> > > >> > > > code.
> > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > > > * don't rely on things
> > outside
> > > of
> > > >> > > HBase
> > > >> > > > > for
> > > >> > > > > >> > > normal
> > > >> > > > > >> > > > > >> > operation
> > > >> > > > > >> > > > > >> > > >> > (okay
> > > >> > > > > >> > > > > >> > > >> > > > for
> > > >> > > > > >> > > > > >> > > >> > > > > > > advanced operation)
> > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > > > We do not.
> > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > > > Enormous time has been
> spent
> > > >> already
> > > >> > > on
> > > >> > > > > the
> > > >> > > > > >> > > > > development
> > > >> > > > > >> > > > > >> > and
> > > >> > > > > >> > > > > >> > > >> > testing
> > > >> > > > > >> > > > > >> > > >> > > > the
> > > >> > > > > >> > > > > >> > > >> > > > > > > feature, it has passed our
> > > >> internal
> > > >> > > > tests
> > > >> > > > > and
> > > >> > > > > >> > > many
> > > >> > > > > >> > > > > >> rounds
> > > >> > > > > >> > > > > >> > of
> > > >> > > > > >> > > > > >> > > >> code
> > > >> > > > > >> > > > > >> > > >> > > > > reviews
> > > >> > > > > >> > > > > >> > > >> > > > > > > by HBase committers. We do
> > not
> > > >> mind
> > > >> > if
> > > >> > > > > >> someone
> > > >> > > > > >> > > from
> > > >> > > > > >> > > > > >> HBase
> > > >> > > > > >> > > > > >> > > >> > community
> > > >> > > > > >> > > > > >> > > >> > > > > > > (outside of HW) will review
> > the
> > > >> > code,
> > > >> > > > but
> > > >> > > > > it
> > > >> > > > > >> > will
> > > >> > > > > >> > > > > >> probably
> > > >> > > > > >> > > > > >> > > >> takes
> > > >> > > > > >> > > > > >> > > >> > > > > forever
> > > >> > > > > >> > > > > >> > > >> > > > > > to
> > > >> > > > > >> > > > > >> > > >> > > > > > > wait for volunteer?, the
> > > feature
> > > >> is
> > > >> > > > quite
> > > >> > > > > >> large
> > > >> > > > > >> > > > (1MB+
> > > >> > > > > >> > > > > >> > > >> cumulative
> > > >> > > > > >> > > > > >> > > >> > > > patch)
> > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > > > 2.0 branch is full of half
> > > baked
> > > >> > > > features,
> > > >> > > > > >> most
> > > >> > > > > >> > > of
> > > >> > > > > >> > > > > them
> > > >> > > > > >> > > > > >> > are
> > > >> > > > > >> > > > > >> > > in
> > > >> > > > > >> > > > > >> > > >> > > active
> > > >> > > > > >> > > > > >> > > >> > > > > > > development, therefore I am
> > not
> > > >> > > > following
> > > >> > > > > you
> > > >> > > > > >> > > here,
> > > >> > > > > >> > > > > >> Sean?
> > > >> > > > > >> > > > > >> > > Why
> > > >> > > > > >> > > > > >> > > >> > > > > HBASE-7912
> > > >> > > > > >> > > > > >> > > >> > > > > > is
> > > >> > > > > >> > > > > >> > > >> > > > > > > not good enough yet to be
> > > >> integrated
> > > >> > > > into
> > > >> > > > > 2.0
> > > >> > > > > >> > > > branch?
> > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > > > -Vlad
> > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > > > On Wed, Sep 7, 2016 at 8:23
> > AM,
> > > >> Sean
> > > >> > > > > Busbey <
> > > >> > > > > >> > > > > >> > > busbey@apache.org
> > > >> > > > > >> > > > > >> > > >> >
> > > >> > > > > >> > > > > >> > > >> > > > wrote:
> > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > > > > On Tue, Sep 6, 2016 at
> > 10:36
> > > >> PM,
> > > >> > > Josh
> > > >> > > > > >> Elser <
> > > >> > > > > >> > > > > >> > > >> > > josh.elser@gmail.com>
> > > >> > > > > >> > > > > >> > > >> > > > > > > wrote:
> > > >> > > > > >> > > > > >> > > >> > > > > > > > > So, the answer to
> Sean's
> > > >> > original
> > > >> > > > > >> question
> > > >> > > > > >> > is
> > > >> > > > > >> > > > "as
> > > >> > > > > >> > > > > >> > > robust as
> > > >> > > > > >> > > > > >> > > >> > > > > snapshots
> > > >> > > > > >> > > > > >> > > >> > > > > > > > > presently are"?
> > > >> (independence of
> > > >> > > > > >> > > backup/restore
> > > >> > > > > >> > > > > >> > failure
> > > >> > > > > >> > > > > >> > > >> > > tolerance
> > > >> > > > > >> > > > > >> > > >> > > > > > from
> > > >> > > > > >> > > > > >> > > >> > > > > > > > > snapshot failure
> > tolerance)
> > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > > > > > Is this just a question
> > WRT
> > > >> > > context
> > > >> > > > of
> > > >> > > > > >> the
> > > >> > > > > >> > > > > change,
> > > >> > > > > >> > > > > >> or
> > > >> > > > > >> > > > > >> > > is it
> > > >> > > > > >> > > > > >> > > >> > > means
> > > >> > > > > >> > > > > >> > > >> > > > > > for a
> > > >> > > > > >> > > > > >> > > >> > > > > > > > veto
> > > >> > > > > >> > > > > >> > > >> > > > > > > > > from you, Sean? Just
> > trying
> > > >> to
> > > >> > > make
> > > >> > > > > sure
> > > >> > > > > >> > I'm
> > > >> > > > > >> > > > > >> following
> > > >> > > > > >> > > > > >> > > >> along
> > > >> > > > > >> > > > > >> > > >> > > > > > > adequately.
> > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > > > > I'd say ATM I'm -0,
> > bordering
> > > >> on
> > > >> > -1
> > > >> > > > but
> > > >> > > > > not
> > > >> > > > > >> > for
> > > >> > > > > >> > > > > >> reasons
> > > >> > > > > >> > > > > >> > I
> > > >> > > > > >> > > > > >> > > can
> > > >> > > > > >> > > > > >> > > >> > > > > > articulate
> > > >> > > > > >> > > > > >> > > >> > > > > > > > well.
> > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > > > > Here's an attempt.
> > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > > > > We've been trying to
> move,
> > > as a
> > > >> > > > > community,
> > > >> > > > > >> > > > towards
> > > >> > > > > >> > > > > >> > > minimizing
> > > >> > > > > >> > > > > >> > > >> > > risk
> > > >> > > > > >> > > > > >> > > >> > > > to
> > > >> > > > > >> > > > > >> > > >> > > > > > > > downstream folks by
> getting
> > > >> > > "complete
> > > >> > > > > >> enough
> > > >> > > > > >> > > for
> > > >> > > > > >> > > > > use"
> > > >> > > > > >> > > > > >> > > gates
> > > >> > > > > >> > > > > >> > > >> in
> > > >> > > > > >> > > > > >> > > >> > > > place
> > > >> > > > > >> > > > > >> > > >> > > > > > > > before we introduce new
> > > >> features.
> > > >> > > This
> > > >> > > > > was
> > > >> > > > > >> > > > spurred
> > > >> > > > > >> > > > > >> by a
> > > >> > > > > >> > > > > >> > > some
> > > >> > > > > >> > > > > >> > > >> > > > features
> > > >> > > > > >> > > > > >> > > >> > > > > > > > getting in half-baked and
> > > never
> > > >> > > making
> > > >> > > > > it
> > > >> > > > > >> to
> > > >> > > > > >> > > "can
> > > >> > > > > >> > > > > >> really
> > > >> > > > > >> > > > > >> > > use"
> > > >> > > > > >> > > > > >> > > >> > > > status
> > > >> > > > > >> > > > > >> > > >> > > > > > > > (I'm thinking of
> > distributed
> > > >> log
> > > >> > > > replay
> > > >> > > > > and
> > > >> > > > > >> > the
> > > >> > > > > >> > > > > >> zk-less
> > > >> > > > > >> > > > > >> > > >> > > assignment
> > > >> > > > > >> > > > > >> > > >> > > > > > > > stuff, I don't recall if
> > > there
> > > >> was
> > > >> > > > > more).
> > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > > > > The gates, generally,
> > > included
> > > >> > > things
> > > >> > > > > like:
> > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > > > > * have docs
> > > >> > > > > >> > > > > >> > > >> > > > > > > > * have sunny-day
> > correctness
> > > >> tests
> > > >> > > > > >> > > > > >> > > >> > > > > > > > * have
> > > >> > > correctness-in-face-of-failure
> > > >> > > > > tests
> > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't rely on things
> > > outside
> > > >> of
> > > >> > > > HBase
> > > >> > > > > for
> > > >> > > > > >> > > > normal
> > > >> > > > > >> > > > > >> > > operation
> > > >> > > > > >> > > > > >> > > >> > > (okay
> > > >> > > > > >> > > > > >> > > >> > > > > for
> > > >> > > > > >> > > > > >> > > >> > > > > > > > advanced operation)
> > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > > > > As an example, we kept
> the
> > > MOB
> > > >> > work
> > > >> > > > off
> > > >> > > > > in
> > > >> > > > > >> a
> > > >> > > > > >> > > > branch
> > > >> > > > > >> > > > > >> and
> > > >> > > > > >> > > > > >> > > out
> > > >> > > > > >> > > > > >> > > >> of
> > > >> > > > > >> > > > > >> > > >> > > > master
> > > >> > > > > >> > > > > >> > > >> > > > > > > > until it could pass these
> > > >> > criteria.
> > > >> > > > The
> > > >> > > > > big
> > > >> > > > > >> > > > > exemption
> > > >> > > > > >> > > > > >> > > we've
> > > >> > > > > >> > > > > >> > > >> had
> > > >> > > > > >> > > > > >> > > >> > > to
> > > >> > > > > >> > > > > >> > > >> > > > > > > > this was the hbase-spark
> > > >> > > integration,
> > > >> > > > > where
> > > >> > > > > >> > we
> > > >> > > > > >> > > > all
> > > >> > > > > >> > > > > >> > agreed
> > > >> > > > > >> > > > > >> > > it
> > > >> > > > > >> > > > > >> > > >> > > could
> > > >> > > > > >> > > > > >> > > >> > > > > > > > land in master because it
> > was
> > > >> very
> > > >> > > > well
> > > >> > > > > >> > > isolated
> > > >> > > > > >> > > > > (the
> > > >> > > > > >> > > > > >> > > slide
> > > >> > > > > >> > > > > >> > > >> > away
> > > >> > > > > >> > > > > >> > > >> > > > from
> > > >> > > > > >> > > > > >> > > >> > > > > > > > including docs as a
> > > first-class
> > > >> > part
> > > >> > > > of
> > > >> > > > > >> > > building
> > > >> > > > > >> > > > up
> > > >> > > > > >> > > > > >> that
> > > >> > > > > >> > > > > >> > > >> > > > integration
> > > >> > > > > >> > > > > >> > > >> > > > > > > > has led me to doubt the
> > > wisdom
> > > >> of
> > > >> > > this
> > > >> > > > > >> > > decision).
> > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > > > > We've also been treating
> > > >> inclusion
> > > >> > > in
> > > >> > > > a
> > > >> > > > > >> > > "probably
> > > >> > > > > >> > > > > >> will
> > > >> > > > > >> > > > > >> > be
> > > >> > > > > >> > > > > >> > > >> > > released
> > > >> > > > > >> > > > > >> > > >> > > > to
> > > >> > > > > >> > > > > >> > > >> > > > > > > > downstream" branches as a
> > > >> higher
> > > >> > > bar,
> > > >> > > > > >> > requiring
> > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't moderately impact
> > > >> > > performance
> > > >> > > > > when
> > > >> > > > > >> > the
> > > >> > > > > >> > > > > >> feature
> > > >> > > > > >> > > > > >> > > isn't
> > > >> > > > > >> > > > > >> > > >> in
> > > >> > > > > >> > > > > >> > > >> > > use
> > > >> > > > > >> > > > > >> > > >> > > > > > > > * don't severely impact
> > > >> > performance
> > > >> > > > when
> > > >> > > > > >> the
> > > >> > > > > >> > > > > feature
> > > >> > > > > >> > > > > >> is
> > > >> > > > > >> > > > > >> > in
> > > >> > > > > >> > > > > >> > > >> use
> > > >> > > > > >> > > > > >> > > >> > > > > > > > * either default-to-on or
> > > show
> > > >> > > enough
> > > >> > > > > >> demand
> > > >> > > > > >> > to
> > > >> > > > > >> > > > > >> believe
> > > >> > > > > >> > > > > >> > a
> > > >> > > > > >> > > > > >> > > >> > > > non-trivial
> > > >> > > > > >> > > > > >> > > >> > > > > > > > number of folks will turn
> > the
> > > >> > > feature
> > > >> > > > on
> > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > > > > The above has kept MOB
> and
> > > >> > > hbase-spark
> > > >> > > > > >> > > > integration
> > > >> > > > > >> > > > > >> out
> > > >> > > > > >> > > > > >> > of
> > > >> > > > > >> > > > > >> > > >> > > branch-1,
> > > >> > > > > >> > > > > >> > > >> > > > > > > > presumably while they've
> > > >> "gotten
> > > >> > > more
> > > >> > > > > >> stable"
> > > >> > > > > >> > > in
> > > >> > > > > >> > > > > >> master
> > > >> > > > > >> > > > > >> > > from
> > > >> > > > > >> > > > > >> > > >> > the
> > > >> > > > > >> > > > > >> > > >> > > > odd
> > > >> > > > > >> > > > > >> > > >> > > > > > > > vendor inclusion.
> > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > > > > Are we going to have a
> 2.0
> > > >> release
> > > >> > > > > before
> > > >> > > > > >> the
> > > >> > > > > >> > > end
> > > >> > > > > >> > > > > of
> > > >> > > > > >> > > > > >> the
> > > >> > > > > >> > > > > >> > > >> year?
> > > >> > > > > >> > > > > >> > > >> > > > We're
> > > >> > > > > >> > > > > >> > > >> > > > > > > > coming up on 1.5 years
> > since
> > > >> the
> > > >> > > > > release of
> > > >> > > > > >> > > > version
> > > >> > > > > >> > > > > >> 1.0;
> > > >> > > > > >> > > > > >> > > >> seems
> > > >> > > > > >> > > > > >> > > >> > > like
> > > >> > > > > >> > > > > >> > > >> > > > > > > > it's about time, though I
> > > >> haven't
> > > >> > > seen
> > > >> > > > > any
> > > >> > > > > >> > > > concrete
> > > >> > > > > >> > > > > >> > plans
> > > >> > > > > >> > > > > >> > > >> this
> > > >> > > > > >> > > > > >> > > >> > > > year.
> > > >> > > > > >> > > > > >> > > >> > > > > > > > Presuming we are going to
> > > have
> > > >> one
> > > >> > > by
> > > >> > > > > the
> > > >> > > > > >> end
> > > >> > > > > >> > > of
> > > >> > > > > >> > > > > the
> > > >> > > > > >> > > > > >> > > year, it
> > > >> > > > > >> > > > > >> > > >> > > > seems a
> > > >> > > > > >> > > > > >> > > >> > > > > > > > bit close to still be
> > adding
> > > in
> > > >> > > > > "features
> > > >> > > > > >> > that
> > > >> > > > > >> > > > need
> > > >> > > > > >> > > > > >> > > maturing"
> > > >> > > > > >> > > > > >> > > >> > on
> > > >> > > > > >> > > > > >> > > >> > > > the
> > > >> > > > > >> > > > > >> > > >> > > > > > > > branch.
> > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > > > > The lack of a concrete
> plan
> > > for
> > > >> > 2.0
> > > >> > > > > keeps
> > > >> > > > > >> me
> > > >> > > > > >> > > from
> > > >> > > > > >> > > > > >> > > considering
> > > >> > > > > >> > > > > >> > > >> > > these
> > > >> > > > > >> > > > > >> > > >> > > > > > > > things blocker at the
> > moment.
> > > >> But
> > > >> > I
> > > >> > > > know
> > > >> > > > > >> > first
> > > >> > > > > >> > > > hand
> > > >> > > > > >> > > > > >> how
> > > >> > > > > >> > > > > >> > > much
> > > >> > > > > >> > > > > >> > > >> > > > trouble
> > > >> > > > > >> > > > > >> > > >> > > > > > > > folks have had with other
> > > >> features
> > > >> > > > that
> > > >> > > > > >> have
> > > >> > > > > >> > > gone
> > > >> > > > > >> > > > > >> into
> > > >> > > > > >> > > > > >> > > >> > downstream
> > > >> > > > > >> > > > > >> > > >> > > > > > > > facing releases without
> > > >> robustness
> > > >> > > > > checks
> > > >> > > > > >> > (i.e.
> > > >> > > > > >> > > > > >> > > replication),
> > > >> > > > > >> > > > > >> > > >> > and
> > > >> > > > > >> > > > > >> > > >> > > > I'm
> > > >> > > > > >> > > > > >> > > >> > > > > > > > concerned about what
> we're
> > > >> setting
> > > >> > > up
> > > >> > > > if
> > > >> > > > > >> 2.0
> > > >> > > > > >> > > goes
> > > >> > > > > >> > > > > out
> > > >> > > > > >> > > > > >> > with
> > > >> > > > > >> > > > > >> > > >> this
> > > >> > > > > >> > > > > >> > > >> > > > > > > > feature in its current
> > state.
> > > >> > > > > >> > > > > >> > > >> > > > > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > >
> > > >> > > > > >> > > > > >> > > >> > > > >
> > > >> > > > > >> > > > > >> > > >> > > > >
> > > >> > > > > >> > > > > >> > > >> > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > --
> > > >> > > > > >> > > > > >> > > >> > > > > Best regards,
> > > >> > > > > >> > > > > >> > > >> > > > >
> > > >> > > > > >> > > > > >> > > >> > > > >    - Andy
> > > >> > > > > >> > > > > >> > > >> > > > >
> > > >> > > > > >> > > > > >> > > >> > > > > Problems worthy of attack prove
> > > their
> > > >> > > worth
> > > >> > > > by
> > > >> > > > > >> > > hitting
> > > >> > > > > >> > > > > >> back. -
> > > >> > > > > >> > > > > >> > > Piet
> > > >> > > > > >> > > > > >> > > >> > > Hein
> > > >> > > > > >> > > > > >> > > >> > > > > (via Tom White)
> > > >> > > > > >> > > > > >> > > >> > > > >
> > > >> > > > > >> > > > > >> > > >> > > >
> > > >> > > > > >> > > > > >> > > >> > >
> > > >> > > > > >> > > > > >> > > >> > >
> > > >> > > > > >> > > > > >> > > >> > >
> > > >> > > > > >> > > > > >> > > >> > > --
> > > >> > > > > >> > > > > >> > > >> > > Best regards,
> > > >> > > > > >> > > > > >> > > >> > >
> > > >> > > > > >> > > > > >> > > >> > >    - Andy
> > > >> > > > > >> > > > > >> > > >> > >
> > > >> > > > > >> > > > > >> > > >> > > Problems worthy of attack prove
> their
> > > >> worth
> > > >> > by
> > > >> > > > > >> hitting
> > > >> > > > > >> > > > back.
> > > >> > > > > >> > > > > -
> > > >> > > > > >> > > > > >> > Piet
> > > >> > > > > >> > > > > >> > > >> Hein
> > > >> > > > > >> > > > > >> > > >> > > (via Tom White)
> > > >> > > > > >> > > > > >> > > >> > >
> > > >> > > > > >> > > > > >> > > >> >
> > > >> > > > > >> > > > > >> > > >>
> > > >> > > > > >> > > > > >> > >
> > > >> > > > > >> > > > > >> >
> > > >> > > > > >> > > > > >>
> > > >> > > > > >> > > > > >
> > > >> > > > > >> > > > > >
> > > >> > > > > >> > > > >
> > > >> > > > > >> > > >
> > > >> > > > > >> > >
> > > >> > > > > >> >
> > > >> > > > > >>
> > > >> > > > >
> > > >> > > > >
> > > >> > > > >
> > > >> > > > > --
> > > >> > > > > busbey
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > > >
> > > >
> > >
> >
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Vladimir Rodionov <vl...@gmail.com>]

Michael,

Its in HBASE-7912

This is tip of git log:

commit a072f6f49a26a7259ff2aaef6cb56d85eb592482
Author: Frank Welsch <fw...@jps.net>
Date:   Fri Sep 23 18:00:42 2016 -0400

    HBASE-16574 Book updates for backup and restore

commit b14e2ab1c24e65ff88dd4c579acf83cb4ed0605e
Author: tedyu <yu...@gmail.com>
Date:   Wed Oct 5 16:29:40 2016 -0700

    HBASE-16727 Backup refactoring: remove MR dependencies from HMaster
(Vladimir Rodionov)

-Vlad

On Tue, Oct 11, 2016 at 1:44 PM, Stack <st...@duboce.net> wrote:

> Which branch do I check out to try it? HBASE-7912 is not it. I don't see an
> HBASE-16727...
> Thanks,
> M
>
>
> On Tue, Oct 11, 2016 at 11:41 AM, Vladimir Rodionov <
> vladrodionov@gmail.com>
> wrote:
>
> > The last patch is on review board:
> > https://reviews.apache.org/r/52748
> >
> > On Mon, Oct 10, 2016 at 3:52 PM, Vladimir Rodionov <
> vladrodionov@gmail.com
> > >
> > wrote:
> >
> > > >> How hard to put in an hbase-backup module? hbase-server is fat
> enough
> > > >> already. Could be done as a follow-up.
> > >
> > > https://issues.apache.org/jira/browse/HBASE-16727?
> > > focusedCommentId=15531237&page=com.atlassian.jira.
> > > plugin.system.issuetabpanels:comment-tabpanel#comment-15531237
> > >
> > > Can we do merge first? Then we can discuss separate module.
> > >
> > > -Vlad
> > >
> > > On Mon, Oct 10, 2016 at 3:44 PM, Ted Yu <yu...@gmail.com> wrote:
> > >
> > >> Looks like the first quote was cut off.
> > >> The original sentence was:
> > >>
> > >> bq. no mapreduce job launched from master or region server.
> > >>
> > >> mapreduce job is launched from the node where command line tool is
> run.
> > >>
> > >> On Mon, Oct 10, 2016 at 3:38 PM, Stack <st...@duboce.net> wrote:
> > >>
> > >> > bq. launched from master or region server.
> > >> >
> > >> > What does this mean please? Has to be run from Master or
> RegionServer?
> > >> Can
> > >> > it be run from another node altogether?
> > >> >
> > >> > On Mon, Oct 10, 2016 at 1:44 PM, Vladimir Rodionov <
> > >> vladrodionov@gmail.com
> > >> > >
> > >> > wrote:
> > >> >
> > >> > > >> mapreduce dependency has been moved to client side - no
> mapreduce
> > >> job
> > >> > >
> > >> > > 1. We have no code in the client module anymore, due to dependency
> > on
> > >> > > internal server API (HFile and WAL access).
> > >> > > 2. Backup/ restore are client - driven operations, but all the
> code
> > >> > resides
> > >> > > in the server module
> > >> > >
> > >> >
> > >> > How hard to put in an hbase-backup module? hbase-server is fat
> enough
> > >> > already. Could be done as a follow-up.
> > >> >
> > >> > Thanks,
> > >> > St.Ack
> > >> >
> > >> >
> > >> >
> > >> > > 3. No MR in Master, no procedure - driven execution.
> > >> > > 4. Old good MR from command-line.
> > >> > > 5. Security was simplified and now only super-user is allowed to
> run
> > >> > > backup/restores.
> > >> > > 6. HBase Backup API was gone due to 1. Now only command-line
> access
> > to
> > >> > > backup tools.
> > >> > >
> > >> > > These consequences of refactoring has been discussed in
> HBASE-16727.
> > >> > >
> > >> > > -Vlad
> > >> > >
> > >> > >
> > >> > >
> > >> > >
> > >> > > On Mon, Oct 10, 2016 at 1:31 PM, Ted Yu <yu...@gmail.com>
> > wrote:
> > >> > >
> > >> > > > Reviving this thread.
> > >> > > >
> > >> > > > The following has taken place:
> > >> > > >
> > >> > > > mapreduce dependency has been moved to client side - no
> mapreduce
> > >> job
> > >> > > > launched from master or region server.
> > >> > > > document patch (HBASE-16574) has been integrated.
> > >> > > > Updated mega patch has been attached to HBASE-14123: this covers
> > the
> > >> > > > refactor in #1 above and the protobuf 3 merge.
> > >> > > >
> > >> > > > If community has more feedback on the merge proposal, I would
> love
> > >> to
> > >> > > hear
> > >> > > > it.
> > >> > > >
> > >> > > > Thanks
> > >> > > >
> > >> > > > On Thu, Sep 22, 2016 at 10:31 AM, Sean Busbey <
> > busbey@cloudera.com>
> > >> > > wrote:
> > >> > > >
> > >> > > > > I'd like to see the docs proposed on HBASE-16574 integrated
> into
> > >> our
> > >> > > > > project's documentation prior to merge.
> > >> > > > >
> > >> > > > > On Thu, Sep 22, 2016 at 9:02 AM, Ted Yu <yu...@gmail.com>
> > >> wrote:
> > >> > > > > > This feature can be marked experimental due to some
> > limitations
> > >> > such
> > >> > > as
> > >> > > > > > security.
> > >> > > > > >
> > >> > > > > > Your previous round of comments have been addressed.
> > >> > > > > > Command line tool has gone through:
> > >> > > > > >
> > >> > > > > > HBASE-16620 Fix backup command-line tool usability issues
> > >> > > > > > HBASE-16655 hbase backup describe with incorrect backup id
> > >> results
> > >> > in
> > >> > > > NPE
> > >> > > > > >
> > >> > > > > > The updated doc has been attached to HBASE-16574.
> > >> > > > > >
> > >> > > > > > Cheers
> > >> > > > > >
> > >> > > > > > On Thu, Sep 22, 2016 at 8:53 AM, Stack <st...@duboce.net>
> > >> wrote:
> > >> > > > > >
> > >> > > > > >> On Wed, Sep 21, 2016 at 7:43 AM, Ted Yu <
> yuzhihong@gmail.com
> > >
> > >> > > wrote:
> > >> > > > > >>
> > >> > > > > >> > Are there more (review) comments ?
> > >> > > > > >> >
> > >> > > > > >> >
> > >> > > > > >> Are outstanding comments addressed?
> > >> > > > > >>
> > >> > > > > >> I don't see answer to my 'is this experimental/will it be
> > >> marked
> > >> > > > > >> experimental' question.
> > >> > > > > >>
> > >> > > > > >> I ran into some issues trying to use the feature and
> > suggested
> > >> > that
> > >> > > a
> > >> > > > > >> feature likes this needs polish else it'll just rot,
> unused.
> > >> Has
> > >> > > > polish
> > >> > > > > >> been applied? All ready for another 'user' test? Suggest
> that
> > >> you
> > >> > > > update
> > >> > > > > >> here going forward for the benefit of those trying to
> follow
> > >> along
> > >> > > and
> > >> > > > > who
> > >> > > > > >> are not watching JIRA change fly-by.
> > >> > > > > >>
> > >> > > > > >> It looks like doc got a revision -- I have to check -- to
> > take
> > >> on
> > >> > > > > >> suggestion made above but again, suggest, that this thread
> > gets
> > >> > > > updated.
> > >> > > > > >>
> > >> > > > > >> Thanks,
> > >> > > > > >> St.Ack
> > >> > > > > >>
> > >> > > > > >>
> > >> > > > > >>
> > >> > > > > >> > Thanks
> > >> > > > > >> >
> > >> > > > > >> > On Tue, Sep 20, 2016 at 10:02 AM, Devaraj Das <
> > >> > > ddas@hortonworks.com
> > >> > > > >
> > >> > > > > >> > wrote:
> > >> > > > > >> >
> > >> > > > > >> > > Just reviving this thread. Thanks Sean, Stack, Dima,
> and
> > >> > others
> > >> > > > for
> > >> > > > > the
> > >> > > > > >> > > thorough reviews and testing. Thanks Ted and Vlad for
> > >> taking
> > >> > > care
> > >> > > > of
> > >> > > > > >> the
> > >> > > > > >> > > feedback. Are we all good to do the merge now? Rather
> do
> > >> > sooner
> > >> > > > than
> > >> > > > > >> > later.
> > >> > > > > >> > > ________________________________________
> > >> > > > > >> > > From: saint.ack@gmail.com <sa...@gmail.com> on
> > behalf
> > >> of
> > >> > > > Stack
> > >> > > > > <
> > >> > > > > >> > > stack@duboce.net>
> > >> > > > > >> > > Sent: Monday, September 12, 2016 1:18 PM
> > >> > > > > >> > > To: HBase Dev List
> > >> > > > > >> > > Subject: Re: [DISCUSSION] Merge Backup / Restore -
> Branch
> > >> > > > HBASE-7912
> > >> > > > > >> > >
> > >> > > > > >> > > On Mon, Sep 12, 2016 at 12:19 PM, Ted Yu <
> > >> yuzhihong@gmail.com
> > >> > >
> > >> > > > > wrote:
> > >> > > > > >> > >
> > >> > > > > >> > > > Mega patch (rev 18) is on HBASE-14123.
> > >> > > > > >> > > >
> > >> > > > > >> > > > Please comment on HBASE-14123 on how you want to
> > review.
> > >> > > > > >> > > >
> > >> > > > > >> > >
> > >> > > > > >> > >
> > >> > > > > >> > > Yeah. That was my lost tab. Last rb was 6 months ago.
> > >> Suggest
> > >> > > > > updating
> > >> > > > > >> > it.
> > >> > > > > >> > > RB is pretty good for review. Patch is only 1.5M so
> > should
> > >> be
> > >> > > > fine.
> > >> > > > > >> > >
> > >> > > > > >> > > St.Ack
> > >> > > > > >> > >
> > >> > > > > >> > >
> > >> > > > > >> > > >
> > >> > > > > >> > > > Thanks
> > >> > > > > >> > > >
> > >> > > > > >> > > > On Mon, Sep 12, 2016 at 12:15 PM, Stack <
> > >> stack@duboce.net>
> > >> > > > wrote:
> > >> > > > > >> > > >
> > >> > > > > >> > > > > On review of the 'patch', do I just compare the
> > branch
> > >> to
> > >> > > > > master or
> > >> > > > > >> > is
> > >> > > > > >> > > > > there a megapatch posted somewhere (I think I saw
> one
> > >> but
> > >> > it
> > >> > > > > seemed
> > >> > > > > >> > > stale
> > >> > > > > >> > > > > and then I 'lost' the tab). Sorry for dumb
> question.
> > >> > > > > >> > > > > St.Ack
> > >> > > > > >> > > > >
> > >> > > > > >> > > > > On Mon, Sep 12, 2016 at 12:01 PM, Stack <
> > >> stack@duboce.net
> > >> > >
> > >> > > > > wrote:
> > >> > > > > >> > > > >
> > >> > > > > >> > > > > > Late to the game. A few comments after rereading
> > this
> > >> > > thread
> > >> > > > > as a
> > >> > > > > >> > > > 'user'.
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > > > + Before merge, a user-facing feature like this
> > >> should
> > >> > > work
> > >> > > > > (If
> > >> > > > > >> > this
> > >> > > > > >> > > is
> > >> > > > > >> > > > > "higher-bar
> > >> > > > > >> > > > > > for new features", bring it on -- smile).
> > >> > > > > >> > > > > > + As a user, I tried the branch with tools after
> > >> > reviewing
> > >> > > > the
> > >> > > > > >> > > > > just-posted
> > >> > > > > >> > > > > > doc. I had an 'interesting' experience (left
> > >> comments up
> > >> > > on
> > >> > > > > >> > issue). I
> > >> > > > > >> > > > > think
> > >> > > > > >> > > > > > the tooling/doc. important to get right. If it
> > breaks
> > >> > > easily
> > >> > > > > or
> > >> > > > > >> is
> > >> > > > > >> > > > > > inconsistent (or lacks 'polish'), operators will
> > >> judge
> > >> > the
> > >> > > > > whole
> > >> > > > > >> > > > > > backup/restore tooling chain as not trustworthy
> and
> > >> > > abandon
> > >> > > > > it.
> > >> > > > > >> > Lets
> > >> > > > > >> > > > not
> > >> > > > > >> > > > > > have this happen to this feature.
> > >> > > > > >> > > > > > + Matteo's suggestion (with a helpful starter
> list)
> > >> that
> > >> > > > there
> > >> > > > > >> > needs
> > >> > > > > >> > > to
> > >> > > > > >> > > > > be
> > >> > > > > >> > > > > > explicit qualification on what is actually being
> > >> > delivered
> > >> > > > --
> > >> > > > > >> > > > including a
> > >> > > > > >> > > > > > listing of limitations (some look serious such as
> > >> data
> > >> > > bleed
> > >> > > > > from
> > >> > > > > >> > > other
> > >> > > > > >> > > > > > regions in WALs, but maybe I don't care for my
> use
> > >> > > case...)
> > >> > > > --
> > >> > > > > >> > needs
> > >> > > > > >> > > to
> > >> > > > > >> > > > > > accompany the merge. Lets fold them into the user
> > >> doc.
> > >> > in
> > >> > > > the
> > >> > > > > >> > > technical
> > >> > > > > >> > > > > > overview area as suggested so user expectations
> are
> > >> > > properly
> > >> > > > > >> > managed
> > >> > > > > >> > > > > > (otherwise, they expect the world and will just
> > give
> > >> up
> > >> > > when
> > >> > > > > we
> > >> > > > > >> > fall
> > >> > > > > >> > > > > > short). Vladimir did a list of what is in each of
> > the
> > >> > > phases
> > >> > > > > >> above
> > >> > > > > >> > > > which
> > >> > > > > >> > > > > > would serve as a good start.
> > >> > > > > >> > > > > > + Is this feature 'experimental' (Matteo asks
> > above).
> > >> > I'd
> > >> > > > > prefer
> > >> > > > > >> it
> > >> > > > > >> > > is
> > >> > > > > >> > > > > > not. If it is, it should be labelled all over
> that
> > >> it is
> > >> > > > so. I
> > >> > > > > >> see
> > >> > > > > >> > > > > current
> > >> > > > > >> > > > > > state called out as a '... technical preview
> > >> feature'.
> > >> > > Does
> > >> > > > > this
> > >> > > > > >> > mean
> > >> > > > > >> > > > > > not-for-users?
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > > > St.Ack
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > > > On Mon, Sep 12, 2016 at 8:03 AM, Ted Yu <
> > >> > > > yuzhihong@gmail.com>
> > >> > > > > >> > wrote:
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > > >> Sean:
> > >> > > > > >> > > > > >> Do you have more comments ?
> > >> > > > > >> > > > > >>
> > >> > > > > >> > > > > >> Cheers
> > >> > > > > >> > > > > >>
> > >> > > > > >> > > > > >> On Fri, Sep 9, 2016 at 1:42 PM, Vladimir
> Rodionov
> > <
> > >> > > > > >> > > > > vladrodionov@gmail.com
> > >> > > > > >> > > > > >> >
> > >> > > > > >> > > > > >> wrote:
> > >> > > > > >> > > > > >>
> > >> > > > > >> > > > > >> > Sean,
> > >> > > > > >> > > > > >> >
> > >> > > > > >> > > > > >> > Backup/Restore can fail due to various
> reasons:
> > >> > network
> > >> > > > > outage
> > >> > > > > >> > > > > (cluster
> > >> > > > > >> > > > > >> > wide), various time-outs in HBase and HDFS
> > layer,
> > >> M/R
> > >> > > > > failure
> > >> > > > > >> > due
> > >> > > > > >> > > to
> > >> > > > > >> > > > > >> "HDFS
> > >> > > > > >> > > > > >> > exceeded quota", user error (manual deletion
> of
> > >> data)
> > >> > > and
> > >> > > > > so
> > >> > > > > >> on
> > >> > > > > >> > so
> > >> > > > > >> > > > on.
> > >> > > > > >> > > > > >> That
> > >> > > > > >> > > > > >> > is impossible to enumerate all possible types
> of
> > >> > > failures
> > >> > > > > in a
> > >> > > > > >> > > > > >> distributed
> > >> > > > > >> > > > > >> > system - that is not our goal/task.
> > >> > > > > >> > > > > >> >
> > >> > > > > >> > > > > >> > We focus completely on backup system table
> > >> > consistency
> > >> > > > in a
> > >> > > > > >> > > presence
> > >> > > > > >> > > > > of
> > >> > > > > >> > > > > >> any
> > >> > > > > >> > > > > >> > type of failure. That is what I call
> "tolerance
> > to
> > >> > > > > failures".
> > >> > > > > >> > > > > >> >
> > >> > > > > >> > > > > >> > On a failure:
> > >> > > > > >> > > > > >> >
> > >> > > > > >> > > > > >> > BACKUP. All backup system information (prior
> to
> > >> > backup)
> > >> > > > > will
> > >> > > > > >> be
> > >> > > > > >> > > > > restored
> > >> > > > > >> > > > > >> > and all temporary data, related to a failed
> > >> session,
> > >> > in
> > >> > > > > HDFS
> > >> > > > > >> > will
> > >> > > > > >> > > be
> > >> > > > > >> > > > > >> > deleted
> > >> > > > > >> > > > > >> > RESTORE. We do not care about system data,
> > because
> > >> > > > restore
> > >> > > > > >> does
> > >> > > > > >> > > not
> > >> > > > > >> > > > > >> change
> > >> > > > > >> > > > > >> > it. Temporary data in HDFS will be cleaned up
> > and
> > >> > table
> > >> > > > > will
> > >> > > > > >> be
> > >> > > > > >> > > in a
> > >> > > > > >> > > > > >> state
> > >> > > > > >> > > > > >> > back to where it was before operation started.
> > >> > > > > >> > > > > >> >
> > >> > > > > >> > > > > >> > This is what user should expect in case of a
> > >> failure.
> > >> > > > > >> > > > > >> >
> > >> > > > > >> > > > > >> > -Vlad
> > >> > > > > >> > > > > >> >
> > >> > > > > >> > > > > >> >
> > >> > > > > >> > > > > >> > -Vlad
> > >> > > > > >> > > > > >> >
> > >> > > > > >> > > > > >> > On Fri, Sep 9, 2016 at 12:56 PM, Sean Busbey <
> > >> > > > > >> busbey@apache.org
> > >> > > > > >> > >
> > >> > > > > >> > > > > wrote:
> > >> > > > > >> > > > > >> >
> > >> > > > > >> > > > > >> > > Failing in a consistent way, with docs that
> > >> explain
> > >> > > the
> > >> > > > > >> > various
> > >> > > > > >> > > > > >> > > expected failures would be sufficient.
> > >> > > > > >> > > > > >> > >
> > >> > > > > >> > > > > >> > > On Fri, Sep 9, 2016 at 12:16 PM, Vladimir
> > >> Rodionov
> > >> > > > > >> > > > > >> > > <vl...@gmail.com> wrote:
> > >> > > > > >> > > > > >> > > > Do not worry Sean, doc is coming today as
> a
> > >> > preview
> > >> > > > and
> > >> > > > > >> our
> > >> > > > > >> > > > writer
> > >> > > > > >> > > > > >> > Frank
> > >> > > > > >> > > > > >> > > > will be working on a putting  it into
> Apache
> > >> > repo.
> > >> > > > > >> Timeline
> > >> > > > > >> > > > > depends
> > >> > > > > >> > > > > >> on
> > >> > > > > >> > > > > >> > > > Franks schedule but I hope we will get it
> > >> rather
> > >> > > > sooner
> > >> > > > > >> than
> > >> > > > > >> > > > > later.
> > >> > > > > >> > > > > >> > > >
> > >> > > > > >> > > > > >> > > > As for failure testing, we are focusing
> only
> > >> on a
> > >> > > > > >> consistent
> > >> > > > > >> > > > state
> > >> > > > > >> > > > > >> of
> > >> > > > > >> > > > > >> > > > backup system data in a presence of any
> type
> > >> of
> > >> > > > > failures,
> > >> > > > > >> We
> > >> > > > > >> > > are
> > >> > > > > >> > > > > not
> > >> > > > > >> > > > > >> > > going
> > >> > > > > >> > > > > >> > > > to implement  anything more "fancy", than
> > >> that.
> > >> > We
> > >> > > > > allow
> > >> > > > > >> > both:
> > >> > > > > >> > > > > >> backup
> > >> > > > > >> > > > > >> > and
> > >> > > > > >> > > > > >> > > > restore to fail. What we do not allow is
> to
> > >> have
> > >> > > > system
> > >> > > > > >> data
> > >> > > > > >> > > > > >> corrupted.
> > >> > > > > >> > > > > >> > > > Will it suffice for you? Do you have any
> > other
> > >> > > > > concerns,
> > >> > > > > >> you
> > >> > > > > >> > > > want
> > >> > > > > >> > > > > >> us to
> > >> > > > > >> > > > > >> > > > address?
> > >> > > > > >> > > > > >> > > >
> > >> > > > > >> > > > > >> > > > -Vlad
> > >> > > > > >> > > > > >> > > >
> > >> > > > > >> > > > > >> > > >
> > >> > > > > >> > > > > >> > > > On Fri, Sep 9, 2016 at 10:56 AM, Sean
> > Busbey <
> > >> > > > > >> > > busbey@apache.org
> > >> > > > > >> > > > >
> > >> > > > > >> > > > > >> > wrote:
> > >> > > > > >> > > > > >> > > >
> > >> > > > > >> > > > > >> > > >> "docs will come to Apache soon" does not
> > >> address
> > >> > > my
> > >> > > > > >> concern
> > >> > > > > >> > > > > around
> > >> > > > > >> > > > > >> > docs
> > >> > > > > >> > > > > >> > > at
> > >> > > > > >> > > > > >> > > >> all, unless said docs have already made
> it
> > >> into
> > >> > > the
> > >> > > > > >> project
> > >> > > > > >> > > > > repo. I
> > >> > > > > >> > > > > >> > > don't
> > >> > > > > >> > > > > >> > > >> want third party resources for using a
> > major
> > >> and
> > >> > > > > >> important
> > >> > > > > >> > > > > feature
> > >> > > > > >> > > > > >> of
> > >> > > > > >> > > > > >> > > the
> > >> > > > > >> > > > > >> > > >> project, I want us to provide end users
> > with
> > >> > what
> > >> > > > they
> > >> > > > > >> need
> > >> > > > > >> > > to
> > >> > > > > >> > > > > get
> > >> > > > > >> > > > > >> the
> > >> > > > > >> > > > > >> > > job
> > >> > > > > >> > > > > >> > > >> done.
> > >> > > > > >> > > > > >> > > >>
> > >> > > > > >> > > > > >> > > >> I see some calls for patience on the
> > failure
> > >> > > > testing,
> > >> > > > > but
> > >> > > > > >> > the
> > >> > > > > >> > > > > >> appeal
> > >> > > > > >> > > > > >> > to
> > >> > > > > >> > > > > >> > > us
> > >> > > > > >> > > > > >> > > >> having done a bad job of requiring proper
> > >> tests
> > >> > of
> > >> > > > > >> previous
> > >> > > > > >> > > > > >> features
> > >> > > > > >> > > > > >> > > just
> > >> > > > > >> > > > > >> > > >> makes me more concerned about not getting
> > >> them
> > >> > > > here. I
> > >> > > > > >> > don't
> > >> > > > > >> > > > want
> > >> > > > > >> > > > > >> to
> > >> > > > > >> > > > > >> > set
> > >> > > > > >> > > > > >> > > >> yet another bad example that will then be
> > >> > pointed
> > >> > > to
> > >> > > > > in
> > >> > > > > >> the
> > >> > > > > >> > > > > future.
> > >> > > > > >> > > > > >> > > >>
> > >> > > > > >> > > > > >> > > >> On Sep 8, 2016 10:50, "Ted Yu" <
> > >> > > yuzhihong@gmail.com
> > >> > > > >
> > >> > > > > >> > wrote:
> > >> > > > > >> > > > > >> > > >>
> > >> > > > > >> > > > > >> > > >> > Is there any concern which is not
> > >> addressed ?
> > >> > > > > >> > > > > >> > > >> >
> > >> > > > > >> > > > > >> > > >> > Do we need another Vote thread ?
> > >> > > > > >> > > > > >> > > >> >
> > >> > > > > >> > > > > >> > > >> > Thanks
> > >> > > > > >> > > > > >> > > >> >
> > >> > > > > >> > > > > >> > > >> > On Thu, Sep 8, 2016 at 9:21 AM, Andrew
> > >> > Purtell <
> > >> > > > > >> > > > > >> apurtell@apache.org
> > >> > > > > >> > > > > >> > >
> > >> > > > > >> > > > > >> > > >> > wrote:
> > >> > > > > >> > > > > >> > > >> >
> > >> > > > > >> > > > > >> > > >> > > Vlad,
> > >> > > > > >> > > > > >> > > >> > >
> > >> > > > > >> > > > > >> > > >> > > I apologize for using the term
> > >> 'half-baked'
> > >> > > in a
> > >> > > > > way
> > >> > > > > >> > that
> > >> > > > > >> > > > > could
> > >> > > > > >> > > > > >> > > seem a
> > >> > > > > >> > > > > >> > > >> > > description of HBASE-7912. I meant
> that
> > >> as a
> > >> > > > > general
> > >> > > > > >> > > > > >> hypothetical.
> > >> > > > > >> > > > > >> > > >> > >
> > >> > > > > >> > > > > >> > > >> > > On Wed, Sep 7, 2016 at 9:36 AM,
> > Vladimir
> > >> > > > Rodionov
> > >> > > > > <
> > >> > > > > >> > > > > >> > > >> > vladrodionov@gmail.com>
> > >> > > > > >> > > > > >> > > >> > > wrote:
> > >> > > > > >> > > > > >> > > >> > >
> > >> > > > > >> > > > > >> > > >> > > > >> I'm not sure that "There is
> > already
> > >> > lots
> > >> > > of
> > >> > > > > >> > > half-baked
> > >> > > > > >> > > > > >> code
> > >> > > > > >> > > > > >> > in
> > >> > > > > >> > > > > >> > > the
> > >> > > > > >> > > > > >> > > >> > > > branch,
> > >> > > > > >> > > > > >> > > >> > > > so what's the harm in adding more?"
> > >> > > > > >> > > > > >> > > >> > > >
> > >> > > > > >> > > > > >> > > >> > > > I meant - not production - ready
> yet.
> > >> This
> > >> > > is
> > >> > > > > 2.0
> > >> > > > > >> > > > > development
> > >> > > > > >> > > > > >> > > branch
> > >> > > > > >> > > > > >> > > >> > and,
> > >> > > > > >> > > > > >> > > >> > > > hence many features are in works,
> > >> > > > > >> > > > > >> > > >> > > > not being tested well etc. I do not
> > >> > consider
> > >> > > > > backup
> > >> > > > > >> > as
> > >> > > > > >> > > > half
> > >> > > > > >> > > > > >> > baked
> > >> > > > > >> > > > > >> > > >> > > feature -
> > >> > > > > >> > > > > >> > > >> > > > it has passed our internal QA and
> has
> > >> very
> > >> > > > good
> > >> > > > > >> doc,
> > >> > > > > >> > > > which
> > >> > > > > >> > > > > we
> > >> > > > > >> > > > > >> > will
> > >> > > > > >> > > > > >> > > >> > > provide
> > >> > > > > >> > > > > >> > > >> > > > to Apache shortly.
> > >> > > > > >> > > > > >> > > >> > > >
> > >> > > > > >> > > > > >> > > >> > > > -Vlad
> > >> > > > > >> > > > > >> > > >> > > >
> > >> > > > > >> > > > > >> > > >> > > > On Wed, Sep 7, 2016 at 9:13 AM,
> > Andrew
> > >> > > > Purtell <
> > >> > > > > >> > > > > >> > > apurtell@apache.org>
> > >> > > > > >> > > > > >> > > >> > > > wrote:
> > >> > > > > >> > > > > >> > > >> > > >
> > >> > > > > >> > > > > >> > > >> > > > > We shouldn't admit half baked
> > changes
> > >> > that
> > >> > > > > won't
> > >> > > > > >> be
> > >> > > > > >> > > > > >> finished.
> > >> > > > > >> > > > > >> > > >> However
> > >> > > > > >> > > > > >> > > >> > > in
> > >> > > > > >> > > > > >> > > >> > > > > this case the crew working on
> this
> > >> > feature
> > >> > > > are
> > >> > > > > >> long
> > >> > > > > >> > > > > timers
> > >> > > > > >> > > > > >> and
> > >> > > > > >> > > > > >> > > less
> > >> > > > > >> > > > > >> > > >> > > > likely
> > >> > > > > >> > > > > >> > > >> > > > > than just about anyone to leave
> > >> > something
> > >> > > > in a
> > >> > > > > >> half
> > >> > > > > >> > > > baked
> > >> > > > > >> > > > > >> > > state. Of
> > >> > > > > >> > > > > >> > > >> > > > course
> > >> > > > > >> > > > > >> > > >> > > > > there is no guarantee how
> anything
> > >> will
> > >> > > turn
> > >> > > > > out,
> > >> > > > > >> > > but I
> > >> > > > > >> > > > > am
> > >> > > > > >> > > > > >> > > willing
> > >> > > > > >> > > > > >> > > >> to
> > >> > > > > >> > > > > >> > > >> > > > take
> > >> > > > > >> > > > > >> > > >> > > > > a little on faith if they feel
> > their
> > >> > best
> > >> > > > path
> > >> > > > > >> > > forward
> > >> > > > > >> > > > > now
> > >> > > > > >> > > > > >> is
> > >> > > > > >> > > > > >> > to
> > >> > > > > >> > > > > >> > > >> > merge
> > >> > > > > >> > > > > >> > > >> > > to
> > >> > > > > >> > > > > >> > > >> > > > > trunk. I only wish I had
> bandwidth
> > to
> > >> > have
> > >> > > > > done
> > >> > > > > >> > some
> > >> > > > > >> > > > real
> > >> > > > > >> > > > > >> > > kicking
> > >> > > > > >> > > > > >> > > >> of
> > >> > > > > >> > > > > >> > > >> > > the
> > >> > > > > >> > > > > >> > > >> > > > > tires by now. Maybe this week.
> > >> > > > > >> > > > > >> > > >> > > > >
> > >> > > > > >> > > > > >> > > >> > > > > (Yes, I'm using some of that time
> > for
> > >> > this
> > >> > > > > email
> > >> > > > > >> > :-)
> > >> > > > > >> > > > but
> > >> > > > > >> > > > > I
> > >> > > > > >> > > > > >> > type
> > >> > > > > >> > > > > >> > > >> > fast.)
> > >> > > > > >> > > > > >> > > >> > > > >
> > >> > > > > >> > > > > >> > > >> > > > > That said, I would like to
> agitate
> > >> for
> > >> > > > making
> > >> > > > > 2.0
> > >> > > > > >> > > more
> > >> > > > > >> > > > > real
> > >> > > > > >> > > > > >> > and
> > >> > > > > >> > > > > >> > > >> spend
> > >> > > > > >> > > > > >> > > >> > > > some
> > >> > > > > >> > > > > >> > > >> > > > > time on it now that I'm winding
> > down
> > >> > with
> > >> > > > > 0.98. I
> > >> > > > > >> > > think
> > >> > > > > >> > > > > >> that
> > >> > > > > >> > > > > >> > > means
> > >> > > > > >> > > > > >> > > >> > > > > branching for 2.0 real soon now
> and
> > >> even
> > >> > > > > evicting
> > >> > > > > >> > > > things
> > >> > > > > >> > > > > >> from
> > >> > > > > >> > > > > >> > > 2.0
> > >> > > > > >> > > > > >> > > >> > > branch
> > >> > > > > >> > > > > >> > > >> > > > > that aren't finished or stable,
> > >> leaving
> > >> > > them
> > >> > > > > only
> > >> > > > > >> > > once
> > >> > > > > >> > > > > >> again
> > >> > > > > >> > > > > >> > in
> > >> > > > > >> > > > > >> > > the
> > >> > > > > >> > > > > >> > > >> > > > master
> > >> > > > > >> > > > > >> > > >> > > > > branch. Or, maybe just evicting
> > them.
> > >> > > Let's
> > >> > > > > take
> > >> > > > > >> it
> > >> > > > > >> > > > case
> > >> > > > > >> > > > > by
> > >> > > > > >> > > > > >> > > case.
> > >> > > > > >> > > > > >> > > >> > > > >
> > >> > > > > >> > > > > >> > > >> > > > > I think this feature can come in
> > >> > > relatively
> > >> > > > > >> safely.
> > >> > > > > >> > > As
> > >> > > > > >> > > > > >> added
> > >> > > > > >> > > > > >> > > >> > insurance,
> > >> > > > > >> > > > > >> > > >> > > > > let's admit the possibility it
> > could
> > >> be
> > >> > > > > reverted
> > >> > > > > >> on
> > >> > > > > >> > > the
> > >> > > > > >> > > > > 2.0
> > >> > > > > >> > > > > >> > > branch
> > >> > > > > >> > > > > >> > > >> if
> > >> > > > > >> > > > > >> > > >> > > > folks
> > >> > > > > >> > > > > >> > > >> > > > > working on stabilizing 2.0 decide
> > to
> > >> > evict
> > >> > > > it
> > >> > > > > >> > because
> > >> > > > > >> > > > it
> > >> > > > > >> > > > > is
> > >> > > > > >> > > > > >> > > >> > unfinished
> > >> > > > > >> > > > > >> > > >> > > or
> > >> > > > > >> > > > > >> > > >> > > > > unstable, because that certainly
> > can
> > >> > > > happen. I
> > >> > > > > >> > would
> > >> > > > > >> > > > > >> expect if
> > >> > > > > >> > > > > >> > > talk
> > >> > > > > >> > > > > >> > > >> > > like
> > >> > > > > >> > > > > >> > > >> > > > > that starts, we'd get help
> > finishing
> > >> or
> > >> > > > > >> stabilizing
> > >> > > > > >> > > > > what's
> > >> > > > > >> > > > > >> > under
> > >> > > > > >> > > > > >> > > >> > > > discussion
> > >> > > > > >> > > > > >> > > >> > > > > for revert. Or, we'd have a
> revert.
> > >> > Either
> > >> > > > way
> > >> > > > > >> the
> > >> > > > > >> > > > > outcome
> > >> > > > > >> > > > > >> is
> > >> > > > > >> > > > > >> > > >> > > acceptable.
> > >> > > > > >> > > > > >> > > >> > > > >
> > >> > > > > >> > > > > >> > > >> > > > >
> > >> > > > > >> > > > > >> > > >> > > > > On Wed, Sep 7, 2016 at 8:56 AM,
> > Dima
> > >> > > Spivak
> > >> > > > <
> > >> > > > > >> > > > > >> > > dimaspivak@apache.org
> > >> > > > > >> > > > > >> > > >> >
> > >> > > > > >> > > > > >> > > >> > > > wrote:
> > >> > > > > >> > > > > >> > > >> > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > I'm not sure that "There is
> > already
> > >> > lots
> > >> > > > of
> > >> > > > > >> > > > half-baked
> > >> > > > > >> > > > > >> code
> > >> > > > > >> > > > > >> > in
> > >> > > > > >> > > > > >> > > >> the
> > >> > > > > >> > > > > >> > > >> > > > > branch,
> > >> > > > > >> > > > > >> > > >> > > > > > so what's the harm in adding
> > more?"
> > >> > is a
> > >> > > > > good
> > >> > > > > >> > code
> > >> > > > > >> > > > > commit
> > >> > > > > >> > > > > >> > > >> > philosophy
> > >> > > > > >> > > > > >> > > >> > > > for
> > >> > > > > >> > > > > >> > > >> > > > > a
> > >> > > > > >> > > > > >> > > >> > > > > > fault-tolerant distributed data
> > >> store.
> > >> > > ;)
> > >> > > > > >> > > > > >> > > >> > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > More seriously, a lack of test
> > >> > coverage
> > >> > > > for
> > >> > > > > >> > > existing
> > >> > > > > >> > > > > >> > features
> > >> > > > > >> > > > > >> > > >> > > shouldn't
> > >> > > > > >> > > > > >> > > >> > > > > be
> > >> > > > > >> > > > > >> > > >> > > > > > used as justification for
> > >> introducing
> > >> > > new
> > >> > > > > >> > features
> > >> > > > > >> > > > with
> > >> > > > > >> > > > > >> the
> > >> > > > > >> > > > > >> > > same
> > >> > > > > >> > > > > >> > > >> > > > > > shortcomings. Ultimately, it's
> > the
> > >> end
> > >> > > > user
> > >> > > > > who
> > >> > > > > >> > > will
> > >> > > > > >> > > > > feel
> > >> > > > > >> > > > > >> > the
> > >> > > > > >> > > > > >> > > >> pain,
> > >> > > > > >> > > > > >> > > >> > > so
> > >> > > > > >> > > > > >> > > >> > > > > > shouldn't we do everything we
> can
> > >> to
> > >> > > > > mitigate
> > >> > > > > >> > that?
> > >> > > > > >> > > > > >> > > >> > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > -Dima
> > >> > > > > >> > > > > >> > > >> > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > On Wed, Sep 7, 2016 at 8:46 AM,
> > >> > Vladimir
> > >> > > > > >> > Rodionov <
> > >> > > > > >> > > > > >> > > >> > > > > vladrodionov@gmail.com>
> > >> > > > > >> > > > > >> > > >> > > > > > wrote:
> > >> > > > > >> > > > > >> > > >> > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > Sean,
> > >> > > > > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > * have docs
> > >> > > > > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > Agree. We have a doc and
> backup
> > >> is
> > >> > the
> > >> > > > > most
> > >> > > > > >> > > > > documented
> > >> > > > > >> > > > > >> > > feature
> > >> > > > > >> > > > > >> > > >> > :),
> > >> > > > > >> > > > > >> > > >> > > we
> > >> > > > > >> > > > > >> > > >> > > > > > will
> > >> > > > > >> > > > > >> > > >> > > > > > > release it shortly to Apache.
> > >> > > > > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > * have sunny-day correctness
> > >> tests
> > >> > > > > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > Feature has  close to 60 test
> > >> cases,
> > >> > > > which
> > >> > > > > >> run
> > >> > > > > >> > > for
> > >> > > > > >> > > > > >> approx
> > >> > > > > >> > > > > >> > 30
> > >> > > > > >> > > > > >> > > >> min.
> > >> > > > > >> > > > > >> > > >> > > We
> > >> > > > > >> > > > > >> > > >> > > > > can
> > >> > > > > >> > > > > >> > > >> > > > > > > add more, if community do not
> > >> mind
> > >> > :)
> > >> > > > > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > * have
> > >> > correctness-in-face-of-failure
> > >> > > > > tests
> > >> > > > > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > Any examples of these tests
> in
> > >> > > existing
> > >> > > > > >> > features?
> > >> > > > > >> > > > In
> > >> > > > > >> > > > > >> > works,
> > >> > > > > >> > > > > >> > > we
> > >> > > > > >> > > > > >> > > >> > > have a
> > >> > > > > >> > > > > >> > > >> > > > > > clear
> > >> > > > > >> > > > > >> > > >> > > > > > > understanding of what should
> be
> > >> done
> > >> > > by
> > >> > > > > the
> > >> > > > > >> > time
> > >> > > > > >> > > of
> > >> > > > > >> > > > > 2.0
> > >> > > > > >> > > > > >> > > >> release.
> > >> > > > > >> > > > > >> > > >> > > > > > > That is very close goal for
> us,
> > >> to
> > >> > > > verify
> > >> > > > > IT
> > >> > > > > >> > > monkey
> > >> > > > > >> > > > > for
> > >> > > > > >> > > > > >> > > >> existing
> > >> > > > > >> > > > > >> > > >> > > > code.
> > >> > > > > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > * don't rely on things
> outside
> > of
> > >> > > HBase
> > >> > > > > for
> > >> > > > > >> > > normal
> > >> > > > > >> > > > > >> > operation
> > >> > > > > >> > > > > >> > > >> > (okay
> > >> > > > > >> > > > > >> > > >> > > > for
> > >> > > > > >> > > > > >> > > >> > > > > > > advanced operation)
> > >> > > > > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > We do not.
> > >> > > > > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > Enormous time has been spent
> > >> already
> > >> > > on
> > >> > > > > the
> > >> > > > > >> > > > > development
> > >> > > > > >> > > > > >> > and
> > >> > > > > >> > > > > >> > > >> > testing
> > >> > > > > >> > > > > >> > > >> > > > the
> > >> > > > > >> > > > > >> > > >> > > > > > > feature, it has passed our
> > >> internal
> > >> > > > tests
> > >> > > > > and
> > >> > > > > >> > > many
> > >> > > > > >> > > > > >> rounds
> > >> > > > > >> > > > > >> > of
> > >> > > > > >> > > > > >> > > >> code
> > >> > > > > >> > > > > >> > > >> > > > > reviews
> > >> > > > > >> > > > > >> > > >> > > > > > > by HBase committers. We do
> not
> > >> mind
> > >> > if
> > >> > > > > >> someone
> > >> > > > > >> > > from
> > >> > > > > >> > > > > >> HBase
> > >> > > > > >> > > > > >> > > >> > community
> > >> > > > > >> > > > > >> > > >> > > > > > > (outside of HW) will review
> the
> > >> > code,
> > >> > > > but
> > >> > > > > it
> > >> > > > > >> > will
> > >> > > > > >> > > > > >> probably
> > >> > > > > >> > > > > >> > > >> takes
> > >> > > > > >> > > > > >> > > >> > > > > forever
> > >> > > > > >> > > > > >> > > >> > > > > > to
> > >> > > > > >> > > > > >> > > >> > > > > > > wait for volunteer?, the
> > feature
> > >> is
> > >> > > > quite
> > >> > > > > >> large
> > >> > > > > >> > > > (1MB+
> > >> > > > > >> > > > > >> > > >> cumulative
> > >> > > > > >> > > > > >> > > >> > > > patch)
> > >> > > > > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > 2.0 branch is full of half
> > baked
> > >> > > > features,
> > >> > > > > >> most
> > >> > > > > >> > > of
> > >> > > > > >> > > > > them
> > >> > > > > >> > > > > >> > are
> > >> > > > > >> > > > > >> > > in
> > >> > > > > >> > > > > >> > > >> > > active
> > >> > > > > >> > > > > >> > > >> > > > > > > development, therefore I am
> not
> > >> > > > following
> > >> > > > > you
> > >> > > > > >> > > here,
> > >> > > > > >> > > > > >> Sean?
> > >> > > > > >> > > > > >> > > Why
> > >> > > > > >> > > > > >> > > >> > > > > HBASE-7912
> > >> > > > > >> > > > > >> > > >> > > > > > is
> > >> > > > > >> > > > > >> > > >> > > > > > > not good enough yet to be
> > >> integrated
> > >> > > > into
> > >> > > > > 2.0
> > >> > > > > >> > > > branch?
> > >> > > > > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > -Vlad
> > >> > > > > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > On Wed, Sep 7, 2016 at 8:23
> AM,
> > >> Sean
> > >> > > > > Busbey <
> > >> > > > > >> > > > > >> > > busbey@apache.org
> > >> > > > > >> > > > > >> > > >> >
> > >> > > > > >> > > > > >> > > >> > > > wrote:
> > >> > > > > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > > On Tue, Sep 6, 2016 at
> 10:36
> > >> PM,
> > >> > > Josh
> > >> > > > > >> Elser <
> > >> > > > > >> > > > > >> > > >> > > josh.elser@gmail.com>
> > >> > > > > >> > > > > >> > > >> > > > > > > wrote:
> > >> > > > > >> > > > > >> > > >> > > > > > > > > So, the answer to Sean's
> > >> > original
> > >> > > > > >> question
> > >> > > > > >> > is
> > >> > > > > >> > > > "as
> > >> > > > > >> > > > > >> > > robust as
> > >> > > > > >> > > > > >> > > >> > > > > snapshots
> > >> > > > > >> > > > > >> > > >> > > > > > > > > presently are"?
> > >> (independence of
> > >> > > > > >> > > backup/restore
> > >> > > > > >> > > > > >> > failure
> > >> > > > > >> > > > > >> > > >> > > tolerance
> > >> > > > > >> > > > > >> > > >> > > > > > from
> > >> > > > > >> > > > > >> > > >> > > > > > > > > snapshot failure
> tolerance)
> > >> > > > > >> > > > > >> > > >> > > > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > > > Is this just a question
> WRT
> > >> > > context
> > >> > > > of
> > >> > > > > >> the
> > >> > > > > >> > > > > change,
> > >> > > > > >> > > > > >> or
> > >> > > > > >> > > > > >> > > is it
> > >> > > > > >> > > > > >> > > >> > > means
> > >> > > > > >> > > > > >> > > >> > > > > > for a
> > >> > > > > >> > > > > >> > > >> > > > > > > > veto
> > >> > > > > >> > > > > >> > > >> > > > > > > > > from you, Sean? Just
> trying
> > >> to
> > >> > > make
> > >> > > > > sure
> > >> > > > > >> > I'm
> > >> > > > > >> > > > > >> following
> > >> > > > > >> > > > > >> > > >> along
> > >> > > > > >> > > > > >> > > >> > > > > > > adequately.
> > >> > > > > >> > > > > >> > > >> > > > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > > I'd say ATM I'm -0,
> bordering
> > >> on
> > >> > -1
> > >> > > > but
> > >> > > > > not
> > >> > > > > >> > for
> > >> > > > > >> > > > > >> reasons
> > >> > > > > >> > > > > >> > I
> > >> > > > > >> > > > > >> > > can
> > >> > > > > >> > > > > >> > > >> > > > > > articulate
> > >> > > > > >> > > > > >> > > >> > > > > > > > well.
> > >> > > > > >> > > > > >> > > >> > > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > > Here's an attempt.
> > >> > > > > >> > > > > >> > > >> > > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > > We've been trying to move,
> > as a
> > >> > > > > community,
> > >> > > > > >> > > > towards
> > >> > > > > >> > > > > >> > > minimizing
> > >> > > > > >> > > > > >> > > >> > > risk
> > >> > > > > >> > > > > >> > > >> > > > to
> > >> > > > > >> > > > > >> > > >> > > > > > > > downstream folks by getting
> > >> > > "complete
> > >> > > > > >> enough
> > >> > > > > >> > > for
> > >> > > > > >> > > > > use"
> > >> > > > > >> > > > > >> > > gates
> > >> > > > > >> > > > > >> > > >> in
> > >> > > > > >> > > > > >> > > >> > > > place
> > >> > > > > >> > > > > >> > > >> > > > > > > > before we introduce new
> > >> features.
> > >> > > This
> > >> > > > > was
> > >> > > > > >> > > > spurred
> > >> > > > > >> > > > > >> by a
> > >> > > > > >> > > > > >> > > some
> > >> > > > > >> > > > > >> > > >> > > > features
> > >> > > > > >> > > > > >> > > >> > > > > > > > getting in half-baked and
> > never
> > >> > > making
> > >> > > > > it
> > >> > > > > >> to
> > >> > > > > >> > > "can
> > >> > > > > >> > > > > >> really
> > >> > > > > >> > > > > >> > > use"
> > >> > > > > >> > > > > >> > > >> > > > status
> > >> > > > > >> > > > > >> > > >> > > > > > > > (I'm thinking of
> distributed
> > >> log
> > >> > > > replay
> > >> > > > > and
> > >> > > > > >> > the
> > >> > > > > >> > > > > >> zk-less
> > >> > > > > >> > > > > >> > > >> > > assignment
> > >> > > > > >> > > > > >> > > >> > > > > > > > stuff, I don't recall if
> > there
> > >> was
> > >> > > > > more).
> > >> > > > > >> > > > > >> > > >> > > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > > The gates, generally,
> > included
> > >> > > things
> > >> > > > > like:
> > >> > > > > >> > > > > >> > > >> > > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > > * have docs
> > >> > > > > >> > > > > >> > > >> > > > > > > > * have sunny-day
> correctness
> > >> tests
> > >> > > > > >> > > > > >> > > >> > > > > > > > * have
> > >> > > correctness-in-face-of-failure
> > >> > > > > tests
> > >> > > > > >> > > > > >> > > >> > > > > > > > * don't rely on things
> > outside
> > >> of
> > >> > > > HBase
> > >> > > > > for
> > >> > > > > >> > > > normal
> > >> > > > > >> > > > > >> > > operation
> > >> > > > > >> > > > > >> > > >> > > (okay
> > >> > > > > >> > > > > >> > > >> > > > > for
> > >> > > > > >> > > > > >> > > >> > > > > > > > advanced operation)
> > >> > > > > >> > > > > >> > > >> > > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > > As an example, we kept the
> > MOB
> > >> > work
> > >> > > > off
> > >> > > > > in
> > >> > > > > >> a
> > >> > > > > >> > > > branch
> > >> > > > > >> > > > > >> and
> > >> > > > > >> > > > > >> > > out
> > >> > > > > >> > > > > >> > > >> of
> > >> > > > > >> > > > > >> > > >> > > > master
> > >> > > > > >> > > > > >> > > >> > > > > > > > until it could pass these
> > >> > criteria.
> > >> > > > The
> > >> > > > > big
> > >> > > > > >> > > > > exemption
> > >> > > > > >> > > > > >> > > we've
> > >> > > > > >> > > > > >> > > >> had
> > >> > > > > >> > > > > >> > > >> > > to
> > >> > > > > >> > > > > >> > > >> > > > > > > > this was the hbase-spark
> > >> > > integration,
> > >> > > > > where
> > >> > > > > >> > we
> > >> > > > > >> > > > all
> > >> > > > > >> > > > > >> > agreed
> > >> > > > > >> > > > > >> > > it
> > >> > > > > >> > > > > >> > > >> > > could
> > >> > > > > >> > > > > >> > > >> > > > > > > > land in master because it
> was
> > >> very
> > >> > > > well
> > >> > > > > >> > > isolated
> > >> > > > > >> > > > > (the
> > >> > > > > >> > > > > >> > > slide
> > >> > > > > >> > > > > >> > > >> > away
> > >> > > > > >> > > > > >> > > >> > > > from
> > >> > > > > >> > > > > >> > > >> > > > > > > > including docs as a
> > first-class
> > >> > part
> > >> > > > of
> > >> > > > > >> > > building
> > >> > > > > >> > > > up
> > >> > > > > >> > > > > >> that
> > >> > > > > >> > > > > >> > > >> > > > integration
> > >> > > > > >> > > > > >> > > >> > > > > > > > has led me to doubt the
> > wisdom
> > >> of
> > >> > > this
> > >> > > > > >> > > decision).
> > >> > > > > >> > > > > >> > > >> > > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > > We've also been treating
> > >> inclusion
> > >> > > in
> > >> > > > a
> > >> > > > > >> > > "probably
> > >> > > > > >> > > > > >> will
> > >> > > > > >> > > > > >> > be
> > >> > > > > >> > > > > >> > > >> > > released
> > >> > > > > >> > > > > >> > > >> > > > to
> > >> > > > > >> > > > > >> > > >> > > > > > > > downstream" branches as a
> > >> higher
> > >> > > bar,
> > >> > > > > >> > requiring
> > >> > > > > >> > > > > >> > > >> > > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > > * don't moderately impact
> > >> > > performance
> > >> > > > > when
> > >> > > > > >> > the
> > >> > > > > >> > > > > >> feature
> > >> > > > > >> > > > > >> > > isn't
> > >> > > > > >> > > > > >> > > >> in
> > >> > > > > >> > > > > >> > > >> > > use
> > >> > > > > >> > > > > >> > > >> > > > > > > > * don't severely impact
> > >> > performance
> > >> > > > when
> > >> > > > > >> the
> > >> > > > > >> > > > > feature
> > >> > > > > >> > > > > >> is
> > >> > > > > >> > > > > >> > in
> > >> > > > > >> > > > > >> > > >> use
> > >> > > > > >> > > > > >> > > >> > > > > > > > * either default-to-on or
> > show
> > >> > > enough
> > >> > > > > >> demand
> > >> > > > > >> > to
> > >> > > > > >> > > > > >> believe
> > >> > > > > >> > > > > >> > a
> > >> > > > > >> > > > > >> > > >> > > > non-trivial
> > >> > > > > >> > > > > >> > > >> > > > > > > > number of folks will turn
> the
> > >> > > feature
> > >> > > > on
> > >> > > > > >> > > > > >> > > >> > > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > > The above has kept MOB and
> > >> > > hbase-spark
> > >> > > > > >> > > > integration
> > >> > > > > >> > > > > >> out
> > >> > > > > >> > > > > >> > of
> > >> > > > > >> > > > > >> > > >> > > branch-1,
> > >> > > > > >> > > > > >> > > >> > > > > > > > presumably while they've
> > >> "gotten
> > >> > > more
> > >> > > > > >> stable"
> > >> > > > > >> > > in
> > >> > > > > >> > > > > >> master
> > >> > > > > >> > > > > >> > > from
> > >> > > > > >> > > > > >> > > >> > the
> > >> > > > > >> > > > > >> > > >> > > > odd
> > >> > > > > >> > > > > >> > > >> > > > > > > > vendor inclusion.
> > >> > > > > >> > > > > >> > > >> > > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > > Are we going to have a 2.0
> > >> release
> > >> > > > > before
> > >> > > > > >> the
> > >> > > > > >> > > end
> > >> > > > > >> > > > > of
> > >> > > > > >> > > > > >> the
> > >> > > > > >> > > > > >> > > >> year?
> > >> > > > > >> > > > > >> > > >> > > > We're
> > >> > > > > >> > > > > >> > > >> > > > > > > > coming up on 1.5 years
> since
> > >> the
> > >> > > > > release of
> > >> > > > > >> > > > version
> > >> > > > > >> > > > > >> 1.0;
> > >> > > > > >> > > > > >> > > >> seems
> > >> > > > > >> > > > > >> > > >> > > like
> > >> > > > > >> > > > > >> > > >> > > > > > > > it's about time, though I
> > >> haven't
> > >> > > seen
> > >> > > > > any
> > >> > > > > >> > > > concrete
> > >> > > > > >> > > > > >> > plans
> > >> > > > > >> > > > > >> > > >> this
> > >> > > > > >> > > > > >> > > >> > > > year.
> > >> > > > > >> > > > > >> > > >> > > > > > > > Presuming we are going to
> > have
> > >> one
> > >> > > by
> > >> > > > > the
> > >> > > > > >> end
> > >> > > > > >> > > of
> > >> > > > > >> > > > > the
> > >> > > > > >> > > > > >> > > year, it
> > >> > > > > >> > > > > >> > > >> > > > seems a
> > >> > > > > >> > > > > >> > > >> > > > > > > > bit close to still be
> adding
> > in
> > >> > > > > "features
> > >> > > > > >> > that
> > >> > > > > >> > > > need
> > >> > > > > >> > > > > >> > > maturing"
> > >> > > > > >> > > > > >> > > >> > on
> > >> > > > > >> > > > > >> > > >> > > > the
> > >> > > > > >> > > > > >> > > >> > > > > > > > branch.
> > >> > > > > >> > > > > >> > > >> > > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > > The lack of a concrete plan
> > for
> > >> > 2.0
> > >> > > > > keeps
> > >> > > > > >> me
> > >> > > > > >> > > from
> > >> > > > > >> > > > > >> > > considering
> > >> > > > > >> > > > > >> > > >> > > these
> > >> > > > > >> > > > > >> > > >> > > > > > > > things blocker at the
> moment.
> > >> But
> > >> > I
> > >> > > > know
> > >> > > > > >> > first
> > >> > > > > >> > > > hand
> > >> > > > > >> > > > > >> how
> > >> > > > > >> > > > > >> > > much
> > >> > > > > >> > > > > >> > > >> > > > trouble
> > >> > > > > >> > > > > >> > > >> > > > > > > > folks have had with other
> > >> features
> > >> > > > that
> > >> > > > > >> have
> > >> > > > > >> > > gone
> > >> > > > > >> > > > > >> into
> > >> > > > > >> > > > > >> > > >> > downstream
> > >> > > > > >> > > > > >> > > >> > > > > > > > facing releases without
> > >> robustness
> > >> > > > > checks
> > >> > > > > >> > (i.e.
> > >> > > > > >> > > > > >> > > replication),
> > >> > > > > >> > > > > >> > > >> > and
> > >> > > > > >> > > > > >> > > >> > > > I'm
> > >> > > > > >> > > > > >> > > >> > > > > > > > concerned about what we're
> > >> setting
> > >> > > up
> > >> > > > if
> > >> > > > > >> 2.0
> > >> > > > > >> > > goes
> > >> > > > > >> > > > > out
> > >> > > > > >> > > > > >> > with
> > >> > > > > >> > > > > >> > > >> this
> > >> > > > > >> > > > > >> > > >> > > > > > > > feature in its current
> state.
> > >> > > > > >> > > > > >> > > >> > > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > >
> > >> > > > > >> > > > > >> > > >> > > > >
> > >> > > > > >> > > > > >> > > >> > > > >
> > >> > > > > >> > > > > >> > > >> > > > >
> > >> > > > > >> > > > > >> > > >> > > > > --
> > >> > > > > >> > > > > >> > > >> > > > > Best regards,
> > >> > > > > >> > > > > >> > > >> > > > >
> > >> > > > > >> > > > > >> > > >> > > > >    - Andy
> > >> > > > > >> > > > > >> > > >> > > > >
> > >> > > > > >> > > > > >> > > >> > > > > Problems worthy of attack prove
> > their
> > >> > > worth
> > >> > > > by
> > >> > > > > >> > > hitting
> > >> > > > > >> > > > > >> back. -
> > >> > > > > >> > > > > >> > > Piet
> > >> > > > > >> > > > > >> > > >> > > Hein
> > >> > > > > >> > > > > >> > > >> > > > > (via Tom White)
> > >> > > > > >> > > > > >> > > >> > > > >
> > >> > > > > >> > > > > >> > > >> > > >
> > >> > > > > >> > > > > >> > > >> > >
> > >> > > > > >> > > > > >> > > >> > >
> > >> > > > > >> > > > > >> > > >> > >
> > >> > > > > >> > > > > >> > > >> > > --
> > >> > > > > >> > > > > >> > > >> > > Best regards,
> > >> > > > > >> > > > > >> > > >> > >
> > >> > > > > >> > > > > >> > > >> > >    - Andy
> > >> > > > > >> > > > > >> > > >> > >
> > >> > > > > >> > > > > >> > > >> > > Problems worthy of attack prove their
> > >> worth
> > >> > by
> > >> > > > > >> hitting
> > >> > > > > >> > > > back.
> > >> > > > > >> > > > > -
> > >> > > > > >> > > > > >> > Piet
> > >> > > > > >> > > > > >> > > >> Hein
> > >> > > > > >> > > > > >> > > >> > > (via Tom White)
> > >> > > > > >> > > > > >> > > >> > >
> > >> > > > > >> > > > > >> > > >> >
> > >> > > > > >> > > > > >> > > >>
> > >> > > > > >> > > > > >> > >
> > >> > > > > >> > > > > >> >
> > >> > > > > >> > > > > >>
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > >
> > >> > > > > >> > > >
> > >> > > > > >> > >
> > >> > > > > >> >
> > >> > > > > >>
> > >> > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > > --
> > >> > > > > busbey
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> > >
> >
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Stack <st...@duboce.net>]

Which branch do I check out to try it? HBASE-7912 is not it. I don't see an
HBASE-16727...
Thanks,
M


On Tue, Oct 11, 2016 at 11:41 AM, Vladimir Rodionov <vl...@gmail.com>
wrote:

> The last patch is on review board:
> https://reviews.apache.org/r/52748
>
> On Mon, Oct 10, 2016 at 3:52 PM, Vladimir Rodionov <vladrodionov@gmail.com
> >
> wrote:
>
> > >> How hard to put in an hbase-backup module? hbase-server is fat enough
> > >> already. Could be done as a follow-up.
> >
> > https://issues.apache.org/jira/browse/HBASE-16727?
> > focusedCommentId=15531237&page=com.atlassian.jira.
> > plugin.system.issuetabpanels:comment-tabpanel#comment-15531237
> >
> > Can we do merge first? Then we can discuss separate module.
> >
> > -Vlad
> >
> > On Mon, Oct 10, 2016 at 3:44 PM, Ted Yu <yu...@gmail.com> wrote:
> >
> >> Looks like the first quote was cut off.
> >> The original sentence was:
> >>
> >> bq. no mapreduce job launched from master or region server.
> >>
> >> mapreduce job is launched from the node where command line tool is run.
> >>
> >> On Mon, Oct 10, 2016 at 3:38 PM, Stack <st...@duboce.net> wrote:
> >>
> >> > bq. launched from master or region server.
> >> >
> >> > What does this mean please? Has to be run from Master or RegionServer?
> >> Can
> >> > it be run from another node altogether?
> >> >
> >> > On Mon, Oct 10, 2016 at 1:44 PM, Vladimir Rodionov <
> >> vladrodionov@gmail.com
> >> > >
> >> > wrote:
> >> >
> >> > > >> mapreduce dependency has been moved to client side - no mapreduce
> >> job
> >> > >
> >> > > 1. We have no code in the client module anymore, due to dependency
> on
> >> > > internal server API (HFile and WAL access).
> >> > > 2. Backup/ restore are client - driven operations, but all the code
> >> > resides
> >> > > in the server module
> >> > >
> >> >
> >> > How hard to put in an hbase-backup module? hbase-server is fat enough
> >> > already. Could be done as a follow-up.
> >> >
> >> > Thanks,
> >> > St.Ack
> >> >
> >> >
> >> >
> >> > > 3. No MR in Master, no procedure - driven execution.
> >> > > 4. Old good MR from command-line.
> >> > > 5. Security was simplified and now only super-user is allowed to run
> >> > > backup/restores.
> >> > > 6. HBase Backup API was gone due to 1. Now only command-line access
> to
> >> > > backup tools.
> >> > >
> >> > > These consequences of refactoring has been discussed in HBASE-16727.
> >> > >
> >> > > -Vlad
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > On Mon, Oct 10, 2016 at 1:31 PM, Ted Yu <yu...@gmail.com>
> wrote:
> >> > >
> >> > > > Reviving this thread.
> >> > > >
> >> > > > The following has taken place:
> >> > > >
> >> > > > mapreduce dependency has been moved to client side - no mapreduce
> >> job
> >> > > > launched from master or region server.
> >> > > > document patch (HBASE-16574) has been integrated.
> >> > > > Updated mega patch has been attached to HBASE-14123: this covers
> the
> >> > > > refactor in #1 above and the protobuf 3 merge.
> >> > > >
> >> > > > If community has more feedback on the merge proposal, I would love
> >> to
> >> > > hear
> >> > > > it.
> >> > > >
> >> > > > Thanks
> >> > > >
> >> > > > On Thu, Sep 22, 2016 at 10:31 AM, Sean Busbey <
> busbey@cloudera.com>
> >> > > wrote:
> >> > > >
> >> > > > > I'd like to see the docs proposed on HBASE-16574 integrated into
> >> our
> >> > > > > project's documentation prior to merge.
> >> > > > >
> >> > > > > On Thu, Sep 22, 2016 at 9:02 AM, Ted Yu <yu...@gmail.com>
> >> wrote:
> >> > > > > > This feature can be marked experimental due to some
> limitations
> >> > such
> >> > > as
> >> > > > > > security.
> >> > > > > >
> >> > > > > > Your previous round of comments have been addressed.
> >> > > > > > Command line tool has gone through:
> >> > > > > >
> >> > > > > > HBASE-16620 Fix backup command-line tool usability issues
> >> > > > > > HBASE-16655 hbase backup describe with incorrect backup id
> >> results
> >> > in
> >> > > > NPE
> >> > > > > >
> >> > > > > > The updated doc has been attached to HBASE-16574.
> >> > > > > >
> >> > > > > > Cheers
> >> > > > > >
> >> > > > > > On Thu, Sep 22, 2016 at 8:53 AM, Stack <st...@duboce.net>
> >> wrote:
> >> > > > > >
> >> > > > > >> On Wed, Sep 21, 2016 at 7:43 AM, Ted Yu <yuzhihong@gmail.com
> >
> >> > > wrote:
> >> > > > > >>
> >> > > > > >> > Are there more (review) comments ?
> >> > > > > >> >
> >> > > > > >> >
> >> > > > > >> Are outstanding comments addressed?
> >> > > > > >>
> >> > > > > >> I don't see answer to my 'is this experimental/will it be
> >> marked
> >> > > > > >> experimental' question.
> >> > > > > >>
> >> > > > > >> I ran into some issues trying to use the feature and
> suggested
> >> > that
> >> > > a
> >> > > > > >> feature likes this needs polish else it'll just rot, unused.
> >> Has
> >> > > > polish
> >> > > > > >> been applied? All ready for another 'user' test? Suggest that
> >> you
> >> > > > update
> >> > > > > >> here going forward for the benefit of those trying to follow
> >> along
> >> > > and
> >> > > > > who
> >> > > > > >> are not watching JIRA change fly-by.
> >> > > > > >>
> >> > > > > >> It looks like doc got a revision -- I have to check -- to
> take
> >> on
> >> > > > > >> suggestion made above but again, suggest, that this thread
> gets
> >> > > > updated.
> >> > > > > >>
> >> > > > > >> Thanks,
> >> > > > > >> St.Ack
> >> > > > > >>
> >> > > > > >>
> >> > > > > >>
> >> > > > > >> > Thanks
> >> > > > > >> >
> >> > > > > >> > On Tue, Sep 20, 2016 at 10:02 AM, Devaraj Das <
> >> > > ddas@hortonworks.com
> >> > > > >
> >> > > > > >> > wrote:
> >> > > > > >> >
> >> > > > > >> > > Just reviving this thread. Thanks Sean, Stack, Dima, and
> >> > others
> >> > > > for
> >> > > > > the
> >> > > > > >> > > thorough reviews and testing. Thanks Ted and Vlad for
> >> taking
> >> > > care
> >> > > > of
> >> > > > > >> the
> >> > > > > >> > > feedback. Are we all good to do the merge now? Rather do
> >> > sooner
> >> > > > than
> >> > > > > >> > later.
> >> > > > > >> > > ________________________________________
> >> > > > > >> > > From: saint.ack@gmail.com <sa...@gmail.com> on
> behalf
> >> of
> >> > > > Stack
> >> > > > > <
> >> > > > > >> > > stack@duboce.net>
> >> > > > > >> > > Sent: Monday, September 12, 2016 1:18 PM
> >> > > > > >> > > To: HBase Dev List
> >> > > > > >> > > Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch
> >> > > > HBASE-7912
> >> > > > > >> > >
> >> > > > > >> > > On Mon, Sep 12, 2016 at 12:19 PM, Ted Yu <
> >> yuzhihong@gmail.com
> >> > >
> >> > > > > wrote:
> >> > > > > >> > >
> >> > > > > >> > > > Mega patch (rev 18) is on HBASE-14123.
> >> > > > > >> > > >
> >> > > > > >> > > > Please comment on HBASE-14123 on how you want to
> review.
> >> > > > > >> > > >
> >> > > > > >> > >
> >> > > > > >> > >
> >> > > > > >> > > Yeah. That was my lost tab. Last rb was 6 months ago.
> >> Suggest
> >> > > > > updating
> >> > > > > >> > it.
> >> > > > > >> > > RB is pretty good for review. Patch is only 1.5M so
> should
> >> be
> >> > > > fine.
> >> > > > > >> > >
> >> > > > > >> > > St.Ack
> >> > > > > >> > >
> >> > > > > >> > >
> >> > > > > >> > > >
> >> > > > > >> > > > Thanks
> >> > > > > >> > > >
> >> > > > > >> > > > On Mon, Sep 12, 2016 at 12:15 PM, Stack <
> >> stack@duboce.net>
> >> > > > wrote:
> >> > > > > >> > > >
> >> > > > > >> > > > > On review of the 'patch', do I just compare the
> branch
> >> to
> >> > > > > master or
> >> > > > > >> > is
> >> > > > > >> > > > > there a megapatch posted somewhere (I think I saw one
> >> but
> >> > it
> >> > > > > seemed
> >> > > > > >> > > stale
> >> > > > > >> > > > > and then I 'lost' the tab). Sorry for dumb question.
> >> > > > > >> > > > > St.Ack
> >> > > > > >> > > > >
> >> > > > > >> > > > > On Mon, Sep 12, 2016 at 12:01 PM, Stack <
> >> stack@duboce.net
> >> > >
> >> > > > > wrote:
> >> > > > > >> > > > >
> >> > > > > >> > > > > > Late to the game. A few comments after rereading
> this
> >> > > thread
> >> > > > > as a
> >> > > > > >> > > > 'user'.
> >> > > > > >> > > > > >
> >> > > > > >> > > > > > + Before merge, a user-facing feature like this
> >> should
> >> > > work
> >> > > > > (If
> >> > > > > >> > this
> >> > > > > >> > > is
> >> > > > > >> > > > > "higher-bar
> >> > > > > >> > > > > > for new features", bring it on -- smile).
> >> > > > > >> > > > > > + As a user, I tried the branch with tools after
> >> > reviewing
> >> > > > the
> >> > > > > >> > > > > just-posted
> >> > > > > >> > > > > > doc. I had an 'interesting' experience (left
> >> comments up
> >> > > on
> >> > > > > >> > issue). I
> >> > > > > >> > > > > think
> >> > > > > >> > > > > > the tooling/doc. important to get right. If it
> breaks
> >> > > easily
> >> > > > > or
> >> > > > > >> is
> >> > > > > >> > > > > > inconsistent (or lacks 'polish'), operators will
> >> judge
> >> > the
> >> > > > > whole
> >> > > > > >> > > > > > backup/restore tooling chain as not trustworthy and
> >> > > abandon
> >> > > > > it.
> >> > > > > >> > Lets
> >> > > > > >> > > > not
> >> > > > > >> > > > > > have this happen to this feature.
> >> > > > > >> > > > > > + Matteo's suggestion (with a helpful starter list)
> >> that
> >> > > > there
> >> > > > > >> > needs
> >> > > > > >> > > to
> >> > > > > >> > > > > be
> >> > > > > >> > > > > > explicit qualification on what is actually being
> >> > delivered
> >> > > > --
> >> > > > > >> > > > including a
> >> > > > > >> > > > > > listing of limitations (some look serious such as
> >> data
> >> > > bleed
> >> > > > > from
> >> > > > > >> > > other
> >> > > > > >> > > > > > regions in WALs, but maybe I don't care for my use
> >> > > case...)
> >> > > > --
> >> > > > > >> > needs
> >> > > > > >> > > to
> >> > > > > >> > > > > > accompany the merge. Lets fold them into the user
> >> doc.
> >> > in
> >> > > > the
> >> > > > > >> > > technical
> >> > > > > >> > > > > > overview area as suggested so user expectations are
> >> > > properly
> >> > > > > >> > managed
> >> > > > > >> > > > > > (otherwise, they expect the world and will just
> give
> >> up
> >> > > when
> >> > > > > we
> >> > > > > >> > fall
> >> > > > > >> > > > > > short). Vladimir did a list of what is in each of
> the
> >> > > phases
> >> > > > > >> above
> >> > > > > >> > > > which
> >> > > > > >> > > > > > would serve as a good start.
> >> > > > > >> > > > > > + Is this feature 'experimental' (Matteo asks
> above).
> >> > I'd
> >> > > > > prefer
> >> > > > > >> it
> >> > > > > >> > > is
> >> > > > > >> > > > > > not. If it is, it should be labelled all over that
> >> it is
> >> > > > so. I
> >> > > > > >> see
> >> > > > > >> > > > > current
> >> > > > > >> > > > > > state called out as a '... technical preview
> >> feature'.
> >> > > Does
> >> > > > > this
> >> > > > > >> > mean
> >> > > > > >> > > > > > not-for-users?
> >> > > > > >> > > > > >
> >> > > > > >> > > > > > St.Ack
> >> > > > > >> > > > > >
> >> > > > > >> > > > > >
> >> > > > > >> > > > > >
> >> > > > > >> > > > > >
> >> > > > > >> > > > > >
> >> > > > > >> > > > > >
> >> > > > > >> > > > > >
> >> > > > > >> > > > > >
> >> > > > > >> > > > > >
> >> > > > > >> > > > > >
> >> > > > > >> > > > > >
> >> > > > > >> > > > > > On Mon, Sep 12, 2016 at 8:03 AM, Ted Yu <
> >> > > > yuzhihong@gmail.com>
> >> > > > > >> > wrote:
> >> > > > > >> > > > > >
> >> > > > > >> > > > > >> Sean:
> >> > > > > >> > > > > >> Do you have more comments ?
> >> > > > > >> > > > > >>
> >> > > > > >> > > > > >> Cheers
> >> > > > > >> > > > > >>
> >> > > > > >> > > > > >> On Fri, Sep 9, 2016 at 1:42 PM, Vladimir Rodionov
> <
> >> > > > > >> > > > > vladrodionov@gmail.com
> >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> wrote:
> >> > > > > >> > > > > >>
> >> > > > > >> > > > > >> > Sean,
> >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > Backup/Restore can fail due to various reasons:
> >> > network
> >> > > > > outage
> >> > > > > >> > > > > (cluster
> >> > > > > >> > > > > >> > wide), various time-outs in HBase and HDFS
> layer,
> >> M/R
> >> > > > > failure
> >> > > > > >> > due
> >> > > > > >> > > to
> >> > > > > >> > > > > >> "HDFS
> >> > > > > >> > > > > >> > exceeded quota", user error (manual deletion of
> >> data)
> >> > > and
> >> > > > > so
> >> > > > > >> on
> >> > > > > >> > so
> >> > > > > >> > > > on.
> >> > > > > >> > > > > >> That
> >> > > > > >> > > > > >> > is impossible to enumerate all possible types of
> >> > > failures
> >> > > > > in a
> >> > > > > >> > > > > >> distributed
> >> > > > > >> > > > > >> > system - that is not our goal/task.
> >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > We focus completely on backup system table
> >> > consistency
> >> > > > in a
> >> > > > > >> > > presence
> >> > > > > >> > > > > of
> >> > > > > >> > > > > >> any
> >> > > > > >> > > > > >> > type of failure. That is what I call "tolerance
> to
> >> > > > > failures".
> >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > On a failure:
> >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > BACKUP. All backup system information (prior to
> >> > backup)
> >> > > > > will
> >> > > > > >> be
> >> > > > > >> > > > > restored
> >> > > > > >> > > > > >> > and all temporary data, related to a failed
> >> session,
> >> > in
> >> > > > > HDFS
> >> > > > > >> > will
> >> > > > > >> > > be
> >> > > > > >> > > > > >> > deleted
> >> > > > > >> > > > > >> > RESTORE. We do not care about system data,
> because
> >> > > > restore
> >> > > > > >> does
> >> > > > > >> > > not
> >> > > > > >> > > > > >> change
> >> > > > > >> > > > > >> > it. Temporary data in HDFS will be cleaned up
> and
> >> > table
> >> > > > > will
> >> > > > > >> be
> >> > > > > >> > > in a
> >> > > > > >> > > > > >> state
> >> > > > > >> > > > > >> > back to where it was before operation started.
> >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > This is what user should expect in case of a
> >> failure.
> >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > -Vlad
> >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > -Vlad
> >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > On Fri, Sep 9, 2016 at 12:56 PM, Sean Busbey <
> >> > > > > >> busbey@apache.org
> >> > > > > >> > >
> >> > > > > >> > > > > wrote:
> >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > > Failing in a consistent way, with docs that
> >> explain
> >> > > the
> >> > > > > >> > various
> >> > > > > >> > > > > >> > > expected failures would be sufficient.
> >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> > > On Fri, Sep 9, 2016 at 12:16 PM, Vladimir
> >> Rodionov
> >> > > > > >> > > > > >> > > <vl...@gmail.com> wrote:
> >> > > > > >> > > > > >> > > > Do not worry Sean, doc is coming today as a
> >> > preview
> >> > > > and
> >> > > > > >> our
> >> > > > > >> > > > writer
> >> > > > > >> > > > > >> > Frank
> >> > > > > >> > > > > >> > > > will be working on a putting  it into Apache
> >> > repo.
> >> > > > > >> Timeline
> >> > > > > >> > > > > depends
> >> > > > > >> > > > > >> on
> >> > > > > >> > > > > >> > > > Franks schedule but I hope we will get it
> >> rather
> >> > > > sooner
> >> > > > > >> than
> >> > > > > >> > > > > later.
> >> > > > > >> > > > > >> > > >
> >> > > > > >> > > > > >> > > > As for failure testing, we are focusing only
> >> on a
> >> > > > > >> consistent
> >> > > > > >> > > > state
> >> > > > > >> > > > > >> of
> >> > > > > >> > > > > >> > > > backup system data in a presence of any type
> >> of
> >> > > > > failures,
> >> > > > > >> We
> >> > > > > >> > > are
> >> > > > > >> > > > > not
> >> > > > > >> > > > > >> > > going
> >> > > > > >> > > > > >> > > > to implement  anything more "fancy", than
> >> that.
> >> > We
> >> > > > > allow
> >> > > > > >> > both:
> >> > > > > >> > > > > >> backup
> >> > > > > >> > > > > >> > and
> >> > > > > >> > > > > >> > > > restore to fail. What we do not allow is to
> >> have
> >> > > > system
> >> > > > > >> data
> >> > > > > >> > > > > >> corrupted.
> >> > > > > >> > > > > >> > > > Will it suffice for you? Do you have any
> other
> >> > > > > concerns,
> >> > > > > >> you
> >> > > > > >> > > > want
> >> > > > > >> > > > > >> us to
> >> > > > > >> > > > > >> > > > address?
> >> > > > > >> > > > > >> > > >
> >> > > > > >> > > > > >> > > > -Vlad
> >> > > > > >> > > > > >> > > >
> >> > > > > >> > > > > >> > > >
> >> > > > > >> > > > > >> > > > On Fri, Sep 9, 2016 at 10:56 AM, Sean
> Busbey <
> >> > > > > >> > > busbey@apache.org
> >> > > > > >> > > > >
> >> > > > > >> > > > > >> > wrote:
> >> > > > > >> > > > > >> > > >
> >> > > > > >> > > > > >> > > >> "docs will come to Apache soon" does not
> >> address
> >> > > my
> >> > > > > >> concern
> >> > > > > >> > > > > around
> >> > > > > >> > > > > >> > docs
> >> > > > > >> > > > > >> > > at
> >> > > > > >> > > > > >> > > >> all, unless said docs have already made it
> >> into
> >> > > the
> >> > > > > >> project
> >> > > > > >> > > > > repo. I
> >> > > > > >> > > > > >> > > don't
> >> > > > > >> > > > > >> > > >> want third party resources for using a
> major
> >> and
> >> > > > > >> important
> >> > > > > >> > > > > feature
> >> > > > > >> > > > > >> of
> >> > > > > >> > > > > >> > > the
> >> > > > > >> > > > > >> > > >> project, I want us to provide end users
> with
> >> > what
> >> > > > they
> >> > > > > >> need
> >> > > > > >> > > to
> >> > > > > >> > > > > get
> >> > > > > >> > > > > >> the
> >> > > > > >> > > > > >> > > job
> >> > > > > >> > > > > >> > > >> done.
> >> > > > > >> > > > > >> > > >>
> >> > > > > >> > > > > >> > > >> I see some calls for patience on the
> failure
> >> > > > testing,
> >> > > > > but
> >> > > > > >> > the
> >> > > > > >> > > > > >> appeal
> >> > > > > >> > > > > >> > to
> >> > > > > >> > > > > >> > > us
> >> > > > > >> > > > > >> > > >> having done a bad job of requiring proper
> >> tests
> >> > of
> >> > > > > >> previous
> >> > > > > >> > > > > >> features
> >> > > > > >> > > > > >> > > just
> >> > > > > >> > > > > >> > > >> makes me more concerned about not getting
> >> them
> >> > > > here. I
> >> > > > > >> > don't
> >> > > > > >> > > > want
> >> > > > > >> > > > > >> to
> >> > > > > >> > > > > >> > set
> >> > > > > >> > > > > >> > > >> yet another bad example that will then be
> >> > pointed
> >> > > to
> >> > > > > in
> >> > > > > >> the
> >> > > > > >> > > > > future.
> >> > > > > >> > > > > >> > > >>
> >> > > > > >> > > > > >> > > >> On Sep 8, 2016 10:50, "Ted Yu" <
> >> > > yuzhihong@gmail.com
> >> > > > >
> >> > > > > >> > wrote:
> >> > > > > >> > > > > >> > > >>
> >> > > > > >> > > > > >> > > >> > Is there any concern which is not
> >> addressed ?
> >> > > > > >> > > > > >> > > >> >
> >> > > > > >> > > > > >> > > >> > Do we need another Vote thread ?
> >> > > > > >> > > > > >> > > >> >
> >> > > > > >> > > > > >> > > >> > Thanks
> >> > > > > >> > > > > >> > > >> >
> >> > > > > >> > > > > >> > > >> > On Thu, Sep 8, 2016 at 9:21 AM, Andrew
> >> > Purtell <
> >> > > > > >> > > > > >> apurtell@apache.org
> >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> > > >> > wrote:
> >> > > > > >> > > > > >> > > >> >
> >> > > > > >> > > > > >> > > >> > > Vlad,
> >> > > > > >> > > > > >> > > >> > >
> >> > > > > >> > > > > >> > > >> > > I apologize for using the term
> >> 'half-baked'
> >> > > in a
> >> > > > > way
> >> > > > > >> > that
> >> > > > > >> > > > > could
> >> > > > > >> > > > > >> > > seem a
> >> > > > > >> > > > > >> > > >> > > description of HBASE-7912. I meant that
> >> as a
> >> > > > > general
> >> > > > > >> > > > > >> hypothetical.
> >> > > > > >> > > > > >> > > >> > >
> >> > > > > >> > > > > >> > > >> > > On Wed, Sep 7, 2016 at 9:36 AM,
> Vladimir
> >> > > > Rodionov
> >> > > > > <
> >> > > > > >> > > > > >> > > >> > vladrodionov@gmail.com>
> >> > > > > >> > > > > >> > > >> > > wrote:
> >> > > > > >> > > > > >> > > >> > >
> >> > > > > >> > > > > >> > > >> > > > >> I'm not sure that "There is
> already
> >> > lots
> >> > > of
> >> > > > > >> > > half-baked
> >> > > > > >> > > > > >> code
> >> > > > > >> > > > > >> > in
> >> > > > > >> > > > > >> > > the
> >> > > > > >> > > > > >> > > >> > > > branch,
> >> > > > > >> > > > > >> > > >> > > > so what's the harm in adding more?"
> >> > > > > >> > > > > >> > > >> > > >
> >> > > > > >> > > > > >> > > >> > > > I meant - not production - ready yet.
> >> This
> >> > > is
> >> > > > > 2.0
> >> > > > > >> > > > > development
> >> > > > > >> > > > > >> > > branch
> >> > > > > >> > > > > >> > > >> > and,
> >> > > > > >> > > > > >> > > >> > > > hence many features are in works,
> >> > > > > >> > > > > >> > > >> > > > not being tested well etc. I do not
> >> > consider
> >> > > > > backup
> >> > > > > >> > as
> >> > > > > >> > > > half
> >> > > > > >> > > > > >> > baked
> >> > > > > >> > > > > >> > > >> > > feature -
> >> > > > > >> > > > > >> > > >> > > > it has passed our internal QA and has
> >> very
> >> > > > good
> >> > > > > >> doc,
> >> > > > > >> > > > which
> >> > > > > >> > > > > we
> >> > > > > >> > > > > >> > will
> >> > > > > >> > > > > >> > > >> > > provide
> >> > > > > >> > > > > >> > > >> > > > to Apache shortly.
> >> > > > > >> > > > > >> > > >> > > >
> >> > > > > >> > > > > >> > > >> > > > -Vlad
> >> > > > > >> > > > > >> > > >> > > >
> >> > > > > >> > > > > >> > > >> > > > On Wed, Sep 7, 2016 at 9:13 AM,
> Andrew
> >> > > > Purtell <
> >> > > > > >> > > > > >> > > apurtell@apache.org>
> >> > > > > >> > > > > >> > > >> > > > wrote:
> >> > > > > >> > > > > >> > > >> > > >
> >> > > > > >> > > > > >> > > >> > > > > We shouldn't admit half baked
> changes
> >> > that
> >> > > > > won't
> >> > > > > >> be
> >> > > > > >> > > > > >> finished.
> >> > > > > >> > > > > >> > > >> However
> >> > > > > >> > > > > >> > > >> > > in
> >> > > > > >> > > > > >> > > >> > > > > this case the crew working on this
> >> > feature
> >> > > > are
> >> > > > > >> long
> >> > > > > >> > > > > timers
> >> > > > > >> > > > > >> and
> >> > > > > >> > > > > >> > > less
> >> > > > > >> > > > > >> > > >> > > > likely
> >> > > > > >> > > > > >> > > >> > > > > than just about anyone to leave
> >> > something
> >> > > > in a
> >> > > > > >> half
> >> > > > > >> > > > baked
> >> > > > > >> > > > > >> > > state. Of
> >> > > > > >> > > > > >> > > >> > > > course
> >> > > > > >> > > > > >> > > >> > > > > there is no guarantee how anything
> >> will
> >> > > turn
> >> > > > > out,
> >> > > > > >> > > but I
> >> > > > > >> > > > > am
> >> > > > > >> > > > > >> > > willing
> >> > > > > >> > > > > >> > > >> to
> >> > > > > >> > > > > >> > > >> > > > take
> >> > > > > >> > > > > >> > > >> > > > > a little on faith if they feel
> their
> >> > best
> >> > > > path
> >> > > > > >> > > forward
> >> > > > > >> > > > > now
> >> > > > > >> > > > > >> is
> >> > > > > >> > > > > >> > to
> >> > > > > >> > > > > >> > > >> > merge
> >> > > > > >> > > > > >> > > >> > > to
> >> > > > > >> > > > > >> > > >> > > > > trunk. I only wish I had bandwidth
> to
> >> > have
> >> > > > > done
> >> > > > > >> > some
> >> > > > > >> > > > real
> >> > > > > >> > > > > >> > > kicking
> >> > > > > >> > > > > >> > > >> of
> >> > > > > >> > > > > >> > > >> > > the
> >> > > > > >> > > > > >> > > >> > > > > tires by now. Maybe this week.
> >> > > > > >> > > > > >> > > >> > > > >
> >> > > > > >> > > > > >> > > >> > > > > (Yes, I'm using some of that time
> for
> >> > this
> >> > > > > email
> >> > > > > >> > :-)
> >> > > > > >> > > > but
> >> > > > > >> > > > > I
> >> > > > > >> > > > > >> > type
> >> > > > > >> > > > > >> > > >> > fast.)
> >> > > > > >> > > > > >> > > >> > > > >
> >> > > > > >> > > > > >> > > >> > > > > That said, I would like to agitate
> >> for
> >> > > > making
> >> > > > > 2.0
> >> > > > > >> > > more
> >> > > > > >> > > > > real
> >> > > > > >> > > > > >> > and
> >> > > > > >> > > > > >> > > >> spend
> >> > > > > >> > > > > >> > > >> > > > some
> >> > > > > >> > > > > >> > > >> > > > > time on it now that I'm winding
> down
> >> > with
> >> > > > > 0.98. I
> >> > > > > >> > > think
> >> > > > > >> > > > > >> that
> >> > > > > >> > > > > >> > > means
> >> > > > > >> > > > > >> > > >> > > > > branching for 2.0 real soon now and
> >> even
> >> > > > > evicting
> >> > > > > >> > > > things
> >> > > > > >> > > > > >> from
> >> > > > > >> > > > > >> > > 2.0
> >> > > > > >> > > > > >> > > >> > > branch
> >> > > > > >> > > > > >> > > >> > > > > that aren't finished or stable,
> >> leaving
> >> > > them
> >> > > > > only
> >> > > > > >> > > once
> >> > > > > >> > > > > >> again
> >> > > > > >> > > > > >> > in
> >> > > > > >> > > > > >> > > the
> >> > > > > >> > > > > >> > > >> > > > master
> >> > > > > >> > > > > >> > > >> > > > > branch. Or, maybe just evicting
> them.
> >> > > Let's
> >> > > > > take
> >> > > > > >> it
> >> > > > > >> > > > case
> >> > > > > >> > > > > by
> >> > > > > >> > > > > >> > > case.
> >> > > > > >> > > > > >> > > >> > > > >
> >> > > > > >> > > > > >> > > >> > > > > I think this feature can come in
> >> > > relatively
> >> > > > > >> safely.
> >> > > > > >> > > As
> >> > > > > >> > > > > >> added
> >> > > > > >> > > > > >> > > >> > insurance,
> >> > > > > >> > > > > >> > > >> > > > > let's admit the possibility it
> could
> >> be
> >> > > > > reverted
> >> > > > > >> on
> >> > > > > >> > > the
> >> > > > > >> > > > > 2.0
> >> > > > > >> > > > > >> > > branch
> >> > > > > >> > > > > >> > > >> if
> >> > > > > >> > > > > >> > > >> > > > folks
> >> > > > > >> > > > > >> > > >> > > > > working on stabilizing 2.0 decide
> to
> >> > evict
> >> > > > it
> >> > > > > >> > because
> >> > > > > >> > > > it
> >> > > > > >> > > > > is
> >> > > > > >> > > > > >> > > >> > unfinished
> >> > > > > >> > > > > >> > > >> > > or
> >> > > > > >> > > > > >> > > >> > > > > unstable, because that certainly
> can
> >> > > > happen. I
> >> > > > > >> > would
> >> > > > > >> > > > > >> expect if
> >> > > > > >> > > > > >> > > talk
> >> > > > > >> > > > > >> > > >> > > like
> >> > > > > >> > > > > >> > > >> > > > > that starts, we'd get help
> finishing
> >> or
> >> > > > > >> stabilizing
> >> > > > > >> > > > > what's
> >> > > > > >> > > > > >> > under
> >> > > > > >> > > > > >> > > >> > > > discussion
> >> > > > > >> > > > > >> > > >> > > > > for revert. Or, we'd have a revert.
> >> > Either
> >> > > > way
> >> > > > > >> the
> >> > > > > >> > > > > outcome
> >> > > > > >> > > > > >> is
> >> > > > > >> > > > > >> > > >> > > acceptable.
> >> > > > > >> > > > > >> > > >> > > > >
> >> > > > > >> > > > > >> > > >> > > > >
> >> > > > > >> > > > > >> > > >> > > > > On Wed, Sep 7, 2016 at 8:56 AM,
> Dima
> >> > > Spivak
> >> > > > <
> >> > > > > >> > > > > >> > > dimaspivak@apache.org
> >> > > > > >> > > > > >> > > >> >
> >> > > > > >> > > > > >> > > >> > > > wrote:
> >> > > > > >> > > > > >> > > >> > > > >
> >> > > > > >> > > > > >> > > >> > > > > > I'm not sure that "There is
> already
> >> > lots
> >> > > > of
> >> > > > > >> > > > half-baked
> >> > > > > >> > > > > >> code
> >> > > > > >> > > > > >> > in
> >> > > > > >> > > > > >> > > >> the
> >> > > > > >> > > > > >> > > >> > > > > branch,
> >> > > > > >> > > > > >> > > >> > > > > > so what's the harm in adding
> more?"
> >> > is a
> >> > > > > good
> >> > > > > >> > code
> >> > > > > >> > > > > commit
> >> > > > > >> > > > > >> > > >> > philosophy
> >> > > > > >> > > > > >> > > >> > > > for
> >> > > > > >> > > > > >> > > >> > > > > a
> >> > > > > >> > > > > >> > > >> > > > > > fault-tolerant distributed data
> >> store.
> >> > > ;)
> >> > > > > >> > > > > >> > > >> > > > > >
> >> > > > > >> > > > > >> > > >> > > > > > More seriously, a lack of test
> >> > coverage
> >> > > > for
> >> > > > > >> > > existing
> >> > > > > >> > > > > >> > features
> >> > > > > >> > > > > >> > > >> > > shouldn't
> >> > > > > >> > > > > >> > > >> > > > > be
> >> > > > > >> > > > > >> > > >> > > > > > used as justification for
> >> introducing
> >> > > new
> >> > > > > >> > features
> >> > > > > >> > > > with
> >> > > > > >> > > > > >> the
> >> > > > > >> > > > > >> > > same
> >> > > > > >> > > > > >> > > >> > > > > > shortcomings. Ultimately, it's
> the
> >> end
> >> > > > user
> >> > > > > who
> >> > > > > >> > > will
> >> > > > > >> > > > > feel
> >> > > > > >> > > > > >> > the
> >> > > > > >> > > > > >> > > >> pain,
> >> > > > > >> > > > > >> > > >> > > so
> >> > > > > >> > > > > >> > > >> > > > > > shouldn't we do everything we can
> >> to
> >> > > > > mitigate
> >> > > > > >> > that?
> >> > > > > >> > > > > >> > > >> > > > > >
> >> > > > > >> > > > > >> > > >> > > > > > -Dima
> >> > > > > >> > > > > >> > > >> > > > > >
> >> > > > > >> > > > > >> > > >> > > > > > On Wed, Sep 7, 2016 at 8:46 AM,
> >> > Vladimir
> >> > > > > >> > Rodionov <
> >> > > > > >> > > > > >> > > >> > > > > vladrodionov@gmail.com>
> >> > > > > >> > > > > >> > > >> > > > > > wrote:
> >> > > > > >> > > > > >> > > >> > > > > >
> >> > > > > >> > > > > >> > > >> > > > > > > Sean,
> >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > >> > > > > > > * have docs
> >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > >> > > > > > > Agree. We have a doc and backup
> >> is
> >> > the
> >> > > > > most
> >> > > > > >> > > > > documented
> >> > > > > >> > > > > >> > > feature
> >> > > > > >> > > > > >> > > >> > :),
> >> > > > > >> > > > > >> > > >> > > we
> >> > > > > >> > > > > >> > > >> > > > > > will
> >> > > > > >> > > > > >> > > >> > > > > > > release it shortly to Apache.
> >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > >> > > > > > > * have sunny-day correctness
> >> tests
> >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > >> > > > > > > Feature has  close to 60 test
> >> cases,
> >> > > > which
> >> > > > > >> run
> >> > > > > >> > > for
> >> > > > > >> > > > > >> approx
> >> > > > > >> > > > > >> > 30
> >> > > > > >> > > > > >> > > >> min.
> >> > > > > >> > > > > >> > > >> > > We
> >> > > > > >> > > > > >> > > >> > > > > can
> >> > > > > >> > > > > >> > > >> > > > > > > add more, if community do not
> >> mind
> >> > :)
> >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > >> > > > > > > * have
> >> > correctness-in-face-of-failure
> >> > > > > tests
> >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > >> > > > > > > Any examples of these tests in
> >> > > existing
> >> > > > > >> > features?
> >> > > > > >> > > > In
> >> > > > > >> > > > > >> > works,
> >> > > > > >> > > > > >> > > we
> >> > > > > >> > > > > >> > > >> > > have a
> >> > > > > >> > > > > >> > > >> > > > > > clear
> >> > > > > >> > > > > >> > > >> > > > > > > understanding of what should be
> >> done
> >> > > by
> >> > > > > the
> >> > > > > >> > time
> >> > > > > >> > > of
> >> > > > > >> > > > > 2.0
> >> > > > > >> > > > > >> > > >> release.
> >> > > > > >> > > > > >> > > >> > > > > > > That is very close goal for us,
> >> to
> >> > > > verify
> >> > > > > IT
> >> > > > > >> > > monkey
> >> > > > > >> > > > > for
> >> > > > > >> > > > > >> > > >> existing
> >> > > > > >> > > > > >> > > >> > > > code.
> >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > >> > > > > > > * don't rely on things outside
> of
> >> > > HBase
> >> > > > > for
> >> > > > > >> > > normal
> >> > > > > >> > > > > >> > operation
> >> > > > > >> > > > > >> > > >> > (okay
> >> > > > > >> > > > > >> > > >> > > > for
> >> > > > > >> > > > > >> > > >> > > > > > > advanced operation)
> >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > >> > > > > > > We do not.
> >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > >> > > > > > > Enormous time has been spent
> >> already
> >> > > on
> >> > > > > the
> >> > > > > >> > > > > development
> >> > > > > >> > > > > >> > and
> >> > > > > >> > > > > >> > > >> > testing
> >> > > > > >> > > > > >> > > >> > > > the
> >> > > > > >> > > > > >> > > >> > > > > > > feature, it has passed our
> >> internal
> >> > > > tests
> >> > > > > and
> >> > > > > >> > > many
> >> > > > > >> > > > > >> rounds
> >> > > > > >> > > > > >> > of
> >> > > > > >> > > > > >> > > >> code
> >> > > > > >> > > > > >> > > >> > > > > reviews
> >> > > > > >> > > > > >> > > >> > > > > > > by HBase committers. We do not
> >> mind
> >> > if
> >> > > > > >> someone
> >> > > > > >> > > from
> >> > > > > >> > > > > >> HBase
> >> > > > > >> > > > > >> > > >> > community
> >> > > > > >> > > > > >> > > >> > > > > > > (outside of HW) will review the
> >> > code,
> >> > > > but
> >> > > > > it
> >> > > > > >> > will
> >> > > > > >> > > > > >> probably
> >> > > > > >> > > > > >> > > >> takes
> >> > > > > >> > > > > >> > > >> > > > > forever
> >> > > > > >> > > > > >> > > >> > > > > > to
> >> > > > > >> > > > > >> > > >> > > > > > > wait for volunteer?, the
> feature
> >> is
> >> > > > quite
> >> > > > > >> large
> >> > > > > >> > > > (1MB+
> >> > > > > >> > > > > >> > > >> cumulative
> >> > > > > >> > > > > >> > > >> > > > patch)
> >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > >> > > > > > > 2.0 branch is full of half
> baked
> >> > > > features,
> >> > > > > >> most
> >> > > > > >> > > of
> >> > > > > >> > > > > them
> >> > > > > >> > > > > >> > are
> >> > > > > >> > > > > >> > > in
> >> > > > > >> > > > > >> > > >> > > active
> >> > > > > >> > > > > >> > > >> > > > > > > development, therefore I am not
> >> > > > following
> >> > > > > you
> >> > > > > >> > > here,
> >> > > > > >> > > > > >> Sean?
> >> > > > > >> > > > > >> > > Why
> >> > > > > >> > > > > >> > > >> > > > > HBASE-7912
> >> > > > > >> > > > > >> > > >> > > > > > is
> >> > > > > >> > > > > >> > > >> > > > > > > not good enough yet to be
> >> integrated
> >> > > > into
> >> > > > > 2.0
> >> > > > > >> > > > branch?
> >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > >> > > > > > > -Vlad
> >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > >> > > > > > > On Wed, Sep 7, 2016 at 8:23 AM,
> >> Sean
> >> > > > > Busbey <
> >> > > > > >> > > > > >> > > busbey@apache.org
> >> > > > > >> > > > > >> > > >> >
> >> > > > > >> > > > > >> > > >> > > > wrote:
> >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > >> > > > > > > > On Tue, Sep 6, 2016 at 10:36
> >> PM,
> >> > > Josh
> >> > > > > >> Elser <
> >> > > > > >> > > > > >> > > >> > > josh.elser@gmail.com>
> >> > > > > >> > > > > >> > > >> > > > > > > wrote:
> >> > > > > >> > > > > >> > > >> > > > > > > > > So, the answer to Sean's
> >> > original
> >> > > > > >> question
> >> > > > > >> > is
> >> > > > > >> > > > "as
> >> > > > > >> > > > > >> > > robust as
> >> > > > > >> > > > > >> > > >> > > > > snapshots
> >> > > > > >> > > > > >> > > >> > > > > > > > > presently are"?
> >> (independence of
> >> > > > > >> > > backup/restore
> >> > > > > >> > > > > >> > failure
> >> > > > > >> > > > > >> > > >> > > tolerance
> >> > > > > >> > > > > >> > > >> > > > > > from
> >> > > > > >> > > > > >> > > >> > > > > > > > > snapshot failure tolerance)
> >> > > > > >> > > > > >> > > >> > > > > > > > >
> >> > > > > >> > > > > >> > > >> > > > > > > > > Is this just a question WRT
> >> > > context
> >> > > > of
> >> > > > > >> the
> >> > > > > >> > > > > change,
> >> > > > > >> > > > > >> or
> >> > > > > >> > > > > >> > > is it
> >> > > > > >> > > > > >> > > >> > > means
> >> > > > > >> > > > > >> > > >> > > > > > for a
> >> > > > > >> > > > > >> > > >> > > > > > > > veto
> >> > > > > >> > > > > >> > > >> > > > > > > > > from you, Sean? Just trying
> >> to
> >> > > make
> >> > > > > sure
> >> > > > > >> > I'm
> >> > > > > >> > > > > >> following
> >> > > > > >> > > > > >> > > >> along
> >> > > > > >> > > > > >> > > >> > > > > > > adequately.
> >> > > > > >> > > > > >> > > >> > > > > > > > >
> >> > > > > >> > > > > >> > > >> > > > > > > > >
> >> > > > > >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > > > >> > > >> > > > > > > > I'd say ATM I'm -0, bordering
> >> on
> >> > -1
> >> > > > but
> >> > > > > not
> >> > > > > >> > for
> >> > > > > >> > > > > >> reasons
> >> > > > > >> > > > > >> > I
> >> > > > > >> > > > > >> > > can
> >> > > > > >> > > > > >> > > >> > > > > > articulate
> >> > > > > >> > > > > >> > > >> > > > > > > > well.
> >> > > > > >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > > > >> > > >> > > > > > > > Here's an attempt.
> >> > > > > >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > > > >> > > >> > > > > > > > We've been trying to move,
> as a
> >> > > > > community,
> >> > > > > >> > > > towards
> >> > > > > >> > > > > >> > > minimizing
> >> > > > > >> > > > > >> > > >> > > risk
> >> > > > > >> > > > > >> > > >> > > > to
> >> > > > > >> > > > > >> > > >> > > > > > > > downstream folks by getting
> >> > > "complete
> >> > > > > >> enough
> >> > > > > >> > > for
> >> > > > > >> > > > > use"
> >> > > > > >> > > > > >> > > gates
> >> > > > > >> > > > > >> > > >> in
> >> > > > > >> > > > > >> > > >> > > > place
> >> > > > > >> > > > > >> > > >> > > > > > > > before we introduce new
> >> features.
> >> > > This
> >> > > > > was
> >> > > > > >> > > > spurred
> >> > > > > >> > > > > >> by a
> >> > > > > >> > > > > >> > > some
> >> > > > > >> > > > > >> > > >> > > > features
> >> > > > > >> > > > > >> > > >> > > > > > > > getting in half-baked and
> never
> >> > > making
> >> > > > > it
> >> > > > > >> to
> >> > > > > >> > > "can
> >> > > > > >> > > > > >> really
> >> > > > > >> > > > > >> > > use"
> >> > > > > >> > > > > >> > > >> > > > status
> >> > > > > >> > > > > >> > > >> > > > > > > > (I'm thinking of distributed
> >> log
> >> > > > replay
> >> > > > > and
> >> > > > > >> > the
> >> > > > > >> > > > > >> zk-less
> >> > > > > >> > > > > >> > > >> > > assignment
> >> > > > > >> > > > > >> > > >> > > > > > > > stuff, I don't recall if
> there
> >> was
> >> > > > > more).
> >> > > > > >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > > > >> > > >> > > > > > > > The gates, generally,
> included
> >> > > things
> >> > > > > like:
> >> > > > > >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > > > >> > > >> > > > > > > > * have docs
> >> > > > > >> > > > > >> > > >> > > > > > > > * have sunny-day correctness
> >> tests
> >> > > > > >> > > > > >> > > >> > > > > > > > * have
> >> > > correctness-in-face-of-failure
> >> > > > > tests
> >> > > > > >> > > > > >> > > >> > > > > > > > * don't rely on things
> outside
> >> of
> >> > > > HBase
> >> > > > > for
> >> > > > > >> > > > normal
> >> > > > > >> > > > > >> > > operation
> >> > > > > >> > > > > >> > > >> > > (okay
> >> > > > > >> > > > > >> > > >> > > > > for
> >> > > > > >> > > > > >> > > >> > > > > > > > advanced operation)
> >> > > > > >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > > > >> > > >> > > > > > > > As an example, we kept the
> MOB
> >> > work
> >> > > > off
> >> > > > > in
> >> > > > > >> a
> >> > > > > >> > > > branch
> >> > > > > >> > > > > >> and
> >> > > > > >> > > > > >> > > out
> >> > > > > >> > > > > >> > > >> of
> >> > > > > >> > > > > >> > > >> > > > master
> >> > > > > >> > > > > >> > > >> > > > > > > > until it could pass these
> >> > criteria.
> >> > > > The
> >> > > > > big
> >> > > > > >> > > > > exemption
> >> > > > > >> > > > > >> > > we've
> >> > > > > >> > > > > >> > > >> had
> >> > > > > >> > > > > >> > > >> > > to
> >> > > > > >> > > > > >> > > >> > > > > > > > this was the hbase-spark
> >> > > integration,
> >> > > > > where
> >> > > > > >> > we
> >> > > > > >> > > > all
> >> > > > > >> > > > > >> > agreed
> >> > > > > >> > > > > >> > > it
> >> > > > > >> > > > > >> > > >> > > could
> >> > > > > >> > > > > >> > > >> > > > > > > > land in master because it was
> >> very
> >> > > > well
> >> > > > > >> > > isolated
> >> > > > > >> > > > > (the
> >> > > > > >> > > > > >> > > slide
> >> > > > > >> > > > > >> > > >> > away
> >> > > > > >> > > > > >> > > >> > > > from
> >> > > > > >> > > > > >> > > >> > > > > > > > including docs as a
> first-class
> >> > part
> >> > > > of
> >> > > > > >> > > building
> >> > > > > >> > > > up
> >> > > > > >> > > > > >> that
> >> > > > > >> > > > > >> > > >> > > > integration
> >> > > > > >> > > > > >> > > >> > > > > > > > has led me to doubt the
> wisdom
> >> of
> >> > > this
> >> > > > > >> > > decision).
> >> > > > > >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > > > >> > > >> > > > > > > > We've also been treating
> >> inclusion
> >> > > in
> >> > > > a
> >> > > > > >> > > "probably
> >> > > > > >> > > > > >> will
> >> > > > > >> > > > > >> > be
> >> > > > > >> > > > > >> > > >> > > released
> >> > > > > >> > > > > >> > > >> > > > to
> >> > > > > >> > > > > >> > > >> > > > > > > > downstream" branches as a
> >> higher
> >> > > bar,
> >> > > > > >> > requiring
> >> > > > > >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > > > >> > > >> > > > > > > > * don't moderately impact
> >> > > performance
> >> > > > > when
> >> > > > > >> > the
> >> > > > > >> > > > > >> feature
> >> > > > > >> > > > > >> > > isn't
> >> > > > > >> > > > > >> > > >> in
> >> > > > > >> > > > > >> > > >> > > use
> >> > > > > >> > > > > >> > > >> > > > > > > > * don't severely impact
> >> > performance
> >> > > > when
> >> > > > > >> the
> >> > > > > >> > > > > feature
> >> > > > > >> > > > > >> is
> >> > > > > >> > > > > >> > in
> >> > > > > >> > > > > >> > > >> use
> >> > > > > >> > > > > >> > > >> > > > > > > > * either default-to-on or
> show
> >> > > enough
> >> > > > > >> demand
> >> > > > > >> > to
> >> > > > > >> > > > > >> believe
> >> > > > > >> > > > > >> > a
> >> > > > > >> > > > > >> > > >> > > > non-trivial
> >> > > > > >> > > > > >> > > >> > > > > > > > number of folks will turn the
> >> > > feature
> >> > > > on
> >> > > > > >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > > > >> > > >> > > > > > > > The above has kept MOB and
> >> > > hbase-spark
> >> > > > > >> > > > integration
> >> > > > > >> > > > > >> out
> >> > > > > >> > > > > >> > of
> >> > > > > >> > > > > >> > > >> > > branch-1,
> >> > > > > >> > > > > >> > > >> > > > > > > > presumably while they've
> >> "gotten
> >> > > more
> >> > > > > >> stable"
> >> > > > > >> > > in
> >> > > > > >> > > > > >> master
> >> > > > > >> > > > > >> > > from
> >> > > > > >> > > > > >> > > >> > the
> >> > > > > >> > > > > >> > > >> > > > odd
> >> > > > > >> > > > > >> > > >> > > > > > > > vendor inclusion.
> >> > > > > >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > > > >> > > >> > > > > > > > Are we going to have a 2.0
> >> release
> >> > > > > before
> >> > > > > >> the
> >> > > > > >> > > end
> >> > > > > >> > > > > of
> >> > > > > >> > > > > >> the
> >> > > > > >> > > > > >> > > >> year?
> >> > > > > >> > > > > >> > > >> > > > We're
> >> > > > > >> > > > > >> > > >> > > > > > > > coming up on 1.5 years since
> >> the
> >> > > > > release of
> >> > > > > >> > > > version
> >> > > > > >> > > > > >> 1.0;
> >> > > > > >> > > > > >> > > >> seems
> >> > > > > >> > > > > >> > > >> > > like
> >> > > > > >> > > > > >> > > >> > > > > > > > it's about time, though I
> >> haven't
> >> > > seen
> >> > > > > any
> >> > > > > >> > > > concrete
> >> > > > > >> > > > > >> > plans
> >> > > > > >> > > > > >> > > >> this
> >> > > > > >> > > > > >> > > >> > > > year.
> >> > > > > >> > > > > >> > > >> > > > > > > > Presuming we are going to
> have
> >> one
> >> > > by
> >> > > > > the
> >> > > > > >> end
> >> > > > > >> > > of
> >> > > > > >> > > > > the
> >> > > > > >> > > > > >> > > year, it
> >> > > > > >> > > > > >> > > >> > > > seems a
> >> > > > > >> > > > > >> > > >> > > > > > > > bit close to still be adding
> in
> >> > > > > "features
> >> > > > > >> > that
> >> > > > > >> > > > need
> >> > > > > >> > > > > >> > > maturing"
> >> > > > > >> > > > > >> > > >> > on
> >> > > > > >> > > > > >> > > >> > > > the
> >> > > > > >> > > > > >> > > >> > > > > > > > branch.
> >> > > > > >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > > > >> > > >> > > > > > > > The lack of a concrete plan
> for
> >> > 2.0
> >> > > > > keeps
> >> > > > > >> me
> >> > > > > >> > > from
> >> > > > > >> > > > > >> > > considering
> >> > > > > >> > > > > >> > > >> > > these
> >> > > > > >> > > > > >> > > >> > > > > > > > things blocker at the moment.
> >> But
> >> > I
> >> > > > know
> >> > > > > >> > first
> >> > > > > >> > > > hand
> >> > > > > >> > > > > >> how
> >> > > > > >> > > > > >> > > much
> >> > > > > >> > > > > >> > > >> > > > trouble
> >> > > > > >> > > > > >> > > >> > > > > > > > folks have had with other
> >> features
> >> > > > that
> >> > > > > >> have
> >> > > > > >> > > gone
> >> > > > > >> > > > > >> into
> >> > > > > >> > > > > >> > > >> > downstream
> >> > > > > >> > > > > >> > > >> > > > > > > > facing releases without
> >> robustness
> >> > > > > checks
> >> > > > > >> > (i.e.
> >> > > > > >> > > > > >> > > replication),
> >> > > > > >> > > > > >> > > >> > and
> >> > > > > >> > > > > >> > > >> > > > I'm
> >> > > > > >> > > > > >> > > >> > > > > > > > concerned about what we're
> >> setting
> >> > > up
> >> > > > if
> >> > > > > >> 2.0
> >> > > > > >> > > goes
> >> > > > > >> > > > > out
> >> > > > > >> > > > > >> > with
> >> > > > > >> > > > > >> > > >> this
> >> > > > > >> > > > > >> > > >> > > > > > > > feature in its current state.
> >> > > > > >> > > > > >> > > >> > > > > > > >
> >> > > > > >> > > > > >> > > >> > > > > > >
> >> > > > > >> > > > > >> > > >> > > > > >
> >> > > > > >> > > > > >> > > >> > > > >
> >> > > > > >> > > > > >> > > >> > > > >
> >> > > > > >> > > > > >> > > >> > > > >
> >> > > > > >> > > > > >> > > >> > > > > --
> >> > > > > >> > > > > >> > > >> > > > > Best regards,
> >> > > > > >> > > > > >> > > >> > > > >
> >> > > > > >> > > > > >> > > >> > > > >    - Andy
> >> > > > > >> > > > > >> > > >> > > > >
> >> > > > > >> > > > > >> > > >> > > > > Problems worthy of attack prove
> their
> >> > > worth
> >> > > > by
> >> > > > > >> > > hitting
> >> > > > > >> > > > > >> back. -
> >> > > > > >> > > > > >> > > Piet
> >> > > > > >> > > > > >> > > >> > > Hein
> >> > > > > >> > > > > >> > > >> > > > > (via Tom White)
> >> > > > > >> > > > > >> > > >> > > > >
> >> > > > > >> > > > > >> > > >> > > >
> >> > > > > >> > > > > >> > > >> > >
> >> > > > > >> > > > > >> > > >> > >
> >> > > > > >> > > > > >> > > >> > >
> >> > > > > >> > > > > >> > > >> > > --
> >> > > > > >> > > > > >> > > >> > > Best regards,
> >> > > > > >> > > > > >> > > >> > >
> >> > > > > >> > > > > >> > > >> > >    - Andy
> >> > > > > >> > > > > >> > > >> > >
> >> > > > > >> > > > > >> > > >> > > Problems worthy of attack prove their
> >> worth
> >> > by
> >> > > > > >> hitting
> >> > > > > >> > > > back.
> >> > > > > >> > > > > -
> >> > > > > >> > > > > >> > Piet
> >> > > > > >> > > > > >> > > >> Hein
> >> > > > > >> > > > > >> > > >> > > (via Tom White)
> >> > > > > >> > > > > >> > > >> > >
> >> > > > > >> > > > > >> > > >> >
> >> > > > > >> > > > > >> > > >>
> >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >>
> >> > > > > >> > > > > >
> >> > > > > >> > > > > >
> >> > > > > >> > > > >
> >> > > > > >> > > >
> >> > > > > >> > >
> >> > > > > >> >
> >> > > > > >>
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > --
> >> > > > > busbey
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Vladimir Rodionov <vl...@gmail.com>]

The last patch is on review board:
https://reviews.apache.org/r/52748

On Mon, Oct 10, 2016 at 3:52 PM, Vladimir Rodionov <vl...@gmail.com>
wrote:

> >> How hard to put in an hbase-backup module? hbase-server is fat enough
> >> already. Could be done as a follow-up.
>
> https://issues.apache.org/jira/browse/HBASE-16727?
> focusedCommentId=15531237&page=com.atlassian.jira.
> plugin.system.issuetabpanels:comment-tabpanel#comment-15531237
>
> Can we do merge first? Then we can discuss separate module.
>
> -Vlad
>
> On Mon, Oct 10, 2016 at 3:44 PM, Ted Yu <yu...@gmail.com> wrote:
>
>> Looks like the first quote was cut off.
>> The original sentence was:
>>
>> bq. no mapreduce job launched from master or region server.
>>
>> mapreduce job is launched from the node where command line tool is run.
>>
>> On Mon, Oct 10, 2016 at 3:38 PM, Stack <st...@duboce.net> wrote:
>>
>> > bq. launched from master or region server.
>> >
>> > What does this mean please? Has to be run from Master or RegionServer?
>> Can
>> > it be run from another node altogether?
>> >
>> > On Mon, Oct 10, 2016 at 1:44 PM, Vladimir Rodionov <
>> vladrodionov@gmail.com
>> > >
>> > wrote:
>> >
>> > > >> mapreduce dependency has been moved to client side - no mapreduce
>> job
>> > >
>> > > 1. We have no code in the client module anymore, due to dependency on
>> > > internal server API (HFile and WAL access).
>> > > 2. Backup/ restore are client - driven operations, but all the code
>> > resides
>> > > in the server module
>> > >
>> >
>> > How hard to put in an hbase-backup module? hbase-server is fat enough
>> > already. Could be done as a follow-up.
>> >
>> > Thanks,
>> > St.Ack
>> >
>> >
>> >
>> > > 3. No MR in Master, no procedure - driven execution.
>> > > 4. Old good MR from command-line.
>> > > 5. Security was simplified and now only super-user is allowed to run
>> > > backup/restores.
>> > > 6. HBase Backup API was gone due to 1. Now only command-line access to
>> > > backup tools.
>> > >
>> > > These consequences of refactoring has been discussed in HBASE-16727.
>> > >
>> > > -Vlad
>> > >
>> > >
>> > >
>> > >
>> > > On Mon, Oct 10, 2016 at 1:31 PM, Ted Yu <yu...@gmail.com> wrote:
>> > >
>> > > > Reviving this thread.
>> > > >
>> > > > The following has taken place:
>> > > >
>> > > > mapreduce dependency has been moved to client side - no mapreduce
>> job
>> > > > launched from master or region server.
>> > > > document patch (HBASE-16574) has been integrated.
>> > > > Updated mega patch has been attached to HBASE-14123: this covers the
>> > > > refactor in #1 above and the protobuf 3 merge.
>> > > >
>> > > > If community has more feedback on the merge proposal, I would love
>> to
>> > > hear
>> > > > it.
>> > > >
>> > > > Thanks
>> > > >
>> > > > On Thu, Sep 22, 2016 at 10:31 AM, Sean Busbey <bu...@cloudera.com>
>> > > wrote:
>> > > >
>> > > > > I'd like to see the docs proposed on HBASE-16574 integrated into
>> our
>> > > > > project's documentation prior to merge.
>> > > > >
>> > > > > On Thu, Sep 22, 2016 at 9:02 AM, Ted Yu <yu...@gmail.com>
>> wrote:
>> > > > > > This feature can be marked experimental due to some limitations
>> > such
>> > > as
>> > > > > > security.
>> > > > > >
>> > > > > > Your previous round of comments have been addressed.
>> > > > > > Command line tool has gone through:
>> > > > > >
>> > > > > > HBASE-16620 Fix backup command-line tool usability issues
>> > > > > > HBASE-16655 hbase backup describe with incorrect backup id
>> results
>> > in
>> > > > NPE
>> > > > > >
>> > > > > > The updated doc has been attached to HBASE-16574.
>> > > > > >
>> > > > > > Cheers
>> > > > > >
>> > > > > > On Thu, Sep 22, 2016 at 8:53 AM, Stack <st...@duboce.net>
>> wrote:
>> > > > > >
>> > > > > >> On Wed, Sep 21, 2016 at 7:43 AM, Ted Yu <yu...@gmail.com>
>> > > wrote:
>> > > > > >>
>> > > > > >> > Are there more (review) comments ?
>> > > > > >> >
>> > > > > >> >
>> > > > > >> Are outstanding comments addressed?
>> > > > > >>
>> > > > > >> I don't see answer to my 'is this experimental/will it be
>> marked
>> > > > > >> experimental' question.
>> > > > > >>
>> > > > > >> I ran into some issues trying to use the feature and suggested
>> > that
>> > > a
>> > > > > >> feature likes this needs polish else it'll just rot, unused.
>> Has
>> > > > polish
>> > > > > >> been applied? All ready for another 'user' test? Suggest that
>> you
>> > > > update
>> > > > > >> here going forward for the benefit of those trying to follow
>> along
>> > > and
>> > > > > who
>> > > > > >> are not watching JIRA change fly-by.
>> > > > > >>
>> > > > > >> It looks like doc got a revision -- I have to check -- to take
>> on
>> > > > > >> suggestion made above but again, suggest, that this thread gets
>> > > > updated.
>> > > > > >>
>> > > > > >> Thanks,
>> > > > > >> St.Ack
>> > > > > >>
>> > > > > >>
>> > > > > >>
>> > > > > >> > Thanks
>> > > > > >> >
>> > > > > >> > On Tue, Sep 20, 2016 at 10:02 AM, Devaraj Das <
>> > > ddas@hortonworks.com
>> > > > >
>> > > > > >> > wrote:
>> > > > > >> >
>> > > > > >> > > Just reviving this thread. Thanks Sean, Stack, Dima, and
>> > others
>> > > > for
>> > > > > the
>> > > > > >> > > thorough reviews and testing. Thanks Ted and Vlad for
>> taking
>> > > care
>> > > > of
>> > > > > >> the
>> > > > > >> > > feedback. Are we all good to do the merge now? Rather do
>> > sooner
>> > > > than
>> > > > > >> > later.
>> > > > > >> > > ________________________________________
>> > > > > >> > > From: saint.ack@gmail.com <sa...@gmail.com> on behalf
>> of
>> > > > Stack
>> > > > > <
>> > > > > >> > > stack@duboce.net>
>> > > > > >> > > Sent: Monday, September 12, 2016 1:18 PM
>> > > > > >> > > To: HBase Dev List
>> > > > > >> > > Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch
>> > > > HBASE-7912
>> > > > > >> > >
>> > > > > >> > > On Mon, Sep 12, 2016 at 12:19 PM, Ted Yu <
>> yuzhihong@gmail.com
>> > >
>> > > > > wrote:
>> > > > > >> > >
>> > > > > >> > > > Mega patch (rev 18) is on HBASE-14123.
>> > > > > >> > > >
>> > > > > >> > > > Please comment on HBASE-14123 on how you want to review.
>> > > > > >> > > >
>> > > > > >> > >
>> > > > > >> > >
>> > > > > >> > > Yeah. That was my lost tab. Last rb was 6 months ago.
>> Suggest
>> > > > > updating
>> > > > > >> > it.
>> > > > > >> > > RB is pretty good for review. Patch is only 1.5M so should
>> be
>> > > > fine.
>> > > > > >> > >
>> > > > > >> > > St.Ack
>> > > > > >> > >
>> > > > > >> > >
>> > > > > >> > > >
>> > > > > >> > > > Thanks
>> > > > > >> > > >
>> > > > > >> > > > On Mon, Sep 12, 2016 at 12:15 PM, Stack <
>> stack@duboce.net>
>> > > > wrote:
>> > > > > >> > > >
>> > > > > >> > > > > On review of the 'patch', do I just compare the branch
>> to
>> > > > > master or
>> > > > > >> > is
>> > > > > >> > > > > there a megapatch posted somewhere (I think I saw one
>> but
>> > it
>> > > > > seemed
>> > > > > >> > > stale
>> > > > > >> > > > > and then I 'lost' the tab). Sorry for dumb question.
>> > > > > >> > > > > St.Ack
>> > > > > >> > > > >
>> > > > > >> > > > > On Mon, Sep 12, 2016 at 12:01 PM, Stack <
>> stack@duboce.net
>> > >
>> > > > > wrote:
>> > > > > >> > > > >
>> > > > > >> > > > > > Late to the game. A few comments after rereading this
>> > > thread
>> > > > > as a
>> > > > > >> > > > 'user'.
>> > > > > >> > > > > >
>> > > > > >> > > > > > + Before merge, a user-facing feature like this
>> should
>> > > work
>> > > > > (If
>> > > > > >> > this
>> > > > > >> > > is
>> > > > > >> > > > > "higher-bar
>> > > > > >> > > > > > for new features", bring it on -- smile).
>> > > > > >> > > > > > + As a user, I tried the branch with tools after
>> > reviewing
>> > > > the
>> > > > > >> > > > > just-posted
>> > > > > >> > > > > > doc. I had an 'interesting' experience (left
>> comments up
>> > > on
>> > > > > >> > issue). I
>> > > > > >> > > > > think
>> > > > > >> > > > > > the tooling/doc. important to get right. If it breaks
>> > > easily
>> > > > > or
>> > > > > >> is
>> > > > > >> > > > > > inconsistent (or lacks 'polish'), operators will
>> judge
>> > the
>> > > > > whole
>> > > > > >> > > > > > backup/restore tooling chain as not trustworthy and
>> > > abandon
>> > > > > it.
>> > > > > >> > Lets
>> > > > > >> > > > not
>> > > > > >> > > > > > have this happen to this feature.
>> > > > > >> > > > > > + Matteo's suggestion (with a helpful starter list)
>> that
>> > > > there
>> > > > > >> > needs
>> > > > > >> > > to
>> > > > > >> > > > > be
>> > > > > >> > > > > > explicit qualification on what is actually being
>> > delivered
>> > > > --
>> > > > > >> > > > including a
>> > > > > >> > > > > > listing of limitations (some look serious such as
>> data
>> > > bleed
>> > > > > from
>> > > > > >> > > other
>> > > > > >> > > > > > regions in WALs, but maybe I don't care for my use
>> > > case...)
>> > > > --
>> > > > > >> > needs
>> > > > > >> > > to
>> > > > > >> > > > > > accompany the merge. Lets fold them into the user
>> doc.
>> > in
>> > > > the
>> > > > > >> > > technical
>> > > > > >> > > > > > overview area as suggested so user expectations are
>> > > properly
>> > > > > >> > managed
>> > > > > >> > > > > > (otherwise, they expect the world and will just give
>> up
>> > > when
>> > > > > we
>> > > > > >> > fall
>> > > > > >> > > > > > short). Vladimir did a list of what is in each of the
>> > > phases
>> > > > > >> above
>> > > > > >> > > > which
>> > > > > >> > > > > > would serve as a good start.
>> > > > > >> > > > > > + Is this feature 'experimental' (Matteo asks above).
>> > I'd
>> > > > > prefer
>> > > > > >> it
>> > > > > >> > > is
>> > > > > >> > > > > > not. If it is, it should be labelled all over that
>> it is
>> > > > so. I
>> > > > > >> see
>> > > > > >> > > > > current
>> > > > > >> > > > > > state called out as a '... technical preview
>> feature'.
>> > > Does
>> > > > > this
>> > > > > >> > mean
>> > > > > >> > > > > > not-for-users?
>> > > > > >> > > > > >
>> > > > > >> > > > > > St.Ack
>> > > > > >> > > > > >
>> > > > > >> > > > > >
>> > > > > >> > > > > >
>> > > > > >> > > > > >
>> > > > > >> > > > > >
>> > > > > >> > > > > >
>> > > > > >> > > > > >
>> > > > > >> > > > > >
>> > > > > >> > > > > >
>> > > > > >> > > > > >
>> > > > > >> > > > > >
>> > > > > >> > > > > > On Mon, Sep 12, 2016 at 8:03 AM, Ted Yu <
>> > > > yuzhihong@gmail.com>
>> > > > > >> > wrote:
>> > > > > >> > > > > >
>> > > > > >> > > > > >> Sean:
>> > > > > >> > > > > >> Do you have more comments ?
>> > > > > >> > > > > >>
>> > > > > >> > > > > >> Cheers
>> > > > > >> > > > > >>
>> > > > > >> > > > > >> On Fri, Sep 9, 2016 at 1:42 PM, Vladimir Rodionov <
>> > > > > >> > > > > vladrodionov@gmail.com
>> > > > > >> > > > > >> >
>> > > > > >> > > > > >> wrote:
>> > > > > >> > > > > >>
>> > > > > >> > > > > >> > Sean,
>> > > > > >> > > > > >> >
>> > > > > >> > > > > >> > Backup/Restore can fail due to various reasons:
>> > network
>> > > > > outage
>> > > > > >> > > > > (cluster
>> > > > > >> > > > > >> > wide), various time-outs in HBase and HDFS layer,
>> M/R
>> > > > > failure
>> > > > > >> > due
>> > > > > >> > > to
>> > > > > >> > > > > >> "HDFS
>> > > > > >> > > > > >> > exceeded quota", user error (manual deletion of
>> data)
>> > > and
>> > > > > so
>> > > > > >> on
>> > > > > >> > so
>> > > > > >> > > > on.
>> > > > > >> > > > > >> That
>> > > > > >> > > > > >> > is impossible to enumerate all possible types of
>> > > failures
>> > > > > in a
>> > > > > >> > > > > >> distributed
>> > > > > >> > > > > >> > system - that is not our goal/task.
>> > > > > >> > > > > >> >
>> > > > > >> > > > > >> > We focus completely on backup system table
>> > consistency
>> > > > in a
>> > > > > >> > > presence
>> > > > > >> > > > > of
>> > > > > >> > > > > >> any
>> > > > > >> > > > > >> > type of failure. That is what I call "tolerance to
>> > > > > failures".
>> > > > > >> > > > > >> >
>> > > > > >> > > > > >> > On a failure:
>> > > > > >> > > > > >> >
>> > > > > >> > > > > >> > BACKUP. All backup system information (prior to
>> > backup)
>> > > > > will
>> > > > > >> be
>> > > > > >> > > > > restored
>> > > > > >> > > > > >> > and all temporary data, related to a failed
>> session,
>> > in
>> > > > > HDFS
>> > > > > >> > will
>> > > > > >> > > be
>> > > > > >> > > > > >> > deleted
>> > > > > >> > > > > >> > RESTORE. We do not care about system data, because
>> > > > restore
>> > > > > >> does
>> > > > > >> > > not
>> > > > > >> > > > > >> change
>> > > > > >> > > > > >> > it. Temporary data in HDFS will be cleaned up and
>> > table
>> > > > > will
>> > > > > >> be
>> > > > > >> > > in a
>> > > > > >> > > > > >> state
>> > > > > >> > > > > >> > back to where it was before operation started.
>> > > > > >> > > > > >> >
>> > > > > >> > > > > >> > This is what user should expect in case of a
>> failure.
>> > > > > >> > > > > >> >
>> > > > > >> > > > > >> > -Vlad
>> > > > > >> > > > > >> >
>> > > > > >> > > > > >> >
>> > > > > >> > > > > >> > -Vlad
>> > > > > >> > > > > >> >
>> > > > > >> > > > > >> > On Fri, Sep 9, 2016 at 12:56 PM, Sean Busbey <
>> > > > > >> busbey@apache.org
>> > > > > >> > >
>> > > > > >> > > > > wrote:
>> > > > > >> > > > > >> >
>> > > > > >> > > > > >> > > Failing in a consistent way, with docs that
>> explain
>> > > the
>> > > > > >> > various
>> > > > > >> > > > > >> > > expected failures would be sufficient.
>> > > > > >> > > > > >> > >
>> > > > > >> > > > > >> > > On Fri, Sep 9, 2016 at 12:16 PM, Vladimir
>> Rodionov
>> > > > > >> > > > > >> > > <vl...@gmail.com> wrote:
>> > > > > >> > > > > >> > > > Do not worry Sean, doc is coming today as a
>> > preview
>> > > > and
>> > > > > >> our
>> > > > > >> > > > writer
>> > > > > >> > > > > >> > Frank
>> > > > > >> > > > > >> > > > will be working on a putting  it into Apache
>> > repo.
>> > > > > >> Timeline
>> > > > > >> > > > > depends
>> > > > > >> > > > > >> on
>> > > > > >> > > > > >> > > > Franks schedule but I hope we will get it
>> rather
>> > > > sooner
>> > > > > >> than
>> > > > > >> > > > > later.
>> > > > > >> > > > > >> > > >
>> > > > > >> > > > > >> > > > As for failure testing, we are focusing only
>> on a
>> > > > > >> consistent
>> > > > > >> > > > state
>> > > > > >> > > > > >> of
>> > > > > >> > > > > >> > > > backup system data in a presence of any type
>> of
>> > > > > failures,
>> > > > > >> We
>> > > > > >> > > are
>> > > > > >> > > > > not
>> > > > > >> > > > > >> > > going
>> > > > > >> > > > > >> > > > to implement  anything more "fancy", than
>> that.
>> > We
>> > > > > allow
>> > > > > >> > both:
>> > > > > >> > > > > >> backup
>> > > > > >> > > > > >> > and
>> > > > > >> > > > > >> > > > restore to fail. What we do not allow is to
>> have
>> > > > system
>> > > > > >> data
>> > > > > >> > > > > >> corrupted.
>> > > > > >> > > > > >> > > > Will it suffice for you? Do you have any other
>> > > > > concerns,
>> > > > > >> you
>> > > > > >> > > > want
>> > > > > >> > > > > >> us to
>> > > > > >> > > > > >> > > > address?
>> > > > > >> > > > > >> > > >
>> > > > > >> > > > > >> > > > -Vlad
>> > > > > >> > > > > >> > > >
>> > > > > >> > > > > >> > > >
>> > > > > >> > > > > >> > > > On Fri, Sep 9, 2016 at 10:56 AM, Sean Busbey <
>> > > > > >> > > busbey@apache.org
>> > > > > >> > > > >
>> > > > > >> > > > > >> > wrote:
>> > > > > >> > > > > >> > > >
>> > > > > >> > > > > >> > > >> "docs will come to Apache soon" does not
>> address
>> > > my
>> > > > > >> concern
>> > > > > >> > > > > around
>> > > > > >> > > > > >> > docs
>> > > > > >> > > > > >> > > at
>> > > > > >> > > > > >> > > >> all, unless said docs have already made it
>> into
>> > > the
>> > > > > >> project
>> > > > > >> > > > > repo. I
>> > > > > >> > > > > >> > > don't
>> > > > > >> > > > > >> > > >> want third party resources for using a major
>> and
>> > > > > >> important
>> > > > > >> > > > > feature
>> > > > > >> > > > > >> of
>> > > > > >> > > > > >> > > the
>> > > > > >> > > > > >> > > >> project, I want us to provide end users with
>> > what
>> > > > they
>> > > > > >> need
>> > > > > >> > > to
>> > > > > >> > > > > get
>> > > > > >> > > > > >> the
>> > > > > >> > > > > >> > > job
>> > > > > >> > > > > >> > > >> done.
>> > > > > >> > > > > >> > > >>
>> > > > > >> > > > > >> > > >> I see some calls for patience on the failure
>> > > > testing,
>> > > > > but
>> > > > > >> > the
>> > > > > >> > > > > >> appeal
>> > > > > >> > > > > >> > to
>> > > > > >> > > > > >> > > us
>> > > > > >> > > > > >> > > >> having done a bad job of requiring proper
>> tests
>> > of
>> > > > > >> previous
>> > > > > >> > > > > >> features
>> > > > > >> > > > > >> > > just
>> > > > > >> > > > > >> > > >> makes me more concerned about not getting
>> them
>> > > > here. I
>> > > > > >> > don't
>> > > > > >> > > > want
>> > > > > >> > > > > >> to
>> > > > > >> > > > > >> > set
>> > > > > >> > > > > >> > > >> yet another bad example that will then be
>> > pointed
>> > > to
>> > > > > in
>> > > > > >> the
>> > > > > >> > > > > future.
>> > > > > >> > > > > >> > > >>
>> > > > > >> > > > > >> > > >> On Sep 8, 2016 10:50, "Ted Yu" <
>> > > yuzhihong@gmail.com
>> > > > >
>> > > > > >> > wrote:
>> > > > > >> > > > > >> > > >>
>> > > > > >> > > > > >> > > >> > Is there any concern which is not
>> addressed ?
>> > > > > >> > > > > >> > > >> >
>> > > > > >> > > > > >> > > >> > Do we need another Vote thread ?
>> > > > > >> > > > > >> > > >> >
>> > > > > >> > > > > >> > > >> > Thanks
>> > > > > >> > > > > >> > > >> >
>> > > > > >> > > > > >> > > >> > On Thu, Sep 8, 2016 at 9:21 AM, Andrew
>> > Purtell <
>> > > > > >> > > > > >> apurtell@apache.org
>> > > > > >> > > > > >> > >
>> > > > > >> > > > > >> > > >> > wrote:
>> > > > > >> > > > > >> > > >> >
>> > > > > >> > > > > >> > > >> > > Vlad,
>> > > > > >> > > > > >> > > >> > >
>> > > > > >> > > > > >> > > >> > > I apologize for using the term
>> 'half-baked'
>> > > in a
>> > > > > way
>> > > > > >> > that
>> > > > > >> > > > > could
>> > > > > >> > > > > >> > > seem a
>> > > > > >> > > > > >> > > >> > > description of HBASE-7912. I meant that
>> as a
>> > > > > general
>> > > > > >> > > > > >> hypothetical.
>> > > > > >> > > > > >> > > >> > >
>> > > > > >> > > > > >> > > >> > > On Wed, Sep 7, 2016 at 9:36 AM, Vladimir
>> > > > Rodionov
>> > > > > <
>> > > > > >> > > > > >> > > >> > vladrodionov@gmail.com>
>> > > > > >> > > > > >> > > >> > > wrote:
>> > > > > >> > > > > >> > > >> > >
>> > > > > >> > > > > >> > > >> > > > >> I'm not sure that "There is already
>> > lots
>> > > of
>> > > > > >> > > half-baked
>> > > > > >> > > > > >> code
>> > > > > >> > > > > >> > in
>> > > > > >> > > > > >> > > the
>> > > > > >> > > > > >> > > >> > > > branch,
>> > > > > >> > > > > >> > > >> > > > so what's the harm in adding more?"
>> > > > > >> > > > > >> > > >> > > >
>> > > > > >> > > > > >> > > >> > > > I meant - not production - ready yet.
>> This
>> > > is
>> > > > > 2.0
>> > > > > >> > > > > development
>> > > > > >> > > > > >> > > branch
>> > > > > >> > > > > >> > > >> > and,
>> > > > > >> > > > > >> > > >> > > > hence many features are in works,
>> > > > > >> > > > > >> > > >> > > > not being tested well etc. I do not
>> > consider
>> > > > > backup
>> > > > > >> > as
>> > > > > >> > > > half
>> > > > > >> > > > > >> > baked
>> > > > > >> > > > > >> > > >> > > feature -
>> > > > > >> > > > > >> > > >> > > > it has passed our internal QA and has
>> very
>> > > > good
>> > > > > >> doc,
>> > > > > >> > > > which
>> > > > > >> > > > > we
>> > > > > >> > > > > >> > will
>> > > > > >> > > > > >> > > >> > > provide
>> > > > > >> > > > > >> > > >> > > > to Apache shortly.
>> > > > > >> > > > > >> > > >> > > >
>> > > > > >> > > > > >> > > >> > > > -Vlad
>> > > > > >> > > > > >> > > >> > > >
>> > > > > >> > > > > >> > > >> > > > On Wed, Sep 7, 2016 at 9:13 AM, Andrew
>> > > > Purtell <
>> > > > > >> > > > > >> > > apurtell@apache.org>
>> > > > > >> > > > > >> > > >> > > > wrote:
>> > > > > >> > > > > >> > > >> > > >
>> > > > > >> > > > > >> > > >> > > > > We shouldn't admit half baked changes
>> > that
>> > > > > won't
>> > > > > >> be
>> > > > > >> > > > > >> finished.
>> > > > > >> > > > > >> > > >> However
>> > > > > >> > > > > >> > > >> > > in
>> > > > > >> > > > > >> > > >> > > > > this case the crew working on this
>> > feature
>> > > > are
>> > > > > >> long
>> > > > > >> > > > > timers
>> > > > > >> > > > > >> and
>> > > > > >> > > > > >> > > less
>> > > > > >> > > > > >> > > >> > > > likely
>> > > > > >> > > > > >> > > >> > > > > than just about anyone to leave
>> > something
>> > > > in a
>> > > > > >> half
>> > > > > >> > > > baked
>> > > > > >> > > > > >> > > state. Of
>> > > > > >> > > > > >> > > >> > > > course
>> > > > > >> > > > > >> > > >> > > > > there is no guarantee how anything
>> will
>> > > turn
>> > > > > out,
>> > > > > >> > > but I
>> > > > > >> > > > > am
>> > > > > >> > > > > >> > > willing
>> > > > > >> > > > > >> > > >> to
>> > > > > >> > > > > >> > > >> > > > take
>> > > > > >> > > > > >> > > >> > > > > a little on faith if they feel their
>> > best
>> > > > path
>> > > > > >> > > forward
>> > > > > >> > > > > now
>> > > > > >> > > > > >> is
>> > > > > >> > > > > >> > to
>> > > > > >> > > > > >> > > >> > merge
>> > > > > >> > > > > >> > > >> > > to
>> > > > > >> > > > > >> > > >> > > > > trunk. I only wish I had bandwidth to
>> > have
>> > > > > done
>> > > > > >> > some
>> > > > > >> > > > real
>> > > > > >> > > > > >> > > kicking
>> > > > > >> > > > > >> > > >> of
>> > > > > >> > > > > >> > > >> > > the
>> > > > > >> > > > > >> > > >> > > > > tires by now. Maybe this week.
>> > > > > >> > > > > >> > > >> > > > >
>> > > > > >> > > > > >> > > >> > > > > (Yes, I'm using some of that time for
>> > this
>> > > > > email
>> > > > > >> > :-)
>> > > > > >> > > > but
>> > > > > >> > > > > I
>> > > > > >> > > > > >> > type
>> > > > > >> > > > > >> > > >> > fast.)
>> > > > > >> > > > > >> > > >> > > > >
>> > > > > >> > > > > >> > > >> > > > > That said, I would like to agitate
>> for
>> > > > making
>> > > > > 2.0
>> > > > > >> > > more
>> > > > > >> > > > > real
>> > > > > >> > > > > >> > and
>> > > > > >> > > > > >> > > >> spend
>> > > > > >> > > > > >> > > >> > > > some
>> > > > > >> > > > > >> > > >> > > > > time on it now that I'm winding down
>> > with
>> > > > > 0.98. I
>> > > > > >> > > think
>> > > > > >> > > > > >> that
>> > > > > >> > > > > >> > > means
>> > > > > >> > > > > >> > > >> > > > > branching for 2.0 real soon now and
>> even
>> > > > > evicting
>> > > > > >> > > > things
>> > > > > >> > > > > >> from
>> > > > > >> > > > > >> > > 2.0
>> > > > > >> > > > > >> > > >> > > branch
>> > > > > >> > > > > >> > > >> > > > > that aren't finished or stable,
>> leaving
>> > > them
>> > > > > only
>> > > > > >> > > once
>> > > > > >> > > > > >> again
>> > > > > >> > > > > >> > in
>> > > > > >> > > > > >> > > the
>> > > > > >> > > > > >> > > >> > > > master
>> > > > > >> > > > > >> > > >> > > > > branch. Or, maybe just evicting them.
>> > > Let's
>> > > > > take
>> > > > > >> it
>> > > > > >> > > > case
>> > > > > >> > > > > by
>> > > > > >> > > > > >> > > case.
>> > > > > >> > > > > >> > > >> > > > >
>> > > > > >> > > > > >> > > >> > > > > I think this feature can come in
>> > > relatively
>> > > > > >> safely.
>> > > > > >> > > As
>> > > > > >> > > > > >> added
>> > > > > >> > > > > >> > > >> > insurance,
>> > > > > >> > > > > >> > > >> > > > > let's admit the possibility it could
>> be
>> > > > > reverted
>> > > > > >> on
>> > > > > >> > > the
>> > > > > >> > > > > 2.0
>> > > > > >> > > > > >> > > branch
>> > > > > >> > > > > >> > > >> if
>> > > > > >> > > > > >> > > >> > > > folks
>> > > > > >> > > > > >> > > >> > > > > working on stabilizing 2.0 decide to
>> > evict
>> > > > it
>> > > > > >> > because
>> > > > > >> > > > it
>> > > > > >> > > > > is
>> > > > > >> > > > > >> > > >> > unfinished
>> > > > > >> > > > > >> > > >> > > or
>> > > > > >> > > > > >> > > >> > > > > unstable, because that certainly can
>> > > > happen. I
>> > > > > >> > would
>> > > > > >> > > > > >> expect if
>> > > > > >> > > > > >> > > talk
>> > > > > >> > > > > >> > > >> > > like
>> > > > > >> > > > > >> > > >> > > > > that starts, we'd get help finishing
>> or
>> > > > > >> stabilizing
>> > > > > >> > > > > what's
>> > > > > >> > > > > >> > under
>> > > > > >> > > > > >> > > >> > > > discussion
>> > > > > >> > > > > >> > > >> > > > > for revert. Or, we'd have a revert.
>> > Either
>> > > > way
>> > > > > >> the
>> > > > > >> > > > > outcome
>> > > > > >> > > > > >> is
>> > > > > >> > > > > >> > > >> > > acceptable.
>> > > > > >> > > > > >> > > >> > > > >
>> > > > > >> > > > > >> > > >> > > > >
>> > > > > >> > > > > >> > > >> > > > > On Wed, Sep 7, 2016 at 8:56 AM, Dima
>> > > Spivak
>> > > > <
>> > > > > >> > > > > >> > > dimaspivak@apache.org
>> > > > > >> > > > > >> > > >> >
>> > > > > >> > > > > >> > > >> > > > wrote:
>> > > > > >> > > > > >> > > >> > > > >
>> > > > > >> > > > > >> > > >> > > > > > I'm not sure that "There is already
>> > lots
>> > > > of
>> > > > > >> > > > half-baked
>> > > > > >> > > > > >> code
>> > > > > >> > > > > >> > in
>> > > > > >> > > > > >> > > >> the
>> > > > > >> > > > > >> > > >> > > > > branch,
>> > > > > >> > > > > >> > > >> > > > > > so what's the harm in adding more?"
>> > is a
>> > > > > good
>> > > > > >> > code
>> > > > > >> > > > > commit
>> > > > > >> > > > > >> > > >> > philosophy
>> > > > > >> > > > > >> > > >> > > > for
>> > > > > >> > > > > >> > > >> > > > > a
>> > > > > >> > > > > >> > > >> > > > > > fault-tolerant distributed data
>> store.
>> > > ;)
>> > > > > >> > > > > >> > > >> > > > > >
>> > > > > >> > > > > >> > > >> > > > > > More seriously, a lack of test
>> > coverage
>> > > > for
>> > > > > >> > > existing
>> > > > > >> > > > > >> > features
>> > > > > >> > > > > >> > > >> > > shouldn't
>> > > > > >> > > > > >> > > >> > > > > be
>> > > > > >> > > > > >> > > >> > > > > > used as justification for
>> introducing
>> > > new
>> > > > > >> > features
>> > > > > >> > > > with
>> > > > > >> > > > > >> the
>> > > > > >> > > > > >> > > same
>> > > > > >> > > > > >> > > >> > > > > > shortcomings. Ultimately, it's the
>> end
>> > > > user
>> > > > > who
>> > > > > >> > > will
>> > > > > >> > > > > feel
>> > > > > >> > > > > >> > the
>> > > > > >> > > > > >> > > >> pain,
>> > > > > >> > > > > >> > > >> > > so
>> > > > > >> > > > > >> > > >> > > > > > shouldn't we do everything we can
>> to
>> > > > > mitigate
>> > > > > >> > that?
>> > > > > >> > > > > >> > > >> > > > > >
>> > > > > >> > > > > >> > > >> > > > > > -Dima
>> > > > > >> > > > > >> > > >> > > > > >
>> > > > > >> > > > > >> > > >> > > > > > On Wed, Sep 7, 2016 at 8:46 AM,
>> > Vladimir
>> > > > > >> > Rodionov <
>> > > > > >> > > > > >> > > >> > > > > vladrodionov@gmail.com>
>> > > > > >> > > > > >> > > >> > > > > > wrote:
>> > > > > >> > > > > >> > > >> > > > > >
>> > > > > >> > > > > >> > > >> > > > > > > Sean,
>> > > > > >> > > > > >> > > >> > > > > > >
>> > > > > >> > > > > >> > > >> > > > > > > * have docs
>> > > > > >> > > > > >> > > >> > > > > > >
>> > > > > >> > > > > >> > > >> > > > > > > Agree. We have a doc and backup
>> is
>> > the
>> > > > > most
>> > > > > >> > > > > documented
>> > > > > >> > > > > >> > > feature
>> > > > > >> > > > > >> > > >> > :),
>> > > > > >> > > > > >> > > >> > > we
>> > > > > >> > > > > >> > > >> > > > > > will
>> > > > > >> > > > > >> > > >> > > > > > > release it shortly to Apache.
>> > > > > >> > > > > >> > > >> > > > > > >
>> > > > > >> > > > > >> > > >> > > > > > > * have sunny-day correctness
>> tests
>> > > > > >> > > > > >> > > >> > > > > > >
>> > > > > >> > > > > >> > > >> > > > > > > Feature has  close to 60 test
>> cases,
>> > > > which
>> > > > > >> run
>> > > > > >> > > for
>> > > > > >> > > > > >> approx
>> > > > > >> > > > > >> > 30
>> > > > > >> > > > > >> > > >> min.
>> > > > > >> > > > > >> > > >> > > We
>> > > > > >> > > > > >> > > >> > > > > can
>> > > > > >> > > > > >> > > >> > > > > > > add more, if community do not
>> mind
>> > :)
>> > > > > >> > > > > >> > > >> > > > > > >
>> > > > > >> > > > > >> > > >> > > > > > > * have
>> > correctness-in-face-of-failure
>> > > > > tests
>> > > > > >> > > > > >> > > >> > > > > > >
>> > > > > >> > > > > >> > > >> > > > > > > Any examples of these tests in
>> > > existing
>> > > > > >> > features?
>> > > > > >> > > > In
>> > > > > >> > > > > >> > works,
>> > > > > >> > > > > >> > > we
>> > > > > >> > > > > >> > > >> > > have a
>> > > > > >> > > > > >> > > >> > > > > > clear
>> > > > > >> > > > > >> > > >> > > > > > > understanding of what should be
>> done
>> > > by
>> > > > > the
>> > > > > >> > time
>> > > > > >> > > of
>> > > > > >> > > > > 2.0
>> > > > > >> > > > > >> > > >> release.
>> > > > > >> > > > > >> > > >> > > > > > > That is very close goal for us,
>> to
>> > > > verify
>> > > > > IT
>> > > > > >> > > monkey
>> > > > > >> > > > > for
>> > > > > >> > > > > >> > > >> existing
>> > > > > >> > > > > >> > > >> > > > code.
>> > > > > >> > > > > >> > > >> > > > > > >
>> > > > > >> > > > > >> > > >> > > > > > > * don't rely on things outside of
>> > > HBase
>> > > > > for
>> > > > > >> > > normal
>> > > > > >> > > > > >> > operation
>> > > > > >> > > > > >> > > >> > (okay
>> > > > > >> > > > > >> > > >> > > > for
>> > > > > >> > > > > >> > > >> > > > > > > advanced operation)
>> > > > > >> > > > > >> > > >> > > > > > >
>> > > > > >> > > > > >> > > >> > > > > > > We do not.
>> > > > > >> > > > > >> > > >> > > > > > >
>> > > > > >> > > > > >> > > >> > > > > > > Enormous time has been spent
>> already
>> > > on
>> > > > > the
>> > > > > >> > > > > development
>> > > > > >> > > > > >> > and
>> > > > > >> > > > > >> > > >> > testing
>> > > > > >> > > > > >> > > >> > > > the
>> > > > > >> > > > > >> > > >> > > > > > > feature, it has passed our
>> internal
>> > > > tests
>> > > > > and
>> > > > > >> > > many
>> > > > > >> > > > > >> rounds
>> > > > > >> > > > > >> > of
>> > > > > >> > > > > >> > > >> code
>> > > > > >> > > > > >> > > >> > > > > reviews
>> > > > > >> > > > > >> > > >> > > > > > > by HBase committers. We do not
>> mind
>> > if
>> > > > > >> someone
>> > > > > >> > > from
>> > > > > >> > > > > >> HBase
>> > > > > >> > > > > >> > > >> > community
>> > > > > >> > > > > >> > > >> > > > > > > (outside of HW) will review the
>> > code,
>> > > > but
>> > > > > it
>> > > > > >> > will
>> > > > > >> > > > > >> probably
>> > > > > >> > > > > >> > > >> takes
>> > > > > >> > > > > >> > > >> > > > > forever
>> > > > > >> > > > > >> > > >> > > > > > to
>> > > > > >> > > > > >> > > >> > > > > > > wait for volunteer?, the feature
>> is
>> > > > quite
>> > > > > >> large
>> > > > > >> > > > (1MB+
>> > > > > >> > > > > >> > > >> cumulative
>> > > > > >> > > > > >> > > >> > > > patch)
>> > > > > >> > > > > >> > > >> > > > > > >
>> > > > > >> > > > > >> > > >> > > > > > > 2.0 branch is full of half baked
>> > > > features,
>> > > > > >> most
>> > > > > >> > > of
>> > > > > >> > > > > them
>> > > > > >> > > > > >> > are
>> > > > > >> > > > > >> > > in
>> > > > > >> > > > > >> > > >> > > active
>> > > > > >> > > > > >> > > >> > > > > > > development, therefore I am not
>> > > > following
>> > > > > you
>> > > > > >> > > here,
>> > > > > >> > > > > >> Sean?
>> > > > > >> > > > > >> > > Why
>> > > > > >> > > > > >> > > >> > > > > HBASE-7912
>> > > > > >> > > > > >> > > >> > > > > > is
>> > > > > >> > > > > >> > > >> > > > > > > not good enough yet to be
>> integrated
>> > > > into
>> > > > > 2.0
>> > > > > >> > > > branch?
>> > > > > >> > > > > >> > > >> > > > > > >
>> > > > > >> > > > > >> > > >> > > > > > > -Vlad
>> > > > > >> > > > > >> > > >> > > > > > >
>> > > > > >> > > > > >> > > >> > > > > > >
>> > > > > >> > > > > >> > > >> > > > > > >
>> > > > > >> > > > > >> > > >> > > > > > >
>> > > > > >> > > > > >> > > >> > > > > > >
>> > > > > >> > > > > >> > > >> > > > > > > On Wed, Sep 7, 2016 at 8:23 AM,
>> Sean
>> > > > > Busbey <
>> > > > > >> > > > > >> > > busbey@apache.org
>> > > > > >> > > > > >> > > >> >
>> > > > > >> > > > > >> > > >> > > > wrote:
>> > > > > >> > > > > >> > > >> > > > > > >
>> > > > > >> > > > > >> > > >> > > > > > > > On Tue, Sep 6, 2016 at 10:36
>> PM,
>> > > Josh
>> > > > > >> Elser <
>> > > > > >> > > > > >> > > >> > > josh.elser@gmail.com>
>> > > > > >> > > > > >> > > >> > > > > > > wrote:
>> > > > > >> > > > > >> > > >> > > > > > > > > So, the answer to Sean's
>> > original
>> > > > > >> question
>> > > > > >> > is
>> > > > > >> > > > "as
>> > > > > >> > > > > >> > > robust as
>> > > > > >> > > > > >> > > >> > > > > snapshots
>> > > > > >> > > > > >> > > >> > > > > > > > > presently are"?
>> (independence of
>> > > > > >> > > backup/restore
>> > > > > >> > > > > >> > failure
>> > > > > >> > > > > >> > > >> > > tolerance
>> > > > > >> > > > > >> > > >> > > > > > from
>> > > > > >> > > > > >> > > >> > > > > > > > > snapshot failure tolerance)
>> > > > > >> > > > > >> > > >> > > > > > > > >
>> > > > > >> > > > > >> > > >> > > > > > > > > Is this just a question WRT
>> > > context
>> > > > of
>> > > > > >> the
>> > > > > >> > > > > change,
>> > > > > >> > > > > >> or
>> > > > > >> > > > > >> > > is it
>> > > > > >> > > > > >> > > >> > > means
>> > > > > >> > > > > >> > > >> > > > > > for a
>> > > > > >> > > > > >> > > >> > > > > > > > veto
>> > > > > >> > > > > >> > > >> > > > > > > > > from you, Sean? Just trying
>> to
>> > > make
>> > > > > sure
>> > > > > >> > I'm
>> > > > > >> > > > > >> following
>> > > > > >> > > > > >> > > >> along
>> > > > > >> > > > > >> > > >> > > > > > > adequately.
>> > > > > >> > > > > >> > > >> > > > > > > > >
>> > > > > >> > > > > >> > > >> > > > > > > > >
>> > > > > >> > > > > >> > > >> > > > > > > >
>> > > > > >> > > > > >> > > >> > > > > > > > I'd say ATM I'm -0, bordering
>> on
>> > -1
>> > > > but
>> > > > > not
>> > > > > >> > for
>> > > > > >> > > > > >> reasons
>> > > > > >> > > > > >> > I
>> > > > > >> > > > > >> > > can
>> > > > > >> > > > > >> > > >> > > > > > articulate
>> > > > > >> > > > > >> > > >> > > > > > > > well.
>> > > > > >> > > > > >> > > >> > > > > > > >
>> > > > > >> > > > > >> > > >> > > > > > > > Here's an attempt.
>> > > > > >> > > > > >> > > >> > > > > > > >
>> > > > > >> > > > > >> > > >> > > > > > > > We've been trying to move, as a
>> > > > > community,
>> > > > > >> > > > towards
>> > > > > >> > > > > >> > > minimizing
>> > > > > >> > > > > >> > > >> > > risk
>> > > > > >> > > > > >> > > >> > > > to
>> > > > > >> > > > > >> > > >> > > > > > > > downstream folks by getting
>> > > "complete
>> > > > > >> enough
>> > > > > >> > > for
>> > > > > >> > > > > use"
>> > > > > >> > > > > >> > > gates
>> > > > > >> > > > > >> > > >> in
>> > > > > >> > > > > >> > > >> > > > place
>> > > > > >> > > > > >> > > >> > > > > > > > before we introduce new
>> features.
>> > > This
>> > > > > was
>> > > > > >> > > > spurred
>> > > > > >> > > > > >> by a
>> > > > > >> > > > > >> > > some
>> > > > > >> > > > > >> > > >> > > > features
>> > > > > >> > > > > >> > > >> > > > > > > > getting in half-baked and never
>> > > making
>> > > > > it
>> > > > > >> to
>> > > > > >> > > "can
>> > > > > >> > > > > >> really
>> > > > > >> > > > > >> > > use"
>> > > > > >> > > > > >> > > >> > > > status
>> > > > > >> > > > > >> > > >> > > > > > > > (I'm thinking of distributed
>> log
>> > > > replay
>> > > > > and
>> > > > > >> > the
>> > > > > >> > > > > >> zk-less
>> > > > > >> > > > > >> > > >> > > assignment
>> > > > > >> > > > > >> > > >> > > > > > > > stuff, I don't recall if there
>> was
>> > > > > more).
>> > > > > >> > > > > >> > > >> > > > > > > >
>> > > > > >> > > > > >> > > >> > > > > > > > The gates, generally, included
>> > > things
>> > > > > like:
>> > > > > >> > > > > >> > > >> > > > > > > >
>> > > > > >> > > > > >> > > >> > > > > > > > * have docs
>> > > > > >> > > > > >> > > >> > > > > > > > * have sunny-day correctness
>> tests
>> > > > > >> > > > > >> > > >> > > > > > > > * have
>> > > correctness-in-face-of-failure
>> > > > > tests
>> > > > > >> > > > > >> > > >> > > > > > > > * don't rely on things outside
>> of
>> > > > HBase
>> > > > > for
>> > > > > >> > > > normal
>> > > > > >> > > > > >> > > operation
>> > > > > >> > > > > >> > > >> > > (okay
>> > > > > >> > > > > >> > > >> > > > > for
>> > > > > >> > > > > >> > > >> > > > > > > > advanced operation)
>> > > > > >> > > > > >> > > >> > > > > > > >
>> > > > > >> > > > > >> > > >> > > > > > > > As an example, we kept the MOB
>> > work
>> > > > off
>> > > > > in
>> > > > > >> a
>> > > > > >> > > > branch
>> > > > > >> > > > > >> and
>> > > > > >> > > > > >> > > out
>> > > > > >> > > > > >> > > >> of
>> > > > > >> > > > > >> > > >> > > > master
>> > > > > >> > > > > >> > > >> > > > > > > > until it could pass these
>> > criteria.
>> > > > The
>> > > > > big
>> > > > > >> > > > > exemption
>> > > > > >> > > > > >> > > we've
>> > > > > >> > > > > >> > > >> had
>> > > > > >> > > > > >> > > >> > > to
>> > > > > >> > > > > >> > > >> > > > > > > > this was the hbase-spark
>> > > integration,
>> > > > > where
>> > > > > >> > we
>> > > > > >> > > > all
>> > > > > >> > > > > >> > agreed
>> > > > > >> > > > > >> > > it
>> > > > > >> > > > > >> > > >> > > could
>> > > > > >> > > > > >> > > >> > > > > > > > land in master because it was
>> very
>> > > > well
>> > > > > >> > > isolated
>> > > > > >> > > > > (the
>> > > > > >> > > > > >> > > slide
>> > > > > >> > > > > >> > > >> > away
>> > > > > >> > > > > >> > > >> > > > from
>> > > > > >> > > > > >> > > >> > > > > > > > including docs as a first-class
>> > part
>> > > > of
>> > > > > >> > > building
>> > > > > >> > > > up
>> > > > > >> > > > > >> that
>> > > > > >> > > > > >> > > >> > > > integration
>> > > > > >> > > > > >> > > >> > > > > > > > has led me to doubt the wisdom
>> of
>> > > this
>> > > > > >> > > decision).
>> > > > > >> > > > > >> > > >> > > > > > > >
>> > > > > >> > > > > >> > > >> > > > > > > > We've also been treating
>> inclusion
>> > > in
>> > > > a
>> > > > > >> > > "probably
>> > > > > >> > > > > >> will
>> > > > > >> > > > > >> > be
>> > > > > >> > > > > >> > > >> > > released
>> > > > > >> > > > > >> > > >> > > > to
>> > > > > >> > > > > >> > > >> > > > > > > > downstream" branches as a
>> higher
>> > > bar,
>> > > > > >> > requiring
>> > > > > >> > > > > >> > > >> > > > > > > >
>> > > > > >> > > > > >> > > >> > > > > > > > * don't moderately impact
>> > > performance
>> > > > > when
>> > > > > >> > the
>> > > > > >> > > > > >> feature
>> > > > > >> > > > > >> > > isn't
>> > > > > >> > > > > >> > > >> in
>> > > > > >> > > > > >> > > >> > > use
>> > > > > >> > > > > >> > > >> > > > > > > > * don't severely impact
>> > performance
>> > > > when
>> > > > > >> the
>> > > > > >> > > > > feature
>> > > > > >> > > > > >> is
>> > > > > >> > > > > >> > in
>> > > > > >> > > > > >> > > >> use
>> > > > > >> > > > > >> > > >> > > > > > > > * either default-to-on or show
>> > > enough
>> > > > > >> demand
>> > > > > >> > to
>> > > > > >> > > > > >> believe
>> > > > > >> > > > > >> > a
>> > > > > >> > > > > >> > > >> > > > non-trivial
>> > > > > >> > > > > >> > > >> > > > > > > > number of folks will turn the
>> > > feature
>> > > > on
>> > > > > >> > > > > >> > > >> > > > > > > >
>> > > > > >> > > > > >> > > >> > > > > > > > The above has kept MOB and
>> > > hbase-spark
>> > > > > >> > > > integration
>> > > > > >> > > > > >> out
>> > > > > >> > > > > >> > of
>> > > > > >> > > > > >> > > >> > > branch-1,
>> > > > > >> > > > > >> > > >> > > > > > > > presumably while they've
>> "gotten
>> > > more
>> > > > > >> stable"
>> > > > > >> > > in
>> > > > > >> > > > > >> master
>> > > > > >> > > > > >> > > from
>> > > > > >> > > > > >> > > >> > the
>> > > > > >> > > > > >> > > >> > > > odd
>> > > > > >> > > > > >> > > >> > > > > > > > vendor inclusion.
>> > > > > >> > > > > >> > > >> > > > > > > >
>> > > > > >> > > > > >> > > >> > > > > > > > Are we going to have a 2.0
>> release
>> > > > > before
>> > > > > >> the
>> > > > > >> > > end
>> > > > > >> > > > > of
>> > > > > >> > > > > >> the
>> > > > > >> > > > > >> > > >> year?
>> > > > > >> > > > > >> > > >> > > > We're
>> > > > > >> > > > > >> > > >> > > > > > > > coming up on 1.5 years since
>> the
>> > > > > release of
>> > > > > >> > > > version
>> > > > > >> > > > > >> 1.0;
>> > > > > >> > > > > >> > > >> seems
>> > > > > >> > > > > >> > > >> > > like
>> > > > > >> > > > > >> > > >> > > > > > > > it's about time, though I
>> haven't
>> > > seen
>> > > > > any
>> > > > > >> > > > concrete
>> > > > > >> > > > > >> > plans
>> > > > > >> > > > > >> > > >> this
>> > > > > >> > > > > >> > > >> > > > year.
>> > > > > >> > > > > >> > > >> > > > > > > > Presuming we are going to have
>> one
>> > > by
>> > > > > the
>> > > > > >> end
>> > > > > >> > > of
>> > > > > >> > > > > the
>> > > > > >> > > > > >> > > year, it
>> > > > > >> > > > > >> > > >> > > > seems a
>> > > > > >> > > > > >> > > >> > > > > > > > bit close to still be adding in
>> > > > > "features
>> > > > > >> > that
>> > > > > >> > > > need
>> > > > > >> > > > > >> > > maturing"
>> > > > > >> > > > > >> > > >> > on
>> > > > > >> > > > > >> > > >> > > > the
>> > > > > >> > > > > >> > > >> > > > > > > > branch.
>> > > > > >> > > > > >> > > >> > > > > > > >
>> > > > > >> > > > > >> > > >> > > > > > > > The lack of a concrete plan for
>> > 2.0
>> > > > > keeps
>> > > > > >> me
>> > > > > >> > > from
>> > > > > >> > > > > >> > > considering
>> > > > > >> > > > > >> > > >> > > these
>> > > > > >> > > > > >> > > >> > > > > > > > things blocker at the moment.
>> But
>> > I
>> > > > know
>> > > > > >> > first
>> > > > > >> > > > hand
>> > > > > >> > > > > >> how
>> > > > > >> > > > > >> > > much
>> > > > > >> > > > > >> > > >> > > > trouble
>> > > > > >> > > > > >> > > >> > > > > > > > folks have had with other
>> features
>> > > > that
>> > > > > >> have
>> > > > > >> > > gone
>> > > > > >> > > > > >> into
>> > > > > >> > > > > >> > > >> > downstream
>> > > > > >> > > > > >> > > >> > > > > > > > facing releases without
>> robustness
>> > > > > checks
>> > > > > >> > (i.e.
>> > > > > >> > > > > >> > > replication),
>> > > > > >> > > > > >> > > >> > and
>> > > > > >> > > > > >> > > >> > > > I'm
>> > > > > >> > > > > >> > > >> > > > > > > > concerned about what we're
>> setting
>> > > up
>> > > > if
>> > > > > >> 2.0
>> > > > > >> > > goes
>> > > > > >> > > > > out
>> > > > > >> > > > > >> > with
>> > > > > >> > > > > >> > > >> this
>> > > > > >> > > > > >> > > >> > > > > > > > feature in its current state.
>> > > > > >> > > > > >> > > >> > > > > > > >
>> > > > > >> > > > > >> > > >> > > > > > >
>> > > > > >> > > > > >> > > >> > > > > >
>> > > > > >> > > > > >> > > >> > > > >
>> > > > > >> > > > > >> > > >> > > > >
>> > > > > >> > > > > >> > > >> > > > >
>> > > > > >> > > > > >> > > >> > > > > --
>> > > > > >> > > > > >> > > >> > > > > Best regards,
>> > > > > >> > > > > >> > > >> > > > >
>> > > > > >> > > > > >> > > >> > > > >    - Andy
>> > > > > >> > > > > >> > > >> > > > >
>> > > > > >> > > > > >> > > >> > > > > Problems worthy of attack prove their
>> > > worth
>> > > > by
>> > > > > >> > > hitting
>> > > > > >> > > > > >> back. -
>> > > > > >> > > > > >> > > Piet
>> > > > > >> > > > > >> > > >> > > Hein
>> > > > > >> > > > > >> > > >> > > > > (via Tom White)
>> > > > > >> > > > > >> > > >> > > > >
>> > > > > >> > > > > >> > > >> > > >
>> > > > > >> > > > > >> > > >> > >
>> > > > > >> > > > > >> > > >> > >
>> > > > > >> > > > > >> > > >> > >
>> > > > > >> > > > > >> > > >> > > --
>> > > > > >> > > > > >> > > >> > > Best regards,
>> > > > > >> > > > > >> > > >> > >
>> > > > > >> > > > > >> > > >> > >    - Andy
>> > > > > >> > > > > >> > > >> > >
>> > > > > >> > > > > >> > > >> > > Problems worthy of attack prove their
>> worth
>> > by
>> > > > > >> hitting
>> > > > > >> > > > back.
>> > > > > >> > > > > -
>> > > > > >> > > > > >> > Piet
>> > > > > >> > > > > >> > > >> Hein
>> > > > > >> > > > > >> > > >> > > (via Tom White)
>> > > > > >> > > > > >> > > >> > >
>> > > > > >> > > > > >> > > >> >
>> > > > > >> > > > > >> > > >>
>> > > > > >> > > > > >> > >
>> > > > > >> > > > > >> >
>> > > > > >> > > > > >>
>> > > > > >> > > > > >
>> > > > > >> > > > > >
>> > > > > >> > > > >
>> > > > > >> > > >
>> > > > > >> > >
>> > > > > >> >
>> > > > > >>
>> > > > >
>> > > > >
>> > > > >
>> > > > > --
>> > > > > busbey
>> > > > >
>> > > >
>> > >
>> >
>>
>
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Vladimir Rodionov <vl...@gmail.com>]

>> How hard to put in an hbase-backup module? hbase-server is fat enough
>> already. Could be done as a follow-up.

https://issues.apache.org/jira/browse/HBASE-16727?focusedCommentId=15531237&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15531237

Can we do merge first? Then we can discuss separate module.

-Vlad

On Mon, Oct 10, 2016 at 3:44 PM, Ted Yu <yu...@gmail.com> wrote:

> Looks like the first quote was cut off.
> The original sentence was:
>
> bq. no mapreduce job launched from master or region server.
>
> mapreduce job is launched from the node where command line tool is run.
>
> On Mon, Oct 10, 2016 at 3:38 PM, Stack <st...@duboce.net> wrote:
>
> > bq. launched from master or region server.
> >
> > What does this mean please? Has to be run from Master or RegionServer?
> Can
> > it be run from another node altogether?
> >
> > On Mon, Oct 10, 2016 at 1:44 PM, Vladimir Rodionov <
> vladrodionov@gmail.com
> > >
> > wrote:
> >
> > > >> mapreduce dependency has been moved to client side - no mapreduce
> job
> > >
> > > 1. We have no code in the client module anymore, due to dependency on
> > > internal server API (HFile and WAL access).
> > > 2. Backup/ restore are client - driven operations, but all the code
> > resides
> > > in the server module
> > >
> >
> > How hard to put in an hbase-backup module? hbase-server is fat enough
> > already. Could be done as a follow-up.
> >
> > Thanks,
> > St.Ack
> >
> >
> >
> > > 3. No MR in Master, no procedure - driven execution.
> > > 4. Old good MR from command-line.
> > > 5. Security was simplified and now only super-user is allowed to run
> > > backup/restores.
> > > 6. HBase Backup API was gone due to 1. Now only command-line access to
> > > backup tools.
> > >
> > > These consequences of refactoring has been discussed in HBASE-16727.
> > >
> > > -Vlad
> > >
> > >
> > >
> > >
> > > On Mon, Oct 10, 2016 at 1:31 PM, Ted Yu <yu...@gmail.com> wrote:
> > >
> > > > Reviving this thread.
> > > >
> > > > The following has taken place:
> > > >
> > > > mapreduce dependency has been moved to client side - no mapreduce job
> > > > launched from master or region server.
> > > > document patch (HBASE-16574) has been integrated.
> > > > Updated mega patch has been attached to HBASE-14123: this covers the
> > > > refactor in #1 above and the protobuf 3 merge.
> > > >
> > > > If community has more feedback on the merge proposal, I would love to
> > > hear
> > > > it.
> > > >
> > > > Thanks
> > > >
> > > > On Thu, Sep 22, 2016 at 10:31 AM, Sean Busbey <bu...@cloudera.com>
> > > wrote:
> > > >
> > > > > I'd like to see the docs proposed on HBASE-16574 integrated into
> our
> > > > > project's documentation prior to merge.
> > > > >
> > > > > On Thu, Sep 22, 2016 at 9:02 AM, Ted Yu <yu...@gmail.com>
> wrote:
> > > > > > This feature can be marked experimental due to some limitations
> > such
> > > as
> > > > > > security.
> > > > > >
> > > > > > Your previous round of comments have been addressed.
> > > > > > Command line tool has gone through:
> > > > > >
> > > > > > HBASE-16620 Fix backup command-line tool usability issues
> > > > > > HBASE-16655 hbase backup describe with incorrect backup id
> results
> > in
> > > > NPE
> > > > > >
> > > > > > The updated doc has been attached to HBASE-16574.
> > > > > >
> > > > > > Cheers
> > > > > >
> > > > > > On Thu, Sep 22, 2016 at 8:53 AM, Stack <st...@duboce.net> wrote:
> > > > > >
> > > > > >> On Wed, Sep 21, 2016 at 7:43 AM, Ted Yu <yu...@gmail.com>
> > > wrote:
> > > > > >>
> > > > > >> > Are there more (review) comments ?
> > > > > >> >
> > > > > >> >
> > > > > >> Are outstanding comments addressed?
> > > > > >>
> > > > > >> I don't see answer to my 'is this experimental/will it be marked
> > > > > >> experimental' question.
> > > > > >>
> > > > > >> I ran into some issues trying to use the feature and suggested
> > that
> > > a
> > > > > >> feature likes this needs polish else it'll just rot, unused. Has
> > > > polish
> > > > > >> been applied? All ready for another 'user' test? Suggest that
> you
> > > > update
> > > > > >> here going forward for the benefit of those trying to follow
> along
> > > and
> > > > > who
> > > > > >> are not watching JIRA change fly-by.
> > > > > >>
> > > > > >> It looks like doc got a revision -- I have to check -- to take
> on
> > > > > >> suggestion made above but again, suggest, that this thread gets
> > > > updated.
> > > > > >>
> > > > > >> Thanks,
> > > > > >> St.Ack
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> > Thanks
> > > > > >> >
> > > > > >> > On Tue, Sep 20, 2016 at 10:02 AM, Devaraj Das <
> > > ddas@hortonworks.com
> > > > >
> > > > > >> > wrote:
> > > > > >> >
> > > > > >> > > Just reviving this thread. Thanks Sean, Stack, Dima, and
> > others
> > > > for
> > > > > the
> > > > > >> > > thorough reviews and testing. Thanks Ted and Vlad for taking
> > > care
> > > > of
> > > > > >> the
> > > > > >> > > feedback. Are we all good to do the merge now? Rather do
> > sooner
> > > > than
> > > > > >> > later.
> > > > > >> > > ________________________________________
> > > > > >> > > From: saint.ack@gmail.com <sa...@gmail.com> on behalf
> of
> > > > Stack
> > > > > <
> > > > > >> > > stack@duboce.net>
> > > > > >> > > Sent: Monday, September 12, 2016 1:18 PM
> > > > > >> > > To: HBase Dev List
> > > > > >> > > Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch
> > > > HBASE-7912
> > > > > >> > >
> > > > > >> > > On Mon, Sep 12, 2016 at 12:19 PM, Ted Yu <
> yuzhihong@gmail.com
> > >
> > > > > wrote:
> > > > > >> > >
> > > > > >> > > > Mega patch (rev 18) is on HBASE-14123.
> > > > > >> > > >
> > > > > >> > > > Please comment on HBASE-14123 on how you want to review.
> > > > > >> > > >
> > > > > >> > >
> > > > > >> > >
> > > > > >> > > Yeah. That was my lost tab. Last rb was 6 months ago.
> Suggest
> > > > > updating
> > > > > >> > it.
> > > > > >> > > RB is pretty good for review. Patch is only 1.5M so should
> be
> > > > fine.
> > > > > >> > >
> > > > > >> > > St.Ack
> > > > > >> > >
> > > > > >> > >
> > > > > >> > > >
> > > > > >> > > > Thanks
> > > > > >> > > >
> > > > > >> > > > On Mon, Sep 12, 2016 at 12:15 PM, Stack <stack@duboce.net
> >
> > > > wrote:
> > > > > >> > > >
> > > > > >> > > > > On review of the 'patch', do I just compare the branch
> to
> > > > > master or
> > > > > >> > is
> > > > > >> > > > > there a megapatch posted somewhere (I think I saw one
> but
> > it
> > > > > seemed
> > > > > >> > > stale
> > > > > >> > > > > and then I 'lost' the tab). Sorry for dumb question.
> > > > > >> > > > > St.Ack
> > > > > >> > > > >
> > > > > >> > > > > On Mon, Sep 12, 2016 at 12:01 PM, Stack <
> stack@duboce.net
> > >
> > > > > wrote:
> > > > > >> > > > >
> > > > > >> > > > > > Late to the game. A few comments after rereading this
> > > thread
> > > > > as a
> > > > > >> > > > 'user'.
> > > > > >> > > > > >
> > > > > >> > > > > > + Before merge, a user-facing feature like this should
> > > work
> > > > > (If
> > > > > >> > this
> > > > > >> > > is
> > > > > >> > > > > "higher-bar
> > > > > >> > > > > > for new features", bring it on -- smile).
> > > > > >> > > > > > + As a user, I tried the branch with tools after
> > reviewing
> > > > the
> > > > > >> > > > > just-posted
> > > > > >> > > > > > doc. I had an 'interesting' experience (left comments
> up
> > > on
> > > > > >> > issue). I
> > > > > >> > > > > think
> > > > > >> > > > > > the tooling/doc. important to get right. If it breaks
> > > easily
> > > > > or
> > > > > >> is
> > > > > >> > > > > > inconsistent (or lacks 'polish'), operators will judge
> > the
> > > > > whole
> > > > > >> > > > > > backup/restore tooling chain as not trustworthy and
> > > abandon
> > > > > it.
> > > > > >> > Lets
> > > > > >> > > > not
> > > > > >> > > > > > have this happen to this feature.
> > > > > >> > > > > > + Matteo's suggestion (with a helpful starter list)
> that
> > > > there
> > > > > >> > needs
> > > > > >> > > to
> > > > > >> > > > > be
> > > > > >> > > > > > explicit qualification on what is actually being
> > delivered
> > > > --
> > > > > >> > > > including a
> > > > > >> > > > > > listing of limitations (some look serious such as data
> > > bleed
> > > > > from
> > > > > >> > > other
> > > > > >> > > > > > regions in WALs, but maybe I don't care for my use
> > > case...)
> > > > --
> > > > > >> > needs
> > > > > >> > > to
> > > > > >> > > > > > accompany the merge. Lets fold them into the user doc.
> > in
> > > > the
> > > > > >> > > technical
> > > > > >> > > > > > overview area as suggested so user expectations are
> > > properly
> > > > > >> > managed
> > > > > >> > > > > > (otherwise, they expect the world and will just give
> up
> > > when
> > > > > we
> > > > > >> > fall
> > > > > >> > > > > > short). Vladimir did a list of what is in each of the
> > > phases
> > > > > >> above
> > > > > >> > > > which
> > > > > >> > > > > > would serve as a good start.
> > > > > >> > > > > > + Is this feature 'experimental' (Matteo asks above).
> > I'd
> > > > > prefer
> > > > > >> it
> > > > > >> > > is
> > > > > >> > > > > > not. If it is, it should be labelled all over that it
> is
> > > > so. I
> > > > > >> see
> > > > > >> > > > > current
> > > > > >> > > > > > state called out as a '... technical preview feature'.
> > > Does
> > > > > this
> > > > > >> > mean
> > > > > >> > > > > > not-for-users?
> > > > > >> > > > > >
> > > > > >> > > > > > St.Ack
> > > > > >> > > > > >
> > > > > >> > > > > >
> > > > > >> > > > > >
> > > > > >> > > > > >
> > > > > >> > > > > >
> > > > > >> > > > > >
> > > > > >> > > > > >
> > > > > >> > > > > >
> > > > > >> > > > > >
> > > > > >> > > > > >
> > > > > >> > > > > >
> > > > > >> > > > > > On Mon, Sep 12, 2016 at 8:03 AM, Ted Yu <
> > > > yuzhihong@gmail.com>
> > > > > >> > wrote:
> > > > > >> > > > > >
> > > > > >> > > > > >> Sean:
> > > > > >> > > > > >> Do you have more comments ?
> > > > > >> > > > > >>
> > > > > >> > > > > >> Cheers
> > > > > >> > > > > >>
> > > > > >> > > > > >> On Fri, Sep 9, 2016 at 1:42 PM, Vladimir Rodionov <
> > > > > >> > > > > vladrodionov@gmail.com
> > > > > >> > > > > >> >
> > > > > >> > > > > >> wrote:
> > > > > >> > > > > >>
> > > > > >> > > > > >> > Sean,
> > > > > >> > > > > >> >
> > > > > >> > > > > >> > Backup/Restore can fail due to various reasons:
> > network
> > > > > outage
> > > > > >> > > > > (cluster
> > > > > >> > > > > >> > wide), various time-outs in HBase and HDFS layer,
> M/R
> > > > > failure
> > > > > >> > due
> > > > > >> > > to
> > > > > >> > > > > >> "HDFS
> > > > > >> > > > > >> > exceeded quota", user error (manual deletion of
> data)
> > > and
> > > > > so
> > > > > >> on
> > > > > >> > so
> > > > > >> > > > on.
> > > > > >> > > > > >> That
> > > > > >> > > > > >> > is impossible to enumerate all possible types of
> > > failures
> > > > > in a
> > > > > >> > > > > >> distributed
> > > > > >> > > > > >> > system - that is not our goal/task.
> > > > > >> > > > > >> >
> > > > > >> > > > > >> > We focus completely on backup system table
> > consistency
> > > > in a
> > > > > >> > > presence
> > > > > >> > > > > of
> > > > > >> > > > > >> any
> > > > > >> > > > > >> > type of failure. That is what I call "tolerance to
> > > > > failures".
> > > > > >> > > > > >> >
> > > > > >> > > > > >> > On a failure:
> > > > > >> > > > > >> >
> > > > > >> > > > > >> > BACKUP. All backup system information (prior to
> > backup)
> > > > > will
> > > > > >> be
> > > > > >> > > > > restored
> > > > > >> > > > > >> > and all temporary data, related to a failed
> session,
> > in
> > > > > HDFS
> > > > > >> > will
> > > > > >> > > be
> > > > > >> > > > > >> > deleted
> > > > > >> > > > > >> > RESTORE. We do not care about system data, because
> > > > restore
> > > > > >> does
> > > > > >> > > not
> > > > > >> > > > > >> change
> > > > > >> > > > > >> > it. Temporary data in HDFS will be cleaned up and
> > table
> > > > > will
> > > > > >> be
> > > > > >> > > in a
> > > > > >> > > > > >> state
> > > > > >> > > > > >> > back to where it was before operation started.
> > > > > >> > > > > >> >
> > > > > >> > > > > >> > This is what user should expect in case of a
> failure.
> > > > > >> > > > > >> >
> > > > > >> > > > > >> > -Vlad
> > > > > >> > > > > >> >
> > > > > >> > > > > >> >
> > > > > >> > > > > >> > -Vlad
> > > > > >> > > > > >> >
> > > > > >> > > > > >> > On Fri, Sep 9, 2016 at 12:56 PM, Sean Busbey <
> > > > > >> busbey@apache.org
> > > > > >> > >
> > > > > >> > > > > wrote:
> > > > > >> > > > > >> >
> > > > > >> > > > > >> > > Failing in a consistent way, with docs that
> explain
> > > the
> > > > > >> > various
> > > > > >> > > > > >> > > expected failures would be sufficient.
> > > > > >> > > > > >> > >
> > > > > >> > > > > >> > > On Fri, Sep 9, 2016 at 12:16 PM, Vladimir
> Rodionov
> > > > > >> > > > > >> > > <vl...@gmail.com> wrote:
> > > > > >> > > > > >> > > > Do not worry Sean, doc is coming today as a
> > preview
> > > > and
> > > > > >> our
> > > > > >> > > > writer
> > > > > >> > > > > >> > Frank
> > > > > >> > > > > >> > > > will be working on a putting  it into Apache
> > repo.
> > > > > >> Timeline
> > > > > >> > > > > depends
> > > > > >> > > > > >> on
> > > > > >> > > > > >> > > > Franks schedule but I hope we will get it
> rather
> > > > sooner
> > > > > >> than
> > > > > >> > > > > later.
> > > > > >> > > > > >> > > >
> > > > > >> > > > > >> > > > As for failure testing, we are focusing only
> on a
> > > > > >> consistent
> > > > > >> > > > state
> > > > > >> > > > > >> of
> > > > > >> > > > > >> > > > backup system data in a presence of any type of
> > > > > failures,
> > > > > >> We
> > > > > >> > > are
> > > > > >> > > > > not
> > > > > >> > > > > >> > > going
> > > > > >> > > > > >> > > > to implement  anything more "fancy", than that.
> > We
> > > > > allow
> > > > > >> > both:
> > > > > >> > > > > >> backup
> > > > > >> > > > > >> > and
> > > > > >> > > > > >> > > > restore to fail. What we do not allow is to
> have
> > > > system
> > > > > >> data
> > > > > >> > > > > >> corrupted.
> > > > > >> > > > > >> > > > Will it suffice for you? Do you have any other
> > > > > concerns,
> > > > > >> you
> > > > > >> > > > want
> > > > > >> > > > > >> us to
> > > > > >> > > > > >> > > > address?
> > > > > >> > > > > >> > > >
> > > > > >> > > > > >> > > > -Vlad
> > > > > >> > > > > >> > > >
> > > > > >> > > > > >> > > >
> > > > > >> > > > > >> > > > On Fri, Sep 9, 2016 at 10:56 AM, Sean Busbey <
> > > > > >> > > busbey@apache.org
> > > > > >> > > > >
> > > > > >> > > > > >> > wrote:
> > > > > >> > > > > >> > > >
> > > > > >> > > > > >> > > >> "docs will come to Apache soon" does not
> address
> > > my
> > > > > >> concern
> > > > > >> > > > > around
> > > > > >> > > > > >> > docs
> > > > > >> > > > > >> > > at
> > > > > >> > > > > >> > > >> all, unless said docs have already made it
> into
> > > the
> > > > > >> project
> > > > > >> > > > > repo. I
> > > > > >> > > > > >> > > don't
> > > > > >> > > > > >> > > >> want third party resources for using a major
> and
> > > > > >> important
> > > > > >> > > > > feature
> > > > > >> > > > > >> of
> > > > > >> > > > > >> > > the
> > > > > >> > > > > >> > > >> project, I want us to provide end users with
> > what
> > > > they
> > > > > >> need
> > > > > >> > > to
> > > > > >> > > > > get
> > > > > >> > > > > >> the
> > > > > >> > > > > >> > > job
> > > > > >> > > > > >> > > >> done.
> > > > > >> > > > > >> > > >>
> > > > > >> > > > > >> > > >> I see some calls for patience on the failure
> > > > testing,
> > > > > but
> > > > > >> > the
> > > > > >> > > > > >> appeal
> > > > > >> > > > > >> > to
> > > > > >> > > > > >> > > us
> > > > > >> > > > > >> > > >> having done a bad job of requiring proper
> tests
> > of
> > > > > >> previous
> > > > > >> > > > > >> features
> > > > > >> > > > > >> > > just
> > > > > >> > > > > >> > > >> makes me more concerned about not getting them
> > > > here. I
> > > > > >> > don't
> > > > > >> > > > want
> > > > > >> > > > > >> to
> > > > > >> > > > > >> > set
> > > > > >> > > > > >> > > >> yet another bad example that will then be
> > pointed
> > > to
> > > > > in
> > > > > >> the
> > > > > >> > > > > future.
> > > > > >> > > > > >> > > >>
> > > > > >> > > > > >> > > >> On Sep 8, 2016 10:50, "Ted Yu" <
> > > yuzhihong@gmail.com
> > > > >
> > > > > >> > wrote:
> > > > > >> > > > > >> > > >>
> > > > > >> > > > > >> > > >> > Is there any concern which is not addressed
> ?
> > > > > >> > > > > >> > > >> >
> > > > > >> > > > > >> > > >> > Do we need another Vote thread ?
> > > > > >> > > > > >> > > >> >
> > > > > >> > > > > >> > > >> > Thanks
> > > > > >> > > > > >> > > >> >
> > > > > >> > > > > >> > > >> > On Thu, Sep 8, 2016 at 9:21 AM, Andrew
> > Purtell <
> > > > > >> > > > > >> apurtell@apache.org
> > > > > >> > > > > >> > >
> > > > > >> > > > > >> > > >> > wrote:
> > > > > >> > > > > >> > > >> >
> > > > > >> > > > > >> > > >> > > Vlad,
> > > > > >> > > > > >> > > >> > >
> > > > > >> > > > > >> > > >> > > I apologize for using the term
> 'half-baked'
> > > in a
> > > > > way
> > > > > >> > that
> > > > > >> > > > > could
> > > > > >> > > > > >> > > seem a
> > > > > >> > > > > >> > > >> > > description of HBASE-7912. I meant that
> as a
> > > > > general
> > > > > >> > > > > >> hypothetical.
> > > > > >> > > > > >> > > >> > >
> > > > > >> > > > > >> > > >> > > On Wed, Sep 7, 2016 at 9:36 AM, Vladimir
> > > > Rodionov
> > > > > <
> > > > > >> > > > > >> > > >> > vladrodionov@gmail.com>
> > > > > >> > > > > >> > > >> > > wrote:
> > > > > >> > > > > >> > > >> > >
> > > > > >> > > > > >> > > >> > > > >> I'm not sure that "There is already
> > lots
> > > of
> > > > > >> > > half-baked
> > > > > >> > > > > >> code
> > > > > >> > > > > >> > in
> > > > > >> > > > > >> > > the
> > > > > >> > > > > >> > > >> > > > branch,
> > > > > >> > > > > >> > > >> > > > so what's the harm in adding more?"
> > > > > >> > > > > >> > > >> > > >
> > > > > >> > > > > >> > > >> > > > I meant - not production - ready yet.
> This
> > > is
> > > > > 2.0
> > > > > >> > > > > development
> > > > > >> > > > > >> > > branch
> > > > > >> > > > > >> > > >> > and,
> > > > > >> > > > > >> > > >> > > > hence many features are in works,
> > > > > >> > > > > >> > > >> > > > not being tested well etc. I do not
> > consider
> > > > > backup
> > > > > >> > as
> > > > > >> > > > half
> > > > > >> > > > > >> > baked
> > > > > >> > > > > >> > > >> > > feature -
> > > > > >> > > > > >> > > >> > > > it has passed our internal QA and has
> very
> > > > good
> > > > > >> doc,
> > > > > >> > > > which
> > > > > >> > > > > we
> > > > > >> > > > > >> > will
> > > > > >> > > > > >> > > >> > > provide
> > > > > >> > > > > >> > > >> > > > to Apache shortly.
> > > > > >> > > > > >> > > >> > > >
> > > > > >> > > > > >> > > >> > > > -Vlad
> > > > > >> > > > > >> > > >> > > >
> > > > > >> > > > > >> > > >> > > > On Wed, Sep 7, 2016 at 9:13 AM, Andrew
> > > > Purtell <
> > > > > >> > > > > >> > > apurtell@apache.org>
> > > > > >> > > > > >> > > >> > > > wrote:
> > > > > >> > > > > >> > > >> > > >
> > > > > >> > > > > >> > > >> > > > > We shouldn't admit half baked changes
> > that
> > > > > won't
> > > > > >> be
> > > > > >> > > > > >> finished.
> > > > > >> > > > > >> > > >> However
> > > > > >> > > > > >> > > >> > > in
> > > > > >> > > > > >> > > >> > > > > this case the crew working on this
> > feature
> > > > are
> > > > > >> long
> > > > > >> > > > > timers
> > > > > >> > > > > >> and
> > > > > >> > > > > >> > > less
> > > > > >> > > > > >> > > >> > > > likely
> > > > > >> > > > > >> > > >> > > > > than just about anyone to leave
> > something
> > > > in a
> > > > > >> half
> > > > > >> > > > baked
> > > > > >> > > > > >> > > state. Of
> > > > > >> > > > > >> > > >> > > > course
> > > > > >> > > > > >> > > >> > > > > there is no guarantee how anything
> will
> > > turn
> > > > > out,
> > > > > >> > > but I
> > > > > >> > > > > am
> > > > > >> > > > > >> > > willing
> > > > > >> > > > > >> > > >> to
> > > > > >> > > > > >> > > >> > > > take
> > > > > >> > > > > >> > > >> > > > > a little on faith if they feel their
> > best
> > > > path
> > > > > >> > > forward
> > > > > >> > > > > now
> > > > > >> > > > > >> is
> > > > > >> > > > > >> > to
> > > > > >> > > > > >> > > >> > merge
> > > > > >> > > > > >> > > >> > > to
> > > > > >> > > > > >> > > >> > > > > trunk. I only wish I had bandwidth to
> > have
> > > > > done
> > > > > >> > some
> > > > > >> > > > real
> > > > > >> > > > > >> > > kicking
> > > > > >> > > > > >> > > >> of
> > > > > >> > > > > >> > > >> > > the
> > > > > >> > > > > >> > > >> > > > > tires by now. Maybe this week.
> > > > > >> > > > > >> > > >> > > > >
> > > > > >> > > > > >> > > >> > > > > (Yes, I'm using some of that time for
> > this
> > > > > email
> > > > > >> > :-)
> > > > > >> > > > but
> > > > > >> > > > > I
> > > > > >> > > > > >> > type
> > > > > >> > > > > >> > > >> > fast.)
> > > > > >> > > > > >> > > >> > > > >
> > > > > >> > > > > >> > > >> > > > > That said, I would like to agitate for
> > > > making
> > > > > 2.0
> > > > > >> > > more
> > > > > >> > > > > real
> > > > > >> > > > > >> > and
> > > > > >> > > > > >> > > >> spend
> > > > > >> > > > > >> > > >> > > > some
> > > > > >> > > > > >> > > >> > > > > time on it now that I'm winding down
> > with
> > > > > 0.98. I
> > > > > >> > > think
> > > > > >> > > > > >> that
> > > > > >> > > > > >> > > means
> > > > > >> > > > > >> > > >> > > > > branching for 2.0 real soon now and
> even
> > > > > evicting
> > > > > >> > > > things
> > > > > >> > > > > >> from
> > > > > >> > > > > >> > > 2.0
> > > > > >> > > > > >> > > >> > > branch
> > > > > >> > > > > >> > > >> > > > > that aren't finished or stable,
> leaving
> > > them
> > > > > only
> > > > > >> > > once
> > > > > >> > > > > >> again
> > > > > >> > > > > >> > in
> > > > > >> > > > > >> > > the
> > > > > >> > > > > >> > > >> > > > master
> > > > > >> > > > > >> > > >> > > > > branch. Or, maybe just evicting them.
> > > Let's
> > > > > take
> > > > > >> it
> > > > > >> > > > case
> > > > > >> > > > > by
> > > > > >> > > > > >> > > case.
> > > > > >> > > > > >> > > >> > > > >
> > > > > >> > > > > >> > > >> > > > > I think this feature can come in
> > > relatively
> > > > > >> safely.
> > > > > >> > > As
> > > > > >> > > > > >> added
> > > > > >> > > > > >> > > >> > insurance,
> > > > > >> > > > > >> > > >> > > > > let's admit the possibility it could
> be
> > > > > reverted
> > > > > >> on
> > > > > >> > > the
> > > > > >> > > > > 2.0
> > > > > >> > > > > >> > > branch
> > > > > >> > > > > >> > > >> if
> > > > > >> > > > > >> > > >> > > > folks
> > > > > >> > > > > >> > > >> > > > > working on stabilizing 2.0 decide to
> > evict
> > > > it
> > > > > >> > because
> > > > > >> > > > it
> > > > > >> > > > > is
> > > > > >> > > > > >> > > >> > unfinished
> > > > > >> > > > > >> > > >> > > or
> > > > > >> > > > > >> > > >> > > > > unstable, because that certainly can
> > > > happen. I
> > > > > >> > would
> > > > > >> > > > > >> expect if
> > > > > >> > > > > >> > > talk
> > > > > >> > > > > >> > > >> > > like
> > > > > >> > > > > >> > > >> > > > > that starts, we'd get help finishing
> or
> > > > > >> stabilizing
> > > > > >> > > > > what's
> > > > > >> > > > > >> > under
> > > > > >> > > > > >> > > >> > > > discussion
> > > > > >> > > > > >> > > >> > > > > for revert. Or, we'd have a revert.
> > Either
> > > > way
> > > > > >> the
> > > > > >> > > > > outcome
> > > > > >> > > > > >> is
> > > > > >> > > > > >> > > >> > > acceptable.
> > > > > >> > > > > >> > > >> > > > >
> > > > > >> > > > > >> > > >> > > > >
> > > > > >> > > > > >> > > >> > > > > On Wed, Sep 7, 2016 at 8:56 AM, Dima
> > > Spivak
> > > > <
> > > > > >> > > > > >> > > dimaspivak@apache.org
> > > > > >> > > > > >> > > >> >
> > > > > >> > > > > >> > > >> > > > wrote:
> > > > > >> > > > > >> > > >> > > > >
> > > > > >> > > > > >> > > >> > > > > > I'm not sure that "There is already
> > lots
> > > > of
> > > > > >> > > > half-baked
> > > > > >> > > > > >> code
> > > > > >> > > > > >> > in
> > > > > >> > > > > >> > > >> the
> > > > > >> > > > > >> > > >> > > > > branch,
> > > > > >> > > > > >> > > >> > > > > > so what's the harm in adding more?"
> > is a
> > > > > good
> > > > > >> > code
> > > > > >> > > > > commit
> > > > > >> > > > > >> > > >> > philosophy
> > > > > >> > > > > >> > > >> > > > for
> > > > > >> > > > > >> > > >> > > > > a
> > > > > >> > > > > >> > > >> > > > > > fault-tolerant distributed data
> store.
> > > ;)
> > > > > >> > > > > >> > > >> > > > > >
> > > > > >> > > > > >> > > >> > > > > > More seriously, a lack of test
> > coverage
> > > > for
> > > > > >> > > existing
> > > > > >> > > > > >> > features
> > > > > >> > > > > >> > > >> > > shouldn't
> > > > > >> > > > > >> > > >> > > > > be
> > > > > >> > > > > >> > > >> > > > > > used as justification for
> introducing
> > > new
> > > > > >> > features
> > > > > >> > > > with
> > > > > >> > > > > >> the
> > > > > >> > > > > >> > > same
> > > > > >> > > > > >> > > >> > > > > > shortcomings. Ultimately, it's the
> end
> > > > user
> > > > > who
> > > > > >> > > will
> > > > > >> > > > > feel
> > > > > >> > > > > >> > the
> > > > > >> > > > > >> > > >> pain,
> > > > > >> > > > > >> > > >> > > so
> > > > > >> > > > > >> > > >> > > > > > shouldn't we do everything we can to
> > > > > mitigate
> > > > > >> > that?
> > > > > >> > > > > >> > > >> > > > > >
> > > > > >> > > > > >> > > >> > > > > > -Dima
> > > > > >> > > > > >> > > >> > > > > >
> > > > > >> > > > > >> > > >> > > > > > On Wed, Sep 7, 2016 at 8:46 AM,
> > Vladimir
> > > > > >> > Rodionov <
> > > > > >> > > > > >> > > >> > > > > vladrodionov@gmail.com>
> > > > > >> > > > > >> > > >> > > > > > wrote:
> > > > > >> > > > > >> > > >> > > > > >
> > > > > >> > > > > >> > > >> > > > > > > Sean,
> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > * have docs
> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > Agree. We have a doc and backup is
> > the
> > > > > most
> > > > > >> > > > > documented
> > > > > >> > > > > >> > > feature
> > > > > >> > > > > >> > > >> > :),
> > > > > >> > > > > >> > > >> > > we
> > > > > >> > > > > >> > > >> > > > > > will
> > > > > >> > > > > >> > > >> > > > > > > release it shortly to Apache.
> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > * have sunny-day correctness tests
> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > Feature has  close to 60 test
> cases,
> > > > which
> > > > > >> run
> > > > > >> > > for
> > > > > >> > > > > >> approx
> > > > > >> > > > > >> > 30
> > > > > >> > > > > >> > > >> min.
> > > > > >> > > > > >> > > >> > > We
> > > > > >> > > > > >> > > >> > > > > can
> > > > > >> > > > > >> > > >> > > > > > > add more, if community do not mind
> > :)
> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > * have
> > correctness-in-face-of-failure
> > > > > tests
> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > Any examples of these tests in
> > > existing
> > > > > >> > features?
> > > > > >> > > > In
> > > > > >> > > > > >> > works,
> > > > > >> > > > > >> > > we
> > > > > >> > > > > >> > > >> > > have a
> > > > > >> > > > > >> > > >> > > > > > clear
> > > > > >> > > > > >> > > >> > > > > > > understanding of what should be
> done
> > > by
> > > > > the
> > > > > >> > time
> > > > > >> > > of
> > > > > >> > > > > 2.0
> > > > > >> > > > > >> > > >> release.
> > > > > >> > > > > >> > > >> > > > > > > That is very close goal for us, to
> > > > verify
> > > > > IT
> > > > > >> > > monkey
> > > > > >> > > > > for
> > > > > >> > > > > >> > > >> existing
> > > > > >> > > > > >> > > >> > > > code.
> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > * don't rely on things outside of
> > > HBase
> > > > > for
> > > > > >> > > normal
> > > > > >> > > > > >> > operation
> > > > > >> > > > > >> > > >> > (okay
> > > > > >> > > > > >> > > >> > > > for
> > > > > >> > > > > >> > > >> > > > > > > advanced operation)
> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > We do not.
> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > Enormous time has been spent
> already
> > > on
> > > > > the
> > > > > >> > > > > development
> > > > > >> > > > > >> > and
> > > > > >> > > > > >> > > >> > testing
> > > > > >> > > > > >> > > >> > > > the
> > > > > >> > > > > >> > > >> > > > > > > feature, it has passed our
> internal
> > > > tests
> > > > > and
> > > > > >> > > many
> > > > > >> > > > > >> rounds
> > > > > >> > > > > >> > of
> > > > > >> > > > > >> > > >> code
> > > > > >> > > > > >> > > >> > > > > reviews
> > > > > >> > > > > >> > > >> > > > > > > by HBase committers. We do not
> mind
> > if
> > > > > >> someone
> > > > > >> > > from
> > > > > >> > > > > >> HBase
> > > > > >> > > > > >> > > >> > community
> > > > > >> > > > > >> > > >> > > > > > > (outside of HW) will review the
> > code,
> > > > but
> > > > > it
> > > > > >> > will
> > > > > >> > > > > >> probably
> > > > > >> > > > > >> > > >> takes
> > > > > >> > > > > >> > > >> > > > > forever
> > > > > >> > > > > >> > > >> > > > > > to
> > > > > >> > > > > >> > > >> > > > > > > wait for volunteer?, the feature
> is
> > > > quite
> > > > > >> large
> > > > > >> > > > (1MB+
> > > > > >> > > > > >> > > >> cumulative
> > > > > >> > > > > >> > > >> > > > patch)
> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > 2.0 branch is full of half baked
> > > > features,
> > > > > >> most
> > > > > >> > > of
> > > > > >> > > > > them
> > > > > >> > > > > >> > are
> > > > > >> > > > > >> > > in
> > > > > >> > > > > >> > > >> > > active
> > > > > >> > > > > >> > > >> > > > > > > development, therefore I am not
> > > > following
> > > > > you
> > > > > >> > > here,
> > > > > >> > > > > >> Sean?
> > > > > >> > > > > >> > > Why
> > > > > >> > > > > >> > > >> > > > > HBASE-7912
> > > > > >> > > > > >> > > >> > > > > > is
> > > > > >> > > > > >> > > >> > > > > > > not good enough yet to be
> integrated
> > > > into
> > > > > 2.0
> > > > > >> > > > branch?
> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > -Vlad
> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > On Wed, Sep 7, 2016 at 8:23 AM,
> Sean
> > > > > Busbey <
> > > > > >> > > > > >> > > busbey@apache.org
> > > > > >> > > > > >> > > >> >
> > > > > >> > > > > >> > > >> > > > wrote:
> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > > On Tue, Sep 6, 2016 at 10:36 PM,
> > > Josh
> > > > > >> Elser <
> > > > > >> > > > > >> > > >> > > josh.elser@gmail.com>
> > > > > >> > > > > >> > > >> > > > > > > wrote:
> > > > > >> > > > > >> > > >> > > > > > > > > So, the answer to Sean's
> > original
> > > > > >> question
> > > > > >> > is
> > > > > >> > > > "as
> > > > > >> > > > > >> > > robust as
> > > > > >> > > > > >> > > >> > > > > snapshots
> > > > > >> > > > > >> > > >> > > > > > > > > presently are"? (independence
> of
> > > > > >> > > backup/restore
> > > > > >> > > > > >> > failure
> > > > > >> > > > > >> > > >> > > tolerance
> > > > > >> > > > > >> > > >> > > > > > from
> > > > > >> > > > > >> > > >> > > > > > > > > snapshot failure tolerance)
> > > > > >> > > > > >> > > >> > > > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > > > Is this just a question WRT
> > > context
> > > > of
> > > > > >> the
> > > > > >> > > > > change,
> > > > > >> > > > > >> or
> > > > > >> > > > > >> > > is it
> > > > > >> > > > > >> > > >> > > means
> > > > > >> > > > > >> > > >> > > > > > for a
> > > > > >> > > > > >> > > >> > > > > > > > veto
> > > > > >> > > > > >> > > >> > > > > > > > > from you, Sean? Just trying to
> > > make
> > > > > sure
> > > > > >> > I'm
> > > > > >> > > > > >> following
> > > > > >> > > > > >> > > >> along
> > > > > >> > > > > >> > > >> > > > > > > adequately.
> > > > > >> > > > > >> > > >> > > > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > > I'd say ATM I'm -0, bordering on
> > -1
> > > > but
> > > > > not
> > > > > >> > for
> > > > > >> > > > > >> reasons
> > > > > >> > > > > >> > I
> > > > > >> > > > > >> > > can
> > > > > >> > > > > >> > > >> > > > > > articulate
> > > > > >> > > > > >> > > >> > > > > > > > well.
> > > > > >> > > > > >> > > >> > > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > > Here's an attempt.
> > > > > >> > > > > >> > > >> > > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > > We've been trying to move, as a
> > > > > community,
> > > > > >> > > > towards
> > > > > >> > > > > >> > > minimizing
> > > > > >> > > > > >> > > >> > > risk
> > > > > >> > > > > >> > > >> > > > to
> > > > > >> > > > > >> > > >> > > > > > > > downstream folks by getting
> > > "complete
> > > > > >> enough
> > > > > >> > > for
> > > > > >> > > > > use"
> > > > > >> > > > > >> > > gates
> > > > > >> > > > > >> > > >> in
> > > > > >> > > > > >> > > >> > > > place
> > > > > >> > > > > >> > > >> > > > > > > > before we introduce new
> features.
> > > This
> > > > > was
> > > > > >> > > > spurred
> > > > > >> > > > > >> by a
> > > > > >> > > > > >> > > some
> > > > > >> > > > > >> > > >> > > > features
> > > > > >> > > > > >> > > >> > > > > > > > getting in half-baked and never
> > > making
> > > > > it
> > > > > >> to
> > > > > >> > > "can
> > > > > >> > > > > >> really
> > > > > >> > > > > >> > > use"
> > > > > >> > > > > >> > > >> > > > status
> > > > > >> > > > > >> > > >> > > > > > > > (I'm thinking of distributed log
> > > > replay
> > > > > and
> > > > > >> > the
> > > > > >> > > > > >> zk-less
> > > > > >> > > > > >> > > >> > > assignment
> > > > > >> > > > > >> > > >> > > > > > > > stuff, I don't recall if there
> was
> > > > > more).
> > > > > >> > > > > >> > > >> > > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > > The gates, generally, included
> > > things
> > > > > like:
> > > > > >> > > > > >> > > >> > > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > > * have docs
> > > > > >> > > > > >> > > >> > > > > > > > * have sunny-day correctness
> tests
> > > > > >> > > > > >> > > >> > > > > > > > * have
> > > correctness-in-face-of-failure
> > > > > tests
> > > > > >> > > > > >> > > >> > > > > > > > * don't rely on things outside
> of
> > > > HBase
> > > > > for
> > > > > >> > > > normal
> > > > > >> > > > > >> > > operation
> > > > > >> > > > > >> > > >> > > (okay
> > > > > >> > > > > >> > > >> > > > > for
> > > > > >> > > > > >> > > >> > > > > > > > advanced operation)
> > > > > >> > > > > >> > > >> > > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > > As an example, we kept the MOB
> > work
> > > > off
> > > > > in
> > > > > >> a
> > > > > >> > > > branch
> > > > > >> > > > > >> and
> > > > > >> > > > > >> > > out
> > > > > >> > > > > >> > > >> of
> > > > > >> > > > > >> > > >> > > > master
> > > > > >> > > > > >> > > >> > > > > > > > until it could pass these
> > criteria.
> > > > The
> > > > > big
> > > > > >> > > > > exemption
> > > > > >> > > > > >> > > we've
> > > > > >> > > > > >> > > >> had
> > > > > >> > > > > >> > > >> > > to
> > > > > >> > > > > >> > > >> > > > > > > > this was the hbase-spark
> > > integration,
> > > > > where
> > > > > >> > we
> > > > > >> > > > all
> > > > > >> > > > > >> > agreed
> > > > > >> > > > > >> > > it
> > > > > >> > > > > >> > > >> > > could
> > > > > >> > > > > >> > > >> > > > > > > > land in master because it was
> very
> > > > well
> > > > > >> > > isolated
> > > > > >> > > > > (the
> > > > > >> > > > > >> > > slide
> > > > > >> > > > > >> > > >> > away
> > > > > >> > > > > >> > > >> > > > from
> > > > > >> > > > > >> > > >> > > > > > > > including docs as a first-class
> > part
> > > > of
> > > > > >> > > building
> > > > > >> > > > up
> > > > > >> > > > > >> that
> > > > > >> > > > > >> > > >> > > > integration
> > > > > >> > > > > >> > > >> > > > > > > > has led me to doubt the wisdom
> of
> > > this
> > > > > >> > > decision).
> > > > > >> > > > > >> > > >> > > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > > We've also been treating
> inclusion
> > > in
> > > > a
> > > > > >> > > "probably
> > > > > >> > > > > >> will
> > > > > >> > > > > >> > be
> > > > > >> > > > > >> > > >> > > released
> > > > > >> > > > > >> > > >> > > > to
> > > > > >> > > > > >> > > >> > > > > > > > downstream" branches as a higher
> > > bar,
> > > > > >> > requiring
> > > > > >> > > > > >> > > >> > > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > > * don't moderately impact
> > > performance
> > > > > when
> > > > > >> > the
> > > > > >> > > > > >> feature
> > > > > >> > > > > >> > > isn't
> > > > > >> > > > > >> > > >> in
> > > > > >> > > > > >> > > >> > > use
> > > > > >> > > > > >> > > >> > > > > > > > * don't severely impact
> > performance
> > > > when
> > > > > >> the
> > > > > >> > > > > feature
> > > > > >> > > > > >> is
> > > > > >> > > > > >> > in
> > > > > >> > > > > >> > > >> use
> > > > > >> > > > > >> > > >> > > > > > > > * either default-to-on or show
> > > enough
> > > > > >> demand
> > > > > >> > to
> > > > > >> > > > > >> believe
> > > > > >> > > > > >> > a
> > > > > >> > > > > >> > > >> > > > non-trivial
> > > > > >> > > > > >> > > >> > > > > > > > number of folks will turn the
> > > feature
> > > > on
> > > > > >> > > > > >> > > >> > > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > > The above has kept MOB and
> > > hbase-spark
> > > > > >> > > > integration
> > > > > >> > > > > >> out
> > > > > >> > > > > >> > of
> > > > > >> > > > > >> > > >> > > branch-1,
> > > > > >> > > > > >> > > >> > > > > > > > presumably while they've "gotten
> > > more
> > > > > >> stable"
> > > > > >> > > in
> > > > > >> > > > > >> master
> > > > > >> > > > > >> > > from
> > > > > >> > > > > >> > > >> > the
> > > > > >> > > > > >> > > >> > > > odd
> > > > > >> > > > > >> > > >> > > > > > > > vendor inclusion.
> > > > > >> > > > > >> > > >> > > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > > Are we going to have a 2.0
> release
> > > > > before
> > > > > >> the
> > > > > >> > > end
> > > > > >> > > > > of
> > > > > >> > > > > >> the
> > > > > >> > > > > >> > > >> year?
> > > > > >> > > > > >> > > >> > > > We're
> > > > > >> > > > > >> > > >> > > > > > > > coming up on 1.5 years since the
> > > > > release of
> > > > > >> > > > version
> > > > > >> > > > > >> 1.0;
> > > > > >> > > > > >> > > >> seems
> > > > > >> > > > > >> > > >> > > like
> > > > > >> > > > > >> > > >> > > > > > > > it's about time, though I
> haven't
> > > seen
> > > > > any
> > > > > >> > > > concrete
> > > > > >> > > > > >> > plans
> > > > > >> > > > > >> > > >> this
> > > > > >> > > > > >> > > >> > > > year.
> > > > > >> > > > > >> > > >> > > > > > > > Presuming we are going to have
> one
> > > by
> > > > > the
> > > > > >> end
> > > > > >> > > of
> > > > > >> > > > > the
> > > > > >> > > > > >> > > year, it
> > > > > >> > > > > >> > > >> > > > seems a
> > > > > >> > > > > >> > > >> > > > > > > > bit close to still be adding in
> > > > > "features
> > > > > >> > that
> > > > > >> > > > need
> > > > > >> > > > > >> > > maturing"
> > > > > >> > > > > >> > > >> > on
> > > > > >> > > > > >> > > >> > > > the
> > > > > >> > > > > >> > > >> > > > > > > > branch.
> > > > > >> > > > > >> > > >> > > > > > > >
> > > > > >> > > > > >> > > >> > > > > > > > The lack of a concrete plan for
> > 2.0
> > > > > keeps
> > > > > >> me
> > > > > >> > > from
> > > > > >> > > > > >> > > considering
> > > > > >> > > > > >> > > >> > > these
> > > > > >> > > > > >> > > >> > > > > > > > things blocker at the moment.
> But
> > I
> > > > know
> > > > > >> > first
> > > > > >> > > > hand
> > > > > >> > > > > >> how
> > > > > >> > > > > >> > > much
> > > > > >> > > > > >> > > >> > > > trouble
> > > > > >> > > > > >> > > >> > > > > > > > folks have had with other
> features
> > > > that
> > > > > >> have
> > > > > >> > > gone
> > > > > >> > > > > >> into
> > > > > >> > > > > >> > > >> > downstream
> > > > > >> > > > > >> > > >> > > > > > > > facing releases without
> robustness
> > > > > checks
> > > > > >> > (i.e.
> > > > > >> > > > > >> > > replication),
> > > > > >> > > > > >> > > >> > and
> > > > > >> > > > > >> > > >> > > > I'm
> > > > > >> > > > > >> > > >> > > > > > > > concerned about what we're
> setting
> > > up
> > > > if
> > > > > >> 2.0
> > > > > >> > > goes
> > > > > >> > > > > out
> > > > > >> > > > > >> > with
> > > > > >> > > > > >> > > >> this
> > > > > >> > > > > >> > > >> > > > > > > > feature in its current state.
> > > > > >> > > > > >> > > >> > > > > > > >
> > > > > >> > > > > >> > > >> > > > > > >
> > > > > >> > > > > >> > > >> > > > > >
> > > > > >> > > > > >> > > >> > > > >
> > > > > >> > > > > >> > > >> > > > >
> > > > > >> > > > > >> > > >> > > > >
> > > > > >> > > > > >> > > >> > > > > --
> > > > > >> > > > > >> > > >> > > > > Best regards,
> > > > > >> > > > > >> > > >> > > > >
> > > > > >> > > > > >> > > >> > > > >    - Andy
> > > > > >> > > > > >> > > >> > > > >
> > > > > >> > > > > >> > > >> > > > > Problems worthy of attack prove their
> > > worth
> > > > by
> > > > > >> > > hitting
> > > > > >> > > > > >> back. -
> > > > > >> > > > > >> > > Piet
> > > > > >> > > > > >> > > >> > > Hein
> > > > > >> > > > > >> > > >> > > > > (via Tom White)
> > > > > >> > > > > >> > > >> > > > >
> > > > > >> > > > > >> > > >> > > >
> > > > > >> > > > > >> > > >> > >
> > > > > >> > > > > >> > > >> > >
> > > > > >> > > > > >> > > >> > >
> > > > > >> > > > > >> > > >> > > --
> > > > > >> > > > > >> > > >> > > Best regards,
> > > > > >> > > > > >> > > >> > >
> > > > > >> > > > > >> > > >> > >    - Andy
> > > > > >> > > > > >> > > >> > >
> > > > > >> > > > > >> > > >> > > Problems worthy of attack prove their
> worth
> > by
> > > > > >> hitting
> > > > > >> > > > back.
> > > > > >> > > > > -
> > > > > >> > > > > >> > Piet
> > > > > >> > > > > >> > > >> Hein
> > > > > >> > > > > >> > > >> > > (via Tom White)
> > > > > >> > > > > >> > > >> > >
> > > > > >> > > > > >> > > >> >
> > > > > >> > > > > >> > > >>
> > > > > >> > > > > >> > >
> > > > > >> > > > > >> >
> > > > > >> > > > > >>
> > > > > >> > > > > >
> > > > > >> > > > > >
> > > > > >> > > > >
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > busbey
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Ted Yu <yu...@gmail.com>]

Looks like the first quote was cut off.
The original sentence was:

bq. no mapreduce job launched from master or region server.

mapreduce job is launched from the node where command line tool is run.

On Mon, Oct 10, 2016 at 3:38 PM, Stack <st...@duboce.net> wrote:

> bq. launched from master or region server.
>
> What does this mean please? Has to be run from Master or RegionServer? Can
> it be run from another node altogether?
>
> On Mon, Oct 10, 2016 at 1:44 PM, Vladimir Rodionov <vladrodionov@gmail.com
> >
> wrote:
>
> > >> mapreduce dependency has been moved to client side - no mapreduce job
> >
> > 1. We have no code in the client module anymore, due to dependency on
> > internal server API (HFile and WAL access).
> > 2. Backup/ restore are client - driven operations, but all the code
> resides
> > in the server module
> >
>
> How hard to put in an hbase-backup module? hbase-server is fat enough
> already. Could be done as a follow-up.
>
> Thanks,
> St.Ack
>
>
>
> > 3. No MR in Master, no procedure - driven execution.
> > 4. Old good MR from command-line.
> > 5. Security was simplified and now only super-user is allowed to run
> > backup/restores.
> > 6. HBase Backup API was gone due to 1. Now only command-line access to
> > backup tools.
> >
> > These consequences of refactoring has been discussed in HBASE-16727.
> >
> > -Vlad
> >
> >
> >
> >
> > On Mon, Oct 10, 2016 at 1:31 PM, Ted Yu <yu...@gmail.com> wrote:
> >
> > > Reviving this thread.
> > >
> > > The following has taken place:
> > >
> > > mapreduce dependency has been moved to client side - no mapreduce job
> > > launched from master or region server.
> > > document patch (HBASE-16574) has been integrated.
> > > Updated mega patch has been attached to HBASE-14123: this covers the
> > > refactor in #1 above and the protobuf 3 merge.
> > >
> > > If community has more feedback on the merge proposal, I would love to
> > hear
> > > it.
> > >
> > > Thanks
> > >
> > > On Thu, Sep 22, 2016 at 10:31 AM, Sean Busbey <bu...@cloudera.com>
> > wrote:
> > >
> > > > I'd like to see the docs proposed on HBASE-16574 integrated into our
> > > > project's documentation prior to merge.
> > > >
> > > > On Thu, Sep 22, 2016 at 9:02 AM, Ted Yu <yu...@gmail.com> wrote:
> > > > > This feature can be marked experimental due to some limitations
> such
> > as
> > > > > security.
> > > > >
> > > > > Your previous round of comments have been addressed.
> > > > > Command line tool has gone through:
> > > > >
> > > > > HBASE-16620 Fix backup command-line tool usability issues
> > > > > HBASE-16655 hbase backup describe with incorrect backup id results
> in
> > > NPE
> > > > >
> > > > > The updated doc has been attached to HBASE-16574.
> > > > >
> > > > > Cheers
> > > > >
> > > > > On Thu, Sep 22, 2016 at 8:53 AM, Stack <st...@duboce.net> wrote:
> > > > >
> > > > >> On Wed, Sep 21, 2016 at 7:43 AM, Ted Yu <yu...@gmail.com>
> > wrote:
> > > > >>
> > > > >> > Are there more (review) comments ?
> > > > >> >
> > > > >> >
> > > > >> Are outstanding comments addressed?
> > > > >>
> > > > >> I don't see answer to my 'is this experimental/will it be marked
> > > > >> experimental' question.
> > > > >>
> > > > >> I ran into some issues trying to use the feature and suggested
> that
> > a
> > > > >> feature likes this needs polish else it'll just rot, unused. Has
> > > polish
> > > > >> been applied? All ready for another 'user' test? Suggest that you
> > > update
> > > > >> here going forward for the benefit of those trying to follow along
> > and
> > > > who
> > > > >> are not watching JIRA change fly-by.
> > > > >>
> > > > >> It looks like doc got a revision -- I have to check -- to take on
> > > > >> suggestion made above but again, suggest, that this thread gets
> > > updated.
> > > > >>
> > > > >> Thanks,
> > > > >> St.Ack
> > > > >>
> > > > >>
> > > > >>
> > > > >> > Thanks
> > > > >> >
> > > > >> > On Tue, Sep 20, 2016 at 10:02 AM, Devaraj Das <
> > ddas@hortonworks.com
> > > >
> > > > >> > wrote:
> > > > >> >
> > > > >> > > Just reviving this thread. Thanks Sean, Stack, Dima, and
> others
> > > for
> > > > the
> > > > >> > > thorough reviews and testing. Thanks Ted and Vlad for taking
> > care
> > > of
> > > > >> the
> > > > >> > > feedback. Are we all good to do the merge now? Rather do
> sooner
> > > than
> > > > >> > later.
> > > > >> > > ________________________________________
> > > > >> > > From: saint.ack@gmail.com <sa...@gmail.com> on behalf of
> > > Stack
> > > > <
> > > > >> > > stack@duboce.net>
> > > > >> > > Sent: Monday, September 12, 2016 1:18 PM
> > > > >> > > To: HBase Dev List
> > > > >> > > Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch
> > > HBASE-7912
> > > > >> > >
> > > > >> > > On Mon, Sep 12, 2016 at 12:19 PM, Ted Yu <yuzhihong@gmail.com
> >
> > > > wrote:
> > > > >> > >
> > > > >> > > > Mega patch (rev 18) is on HBASE-14123.
> > > > >> > > >
> > > > >> > > > Please comment on HBASE-14123 on how you want to review.
> > > > >> > > >
> > > > >> > >
> > > > >> > >
> > > > >> > > Yeah. That was my lost tab. Last rb was 6 months ago. Suggest
> > > > updating
> > > > >> > it.
> > > > >> > > RB is pretty good for review. Patch is only 1.5M so should be
> > > fine.
> > > > >> > >
> > > > >> > > St.Ack
> > > > >> > >
> > > > >> > >
> > > > >> > > >
> > > > >> > > > Thanks
> > > > >> > > >
> > > > >> > > > On Mon, Sep 12, 2016 at 12:15 PM, Stack <st...@duboce.net>
> > > wrote:
> > > > >> > > >
> > > > >> > > > > On review of the 'patch', do I just compare the branch to
> > > > master or
> > > > >> > is
> > > > >> > > > > there a megapatch posted somewhere (I think I saw one but
> it
> > > > seemed
> > > > >> > > stale
> > > > >> > > > > and then I 'lost' the tab). Sorry for dumb question.
> > > > >> > > > > St.Ack
> > > > >> > > > >
> > > > >> > > > > On Mon, Sep 12, 2016 at 12:01 PM, Stack <stack@duboce.net
> >
> > > > wrote:
> > > > >> > > > >
> > > > >> > > > > > Late to the game. A few comments after rereading this
> > thread
> > > > as a
> > > > >> > > > 'user'.
> > > > >> > > > > >
> > > > >> > > > > > + Before merge, a user-facing feature like this should
> > work
> > > > (If
> > > > >> > this
> > > > >> > > is
> > > > >> > > > > "higher-bar
> > > > >> > > > > > for new features", bring it on -- smile).
> > > > >> > > > > > + As a user, I tried the branch with tools after
> reviewing
> > > the
> > > > >> > > > > just-posted
> > > > >> > > > > > doc. I had an 'interesting' experience (left comments up
> > on
> > > > >> > issue). I
> > > > >> > > > > think
> > > > >> > > > > > the tooling/doc. important to get right. If it breaks
> > easily
> > > > or
> > > > >> is
> > > > >> > > > > > inconsistent (or lacks 'polish'), operators will judge
> the
> > > > whole
> > > > >> > > > > > backup/restore tooling chain as not trustworthy and
> > abandon
> > > > it.
> > > > >> > Lets
> > > > >> > > > not
> > > > >> > > > > > have this happen to this feature.
> > > > >> > > > > > + Matteo's suggestion (with a helpful starter list) that
> > > there
> > > > >> > needs
> > > > >> > > to
> > > > >> > > > > be
> > > > >> > > > > > explicit qualification on what is actually being
> delivered
> > > --
> > > > >> > > > including a
> > > > >> > > > > > listing of limitations (some look serious such as data
> > bleed
> > > > from
> > > > >> > > other
> > > > >> > > > > > regions in WALs, but maybe I don't care for my use
> > case...)
> > > --
> > > > >> > needs
> > > > >> > > to
> > > > >> > > > > > accompany the merge. Lets fold them into the user doc.
> in
> > > the
> > > > >> > > technical
> > > > >> > > > > > overview area as suggested so user expectations are
> > properly
> > > > >> > managed
> > > > >> > > > > > (otherwise, they expect the world and will just give up
> > when
> > > > we
> > > > >> > fall
> > > > >> > > > > > short). Vladimir did a list of what is in each of the
> > phases
> > > > >> above
> > > > >> > > > which
> > > > >> > > > > > would serve as a good start.
> > > > >> > > > > > + Is this feature 'experimental' (Matteo asks above).
> I'd
> > > > prefer
> > > > >> it
> > > > >> > > is
> > > > >> > > > > > not. If it is, it should be labelled all over that it is
> > > so. I
> > > > >> see
> > > > >> > > > > current
> > > > >> > > > > > state called out as a '... technical preview feature'.
> > Does
> > > > this
> > > > >> > mean
> > > > >> > > > > > not-for-users?
> > > > >> > > > > >
> > > > >> > > > > > St.Ack
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > > > On Mon, Sep 12, 2016 at 8:03 AM, Ted Yu <
> > > yuzhihong@gmail.com>
> > > > >> > wrote:
> > > > >> > > > > >
> > > > >> > > > > >> Sean:
> > > > >> > > > > >> Do you have more comments ?
> > > > >> > > > > >>
> > > > >> > > > > >> Cheers
> > > > >> > > > > >>
> > > > >> > > > > >> On Fri, Sep 9, 2016 at 1:42 PM, Vladimir Rodionov <
> > > > >> > > > > vladrodionov@gmail.com
> > > > >> > > > > >> >
> > > > >> > > > > >> wrote:
> > > > >> > > > > >>
> > > > >> > > > > >> > Sean,
> > > > >> > > > > >> >
> > > > >> > > > > >> > Backup/Restore can fail due to various reasons:
> network
> > > > outage
> > > > >> > > > > (cluster
> > > > >> > > > > >> > wide), various time-outs in HBase and HDFS layer, M/R
> > > > failure
> > > > >> > due
> > > > >> > > to
> > > > >> > > > > >> "HDFS
> > > > >> > > > > >> > exceeded quota", user error (manual deletion of data)
> > and
> > > > so
> > > > >> on
> > > > >> > so
> > > > >> > > > on.
> > > > >> > > > > >> That
> > > > >> > > > > >> > is impossible to enumerate all possible types of
> > failures
> > > > in a
> > > > >> > > > > >> distributed
> > > > >> > > > > >> > system - that is not our goal/task.
> > > > >> > > > > >> >
> > > > >> > > > > >> > We focus completely on backup system table
> consistency
> > > in a
> > > > >> > > presence
> > > > >> > > > > of
> > > > >> > > > > >> any
> > > > >> > > > > >> > type of failure. That is what I call "tolerance to
> > > > failures".
> > > > >> > > > > >> >
> > > > >> > > > > >> > On a failure:
> > > > >> > > > > >> >
> > > > >> > > > > >> > BACKUP. All backup system information (prior to
> backup)
> > > > will
> > > > >> be
> > > > >> > > > > restored
> > > > >> > > > > >> > and all temporary data, related to a failed session,
> in
> > > > HDFS
> > > > >> > will
> > > > >> > > be
> > > > >> > > > > >> > deleted
> > > > >> > > > > >> > RESTORE. We do not care about system data, because
> > > restore
> > > > >> does
> > > > >> > > not
> > > > >> > > > > >> change
> > > > >> > > > > >> > it. Temporary data in HDFS will be cleaned up and
> table
> > > > will
> > > > >> be
> > > > >> > > in a
> > > > >> > > > > >> state
> > > > >> > > > > >> > back to where it was before operation started.
> > > > >> > > > > >> >
> > > > >> > > > > >> > This is what user should expect in case of a failure.
> > > > >> > > > > >> >
> > > > >> > > > > >> > -Vlad
> > > > >> > > > > >> >
> > > > >> > > > > >> >
> > > > >> > > > > >> > -Vlad
> > > > >> > > > > >> >
> > > > >> > > > > >> > On Fri, Sep 9, 2016 at 12:56 PM, Sean Busbey <
> > > > >> busbey@apache.org
> > > > >> > >
> > > > >> > > > > wrote:
> > > > >> > > > > >> >
> > > > >> > > > > >> > > Failing in a consistent way, with docs that explain
> > the
> > > > >> > various
> > > > >> > > > > >> > > expected failures would be sufficient.
> > > > >> > > > > >> > >
> > > > >> > > > > >> > > On Fri, Sep 9, 2016 at 12:16 PM, Vladimir Rodionov
> > > > >> > > > > >> > > <vl...@gmail.com> wrote:
> > > > >> > > > > >> > > > Do not worry Sean, doc is coming today as a
> preview
> > > and
> > > > >> our
> > > > >> > > > writer
> > > > >> > > > > >> > Frank
> > > > >> > > > > >> > > > will be working on a putting  it into Apache
> repo.
> > > > >> Timeline
> > > > >> > > > > depends
> > > > >> > > > > >> on
> > > > >> > > > > >> > > > Franks schedule but I hope we will get it rather
> > > sooner
> > > > >> than
> > > > >> > > > > later.
> > > > >> > > > > >> > > >
> > > > >> > > > > >> > > > As for failure testing, we are focusing only on a
> > > > >> consistent
> > > > >> > > > state
> > > > >> > > > > >> of
> > > > >> > > > > >> > > > backup system data in a presence of any type of
> > > > failures,
> > > > >> We
> > > > >> > > are
> > > > >> > > > > not
> > > > >> > > > > >> > > going
> > > > >> > > > > >> > > > to implement  anything more "fancy", than that.
> We
> > > > allow
> > > > >> > both:
> > > > >> > > > > >> backup
> > > > >> > > > > >> > and
> > > > >> > > > > >> > > > restore to fail. What we do not allow is to have
> > > system
> > > > >> data
> > > > >> > > > > >> corrupted.
> > > > >> > > > > >> > > > Will it suffice for you? Do you have any other
> > > > concerns,
> > > > >> you
> > > > >> > > > want
> > > > >> > > > > >> us to
> > > > >> > > > > >> > > > address?
> > > > >> > > > > >> > > >
> > > > >> > > > > >> > > > -Vlad
> > > > >> > > > > >> > > >
> > > > >> > > > > >> > > >
> > > > >> > > > > >> > > > On Fri, Sep 9, 2016 at 10:56 AM, Sean Busbey <
> > > > >> > > busbey@apache.org
> > > > >> > > > >
> > > > >> > > > > >> > wrote:
> > > > >> > > > > >> > > >
> > > > >> > > > > >> > > >> "docs will come to Apache soon" does not address
> > my
> > > > >> concern
> > > > >> > > > > around
> > > > >> > > > > >> > docs
> > > > >> > > > > >> > > at
> > > > >> > > > > >> > > >> all, unless said docs have already made it into
> > the
> > > > >> project
> > > > >> > > > > repo. I
> > > > >> > > > > >> > > don't
> > > > >> > > > > >> > > >> want third party resources for using a major and
> > > > >> important
> > > > >> > > > > feature
> > > > >> > > > > >> of
> > > > >> > > > > >> > > the
> > > > >> > > > > >> > > >> project, I want us to provide end users with
> what
> > > they
> > > > >> need
> > > > >> > > to
> > > > >> > > > > get
> > > > >> > > > > >> the
> > > > >> > > > > >> > > job
> > > > >> > > > > >> > > >> done.
> > > > >> > > > > >> > > >>
> > > > >> > > > > >> > > >> I see some calls for patience on the failure
> > > testing,
> > > > but
> > > > >> > the
> > > > >> > > > > >> appeal
> > > > >> > > > > >> > to
> > > > >> > > > > >> > > us
> > > > >> > > > > >> > > >> having done a bad job of requiring proper tests
> of
> > > > >> previous
> > > > >> > > > > >> features
> > > > >> > > > > >> > > just
> > > > >> > > > > >> > > >> makes me more concerned about not getting them
> > > here. I
> > > > >> > don't
> > > > >> > > > want
> > > > >> > > > > >> to
> > > > >> > > > > >> > set
> > > > >> > > > > >> > > >> yet another bad example that will then be
> pointed
> > to
> > > > in
> > > > >> the
> > > > >> > > > > future.
> > > > >> > > > > >> > > >>
> > > > >> > > > > >> > > >> On Sep 8, 2016 10:50, "Ted Yu" <
> > yuzhihong@gmail.com
> > > >
> > > > >> > wrote:
> > > > >> > > > > >> > > >>
> > > > >> > > > > >> > > >> > Is there any concern which is not addressed ?
> > > > >> > > > > >> > > >> >
> > > > >> > > > > >> > > >> > Do we need another Vote thread ?
> > > > >> > > > > >> > > >> >
> > > > >> > > > > >> > > >> > Thanks
> > > > >> > > > > >> > > >> >
> > > > >> > > > > >> > > >> > On Thu, Sep 8, 2016 at 9:21 AM, Andrew
> Purtell <
> > > > >> > > > > >> apurtell@apache.org
> > > > >> > > > > >> > >
> > > > >> > > > > >> > > >> > wrote:
> > > > >> > > > > >> > > >> >
> > > > >> > > > > >> > > >> > > Vlad,
> > > > >> > > > > >> > > >> > >
> > > > >> > > > > >> > > >> > > I apologize for using the term 'half-baked'
> > in a
> > > > way
> > > > >> > that
> > > > >> > > > > could
> > > > >> > > > > >> > > seem a
> > > > >> > > > > >> > > >> > > description of HBASE-7912. I meant that as a
> > > > general
> > > > >> > > > > >> hypothetical.
> > > > >> > > > > >> > > >> > >
> > > > >> > > > > >> > > >> > > On Wed, Sep 7, 2016 at 9:36 AM, Vladimir
> > > Rodionov
> > > > <
> > > > >> > > > > >> > > >> > vladrodionov@gmail.com>
> > > > >> > > > > >> > > >> > > wrote:
> > > > >> > > > > >> > > >> > >
> > > > >> > > > > >> > > >> > > > >> I'm not sure that "There is already
> lots
> > of
> > > > >> > > half-baked
> > > > >> > > > > >> code
> > > > >> > > > > >> > in
> > > > >> > > > > >> > > the
> > > > >> > > > > >> > > >> > > > branch,
> > > > >> > > > > >> > > >> > > > so what's the harm in adding more?"
> > > > >> > > > > >> > > >> > > >
> > > > >> > > > > >> > > >> > > > I meant - not production - ready yet. This
> > is
> > > > 2.0
> > > > >> > > > > development
> > > > >> > > > > >> > > branch
> > > > >> > > > > >> > > >> > and,
> > > > >> > > > > >> > > >> > > > hence many features are in works,
> > > > >> > > > > >> > > >> > > > not being tested well etc. I do not
> consider
> > > > backup
> > > > >> > as
> > > > >> > > > half
> > > > >> > > > > >> > baked
> > > > >> > > > > >> > > >> > > feature -
> > > > >> > > > > >> > > >> > > > it has passed our internal QA and has very
> > > good
> > > > >> doc,
> > > > >> > > > which
> > > > >> > > > > we
> > > > >> > > > > >> > will
> > > > >> > > > > >> > > >> > > provide
> > > > >> > > > > >> > > >> > > > to Apache shortly.
> > > > >> > > > > >> > > >> > > >
> > > > >> > > > > >> > > >> > > > -Vlad
> > > > >> > > > > >> > > >> > > >
> > > > >> > > > > >> > > >> > > > On Wed, Sep 7, 2016 at 9:13 AM, Andrew
> > > Purtell <
> > > > >> > > > > >> > > apurtell@apache.org>
> > > > >> > > > > >> > > >> > > > wrote:
> > > > >> > > > > >> > > >> > > >
> > > > >> > > > > >> > > >> > > > > We shouldn't admit half baked changes
> that
> > > > won't
> > > > >> be
> > > > >> > > > > >> finished.
> > > > >> > > > > >> > > >> However
> > > > >> > > > > >> > > >> > > in
> > > > >> > > > > >> > > >> > > > > this case the crew working on this
> feature
> > > are
> > > > >> long
> > > > >> > > > > timers
> > > > >> > > > > >> and
> > > > >> > > > > >> > > less
> > > > >> > > > > >> > > >> > > > likely
> > > > >> > > > > >> > > >> > > > > than just about anyone to leave
> something
> > > in a
> > > > >> half
> > > > >> > > > baked
> > > > >> > > > > >> > > state. Of
> > > > >> > > > > >> > > >> > > > course
> > > > >> > > > > >> > > >> > > > > there is no guarantee how anything will
> > turn
> > > > out,
> > > > >> > > but I
> > > > >> > > > > am
> > > > >> > > > > >> > > willing
> > > > >> > > > > >> > > >> to
> > > > >> > > > > >> > > >> > > > take
> > > > >> > > > > >> > > >> > > > > a little on faith if they feel their
> best
> > > path
> > > > >> > > forward
> > > > >> > > > > now
> > > > >> > > > > >> is
> > > > >> > > > > >> > to
> > > > >> > > > > >> > > >> > merge
> > > > >> > > > > >> > > >> > > to
> > > > >> > > > > >> > > >> > > > > trunk. I only wish I had bandwidth to
> have
> > > > done
> > > > >> > some
> > > > >> > > > real
> > > > >> > > > > >> > > kicking
> > > > >> > > > > >> > > >> of
> > > > >> > > > > >> > > >> > > the
> > > > >> > > > > >> > > >> > > > > tires by now. Maybe this week.
> > > > >> > > > > >> > > >> > > > >
> > > > >> > > > > >> > > >> > > > > (Yes, I'm using some of that time for
> this
> > > > email
> > > > >> > :-)
> > > > >> > > > but
> > > > >> > > > > I
> > > > >> > > > > >> > type
> > > > >> > > > > >> > > >> > fast.)
> > > > >> > > > > >> > > >> > > > >
> > > > >> > > > > >> > > >> > > > > That said, I would like to agitate for
> > > making
> > > > 2.0
> > > > >> > > more
> > > > >> > > > > real
> > > > >> > > > > >> > and
> > > > >> > > > > >> > > >> spend
> > > > >> > > > > >> > > >> > > > some
> > > > >> > > > > >> > > >> > > > > time on it now that I'm winding down
> with
> > > > 0.98. I
> > > > >> > > think
> > > > >> > > > > >> that
> > > > >> > > > > >> > > means
> > > > >> > > > > >> > > >> > > > > branching for 2.0 real soon now and even
> > > > evicting
> > > > >> > > > things
> > > > >> > > > > >> from
> > > > >> > > > > >> > > 2.0
> > > > >> > > > > >> > > >> > > branch
> > > > >> > > > > >> > > >> > > > > that aren't finished or stable, leaving
> > them
> > > > only
> > > > >> > > once
> > > > >> > > > > >> again
> > > > >> > > > > >> > in
> > > > >> > > > > >> > > the
> > > > >> > > > > >> > > >> > > > master
> > > > >> > > > > >> > > >> > > > > branch. Or, maybe just evicting them.
> > Let's
> > > > take
> > > > >> it
> > > > >> > > > case
> > > > >> > > > > by
> > > > >> > > > > >> > > case.
> > > > >> > > > > >> > > >> > > > >
> > > > >> > > > > >> > > >> > > > > I think this feature can come in
> > relatively
> > > > >> safely.
> > > > >> > > As
> > > > >> > > > > >> added
> > > > >> > > > > >> > > >> > insurance,
> > > > >> > > > > >> > > >> > > > > let's admit the possibility it could be
> > > > reverted
> > > > >> on
> > > > >> > > the
> > > > >> > > > > 2.0
> > > > >> > > > > >> > > branch
> > > > >> > > > > >> > > >> if
> > > > >> > > > > >> > > >> > > > folks
> > > > >> > > > > >> > > >> > > > > working on stabilizing 2.0 decide to
> evict
> > > it
> > > > >> > because
> > > > >> > > > it
> > > > >> > > > > is
> > > > >> > > > > >> > > >> > unfinished
> > > > >> > > > > >> > > >> > > or
> > > > >> > > > > >> > > >> > > > > unstable, because that certainly can
> > > happen. I
> > > > >> > would
> > > > >> > > > > >> expect if
> > > > >> > > > > >> > > talk
> > > > >> > > > > >> > > >> > > like
> > > > >> > > > > >> > > >> > > > > that starts, we'd get help finishing or
> > > > >> stabilizing
> > > > >> > > > > what's
> > > > >> > > > > >> > under
> > > > >> > > > > >> > > >> > > > discussion
> > > > >> > > > > >> > > >> > > > > for revert. Or, we'd have a revert.
> Either
> > > way
> > > > >> the
> > > > >> > > > > outcome
> > > > >> > > > > >> is
> > > > >> > > > > >> > > >> > > acceptable.
> > > > >> > > > > >> > > >> > > > >
> > > > >> > > > > >> > > >> > > > >
> > > > >> > > > > >> > > >> > > > > On Wed, Sep 7, 2016 at 8:56 AM, Dima
> > Spivak
> > > <
> > > > >> > > > > >> > > dimaspivak@apache.org
> > > > >> > > > > >> > > >> >
> > > > >> > > > > >> > > >> > > > wrote:
> > > > >> > > > > >> > > >> > > > >
> > > > >> > > > > >> > > >> > > > > > I'm not sure that "There is already
> lots
> > > of
> > > > >> > > > half-baked
> > > > >> > > > > >> code
> > > > >> > > > > >> > in
> > > > >> > > > > >> > > >> the
> > > > >> > > > > >> > > >> > > > > branch,
> > > > >> > > > > >> > > >> > > > > > so what's the harm in adding more?"
> is a
> > > > good
> > > > >> > code
> > > > >> > > > > commit
> > > > >> > > > > >> > > >> > philosophy
> > > > >> > > > > >> > > >> > > > for
> > > > >> > > > > >> > > >> > > > > a
> > > > >> > > > > >> > > >> > > > > > fault-tolerant distributed data store.
> > ;)
> > > > >> > > > > >> > > >> > > > > >
> > > > >> > > > > >> > > >> > > > > > More seriously, a lack of test
> coverage
> > > for
> > > > >> > > existing
> > > > >> > > > > >> > features
> > > > >> > > > > >> > > >> > > shouldn't
> > > > >> > > > > >> > > >> > > > > be
> > > > >> > > > > >> > > >> > > > > > used as justification for introducing
> > new
> > > > >> > features
> > > > >> > > > with
> > > > >> > > > > >> the
> > > > >> > > > > >> > > same
> > > > >> > > > > >> > > >> > > > > > shortcomings. Ultimately, it's the end
> > > user
> > > > who
> > > > >> > > will
> > > > >> > > > > feel
> > > > >> > > > > >> > the
> > > > >> > > > > >> > > >> pain,
> > > > >> > > > > >> > > >> > > so
> > > > >> > > > > >> > > >> > > > > > shouldn't we do everything we can to
> > > > mitigate
> > > > >> > that?
> > > > >> > > > > >> > > >> > > > > >
> > > > >> > > > > >> > > >> > > > > > -Dima
> > > > >> > > > > >> > > >> > > > > >
> > > > >> > > > > >> > > >> > > > > > On Wed, Sep 7, 2016 at 8:46 AM,
> Vladimir
> > > > >> > Rodionov <
> > > > >> > > > > >> > > >> > > > > vladrodionov@gmail.com>
> > > > >> > > > > >> > > >> > > > > > wrote:
> > > > >> > > > > >> > > >> > > > > >
> > > > >> > > > > >> > > >> > > > > > > Sean,
> > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > >> > > > > > > * have docs
> > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > >> > > > > > > Agree. We have a doc and backup is
> the
> > > > most
> > > > >> > > > > documented
> > > > >> > > > > >> > > feature
> > > > >> > > > > >> > > >> > :),
> > > > >> > > > > >> > > >> > > we
> > > > >> > > > > >> > > >> > > > > > will
> > > > >> > > > > >> > > >> > > > > > > release it shortly to Apache.
> > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > >> > > > > > > * have sunny-day correctness tests
> > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > >> > > > > > > Feature has  close to 60 test cases,
> > > which
> > > > >> run
> > > > >> > > for
> > > > >> > > > > >> approx
> > > > >> > > > > >> > 30
> > > > >> > > > > >> > > >> min.
> > > > >> > > > > >> > > >> > > We
> > > > >> > > > > >> > > >> > > > > can
> > > > >> > > > > >> > > >> > > > > > > add more, if community do not mind
> :)
> > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > >> > > > > > > * have
> correctness-in-face-of-failure
> > > > tests
> > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > >> > > > > > > Any examples of these tests in
> > existing
> > > > >> > features?
> > > > >> > > > In
> > > > >> > > > > >> > works,
> > > > >> > > > > >> > > we
> > > > >> > > > > >> > > >> > > have a
> > > > >> > > > > >> > > >> > > > > > clear
> > > > >> > > > > >> > > >> > > > > > > understanding of what should be done
> > by
> > > > the
> > > > >> > time
> > > > >> > > of
> > > > >> > > > > 2.0
> > > > >> > > > > >> > > >> release.
> > > > >> > > > > >> > > >> > > > > > > That is very close goal for us, to
> > > verify
> > > > IT
> > > > >> > > monkey
> > > > >> > > > > for
> > > > >> > > > > >> > > >> existing
> > > > >> > > > > >> > > >> > > > code.
> > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > >> > > > > > > * don't rely on things outside of
> > HBase
> > > > for
> > > > >> > > normal
> > > > >> > > > > >> > operation
> > > > >> > > > > >> > > >> > (okay
> > > > >> > > > > >> > > >> > > > for
> > > > >> > > > > >> > > >> > > > > > > advanced operation)
> > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > >> > > > > > > We do not.
> > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > >> > > > > > > Enormous time has been spent already
> > on
> > > > the
> > > > >> > > > > development
> > > > >> > > > > >> > and
> > > > >> > > > > >> > > >> > testing
> > > > >> > > > > >> > > >> > > > the
> > > > >> > > > > >> > > >> > > > > > > feature, it has passed our internal
> > > tests
> > > > and
> > > > >> > > many
> > > > >> > > > > >> rounds
> > > > >> > > > > >> > of
> > > > >> > > > > >> > > >> code
> > > > >> > > > > >> > > >> > > > > reviews
> > > > >> > > > > >> > > >> > > > > > > by HBase committers. We do not mind
> if
> > > > >> someone
> > > > >> > > from
> > > > >> > > > > >> HBase
> > > > >> > > > > >> > > >> > community
> > > > >> > > > > >> > > >> > > > > > > (outside of HW) will review the
> code,
> > > but
> > > > it
> > > > >> > will
> > > > >> > > > > >> probably
> > > > >> > > > > >> > > >> takes
> > > > >> > > > > >> > > >> > > > > forever
> > > > >> > > > > >> > > >> > > > > > to
> > > > >> > > > > >> > > >> > > > > > > wait for volunteer?, the feature is
> > > quite
> > > > >> large
> > > > >> > > > (1MB+
> > > > >> > > > > >> > > >> cumulative
> > > > >> > > > > >> > > >> > > > patch)
> > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > >> > > > > > > 2.0 branch is full of half baked
> > > features,
> > > > >> most
> > > > >> > > of
> > > > >> > > > > them
> > > > >> > > > > >> > are
> > > > >> > > > > >> > > in
> > > > >> > > > > >> > > >> > > active
> > > > >> > > > > >> > > >> > > > > > > development, therefore I am not
> > > following
> > > > you
> > > > >> > > here,
> > > > >> > > > > >> Sean?
> > > > >> > > > > >> > > Why
> > > > >> > > > > >> > > >> > > > > HBASE-7912
> > > > >> > > > > >> > > >> > > > > > is
> > > > >> > > > > >> > > >> > > > > > > not good enough yet to be integrated
> > > into
> > > > 2.0
> > > > >> > > > branch?
> > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > >> > > > > > > -Vlad
> > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > >> > > > > > > On Wed, Sep 7, 2016 at 8:23 AM, Sean
> > > > Busbey <
> > > > >> > > > > >> > > busbey@apache.org
> > > > >> > > > > >> > > >> >
> > > > >> > > > > >> > > >> > > > wrote:
> > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > >> > > > > > > > On Tue, Sep 6, 2016 at 10:36 PM,
> > Josh
> > > > >> Elser <
> > > > >> > > > > >> > > >> > > josh.elser@gmail.com>
> > > > >> > > > > >> > > >> > > > > > > wrote:
> > > > >> > > > > >> > > >> > > > > > > > > So, the answer to Sean's
> original
> > > > >> question
> > > > >> > is
> > > > >> > > > "as
> > > > >> > > > > >> > > robust as
> > > > >> > > > > >> > > >> > > > > snapshots
> > > > >> > > > > >> > > >> > > > > > > > > presently are"? (independence of
> > > > >> > > backup/restore
> > > > >> > > > > >> > failure
> > > > >> > > > > >> > > >> > > tolerance
> > > > >> > > > > >> > > >> > > > > > from
> > > > >> > > > > >> > > >> > > > > > > > > snapshot failure tolerance)
> > > > >> > > > > >> > > >> > > > > > > > >
> > > > >> > > > > >> > > >> > > > > > > > > Is this just a question WRT
> > context
> > > of
> > > > >> the
> > > > >> > > > > change,
> > > > >> > > > > >> or
> > > > >> > > > > >> > > is it
> > > > >> > > > > >> > > >> > > means
> > > > >> > > > > >> > > >> > > > > > for a
> > > > >> > > > > >> > > >> > > > > > > > veto
> > > > >> > > > > >> > > >> > > > > > > > > from you, Sean? Just trying to
> > make
> > > > sure
> > > > >> > I'm
> > > > >> > > > > >> following
> > > > >> > > > > >> > > >> along
> > > > >> > > > > >> > > >> > > > > > > adequately.
> > > > >> > > > > >> > > >> > > > > > > > >
> > > > >> > > > > >> > > >> > > > > > > > >
> > > > >> > > > > >> > > >> > > > > > > >
> > > > >> > > > > >> > > >> > > > > > > > I'd say ATM I'm -0, bordering on
> -1
> > > but
> > > > not
> > > > >> > for
> > > > >> > > > > >> reasons
> > > > >> > > > > >> > I
> > > > >> > > > > >> > > can
> > > > >> > > > > >> > > >> > > > > > articulate
> > > > >> > > > > >> > > >> > > > > > > > well.
> > > > >> > > > > >> > > >> > > > > > > >
> > > > >> > > > > >> > > >> > > > > > > > Here's an attempt.
> > > > >> > > > > >> > > >> > > > > > > >
> > > > >> > > > > >> > > >> > > > > > > > We've been trying to move, as a
> > > > community,
> > > > >> > > > towards
> > > > >> > > > > >> > > minimizing
> > > > >> > > > > >> > > >> > > risk
> > > > >> > > > > >> > > >> > > > to
> > > > >> > > > > >> > > >> > > > > > > > downstream folks by getting
> > "complete
> > > > >> enough
> > > > >> > > for
> > > > >> > > > > use"
> > > > >> > > > > >> > > gates
> > > > >> > > > > >> > > >> in
> > > > >> > > > > >> > > >> > > > place
> > > > >> > > > > >> > > >> > > > > > > > before we introduce new features.
> > This
> > > > was
> > > > >> > > > spurred
> > > > >> > > > > >> by a
> > > > >> > > > > >> > > some
> > > > >> > > > > >> > > >> > > > features
> > > > >> > > > > >> > > >> > > > > > > > getting in half-baked and never
> > making
> > > > it
> > > > >> to
> > > > >> > > "can
> > > > >> > > > > >> really
> > > > >> > > > > >> > > use"
> > > > >> > > > > >> > > >> > > > status
> > > > >> > > > > >> > > >> > > > > > > > (I'm thinking of distributed log
> > > replay
> > > > and
> > > > >> > the
> > > > >> > > > > >> zk-less
> > > > >> > > > > >> > > >> > > assignment
> > > > >> > > > > >> > > >> > > > > > > > stuff, I don't recall if there was
> > > > more).
> > > > >> > > > > >> > > >> > > > > > > >
> > > > >> > > > > >> > > >> > > > > > > > The gates, generally, included
> > things
> > > > like:
> > > > >> > > > > >> > > >> > > > > > > >
> > > > >> > > > > >> > > >> > > > > > > > * have docs
> > > > >> > > > > >> > > >> > > > > > > > * have sunny-day correctness tests
> > > > >> > > > > >> > > >> > > > > > > > * have
> > correctness-in-face-of-failure
> > > > tests
> > > > >> > > > > >> > > >> > > > > > > > * don't rely on things outside of
> > > HBase
> > > > for
> > > > >> > > > normal
> > > > >> > > > > >> > > operation
> > > > >> > > > > >> > > >> > > (okay
> > > > >> > > > > >> > > >> > > > > for
> > > > >> > > > > >> > > >> > > > > > > > advanced operation)
> > > > >> > > > > >> > > >> > > > > > > >
> > > > >> > > > > >> > > >> > > > > > > > As an example, we kept the MOB
> work
> > > off
> > > > in
> > > > >> a
> > > > >> > > > branch
> > > > >> > > > > >> and
> > > > >> > > > > >> > > out
> > > > >> > > > > >> > > >> of
> > > > >> > > > > >> > > >> > > > master
> > > > >> > > > > >> > > >> > > > > > > > until it could pass these
> criteria.
> > > The
> > > > big
> > > > >> > > > > exemption
> > > > >> > > > > >> > > we've
> > > > >> > > > > >> > > >> had
> > > > >> > > > > >> > > >> > > to
> > > > >> > > > > >> > > >> > > > > > > > this was the hbase-spark
> > integration,
> > > > where
> > > > >> > we
> > > > >> > > > all
> > > > >> > > > > >> > agreed
> > > > >> > > > > >> > > it
> > > > >> > > > > >> > > >> > > could
> > > > >> > > > > >> > > >> > > > > > > > land in master because it was very
> > > well
> > > > >> > > isolated
> > > > >> > > > > (the
> > > > >> > > > > >> > > slide
> > > > >> > > > > >> > > >> > away
> > > > >> > > > > >> > > >> > > > from
> > > > >> > > > > >> > > >> > > > > > > > including docs as a first-class
> part
> > > of
> > > > >> > > building
> > > > >> > > > up
> > > > >> > > > > >> that
> > > > >> > > > > >> > > >> > > > integration
> > > > >> > > > > >> > > >> > > > > > > > has led me to doubt the wisdom of
> > this
> > > > >> > > decision).
> > > > >> > > > > >> > > >> > > > > > > >
> > > > >> > > > > >> > > >> > > > > > > > We've also been treating inclusion
> > in
> > > a
> > > > >> > > "probably
> > > > >> > > > > >> will
> > > > >> > > > > >> > be
> > > > >> > > > > >> > > >> > > released
> > > > >> > > > > >> > > >> > > > to
> > > > >> > > > > >> > > >> > > > > > > > downstream" branches as a higher
> > bar,
> > > > >> > requiring
> > > > >> > > > > >> > > >> > > > > > > >
> > > > >> > > > > >> > > >> > > > > > > > * don't moderately impact
> > performance
> > > > when
> > > > >> > the
> > > > >> > > > > >> feature
> > > > >> > > > > >> > > isn't
> > > > >> > > > > >> > > >> in
> > > > >> > > > > >> > > >> > > use
> > > > >> > > > > >> > > >> > > > > > > > * don't severely impact
> performance
> > > when
> > > > >> the
> > > > >> > > > > feature
> > > > >> > > > > >> is
> > > > >> > > > > >> > in
> > > > >> > > > > >> > > >> use
> > > > >> > > > > >> > > >> > > > > > > > * either default-to-on or show
> > enough
> > > > >> demand
> > > > >> > to
> > > > >> > > > > >> believe
> > > > >> > > > > >> > a
> > > > >> > > > > >> > > >> > > > non-trivial
> > > > >> > > > > >> > > >> > > > > > > > number of folks will turn the
> > feature
> > > on
> > > > >> > > > > >> > > >> > > > > > > >
> > > > >> > > > > >> > > >> > > > > > > > The above has kept MOB and
> > hbase-spark
> > > > >> > > > integration
> > > > >> > > > > >> out
> > > > >> > > > > >> > of
> > > > >> > > > > >> > > >> > > branch-1,
> > > > >> > > > > >> > > >> > > > > > > > presumably while they've "gotten
> > more
> > > > >> stable"
> > > > >> > > in
> > > > >> > > > > >> master
> > > > >> > > > > >> > > from
> > > > >> > > > > >> > > >> > the
> > > > >> > > > > >> > > >> > > > odd
> > > > >> > > > > >> > > >> > > > > > > > vendor inclusion.
> > > > >> > > > > >> > > >> > > > > > > >
> > > > >> > > > > >> > > >> > > > > > > > Are we going to have a 2.0 release
> > > > before
> > > > >> the
> > > > >> > > end
> > > > >> > > > > of
> > > > >> > > > > >> the
> > > > >> > > > > >> > > >> year?
> > > > >> > > > > >> > > >> > > > We're
> > > > >> > > > > >> > > >> > > > > > > > coming up on 1.5 years since the
> > > > release of
> > > > >> > > > version
> > > > >> > > > > >> 1.0;
> > > > >> > > > > >> > > >> seems
> > > > >> > > > > >> > > >> > > like
> > > > >> > > > > >> > > >> > > > > > > > it's about time, though I haven't
> > seen
> > > > any
> > > > >> > > > concrete
> > > > >> > > > > >> > plans
> > > > >> > > > > >> > > >> this
> > > > >> > > > > >> > > >> > > > year.
> > > > >> > > > > >> > > >> > > > > > > > Presuming we are going to have one
> > by
> > > > the
> > > > >> end
> > > > >> > > of
> > > > >> > > > > the
> > > > >> > > > > >> > > year, it
> > > > >> > > > > >> > > >> > > > seems a
> > > > >> > > > > >> > > >> > > > > > > > bit close to still be adding in
> > > > "features
> > > > >> > that
> > > > >> > > > need
> > > > >> > > > > >> > > maturing"
> > > > >> > > > > >> > > >> > on
> > > > >> > > > > >> > > >> > > > the
> > > > >> > > > > >> > > >> > > > > > > > branch.
> > > > >> > > > > >> > > >> > > > > > > >
> > > > >> > > > > >> > > >> > > > > > > > The lack of a concrete plan for
> 2.0
> > > > keeps
> > > > >> me
> > > > >> > > from
> > > > >> > > > > >> > > considering
> > > > >> > > > > >> > > >> > > these
> > > > >> > > > > >> > > >> > > > > > > > things blocker at the moment. But
> I
> > > know
> > > > >> > first
> > > > >> > > > hand
> > > > >> > > > > >> how
> > > > >> > > > > >> > > much
> > > > >> > > > > >> > > >> > > > trouble
> > > > >> > > > > >> > > >> > > > > > > > folks have had with other features
> > > that
> > > > >> have
> > > > >> > > gone
> > > > >> > > > > >> into
> > > > >> > > > > >> > > >> > downstream
> > > > >> > > > > >> > > >> > > > > > > > facing releases without robustness
> > > > checks
> > > > >> > (i.e.
> > > > >> > > > > >> > > replication),
> > > > >> > > > > >> > > >> > and
> > > > >> > > > > >> > > >> > > > I'm
> > > > >> > > > > >> > > >> > > > > > > > concerned about what we're setting
> > up
> > > if
> > > > >> 2.0
> > > > >> > > goes
> > > > >> > > > > out
> > > > >> > > > > >> > with
> > > > >> > > > > >> > > >> this
> > > > >> > > > > >> > > >> > > > > > > > feature in its current state.
> > > > >> > > > > >> > > >> > > > > > > >
> > > > >> > > > > >> > > >> > > > > > >
> > > > >> > > > > >> > > >> > > > > >
> > > > >> > > > > >> > > >> > > > >
> > > > >> > > > > >> > > >> > > > >
> > > > >> > > > > >> > > >> > > > >
> > > > >> > > > > >> > > >> > > > > --
> > > > >> > > > > >> > > >> > > > > Best regards,
> > > > >> > > > > >> > > >> > > > >
> > > > >> > > > > >> > > >> > > > >    - Andy
> > > > >> > > > > >> > > >> > > > >
> > > > >> > > > > >> > > >> > > > > Problems worthy of attack prove their
> > worth
> > > by
> > > > >> > > hitting
> > > > >> > > > > >> back. -
> > > > >> > > > > >> > > Piet
> > > > >> > > > > >> > > >> > > Hein
> > > > >> > > > > >> > > >> > > > > (via Tom White)
> > > > >> > > > > >> > > >> > > > >
> > > > >> > > > > >> > > >> > > >
> > > > >> > > > > >> > > >> > >
> > > > >> > > > > >> > > >> > >
> > > > >> > > > > >> > > >> > >
> > > > >> > > > > >> > > >> > > --
> > > > >> > > > > >> > > >> > > Best regards,
> > > > >> > > > > >> > > >> > >
> > > > >> > > > > >> > > >> > >    - Andy
> > > > >> > > > > >> > > >> > >
> > > > >> > > > > >> > > >> > > Problems worthy of attack prove their worth
> by
> > > > >> hitting
> > > > >> > > > back.
> > > > >> > > > > -
> > > > >> > > > > >> > Piet
> > > > >> > > > > >> > > >> Hein
> > > > >> > > > > >> > > >> > > (via Tom White)
> > > > >> > > > > >> > > >> > >
> > > > >> > > > > >> > > >> >
> > > > >> > > > > >> > > >>
> > > > >> > > > > >> > >
> > > > >> > > > > >> >
> > > > >> > > > > >>
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > busbey
> > > >
> > >
> >
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Stack <st...@duboce.net>]

bq. launched from master or region server.

What does this mean please? Has to be run from Master or RegionServer? Can
it be run from another node altogether?

On Mon, Oct 10, 2016 at 1:44 PM, Vladimir Rodionov <vl...@gmail.com>
wrote:

> >> mapreduce dependency has been moved to client side - no mapreduce job
>
> 1. We have no code in the client module anymore, due to dependency on
> internal server API (HFile and WAL access).
> 2. Backup/ restore are client - driven operations, but all the code resides
> in the server module
>

How hard to put in an hbase-backup module? hbase-server is fat enough
already. Could be done as a follow-up.

Thanks,
St.Ack



> 3. No MR in Master, no procedure - driven execution.
> 4. Old good MR from command-line.
> 5. Security was simplified and now only super-user is allowed to run
> backup/restores.
> 6. HBase Backup API was gone due to 1. Now only command-line access to
> backup tools.
>
> These consequences of refactoring has been discussed in HBASE-16727.
>
> -Vlad
>
>
>
>
> On Mon, Oct 10, 2016 at 1:31 PM, Ted Yu <yu...@gmail.com> wrote:
>
> > Reviving this thread.
> >
> > The following has taken place:
> >
> > mapreduce dependency has been moved to client side - no mapreduce job
> > launched from master or region server.
> > document patch (HBASE-16574) has been integrated.
> > Updated mega patch has been attached to HBASE-14123: this covers the
> > refactor in #1 above and the protobuf 3 merge.
> >
> > If community has more feedback on the merge proposal, I would love to
> hear
> > it.
> >
> > Thanks
> >
> > On Thu, Sep 22, 2016 at 10:31 AM, Sean Busbey <bu...@cloudera.com>
> wrote:
> >
> > > I'd like to see the docs proposed on HBASE-16574 integrated into our
> > > project's documentation prior to merge.
> > >
> > > On Thu, Sep 22, 2016 at 9:02 AM, Ted Yu <yu...@gmail.com> wrote:
> > > > This feature can be marked experimental due to some limitations such
> as
> > > > security.
> > > >
> > > > Your previous round of comments have been addressed.
> > > > Command line tool has gone through:
> > > >
> > > > HBASE-16620 Fix backup command-line tool usability issues
> > > > HBASE-16655 hbase backup describe with incorrect backup id results in
> > NPE
> > > >
> > > > The updated doc has been attached to HBASE-16574.
> > > >
> > > > Cheers
> > > >
> > > > On Thu, Sep 22, 2016 at 8:53 AM, Stack <st...@duboce.net> wrote:
> > > >
> > > >> On Wed, Sep 21, 2016 at 7:43 AM, Ted Yu <yu...@gmail.com>
> wrote:
> > > >>
> > > >> > Are there more (review) comments ?
> > > >> >
> > > >> >
> > > >> Are outstanding comments addressed?
> > > >>
> > > >> I don't see answer to my 'is this experimental/will it be marked
> > > >> experimental' question.
> > > >>
> > > >> I ran into some issues trying to use the feature and suggested that
> a
> > > >> feature likes this needs polish else it'll just rot, unused. Has
> > polish
> > > >> been applied? All ready for another 'user' test? Suggest that you
> > update
> > > >> here going forward for the benefit of those trying to follow along
> and
> > > who
> > > >> are not watching JIRA change fly-by.
> > > >>
> > > >> It looks like doc got a revision -- I have to check -- to take on
> > > >> suggestion made above but again, suggest, that this thread gets
> > updated.
> > > >>
> > > >> Thanks,
> > > >> St.Ack
> > > >>
> > > >>
> > > >>
> > > >> > Thanks
> > > >> >
> > > >> > On Tue, Sep 20, 2016 at 10:02 AM, Devaraj Das <
> ddas@hortonworks.com
> > >
> > > >> > wrote:
> > > >> >
> > > >> > > Just reviving this thread. Thanks Sean, Stack, Dima, and others
> > for
> > > the
> > > >> > > thorough reviews and testing. Thanks Ted and Vlad for taking
> care
> > of
> > > >> the
> > > >> > > feedback. Are we all good to do the merge now? Rather do sooner
> > than
> > > >> > later.
> > > >> > > ________________________________________
> > > >> > > From: saint.ack@gmail.com <sa...@gmail.com> on behalf of
> > Stack
> > > <
> > > >> > > stack@duboce.net>
> > > >> > > Sent: Monday, September 12, 2016 1:18 PM
> > > >> > > To: HBase Dev List
> > > >> > > Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch
> > HBASE-7912
> > > >> > >
> > > >> > > On Mon, Sep 12, 2016 at 12:19 PM, Ted Yu <yu...@gmail.com>
> > > wrote:
> > > >> > >
> > > >> > > > Mega patch (rev 18) is on HBASE-14123.
> > > >> > > >
> > > >> > > > Please comment on HBASE-14123 on how you want to review.
> > > >> > > >
> > > >> > >
> > > >> > >
> > > >> > > Yeah. That was my lost tab. Last rb was 6 months ago. Suggest
> > > updating
> > > >> > it.
> > > >> > > RB is pretty good for review. Patch is only 1.5M so should be
> > fine.
> > > >> > >
> > > >> > > St.Ack
> > > >> > >
> > > >> > >
> > > >> > > >
> > > >> > > > Thanks
> > > >> > > >
> > > >> > > > On Mon, Sep 12, 2016 at 12:15 PM, Stack <st...@duboce.net>
> > wrote:
> > > >> > > >
> > > >> > > > > On review of the 'patch', do I just compare the branch to
> > > master or
> > > >> > is
> > > >> > > > > there a megapatch posted somewhere (I think I saw one but it
> > > seemed
> > > >> > > stale
> > > >> > > > > and then I 'lost' the tab). Sorry for dumb question.
> > > >> > > > > St.Ack
> > > >> > > > >
> > > >> > > > > On Mon, Sep 12, 2016 at 12:01 PM, Stack <st...@duboce.net>
> > > wrote:
> > > >> > > > >
> > > >> > > > > > Late to the game. A few comments after rereading this
> thread
> > > as a
> > > >> > > > 'user'.
> > > >> > > > > >
> > > >> > > > > > + Before merge, a user-facing feature like this should
> work
> > > (If
> > > >> > this
> > > >> > > is
> > > >> > > > > "higher-bar
> > > >> > > > > > for new features", bring it on -- smile).
> > > >> > > > > > + As a user, I tried the branch with tools after reviewing
> > the
> > > >> > > > > just-posted
> > > >> > > > > > doc. I had an 'interesting' experience (left comments up
> on
> > > >> > issue). I
> > > >> > > > > think
> > > >> > > > > > the tooling/doc. important to get right. If it breaks
> easily
> > > or
> > > >> is
> > > >> > > > > > inconsistent (or lacks 'polish'), operators will judge the
> > > whole
> > > >> > > > > > backup/restore tooling chain as not trustworthy and
> abandon
> > > it.
> > > >> > Lets
> > > >> > > > not
> > > >> > > > > > have this happen to this feature.
> > > >> > > > > > + Matteo's suggestion (with a helpful starter list) that
> > there
> > > >> > needs
> > > >> > > to
> > > >> > > > > be
> > > >> > > > > > explicit qualification on what is actually being delivered
> > --
> > > >> > > > including a
> > > >> > > > > > listing of limitations (some look serious such as data
> bleed
> > > from
> > > >> > > other
> > > >> > > > > > regions in WALs, but maybe I don't care for my use
> case...)
> > --
> > > >> > needs
> > > >> > > to
> > > >> > > > > > accompany the merge. Lets fold them into the user doc. in
> > the
> > > >> > > technical
> > > >> > > > > > overview area as suggested so user expectations are
> properly
> > > >> > managed
> > > >> > > > > > (otherwise, they expect the world and will just give up
> when
> > > we
> > > >> > fall
> > > >> > > > > > short). Vladimir did a list of what is in each of the
> phases
> > > >> above
> > > >> > > > which
> > > >> > > > > > would serve as a good start.
> > > >> > > > > > + Is this feature 'experimental' (Matteo asks above). I'd
> > > prefer
> > > >> it
> > > >> > > is
> > > >> > > > > > not. If it is, it should be labelled all over that it is
> > so. I
> > > >> see
> > > >> > > > > current
> > > >> > > > > > state called out as a '... technical preview feature'.
> Does
> > > this
> > > >> > mean
> > > >> > > > > > not-for-users?
> > > >> > > > > >
> > > >> > > > > > St.Ack
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > > On Mon, Sep 12, 2016 at 8:03 AM, Ted Yu <
> > yuzhihong@gmail.com>
> > > >> > wrote:
> > > >> > > > > >
> > > >> > > > > >> Sean:
> > > >> > > > > >> Do you have more comments ?
> > > >> > > > > >>
> > > >> > > > > >> Cheers
> > > >> > > > > >>
> > > >> > > > > >> On Fri, Sep 9, 2016 at 1:42 PM, Vladimir Rodionov <
> > > >> > > > > vladrodionov@gmail.com
> > > >> > > > > >> >
> > > >> > > > > >> wrote:
> > > >> > > > > >>
> > > >> > > > > >> > Sean,
> > > >> > > > > >> >
> > > >> > > > > >> > Backup/Restore can fail due to various reasons: network
> > > outage
> > > >> > > > > (cluster
> > > >> > > > > >> > wide), various time-outs in HBase and HDFS layer, M/R
> > > failure
> > > >> > due
> > > >> > > to
> > > >> > > > > >> "HDFS
> > > >> > > > > >> > exceeded quota", user error (manual deletion of data)
> and
> > > so
> > > >> on
> > > >> > so
> > > >> > > > on.
> > > >> > > > > >> That
> > > >> > > > > >> > is impossible to enumerate all possible types of
> failures
> > > in a
> > > >> > > > > >> distributed
> > > >> > > > > >> > system - that is not our goal/task.
> > > >> > > > > >> >
> > > >> > > > > >> > We focus completely on backup system table consistency
> > in a
> > > >> > > presence
> > > >> > > > > of
> > > >> > > > > >> any
> > > >> > > > > >> > type of failure. That is what I call "tolerance to
> > > failures".
> > > >> > > > > >> >
> > > >> > > > > >> > On a failure:
> > > >> > > > > >> >
> > > >> > > > > >> > BACKUP. All backup system information (prior to backup)
> > > will
> > > >> be
> > > >> > > > > restored
> > > >> > > > > >> > and all temporary data, related to a failed session, in
> > > HDFS
> > > >> > will
> > > >> > > be
> > > >> > > > > >> > deleted
> > > >> > > > > >> > RESTORE. We do not care about system data, because
> > restore
> > > >> does
> > > >> > > not
> > > >> > > > > >> change
> > > >> > > > > >> > it. Temporary data in HDFS will be cleaned up and table
> > > will
> > > >> be
> > > >> > > in a
> > > >> > > > > >> state
> > > >> > > > > >> > back to where it was before operation started.
> > > >> > > > > >> >
> > > >> > > > > >> > This is what user should expect in case of a failure.
> > > >> > > > > >> >
> > > >> > > > > >> > -Vlad
> > > >> > > > > >> >
> > > >> > > > > >> >
> > > >> > > > > >> > -Vlad
> > > >> > > > > >> >
> > > >> > > > > >> > On Fri, Sep 9, 2016 at 12:56 PM, Sean Busbey <
> > > >> busbey@apache.org
> > > >> > >
> > > >> > > > > wrote:
> > > >> > > > > >> >
> > > >> > > > > >> > > Failing in a consistent way, with docs that explain
> the
> > > >> > various
> > > >> > > > > >> > > expected failures would be sufficient.
> > > >> > > > > >> > >
> > > >> > > > > >> > > On Fri, Sep 9, 2016 at 12:16 PM, Vladimir Rodionov
> > > >> > > > > >> > > <vl...@gmail.com> wrote:
> > > >> > > > > >> > > > Do not worry Sean, doc is coming today as a preview
> > and
> > > >> our
> > > >> > > > writer
> > > >> > > > > >> > Frank
> > > >> > > > > >> > > > will be working on a putting  it into Apache repo.
> > > >> Timeline
> > > >> > > > > depends
> > > >> > > > > >> on
> > > >> > > > > >> > > > Franks schedule but I hope we will get it rather
> > sooner
> > > >> than
> > > >> > > > > later.
> > > >> > > > > >> > > >
> > > >> > > > > >> > > > As for failure testing, we are focusing only on a
> > > >> consistent
> > > >> > > > state
> > > >> > > > > >> of
> > > >> > > > > >> > > > backup system data in a presence of any type of
> > > failures,
> > > >> We
> > > >> > > are
> > > >> > > > > not
> > > >> > > > > >> > > going
> > > >> > > > > >> > > > to implement  anything more "fancy", than that. We
> > > allow
> > > >> > both:
> > > >> > > > > >> backup
> > > >> > > > > >> > and
> > > >> > > > > >> > > > restore to fail. What we do not allow is to have
> > system
> > > >> data
> > > >> > > > > >> corrupted.
> > > >> > > > > >> > > > Will it suffice for you? Do you have any other
> > > concerns,
> > > >> you
> > > >> > > > want
> > > >> > > > > >> us to
> > > >> > > > > >> > > > address?
> > > >> > > > > >> > > >
> > > >> > > > > >> > > > -Vlad
> > > >> > > > > >> > > >
> > > >> > > > > >> > > >
> > > >> > > > > >> > > > On Fri, Sep 9, 2016 at 10:56 AM, Sean Busbey <
> > > >> > > busbey@apache.org
> > > >> > > > >
> > > >> > > > > >> > wrote:
> > > >> > > > > >> > > >
> > > >> > > > > >> > > >> "docs will come to Apache soon" does not address
> my
> > > >> concern
> > > >> > > > > around
> > > >> > > > > >> > docs
> > > >> > > > > >> > > at
> > > >> > > > > >> > > >> all, unless said docs have already made it into
> the
> > > >> project
> > > >> > > > > repo. I
> > > >> > > > > >> > > don't
> > > >> > > > > >> > > >> want third party resources for using a major and
> > > >> important
> > > >> > > > > feature
> > > >> > > > > >> of
> > > >> > > > > >> > > the
> > > >> > > > > >> > > >> project, I want us to provide end users with what
> > they
> > > >> need
> > > >> > > to
> > > >> > > > > get
> > > >> > > > > >> the
> > > >> > > > > >> > > job
> > > >> > > > > >> > > >> done.
> > > >> > > > > >> > > >>
> > > >> > > > > >> > > >> I see some calls for patience on the failure
> > testing,
> > > but
> > > >> > the
> > > >> > > > > >> appeal
> > > >> > > > > >> > to
> > > >> > > > > >> > > us
> > > >> > > > > >> > > >> having done a bad job of requiring proper tests of
> > > >> previous
> > > >> > > > > >> features
> > > >> > > > > >> > > just
> > > >> > > > > >> > > >> makes me more concerned about not getting them
> > here. I
> > > >> > don't
> > > >> > > > want
> > > >> > > > > >> to
> > > >> > > > > >> > set
> > > >> > > > > >> > > >> yet another bad example that will then be pointed
> to
> > > in
> > > >> the
> > > >> > > > > future.
> > > >> > > > > >> > > >>
> > > >> > > > > >> > > >> On Sep 8, 2016 10:50, "Ted Yu" <
> yuzhihong@gmail.com
> > >
> > > >> > wrote:
> > > >> > > > > >> > > >>
> > > >> > > > > >> > > >> > Is there any concern which is not addressed ?
> > > >> > > > > >> > > >> >
> > > >> > > > > >> > > >> > Do we need another Vote thread ?
> > > >> > > > > >> > > >> >
> > > >> > > > > >> > > >> > Thanks
> > > >> > > > > >> > > >> >
> > > >> > > > > >> > > >> > On Thu, Sep 8, 2016 at 9:21 AM, Andrew Purtell <
> > > >> > > > > >> apurtell@apache.org
> > > >> > > > > >> > >
> > > >> > > > > >> > > >> > wrote:
> > > >> > > > > >> > > >> >
> > > >> > > > > >> > > >> > > Vlad,
> > > >> > > > > >> > > >> > >
> > > >> > > > > >> > > >> > > I apologize for using the term 'half-baked'
> in a
> > > way
> > > >> > that
> > > >> > > > > could
> > > >> > > > > >> > > seem a
> > > >> > > > > >> > > >> > > description of HBASE-7912. I meant that as a
> > > general
> > > >> > > > > >> hypothetical.
> > > >> > > > > >> > > >> > >
> > > >> > > > > >> > > >> > > On Wed, Sep 7, 2016 at 9:36 AM, Vladimir
> > Rodionov
> > > <
> > > >> > > > > >> > > >> > vladrodionov@gmail.com>
> > > >> > > > > >> > > >> > > wrote:
> > > >> > > > > >> > > >> > >
> > > >> > > > > >> > > >> > > > >> I'm not sure that "There is already lots
> of
> > > >> > > half-baked
> > > >> > > > > >> code
> > > >> > > > > >> > in
> > > >> > > > > >> > > the
> > > >> > > > > >> > > >> > > > branch,
> > > >> > > > > >> > > >> > > > so what's the harm in adding more?"
> > > >> > > > > >> > > >> > > >
> > > >> > > > > >> > > >> > > > I meant - not production - ready yet. This
> is
> > > 2.0
> > > >> > > > > development
> > > >> > > > > >> > > branch
> > > >> > > > > >> > > >> > and,
> > > >> > > > > >> > > >> > > > hence many features are in works,
> > > >> > > > > >> > > >> > > > not being tested well etc. I do not consider
> > > backup
> > > >> > as
> > > >> > > > half
> > > >> > > > > >> > baked
> > > >> > > > > >> > > >> > > feature -
> > > >> > > > > >> > > >> > > > it has passed our internal QA and has very
> > good
> > > >> doc,
> > > >> > > > which
> > > >> > > > > we
> > > >> > > > > >> > will
> > > >> > > > > >> > > >> > > provide
> > > >> > > > > >> > > >> > > > to Apache shortly.
> > > >> > > > > >> > > >> > > >
> > > >> > > > > >> > > >> > > > -Vlad
> > > >> > > > > >> > > >> > > >
> > > >> > > > > >> > > >> > > > On Wed, Sep 7, 2016 at 9:13 AM, Andrew
> > Purtell <
> > > >> > > > > >> > > apurtell@apache.org>
> > > >> > > > > >> > > >> > > > wrote:
> > > >> > > > > >> > > >> > > >
> > > >> > > > > >> > > >> > > > > We shouldn't admit half baked changes that
> > > won't
> > > >> be
> > > >> > > > > >> finished.
> > > >> > > > > >> > > >> However
> > > >> > > > > >> > > >> > > in
> > > >> > > > > >> > > >> > > > > this case the crew working on this feature
> > are
> > > >> long
> > > >> > > > > timers
> > > >> > > > > >> and
> > > >> > > > > >> > > less
> > > >> > > > > >> > > >> > > > likely
> > > >> > > > > >> > > >> > > > > than just about anyone to leave something
> > in a
> > > >> half
> > > >> > > > baked
> > > >> > > > > >> > > state. Of
> > > >> > > > > >> > > >> > > > course
> > > >> > > > > >> > > >> > > > > there is no guarantee how anything will
> turn
> > > out,
> > > >> > > but I
> > > >> > > > > am
> > > >> > > > > >> > > willing
> > > >> > > > > >> > > >> to
> > > >> > > > > >> > > >> > > > take
> > > >> > > > > >> > > >> > > > > a little on faith if they feel their best
> > path
> > > >> > > forward
> > > >> > > > > now
> > > >> > > > > >> is
> > > >> > > > > >> > to
> > > >> > > > > >> > > >> > merge
> > > >> > > > > >> > > >> > > to
> > > >> > > > > >> > > >> > > > > trunk. I only wish I had bandwidth to have
> > > done
> > > >> > some
> > > >> > > > real
> > > >> > > > > >> > > kicking
> > > >> > > > > >> > > >> of
> > > >> > > > > >> > > >> > > the
> > > >> > > > > >> > > >> > > > > tires by now. Maybe this week.
> > > >> > > > > >> > > >> > > > >
> > > >> > > > > >> > > >> > > > > (Yes, I'm using some of that time for this
> > > email
> > > >> > :-)
> > > >> > > > but
> > > >> > > > > I
> > > >> > > > > >> > type
> > > >> > > > > >> > > >> > fast.)
> > > >> > > > > >> > > >> > > > >
> > > >> > > > > >> > > >> > > > > That said, I would like to agitate for
> > making
> > > 2.0
> > > >> > > more
> > > >> > > > > real
> > > >> > > > > >> > and
> > > >> > > > > >> > > >> spend
> > > >> > > > > >> > > >> > > > some
> > > >> > > > > >> > > >> > > > > time on it now that I'm winding down with
> > > 0.98. I
> > > >> > > think
> > > >> > > > > >> that
> > > >> > > > > >> > > means
> > > >> > > > > >> > > >> > > > > branching for 2.0 real soon now and even
> > > evicting
> > > >> > > > things
> > > >> > > > > >> from
> > > >> > > > > >> > > 2.0
> > > >> > > > > >> > > >> > > branch
> > > >> > > > > >> > > >> > > > > that aren't finished or stable, leaving
> them
> > > only
> > > >> > > once
> > > >> > > > > >> again
> > > >> > > > > >> > in
> > > >> > > > > >> > > the
> > > >> > > > > >> > > >> > > > master
> > > >> > > > > >> > > >> > > > > branch. Or, maybe just evicting them.
> Let's
> > > take
> > > >> it
> > > >> > > > case
> > > >> > > > > by
> > > >> > > > > >> > > case.
> > > >> > > > > >> > > >> > > > >
> > > >> > > > > >> > > >> > > > > I think this feature can come in
> relatively
> > > >> safely.
> > > >> > > As
> > > >> > > > > >> added
> > > >> > > > > >> > > >> > insurance,
> > > >> > > > > >> > > >> > > > > let's admit the possibility it could be
> > > reverted
> > > >> on
> > > >> > > the
> > > >> > > > > 2.0
> > > >> > > > > >> > > branch
> > > >> > > > > >> > > >> if
> > > >> > > > > >> > > >> > > > folks
> > > >> > > > > >> > > >> > > > > working on stabilizing 2.0 decide to evict
> > it
> > > >> > because
> > > >> > > > it
> > > >> > > > > is
> > > >> > > > > >> > > >> > unfinished
> > > >> > > > > >> > > >> > > or
> > > >> > > > > >> > > >> > > > > unstable, because that certainly can
> > happen. I
> > > >> > would
> > > >> > > > > >> expect if
> > > >> > > > > >> > > talk
> > > >> > > > > >> > > >> > > like
> > > >> > > > > >> > > >> > > > > that starts, we'd get help finishing or
> > > >> stabilizing
> > > >> > > > > what's
> > > >> > > > > >> > under
> > > >> > > > > >> > > >> > > > discussion
> > > >> > > > > >> > > >> > > > > for revert. Or, we'd have a revert. Either
> > way
> > > >> the
> > > >> > > > > outcome
> > > >> > > > > >> is
> > > >> > > > > >> > > >> > > acceptable.
> > > >> > > > > >> > > >> > > > >
> > > >> > > > > >> > > >> > > > >
> > > >> > > > > >> > > >> > > > > On Wed, Sep 7, 2016 at 8:56 AM, Dima
> Spivak
> > <
> > > >> > > > > >> > > dimaspivak@apache.org
> > > >> > > > > >> > > >> >
> > > >> > > > > >> > > >> > > > wrote:
> > > >> > > > > >> > > >> > > > >
> > > >> > > > > >> > > >> > > > > > I'm not sure that "There is already lots
> > of
> > > >> > > > half-baked
> > > >> > > > > >> code
> > > >> > > > > >> > in
> > > >> > > > > >> > > >> the
> > > >> > > > > >> > > >> > > > > branch,
> > > >> > > > > >> > > >> > > > > > so what's the harm in adding more?" is a
> > > good
> > > >> > code
> > > >> > > > > commit
> > > >> > > > > >> > > >> > philosophy
> > > >> > > > > >> > > >> > > > for
> > > >> > > > > >> > > >> > > > > a
> > > >> > > > > >> > > >> > > > > > fault-tolerant distributed data store.
> ;)
> > > >> > > > > >> > > >> > > > > >
> > > >> > > > > >> > > >> > > > > > More seriously, a lack of test coverage
> > for
> > > >> > > existing
> > > >> > > > > >> > features
> > > >> > > > > >> > > >> > > shouldn't
> > > >> > > > > >> > > >> > > > > be
> > > >> > > > > >> > > >> > > > > > used as justification for introducing
> new
> > > >> > features
> > > >> > > > with
> > > >> > > > > >> the
> > > >> > > > > >> > > same
> > > >> > > > > >> > > >> > > > > > shortcomings. Ultimately, it's the end
> > user
> > > who
> > > >> > > will
> > > >> > > > > feel
> > > >> > > > > >> > the
> > > >> > > > > >> > > >> pain,
> > > >> > > > > >> > > >> > > so
> > > >> > > > > >> > > >> > > > > > shouldn't we do everything we can to
> > > mitigate
> > > >> > that?
> > > >> > > > > >> > > >> > > > > >
> > > >> > > > > >> > > >> > > > > > -Dima
> > > >> > > > > >> > > >> > > > > >
> > > >> > > > > >> > > >> > > > > > On Wed, Sep 7, 2016 at 8:46 AM, Vladimir
> > > >> > Rodionov <
> > > >> > > > > >> > > >> > > > > vladrodionov@gmail.com>
> > > >> > > > > >> > > >> > > > > > wrote:
> > > >> > > > > >> > > >> > > > > >
> > > >> > > > > >> > > >> > > > > > > Sean,
> > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > >> > > > > > > * have docs
> > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > >> > > > > > > Agree. We have a doc and backup is the
> > > most
> > > >> > > > > documented
> > > >> > > > > >> > > feature
> > > >> > > > > >> > > >> > :),
> > > >> > > > > >> > > >> > > we
> > > >> > > > > >> > > >> > > > > > will
> > > >> > > > > >> > > >> > > > > > > release it shortly to Apache.
> > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > >> > > > > > > * have sunny-day correctness tests
> > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > >> > > > > > > Feature has  close to 60 test cases,
> > which
> > > >> run
> > > >> > > for
> > > >> > > > > >> approx
> > > >> > > > > >> > 30
> > > >> > > > > >> > > >> min.
> > > >> > > > > >> > > >> > > We
> > > >> > > > > >> > > >> > > > > can
> > > >> > > > > >> > > >> > > > > > > add more, if community do not mind :)
> > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > >> > > > > > > * have correctness-in-face-of-failure
> > > tests
> > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > >> > > > > > > Any examples of these tests in
> existing
> > > >> > features?
> > > >> > > > In
> > > >> > > > > >> > works,
> > > >> > > > > >> > > we
> > > >> > > > > >> > > >> > > have a
> > > >> > > > > >> > > >> > > > > > clear
> > > >> > > > > >> > > >> > > > > > > understanding of what should be done
> by
> > > the
> > > >> > time
> > > >> > > of
> > > >> > > > > 2.0
> > > >> > > > > >> > > >> release.
> > > >> > > > > >> > > >> > > > > > > That is very close goal for us, to
> > verify
> > > IT
> > > >> > > monkey
> > > >> > > > > for
> > > >> > > > > >> > > >> existing
> > > >> > > > > >> > > >> > > > code.
> > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > >> > > > > > > * don't rely on things outside of
> HBase
> > > for
> > > >> > > normal
> > > >> > > > > >> > operation
> > > >> > > > > >> > > >> > (okay
> > > >> > > > > >> > > >> > > > for
> > > >> > > > > >> > > >> > > > > > > advanced operation)
> > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > >> > > > > > > We do not.
> > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > >> > > > > > > Enormous time has been spent already
> on
> > > the
> > > >> > > > > development
> > > >> > > > > >> > and
> > > >> > > > > >> > > >> > testing
> > > >> > > > > >> > > >> > > > the
> > > >> > > > > >> > > >> > > > > > > feature, it has passed our internal
> > tests
> > > and
> > > >> > > many
> > > >> > > > > >> rounds
> > > >> > > > > >> > of
> > > >> > > > > >> > > >> code
> > > >> > > > > >> > > >> > > > > reviews
> > > >> > > > > >> > > >> > > > > > > by HBase committers. We do not mind if
> > > >> someone
> > > >> > > from
> > > >> > > > > >> HBase
> > > >> > > > > >> > > >> > community
> > > >> > > > > >> > > >> > > > > > > (outside of HW) will review the code,
> > but
> > > it
> > > >> > will
> > > >> > > > > >> probably
> > > >> > > > > >> > > >> takes
> > > >> > > > > >> > > >> > > > > forever
> > > >> > > > > >> > > >> > > > > > to
> > > >> > > > > >> > > >> > > > > > > wait for volunteer?, the feature is
> > quite
> > > >> large
> > > >> > > > (1MB+
> > > >> > > > > >> > > >> cumulative
> > > >> > > > > >> > > >> > > > patch)
> > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > >> > > > > > > 2.0 branch is full of half baked
> > features,
> > > >> most
> > > >> > > of
> > > >> > > > > them
> > > >> > > > > >> > are
> > > >> > > > > >> > > in
> > > >> > > > > >> > > >> > > active
> > > >> > > > > >> > > >> > > > > > > development, therefore I am not
> > following
> > > you
> > > >> > > here,
> > > >> > > > > >> Sean?
> > > >> > > > > >> > > Why
> > > >> > > > > >> > > >> > > > > HBASE-7912
> > > >> > > > > >> > > >> > > > > > is
> > > >> > > > > >> > > >> > > > > > > not good enough yet to be integrated
> > into
> > > 2.0
> > > >> > > > branch?
> > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > >> > > > > > > -Vlad
> > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > >> > > > > > > On Wed, Sep 7, 2016 at 8:23 AM, Sean
> > > Busbey <
> > > >> > > > > >> > > busbey@apache.org
> > > >> > > > > >> > > >> >
> > > >> > > > > >> > > >> > > > wrote:
> > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > >> > > > > > > > On Tue, Sep 6, 2016 at 10:36 PM,
> Josh
> > > >> Elser <
> > > >> > > > > >> > > >> > > josh.elser@gmail.com>
> > > >> > > > > >> > > >> > > > > > > wrote:
> > > >> > > > > >> > > >> > > > > > > > > So, the answer to Sean's original
> > > >> question
> > > >> > is
> > > >> > > > "as
> > > >> > > > > >> > > robust as
> > > >> > > > > >> > > >> > > > > snapshots
> > > >> > > > > >> > > >> > > > > > > > > presently are"? (independence of
> > > >> > > backup/restore
> > > >> > > > > >> > failure
> > > >> > > > > >> > > >> > > tolerance
> > > >> > > > > >> > > >> > > > > > from
> > > >> > > > > >> > > >> > > > > > > > > snapshot failure tolerance)
> > > >> > > > > >> > > >> > > > > > > > >
> > > >> > > > > >> > > >> > > > > > > > > Is this just a question WRT
> context
> > of
> > > >> the
> > > >> > > > > change,
> > > >> > > > > >> or
> > > >> > > > > >> > > is it
> > > >> > > > > >> > > >> > > means
> > > >> > > > > >> > > >> > > > > > for a
> > > >> > > > > >> > > >> > > > > > > > veto
> > > >> > > > > >> > > >> > > > > > > > > from you, Sean? Just trying to
> make
> > > sure
> > > >> > I'm
> > > >> > > > > >> following
> > > >> > > > > >> > > >> along
> > > >> > > > > >> > > >> > > > > > > adequately.
> > > >> > > > > >> > > >> > > > > > > > >
> > > >> > > > > >> > > >> > > > > > > > >
> > > >> > > > > >> > > >> > > > > > > >
> > > >> > > > > >> > > >> > > > > > > > I'd say ATM I'm -0, bordering on -1
> > but
> > > not
> > > >> > for
> > > >> > > > > >> reasons
> > > >> > > > > >> > I
> > > >> > > > > >> > > can
> > > >> > > > > >> > > >> > > > > > articulate
> > > >> > > > > >> > > >> > > > > > > > well.
> > > >> > > > > >> > > >> > > > > > > >
> > > >> > > > > >> > > >> > > > > > > > Here's an attempt.
> > > >> > > > > >> > > >> > > > > > > >
> > > >> > > > > >> > > >> > > > > > > > We've been trying to move, as a
> > > community,
> > > >> > > > towards
> > > >> > > > > >> > > minimizing
> > > >> > > > > >> > > >> > > risk
> > > >> > > > > >> > > >> > > > to
> > > >> > > > > >> > > >> > > > > > > > downstream folks by getting
> "complete
> > > >> enough
> > > >> > > for
> > > >> > > > > use"
> > > >> > > > > >> > > gates
> > > >> > > > > >> > > >> in
> > > >> > > > > >> > > >> > > > place
> > > >> > > > > >> > > >> > > > > > > > before we introduce new features.
> This
> > > was
> > > >> > > > spurred
> > > >> > > > > >> by a
> > > >> > > > > >> > > some
> > > >> > > > > >> > > >> > > > features
> > > >> > > > > >> > > >> > > > > > > > getting in half-baked and never
> making
> > > it
> > > >> to
> > > >> > > "can
> > > >> > > > > >> really
> > > >> > > > > >> > > use"
> > > >> > > > > >> > > >> > > > status
> > > >> > > > > >> > > >> > > > > > > > (I'm thinking of distributed log
> > replay
> > > and
> > > >> > the
> > > >> > > > > >> zk-less
> > > >> > > > > >> > > >> > > assignment
> > > >> > > > > >> > > >> > > > > > > > stuff, I don't recall if there was
> > > more).
> > > >> > > > > >> > > >> > > > > > > >
> > > >> > > > > >> > > >> > > > > > > > The gates, generally, included
> things
> > > like:
> > > >> > > > > >> > > >> > > > > > > >
> > > >> > > > > >> > > >> > > > > > > > * have docs
> > > >> > > > > >> > > >> > > > > > > > * have sunny-day correctness tests
> > > >> > > > > >> > > >> > > > > > > > * have
> correctness-in-face-of-failure
> > > tests
> > > >> > > > > >> > > >> > > > > > > > * don't rely on things outside of
> > HBase
> > > for
> > > >> > > > normal
> > > >> > > > > >> > > operation
> > > >> > > > > >> > > >> > > (okay
> > > >> > > > > >> > > >> > > > > for
> > > >> > > > > >> > > >> > > > > > > > advanced operation)
> > > >> > > > > >> > > >> > > > > > > >
> > > >> > > > > >> > > >> > > > > > > > As an example, we kept the MOB work
> > off
> > > in
> > > >> a
> > > >> > > > branch
> > > >> > > > > >> and
> > > >> > > > > >> > > out
> > > >> > > > > >> > > >> of
> > > >> > > > > >> > > >> > > > master
> > > >> > > > > >> > > >> > > > > > > > until it could pass these criteria.
> > The
> > > big
> > > >> > > > > exemption
> > > >> > > > > >> > > we've
> > > >> > > > > >> > > >> had
> > > >> > > > > >> > > >> > > to
> > > >> > > > > >> > > >> > > > > > > > this was the hbase-spark
> integration,
> > > where
> > > >> > we
> > > >> > > > all
> > > >> > > > > >> > agreed
> > > >> > > > > >> > > it
> > > >> > > > > >> > > >> > > could
> > > >> > > > > >> > > >> > > > > > > > land in master because it was very
> > well
> > > >> > > isolated
> > > >> > > > > (the
> > > >> > > > > >> > > slide
> > > >> > > > > >> > > >> > away
> > > >> > > > > >> > > >> > > > from
> > > >> > > > > >> > > >> > > > > > > > including docs as a first-class part
> > of
> > > >> > > building
> > > >> > > > up
> > > >> > > > > >> that
> > > >> > > > > >> > > >> > > > integration
> > > >> > > > > >> > > >> > > > > > > > has led me to doubt the wisdom of
> this
> > > >> > > decision).
> > > >> > > > > >> > > >> > > > > > > >
> > > >> > > > > >> > > >> > > > > > > > We've also been treating inclusion
> in
> > a
> > > >> > > "probably
> > > >> > > > > >> will
> > > >> > > > > >> > be
> > > >> > > > > >> > > >> > > released
> > > >> > > > > >> > > >> > > > to
> > > >> > > > > >> > > >> > > > > > > > downstream" branches as a higher
> bar,
> > > >> > requiring
> > > >> > > > > >> > > >> > > > > > > >
> > > >> > > > > >> > > >> > > > > > > > * don't moderately impact
> performance
> > > when
> > > >> > the
> > > >> > > > > >> feature
> > > >> > > > > >> > > isn't
> > > >> > > > > >> > > >> in
> > > >> > > > > >> > > >> > > use
> > > >> > > > > >> > > >> > > > > > > > * don't severely impact performance
> > when
> > > >> the
> > > >> > > > > feature
> > > >> > > > > >> is
> > > >> > > > > >> > in
> > > >> > > > > >> > > >> use
> > > >> > > > > >> > > >> > > > > > > > * either default-to-on or show
> enough
> > > >> demand
> > > >> > to
> > > >> > > > > >> believe
> > > >> > > > > >> > a
> > > >> > > > > >> > > >> > > > non-trivial
> > > >> > > > > >> > > >> > > > > > > > number of folks will turn the
> feature
> > on
> > > >> > > > > >> > > >> > > > > > > >
> > > >> > > > > >> > > >> > > > > > > > The above has kept MOB and
> hbase-spark
> > > >> > > > integration
> > > >> > > > > >> out
> > > >> > > > > >> > of
> > > >> > > > > >> > > >> > > branch-1,
> > > >> > > > > >> > > >> > > > > > > > presumably while they've "gotten
> more
> > > >> stable"
> > > >> > > in
> > > >> > > > > >> master
> > > >> > > > > >> > > from
> > > >> > > > > >> > > >> > the
> > > >> > > > > >> > > >> > > > odd
> > > >> > > > > >> > > >> > > > > > > > vendor inclusion.
> > > >> > > > > >> > > >> > > > > > > >
> > > >> > > > > >> > > >> > > > > > > > Are we going to have a 2.0 release
> > > before
> > > >> the
> > > >> > > end
> > > >> > > > > of
> > > >> > > > > >> the
> > > >> > > > > >> > > >> year?
> > > >> > > > > >> > > >> > > > We're
> > > >> > > > > >> > > >> > > > > > > > coming up on 1.5 years since the
> > > release of
> > > >> > > > version
> > > >> > > > > >> 1.0;
> > > >> > > > > >> > > >> seems
> > > >> > > > > >> > > >> > > like
> > > >> > > > > >> > > >> > > > > > > > it's about time, though I haven't
> seen
> > > any
> > > >> > > > concrete
> > > >> > > > > >> > plans
> > > >> > > > > >> > > >> this
> > > >> > > > > >> > > >> > > > year.
> > > >> > > > > >> > > >> > > > > > > > Presuming we are going to have one
> by
> > > the
> > > >> end
> > > >> > > of
> > > >> > > > > the
> > > >> > > > > >> > > year, it
> > > >> > > > > >> > > >> > > > seems a
> > > >> > > > > >> > > >> > > > > > > > bit close to still be adding in
> > > "features
> > > >> > that
> > > >> > > > need
> > > >> > > > > >> > > maturing"
> > > >> > > > > >> > > >> > on
> > > >> > > > > >> > > >> > > > the
> > > >> > > > > >> > > >> > > > > > > > branch.
> > > >> > > > > >> > > >> > > > > > > >
> > > >> > > > > >> > > >> > > > > > > > The lack of a concrete plan for 2.0
> > > keeps
> > > >> me
> > > >> > > from
> > > >> > > > > >> > > considering
> > > >> > > > > >> > > >> > > these
> > > >> > > > > >> > > >> > > > > > > > things blocker at the moment. But I
> > know
> > > >> > first
> > > >> > > > hand
> > > >> > > > > >> how
> > > >> > > > > >> > > much
> > > >> > > > > >> > > >> > > > trouble
> > > >> > > > > >> > > >> > > > > > > > folks have had with other features
> > that
> > > >> have
> > > >> > > gone
> > > >> > > > > >> into
> > > >> > > > > >> > > >> > downstream
> > > >> > > > > >> > > >> > > > > > > > facing releases without robustness
> > > checks
> > > >> > (i.e.
> > > >> > > > > >> > > replication),
> > > >> > > > > >> > > >> > and
> > > >> > > > > >> > > >> > > > I'm
> > > >> > > > > >> > > >> > > > > > > > concerned about what we're setting
> up
> > if
> > > >> 2.0
> > > >> > > goes
> > > >> > > > > out
> > > >> > > > > >> > with
> > > >> > > > > >> > > >> this
> > > >> > > > > >> > > >> > > > > > > > feature in its current state.
> > > >> > > > > >> > > >> > > > > > > >
> > > >> > > > > >> > > >> > > > > > >
> > > >> > > > > >> > > >> > > > > >
> > > >> > > > > >> > > >> > > > >
> > > >> > > > > >> > > >> > > > >
> > > >> > > > > >> > > >> > > > >
> > > >> > > > > >> > > >> > > > > --
> > > >> > > > > >> > > >> > > > > Best regards,
> > > >> > > > > >> > > >> > > > >
> > > >> > > > > >> > > >> > > > >    - Andy
> > > >> > > > > >> > > >> > > > >
> > > >> > > > > >> > > >> > > > > Problems worthy of attack prove their
> worth
> > by
> > > >> > > hitting
> > > >> > > > > >> back. -
> > > >> > > > > >> > > Piet
> > > >> > > > > >> > > >> > > Hein
> > > >> > > > > >> > > >> > > > > (via Tom White)
> > > >> > > > > >> > > >> > > > >
> > > >> > > > > >> > > >> > > >
> > > >> > > > > >> > > >> > >
> > > >> > > > > >> > > >> > >
> > > >> > > > > >> > > >> > >
> > > >> > > > > >> > > >> > > --
> > > >> > > > > >> > > >> > > Best regards,
> > > >> > > > > >> > > >> > >
> > > >> > > > > >> > > >> > >    - Andy
> > > >> > > > > >> > > >> > >
> > > >> > > > > >> > > >> > > Problems worthy of attack prove their worth by
> > > >> hitting
> > > >> > > > back.
> > > >> > > > > -
> > > >> > > > > >> > Piet
> > > >> > > > > >> > > >> Hein
> > > >> > > > > >> > > >> > > (via Tom White)
> > > >> > > > > >> > > >> > >
> > > >> > > > > >> > > >> >
> > > >> > > > > >> > > >>
> > > >> > > > > >> > >
> > > >> > > > > >> >
> > > >> > > > > >>
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> > >
> > >
> > > --
> > > busbey
> > >
> >
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Vladimir Rodionov <vl...@gmail.com>]

>> mapreduce dependency has been moved to client side - no mapreduce job

1. We have no code in the client module anymore, due to dependency on
internal server API (HFile and WAL access).
2. Backup/ restore are client - driven operations, but all the code resides
in the server module
3. No MR in Master, no procedure - driven execution.
4. Old good MR from command-line.
5. Security was simplified and now only super-user is allowed to run
backup/restores.
6. HBase Backup API was gone due to 1. Now only command-line access to
backup tools.

These consequences of refactoring has been discussed in HBASE-16727.

-Vlad




On Mon, Oct 10, 2016 at 1:31 PM, Ted Yu <yu...@gmail.com> wrote:

> Reviving this thread.
>
> The following has taken place:
>
> mapreduce dependency has been moved to client side - no mapreduce job
> launched from master or region server.
> document patch (HBASE-16574) has been integrated.
> Updated mega patch has been attached to HBASE-14123: this covers the
> refactor in #1 above and the protobuf 3 merge.
>
> If community has more feedback on the merge proposal, I would love to hear
> it.
>
> Thanks
>
> On Thu, Sep 22, 2016 at 10:31 AM, Sean Busbey <bu...@cloudera.com> wrote:
>
> > I'd like to see the docs proposed on HBASE-16574 integrated into our
> > project's documentation prior to merge.
> >
> > On Thu, Sep 22, 2016 at 9:02 AM, Ted Yu <yu...@gmail.com> wrote:
> > > This feature can be marked experimental due to some limitations such as
> > > security.
> > >
> > > Your previous round of comments have been addressed.
> > > Command line tool has gone through:
> > >
> > > HBASE-16620 Fix backup command-line tool usability issues
> > > HBASE-16655 hbase backup describe with incorrect backup id results in
> NPE
> > >
> > > The updated doc has been attached to HBASE-16574.
> > >
> > > Cheers
> > >
> > > On Thu, Sep 22, 2016 at 8:53 AM, Stack <st...@duboce.net> wrote:
> > >
> > >> On Wed, Sep 21, 2016 at 7:43 AM, Ted Yu <yu...@gmail.com> wrote:
> > >>
> > >> > Are there more (review) comments ?
> > >> >
> > >> >
> > >> Are outstanding comments addressed?
> > >>
> > >> I don't see answer to my 'is this experimental/will it be marked
> > >> experimental' question.
> > >>
> > >> I ran into some issues trying to use the feature and suggested that a
> > >> feature likes this needs polish else it'll just rot, unused. Has
> polish
> > >> been applied? All ready for another 'user' test? Suggest that you
> update
> > >> here going forward for the benefit of those trying to follow along and
> > who
> > >> are not watching JIRA change fly-by.
> > >>
> > >> It looks like doc got a revision -- I have to check -- to take on
> > >> suggestion made above but again, suggest, that this thread gets
> updated.
> > >>
> > >> Thanks,
> > >> St.Ack
> > >>
> > >>
> > >>
> > >> > Thanks
> > >> >
> > >> > On Tue, Sep 20, 2016 at 10:02 AM, Devaraj Das <ddas@hortonworks.com
> >
> > >> > wrote:
> > >> >
> > >> > > Just reviving this thread. Thanks Sean, Stack, Dima, and others
> for
> > the
> > >> > > thorough reviews and testing. Thanks Ted and Vlad for taking care
> of
> > >> the
> > >> > > feedback. Are we all good to do the merge now? Rather do sooner
> than
> > >> > later.
> > >> > > ________________________________________
> > >> > > From: saint.ack@gmail.com <sa...@gmail.com> on behalf of
> Stack
> > <
> > >> > > stack@duboce.net>
> > >> > > Sent: Monday, September 12, 2016 1:18 PM
> > >> > > To: HBase Dev List
> > >> > > Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch
> HBASE-7912
> > >> > >
> > >> > > On Mon, Sep 12, 2016 at 12:19 PM, Ted Yu <yu...@gmail.com>
> > wrote:
> > >> > >
> > >> > > > Mega patch (rev 18) is on HBASE-14123.
> > >> > > >
> > >> > > > Please comment on HBASE-14123 on how you want to review.
> > >> > > >
> > >> > >
> > >> > >
> > >> > > Yeah. That was my lost tab. Last rb was 6 months ago. Suggest
> > updating
> > >> > it.
> > >> > > RB is pretty good for review. Patch is only 1.5M so should be
> fine.
> > >> > >
> > >> > > St.Ack
> > >> > >
> > >> > >
> > >> > > >
> > >> > > > Thanks
> > >> > > >
> > >> > > > On Mon, Sep 12, 2016 at 12:15 PM, Stack <st...@duboce.net>
> wrote:
> > >> > > >
> > >> > > > > On review of the 'patch', do I just compare the branch to
> > master or
> > >> > is
> > >> > > > > there a megapatch posted somewhere (I think I saw one but it
> > seemed
> > >> > > stale
> > >> > > > > and then I 'lost' the tab). Sorry for dumb question.
> > >> > > > > St.Ack
> > >> > > > >
> > >> > > > > On Mon, Sep 12, 2016 at 12:01 PM, Stack <st...@duboce.net>
> > wrote:
> > >> > > > >
> > >> > > > > > Late to the game. A few comments after rereading this thread
> > as a
> > >> > > > 'user'.
> > >> > > > > >
> > >> > > > > > + Before merge, a user-facing feature like this should work
> > (If
> > >> > this
> > >> > > is
> > >> > > > > "higher-bar
> > >> > > > > > for new features", bring it on -- smile).
> > >> > > > > > + As a user, I tried the branch with tools after reviewing
> the
> > >> > > > > just-posted
> > >> > > > > > doc. I had an 'interesting' experience (left comments up on
> > >> > issue). I
> > >> > > > > think
> > >> > > > > > the tooling/doc. important to get right. If it breaks easily
> > or
> > >> is
> > >> > > > > > inconsistent (or lacks 'polish'), operators will judge the
> > whole
> > >> > > > > > backup/restore tooling chain as not trustworthy and abandon
> > it.
> > >> > Lets
> > >> > > > not
> > >> > > > > > have this happen to this feature.
> > >> > > > > > + Matteo's suggestion (with a helpful starter list) that
> there
> > >> > needs
> > >> > > to
> > >> > > > > be
> > >> > > > > > explicit qualification on what is actually being delivered
> --
> > >> > > > including a
> > >> > > > > > listing of limitations (some look serious such as data bleed
> > from
> > >> > > other
> > >> > > > > > regions in WALs, but maybe I don't care for my use case...)
> --
> > >> > needs
> > >> > > to
> > >> > > > > > accompany the merge. Lets fold them into the user doc. in
> the
> > >> > > technical
> > >> > > > > > overview area as suggested so user expectations are properly
> > >> > managed
> > >> > > > > > (otherwise, they expect the world and will just give up when
> > we
> > >> > fall
> > >> > > > > > short). Vladimir did a list of what is in each of the phases
> > >> above
> > >> > > > which
> > >> > > > > > would serve as a good start.
> > >> > > > > > + Is this feature 'experimental' (Matteo asks above). I'd
> > prefer
> > >> it
> > >> > > is
> > >> > > > > > not. If it is, it should be labelled all over that it is
> so. I
> > >> see
> > >> > > > > current
> > >> > > > > > state called out as a '... technical preview feature'. Does
> > this
> > >> > mean
> > >> > > > > > not-for-users?
> > >> > > > > >
> > >> > > > > > St.Ack
> > >> > > > > >
> > >> > > > > >
> > >> > > > > >
> > >> > > > > >
> > >> > > > > >
> > >> > > > > >
> > >> > > > > >
> > >> > > > > >
> > >> > > > > >
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > On Mon, Sep 12, 2016 at 8:03 AM, Ted Yu <
> yuzhihong@gmail.com>
> > >> > wrote:
> > >> > > > > >
> > >> > > > > >> Sean:
> > >> > > > > >> Do you have more comments ?
> > >> > > > > >>
> > >> > > > > >> Cheers
> > >> > > > > >>
> > >> > > > > >> On Fri, Sep 9, 2016 at 1:42 PM, Vladimir Rodionov <
> > >> > > > > vladrodionov@gmail.com
> > >> > > > > >> >
> > >> > > > > >> wrote:
> > >> > > > > >>
> > >> > > > > >> > Sean,
> > >> > > > > >> >
> > >> > > > > >> > Backup/Restore can fail due to various reasons: network
> > outage
> > >> > > > > (cluster
> > >> > > > > >> > wide), various time-outs in HBase and HDFS layer, M/R
> > failure
> > >> > due
> > >> > > to
> > >> > > > > >> "HDFS
> > >> > > > > >> > exceeded quota", user error (manual deletion of data) and
> > so
> > >> on
> > >> > so
> > >> > > > on.
> > >> > > > > >> That
> > >> > > > > >> > is impossible to enumerate all possible types of failures
> > in a
> > >> > > > > >> distributed
> > >> > > > > >> > system - that is not our goal/task.
> > >> > > > > >> >
> > >> > > > > >> > We focus completely on backup system table consistency
> in a
> > >> > > presence
> > >> > > > > of
> > >> > > > > >> any
> > >> > > > > >> > type of failure. That is what I call "tolerance to
> > failures".
> > >> > > > > >> >
> > >> > > > > >> > On a failure:
> > >> > > > > >> >
> > >> > > > > >> > BACKUP. All backup system information (prior to backup)
> > will
> > >> be
> > >> > > > > restored
> > >> > > > > >> > and all temporary data, related to a failed session, in
> > HDFS
> > >> > will
> > >> > > be
> > >> > > > > >> > deleted
> > >> > > > > >> > RESTORE. We do not care about system data, because
> restore
> > >> does
> > >> > > not
> > >> > > > > >> change
> > >> > > > > >> > it. Temporary data in HDFS will be cleaned up and table
> > will
> > >> be
> > >> > > in a
> > >> > > > > >> state
> > >> > > > > >> > back to where it was before operation started.
> > >> > > > > >> >
> > >> > > > > >> > This is what user should expect in case of a failure.
> > >> > > > > >> >
> > >> > > > > >> > -Vlad
> > >> > > > > >> >
> > >> > > > > >> >
> > >> > > > > >> > -Vlad
> > >> > > > > >> >
> > >> > > > > >> > On Fri, Sep 9, 2016 at 12:56 PM, Sean Busbey <
> > >> busbey@apache.org
> > >> > >
> > >> > > > > wrote:
> > >> > > > > >> >
> > >> > > > > >> > > Failing in a consistent way, with docs that explain the
> > >> > various
> > >> > > > > >> > > expected failures would be sufficient.
> > >> > > > > >> > >
> > >> > > > > >> > > On Fri, Sep 9, 2016 at 12:16 PM, Vladimir Rodionov
> > >> > > > > >> > > <vl...@gmail.com> wrote:
> > >> > > > > >> > > > Do not worry Sean, doc is coming today as a preview
> and
> > >> our
> > >> > > > writer
> > >> > > > > >> > Frank
> > >> > > > > >> > > > will be working on a putting  it into Apache repo.
> > >> Timeline
> > >> > > > > depends
> > >> > > > > >> on
> > >> > > > > >> > > > Franks schedule but I hope we will get it rather
> sooner
> > >> than
> > >> > > > > later.
> > >> > > > > >> > > >
> > >> > > > > >> > > > As for failure testing, we are focusing only on a
> > >> consistent
> > >> > > > state
> > >> > > > > >> of
> > >> > > > > >> > > > backup system data in a presence of any type of
> > failures,
> > >> We
> > >> > > are
> > >> > > > > not
> > >> > > > > >> > > going
> > >> > > > > >> > > > to implement  anything more "fancy", than that. We
> > allow
> > >> > both:
> > >> > > > > >> backup
> > >> > > > > >> > and
> > >> > > > > >> > > > restore to fail. What we do not allow is to have
> system
> > >> data
> > >> > > > > >> corrupted.
> > >> > > > > >> > > > Will it suffice for you? Do you have any other
> > concerns,
> > >> you
> > >> > > > want
> > >> > > > > >> us to
> > >> > > > > >> > > > address?
> > >> > > > > >> > > >
> > >> > > > > >> > > > -Vlad
> > >> > > > > >> > > >
> > >> > > > > >> > > >
> > >> > > > > >> > > > On Fri, Sep 9, 2016 at 10:56 AM, Sean Busbey <
> > >> > > busbey@apache.org
> > >> > > > >
> > >> > > > > >> > wrote:
> > >> > > > > >> > > >
> > >> > > > > >> > > >> "docs will come to Apache soon" does not address my
> > >> concern
> > >> > > > > around
> > >> > > > > >> > docs
> > >> > > > > >> > > at
> > >> > > > > >> > > >> all, unless said docs have already made it into the
> > >> project
> > >> > > > > repo. I
> > >> > > > > >> > > don't
> > >> > > > > >> > > >> want third party resources for using a major and
> > >> important
> > >> > > > > feature
> > >> > > > > >> of
> > >> > > > > >> > > the
> > >> > > > > >> > > >> project, I want us to provide end users with what
> they
> > >> need
> > >> > > to
> > >> > > > > get
> > >> > > > > >> the
> > >> > > > > >> > > job
> > >> > > > > >> > > >> done.
> > >> > > > > >> > > >>
> > >> > > > > >> > > >> I see some calls for patience on the failure
> testing,
> > but
> > >> > the
> > >> > > > > >> appeal
> > >> > > > > >> > to
> > >> > > > > >> > > us
> > >> > > > > >> > > >> having done a bad job of requiring proper tests of
> > >> previous
> > >> > > > > >> features
> > >> > > > > >> > > just
> > >> > > > > >> > > >> makes me more concerned about not getting them
> here. I
> > >> > don't
> > >> > > > want
> > >> > > > > >> to
> > >> > > > > >> > set
> > >> > > > > >> > > >> yet another bad example that will then be pointed to
> > in
> > >> the
> > >> > > > > future.
> > >> > > > > >> > > >>
> > >> > > > > >> > > >> On Sep 8, 2016 10:50, "Ted Yu" <yuzhihong@gmail.com
> >
> > >> > wrote:
> > >> > > > > >> > > >>
> > >> > > > > >> > > >> > Is there any concern which is not addressed ?
> > >> > > > > >> > > >> >
> > >> > > > > >> > > >> > Do we need another Vote thread ?
> > >> > > > > >> > > >> >
> > >> > > > > >> > > >> > Thanks
> > >> > > > > >> > > >> >
> > >> > > > > >> > > >> > On Thu, Sep 8, 2016 at 9:21 AM, Andrew Purtell <
> > >> > > > > >> apurtell@apache.org
> > >> > > > > >> > >
> > >> > > > > >> > > >> > wrote:
> > >> > > > > >> > > >> >
> > >> > > > > >> > > >> > > Vlad,
> > >> > > > > >> > > >> > >
> > >> > > > > >> > > >> > > I apologize for using the term 'half-baked' in a
> > way
> > >> > that
> > >> > > > > could
> > >> > > > > >> > > seem a
> > >> > > > > >> > > >> > > description of HBASE-7912. I meant that as a
> > general
> > >> > > > > >> hypothetical.
> > >> > > > > >> > > >> > >
> > >> > > > > >> > > >> > > On Wed, Sep 7, 2016 at 9:36 AM, Vladimir
> Rodionov
> > <
> > >> > > > > >> > > >> > vladrodionov@gmail.com>
> > >> > > > > >> > > >> > > wrote:
> > >> > > > > >> > > >> > >
> > >> > > > > >> > > >> > > > >> I'm not sure that "There is already lots of
> > >> > > half-baked
> > >> > > > > >> code
> > >> > > > > >> > in
> > >> > > > > >> > > the
> > >> > > > > >> > > >> > > > branch,
> > >> > > > > >> > > >> > > > so what's the harm in adding more?"
> > >> > > > > >> > > >> > > >
> > >> > > > > >> > > >> > > > I meant - not production - ready yet. This is
> > 2.0
> > >> > > > > development
> > >> > > > > >> > > branch
> > >> > > > > >> > > >> > and,
> > >> > > > > >> > > >> > > > hence many features are in works,
> > >> > > > > >> > > >> > > > not being tested well etc. I do not consider
> > backup
> > >> > as
> > >> > > > half
> > >> > > > > >> > baked
> > >> > > > > >> > > >> > > feature -
> > >> > > > > >> > > >> > > > it has passed our internal QA and has very
> good
> > >> doc,
> > >> > > > which
> > >> > > > > we
> > >> > > > > >> > will
> > >> > > > > >> > > >> > > provide
> > >> > > > > >> > > >> > > > to Apache shortly.
> > >> > > > > >> > > >> > > >
> > >> > > > > >> > > >> > > > -Vlad
> > >> > > > > >> > > >> > > >
> > >> > > > > >> > > >> > > > On Wed, Sep 7, 2016 at 9:13 AM, Andrew
> Purtell <
> > >> > > > > >> > > apurtell@apache.org>
> > >> > > > > >> > > >> > > > wrote:
> > >> > > > > >> > > >> > > >
> > >> > > > > >> > > >> > > > > We shouldn't admit half baked changes that
> > won't
> > >> be
> > >> > > > > >> finished.
> > >> > > > > >> > > >> However
> > >> > > > > >> > > >> > > in
> > >> > > > > >> > > >> > > > > this case the crew working on this feature
> are
> > >> long
> > >> > > > > timers
> > >> > > > > >> and
> > >> > > > > >> > > less
> > >> > > > > >> > > >> > > > likely
> > >> > > > > >> > > >> > > > > than just about anyone to leave something
> in a
> > >> half
> > >> > > > baked
> > >> > > > > >> > > state. Of
> > >> > > > > >> > > >> > > > course
> > >> > > > > >> > > >> > > > > there is no guarantee how anything will turn
> > out,
> > >> > > but I
> > >> > > > > am
> > >> > > > > >> > > willing
> > >> > > > > >> > > >> to
> > >> > > > > >> > > >> > > > take
> > >> > > > > >> > > >> > > > > a little on faith if they feel their best
> path
> > >> > > forward
> > >> > > > > now
> > >> > > > > >> is
> > >> > > > > >> > to
> > >> > > > > >> > > >> > merge
> > >> > > > > >> > > >> > > to
> > >> > > > > >> > > >> > > > > trunk. I only wish I had bandwidth to have
> > done
> > >> > some
> > >> > > > real
> > >> > > > > >> > > kicking
> > >> > > > > >> > > >> of
> > >> > > > > >> > > >> > > the
> > >> > > > > >> > > >> > > > > tires by now. Maybe this week.
> > >> > > > > >> > > >> > > > >
> > >> > > > > >> > > >> > > > > (Yes, I'm using some of that time for this
> > email
> > >> > :-)
> > >> > > > but
> > >> > > > > I
> > >> > > > > >> > type
> > >> > > > > >> > > >> > fast.)
> > >> > > > > >> > > >> > > > >
> > >> > > > > >> > > >> > > > > That said, I would like to agitate for
> making
> > 2.0
> > >> > > more
> > >> > > > > real
> > >> > > > > >> > and
> > >> > > > > >> > > >> spend
> > >> > > > > >> > > >> > > > some
> > >> > > > > >> > > >> > > > > time on it now that I'm winding down with
> > 0.98. I
> > >> > > think
> > >> > > > > >> that
> > >> > > > > >> > > means
> > >> > > > > >> > > >> > > > > branching for 2.0 real soon now and even
> > evicting
> > >> > > > things
> > >> > > > > >> from
> > >> > > > > >> > > 2.0
> > >> > > > > >> > > >> > > branch
> > >> > > > > >> > > >> > > > > that aren't finished or stable, leaving them
> > only
> > >> > > once
> > >> > > > > >> again
> > >> > > > > >> > in
> > >> > > > > >> > > the
> > >> > > > > >> > > >> > > > master
> > >> > > > > >> > > >> > > > > branch. Or, maybe just evicting them. Let's
> > take
> > >> it
> > >> > > > case
> > >> > > > > by
> > >> > > > > >> > > case.
> > >> > > > > >> > > >> > > > >
> > >> > > > > >> > > >> > > > > I think this feature can come in relatively
> > >> safely.
> > >> > > As
> > >> > > > > >> added
> > >> > > > > >> > > >> > insurance,
> > >> > > > > >> > > >> > > > > let's admit the possibility it could be
> > reverted
> > >> on
> > >> > > the
> > >> > > > > 2.0
> > >> > > > > >> > > branch
> > >> > > > > >> > > >> if
> > >> > > > > >> > > >> > > > folks
> > >> > > > > >> > > >> > > > > working on stabilizing 2.0 decide to evict
> it
> > >> > because
> > >> > > > it
> > >> > > > > is
> > >> > > > > >> > > >> > unfinished
> > >> > > > > >> > > >> > > or
> > >> > > > > >> > > >> > > > > unstable, because that certainly can
> happen. I
> > >> > would
> > >> > > > > >> expect if
> > >> > > > > >> > > talk
> > >> > > > > >> > > >> > > like
> > >> > > > > >> > > >> > > > > that starts, we'd get help finishing or
> > >> stabilizing
> > >> > > > > what's
> > >> > > > > >> > under
> > >> > > > > >> > > >> > > > discussion
> > >> > > > > >> > > >> > > > > for revert. Or, we'd have a revert. Either
> way
> > >> the
> > >> > > > > outcome
> > >> > > > > >> is
> > >> > > > > >> > > >> > > acceptable.
> > >> > > > > >> > > >> > > > >
> > >> > > > > >> > > >> > > > >
> > >> > > > > >> > > >> > > > > On Wed, Sep 7, 2016 at 8:56 AM, Dima Spivak
> <
> > >> > > > > >> > > dimaspivak@apache.org
> > >> > > > > >> > > >> >
> > >> > > > > >> > > >> > > > wrote:
> > >> > > > > >> > > >> > > > >
> > >> > > > > >> > > >> > > > > > I'm not sure that "There is already lots
> of
> > >> > > > half-baked
> > >> > > > > >> code
> > >> > > > > >> > in
> > >> > > > > >> > > >> the
> > >> > > > > >> > > >> > > > > branch,
> > >> > > > > >> > > >> > > > > > so what's the harm in adding more?" is a
> > good
> > >> > code
> > >> > > > > commit
> > >> > > > > >> > > >> > philosophy
> > >> > > > > >> > > >> > > > for
> > >> > > > > >> > > >> > > > > a
> > >> > > > > >> > > >> > > > > > fault-tolerant distributed data store. ;)
> > >> > > > > >> > > >> > > > > >
> > >> > > > > >> > > >> > > > > > More seriously, a lack of test coverage
> for
> > >> > > existing
> > >> > > > > >> > features
> > >> > > > > >> > > >> > > shouldn't
> > >> > > > > >> > > >> > > > > be
> > >> > > > > >> > > >> > > > > > used as justification for introducing new
> > >> > features
> > >> > > > with
> > >> > > > > >> the
> > >> > > > > >> > > same
> > >> > > > > >> > > >> > > > > > shortcomings. Ultimately, it's the end
> user
> > who
> > >> > > will
> > >> > > > > feel
> > >> > > > > >> > the
> > >> > > > > >> > > >> pain,
> > >> > > > > >> > > >> > > so
> > >> > > > > >> > > >> > > > > > shouldn't we do everything we can to
> > mitigate
> > >> > that?
> > >> > > > > >> > > >> > > > > >
> > >> > > > > >> > > >> > > > > > -Dima
> > >> > > > > >> > > >> > > > > >
> > >> > > > > >> > > >> > > > > > On Wed, Sep 7, 2016 at 8:46 AM, Vladimir
> > >> > Rodionov <
> > >> > > > > >> > > >> > > > > vladrodionov@gmail.com>
> > >> > > > > >> > > >> > > > > > wrote:
> > >> > > > > >> > > >> > > > > >
> > >> > > > > >> > > >> > > > > > > Sean,
> > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > >> > > > > > > * have docs
> > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > >> > > > > > > Agree. We have a doc and backup is the
> > most
> > >> > > > > documented
> > >> > > > > >> > > feature
> > >> > > > > >> > > >> > :),
> > >> > > > > >> > > >> > > we
> > >> > > > > >> > > >> > > > > > will
> > >> > > > > >> > > >> > > > > > > release it shortly to Apache.
> > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > >> > > > > > > * have sunny-day correctness tests
> > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > >> > > > > > > Feature has  close to 60 test cases,
> which
> > >> run
> > >> > > for
> > >> > > > > >> approx
> > >> > > > > >> > 30
> > >> > > > > >> > > >> min.
> > >> > > > > >> > > >> > > We
> > >> > > > > >> > > >> > > > > can
> > >> > > > > >> > > >> > > > > > > add more, if community do not mind :)
> > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > >> > > > > > > * have correctness-in-face-of-failure
> > tests
> > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > >> > > > > > > Any examples of these tests in existing
> > >> > features?
> > >> > > > In
> > >> > > > > >> > works,
> > >> > > > > >> > > we
> > >> > > > > >> > > >> > > have a
> > >> > > > > >> > > >> > > > > > clear
> > >> > > > > >> > > >> > > > > > > understanding of what should be done by
> > the
> > >> > time
> > >> > > of
> > >> > > > > 2.0
> > >> > > > > >> > > >> release.
> > >> > > > > >> > > >> > > > > > > That is very close goal for us, to
> verify
> > IT
> > >> > > monkey
> > >> > > > > for
> > >> > > > > >> > > >> existing
> > >> > > > > >> > > >> > > > code.
> > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > >> > > > > > > * don't rely on things outside of HBase
> > for
> > >> > > normal
> > >> > > > > >> > operation
> > >> > > > > >> > > >> > (okay
> > >> > > > > >> > > >> > > > for
> > >> > > > > >> > > >> > > > > > > advanced operation)
> > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > >> > > > > > > We do not.
> > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > >> > > > > > > Enormous time has been spent already on
> > the
> > >> > > > > development
> > >> > > > > >> > and
> > >> > > > > >> > > >> > testing
> > >> > > > > >> > > >> > > > the
> > >> > > > > >> > > >> > > > > > > feature, it has passed our internal
> tests
> > and
> > >> > > many
> > >> > > > > >> rounds
> > >> > > > > >> > of
> > >> > > > > >> > > >> code
> > >> > > > > >> > > >> > > > > reviews
> > >> > > > > >> > > >> > > > > > > by HBase committers. We do not mind if
> > >> someone
> > >> > > from
> > >> > > > > >> HBase
> > >> > > > > >> > > >> > community
> > >> > > > > >> > > >> > > > > > > (outside of HW) will review the code,
> but
> > it
> > >> > will
> > >> > > > > >> probably
> > >> > > > > >> > > >> takes
> > >> > > > > >> > > >> > > > > forever
> > >> > > > > >> > > >> > > > > > to
> > >> > > > > >> > > >> > > > > > > wait for volunteer?, the feature is
> quite
> > >> large
> > >> > > > (1MB+
> > >> > > > > >> > > >> cumulative
> > >> > > > > >> > > >> > > > patch)
> > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > >> > > > > > > 2.0 branch is full of half baked
> features,
> > >> most
> > >> > > of
> > >> > > > > them
> > >> > > > > >> > are
> > >> > > > > >> > > in
> > >> > > > > >> > > >> > > active
> > >> > > > > >> > > >> > > > > > > development, therefore I am not
> following
> > you
> > >> > > here,
> > >> > > > > >> Sean?
> > >> > > > > >> > > Why
> > >> > > > > >> > > >> > > > > HBASE-7912
> > >> > > > > >> > > >> > > > > > is
> > >> > > > > >> > > >> > > > > > > not good enough yet to be integrated
> into
> > 2.0
> > >> > > > branch?
> > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > >> > > > > > > -Vlad
> > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > >> > > > > > > On Wed, Sep 7, 2016 at 8:23 AM, Sean
> > Busbey <
> > >> > > > > >> > > busbey@apache.org
> > >> > > > > >> > > >> >
> > >> > > > > >> > > >> > > > wrote:
> > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > >> > > > > > > > On Tue, Sep 6, 2016 at 10:36 PM, Josh
> > >> Elser <
> > >> > > > > >> > > >> > > josh.elser@gmail.com>
> > >> > > > > >> > > >> > > > > > > wrote:
> > >> > > > > >> > > >> > > > > > > > > So, the answer to Sean's original
> > >> question
> > >> > is
> > >> > > > "as
> > >> > > > > >> > > robust as
> > >> > > > > >> > > >> > > > > snapshots
> > >> > > > > >> > > >> > > > > > > > > presently are"? (independence of
> > >> > > backup/restore
> > >> > > > > >> > failure
> > >> > > > > >> > > >> > > tolerance
> > >> > > > > >> > > >> > > > > > from
> > >> > > > > >> > > >> > > > > > > > > snapshot failure tolerance)
> > >> > > > > >> > > >> > > > > > > > >
> > >> > > > > >> > > >> > > > > > > > > Is this just a question WRT context
> of
> > >> the
> > >> > > > > change,
> > >> > > > > >> or
> > >> > > > > >> > > is it
> > >> > > > > >> > > >> > > means
> > >> > > > > >> > > >> > > > > > for a
> > >> > > > > >> > > >> > > > > > > > veto
> > >> > > > > >> > > >> > > > > > > > > from you, Sean? Just trying to make
> > sure
> > >> > I'm
> > >> > > > > >> following
> > >> > > > > >> > > >> along
> > >> > > > > >> > > >> > > > > > > adequately.
> > >> > > > > >> > > >> > > > > > > > >
> > >> > > > > >> > > >> > > > > > > > >
> > >> > > > > >> > > >> > > > > > > >
> > >> > > > > >> > > >> > > > > > > > I'd say ATM I'm -0, bordering on -1
> but
> > not
> > >> > for
> > >> > > > > >> reasons
> > >> > > > > >> > I
> > >> > > > > >> > > can
> > >> > > > > >> > > >> > > > > > articulate
> > >> > > > > >> > > >> > > > > > > > well.
> > >> > > > > >> > > >> > > > > > > >
> > >> > > > > >> > > >> > > > > > > > Here's an attempt.
> > >> > > > > >> > > >> > > > > > > >
> > >> > > > > >> > > >> > > > > > > > We've been trying to move, as a
> > community,
> > >> > > > towards
> > >> > > > > >> > > minimizing
> > >> > > > > >> > > >> > > risk
> > >> > > > > >> > > >> > > > to
> > >> > > > > >> > > >> > > > > > > > downstream folks by getting "complete
> > >> enough
> > >> > > for
> > >> > > > > use"
> > >> > > > > >> > > gates
> > >> > > > > >> > > >> in
> > >> > > > > >> > > >> > > > place
> > >> > > > > >> > > >> > > > > > > > before we introduce new features. This
> > was
> > >> > > > spurred
> > >> > > > > >> by a
> > >> > > > > >> > > some
> > >> > > > > >> > > >> > > > features
> > >> > > > > >> > > >> > > > > > > > getting in half-baked and never making
> > it
> > >> to
> > >> > > "can
> > >> > > > > >> really
> > >> > > > > >> > > use"
> > >> > > > > >> > > >> > > > status
> > >> > > > > >> > > >> > > > > > > > (I'm thinking of distributed log
> replay
> > and
> > >> > the
> > >> > > > > >> zk-less
> > >> > > > > >> > > >> > > assignment
> > >> > > > > >> > > >> > > > > > > > stuff, I don't recall if there was
> > more).
> > >> > > > > >> > > >> > > > > > > >
> > >> > > > > >> > > >> > > > > > > > The gates, generally, included things
> > like:
> > >> > > > > >> > > >> > > > > > > >
> > >> > > > > >> > > >> > > > > > > > * have docs
> > >> > > > > >> > > >> > > > > > > > * have sunny-day correctness tests
> > >> > > > > >> > > >> > > > > > > > * have correctness-in-face-of-failure
> > tests
> > >> > > > > >> > > >> > > > > > > > * don't rely on things outside of
> HBase
> > for
> > >> > > > normal
> > >> > > > > >> > > operation
> > >> > > > > >> > > >> > > (okay
> > >> > > > > >> > > >> > > > > for
> > >> > > > > >> > > >> > > > > > > > advanced operation)
> > >> > > > > >> > > >> > > > > > > >
> > >> > > > > >> > > >> > > > > > > > As an example, we kept the MOB work
> off
> > in
> > >> a
> > >> > > > branch
> > >> > > > > >> and
> > >> > > > > >> > > out
> > >> > > > > >> > > >> of
> > >> > > > > >> > > >> > > > master
> > >> > > > > >> > > >> > > > > > > > until it could pass these criteria.
> The
> > big
> > >> > > > > exemption
> > >> > > > > >> > > we've
> > >> > > > > >> > > >> had
> > >> > > > > >> > > >> > > to
> > >> > > > > >> > > >> > > > > > > > this was the hbase-spark integration,
> > where
> > >> > we
> > >> > > > all
> > >> > > > > >> > agreed
> > >> > > > > >> > > it
> > >> > > > > >> > > >> > > could
> > >> > > > > >> > > >> > > > > > > > land in master because it was very
> well
> > >> > > isolated
> > >> > > > > (the
> > >> > > > > >> > > slide
> > >> > > > > >> > > >> > away
> > >> > > > > >> > > >> > > > from
> > >> > > > > >> > > >> > > > > > > > including docs as a first-class part
> of
> > >> > > building
> > >> > > > up
> > >> > > > > >> that
> > >> > > > > >> > > >> > > > integration
> > >> > > > > >> > > >> > > > > > > > has led me to doubt the wisdom of this
> > >> > > decision).
> > >> > > > > >> > > >> > > > > > > >
> > >> > > > > >> > > >> > > > > > > > We've also been treating inclusion in
> a
> > >> > > "probably
> > >> > > > > >> will
> > >> > > > > >> > be
> > >> > > > > >> > > >> > > released
> > >> > > > > >> > > >> > > > to
> > >> > > > > >> > > >> > > > > > > > downstream" branches as a higher bar,
> > >> > requiring
> > >> > > > > >> > > >> > > > > > > >
> > >> > > > > >> > > >> > > > > > > > * don't moderately impact performance
> > when
> > >> > the
> > >> > > > > >> feature
> > >> > > > > >> > > isn't
> > >> > > > > >> > > >> in
> > >> > > > > >> > > >> > > use
> > >> > > > > >> > > >> > > > > > > > * don't severely impact performance
> when
> > >> the
> > >> > > > > feature
> > >> > > > > >> is
> > >> > > > > >> > in
> > >> > > > > >> > > >> use
> > >> > > > > >> > > >> > > > > > > > * either default-to-on or show enough
> > >> demand
> > >> > to
> > >> > > > > >> believe
> > >> > > > > >> > a
> > >> > > > > >> > > >> > > > non-trivial
> > >> > > > > >> > > >> > > > > > > > number of folks will turn the feature
> on
> > >> > > > > >> > > >> > > > > > > >
> > >> > > > > >> > > >> > > > > > > > The above has kept MOB and hbase-spark
> > >> > > > integration
> > >> > > > > >> out
> > >> > > > > >> > of
> > >> > > > > >> > > >> > > branch-1,
> > >> > > > > >> > > >> > > > > > > > presumably while they've "gotten more
> > >> stable"
> > >> > > in
> > >> > > > > >> master
> > >> > > > > >> > > from
> > >> > > > > >> > > >> > the
> > >> > > > > >> > > >> > > > odd
> > >> > > > > >> > > >> > > > > > > > vendor inclusion.
> > >> > > > > >> > > >> > > > > > > >
> > >> > > > > >> > > >> > > > > > > > Are we going to have a 2.0 release
> > before
> > >> the
> > >> > > end
> > >> > > > > of
> > >> > > > > >> the
> > >> > > > > >> > > >> year?
> > >> > > > > >> > > >> > > > We're
> > >> > > > > >> > > >> > > > > > > > coming up on 1.5 years since the
> > release of
> > >> > > > version
> > >> > > > > >> 1.0;
> > >> > > > > >> > > >> seems
> > >> > > > > >> > > >> > > like
> > >> > > > > >> > > >> > > > > > > > it's about time, though I haven't seen
> > any
> > >> > > > concrete
> > >> > > > > >> > plans
> > >> > > > > >> > > >> this
> > >> > > > > >> > > >> > > > year.
> > >> > > > > >> > > >> > > > > > > > Presuming we are going to have one by
> > the
> > >> end
> > >> > > of
> > >> > > > > the
> > >> > > > > >> > > year, it
> > >> > > > > >> > > >> > > > seems a
> > >> > > > > >> > > >> > > > > > > > bit close to still be adding in
> > "features
> > >> > that
> > >> > > > need
> > >> > > > > >> > > maturing"
> > >> > > > > >> > > >> > on
> > >> > > > > >> > > >> > > > the
> > >> > > > > >> > > >> > > > > > > > branch.
> > >> > > > > >> > > >> > > > > > > >
> > >> > > > > >> > > >> > > > > > > > The lack of a concrete plan for 2.0
> > keeps
> > >> me
> > >> > > from
> > >> > > > > >> > > considering
> > >> > > > > >> > > >> > > these
> > >> > > > > >> > > >> > > > > > > > things blocker at the moment. But I
> know
> > >> > first
> > >> > > > hand
> > >> > > > > >> how
> > >> > > > > >> > > much
> > >> > > > > >> > > >> > > > trouble
> > >> > > > > >> > > >> > > > > > > > folks have had with other features
> that
> > >> have
> > >> > > gone
> > >> > > > > >> into
> > >> > > > > >> > > >> > downstream
> > >> > > > > >> > > >> > > > > > > > facing releases without robustness
> > checks
> > >> > (i.e.
> > >> > > > > >> > > replication),
> > >> > > > > >> > > >> > and
> > >> > > > > >> > > >> > > > I'm
> > >> > > > > >> > > >> > > > > > > > concerned about what we're setting up
> if
> > >> 2.0
> > >> > > goes
> > >> > > > > out
> > >> > > > > >> > with
> > >> > > > > >> > > >> this
> > >> > > > > >> > > >> > > > > > > > feature in its current state.
> > >> > > > > >> > > >> > > > > > > >
> > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > >> > > > > >
> > >> > > > > >> > > >> > > > >
> > >> > > > > >> > > >> > > > >
> > >> > > > > >> > > >> > > > >
> > >> > > > > >> > > >> > > > > --
> > >> > > > > >> > > >> > > > > Best regards,
> > >> > > > > >> > > >> > > > >
> > >> > > > > >> > > >> > > > >    - Andy
> > >> > > > > >> > > >> > > > >
> > >> > > > > >> > > >> > > > > Problems worthy of attack prove their worth
> by
> > >> > > hitting
> > >> > > > > >> back. -
> > >> > > > > >> > > Piet
> > >> > > > > >> > > >> > > Hein
> > >> > > > > >> > > >> > > > > (via Tom White)
> > >> > > > > >> > > >> > > > >
> > >> > > > > >> > > >> > > >
> > >> > > > > >> > > >> > >
> > >> > > > > >> > > >> > >
> > >> > > > > >> > > >> > >
> > >> > > > > >> > > >> > > --
> > >> > > > > >> > > >> > > Best regards,
> > >> > > > > >> > > >> > >
> > >> > > > > >> > > >> > >    - Andy
> > >> > > > > >> > > >> > >
> > >> > > > > >> > > >> > > Problems worthy of attack prove their worth by
> > >> hitting
> > >> > > > back.
> > >> > > > > -
> > >> > > > > >> > Piet
> > >> > > > > >> > > >> Hein
> > >> > > > > >> > > >> > > (via Tom White)
> > >> > > > > >> > > >> > >
> > >> > > > > >> > > >> >
> > >> > > > > >> > > >>
> > >> > > > > >> > >
> > >> > > > > >> >
> > >> > > > > >>
> > >> > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> >
> >
> > --
> > busbey
> >
>