Re: Moving 2.0 forward

[Stack <st...@duboce.net>]

hbase-2.0.0-beta-1 update (Reminder, beta-1 is where we finish last
remaining features and apply final polish to API. There will be a beta-2
but it is about upgrade/rolling-upgrade and bug fixes ONLY).

Myself and Mr. Drob did a pass over the outstanding hbase-2.0.0-beta-1 list
this morning. See here [1].

There are about ~12 issues in progress with most of these about to land.
There are 37 TODO. Many of these are tests we need to run, some are related
to the backup/restore, but a good few are meaty w/o assignees.

The awkward outstanding ones as I see it are the below:

HBASE-18946 Stochastic load balancer assigns replica regions to the same RS
HBASE-17204 Make L2 off heap cache default ON
HBASE-19112 Suspect methods on Cell to be deprecated
HBASE-19147 All branch-2 unit tests pass

We need to make progress on the above or punt on them (can't punt on the
last one though).

Any ideas on what configs we should update in hbase2? Dump ideas into:
HBASE-19148 Edit of default configuration

Still hoping for an early December beta-1 RC. beta-2 hopefully will be
close behind.

Comments? Thoughts?

Thanks all,
S

1. https://issues.apache.org/jira/projects/HBASE/versions/12340861

On Tue, Oct 31, 2017 at 2:56 PM, Stack <st...@duboce.net> wrote:

>
>
> On Tue, Oct 31, 2017 at 2:31 PM, Mike Drob <md...@apache.org> wrote:
>
>> Hoping to keep momentum going from our Stack working on alpha4, I tried to
>> take a stab at triaging some of the open beta-1 issues.
>>
>> I moved some docs stuff out form beta-1 to 2.0-GA, if it gets done sooner
>> then I'm happy to see it pulled back in. Trying to balance optimism with
>> realism here, and knowing that documentation unfortunately often gets
>> pushed to the back-burner.
>>
>> Also, I poked some folks on unassigned issues that they've filed for
>> beta-1, especially in the last few days. If issues don't have an owner
>> they
>> are unlikely to get worked. I chatted with stack and he agreed to take on
>> some of the tasks, but there's a lot of surface area to cover.
>>
>> If you you're working on issues that are critical for beta-1, please mark
>> them as such. Then the rest of the community will know to help prioritize
>> feedback and reviews there.
>>
>> Do we have a general theme for the betas like we did with the alphas?
>>
>> Beta1 is upgrades work from branch1, beta2 is rolling upgrades work as
>> well? Continue to work on tests throughout?
>>
>>
> Thanks Mike for helping to kick off the beta-1 train.
>
> Regards a theme for beta-1, I'd like it to be 'finish'; beta-1 is what we
> are going to ship (beta-2 is rolling upgrade and any minor items turned up
> in testing/burn-in).
>
> Thanks,
> S
>
>
>
>> Mike
>>
>> On Tue, Oct 31, 2017 at 10:04 AM, Josh Elser <el...@apache.org> wrote:
>>
>> > +1 go from my POV.
>> >
>> >
>> > On 10/31/17 10:07 AM, Stack wrote:
>> >
>> >> I want to push an alpha-4 today. A few items didn't make it
>> (HBASE-19092).
>> >> They need more time. We'll pull them in for beta-1. CP API is basically
>> >> done. There may be some changes for beta-1 but hopefully only changes
>> >> informed by experience trying to port an existing Coprocessor to
>> hbase2.
>> >>
>> >> Shout if there is anything that needs to make alpha-4.
>> >>
>> >> Thanks,
>> >> St.Ack
>> >>
>> >>
>> >> On Sat, Oct 28, 2017 at 2:48 PM, Josh Elser <jo...@gmail.com>
>> wrote:
>> >>
>> >> Yup, that was going to be my plan, Mike!
>> >>>
>> >>> Making a pass now, and will check back later tonight again. I see
>> others
>> >>> have already done some work today on this front.
>> >>>
>> >>>
>> >>> On 10/27/17 11:38 PM, Mike Drob wrote:
>> >>>
>> >>> Josh - Do you want to kick off a bunch of QA runs? (Do you know how
>> to do
>> >>>> it directly on the jenkins job, so you don't have to bother with JIRA
>> >>>> uploads)
>> >>>>
>> >>>> If you're busy, then I can make time tomorrow or Sunday to kick off
>> >>>> jobs,
>> >>>> but I want to make sure we're not duplicating effort and jenkins
>> cycles.
>> >>>>
>> >>>> On Fri, Oct 27, 2017 at 7:10 PM, Josh Elser <el...@apache.org>
>> wrote:
>> >>>>
>> >>>> My turn to bump ;)
>> >>>>
>> >>>>>
>> >>>>> By my take: HBASE-18770 and HBASE-19092 are the only issues that
>> remain
>> >>>>> needing some more work. The rest are just awaiting a good QA run.
>> >>>>>
>> >>>>> Unless I hear otherwise, I'll try to keep an eye on things over the
>> >>>>> weekend, bump them along as necessary, and get them committed.
>> Would be
>> >>>>> great to be able get a vote up on Monday.
>> >>>>>
>> >>>>>
>> >>>>> On 10/24/17 6:03 PM, Stack wrote:
>> >>>>>
>> >>>>> Chatting with my coworker Mr. Mike Drob, we were batting back and
>> forth
>> >>>>>
>> >>>>>> what remains to be done. Surfacing our thoughts here so you all
>> clued
>> >>>>>> in....Or if you think otherwise, please speak up.
>> >>>>>>
>> >>>>>> We have ~13 issues to land:
>> >>>>>> https://issues.apache.org/jira/projects/HBASE/versions/12341594
>> About
>> >>>>>> two
>> >>>>>> are meta-issues that are about process which leaves 11.
>> >>>>>>
>> >>>>>> Duo and Zheng Hu are to merge the FilterList fixes improvements
>> >>>>>> (HBASE-19057, HBASE-18410 et al.). These are blocker because some
>> >>>>>> changed
>> >>>>>> API/semantic that we need to get out earlier rather than later.
>> >>>>>>
>> >>>>>> Once the above is merged, HBASE-13346, change of Filter method
>> names
>> >>>>>> to
>> >>>>>> mention Cell instead of KeyValue can land.
>> >>>>>>
>> >>>>>> HBASE-199048 needs a review (Anoop will probably do it), removing
>> >>>>>> IA.Private objects as params to MasterObserver... Hopefully this
>> goes
>> >>>>>> in
>> >>>>>> soon.
>> >>>>>>
>> >>>>>> Duo is hard at work on trackers for flush and compaction for CPs
>> >>>>>> (HBASE-18905). How is HBASE-19033 looking Duo (facility for
>> Tephra)?
>> >>>>>>
>> >>>>>> I think HBASE-18906 (Phoenix Region#waitFor...) will evaporate
>> after
>> >>>>>> Duo
>> >>>>>> is
>> >>>>>> done w/ his work above.
>> >>>>>>
>> >>>>>> I'm on HBASE-18770 bypass and HBASE-19077 restore some parity after
>> >>>>>> all
>> >>>>>> the
>> >>>>>> purges allowing CPs do direct calls against Regions in same Host.
>> >>>>>>
>> >>>>>> Anoop is on HBASE-19047 (Fixes) and Ram on cleanup of CellUtil.
>> >>>>>>
>> >>>>>> Another day or two?
>> >>>>>>
>> >>>>>> St.Ack
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> On Mon, Oct 23, 2017 at 2:52 PM, Stack <st...@duboce.net> wrote:
>> >>>>>>
>> >>>>>>
>> >>>>>> On Mon, Oct 23, 2017 at 11:59 AM, Josh Elser <el...@apache.org>
>> >>>>>> wrote:
>> >>>>>>
>> >>>>>>>
>> >>>>>>> +1
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>> I was trying to work on helping out on the outstanding alpha-4
>> stuff
>> >>>>>>>> last
>> >>>>>>>> week -- will be continuing to try to do the same this week.
>> >>>>>>>>
>> >>>>>>>> If you need any help, Stack, or if others need reviews where I
>> >>>>>>>> haven't
>> >>>>>>>> noticed on my own: feel free to @mention me.
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> Thanks for the offer Josh. All items seem assigned and are being
>> >>>>>>>>
>> >>>>>>>> actively
>> >>>>>>> worked on. If you get a moment, reviews by you (or anyone else)
>> helps
>> >>>>>>> move
>> >>>>>>> the process along.
>> >>>>>>>
>> >>>>>>> We need to merge in HBASE-18410 branch to pick up Filter
>> >>>>>>> improvements.
>> >>>>>>> Then HBASE-13346 can go in.
>> >>>>>>>
>> >>>>>>> You are already helping out on HBASE-18906, thanks. Looks like
>> that
>> >>>>>>> will
>> >>>>>>> be addressed by other alpha-4s about to land.
>> >>>>>>>
>> >>>>>>> St.Ack
>> >>>>>>> TODOs: https://issues.apache.org/jira
>> /projects/HBASE/versions/12341
>> >>>>>>> 594
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On 10/23/17 12:53 PM, Stack wrote:
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>> (Reviving this thread)
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>> Lets push out alpha-4 this week. Alpha-4 is the release that has
>> >>>>>>>>> the
>> >>>>>>>>> refactor of the Coprocessor API shutting down access to
>> internals
>> >>>>>>>>> marked
>> >>>>>>>>> InterfaceAudience.Private.
>> >>>>>>>>>
>> >>>>>>>>> The outstanding list is here:
>> >>>>>>>>> https://issues.apache.org/jira/projects/HBASE/versions/12341594
>> >>>>>>>>>
>> >>>>>>>>> Please push in anything marked alpha-4 that belongs to you.
>> >>>>>>>>>
>> >>>>>>>>> If issue, talk out loud on this thread. If you need a review to
>> >>>>>>>>> land
>> >>>>>>>>> an
>> >>>>>>>>> item, shout on the issue and here; we'll help you out.
>> >>>>>>>>>
>> >>>>>>>>> As is, items are coming along nicely I'd say. We need to merge
>> the
>> >>>>>>>>> filter
>> >>>>>>>>> branch -- HBASE-18410 -- so APIs are finished for hbase2.
>> >>>>>>>>>
>> >>>>>>>>> Post alpha-4, we'll have to hunt down our downstreamers and help
>> >>>>>>>>> them
>> >>>>>>>>> test
>> >>>>>>>>> on top of alpha-4 so rolling into beta-1, we have confidence our
>> >>>>>>>>> downstreamers know what to expect (or we discover what we missed
>> >>>>>>>>> BEFORE
>> >>>>>>>>> we
>> >>>>>>>>> beta-1).
>> >>>>>>>>>
>> >>>>>>>>> Thanks for time,
>> >>>>>>>>> S
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> On Fri, Sep 8, 2017 at 2:04 PM, Stack <st...@duboce.net> wrote:
>> >>>>>>>>>
>> >>>>>>>>> I'll put up an alpha3 RC Monday, probably Monday night. That
>> should
>> >>>>>>>>> be
>> >>>>>>>>>
>> >>>>>>>>> time, if we all sprint, for the public-facing API fixes to be
>> done.
>> >>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> I had a bunch of Coprocessor refactor and fixup scheduled for
>> >>>>>>>>>> alpha3
>> >>>>>>>>>> but
>> >>>>>>>>>> it is plain that more time is needed (in spite of valiant
>> effort
>> >>>>>>>>>> so
>> >>>>>>>>>> far
>> >>>>>>>>>> by
>> >>>>>>>>>> Anoop, Duo, Appy, etc.). Therefore, lets run a 2.0.0-alpha-4
>> whose
>> >>>>>>>>>> theme is
>> >>>>>>>>>> "Coprocessor Fixup". Hopefully we can put an alpha-4 up by the
>> >>>>>>>>>> following
>> >>>>>>>>>> week.
>> >>>>>>>>>>
>> >>>>>>>>>> We should then be ready for beta (beta == no new features, no
>> API
>> >>>>>>>>>> changes,
>> >>>>>>>>>> just fixes).
>> >>>>>>>>>>
>> >>>>>>>>>> Thanks,
>> >>>>>>>>>> St.Ack
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> On Thu, Aug 17, 2017 at 12:35 PM, Stack <st...@duboce.net>
>> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>> I put up the hbase-2.0.0-alpha2 release candidate. Please vote
>> on
>> >>>>>>>>>> it.
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> For hbase-2.0.0-alpha3, the theme is solidifying API. I hope to
>> >>>>>>>>>>> get a
>> >>>>>>>>>>> release out in the next week or so.
>> >>>>>>>>>>>
>> >>>>>>>>>>> I did a weeding of 2.0.0 issues over the last day. If folks
>> are
>> >>>>>>>>>>> interested in helping out, below are the items I think we need
>> >>>>>>>>>>> done
>> >>>>>>>>>>> for
>> >>>>>>>>>>> alpha3 (below are at least 'Critical' status, are API possibly
>> >>>>>>>>>>> altering
>> >>>>>>>>>>> items, and are absent those JIRAs that are making active
>> >>>>>>>>>>> progress,
>> >>>>>>>>>>> i.e. the
>> >>>>>>>>>>> HTD/HCD revamp by Chia-Ping Tsai). A project NOT listed that
>> >>>>>>>>>>> needs
>> >>>>>>>>>>> doing is
>> >>>>>>>>>>> what Andrew did comparing 1.3. and 1.4 APIs
>> >>>>>>>>>>>
>> >>>>>>>>>>> * HBASE-18622 Mitigate compatibility concerns between branch-1
>> >>>>>>>>>>> and
>> >>>>>>>>>>> branch-2
>> >>>>>>>>>>> This is to do what Andrew did between 1.3 and 1.4 branches
>> only
>> >>>>>>>>>>> do
>> >>>>>>>>>>> it
>> >>>>>>>>>>> between branch-1 and branch-2.
>> >>>>>>>>>>>
>> >>>>>>>>>>> * HBASE-10462 Recategorize some of the client facing Public /
>> >>>>>>>>>>> Private
>> >>>>>>>>>>> interfaces
>> >>>>>>>>>>> This one is almost done. It could do with a finish, attention
>> to
>> >>>>>>>>>>> the
>> >>>>>>>>>>> items in last comment, and then our codebase could do with
>> >>>>>>>>>>> another
>> >>>>>>>>>>> sweep
>> >>>>>>>>>>> after the spirit of this issue since a bunch has gone in since
>> >>>>>>>>>>> the
>> >>>>>>>>>>> pass
>> >>>>>>>>>>> that was the basis of this issue.
>> >>>>>>>>>>>
>> >>>>>>>>>>> * HBASE-10504 Define Replication Interface
>> >>>>>>>>>>> I was going to take a crack at this as part of the revamp
>> forced
>> >>>>>>>>>>> by
>> >>>>>>>>>>> 'HBASE-15982 Interface ReplicationEndpoint extends Guava's
>> >>>>>>>>>>> Service'
>> >>>>>>>>>>> but if
>> >>>>>>>>>>> anyone else is interested, be my guest.
>> >>>>>>>>>>>
>> >>>>>>>>>>> * HBASE-14996 Some more API cleanup for 2.0
>> >>>>>>>>>>> Has a bunch of subtasks, some of which are being worked on.
>> Needs
>> >>>>>>>>>>> finishing.
>> >>>>>>>>>>>
>> >>>>>>>>>>> * HBASE-14998 Unify synchronous and asynchronous methods in
>> Admin
>> >>>>>>>>>>> and
>> >>>>>>>>>>> cleanup
>> >>>>>>>>>>> Needs a pass. Small issue I think. Could also look at new
>> >>>>>>>>>>> AsyncClient
>> >>>>>>>>>>> and
>> >>>>>>>>>>> make sure symmetry.
>> >>>>>>>>>>>
>> >>>>>>>>>>> * HBASE-15607 Remove PB references from Admin for 2.0
>> >>>>>>>>>>> Predicated on result of an ongoing DISCUSSION thread but
>> needs to
>> >>>>>>>>>>> be
>> >>>>>>>>>>> done.
>> >>>>>>>>>>>
>> >>>>>>>>>>> Rolling upgrade will have implications for our API. Would be
>> good
>> >>>>>>>>>>> to
>> >>>>>>>>>>> try
>> >>>>>>>>>>> it and figure what needs fixup (as said above, according to
>> trial
>> >>>>>>>>>>> by
>> >>>>>>>>>>> Sean,
>> >>>>>>>>>>> we might not be too bad here):
>> >>>>>>>>>>> * HBASE-16060 1.x clients cannot access table state talking to
>> >>>>>>>>>>> 2.0
>> >>>>>>>>>>> cluster
>> >>>>>>>>>>> * HBASE-16550 Procedure v2 - Add AM compatibility for 2.x
>> Master
>> >>>>>>>>>>> and
>> >>>>>>>>>>> 1.x
>> >>>>>>>>>>> RSs; i.e. support Rolling Upgrade from hbase-1 to -2.
>> >>>>>>>>>>>
>> >>>>>>>>>>> * HBASE-17442 Move most of the replication related classes to
>> >>>>>>>>>>> hbase-server package
>> >>>>>>>>>>> The above would be good to do generally but it may make for
>> >>>>>>>>>>> ripples
>> >>>>>>>>>>> in
>> >>>>>>>>>>> API so would be good to do now.
>> >>>>>>>>>>>
>> >>>>>>>>>>> * HBASE-18106 Redo ProcedureInfo and LockInfo
>> >>>>>>>>>>> Balazs is working on this. The idea is that we avoid adding
>> two
>> >>>>>>>>>>> new
>> >>>>>>>>>>> types
>> >>>>>>>>>>> to our API, two types that are nought but curtailed, read-only
>> >>>>>>>>>>> views
>> >>>>>>>>>>> on
>> >>>>>>>>>>> internals. Input if you have time appreciated.
>> >>>>>>>>>>>
>> >>>>>>>>>>> * HBASE-18596 A hbase1 cluster should be able to replicate to
>> a
>> >>>>>>>>>>> hbase2
>> >>>>>>>>>>> cluster; verify
>> >>>>>>>>>>> Esteban is looking at this one
>> >>>>>>>>>>>
>> >>>>>>>>>>> * HBASE-9417 SecureBulkLoadEndpoint should be folded in core
>> >>>>>>>>>>> * HBASE-17143 Scan improvement
>> >>>>>>>>>>>
>> >>>>>>>>>>> Our Coprocessor Interface needs a tough edit. It exposes
>> >>>>>>>>>>> implementations
>> >>>>>>>>>>> marked audience Private and returns implementations rather
>> than
>> >>>>>>>>>>> Interfaces.
>> >>>>>>>>>>> In a few locations, we allow returning an alternate
>> >>>>>>>>>>> implementation
>> >>>>>>>>>>> altogether which is probably something we don't want a CP
>> doing.
>> >>>>>>>>>>> To
>> >>>>>>>>>>> that
>> >>>>>>>>>>> end, the following issues started by Duo and Anoop need to be
>> >>>>>>>>>>> taken
>> >>>>>>>>>>> to
>> >>>>>>>>>>> the
>> >>>>>>>>>>> finish line; ideally they'd have an owner:
>> >>>>>>>>>>>
>> >>>>>>>>>>> * HBASE-18169 Coprocessor fix and cleanup before 2.0.0
>> release <=
>> >>>>>>>>>>> The
>> >>>>>>>>>>> umbrella issue.
>> >>>>>>>>>>> * HBASE-18298 RegionServerServices Interface cleanup for CP
>> >>>>>>>>>>> expose
>> >>>>>>>>>>> * HBASE-16769 Deprecate/remove PB references from
>> MasterObserver
>> >>>>>>>>>>> and
>> >>>>>>>>>>> RegionServerObserver
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> Nice-to-haves:
>> >>>>>>>>>>>
>> >>>>>>>>>>> * HBASE-15284 Make TimeRange constructors IA.Private and
>> remove
>> >>>>>>>>>>> unused
>> >>>>>>>>>>> TimeRange constructors
>> >>>>>>>>>>>
>> >>>>>>>>>>> * HBASE-10944 Remove all kv.getBuffer() and kv.getRow()
>> >>>>>>>>>>> references
>> >>>>>>>>>>> existing in the code
>> >>>>>>>>>>> This is the end of an old long-running project moving up on to
>> >>>>>>>>>>> Cell
>> >>>>>>>>>>> Interface. We think it is done but for a few little items
>> >>>>>>>>>>> (deprecate
>> >>>>>>>>>>> KV
>> >>>>>>>>>>> methods in MR and provide Cell versions instead...)
>> >>>>>>>>>>>
>> >>>>>>>>>>> * HBASE-13271 Table#puts(List<Put>) operation is
>> indeterminate;
>> >>>>>>>>>>> needs
>> >>>>>>>>>>> fixing
>> >>>>>>>>>>>
>> >>>>>>>>>>> * HBASE-13346 Clean up Filter package for post 1.0
>> >>>>>>>>>>>
>> >>>>>>>>>>> * HBASE-14255 Simplify Cell creation post 1.0
>> >>>>>>>>>>> * HBASE-14997
>> >>>>>>>>>>> Move compareOp and Comparators out of filter to client package
>> >>>>>>>>>>>
>> >>>>>>>>>>> * HBASE-13740 Stop using Hadoop private interfaces
>> >>>>>>>>>>>
>> >>>>>>>>>>> What about:
>> >>>>>>>>>>>
>> >>>>>>>>>>> * HBASE-18601 Remove Htrace 3.2
>> >>>>>>>>>>> As has been noted, the HTrace API is our 'trace' API.
>> >>>>>>>>>>>
>> >>>>>>>>>>> If interested in any of the above and you need a legup, just
>> ask
>> >>>>>>>>>>> in
>> >>>>>>>>>>> the
>> >>>>>>>>>>> issue and I'll be by....
>> >>>>>>>>>>>
>> >>>>>>>>>>> Thanks,
>> >>>>>>>>>>> St.Ack
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> On Mon, Aug 14, 2017 at 10:54 AM, Stack <st...@duboce.net>
>> >>>>>>>>>>> wrote:
>> >>>>>>>>>>>
>> >>>>>>>>>>> Heads-up:
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> I'm about to put up an hbase-2.0.0-alpha2 Release Candidate.
>> >>>>>>>>>>>> Theme
>> >>>>>>>>>>>> is
>> >>>>>>>>>>>> updated dependencies, reliance on relocated popular libs
>> (guava,
>> >>>>>>>>>>>> netty,
>> >>>>>>>>>>>> protobuf), purge of checked-in generated src, and
>> >>>>>>>>>>>> master-carries-no-regions
>> >>>>>>>>>>>> by default.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> alpha3 I hope will follow soon after (end-of-August?). Its
>> theme
>> >>>>>>>>>>>> will
>> >>>>>>>>>>>> be
>> >>>>>>>>>>>> settling the APIs and compatibility (At first blush, we are
>> not
>> >>>>>>>>>>>> looking too
>> >>>>>>>>>>>> bad; our Sean ran some tests over weekend that have hbase-1
>> >>>>>>>>>>>> client
>> >>>>>>>>>>>> running
>> >>>>>>>>>>>> against an hbase-2 cluster....). The Coprocessor Interface
>> >>>>>>>>>>>> revamp
>> >>>>>>>>>>>> should be
>> >>>>>>>>>>>> done by alpha3 (i.e. returning Interfaces rather than
>> >>>>>>>>>>>> Implementations, and
>> >>>>>>>>>>>> our shutdown of CPs accessing classes in hbase marked
>> >>>>>>>>>>>> InterfaceAudience).
>> >>>>>>>>>>>> We'll also have purged thirdparty classes from our API; e.g.
>> >>>>>>>>>>>> guava
>> >>>>>>>>>>>> 0.12
>> >>>>>>>>>>>> Service showing through in our replication API and protobufs
>> in
>> >>>>>>>>>>>> Admin
>> >>>>>>>>>>>> Interface. On alpha3, we will have to do a bunch of outreach
>> to
>> >>>>>>>>>>>> make
>> >>>>>>>>>>>> sure
>> >>>>>>>>>>>> our downstreamers are up on what is coming down the pipe.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Beta1 in mid-September?
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> I encourage you to check out the items marked for hbase2:
>> >>>>>>>>>>>> https://issues.apache.org/jira/projects/HBASE/versions/12327
>> 188
>> >>>>>>>>>>>> Edit
>> >>>>>>>>>>>> as
>> >>>>>>>>>>>> you see appropriate. Punt if you know the JIRA will not get
>> any
>> >>>>>>>>>>>> attention
>> >>>>>>>>>>>> in next month or so.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> A bunch of issues marked blocker are unassigned. I'll leave
>> them
>> >>>>>>>>>>>> as
>> >>>>>>>>>>>> is
>> >>>>>>>>>>>> another while but I'll boot them soon.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> While I have your attention:
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> + I think we should leave thrift version at 0.9.3. Moving
>> hbase
>> >>>>>>>>>>>> thrift
>> >>>>>>>>>>>> to 0.10.0 will break existing clients. The change is easy
>> enough
>> >>>>>>>>>>>> if
>> >>>>>>>>>>>> folks
>> >>>>>>>>>>>> need to upgrade their hbase thrift. See HBASE-18591.
>> >>>>>>>>>>>> + Upgrade from 0.94 is disallowed. You have to get to 1.0
>> first
>> >>>>>>>>>>>> (0.98?).
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> St.Ack
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> On Wed, Aug 2, 2017 at 9:43 AM, Stack <st...@duboce.net>
>> wrote:
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> On Tue, Aug 1, 2017 at 2:06 PM, Josh Elser <
>> elserj@apache.org>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> On 7/31/17 9:00 AM, Stack wrote:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> On Mon, Jul 24, 2017 at 12:25 PM, Josh Elser<
>> >>>>>>>>>>>>>> elserj@apache.org>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> ...
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> I like the idea of this also hitting 2.0 as it would make
>> the
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> feature a
>> >>>>>>>>>>>>>>>> bit more "real", but am obviously a little nervous (I
>> have
>> >>>>>>>>>>>>>>>> no
>> >>>>>>>>>>>>>>>> reason
>> >>>>>>>>>>>>>>>> to be
>> >>>>>>>>>>>>>>>> nervous though). I am pretty happy with the feature in
>> terms
>> >>>>>>>>>>>>>>>> of
>> >>>>>>>>>>>>>>>> how
>> >>>>>>>>>>>>>>>> much it
>> >>>>>>>>>>>>>>>> is covered via testing.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/HBASE-17748
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Sounds good to me. Whats involved? Backport? If so, +1
>> Josh.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Last think on space quota says that need doc too. See
>> 'Space
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Quota' in
>> >>>>>>>>>>>>>>> here:
>> >>>>>>>>>>>>>>> https://docs.google.com/docume
>> nt/d/1WCsVlnHjJeKUcl7wHwqb4z9i
>> >>>>>>>>>>>>>>> Eu_ktczrlKHK8N4SZzs/edit#heading=h.wuw3a6jukzo5
>> >>>>>>>>>>>>>>> Does this little section need an update Josh?
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>> S
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Yep, just a couple of cherry-picks. Good test coverage and
>> >>>>>>>>>>>>>>> some
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> docs
>> >>>>>>>>>>>>>> already included for 17748.  Happy to put that on my plate
>> if
>> >>>>>>>>>>>>>> you're good
>> >>>>>>>>>>>>>> with it. I can reasonably assume that no one is against it
>> :)
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> I think I had knocked out docs for the "phase 1" stuff
>> before
>> >>>>>>>>>>>>>> we
>> >>>>>>>>>>>>>> merged it in from the original feature branch. I'll double
>> >>>>>>>>>>>>>> check
>> >>>>>>>>>>>>>> and update
>> >>>>>>>>>>>>>> the gdoc. Perhaps this was just a timing thing.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Thanks Josh,
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> S
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>
>> >>>>>>
>> >>>>
>> >>
>>
>
>

Re: Moving 2.0 forward

[Stack <st...@duboce.net>]

In another thread in this list I flag the coming 2.0.0-beta-2. Look for an
RC in the next day or so.

After 2.0.0-beta-2 makes it out, I'll cut branch-2.0. Soon after will come
our first hbase-2.0.0 release candidate. Expect this within a week or so
after beta-2 after remaining blockers are addressed (At this stage,
blockers are mostly documentation). Scream if there is something that has
to make 2.0.0 that I may have missed, but at this point, only critical
bug-fixes or compatibility breakages will be entertained.

I just did a pass over the issues filed against hbase-2.0.0. I did a pretty
brutal edit, sparse on commentary mostly because the count of issues
against 2.0.0 was high. There were a fair amount filed against '2.0.0' just
because 2.0.0 used to be thought of as the end of the rainbow, an event
that was way off on the horizon that would never arrive[1]. I unscheduled
the bulk of these. Reschedule against 2.0.0 if I got it wrong.  A good
bunch were great ideas almost- or half-done or never started (HBASE-14055
RegionServer Refactor, HBASE-15531 Favored Nodes, Andy's security
improvements, etc.). These were also unscheduled. Others that were really
old without recent updates I resolved. A few had been realized via other
channels.

Here is the current list [2]. Help on blockers would be much appreciated.

Your 2.0.0RM

P.S.No scale testing, perf compare, compatibility, or exercise of new
features (at least to my knowledge) has been done. We could do with some
exercise along any of these dimensions if any of you have a few spare
cycles. Thanks.

1. hbase-2.0.0 still has this smell to it, but we do our best.
2.
https://issues.apache.org/jira/projects/HBASE/versions/12327188#release-report-tab-body


On Wed, Dec 27, 2017 at 9:54 AM, Stack <st...@duboce.net> wrote:

> Heads-up, I'm working on cutting a beta-1. Shout if there is something you
> think needs to make it in (The Chia-Ping Tsai cleanups look good). I'm
> mainly waiting on the hbase-thirdparty changes to go in. The rest can wait
> it.
>
> Thanks,
> S
>
>
> On Wed, Dec 13, 2017 at 11:28 PM, Stack <st...@duboce.net> wrote:
>
>> Update. See below.
>>
>> On Tue, Dec 5, 2017 at 10:43 AM, Stack <st...@duboce.net> wrote:
>>
>>> Update on hbase-2.0.0-beta-1. I'm going to put up a beta-1 RC before
>>> Christmas day.
>>>
>>>
>> Still aiming for an hbase2 beta-1 showing up in your christmas stocking.
>>
>>
>>> Here is the list [1]. We're doing pretty good. We've fixed a load since
>>> the last update including some nice cleanups  two weeks ago (e.g. undoing
>>> client dependency on curator/zk watching making the dependence read-only)
>>> but there are still a few hairy ones hanging out there.
>>>
>>>  * HBASE-18946 Stochastic load balancer assigns replica regions to the
>>> same RS
>>>    This one is proving a little tough. Ram has been banging his head on
>>> it. We'll be able to clear a bunch of test failures once this is done.
>>>
>>>
>> This should be going in tomorrow.
>>
>>
>>
>>>  * HBASE-19112 Suspect methods on Cell to be deprecated
>>>    Good discussion up on the JIRA. We need to be crystal clear around
>>> Cell and derivatives API and Audience when we ship 2.0.
>>>
>>>
>> Ram, Chia-Ping Tsai, and Anoop are doing a nice job here and it should be
>> done in next day or so.
>>
>>
>>
>>>  * HBASE-17204 Make L2 off heap cache default ON
>>>    We need to try this at least so offheaping can be default.
>>>
>>>
>> Chatted with Anoop and Ram. Because it is so late in the day and because
>> it critical that the user have a good experience out of the box -- which
>> requires time and testing -- we are going to punt this from 2.0. Will look
>> at it for 2.1 (or 3.0 if soon). 2.0 will still be carrying all of the boys
>> offheap read/write path goodness.
>>
>>
>>
>>> Making asyncfswal default is also ongoing, HBASE-16890
>>> <https://issues.apache.org/jira/browse/HBASE-16890>, making good
>>> progress.
>>>
>>>
>>
>> Good progress here. In perf testing asyncfswal makes us generally faster.
>> Duo making good progress on last few tests that are in the way of our
>> making this default for hbase2.
>>
>> A load of goodness has been landing these last few weeks. Keep up the
>> great work.
>>
>> Your hbase-2 RM.
>>
>>
>>
>>
>>> I punted HBASE-19147 "All branch-2 unit tests pass" out to beta-2. Our
>>> unit test story has gotten better and a few of us are actively working on
>>> flakies [2] but we may make a beta-1 w/o shutting down all failures.
>>>
>>> Speak up if you need help on an issue or if you think we are missing
>>> items from our list.
>>>
>>> Thanks for all the hard work,
>>>
>>> The hbase-2.0.0 RM.
>>>
>>>
>>> 1. https://issues.apache.org/jira/projects/HBASE/versions/12340861
>>> 2. https://builds.apache.org/job/HBASE-Find-Flaky-Tests/last
>>> SuccessfulBuild/artifact/dashboard.html
>>>
>>> On Thu, Nov 16, 2017 at 3:48 PM, Stack <st...@duboce.net> wrote:
>>>
>>>> hbase-2.0.0-beta-1 update (Reminder, beta-1 is where we finish last
>>>> remaining features and apply final polish to API. There will be a beta-2
>>>> but it is about upgrade/rolling-upgrade and bug fixes ONLY).
>>>>
>>>> Myself and Mr. Drob did a pass over the outstanding hbase-2.0.0-beta-1
>>>> list this morning. See here [1].
>>>>
>>>> There are about ~12 issues in progress with most of these about to
>>>> land. There are 37 TODO. Many of these are tests we need to run, some are
>>>> related to the backup/restore, but a good few are meaty w/o assignees.
>>>>
>>>> The awkward outstanding ones as I see it are the below:
>>>>
>>>> HBASE-18946 Stochastic load balancer assigns replica regions to the
>>>> same RS
>>>> HBASE-17204 Make L2 off heap cache default ON
>>>> HBASE-19112 Suspect methods on Cell to be deprecated
>>>> HBASE-19147 All branch-2 unit tests pass
>>>>
>>>> We need to make progress on the above or punt on them (can't punt on
>>>> the last one though).
>>>>
>>>> Any ideas on what configs we should update in hbase2? Dump ideas into:
>>>> HBASE-19148 Edit of default configuration
>>>>
>>>> Still hoping for an early December beta-1 RC. beta-2 hopefully will be
>>>> close behind.
>>>>
>>>> Comments? Thoughts?
>>>>
>>>> Thanks all,
>>>> S
>>>>
>>>> 1. https://issues.apache.org/jira/projects/HBASE/versions/12340861
>>>>
>>>> On Tue, Oct 31, 2017 at 2:56 PM, Stack <st...@duboce.net> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Tue, Oct 31, 2017 at 2:31 PM, Mike Drob <md...@apache.org> wrote:
>>>>>
>>>>>> Hoping to keep momentum going from our Stack working on alpha4, I
>>>>>> tried to
>>>>>> take a stab at triaging some of the open beta-1 issues.
>>>>>>
>>>>>> I moved some docs stuff out form beta-1 to 2.0-GA, if it gets done
>>>>>> sooner
>>>>>> then I'm happy to see it pulled back in. Trying to balance optimism
>>>>>> with
>>>>>> realism here, and knowing that documentation unfortunately often gets
>>>>>> pushed to the back-burner.
>>>>>>
>>>>>> Also, I poked some folks on unassigned issues that they've filed for
>>>>>> beta-1, especially in the last few days. If issues don't have an
>>>>>> owner they
>>>>>> are unlikely to get worked. I chatted with stack and he agreed to
>>>>>> take on
>>>>>> some of the tasks, but there's a lot of surface area to cover.
>>>>>>
>>>>>> If you you're working on issues that are critical for beta-1, please
>>>>>> mark
>>>>>> them as such. Then the rest of the community will know to help
>>>>>> prioritize
>>>>>> feedback and reviews there.
>>>>>>
>>>>>> Do we have a general theme for the betas like we did with the alphas?
>>>>>>
>>>>>> Beta1 is upgrades work from branch1, beta2 is rolling upgrades work as
>>>>>> well? Continue to work on tests throughout?
>>>>>>
>>>>>>
>>>>> Thanks Mike for helping to kick off the beta-1 train.
>>>>>
>>>>> Regards a theme for beta-1, I'd like it to be 'finish'; beta-1 is what
>>>>> we are going to ship (beta-2 is rolling upgrade and any minor items turned
>>>>> up in testing/burn-in).
>>>>>
>>>>> Thanks,
>>>>> S
>>>>>
>>>>>
>>>>>
>>>>>> Mike
>>>>>>
>>>>>> On Tue, Oct 31, 2017 at 10:04 AM, Josh Elser <el...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>> > +1 go from my POV.
>>>>>> >
>>>>>> >
>>>>>> > On 10/31/17 10:07 AM, Stack wrote:
>>>>>> >
>>>>>> >> I want to push an alpha-4 today. A few items didn't make it
>>>>>> (HBASE-19092).
>>>>>> >> They need more time. We'll pull them in for beta-1. CP API is
>>>>>> basically
>>>>>> >> done. There may be some changes for beta-1 but hopefully only
>>>>>> changes
>>>>>> >> informed by experience trying to port an existing Coprocessor to
>>>>>> hbase2.
>>>>>> >>
>>>>>> >> Shout if there is anything that needs to make alpha-4.
>>>>>> >>
>>>>>> >> Thanks,
>>>>>> >> St.Ack
>>>>>> >>
>>>>>> >>
>>>>>> >> On Sat, Oct 28, 2017 at 2:48 PM, Josh Elser <jo...@gmail.com>
>>>>>> wrote:
>>>>>> >>
>>>>>> >> Yup, that was going to be my plan, Mike!
>>>>>> >>>
>>>>>> >>> Making a pass now, and will check back later tonight again. I see
>>>>>> others
>>>>>> >>> have already done some work today on this front.
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> On 10/27/17 11:38 PM, Mike Drob wrote:
>>>>>> >>>
>>>>>> >>> Josh - Do you want to kick off a bunch of QA runs? (Do you know
>>>>>> how to do
>>>>>> >>>> it directly on the jenkins job, so you don't have to bother with
>>>>>> JIRA
>>>>>> >>>> uploads)
>>>>>> >>>>
>>>>>> >>>> If you're busy, then I can make time tomorrow or Sunday to kick
>>>>>> off
>>>>>> >>>> jobs,
>>>>>> >>>> but I want to make sure we're not duplicating effort and jenkins
>>>>>> cycles.
>>>>>> >>>>
>>>>>> >>>> On Fri, Oct 27, 2017 at 7:10 PM, Josh Elser <el...@apache.org>
>>>>>> wrote:
>>>>>> >>>>
>>>>>> >>>> My turn to bump ;)
>>>>>> >>>>
>>>>>> >>>>>
>>>>>> >>>>> By my take: HBASE-18770 and HBASE-19092 are the only issues
>>>>>> that remain
>>>>>> >>>>> needing some more work. The rest are just awaiting a good QA
>>>>>> run.
>>>>>> >>>>>
>>>>>> >>>>> Unless I hear otherwise, I'll try to keep an eye on things over
>>>>>> the
>>>>>> >>>>> weekend, bump them along as necessary, and get them committed.
>>>>>> Would be
>>>>>> >>>>> great to be able get a vote up on Monday.
>>>>>> >>>>>
>>>>>> >>>>>
>>>>>> >>>>> On 10/24/17 6:03 PM, Stack wrote:
>>>>>> >>>>>
>>>>>> >>>>> Chatting with my coworker Mr. Mike Drob, we were batting back
>>>>>> and forth
>>>>>> >>>>>
>>>>>> >>>>>> what remains to be done. Surfacing our thoughts here so you
>>>>>> all clued
>>>>>> >>>>>> in....Or if you think otherwise, please speak up.
>>>>>> >>>>>>
>>>>>> >>>>>> We have ~13 issues to land:
>>>>>> >>>>>> https://issues.apache.org/jira/projects/HBASE/versions/12341
>>>>>> 594 About
>>>>>> >>>>>> two
>>>>>> >>>>>> are meta-issues that are about process which leaves 11.
>>>>>> >>>>>>
>>>>>> >>>>>> Duo and Zheng Hu are to merge the FilterList fixes improvements
>>>>>> >>>>>> (HBASE-19057, HBASE-18410 et al.). These are blocker because
>>>>>> some
>>>>>> >>>>>> changed
>>>>>> >>>>>> API/semantic that we need to get out earlier rather than later.
>>>>>> >>>>>>
>>>>>> >>>>>> Once the above is merged, HBASE-13346, change of Filter method
>>>>>> names
>>>>>> >>>>>> to
>>>>>> >>>>>> mention Cell instead of KeyValue can land.
>>>>>> >>>>>>
>>>>>> >>>>>> HBASE-199048 needs a review (Anoop will probably do it),
>>>>>> removing
>>>>>> >>>>>> IA.Private objects as params to MasterObserver... Hopefully
>>>>>> this goes
>>>>>> >>>>>> in
>>>>>> >>>>>> soon.
>>>>>> >>>>>>
>>>>>> >>>>>> Duo is hard at work on trackers for flush and compaction for
>>>>>> CPs
>>>>>> >>>>>> (HBASE-18905). How is HBASE-19033 looking Duo (facility for
>>>>>> Tephra)?
>>>>>> >>>>>>
>>>>>> >>>>>> I think HBASE-18906 (Phoenix Region#waitFor...) will evaporate
>>>>>> after
>>>>>> >>>>>> Duo
>>>>>> >>>>>> is
>>>>>> >>>>>> done w/ his work above.
>>>>>> >>>>>>
>>>>>> >>>>>> I'm on HBASE-18770 bypass and HBASE-19077 restore some parity
>>>>>> after
>>>>>> >>>>>> all
>>>>>> >>>>>> the
>>>>>> >>>>>> purges allowing CPs do direct calls against Regions in same
>>>>>> Host.
>>>>>> >>>>>>
>>>>>> >>>>>> Anoop is on HBASE-19047 (Fixes) and Ram on cleanup of CellUtil.
>>>>>> >>>>>>
>>>>>> >>>>>> Another day or two?
>>>>>> >>>>>>
>>>>>> >>>>>> St.Ack
>>>>>> >>>>>>
>>>>>> >>>>>>
>>>>>> >>>>>>
>>>>>> >>>>>> On Mon, Oct 23, 2017 at 2:52 PM, Stack <st...@duboce.net>
>>>>>> wrote:
>>>>>> >>>>>>
>>>>>> >>>>>>
>>>>>> >>>>>> On Mon, Oct 23, 2017 at 11:59 AM, Josh Elser <
>>>>>> elserj@apache.org>
>>>>>> >>>>>> wrote:
>>>>>> >>>>>>
>>>>>> >>>>>>>
>>>>>> >>>>>>> +1
>>>>>> >>>>>>>
>>>>>> >>>>>>>
>>>>>> >>>>>>>> I was trying to work on helping out on the outstanding
>>>>>> alpha-4 stuff
>>>>>> >>>>>>>> last
>>>>>> >>>>>>>> week -- will be continuing to try to do the same this week.
>>>>>> >>>>>>>>
>>>>>> >>>>>>>> If you need any help, Stack, or if others need reviews where
>>>>>> I
>>>>>> >>>>>>>> haven't
>>>>>> >>>>>>>> noticed on my own: feel free to @mention me.
>>>>>> >>>>>>>>
>>>>>> >>>>>>>>
>>>>>> >>>>>>>> Thanks for the offer Josh. All items seem assigned and are
>>>>>> being
>>>>>> >>>>>>>>
>>>>>> >>>>>>>> actively
>>>>>> >>>>>>> worked on. If you get a moment, reviews by you (or anyone
>>>>>> else) helps
>>>>>> >>>>>>> move
>>>>>> >>>>>>> the process along.
>>>>>> >>>>>>>
>>>>>> >>>>>>> We need to merge in HBASE-18410 branch to pick up Filter
>>>>>> >>>>>>> improvements.
>>>>>> >>>>>>> Then HBASE-13346 can go in.
>>>>>> >>>>>>>
>>>>>> >>>>>>> You are already helping out on HBASE-18906, thanks. Looks
>>>>>> like that
>>>>>> >>>>>>> will
>>>>>> >>>>>>> be addressed by other alpha-4s about to land.
>>>>>> >>>>>>>
>>>>>> >>>>>>> St.Ack
>>>>>> >>>>>>> TODOs: https://issues.apache.org/jira
>>>>>> /projects/HBASE/versions/12341
>>>>>> >>>>>>> 594
>>>>>> >>>>>>>
>>>>>> >>>>>>>
>>>>>> >>>>>>>
>>>>>> >>>>>>>
>>>>>> >>>>>>>
>>>>>> >>>>>>>
>>>>>> >>>>>>>
>>>>>> >>>>>>>
>>>>>> >>>>>>>
>>>>>> >>>>>>> On 10/23/17 12:53 PM, Stack wrote:
>>>>>> >>>>>>>
>>>>>> >>>>>>>
>>>>>> >>>>>>>> (Reviving this thread)
>>>>>> >>>>>>>>
>>>>>> >>>>>>>>
>>>>>> >>>>>>>>> Lets push out alpha-4 this week. Alpha-4 is the release
>>>>>> that has
>>>>>> >>>>>>>>> the
>>>>>> >>>>>>>>> refactor of the Coprocessor API shutting down access to
>>>>>> internals
>>>>>> >>>>>>>>> marked
>>>>>> >>>>>>>>> InterfaceAudience.Private.
>>>>>> >>>>>>>>>
>>>>>> >>>>>>>>> The outstanding list is here:
>>>>>> >>>>>>>>> https://issues.apache.org/jira
>>>>>> /projects/HBASE/versions/12341594
>>>>>> >>>>>>>>>
>>>>>> >>>>>>>>> Please push in anything marked alpha-4 that belongs to you.
>>>>>> >>>>>>>>>
>>>>>> >>>>>>>>> If issue, talk out loud on this thread. If you need a
>>>>>> review to
>>>>>> >>>>>>>>> land
>>>>>> >>>>>>>>> an
>>>>>> >>>>>>>>> item, shout on the issue and here; we'll help you out.
>>>>>> >>>>>>>>>
>>>>>> >>>>>>>>> As is, items are coming along nicely I'd say. We need to
>>>>>> merge the
>>>>>> >>>>>>>>> filter
>>>>>> >>>>>>>>> branch -- HBASE-18410 -- so APIs are finished for hbase2.
>>>>>> >>>>>>>>>
>>>>>> >>>>>>>>> Post alpha-4, we'll have to hunt down our downstreamers and
>>>>>> help
>>>>>> >>>>>>>>> them
>>>>>> >>>>>>>>> test
>>>>>> >>>>>>>>> on top of alpha-4 so rolling into beta-1, we have
>>>>>> confidence our
>>>>>> >>>>>>>>> downstreamers know what to expect (or we discover what we
>>>>>> missed
>>>>>> >>>>>>>>> BEFORE
>>>>>> >>>>>>>>> we
>>>>>> >>>>>>>>> beta-1).
>>>>>> >>>>>>>>>
>>>>>> >>>>>>>>> Thanks for time,
>>>>>> >>>>>>>>> S
>>>>>> >>>>>>>>>
>>>>>> >>>>>>>>>
>>>>>> >>>>>>>>>
>>>>>> >>>>>>>>>
>>>>>> >>>>>>>>>
>>>>>> >>>>>>>>> On Fri, Sep 8, 2017 at 2:04 PM, Stack <st...@duboce.net>
>>>>>> wrote:
>>>>>> >>>>>>>>>
>>>>>> >>>>>>>>> I'll put up an alpha3 RC Monday, probably Monday night.
>>>>>> That should
>>>>>> >>>>>>>>> be
>>>>>> >>>>>>>>>
>>>>>> >>>>>>>>> time, if we all sprint, for the public-facing API fixes to
>>>>>> be done.
>>>>>> >>>>>>>>>
>>>>>> >>>>>>>>>>
>>>>>> >>>>>>>>>> I had a bunch of Coprocessor refactor and fixup scheduled
>>>>>> for
>>>>>> >>>>>>>>>> alpha3
>>>>>> >>>>>>>>>> but
>>>>>> >>>>>>>>>> it is plain that more time is needed (in spite of valiant
>>>>>> effort
>>>>>> >>>>>>>>>> so
>>>>>> >>>>>>>>>> far
>>>>>> >>>>>>>>>> by
>>>>>> >>>>>>>>>> Anoop, Duo, Appy, etc.). Therefore, lets run a
>>>>>> 2.0.0-alpha-4 whose
>>>>>> >>>>>>>>>> theme is
>>>>>> >>>>>>>>>> "Coprocessor Fixup". Hopefully we can put an alpha-4 up by
>>>>>> the
>>>>>> >>>>>>>>>> following
>>>>>> >>>>>>>>>> week.
>>>>>> >>>>>>>>>>
>>>>>> >>>>>>>>>> We should then be ready for beta (beta == no new features,
>>>>>> no API
>>>>>> >>>>>>>>>> changes,
>>>>>> >>>>>>>>>> just fixes).
>>>>>> >>>>>>>>>>
>>>>>> >>>>>>>>>> Thanks,
>>>>>> >>>>>>>>>> St.Ack
>>>>>> >>>>>>>>>>
>>>>>> >>>>>>>>>>
>>>>>> >>>>>>>>>> On Thu, Aug 17, 2017 at 12:35 PM, Stack <st...@duboce.net>
>>>>>> wrote:
>>>>>> >>>>>>>>>>
>>>>>> >>>>>>>>>> I put up the hbase-2.0.0-alpha2 release candidate. Please
>>>>>> vote on
>>>>>> >>>>>>>>>> it.
>>>>>> >>>>>>>>>>
>>>>>> >>>>>>>>>>
>>>>>> >>>>>>>>>> For hbase-2.0.0-alpha3, the theme is solidifying API. I
>>>>>> hope to
>>>>>> >>>>>>>>>>> get a
>>>>>> >>>>>>>>>>> release out in the next week or so.
>>>>>> >>>>>>>>>>>
>>>>>> >>>>>>>>>>> I did a weeding of 2.0.0 issues over the last day. If
>>>>>> folks are
>>>>>> >>>>>>>>>>> interested in helping out, below are the items I think we
>>>>>> need
>>>>>> >>>>>>>>>>> done
>>>>>> >>>>>>>>>>> for
>>>>>> >>>>>>>>>>> alpha3 (below are at least 'Critical' status, are API
>>>>>> possibly
>>>>>> >>>>>>>>>>> altering
>>>>>> >>>>>>>>>>> items, and are absent those JIRAs that are making active
>>>>>> >>>>>>>>>>> progress,
>>>>>> >>>>>>>>>>> i.e. the
>>>>>> >>>>>>>>>>> HTD/HCD revamp by Chia-Ping Tsai). A project NOT listed
>>>>>> that
>>>>>> >>>>>>>>>>> needs
>>>>>> >>>>>>>>>>> doing is
>>>>>> >>>>>>>>>>> what Andrew did comparing 1.3. and 1.4 APIs
>>>>>> >>>>>>>>>>>
>>>>>> >>>>>>>>>>> * HBASE-18622 Mitigate compatibility concerns between
>>>>>> branch-1
>>>>>> >>>>>>>>>>> and
>>>>>> >>>>>>>>>>> branch-2
>>>>>> >>>>>>>>>>> This is to do what Andrew did between 1.3 and 1.4
>>>>>> branches only
>>>>>> >>>>>>>>>>> do
>>>>>> >>>>>>>>>>> it
>>>>>> >>>>>>>>>>> between branch-1 and branch-2.
>>>>>> >>>>>>>>>>>
>>>>>> >>>>>>>>>>> * HBASE-10462 Recategorize some of the client facing
>>>>>> Public /
>>>>>> >>>>>>>>>>> Private
>>>>>> >>>>>>>>>>> interfaces
>>>>>> >>>>>>>>>>> This one is almost done. It could do with a finish,
>>>>>> attention to
>>>>>> >>>>>>>>>>> the
>>>>>> >>>>>>>>>>> items in last comment, and then our codebase could do with
>>>>>> >>>>>>>>>>> another
>>>>>> >>>>>>>>>>> sweep
>>>>>> >>>>>>>>>>> after the spirit of this issue since a bunch has gone in
>>>>>> since
>>>>>> >>>>>>>>>>> the
>>>>>> >>>>>>>>>>> pass
>>>>>> >>>>>>>>>>> that was the basis of this issue.
>>>>>> >>>>>>>>>>>
>>>>>> >>>>>>>>>>> * HBASE-10504 Define Replication Interface
>>>>>> >>>>>>>>>>> I was going to take a crack at this as part of the revamp
>>>>>> forced
>>>>>> >>>>>>>>>>> by
>>>>>> >>>>>>>>>>> 'HBASE-15982 Interface ReplicationEndpoint extends Guava's
>>>>>> >>>>>>>>>>> Service'
>>>>>> >>>>>>>>>>> but if
>>>>>> >>>>>>>>>>> anyone else is interested, be my guest.
>>>>>> >>>>>>>>>>>
>>>>>> >>>>>>>>>>> * HBASE-14996 Some more API cleanup for 2.0
>>>>>> >>>>>>>>>>> Has a bunch of subtasks, some of which are being worked
>>>>>> on. Needs
>>>>>> >>>>>>>>>>> finishing.
>>>>>> >>>>>>>>>>>
>>>>>> >>>>>>>>>>> * HBASE-14998 Unify synchronous and asynchronous methods
>>>>>> in Admin
>>>>>> >>>>>>>>>>> and
>>>>>> >>>>>>>>>>> cleanup
>>>>>> >>>>>>>>>>> Needs a pass. Small issue I think. Could also look at new
>>>>>> >>>>>>>>>>> AsyncClient
>>>>>> >>>>>>>>>>> and
>>>>>> >>>>>>>>>>> make sure symmetry.
>>>>>> >>>>>>>>>>>
>>>>>> >>>>>>>>>>> * HBASE-15607 Remove PB references from Admin for 2.0
>>>>>> >>>>>>>>>>> Predicated on result of an ongoing DISCUSSION thread but
>>>>>> needs to
>>>>>> >>>>>>>>>>> be
>>>>>> >>>>>>>>>>> done.
>>>>>> >>>>>>>>>>>
>>>>>> >>>>>>>>>>> Rolling upgrade will have implications for our API. Would
>>>>>> be good
>>>>>> >>>>>>>>>>> to
>>>>>> >>>>>>>>>>> try
>>>>>> >>>>>>>>>>> it and figure what needs fixup (as said above, according
>>>>>> to trial
>>>>>> >>>>>>>>>>> by
>>>>>> >>>>>>>>>>> Sean,
>>>>>> >>>>>>>>>>> we might not be too bad here):
>>>>>> >>>>>>>>>>> * HBASE-16060 1.x clients cannot access table state
>>>>>> talking to
>>>>>> >>>>>>>>>>> 2.0
>>>>>> >>>>>>>>>>> cluster
>>>>>> >>>>>>>>>>> * HBASE-16550 Procedure v2 - Add AM compatibility for 2.x
>>>>>> Master
>>>>>> >>>>>>>>>>> and
>>>>>> >>>>>>>>>>> 1.x
>>>>>> >>>>>>>>>>> RSs; i.e. support Rolling Upgrade from hbase-1 to -2.
>>>>>> >>>>>>>>>>>
>>>>>> >>>>>>>>>>> * HBASE-17442 Move most of the replication related
>>>>>> classes to
>>>>>> >>>>>>>>>>> hbase-server package
>>>>>> >>>>>>>>>>> The above would be good to do generally but it may make
>>>>>> for
>>>>>> >>>>>>>>>>> ripples
>>>>>> >>>>>>>>>>> in
>>>>>> >>>>>>>>>>> API so would be good to do now.
>>>>>> >>>>>>>>>>>
>>>>>> >>>>>>>>>>> * HBASE-18106 Redo ProcedureInfo and LockInfo
>>>>>> >>>>>>>>>>> Balazs is working on this. The idea is that we avoid
>>>>>> adding two
>>>>>> >>>>>>>>>>> new
>>>>>> >>>>>>>>>>> types
>>>>>> >>>>>>>>>>> to our API, two types that are nought but curtailed,
>>>>>> read-only
>>>>>> >>>>>>>>>>> views
>>>>>> >>>>>>>>>>> on
>>>>>> >>>>>>>>>>> internals. Input if you have time appreciated.
>>>>>> >>>>>>>>>>>
>>>>>> >>>>>>>>>>> * HBASE-18596 A hbase1 cluster should be able to
>>>>>> replicate to a
>>>>>> >>>>>>>>>>> hbase2
>>>>>> >>>>>>>>>>> cluster; verify
>>>>>> >>>>>>>>>>> Esteban is looking at this one
>>>>>> >>>>>>>>>>>
>>>>>> >>>>>>>>>>> * HBASE-9417 SecureBulkLoadEndpoint should be folded in
>>>>>> core
>>>>>> >>>>>>>>>>> * HBASE-17143 Scan improvement
>>>>>> >>>>>>>>>>>
>>>>>> >>>>>>>>>>> Our Coprocessor Interface needs a tough edit. It exposes
>>>>>> >>>>>>>>>>> implementations
>>>>>> >>>>>>>>>>> marked audience Private and returns implementations
>>>>>> rather than
>>>>>> >>>>>>>>>>> Interfaces.
>>>>>> >>>>>>>>>>> In a few locations, we allow returning an alternate
>>>>>> >>>>>>>>>>> implementation
>>>>>> >>>>>>>>>>> altogether which is probably something we don't want a CP
>>>>>> doing.
>>>>>> >>>>>>>>>>> To
>>>>>> >>>>>>>>>>> that
>>>>>> >>>>>>>>>>> end, the following issues started by Duo and Anoop need
>>>>>> to be
>>>>>> >>>>>>>>>>> taken
>>>>>> >>>>>>>>>>> to
>>>>>> >>>>>>>>>>> the
>>>>>> >>>>>>>>>>> finish line; ideally they'd have an owner:
>>>>>> >>>>>>>>>>>
>>>>>> >>>>>>>>>>> * HBASE-18169 Coprocessor fix and cleanup before 2.0.0
>>>>>> release <=
>>>>>> >>>>>>>>>>> The
>>>>>> >>>>>>>>>>> umbrella issue.
>>>>>> >>>>>>>>>>> * HBASE-18298 RegionServerServices Interface cleanup for
>>>>>> CP
>>>>>> >>>>>>>>>>> expose
>>>>>> >>>>>>>>>>> * HBASE-16769 Deprecate/remove PB references from
>>>>>> MasterObserver
>>>>>> >>>>>>>>>>> and
>>>>>> >>>>>>>>>>> RegionServerObserver
>>>>>> >>>>>>>>>>>
>>>>>> >>>>>>>>>>>
>>>>>> >>>>>>>>>>> Nice-to-haves:
>>>>>> >>>>>>>>>>>
>>>>>> >>>>>>>>>>> * HBASE-15284 Make TimeRange constructors IA.Private and
>>>>>> remove
>>>>>> >>>>>>>>>>> unused
>>>>>> >>>>>>>>>>> TimeRange constructors
>>>>>> >>>>>>>>>>>
>>>>>> >>>>>>>>>>> * HBASE-10944 Remove all kv.getBuffer() and kv.getRow()
>>>>>> >>>>>>>>>>> references
>>>>>> >>>>>>>>>>> existing in the code
>>>>>> >>>>>>>>>>> This is the end of an old long-running project moving up
>>>>>> on to
>>>>>> >>>>>>>>>>> Cell
>>>>>> >>>>>>>>>>> Interface. We think it is done but for a few little items
>>>>>> >>>>>>>>>>> (deprecate
>>>>>> >>>>>>>>>>> KV
>>>>>> >>>>>>>>>>> methods in MR and provide Cell versions instead...)
>>>>>> >>>>>>>>>>>
>>>>>> >>>>>>>>>>> * HBASE-13271 Table#puts(List<Put>) operation is
>>>>>> indeterminate;
>>>>>> >>>>>>>>>>> needs
>>>>>> >>>>>>>>>>> fixing
>>>>>> >>>>>>>>>>>
>>>>>> >>>>>>>>>>> * HBASE-13346 Clean up Filter package for post 1.0
>>>>>> >>>>>>>>>>>
>>>>>> >>>>>>>>>>> * HBASE-14255 Simplify Cell creation post 1.0
>>>>>> >>>>>>>>>>> * HBASE-14997
>>>>>> >>>>>>>>>>> Move compareOp and Comparators out of filter to client
>>>>>> package
>>>>>> >>>>>>>>>>>
>>>>>> >>>>>>>>>>> * HBASE-13740 Stop using Hadoop private interfaces
>>>>>> >>>>>>>>>>>
>>>>>> >>>>>>>>>>> What about:
>>>>>> >>>>>>>>>>>
>>>>>> >>>>>>>>>>> * HBASE-18601 Remove Htrace 3.2
>>>>>> >>>>>>>>>>> As has been noted, the HTrace API is our 'trace' API.
>>>>>> >>>>>>>>>>>
>>>>>> >>>>>>>>>>> If interested in any of the above and you need a legup,
>>>>>> just ask
>>>>>> >>>>>>>>>>> in
>>>>>> >>>>>>>>>>> the
>>>>>> >>>>>>>>>>> issue and I'll be by....
>>>>>> >>>>>>>>>>>
>>>>>> >>>>>>>>>>> Thanks,
>>>>>> >>>>>>>>>>> St.Ack
>>>>>> >>>>>>>>>>>
>>>>>> >>>>>>>>>>>
>>>>>> >>>>>>>>>>>
>>>>>> >>>>>>>>>>> On Mon, Aug 14, 2017 at 10:54 AM, Stack <stack@duboce.net
>>>>>> >
>>>>>> >>>>>>>>>>> wrote:
>>>>>> >>>>>>>>>>>
>>>>>> >>>>>>>>>>> Heads-up:
>>>>>> >>>>>>>>>>>
>>>>>> >>>>>>>>>>>
>>>>>> >>>>>>>>>>> I'm about to put up an hbase-2.0.0-alpha2 Release
>>>>>> Candidate.
>>>>>> >>>>>>>>>>>> Theme
>>>>>> >>>>>>>>>>>> is
>>>>>> >>>>>>>>>>>> updated dependencies, reliance on relocated popular libs
>>>>>> (guava,
>>>>>> >>>>>>>>>>>> netty,
>>>>>> >>>>>>>>>>>> protobuf), purge of checked-in generated src, and
>>>>>> >>>>>>>>>>>> master-carries-no-regions
>>>>>> >>>>>>>>>>>> by default.
>>>>>> >>>>>>>>>>>>
>>>>>> >>>>>>>>>>>> alpha3 I hope will follow soon after (end-of-August?).
>>>>>> Its theme
>>>>>> >>>>>>>>>>>> will
>>>>>> >>>>>>>>>>>> be
>>>>>> >>>>>>>>>>>> settling the APIs and compatibility (At first blush, we
>>>>>> are not
>>>>>> >>>>>>>>>>>> looking too
>>>>>> >>>>>>>>>>>> bad; our Sean ran some tests over weekend that have
>>>>>> hbase-1
>>>>>> >>>>>>>>>>>> client
>>>>>> >>>>>>>>>>>> running
>>>>>> >>>>>>>>>>>> against an hbase-2 cluster....). The Coprocessor
>>>>>> Interface
>>>>>> >>>>>>>>>>>> revamp
>>>>>> >>>>>>>>>>>> should be
>>>>>> >>>>>>>>>>>> done by alpha3 (i.e. returning Interfaces rather than
>>>>>> >>>>>>>>>>>> Implementations, and
>>>>>> >>>>>>>>>>>> our shutdown of CPs accessing classes in hbase marked
>>>>>> >>>>>>>>>>>> InterfaceAudience).
>>>>>> >>>>>>>>>>>> We'll also have purged thirdparty classes from our API;
>>>>>> e.g.
>>>>>> >>>>>>>>>>>> guava
>>>>>> >>>>>>>>>>>> 0.12
>>>>>> >>>>>>>>>>>> Service showing through in our replication API and
>>>>>> protobufs in
>>>>>> >>>>>>>>>>>> Admin
>>>>>> >>>>>>>>>>>> Interface. On alpha3, we will have to do a bunch of
>>>>>> outreach to
>>>>>> >>>>>>>>>>>> make
>>>>>> >>>>>>>>>>>> sure
>>>>>> >>>>>>>>>>>> our downstreamers are up on what is coming down the pipe.
>>>>>> >>>>>>>>>>>>
>>>>>> >>>>>>>>>>>> Beta1 in mid-September?
>>>>>> >>>>>>>>>>>>
>>>>>> >>>>>>>>>>>> I encourage you to check out the items marked for hbase2:
>>>>>> >>>>>>>>>>>> https://issues.apache.org/jira
>>>>>> /projects/HBASE/versions/12327188
>>>>>> >>>>>>>>>>>> Edit
>>>>>> >>>>>>>>>>>> as
>>>>>> >>>>>>>>>>>> you see appropriate. Punt if you know the JIRA will not
>>>>>> get any
>>>>>> >>>>>>>>>>>> attention
>>>>>> >>>>>>>>>>>> in next month or so.
>>>>>> >>>>>>>>>>>>
>>>>>> >>>>>>>>>>>> A bunch of issues marked blocker are unassigned. I'll
>>>>>> leave them
>>>>>> >>>>>>>>>>>> as
>>>>>> >>>>>>>>>>>> is
>>>>>> >>>>>>>>>>>> another while but I'll boot them soon.
>>>>>> >>>>>>>>>>>>
>>>>>> >>>>>>>>>>>> While I have your attention:
>>>>>> >>>>>>>>>>>>
>>>>>> >>>>>>>>>>>> + I think we should leave thrift version at 0.9.3.
>>>>>> Moving hbase
>>>>>> >>>>>>>>>>>> thrift
>>>>>> >>>>>>>>>>>> to 0.10.0 will break existing clients. The change is
>>>>>> easy enough
>>>>>> >>>>>>>>>>>> if
>>>>>> >>>>>>>>>>>> folks
>>>>>> >>>>>>>>>>>> need to upgrade their hbase thrift. See HBASE-18591.
>>>>>> >>>>>>>>>>>> + Upgrade from 0.94 is disallowed. You have to get to
>>>>>> 1.0 first
>>>>>> >>>>>>>>>>>> (0.98?).
>>>>>> >>>>>>>>>>>>
>>>>>> >>>>>>>>>>>> St.Ack
>>>>>> >>>>>>>>>>>>
>>>>>> >>>>>>>>>>>>
>>>>>> >>>>>>>>>>>>
>>>>>> >>>>>>>>>>>> On Wed, Aug 2, 2017 at 9:43 AM, Stack <st...@duboce.net>
>>>>>> wrote:
>>>>>> >>>>>>>>>>>>
>>>>>> >>>>>>>>>>>>
>>>>>> >>>>>>>>>>>>
>>>>>> >>>>>>>>>>>> On Tue, Aug 1, 2017 at 2:06 PM, Josh Elser <
>>>>>> elserj@apache.org>
>>>>>> >>>>>>>>>>>>
>>>>>> >>>>>>>>>>>>> wrote:
>>>>>> >>>>>>>>>>>>>
>>>>>> >>>>>>>>>>>>>
>>>>>> >>>>>>>>>>>>>
>>>>>> >>>>>>>>>>>>> On 7/31/17 9:00 AM, Stack wrote:
>>>>>> >>>>>>>>>>>>>
>>>>>> >>>>>>>>>>>>>>
>>>>>> >>>>>>>>>>>>>> On Mon, Jul 24, 2017 at 12:25 PM, Josh Elser<
>>>>>> >>>>>>>>>>>>>> elserj@apache.org>
>>>>>> >>>>>>>>>>>>>>
>>>>>> >>>>>>>>>>>>>> wrote:
>>>>>> >>>>>>>>>>>>>>
>>>>>> >>>>>>>>>>>>>>>
>>>>>> >>>>>>>>>>>>>>> ...
>>>>>> >>>>>>>>>>>>>>>
>>>>>> >>>>>>>>>>>>>>>
>>>>>> >>>>>>>>>>>>>>> I like the idea of this also hitting 2.0 as it would
>>>>>> make the
>>>>>> >>>>>>>>>>>>>>>
>>>>>> >>>>>>>>>>>>>>>> feature a
>>>>>> >>>>>>>>>>>>>>>> bit more "real", but am obviously a little nervous
>>>>>> (I have
>>>>>> >>>>>>>>>>>>>>>> no
>>>>>> >>>>>>>>>>>>>>>> reason
>>>>>> >>>>>>>>>>>>>>>> to be
>>>>>> >>>>>>>>>>>>>>>> nervous though). I am pretty happy with the feature
>>>>>> in terms
>>>>>> >>>>>>>>>>>>>>>> of
>>>>>> >>>>>>>>>>>>>>>> how
>>>>>> >>>>>>>>>>>>>>>> much it
>>>>>> >>>>>>>>>>>>>>>> is covered via testing.
>>>>>> >>>>>>>>>>>>>>>>
>>>>>> >>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/HBASE-17748
>>>>>> >>>>>>>>>>>>>>>>
>>>>>> >>>>>>>>>>>>>>>>
>>>>>> >>>>>>>>>>>>>>>>
>>>>>> >>>>>>>>>>>>>>>> Sounds good to me. Whats involved? Backport? If so,
>>>>>> +1 Josh.
>>>>>> >>>>>>>>>>>>>>>>
>>>>>> >>>>>>>>>>>>>>>>
>>>>>> >>>>>>>>>>>>>>>> Last think on space quota says that need doc too.
>>>>>> See 'Space
>>>>>> >>>>>>>>>>>>>>>>
>>>>>> >>>>>>>>>>>>>>> Quota' in
>>>>>> >>>>>>>>>>>>>>> here:
>>>>>> >>>>>>>>>>>>>>> https://docs.google.com/docume
>>>>>> nt/d/1WCsVlnHjJeKUcl7wHwqb4z9i
>>>>>> >>>>>>>>>>>>>>> Eu_ktczrlKHK8N4SZzs/edit#heading=h.wuw3a6jukzo5
>>>>>> >>>>>>>>>>>>>>> Does this little section need an update Josh?
>>>>>> >>>>>>>>>>>>>>>
>>>>>> >>>>>>>>>>>>>>> Thanks,
>>>>>> >>>>>>>>>>>>>>> S
>>>>>> >>>>>>>>>>>>>>>
>>>>>> >>>>>>>>>>>>>>>
>>>>>> >>>>>>>>>>>>>>> Yep, just a couple of cherry-picks. Good test
>>>>>> coverage and
>>>>>> >>>>>>>>>>>>>>> some
>>>>>> >>>>>>>>>>>>>>>
>>>>>> >>>>>>>>>>>>>>> docs
>>>>>> >>>>>>>>>>>>>> already included for 17748.  Happy to put that on my
>>>>>> plate if
>>>>>> >>>>>>>>>>>>>> you're good
>>>>>> >>>>>>>>>>>>>> with it. I can reasonably assume that no one is
>>>>>> against it :)
>>>>>> >>>>>>>>>>>>>>
>>>>>> >>>>>>>>>>>>>> I think I had knocked out docs for the "phase 1" stuff
>>>>>> before
>>>>>> >>>>>>>>>>>>>> we
>>>>>> >>>>>>>>>>>>>> merged it in from the original feature branch. I'll
>>>>>> double
>>>>>> >>>>>>>>>>>>>> check
>>>>>> >>>>>>>>>>>>>> and update
>>>>>> >>>>>>>>>>>>>> the gdoc. Perhaps this was just a timing thing.
>>>>>> >>>>>>>>>>>>>>
>>>>>> >>>>>>>>>>>>>>
>>>>>> >>>>>>>>>>>>>> Thanks Josh,
>>>>>> >>>>>>>>>>>>>>
>>>>>> >>>>>>>>>>>>>> S
>>>>>> >>>>>>>>>>>>>
>>>>>> >>>>>>>>>>>>>
>>>>>> >>>>>>>>>>>>>
>>>>>> >>>>>>>>>>>>>
>>>>>> >>>>>>>>>>>>>
>>>>>> >>>>>>>>>>>>
>>>>>> >>>>>>>>>>>>
>>>>>> >>>>>>>>>>>
>>>>>> >>>>>>>>>>
>>>>>> >>>>>>>>>
>>>>>> >>>>>>>
>>>>>> >>>>>>
>>>>>> >>>>
>>>>>> >>
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>

Re: Moving 2.0 forward

[Stack <st...@duboce.net>]

Heads-up, I'm working on cutting a beta-1. Shout if there is something you
think needs to make it in (The Chia-Ping Tsai cleanups look good). I'm
mainly waiting on the hbase-thirdparty changes to go in. The rest can wait
it.

Thanks,
S


On Wed, Dec 13, 2017 at 11:28 PM, Stack <st...@duboce.net> wrote:

> Update. See below.
>
> On Tue, Dec 5, 2017 at 10:43 AM, Stack <st...@duboce.net> wrote:
>
>> Update on hbase-2.0.0-beta-1. I'm going to put up a beta-1 RC before
>> Christmas day.
>>
>>
> Still aiming for an hbase2 beta-1 showing up in your christmas stocking.
>
>
>> Here is the list [1]. We're doing pretty good. We've fixed a load since
>> the last update including some nice cleanups  two weeks ago (e.g. undoing
>> client dependency on curator/zk watching making the dependence read-only)
>> but there are still a few hairy ones hanging out there.
>>
>>  * HBASE-18946 Stochastic load balancer assigns replica regions to the
>> same RS
>>    This one is proving a little tough. Ram has been banging his head on
>> it. We'll be able to clear a bunch of test failures once this is done.
>>
>>
> This should be going in tomorrow.
>
>
>
>>  * HBASE-19112 Suspect methods on Cell to be deprecated
>>    Good discussion up on the JIRA. We need to be crystal clear around
>> Cell and derivatives API and Audience when we ship 2.0.
>>
>>
> Ram, Chia-Ping Tsai, and Anoop are doing a nice job here and it should be
> done in next day or so.
>
>
>
>>  * HBASE-17204 Make L2 off heap cache default ON
>>    We need to try this at least so offheaping can be default.
>>
>>
> Chatted with Anoop and Ram. Because it is so late in the day and because
> it critical that the user have a good experience out of the box -- which
> requires time and testing -- we are going to punt this from 2.0. Will look
> at it for 2.1 (or 3.0 if soon). 2.0 will still be carrying all of the boys
> offheap read/write path goodness.
>
>
>
>> Making asyncfswal default is also ongoing, HBASE-16890
>> <https://issues.apache.org/jira/browse/HBASE-16890>, making good
>> progress.
>>
>>
>
> Good progress here. In perf testing asyncfswal makes us generally faster.
> Duo making good progress on last few tests that are in the way of our
> making this default for hbase2.
>
> A load of goodness has been landing these last few weeks. Keep up the
> great work.
>
> Your hbase-2 RM.
>
>
>
>
>> I punted HBASE-19147 "All branch-2 unit tests pass" out to beta-2. Our
>> unit test story has gotten better and a few of us are actively working on
>> flakies [2] but we may make a beta-1 w/o shutting down all failures.
>>
>> Speak up if you need help on an issue or if you think we are missing
>> items from our list.
>>
>> Thanks for all the hard work,
>>
>> The hbase-2.0.0 RM.
>>
>>
>> 1. https://issues.apache.org/jira/projects/HBASE/versions/12340861
>> 2. https://builds.apache.org/job/HBASE-Find-Flaky-Tests/last
>> SuccessfulBuild/artifact/dashboard.html
>>
>> On Thu, Nov 16, 2017 at 3:48 PM, Stack <st...@duboce.net> wrote:
>>
>>> hbase-2.0.0-beta-1 update (Reminder, beta-1 is where we finish last
>>> remaining features and apply final polish to API. There will be a beta-2
>>> but it is about upgrade/rolling-upgrade and bug fixes ONLY).
>>>
>>> Myself and Mr. Drob did a pass over the outstanding hbase-2.0.0-beta-1
>>> list this morning. See here [1].
>>>
>>> There are about ~12 issues in progress with most of these about to land.
>>> There are 37 TODO. Many of these are tests we need to run, some are related
>>> to the backup/restore, but a good few are meaty w/o assignees.
>>>
>>> The awkward outstanding ones as I see it are the below:
>>>
>>> HBASE-18946 Stochastic load balancer assigns replica regions to the same
>>> RS
>>> HBASE-17204 Make L2 off heap cache default ON
>>> HBASE-19112 Suspect methods on Cell to be deprecated
>>> HBASE-19147 All branch-2 unit tests pass
>>>
>>> We need to make progress on the above or punt on them (can't punt on the
>>> last one though).
>>>
>>> Any ideas on what configs we should update in hbase2? Dump ideas into:
>>> HBASE-19148 Edit of default configuration
>>>
>>> Still hoping for an early December beta-1 RC. beta-2 hopefully will be
>>> close behind.
>>>
>>> Comments? Thoughts?
>>>
>>> Thanks all,
>>> S
>>>
>>> 1. https://issues.apache.org/jira/projects/HBASE/versions/12340861
>>>
>>> On Tue, Oct 31, 2017 at 2:56 PM, Stack <st...@duboce.net> wrote:
>>>
>>>>
>>>>
>>>> On Tue, Oct 31, 2017 at 2:31 PM, Mike Drob <md...@apache.org> wrote:
>>>>
>>>>> Hoping to keep momentum going from our Stack working on alpha4, I
>>>>> tried to
>>>>> take a stab at triaging some of the open beta-1 issues.
>>>>>
>>>>> I moved some docs stuff out form beta-1 to 2.0-GA, if it gets done
>>>>> sooner
>>>>> then I'm happy to see it pulled back in. Trying to balance optimism
>>>>> with
>>>>> realism here, and knowing that documentation unfortunately often gets
>>>>> pushed to the back-burner.
>>>>>
>>>>> Also, I poked some folks on unassigned issues that they've filed for
>>>>> beta-1, especially in the last few days. If issues don't have an owner
>>>>> they
>>>>> are unlikely to get worked. I chatted with stack and he agreed to take
>>>>> on
>>>>> some of the tasks, but there's a lot of surface area to cover.
>>>>>
>>>>> If you you're working on issues that are critical for beta-1, please
>>>>> mark
>>>>> them as such. Then the rest of the community will know to help
>>>>> prioritize
>>>>> feedback and reviews there.
>>>>>
>>>>> Do we have a general theme for the betas like we did with the alphas?
>>>>>
>>>>> Beta1 is upgrades work from branch1, beta2 is rolling upgrades work as
>>>>> well? Continue to work on tests throughout?
>>>>>
>>>>>
>>>> Thanks Mike for helping to kick off the beta-1 train.
>>>>
>>>> Regards a theme for beta-1, I'd like it to be 'finish'; beta-1 is what
>>>> we are going to ship (beta-2 is rolling upgrade and any minor items turned
>>>> up in testing/burn-in).
>>>>
>>>> Thanks,
>>>> S
>>>>
>>>>
>>>>
>>>>> Mike
>>>>>
>>>>> On Tue, Oct 31, 2017 at 10:04 AM, Josh Elser <el...@apache.org>
>>>>> wrote:
>>>>>
>>>>> > +1 go from my POV.
>>>>> >
>>>>> >
>>>>> > On 10/31/17 10:07 AM, Stack wrote:
>>>>> >
>>>>> >> I want to push an alpha-4 today. A few items didn't make it
>>>>> (HBASE-19092).
>>>>> >> They need more time. We'll pull them in for beta-1. CP API is
>>>>> basically
>>>>> >> done. There may be some changes for beta-1 but hopefully only
>>>>> changes
>>>>> >> informed by experience trying to port an existing Coprocessor to
>>>>> hbase2.
>>>>> >>
>>>>> >> Shout if there is anything that needs to make alpha-4.
>>>>> >>
>>>>> >> Thanks,
>>>>> >> St.Ack
>>>>> >>
>>>>> >>
>>>>> >> On Sat, Oct 28, 2017 at 2:48 PM, Josh Elser <jo...@gmail.com>
>>>>> wrote:
>>>>> >>
>>>>> >> Yup, that was going to be my plan, Mike!
>>>>> >>>
>>>>> >>> Making a pass now, and will check back later tonight again. I see
>>>>> others
>>>>> >>> have already done some work today on this front.
>>>>> >>>
>>>>> >>>
>>>>> >>> On 10/27/17 11:38 PM, Mike Drob wrote:
>>>>> >>>
>>>>> >>> Josh - Do you want to kick off a bunch of QA runs? (Do you know
>>>>> how to do
>>>>> >>>> it directly on the jenkins job, so you don't have to bother with
>>>>> JIRA
>>>>> >>>> uploads)
>>>>> >>>>
>>>>> >>>> If you're busy, then I can make time tomorrow or Sunday to kick
>>>>> off
>>>>> >>>> jobs,
>>>>> >>>> but I want to make sure we're not duplicating effort and jenkins
>>>>> cycles.
>>>>> >>>>
>>>>> >>>> On Fri, Oct 27, 2017 at 7:10 PM, Josh Elser <el...@apache.org>
>>>>> wrote:
>>>>> >>>>
>>>>> >>>> My turn to bump ;)
>>>>> >>>>
>>>>> >>>>>
>>>>> >>>>> By my take: HBASE-18770 and HBASE-19092 are the only issues that
>>>>> remain
>>>>> >>>>> needing some more work. The rest are just awaiting a good QA run.
>>>>> >>>>>
>>>>> >>>>> Unless I hear otherwise, I'll try to keep an eye on things over
>>>>> the
>>>>> >>>>> weekend, bump them along as necessary, and get them committed.
>>>>> Would be
>>>>> >>>>> great to be able get a vote up on Monday.
>>>>> >>>>>
>>>>> >>>>>
>>>>> >>>>> On 10/24/17 6:03 PM, Stack wrote:
>>>>> >>>>>
>>>>> >>>>> Chatting with my coworker Mr. Mike Drob, we were batting back
>>>>> and forth
>>>>> >>>>>
>>>>> >>>>>> what remains to be done. Surfacing our thoughts here so you all
>>>>> clued
>>>>> >>>>>> in....Or if you think otherwise, please speak up.
>>>>> >>>>>>
>>>>> >>>>>> We have ~13 issues to land:
>>>>> >>>>>> https://issues.apache.org/jira/projects/HBASE/versions/12341594
>>>>> About
>>>>> >>>>>> two
>>>>> >>>>>> are meta-issues that are about process which leaves 11.
>>>>> >>>>>>
>>>>> >>>>>> Duo and Zheng Hu are to merge the FilterList fixes improvements
>>>>> >>>>>> (HBASE-19057, HBASE-18410 et al.). These are blocker because
>>>>> some
>>>>> >>>>>> changed
>>>>> >>>>>> API/semantic that we need to get out earlier rather than later.
>>>>> >>>>>>
>>>>> >>>>>> Once the above is merged, HBASE-13346, change of Filter method
>>>>> names
>>>>> >>>>>> to
>>>>> >>>>>> mention Cell instead of KeyValue can land.
>>>>> >>>>>>
>>>>> >>>>>> HBASE-199048 needs a review (Anoop will probably do it),
>>>>> removing
>>>>> >>>>>> IA.Private objects as params to MasterObserver... Hopefully
>>>>> this goes
>>>>> >>>>>> in
>>>>> >>>>>> soon.
>>>>> >>>>>>
>>>>> >>>>>> Duo is hard at work on trackers for flush and compaction for CPs
>>>>> >>>>>> (HBASE-18905). How is HBASE-19033 looking Duo (facility for
>>>>> Tephra)?
>>>>> >>>>>>
>>>>> >>>>>> I think HBASE-18906 (Phoenix Region#waitFor...) will evaporate
>>>>> after
>>>>> >>>>>> Duo
>>>>> >>>>>> is
>>>>> >>>>>> done w/ his work above.
>>>>> >>>>>>
>>>>> >>>>>> I'm on HBASE-18770 bypass and HBASE-19077 restore some parity
>>>>> after
>>>>> >>>>>> all
>>>>> >>>>>> the
>>>>> >>>>>> purges allowing CPs do direct calls against Regions in same
>>>>> Host.
>>>>> >>>>>>
>>>>> >>>>>> Anoop is on HBASE-19047 (Fixes) and Ram on cleanup of CellUtil.
>>>>> >>>>>>
>>>>> >>>>>> Another day or two?
>>>>> >>>>>>
>>>>> >>>>>> St.Ack
>>>>> >>>>>>
>>>>> >>>>>>
>>>>> >>>>>>
>>>>> >>>>>> On Mon, Oct 23, 2017 at 2:52 PM, Stack <st...@duboce.net>
>>>>> wrote:
>>>>> >>>>>>
>>>>> >>>>>>
>>>>> >>>>>> On Mon, Oct 23, 2017 at 11:59 AM, Josh Elser <elserj@apache.org
>>>>> >
>>>>> >>>>>> wrote:
>>>>> >>>>>>
>>>>> >>>>>>>
>>>>> >>>>>>> +1
>>>>> >>>>>>>
>>>>> >>>>>>>
>>>>> >>>>>>>> I was trying to work on helping out on the outstanding
>>>>> alpha-4 stuff
>>>>> >>>>>>>> last
>>>>> >>>>>>>> week -- will be continuing to try to do the same this week.
>>>>> >>>>>>>>
>>>>> >>>>>>>> If you need any help, Stack, or if others need reviews where I
>>>>> >>>>>>>> haven't
>>>>> >>>>>>>> noticed on my own: feel free to @mention me.
>>>>> >>>>>>>>
>>>>> >>>>>>>>
>>>>> >>>>>>>> Thanks for the offer Josh. All items seem assigned and are
>>>>> being
>>>>> >>>>>>>>
>>>>> >>>>>>>> actively
>>>>> >>>>>>> worked on. If you get a moment, reviews by you (or anyone
>>>>> else) helps
>>>>> >>>>>>> move
>>>>> >>>>>>> the process along.
>>>>> >>>>>>>
>>>>> >>>>>>> We need to merge in HBASE-18410 branch to pick up Filter
>>>>> >>>>>>> improvements.
>>>>> >>>>>>> Then HBASE-13346 can go in.
>>>>> >>>>>>>
>>>>> >>>>>>> You are already helping out on HBASE-18906, thanks. Looks like
>>>>> that
>>>>> >>>>>>> will
>>>>> >>>>>>> be addressed by other alpha-4s about to land.
>>>>> >>>>>>>
>>>>> >>>>>>> St.Ack
>>>>> >>>>>>> TODOs: https://issues.apache.org/jira
>>>>> /projects/HBASE/versions/12341
>>>>> >>>>>>> 594
>>>>> >>>>>>>
>>>>> >>>>>>>
>>>>> >>>>>>>
>>>>> >>>>>>>
>>>>> >>>>>>>
>>>>> >>>>>>>
>>>>> >>>>>>>
>>>>> >>>>>>>
>>>>> >>>>>>>
>>>>> >>>>>>> On 10/23/17 12:53 PM, Stack wrote:
>>>>> >>>>>>>
>>>>> >>>>>>>
>>>>> >>>>>>>> (Reviving this thread)
>>>>> >>>>>>>>
>>>>> >>>>>>>>
>>>>> >>>>>>>>> Lets push out alpha-4 this week. Alpha-4 is the release that
>>>>> has
>>>>> >>>>>>>>> the
>>>>> >>>>>>>>> refactor of the Coprocessor API shutting down access to
>>>>> internals
>>>>> >>>>>>>>> marked
>>>>> >>>>>>>>> InterfaceAudience.Private.
>>>>> >>>>>>>>>
>>>>> >>>>>>>>> The outstanding list is here:
>>>>> >>>>>>>>> https://issues.apache.org/jira/projects/HBASE/versions/12341
>>>>> 594
>>>>> >>>>>>>>>
>>>>> >>>>>>>>> Please push in anything marked alpha-4 that belongs to you.
>>>>> >>>>>>>>>
>>>>> >>>>>>>>> If issue, talk out loud on this thread. If you need a review
>>>>> to
>>>>> >>>>>>>>> land
>>>>> >>>>>>>>> an
>>>>> >>>>>>>>> item, shout on the issue and here; we'll help you out.
>>>>> >>>>>>>>>
>>>>> >>>>>>>>> As is, items are coming along nicely I'd say. We need to
>>>>> merge the
>>>>> >>>>>>>>> filter
>>>>> >>>>>>>>> branch -- HBASE-18410 -- so APIs are finished for hbase2.
>>>>> >>>>>>>>>
>>>>> >>>>>>>>> Post alpha-4, we'll have to hunt down our downstreamers and
>>>>> help
>>>>> >>>>>>>>> them
>>>>> >>>>>>>>> test
>>>>> >>>>>>>>> on top of alpha-4 so rolling into beta-1, we have confidence
>>>>> our
>>>>> >>>>>>>>> downstreamers know what to expect (or we discover what we
>>>>> missed
>>>>> >>>>>>>>> BEFORE
>>>>> >>>>>>>>> we
>>>>> >>>>>>>>> beta-1).
>>>>> >>>>>>>>>
>>>>> >>>>>>>>> Thanks for time,
>>>>> >>>>>>>>> S
>>>>> >>>>>>>>>
>>>>> >>>>>>>>>
>>>>> >>>>>>>>>
>>>>> >>>>>>>>>
>>>>> >>>>>>>>>
>>>>> >>>>>>>>> On Fri, Sep 8, 2017 at 2:04 PM, Stack <st...@duboce.net>
>>>>> wrote:
>>>>> >>>>>>>>>
>>>>> >>>>>>>>> I'll put up an alpha3 RC Monday, probably Monday night. That
>>>>> should
>>>>> >>>>>>>>> be
>>>>> >>>>>>>>>
>>>>> >>>>>>>>> time, if we all sprint, for the public-facing API fixes to
>>>>> be done.
>>>>> >>>>>>>>>
>>>>> >>>>>>>>>>
>>>>> >>>>>>>>>> I had a bunch of Coprocessor refactor and fixup scheduled
>>>>> for
>>>>> >>>>>>>>>> alpha3
>>>>> >>>>>>>>>> but
>>>>> >>>>>>>>>> it is plain that more time is needed (in spite of valiant
>>>>> effort
>>>>> >>>>>>>>>> so
>>>>> >>>>>>>>>> far
>>>>> >>>>>>>>>> by
>>>>> >>>>>>>>>> Anoop, Duo, Appy, etc.). Therefore, lets run a
>>>>> 2.0.0-alpha-4 whose
>>>>> >>>>>>>>>> theme is
>>>>> >>>>>>>>>> "Coprocessor Fixup". Hopefully we can put an alpha-4 up by
>>>>> the
>>>>> >>>>>>>>>> following
>>>>> >>>>>>>>>> week.
>>>>> >>>>>>>>>>
>>>>> >>>>>>>>>> We should then be ready for beta (beta == no new features,
>>>>> no API
>>>>> >>>>>>>>>> changes,
>>>>> >>>>>>>>>> just fixes).
>>>>> >>>>>>>>>>
>>>>> >>>>>>>>>> Thanks,
>>>>> >>>>>>>>>> St.Ack
>>>>> >>>>>>>>>>
>>>>> >>>>>>>>>>
>>>>> >>>>>>>>>> On Thu, Aug 17, 2017 at 12:35 PM, Stack <st...@duboce.net>
>>>>> wrote:
>>>>> >>>>>>>>>>
>>>>> >>>>>>>>>> I put up the hbase-2.0.0-alpha2 release candidate. Please
>>>>> vote on
>>>>> >>>>>>>>>> it.
>>>>> >>>>>>>>>>
>>>>> >>>>>>>>>>
>>>>> >>>>>>>>>> For hbase-2.0.0-alpha3, the theme is solidifying API. I
>>>>> hope to
>>>>> >>>>>>>>>>> get a
>>>>> >>>>>>>>>>> release out in the next week or so.
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>> I did a weeding of 2.0.0 issues over the last day. If
>>>>> folks are
>>>>> >>>>>>>>>>> interested in helping out, below are the items I think we
>>>>> need
>>>>> >>>>>>>>>>> done
>>>>> >>>>>>>>>>> for
>>>>> >>>>>>>>>>> alpha3 (below are at least 'Critical' status, are API
>>>>> possibly
>>>>> >>>>>>>>>>> altering
>>>>> >>>>>>>>>>> items, and are absent those JIRAs that are making active
>>>>> >>>>>>>>>>> progress,
>>>>> >>>>>>>>>>> i.e. the
>>>>> >>>>>>>>>>> HTD/HCD revamp by Chia-Ping Tsai). A project NOT listed
>>>>> that
>>>>> >>>>>>>>>>> needs
>>>>> >>>>>>>>>>> doing is
>>>>> >>>>>>>>>>> what Andrew did comparing 1.3. and 1.4 APIs
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>> * HBASE-18622 Mitigate compatibility concerns between
>>>>> branch-1
>>>>> >>>>>>>>>>> and
>>>>> >>>>>>>>>>> branch-2
>>>>> >>>>>>>>>>> This is to do what Andrew did between 1.3 and 1.4 branches
>>>>> only
>>>>> >>>>>>>>>>> do
>>>>> >>>>>>>>>>> it
>>>>> >>>>>>>>>>> between branch-1 and branch-2.
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>> * HBASE-10462 Recategorize some of the client facing
>>>>> Public /
>>>>> >>>>>>>>>>> Private
>>>>> >>>>>>>>>>> interfaces
>>>>> >>>>>>>>>>> This one is almost done. It could do with a finish,
>>>>> attention to
>>>>> >>>>>>>>>>> the
>>>>> >>>>>>>>>>> items in last comment, and then our codebase could do with
>>>>> >>>>>>>>>>> another
>>>>> >>>>>>>>>>> sweep
>>>>> >>>>>>>>>>> after the spirit of this issue since a bunch has gone in
>>>>> since
>>>>> >>>>>>>>>>> the
>>>>> >>>>>>>>>>> pass
>>>>> >>>>>>>>>>> that was the basis of this issue.
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>> * HBASE-10504 Define Replication Interface
>>>>> >>>>>>>>>>> I was going to take a crack at this as part of the revamp
>>>>> forced
>>>>> >>>>>>>>>>> by
>>>>> >>>>>>>>>>> 'HBASE-15982 Interface ReplicationEndpoint extends Guava's
>>>>> >>>>>>>>>>> Service'
>>>>> >>>>>>>>>>> but if
>>>>> >>>>>>>>>>> anyone else is interested, be my guest.
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>> * HBASE-14996 Some more API cleanup for 2.0
>>>>> >>>>>>>>>>> Has a bunch of subtasks, some of which are being worked
>>>>> on. Needs
>>>>> >>>>>>>>>>> finishing.
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>> * HBASE-14998 Unify synchronous and asynchronous methods
>>>>> in Admin
>>>>> >>>>>>>>>>> and
>>>>> >>>>>>>>>>> cleanup
>>>>> >>>>>>>>>>> Needs a pass. Small issue I think. Could also look at new
>>>>> >>>>>>>>>>> AsyncClient
>>>>> >>>>>>>>>>> and
>>>>> >>>>>>>>>>> make sure symmetry.
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>> * HBASE-15607 Remove PB references from Admin for 2.0
>>>>> >>>>>>>>>>> Predicated on result of an ongoing DISCUSSION thread but
>>>>> needs to
>>>>> >>>>>>>>>>> be
>>>>> >>>>>>>>>>> done.
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>> Rolling upgrade will have implications for our API. Would
>>>>> be good
>>>>> >>>>>>>>>>> to
>>>>> >>>>>>>>>>> try
>>>>> >>>>>>>>>>> it and figure what needs fixup (as said above, according
>>>>> to trial
>>>>> >>>>>>>>>>> by
>>>>> >>>>>>>>>>> Sean,
>>>>> >>>>>>>>>>> we might not be too bad here):
>>>>> >>>>>>>>>>> * HBASE-16060 1.x clients cannot access table state
>>>>> talking to
>>>>> >>>>>>>>>>> 2.0
>>>>> >>>>>>>>>>> cluster
>>>>> >>>>>>>>>>> * HBASE-16550 Procedure v2 - Add AM compatibility for 2.x
>>>>> Master
>>>>> >>>>>>>>>>> and
>>>>> >>>>>>>>>>> 1.x
>>>>> >>>>>>>>>>> RSs; i.e. support Rolling Upgrade from hbase-1 to -2.
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>> * HBASE-17442 Move most of the replication related classes
>>>>> to
>>>>> >>>>>>>>>>> hbase-server package
>>>>> >>>>>>>>>>> The above would be good to do generally but it may make for
>>>>> >>>>>>>>>>> ripples
>>>>> >>>>>>>>>>> in
>>>>> >>>>>>>>>>> API so would be good to do now.
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>> * HBASE-18106 Redo ProcedureInfo and LockInfo
>>>>> >>>>>>>>>>> Balazs is working on this. The idea is that we avoid
>>>>> adding two
>>>>> >>>>>>>>>>> new
>>>>> >>>>>>>>>>> types
>>>>> >>>>>>>>>>> to our API, two types that are nought but curtailed,
>>>>> read-only
>>>>> >>>>>>>>>>> views
>>>>> >>>>>>>>>>> on
>>>>> >>>>>>>>>>> internals. Input if you have time appreciated.
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>> * HBASE-18596 A hbase1 cluster should be able to replicate
>>>>> to a
>>>>> >>>>>>>>>>> hbase2
>>>>> >>>>>>>>>>> cluster; verify
>>>>> >>>>>>>>>>> Esteban is looking at this one
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>> * HBASE-9417 SecureBulkLoadEndpoint should be folded in
>>>>> core
>>>>> >>>>>>>>>>> * HBASE-17143 Scan improvement
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>> Our Coprocessor Interface needs a tough edit. It exposes
>>>>> >>>>>>>>>>> implementations
>>>>> >>>>>>>>>>> marked audience Private and returns implementations rather
>>>>> than
>>>>> >>>>>>>>>>> Interfaces.
>>>>> >>>>>>>>>>> In a few locations, we allow returning an alternate
>>>>> >>>>>>>>>>> implementation
>>>>> >>>>>>>>>>> altogether which is probably something we don't want a CP
>>>>> doing.
>>>>> >>>>>>>>>>> To
>>>>> >>>>>>>>>>> that
>>>>> >>>>>>>>>>> end, the following issues started by Duo and Anoop need to
>>>>> be
>>>>> >>>>>>>>>>> taken
>>>>> >>>>>>>>>>> to
>>>>> >>>>>>>>>>> the
>>>>> >>>>>>>>>>> finish line; ideally they'd have an owner:
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>> * HBASE-18169 Coprocessor fix and cleanup before 2.0.0
>>>>> release <=
>>>>> >>>>>>>>>>> The
>>>>> >>>>>>>>>>> umbrella issue.
>>>>> >>>>>>>>>>> * HBASE-18298 RegionServerServices Interface cleanup for CP
>>>>> >>>>>>>>>>> expose
>>>>> >>>>>>>>>>> * HBASE-16769 Deprecate/remove PB references from
>>>>> MasterObserver
>>>>> >>>>>>>>>>> and
>>>>> >>>>>>>>>>> RegionServerObserver
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>> Nice-to-haves:
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>> * HBASE-15284 Make TimeRange constructors IA.Private and
>>>>> remove
>>>>> >>>>>>>>>>> unused
>>>>> >>>>>>>>>>> TimeRange constructors
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>> * HBASE-10944 Remove all kv.getBuffer() and kv.getRow()
>>>>> >>>>>>>>>>> references
>>>>> >>>>>>>>>>> existing in the code
>>>>> >>>>>>>>>>> This is the end of an old long-running project moving up
>>>>> on to
>>>>> >>>>>>>>>>> Cell
>>>>> >>>>>>>>>>> Interface. We think it is done but for a few little items
>>>>> >>>>>>>>>>> (deprecate
>>>>> >>>>>>>>>>> KV
>>>>> >>>>>>>>>>> methods in MR and provide Cell versions instead...)
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>> * HBASE-13271 Table#puts(List<Put>) operation is
>>>>> indeterminate;
>>>>> >>>>>>>>>>> needs
>>>>> >>>>>>>>>>> fixing
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>> * HBASE-13346 Clean up Filter package for post 1.0
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>> * HBASE-14255 Simplify Cell creation post 1.0
>>>>> >>>>>>>>>>> * HBASE-14997
>>>>> >>>>>>>>>>> Move compareOp and Comparators out of filter to client
>>>>> package
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>> * HBASE-13740 Stop using Hadoop private interfaces
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>> What about:
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>> * HBASE-18601 Remove Htrace 3.2
>>>>> >>>>>>>>>>> As has been noted, the HTrace API is our 'trace' API.
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>> If interested in any of the above and you need a legup,
>>>>> just ask
>>>>> >>>>>>>>>>> in
>>>>> >>>>>>>>>>> the
>>>>> >>>>>>>>>>> issue and I'll be by....
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>> Thanks,
>>>>> >>>>>>>>>>> St.Ack
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>> On Mon, Aug 14, 2017 at 10:54 AM, Stack <st...@duboce.net>
>>>>> >>>>>>>>>>> wrote:
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>> Heads-up:
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>> I'm about to put up an hbase-2.0.0-alpha2 Release
>>>>> Candidate.
>>>>> >>>>>>>>>>>> Theme
>>>>> >>>>>>>>>>>> is
>>>>> >>>>>>>>>>>> updated dependencies, reliance on relocated popular libs
>>>>> (guava,
>>>>> >>>>>>>>>>>> netty,
>>>>> >>>>>>>>>>>> protobuf), purge of checked-in generated src, and
>>>>> >>>>>>>>>>>> master-carries-no-regions
>>>>> >>>>>>>>>>>> by default.
>>>>> >>>>>>>>>>>>
>>>>> >>>>>>>>>>>> alpha3 I hope will follow soon after (end-of-August?).
>>>>> Its theme
>>>>> >>>>>>>>>>>> will
>>>>> >>>>>>>>>>>> be
>>>>> >>>>>>>>>>>> settling the APIs and compatibility (At first blush, we
>>>>> are not
>>>>> >>>>>>>>>>>> looking too
>>>>> >>>>>>>>>>>> bad; our Sean ran some tests over weekend that have
>>>>> hbase-1
>>>>> >>>>>>>>>>>> client
>>>>> >>>>>>>>>>>> running
>>>>> >>>>>>>>>>>> against an hbase-2 cluster....). The Coprocessor Interface
>>>>> >>>>>>>>>>>> revamp
>>>>> >>>>>>>>>>>> should be
>>>>> >>>>>>>>>>>> done by alpha3 (i.e. returning Interfaces rather than
>>>>> >>>>>>>>>>>> Implementations, and
>>>>> >>>>>>>>>>>> our shutdown of CPs accessing classes in hbase marked
>>>>> >>>>>>>>>>>> InterfaceAudience).
>>>>> >>>>>>>>>>>> We'll also have purged thirdparty classes from our API;
>>>>> e.g.
>>>>> >>>>>>>>>>>> guava
>>>>> >>>>>>>>>>>> 0.12
>>>>> >>>>>>>>>>>> Service showing through in our replication API and
>>>>> protobufs in
>>>>> >>>>>>>>>>>> Admin
>>>>> >>>>>>>>>>>> Interface. On alpha3, we will have to do a bunch of
>>>>> outreach to
>>>>> >>>>>>>>>>>> make
>>>>> >>>>>>>>>>>> sure
>>>>> >>>>>>>>>>>> our downstreamers are up on what is coming down the pipe.
>>>>> >>>>>>>>>>>>
>>>>> >>>>>>>>>>>> Beta1 in mid-September?
>>>>> >>>>>>>>>>>>
>>>>> >>>>>>>>>>>> I encourage you to check out the items marked for hbase2:
>>>>> >>>>>>>>>>>> https://issues.apache.org/jira
>>>>> /projects/HBASE/versions/12327188
>>>>> >>>>>>>>>>>> Edit
>>>>> >>>>>>>>>>>> as
>>>>> >>>>>>>>>>>> you see appropriate. Punt if you know the JIRA will not
>>>>> get any
>>>>> >>>>>>>>>>>> attention
>>>>> >>>>>>>>>>>> in next month or so.
>>>>> >>>>>>>>>>>>
>>>>> >>>>>>>>>>>> A bunch of issues marked blocker are unassigned. I'll
>>>>> leave them
>>>>> >>>>>>>>>>>> as
>>>>> >>>>>>>>>>>> is
>>>>> >>>>>>>>>>>> another while but I'll boot them soon.
>>>>> >>>>>>>>>>>>
>>>>> >>>>>>>>>>>> While I have your attention:
>>>>> >>>>>>>>>>>>
>>>>> >>>>>>>>>>>> + I think we should leave thrift version at 0.9.3. Moving
>>>>> hbase
>>>>> >>>>>>>>>>>> thrift
>>>>> >>>>>>>>>>>> to 0.10.0 will break existing clients. The change is easy
>>>>> enough
>>>>> >>>>>>>>>>>> if
>>>>> >>>>>>>>>>>> folks
>>>>> >>>>>>>>>>>> need to upgrade their hbase thrift. See HBASE-18591.
>>>>> >>>>>>>>>>>> + Upgrade from 0.94 is disallowed. You have to get to 1.0
>>>>> first
>>>>> >>>>>>>>>>>> (0.98?).
>>>>> >>>>>>>>>>>>
>>>>> >>>>>>>>>>>> St.Ack
>>>>> >>>>>>>>>>>>
>>>>> >>>>>>>>>>>>
>>>>> >>>>>>>>>>>>
>>>>> >>>>>>>>>>>> On Wed, Aug 2, 2017 at 9:43 AM, Stack <st...@duboce.net>
>>>>> wrote:
>>>>> >>>>>>>>>>>>
>>>>> >>>>>>>>>>>>
>>>>> >>>>>>>>>>>>
>>>>> >>>>>>>>>>>> On Tue, Aug 1, 2017 at 2:06 PM, Josh Elser <
>>>>> elserj@apache.org>
>>>>> >>>>>>>>>>>>
>>>>> >>>>>>>>>>>>> wrote:
>>>>> >>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>> On 7/31/17 9:00 AM, Stack wrote:
>>>>> >>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>> On Mon, Jul 24, 2017 at 12:25 PM, Josh Elser<
>>>>> >>>>>>>>>>>>>> elserj@apache.org>
>>>>> >>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>> wrote:
>>>>> >>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>> ...
>>>>> >>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>> I like the idea of this also hitting 2.0 as it would
>>>>> make the
>>>>> >>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>> feature a
>>>>> >>>>>>>>>>>>>>>> bit more "real", but am obviously a little nervous (I
>>>>> have
>>>>> >>>>>>>>>>>>>>>> no
>>>>> >>>>>>>>>>>>>>>> reason
>>>>> >>>>>>>>>>>>>>>> to be
>>>>> >>>>>>>>>>>>>>>> nervous though). I am pretty happy with the feature
>>>>> in terms
>>>>> >>>>>>>>>>>>>>>> of
>>>>> >>>>>>>>>>>>>>>> how
>>>>> >>>>>>>>>>>>>>>> much it
>>>>> >>>>>>>>>>>>>>>> is covered via testing.
>>>>> >>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/HBASE-17748
>>>>> >>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>> Sounds good to me. Whats involved? Backport? If so,
>>>>> +1 Josh.
>>>>> >>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>> Last think on space quota says that need doc too. See
>>>>> 'Space
>>>>> >>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>> Quota' in
>>>>> >>>>>>>>>>>>>>> here:
>>>>> >>>>>>>>>>>>>>> https://docs.google.com/docume
>>>>> nt/d/1WCsVlnHjJeKUcl7wHwqb4z9i
>>>>> >>>>>>>>>>>>>>> Eu_ktczrlKHK8N4SZzs/edit#heading=h.wuw3a6jukzo5
>>>>> >>>>>>>>>>>>>>> Does this little section need an update Josh?
>>>>> >>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>> Thanks,
>>>>> >>>>>>>>>>>>>>> S
>>>>> >>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>> Yep, just a couple of cherry-picks. Good test coverage
>>>>> and
>>>>> >>>>>>>>>>>>>>> some
>>>>> >>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>> docs
>>>>> >>>>>>>>>>>>>> already included for 17748.  Happy to put that on my
>>>>> plate if
>>>>> >>>>>>>>>>>>>> you're good
>>>>> >>>>>>>>>>>>>> with it. I can reasonably assume that no one is against
>>>>> it :)
>>>>> >>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>> I think I had knocked out docs for the "phase 1" stuff
>>>>> before
>>>>> >>>>>>>>>>>>>> we
>>>>> >>>>>>>>>>>>>> merged it in from the original feature branch. I'll
>>>>> double
>>>>> >>>>>>>>>>>>>> check
>>>>> >>>>>>>>>>>>>> and update
>>>>> >>>>>>>>>>>>>> the gdoc. Perhaps this was just a timing thing.
>>>>> >>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>> Thanks Josh,
>>>>> >>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>> S
>>>>> >>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>
>>>>> >>>>>>>>>>>>
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>
>>>>> >>>>>>>>>
>>>>> >>>>>>>
>>>>> >>>>>>
>>>>> >>>>
>>>>> >>
>>>>>
>>>>
>>>>
>>>
>>
>

Re: Moving 2.0 forward

[Stack <st...@duboce.net>]

Update. See below.

On Tue, Dec 5, 2017 at 10:43 AM, Stack <st...@duboce.net> wrote:

> Update on hbase-2.0.0-beta-1. I'm going to put up a beta-1 RC before
> Christmas day.
>
>
Still aiming for an hbase2 beta-1 showing up in your christmas stocking.


> Here is the list [1]. We're doing pretty good. We've fixed a load since
> the last update including some nice cleanups  two weeks ago (e.g. undoing
> client dependency on curator/zk watching making the dependence read-only)
> but there are still a few hairy ones hanging out there.
>
>  * HBASE-18946 Stochastic load balancer assigns replica regions to the
> same RS
>    This one is proving a little tough. Ram has been banging his head on
> it. We'll be able to clear a bunch of test failures once this is done.
>
>
This should be going in tomorrow.



>  * HBASE-19112 Suspect methods on Cell to be deprecated
>    Good discussion up on the JIRA. We need to be crystal clear around Cell
> and derivatives API and Audience when we ship 2.0.
>
>
Ram, Chia-Ping Tsai, and Anoop are doing a nice job here and it should be
done in next day or so.



>  * HBASE-17204 Make L2 off heap cache default ON
>    We need to try this at least so offheaping can be default.
>
>
Chatted with Anoop and Ram. Because it is so late in the day and because it
critical that the user have a good experience out of the box -- which
requires time and testing -- we are going to punt this from 2.0. Will look
at it for 2.1 (or 3.0 if soon). 2.0 will still be carrying all of the boys
offheap read/write path goodness.



> Making asyncfswal default is also ongoing, HBASE-16890
> <https://issues.apache.org/jira/browse/HBASE-16890>, making good progress.
>
>

Good progress here. In perf testing asyncfswal makes us generally faster.
Duo making good progress on last few tests that are in the way of our
making this default for hbase2.

A load of goodness has been landing these last few weeks. Keep up the great
work.

Your hbase-2 RM.




> I punted HBASE-19147 "All branch-2 unit tests pass" out to beta-2. Our
> unit test story has gotten better and a few of us are actively working on
> flakies [2] but we may make a beta-1 w/o shutting down all failures.
>
> Speak up if you need help on an issue or if you think we are missing items
> from our list.
>
> Thanks for all the hard work,
>
> The hbase-2.0.0 RM.
>
>
> 1. https://issues.apache.org/jira/projects/HBASE/versions/12340861
> 2. https://builds.apache.org/job/HBASE-Find-Flaky-Tests/
> lastSuccessfulBuild/artifact/dashboard.html
>
> On Thu, Nov 16, 2017 at 3:48 PM, Stack <st...@duboce.net> wrote:
>
>> hbase-2.0.0-beta-1 update (Reminder, beta-1 is where we finish last
>> remaining features and apply final polish to API. There will be a beta-2
>> but it is about upgrade/rolling-upgrade and bug fixes ONLY).
>>
>> Myself and Mr. Drob did a pass over the outstanding hbase-2.0.0-beta-1
>> list this morning. See here [1].
>>
>> There are about ~12 issues in progress with most of these about to land.
>> There are 37 TODO. Many of these are tests we need to run, some are related
>> to the backup/restore, but a good few are meaty w/o assignees.
>>
>> The awkward outstanding ones as I see it are the below:
>>
>> HBASE-18946 Stochastic load balancer assigns replica regions to the same
>> RS
>> HBASE-17204 Make L2 off heap cache default ON
>> HBASE-19112 Suspect methods on Cell to be deprecated
>> HBASE-19147 All branch-2 unit tests pass
>>
>> We need to make progress on the above or punt on them (can't punt on the
>> last one though).
>>
>> Any ideas on what configs we should update in hbase2? Dump ideas into:
>> HBASE-19148 Edit of default configuration
>>
>> Still hoping for an early December beta-1 RC. beta-2 hopefully will be
>> close behind.
>>
>> Comments? Thoughts?
>>
>> Thanks all,
>> S
>>
>> 1. https://issues.apache.org/jira/projects/HBASE/versions/12340861
>>
>> On Tue, Oct 31, 2017 at 2:56 PM, Stack <st...@duboce.net> wrote:
>>
>>>
>>>
>>> On Tue, Oct 31, 2017 at 2:31 PM, Mike Drob <md...@apache.org> wrote:
>>>
>>>> Hoping to keep momentum going from our Stack working on alpha4, I tried
>>>> to
>>>> take a stab at triaging some of the open beta-1 issues.
>>>>
>>>> I moved some docs stuff out form beta-1 to 2.0-GA, if it gets done
>>>> sooner
>>>> then I'm happy to see it pulled back in. Trying to balance optimism with
>>>> realism here, and knowing that documentation unfortunately often gets
>>>> pushed to the back-burner.
>>>>
>>>> Also, I poked some folks on unassigned issues that they've filed for
>>>> beta-1, especially in the last few days. If issues don't have an owner
>>>> they
>>>> are unlikely to get worked. I chatted with stack and he agreed to take
>>>> on
>>>> some of the tasks, but there's a lot of surface area to cover.
>>>>
>>>> If you you're working on issues that are critical for beta-1, please
>>>> mark
>>>> them as such. Then the rest of the community will know to help
>>>> prioritize
>>>> feedback and reviews there.
>>>>
>>>> Do we have a general theme for the betas like we did with the alphas?
>>>>
>>>> Beta1 is upgrades work from branch1, beta2 is rolling upgrades work as
>>>> well? Continue to work on tests throughout?
>>>>
>>>>
>>> Thanks Mike for helping to kick off the beta-1 train.
>>>
>>> Regards a theme for beta-1, I'd like it to be 'finish'; beta-1 is what
>>> we are going to ship (beta-2 is rolling upgrade and any minor items turned
>>> up in testing/burn-in).
>>>
>>> Thanks,
>>> S
>>>
>>>
>>>
>>>> Mike
>>>>
>>>> On Tue, Oct 31, 2017 at 10:04 AM, Josh Elser <el...@apache.org> wrote:
>>>>
>>>> > +1 go from my POV.
>>>> >
>>>> >
>>>> > On 10/31/17 10:07 AM, Stack wrote:
>>>> >
>>>> >> I want to push an alpha-4 today. A few items didn't make it
>>>> (HBASE-19092).
>>>> >> They need more time. We'll pull them in for beta-1. CP API is
>>>> basically
>>>> >> done. There may be some changes for beta-1 but hopefully only changes
>>>> >> informed by experience trying to port an existing Coprocessor to
>>>> hbase2.
>>>> >>
>>>> >> Shout if there is anything that needs to make alpha-4.
>>>> >>
>>>> >> Thanks,
>>>> >> St.Ack
>>>> >>
>>>> >>
>>>> >> On Sat, Oct 28, 2017 at 2:48 PM, Josh Elser <jo...@gmail.com>
>>>> wrote:
>>>> >>
>>>> >> Yup, that was going to be my plan, Mike!
>>>> >>>
>>>> >>> Making a pass now, and will check back later tonight again. I see
>>>> others
>>>> >>> have already done some work today on this front.
>>>> >>>
>>>> >>>
>>>> >>> On 10/27/17 11:38 PM, Mike Drob wrote:
>>>> >>>
>>>> >>> Josh - Do you want to kick off a bunch of QA runs? (Do you know how
>>>> to do
>>>> >>>> it directly on the jenkins job, so you don't have to bother with
>>>> JIRA
>>>> >>>> uploads)
>>>> >>>>
>>>> >>>> If you're busy, then I can make time tomorrow or Sunday to kick off
>>>> >>>> jobs,
>>>> >>>> but I want to make sure we're not duplicating effort and jenkins
>>>> cycles.
>>>> >>>>
>>>> >>>> On Fri, Oct 27, 2017 at 7:10 PM, Josh Elser <el...@apache.org>
>>>> wrote:
>>>> >>>>
>>>> >>>> My turn to bump ;)
>>>> >>>>
>>>> >>>>>
>>>> >>>>> By my take: HBASE-18770 and HBASE-19092 are the only issues that
>>>> remain
>>>> >>>>> needing some more work. The rest are just awaiting a good QA run.
>>>> >>>>>
>>>> >>>>> Unless I hear otherwise, I'll try to keep an eye on things over
>>>> the
>>>> >>>>> weekend, bump them along as necessary, and get them committed.
>>>> Would be
>>>> >>>>> great to be able get a vote up on Monday.
>>>> >>>>>
>>>> >>>>>
>>>> >>>>> On 10/24/17 6:03 PM, Stack wrote:
>>>> >>>>>
>>>> >>>>> Chatting with my coworker Mr. Mike Drob, we were batting back and
>>>> forth
>>>> >>>>>
>>>> >>>>>> what remains to be done. Surfacing our thoughts here so you all
>>>> clued
>>>> >>>>>> in....Or if you think otherwise, please speak up.
>>>> >>>>>>
>>>> >>>>>> We have ~13 issues to land:
>>>> >>>>>> https://issues.apache.org/jira/projects/HBASE/versions/12341594
>>>> About
>>>> >>>>>> two
>>>> >>>>>> are meta-issues that are about process which leaves 11.
>>>> >>>>>>
>>>> >>>>>> Duo and Zheng Hu are to merge the FilterList fixes improvements
>>>> >>>>>> (HBASE-19057, HBASE-18410 et al.). These are blocker because some
>>>> >>>>>> changed
>>>> >>>>>> API/semantic that we need to get out earlier rather than later.
>>>> >>>>>>
>>>> >>>>>> Once the above is merged, HBASE-13346, change of Filter method
>>>> names
>>>> >>>>>> to
>>>> >>>>>> mention Cell instead of KeyValue can land.
>>>> >>>>>>
>>>> >>>>>> HBASE-199048 needs a review (Anoop will probably do it), removing
>>>> >>>>>> IA.Private objects as params to MasterObserver... Hopefully this
>>>> goes
>>>> >>>>>> in
>>>> >>>>>> soon.
>>>> >>>>>>
>>>> >>>>>> Duo is hard at work on trackers for flush and compaction for CPs
>>>> >>>>>> (HBASE-18905). How is HBASE-19033 looking Duo (facility for
>>>> Tephra)?
>>>> >>>>>>
>>>> >>>>>> I think HBASE-18906 (Phoenix Region#waitFor...) will evaporate
>>>> after
>>>> >>>>>> Duo
>>>> >>>>>> is
>>>> >>>>>> done w/ his work above.
>>>> >>>>>>
>>>> >>>>>> I'm on HBASE-18770 bypass and HBASE-19077 restore some parity
>>>> after
>>>> >>>>>> all
>>>> >>>>>> the
>>>> >>>>>> purges allowing CPs do direct calls against Regions in same Host.
>>>> >>>>>>
>>>> >>>>>> Anoop is on HBASE-19047 (Fixes) and Ram on cleanup of CellUtil.
>>>> >>>>>>
>>>> >>>>>> Another day or two?
>>>> >>>>>>
>>>> >>>>>> St.Ack
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>> On Mon, Oct 23, 2017 at 2:52 PM, Stack <st...@duboce.net> wrote:
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>> On Mon, Oct 23, 2017 at 11:59 AM, Josh Elser <el...@apache.org>
>>>> >>>>>> wrote:
>>>> >>>>>>
>>>> >>>>>>>
>>>> >>>>>>> +1
>>>> >>>>>>>
>>>> >>>>>>>
>>>> >>>>>>>> I was trying to work on helping out on the outstanding alpha-4
>>>> stuff
>>>> >>>>>>>> last
>>>> >>>>>>>> week -- will be continuing to try to do the same this week.
>>>> >>>>>>>>
>>>> >>>>>>>> If you need any help, Stack, or if others need reviews where I
>>>> >>>>>>>> haven't
>>>> >>>>>>>> noticed on my own: feel free to @mention me.
>>>> >>>>>>>>
>>>> >>>>>>>>
>>>> >>>>>>>> Thanks for the offer Josh. All items seem assigned and are
>>>> being
>>>> >>>>>>>>
>>>> >>>>>>>> actively
>>>> >>>>>>> worked on. If you get a moment, reviews by you (or anyone else)
>>>> helps
>>>> >>>>>>> move
>>>> >>>>>>> the process along.
>>>> >>>>>>>
>>>> >>>>>>> We need to merge in HBASE-18410 branch to pick up Filter
>>>> >>>>>>> improvements.
>>>> >>>>>>> Then HBASE-13346 can go in.
>>>> >>>>>>>
>>>> >>>>>>> You are already helping out on HBASE-18906, thanks. Looks like
>>>> that
>>>> >>>>>>> will
>>>> >>>>>>> be addressed by other alpha-4s about to land.
>>>> >>>>>>>
>>>> >>>>>>> St.Ack
>>>> >>>>>>> TODOs: https://issues.apache.org/jira
>>>> /projects/HBASE/versions/12341
>>>> >>>>>>> 594
>>>> >>>>>>>
>>>> >>>>>>>
>>>> >>>>>>>
>>>> >>>>>>>
>>>> >>>>>>>
>>>> >>>>>>>
>>>> >>>>>>>
>>>> >>>>>>>
>>>> >>>>>>>
>>>> >>>>>>> On 10/23/17 12:53 PM, Stack wrote:
>>>> >>>>>>>
>>>> >>>>>>>
>>>> >>>>>>>> (Reviving this thread)
>>>> >>>>>>>>
>>>> >>>>>>>>
>>>> >>>>>>>>> Lets push out alpha-4 this week. Alpha-4 is the release that
>>>> has
>>>> >>>>>>>>> the
>>>> >>>>>>>>> refactor of the Coprocessor API shutting down access to
>>>> internals
>>>> >>>>>>>>> marked
>>>> >>>>>>>>> InterfaceAudience.Private.
>>>> >>>>>>>>>
>>>> >>>>>>>>> The outstanding list is here:
>>>> >>>>>>>>> https://issues.apache.org/jira/projects/HBASE/versions/12341
>>>> 594
>>>> >>>>>>>>>
>>>> >>>>>>>>> Please push in anything marked alpha-4 that belongs to you.
>>>> >>>>>>>>>
>>>> >>>>>>>>> If issue, talk out loud on this thread. If you need a review
>>>> to
>>>> >>>>>>>>> land
>>>> >>>>>>>>> an
>>>> >>>>>>>>> item, shout on the issue and here; we'll help you out.
>>>> >>>>>>>>>
>>>> >>>>>>>>> As is, items are coming along nicely I'd say. We need to
>>>> merge the
>>>> >>>>>>>>> filter
>>>> >>>>>>>>> branch -- HBASE-18410 -- so APIs are finished for hbase2.
>>>> >>>>>>>>>
>>>> >>>>>>>>> Post alpha-4, we'll have to hunt down our downstreamers and
>>>> help
>>>> >>>>>>>>> them
>>>> >>>>>>>>> test
>>>> >>>>>>>>> on top of alpha-4 so rolling into beta-1, we have confidence
>>>> our
>>>> >>>>>>>>> downstreamers know what to expect (or we discover what we
>>>> missed
>>>> >>>>>>>>> BEFORE
>>>> >>>>>>>>> we
>>>> >>>>>>>>> beta-1).
>>>> >>>>>>>>>
>>>> >>>>>>>>> Thanks for time,
>>>> >>>>>>>>> S
>>>> >>>>>>>>>
>>>> >>>>>>>>>
>>>> >>>>>>>>>
>>>> >>>>>>>>>
>>>> >>>>>>>>>
>>>> >>>>>>>>> On Fri, Sep 8, 2017 at 2:04 PM, Stack <st...@duboce.net>
>>>> wrote:
>>>> >>>>>>>>>
>>>> >>>>>>>>> I'll put up an alpha3 RC Monday, probably Monday night. That
>>>> should
>>>> >>>>>>>>> be
>>>> >>>>>>>>>
>>>> >>>>>>>>> time, if we all sprint, for the public-facing API fixes to be
>>>> done.
>>>> >>>>>>>>>
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> I had a bunch of Coprocessor refactor and fixup scheduled for
>>>> >>>>>>>>>> alpha3
>>>> >>>>>>>>>> but
>>>> >>>>>>>>>> it is plain that more time is needed (in spite of valiant
>>>> effort
>>>> >>>>>>>>>> so
>>>> >>>>>>>>>> far
>>>> >>>>>>>>>> by
>>>> >>>>>>>>>> Anoop, Duo, Appy, etc.). Therefore, lets run a 2.0.0-alpha-4
>>>> whose
>>>> >>>>>>>>>> theme is
>>>> >>>>>>>>>> "Coprocessor Fixup". Hopefully we can put an alpha-4 up by
>>>> the
>>>> >>>>>>>>>> following
>>>> >>>>>>>>>> week.
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> We should then be ready for beta (beta == no new features,
>>>> no API
>>>> >>>>>>>>>> changes,
>>>> >>>>>>>>>> just fixes).
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> Thanks,
>>>> >>>>>>>>>> St.Ack
>>>> >>>>>>>>>>
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> On Thu, Aug 17, 2017 at 12:35 PM, Stack <st...@duboce.net>
>>>> wrote:
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> I put up the hbase-2.0.0-alpha2 release candidate. Please
>>>> vote on
>>>> >>>>>>>>>> it.
>>>> >>>>>>>>>>
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> For hbase-2.0.0-alpha3, the theme is solidifying API. I hope
>>>> to
>>>> >>>>>>>>>>> get a
>>>> >>>>>>>>>>> release out in the next week or so.
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> I did a weeding of 2.0.0 issues over the last day. If folks
>>>> are
>>>> >>>>>>>>>>> interested in helping out, below are the items I think we
>>>> need
>>>> >>>>>>>>>>> done
>>>> >>>>>>>>>>> for
>>>> >>>>>>>>>>> alpha3 (below are at least 'Critical' status, are API
>>>> possibly
>>>> >>>>>>>>>>> altering
>>>> >>>>>>>>>>> items, and are absent those JIRAs that are making active
>>>> >>>>>>>>>>> progress,
>>>> >>>>>>>>>>> i.e. the
>>>> >>>>>>>>>>> HTD/HCD revamp by Chia-Ping Tsai). A project NOT listed that
>>>> >>>>>>>>>>> needs
>>>> >>>>>>>>>>> doing is
>>>> >>>>>>>>>>> what Andrew did comparing 1.3. and 1.4 APIs
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> * HBASE-18622 Mitigate compatibility concerns between
>>>> branch-1
>>>> >>>>>>>>>>> and
>>>> >>>>>>>>>>> branch-2
>>>> >>>>>>>>>>> This is to do what Andrew did between 1.3 and 1.4 branches
>>>> only
>>>> >>>>>>>>>>> do
>>>> >>>>>>>>>>> it
>>>> >>>>>>>>>>> between branch-1 and branch-2.
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> * HBASE-10462 Recategorize some of the client facing Public
>>>> /
>>>> >>>>>>>>>>> Private
>>>> >>>>>>>>>>> interfaces
>>>> >>>>>>>>>>> This one is almost done. It could do with a finish,
>>>> attention to
>>>> >>>>>>>>>>> the
>>>> >>>>>>>>>>> items in last comment, and then our codebase could do with
>>>> >>>>>>>>>>> another
>>>> >>>>>>>>>>> sweep
>>>> >>>>>>>>>>> after the spirit of this issue since a bunch has gone in
>>>> since
>>>> >>>>>>>>>>> the
>>>> >>>>>>>>>>> pass
>>>> >>>>>>>>>>> that was the basis of this issue.
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> * HBASE-10504 Define Replication Interface
>>>> >>>>>>>>>>> I was going to take a crack at this as part of the revamp
>>>> forced
>>>> >>>>>>>>>>> by
>>>> >>>>>>>>>>> 'HBASE-15982 Interface ReplicationEndpoint extends Guava's
>>>> >>>>>>>>>>> Service'
>>>> >>>>>>>>>>> but if
>>>> >>>>>>>>>>> anyone else is interested, be my guest.
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> * HBASE-14996 Some more API cleanup for 2.0
>>>> >>>>>>>>>>> Has a bunch of subtasks, some of which are being worked on.
>>>> Needs
>>>> >>>>>>>>>>> finishing.
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> * HBASE-14998 Unify synchronous and asynchronous methods in
>>>> Admin
>>>> >>>>>>>>>>> and
>>>> >>>>>>>>>>> cleanup
>>>> >>>>>>>>>>> Needs a pass. Small issue I think. Could also look at new
>>>> >>>>>>>>>>> AsyncClient
>>>> >>>>>>>>>>> and
>>>> >>>>>>>>>>> make sure symmetry.
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> * HBASE-15607 Remove PB references from Admin for 2.0
>>>> >>>>>>>>>>> Predicated on result of an ongoing DISCUSSION thread but
>>>> needs to
>>>> >>>>>>>>>>> be
>>>> >>>>>>>>>>> done.
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> Rolling upgrade will have implications for our API. Would
>>>> be good
>>>> >>>>>>>>>>> to
>>>> >>>>>>>>>>> try
>>>> >>>>>>>>>>> it and figure what needs fixup (as said above, according to
>>>> trial
>>>> >>>>>>>>>>> by
>>>> >>>>>>>>>>> Sean,
>>>> >>>>>>>>>>> we might not be too bad here):
>>>> >>>>>>>>>>> * HBASE-16060 1.x clients cannot access table state talking
>>>> to
>>>> >>>>>>>>>>> 2.0
>>>> >>>>>>>>>>> cluster
>>>> >>>>>>>>>>> * HBASE-16550 Procedure v2 - Add AM compatibility for 2.x
>>>> Master
>>>> >>>>>>>>>>> and
>>>> >>>>>>>>>>> 1.x
>>>> >>>>>>>>>>> RSs; i.e. support Rolling Upgrade from hbase-1 to -2.
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> * HBASE-17442 Move most of the replication related classes
>>>> to
>>>> >>>>>>>>>>> hbase-server package
>>>> >>>>>>>>>>> The above would be good to do generally but it may make for
>>>> >>>>>>>>>>> ripples
>>>> >>>>>>>>>>> in
>>>> >>>>>>>>>>> API so would be good to do now.
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> * HBASE-18106 Redo ProcedureInfo and LockInfo
>>>> >>>>>>>>>>> Balazs is working on this. The idea is that we avoid adding
>>>> two
>>>> >>>>>>>>>>> new
>>>> >>>>>>>>>>> types
>>>> >>>>>>>>>>> to our API, two types that are nought but curtailed,
>>>> read-only
>>>> >>>>>>>>>>> views
>>>> >>>>>>>>>>> on
>>>> >>>>>>>>>>> internals. Input if you have time appreciated.
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> * HBASE-18596 A hbase1 cluster should be able to replicate
>>>> to a
>>>> >>>>>>>>>>> hbase2
>>>> >>>>>>>>>>> cluster; verify
>>>> >>>>>>>>>>> Esteban is looking at this one
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> * HBASE-9417 SecureBulkLoadEndpoint should be folded in core
>>>> >>>>>>>>>>> * HBASE-17143 Scan improvement
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> Our Coprocessor Interface needs a tough edit. It exposes
>>>> >>>>>>>>>>> implementations
>>>> >>>>>>>>>>> marked audience Private and returns implementations rather
>>>> than
>>>> >>>>>>>>>>> Interfaces.
>>>> >>>>>>>>>>> In a few locations, we allow returning an alternate
>>>> >>>>>>>>>>> implementation
>>>> >>>>>>>>>>> altogether which is probably something we don't want a CP
>>>> doing.
>>>> >>>>>>>>>>> To
>>>> >>>>>>>>>>> that
>>>> >>>>>>>>>>> end, the following issues started by Duo and Anoop need to
>>>> be
>>>> >>>>>>>>>>> taken
>>>> >>>>>>>>>>> to
>>>> >>>>>>>>>>> the
>>>> >>>>>>>>>>> finish line; ideally they'd have an owner:
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> * HBASE-18169 Coprocessor fix and cleanup before 2.0.0
>>>> release <=
>>>> >>>>>>>>>>> The
>>>> >>>>>>>>>>> umbrella issue.
>>>> >>>>>>>>>>> * HBASE-18298 RegionServerServices Interface cleanup for CP
>>>> >>>>>>>>>>> expose
>>>> >>>>>>>>>>> * HBASE-16769 Deprecate/remove PB references from
>>>> MasterObserver
>>>> >>>>>>>>>>> and
>>>> >>>>>>>>>>> RegionServerObserver
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> Nice-to-haves:
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> * HBASE-15284 Make TimeRange constructors IA.Private and
>>>> remove
>>>> >>>>>>>>>>> unused
>>>> >>>>>>>>>>> TimeRange constructors
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> * HBASE-10944 Remove all kv.getBuffer() and kv.getRow()
>>>> >>>>>>>>>>> references
>>>> >>>>>>>>>>> existing in the code
>>>> >>>>>>>>>>> This is the end of an old long-running project moving up on
>>>> to
>>>> >>>>>>>>>>> Cell
>>>> >>>>>>>>>>> Interface. We think it is done but for a few little items
>>>> >>>>>>>>>>> (deprecate
>>>> >>>>>>>>>>> KV
>>>> >>>>>>>>>>> methods in MR and provide Cell versions instead...)
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> * HBASE-13271 Table#puts(List<Put>) operation is
>>>> indeterminate;
>>>> >>>>>>>>>>> needs
>>>> >>>>>>>>>>> fixing
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> * HBASE-13346 Clean up Filter package for post 1.0
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> * HBASE-14255 Simplify Cell creation post 1.0
>>>> >>>>>>>>>>> * HBASE-14997
>>>> >>>>>>>>>>> Move compareOp and Comparators out of filter to client
>>>> package
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> * HBASE-13740 Stop using Hadoop private interfaces
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> What about:
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> * HBASE-18601 Remove Htrace 3.2
>>>> >>>>>>>>>>> As has been noted, the HTrace API is our 'trace' API.
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> If interested in any of the above and you need a legup,
>>>> just ask
>>>> >>>>>>>>>>> in
>>>> >>>>>>>>>>> the
>>>> >>>>>>>>>>> issue and I'll be by....
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> Thanks,
>>>> >>>>>>>>>>> St.Ack
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> On Mon, Aug 14, 2017 at 10:54 AM, Stack <st...@duboce.net>
>>>> >>>>>>>>>>> wrote:
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> Heads-up:
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> I'm about to put up an hbase-2.0.0-alpha2 Release Candidate.
>>>> >>>>>>>>>>>> Theme
>>>> >>>>>>>>>>>> is
>>>> >>>>>>>>>>>> updated dependencies, reliance on relocated popular libs
>>>> (guava,
>>>> >>>>>>>>>>>> netty,
>>>> >>>>>>>>>>>> protobuf), purge of checked-in generated src, and
>>>> >>>>>>>>>>>> master-carries-no-regions
>>>> >>>>>>>>>>>> by default.
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> alpha3 I hope will follow soon after (end-of-August?). Its
>>>> theme
>>>> >>>>>>>>>>>> will
>>>> >>>>>>>>>>>> be
>>>> >>>>>>>>>>>> settling the APIs and compatibility (At first blush, we
>>>> are not
>>>> >>>>>>>>>>>> looking too
>>>> >>>>>>>>>>>> bad; our Sean ran some tests over weekend that have hbase-1
>>>> >>>>>>>>>>>> client
>>>> >>>>>>>>>>>> running
>>>> >>>>>>>>>>>> against an hbase-2 cluster....). The Coprocessor Interface
>>>> >>>>>>>>>>>> revamp
>>>> >>>>>>>>>>>> should be
>>>> >>>>>>>>>>>> done by alpha3 (i.e. returning Interfaces rather than
>>>> >>>>>>>>>>>> Implementations, and
>>>> >>>>>>>>>>>> our shutdown of CPs accessing classes in hbase marked
>>>> >>>>>>>>>>>> InterfaceAudience).
>>>> >>>>>>>>>>>> We'll also have purged thirdparty classes from our API;
>>>> e.g.
>>>> >>>>>>>>>>>> guava
>>>> >>>>>>>>>>>> 0.12
>>>> >>>>>>>>>>>> Service showing through in our replication API and
>>>> protobufs in
>>>> >>>>>>>>>>>> Admin
>>>> >>>>>>>>>>>> Interface. On alpha3, we will have to do a bunch of
>>>> outreach to
>>>> >>>>>>>>>>>> make
>>>> >>>>>>>>>>>> sure
>>>> >>>>>>>>>>>> our downstreamers are up on what is coming down the pipe.
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> Beta1 in mid-September?
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> I encourage you to check out the items marked for hbase2:
>>>> >>>>>>>>>>>> https://issues.apache.org/jira
>>>> /projects/HBASE/versions/12327188
>>>> >>>>>>>>>>>> Edit
>>>> >>>>>>>>>>>> as
>>>> >>>>>>>>>>>> you see appropriate. Punt if you know the JIRA will not
>>>> get any
>>>> >>>>>>>>>>>> attention
>>>> >>>>>>>>>>>> in next month or so.
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> A bunch of issues marked blocker are unassigned. I'll
>>>> leave them
>>>> >>>>>>>>>>>> as
>>>> >>>>>>>>>>>> is
>>>> >>>>>>>>>>>> another while but I'll boot them soon.
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> While I have your attention:
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> + I think we should leave thrift version at 0.9.3. Moving
>>>> hbase
>>>> >>>>>>>>>>>> thrift
>>>> >>>>>>>>>>>> to 0.10.0 will break existing clients. The change is easy
>>>> enough
>>>> >>>>>>>>>>>> if
>>>> >>>>>>>>>>>> folks
>>>> >>>>>>>>>>>> need to upgrade their hbase thrift. See HBASE-18591.
>>>> >>>>>>>>>>>> + Upgrade from 0.94 is disallowed. You have to get to 1.0
>>>> first
>>>> >>>>>>>>>>>> (0.98?).
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> St.Ack
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> On Wed, Aug 2, 2017 at 9:43 AM, Stack <st...@duboce.net>
>>>> wrote:
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> On Tue, Aug 1, 2017 at 2:06 PM, Josh Elser <
>>>> elserj@apache.org>
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>>> wrote:
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>> On 7/31/17 9:00 AM, Stack wrote:
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>> On Mon, Jul 24, 2017 at 12:25 PM, Josh Elser<
>>>> >>>>>>>>>>>>>> elserj@apache.org>
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>> wrote:
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>> ...
>>>> >>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>> I like the idea of this also hitting 2.0 as it would
>>>> make the
>>>> >>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>> feature a
>>>> >>>>>>>>>>>>>>>> bit more "real", but am obviously a little nervous (I
>>>> have
>>>> >>>>>>>>>>>>>>>> no
>>>> >>>>>>>>>>>>>>>> reason
>>>> >>>>>>>>>>>>>>>> to be
>>>> >>>>>>>>>>>>>>>> nervous though). I am pretty happy with the feature in
>>>> terms
>>>> >>>>>>>>>>>>>>>> of
>>>> >>>>>>>>>>>>>>>> how
>>>> >>>>>>>>>>>>>>>> much it
>>>> >>>>>>>>>>>>>>>> is covered via testing.
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/HBASE-17748
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>> Sounds good to me. Whats involved? Backport? If so, +1
>>>> Josh.
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>> Last think on space quota says that need doc too. See
>>>> 'Space
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>> Quota' in
>>>> >>>>>>>>>>>>>>> here:
>>>> >>>>>>>>>>>>>>> https://docs.google.com/docume
>>>> nt/d/1WCsVlnHjJeKUcl7wHwqb4z9i
>>>> >>>>>>>>>>>>>>> Eu_ktczrlKHK8N4SZzs/edit#heading=h.wuw3a6jukzo5
>>>> >>>>>>>>>>>>>>> Does this little section need an update Josh?
>>>> >>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>> Thanks,
>>>> >>>>>>>>>>>>>>> S
>>>> >>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>> Yep, just a couple of cherry-picks. Good test coverage
>>>> and
>>>> >>>>>>>>>>>>>>> some
>>>> >>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>> docs
>>>> >>>>>>>>>>>>>> already included for 17748.  Happy to put that on my
>>>> plate if
>>>> >>>>>>>>>>>>>> you're good
>>>> >>>>>>>>>>>>>> with it. I can reasonably assume that no one is against
>>>> it :)
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>> I think I had knocked out docs for the "phase 1" stuff
>>>> before
>>>> >>>>>>>>>>>>>> we
>>>> >>>>>>>>>>>>>> merged it in from the original feature branch. I'll
>>>> double
>>>> >>>>>>>>>>>>>> check
>>>> >>>>>>>>>>>>>> and update
>>>> >>>>>>>>>>>>>> the gdoc. Perhaps this was just a timing thing.
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>> Thanks Josh,
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>> S
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>
>>>> >>>>>>>>>
>>>> >>>>>>>
>>>> >>>>>>
>>>> >>>>
>>>> >>
>>>>
>>>
>>>
>>
>

Re: Moving 2.0 forward

[Stack <st...@duboce.net>]

Update on hbase-2.0.0-beta-1. I'm going to put up a beta-1 RC before
Christmas day.

Here is the list [1]. We're doing pretty good. We've fixed a load since the
last update including some nice cleanups  two weeks ago (e.g. undoing
client dependency on curator/zk watching making the dependence read-only)
but there are still a few hairy ones hanging out there.

 * HBASE-18946 Stochastic load balancer assigns replica regions to the same
RS
   This one is proving a little tough. Ram has been banging his head on it.
We'll be able to clear a bunch of test failures once this is done.

 * HBASE-19112 Suspect methods on Cell to be deprecated
   Good discussion up on the JIRA. We need to be crystal clear around Cell
and derivatives API and Audience when we ship 2.0.

 * HBASE-17204 Make L2 off heap cache default ON
   We need to try this at least so offheaping can be default.

Making asyncfswal default is also ongoing, HBASE-16890
<https://issues.apache.org/jira/browse/HBASE-16890>, making good progress.

I punted HBASE-19147 "All branch-2 unit tests pass" out to beta-2. Our unit
test story has gotten better and a few of us are actively working on
flakies [2] but we may make a beta-1 w/o shutting down all failures.

Speak up if you need help on an issue or if you think we are missing items
from our list.

Thanks for all the hard work,

The hbase-2.0.0 RM.


1. https://issues.apache.org/jira/projects/HBASE/versions/12340861
2.
https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html

On Thu, Nov 16, 2017 at 3:48 PM, Stack <st...@duboce.net> wrote:

> hbase-2.0.0-beta-1 update (Reminder, beta-1 is where we finish last
> remaining features and apply final polish to API. There will be a beta-2
> but it is about upgrade/rolling-upgrade and bug fixes ONLY).
>
> Myself and Mr. Drob did a pass over the outstanding hbase-2.0.0-beta-1
> list this morning. See here [1].
>
> There are about ~12 issues in progress with most of these about to land.
> There are 37 TODO. Many of these are tests we need to run, some are related
> to the backup/restore, but a good few are meaty w/o assignees.
>
> The awkward outstanding ones as I see it are the below:
>
> HBASE-18946 Stochastic load balancer assigns replica regions to the same RS
> HBASE-17204 Make L2 off heap cache default ON
> HBASE-19112 Suspect methods on Cell to be deprecated
> HBASE-19147 All branch-2 unit tests pass
>
> We need to make progress on the above or punt on them (can't punt on the
> last one though).
>
> Any ideas on what configs we should update in hbase2? Dump ideas into:
> HBASE-19148 Edit of default configuration
>
> Still hoping for an early December beta-1 RC. beta-2 hopefully will be
> close behind.
>
> Comments? Thoughts?
>
> Thanks all,
> S
>
> 1. https://issues.apache.org/jira/projects/HBASE/versions/12340861
>
> On Tue, Oct 31, 2017 at 2:56 PM, Stack <st...@duboce.net> wrote:
>
>>
>>
>> On Tue, Oct 31, 2017 at 2:31 PM, Mike Drob <md...@apache.org> wrote:
>>
>>> Hoping to keep momentum going from our Stack working on alpha4, I tried
>>> to
>>> take a stab at triaging some of the open beta-1 issues.
>>>
>>> I moved some docs stuff out form beta-1 to 2.0-GA, if it gets done sooner
>>> then I'm happy to see it pulled back in. Trying to balance optimism with
>>> realism here, and knowing that documentation unfortunately often gets
>>> pushed to the back-burner.
>>>
>>> Also, I poked some folks on unassigned issues that they've filed for
>>> beta-1, especially in the last few days. If issues don't have an owner
>>> they
>>> are unlikely to get worked. I chatted with stack and he agreed to take on
>>> some of the tasks, but there's a lot of surface area to cover.
>>>
>>> If you you're working on issues that are critical for beta-1, please mark
>>> them as such. Then the rest of the community will know to help prioritize
>>> feedback and reviews there.
>>>
>>> Do we have a general theme for the betas like we did with the alphas?
>>>
>>> Beta1 is upgrades work from branch1, beta2 is rolling upgrades work as
>>> well? Continue to work on tests throughout?
>>>
>>>
>> Thanks Mike for helping to kick off the beta-1 train.
>>
>> Regards a theme for beta-1, I'd like it to be 'finish'; beta-1 is what we
>> are going to ship (beta-2 is rolling upgrade and any minor items turned up
>> in testing/burn-in).
>>
>> Thanks,
>> S
>>
>>
>>
>>> Mike
>>>
>>> On Tue, Oct 31, 2017 at 10:04 AM, Josh Elser <el...@apache.org> wrote:
>>>
>>> > +1 go from my POV.
>>> >
>>> >
>>> > On 10/31/17 10:07 AM, Stack wrote:
>>> >
>>> >> I want to push an alpha-4 today. A few items didn't make it
>>> (HBASE-19092).
>>> >> They need more time. We'll pull them in for beta-1. CP API is
>>> basically
>>> >> done. There may be some changes for beta-1 but hopefully only changes
>>> >> informed by experience trying to port an existing Coprocessor to
>>> hbase2.
>>> >>
>>> >> Shout if there is anything that needs to make alpha-4.
>>> >>
>>> >> Thanks,
>>> >> St.Ack
>>> >>
>>> >>
>>> >> On Sat, Oct 28, 2017 at 2:48 PM, Josh Elser <jo...@gmail.com>
>>> wrote:
>>> >>
>>> >> Yup, that was going to be my plan, Mike!
>>> >>>
>>> >>> Making a pass now, and will check back later tonight again. I see
>>> others
>>> >>> have already done some work today on this front.
>>> >>>
>>> >>>
>>> >>> On 10/27/17 11:38 PM, Mike Drob wrote:
>>> >>>
>>> >>> Josh - Do you want to kick off a bunch of QA runs? (Do you know how
>>> to do
>>> >>>> it directly on the jenkins job, so you don't have to bother with
>>> JIRA
>>> >>>> uploads)
>>> >>>>
>>> >>>> If you're busy, then I can make time tomorrow or Sunday to kick off
>>> >>>> jobs,
>>> >>>> but I want to make sure we're not duplicating effort and jenkins
>>> cycles.
>>> >>>>
>>> >>>> On Fri, Oct 27, 2017 at 7:10 PM, Josh Elser <el...@apache.org>
>>> wrote:
>>> >>>>
>>> >>>> My turn to bump ;)
>>> >>>>
>>> >>>>>
>>> >>>>> By my take: HBASE-18770 and HBASE-19092 are the only issues that
>>> remain
>>> >>>>> needing some more work. The rest are just awaiting a good QA run.
>>> >>>>>
>>> >>>>> Unless I hear otherwise, I'll try to keep an eye on things over the
>>> >>>>> weekend, bump them along as necessary, and get them committed.
>>> Would be
>>> >>>>> great to be able get a vote up on Monday.
>>> >>>>>
>>> >>>>>
>>> >>>>> On 10/24/17 6:03 PM, Stack wrote:
>>> >>>>>
>>> >>>>> Chatting with my coworker Mr. Mike Drob, we were batting back and
>>> forth
>>> >>>>>
>>> >>>>>> what remains to be done. Surfacing our thoughts here so you all
>>> clued
>>> >>>>>> in....Or if you think otherwise, please speak up.
>>> >>>>>>
>>> >>>>>> We have ~13 issues to land:
>>> >>>>>> https://issues.apache.org/jira/projects/HBASE/versions/12341594
>>> About
>>> >>>>>> two
>>> >>>>>> are meta-issues that are about process which leaves 11.
>>> >>>>>>
>>> >>>>>> Duo and Zheng Hu are to merge the FilterList fixes improvements
>>> >>>>>> (HBASE-19057, HBASE-18410 et al.). These are blocker because some
>>> >>>>>> changed
>>> >>>>>> API/semantic that we need to get out earlier rather than later.
>>> >>>>>>
>>> >>>>>> Once the above is merged, HBASE-13346, change of Filter method
>>> names
>>> >>>>>> to
>>> >>>>>> mention Cell instead of KeyValue can land.
>>> >>>>>>
>>> >>>>>> HBASE-199048 needs a review (Anoop will probably do it), removing
>>> >>>>>> IA.Private objects as params to MasterObserver... Hopefully this
>>> goes
>>> >>>>>> in
>>> >>>>>> soon.
>>> >>>>>>
>>> >>>>>> Duo is hard at work on trackers for flush and compaction for CPs
>>> >>>>>> (HBASE-18905). How is HBASE-19033 looking Duo (facility for
>>> Tephra)?
>>> >>>>>>
>>> >>>>>> I think HBASE-18906 (Phoenix Region#waitFor...) will evaporate
>>> after
>>> >>>>>> Duo
>>> >>>>>> is
>>> >>>>>> done w/ his work above.
>>> >>>>>>
>>> >>>>>> I'm on HBASE-18770 bypass and HBASE-19077 restore some parity
>>> after
>>> >>>>>> all
>>> >>>>>> the
>>> >>>>>> purges allowing CPs do direct calls against Regions in same Host.
>>> >>>>>>
>>> >>>>>> Anoop is on HBASE-19047 (Fixes) and Ram on cleanup of CellUtil.
>>> >>>>>>
>>> >>>>>> Another day or two?
>>> >>>>>>
>>> >>>>>> St.Ack
>>> >>>>>>
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> On Mon, Oct 23, 2017 at 2:52 PM, Stack <st...@duboce.net> wrote:
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> On Mon, Oct 23, 2017 at 11:59 AM, Josh Elser <el...@apache.org>
>>> >>>>>> wrote:
>>> >>>>>>
>>> >>>>>>>
>>> >>>>>>> +1
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>> I was trying to work on helping out on the outstanding alpha-4
>>> stuff
>>> >>>>>>>> last
>>> >>>>>>>> week -- will be continuing to try to do the same this week.
>>> >>>>>>>>
>>> >>>>>>>> If you need any help, Stack, or if others need reviews where I
>>> >>>>>>>> haven't
>>> >>>>>>>> noticed on my own: feel free to @mention me.
>>> >>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>> Thanks for the offer Josh. All items seem assigned and are being
>>> >>>>>>>>
>>> >>>>>>>> actively
>>> >>>>>>> worked on. If you get a moment, reviews by you (or anyone else)
>>> helps
>>> >>>>>>> move
>>> >>>>>>> the process along.
>>> >>>>>>>
>>> >>>>>>> We need to merge in HBASE-18410 branch to pick up Filter
>>> >>>>>>> improvements.
>>> >>>>>>> Then HBASE-13346 can go in.
>>> >>>>>>>
>>> >>>>>>> You are already helping out on HBASE-18906, thanks. Looks like
>>> that
>>> >>>>>>> will
>>> >>>>>>> be addressed by other alpha-4s about to land.
>>> >>>>>>>
>>> >>>>>>> St.Ack
>>> >>>>>>> TODOs: https://issues.apache.org/jira
>>> /projects/HBASE/versions/12341
>>> >>>>>>> 594
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> On 10/23/17 12:53 PM, Stack wrote:
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>> (Reviving this thread)
>>> >>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>>> Lets push out alpha-4 this week. Alpha-4 is the release that
>>> has
>>> >>>>>>>>> the
>>> >>>>>>>>> refactor of the Coprocessor API shutting down access to
>>> internals
>>> >>>>>>>>> marked
>>> >>>>>>>>> InterfaceAudience.Private.
>>> >>>>>>>>>
>>> >>>>>>>>> The outstanding list is here:
>>> >>>>>>>>> https://issues.apache.org/jira/projects/HBASE/versions/12341
>>> 594
>>> >>>>>>>>>
>>> >>>>>>>>> Please push in anything marked alpha-4 that belongs to you.
>>> >>>>>>>>>
>>> >>>>>>>>> If issue, talk out loud on this thread. If you need a review to
>>> >>>>>>>>> land
>>> >>>>>>>>> an
>>> >>>>>>>>> item, shout on the issue and here; we'll help you out.
>>> >>>>>>>>>
>>> >>>>>>>>> As is, items are coming along nicely I'd say. We need to merge
>>> the
>>> >>>>>>>>> filter
>>> >>>>>>>>> branch -- HBASE-18410 -- so APIs are finished for hbase2.
>>> >>>>>>>>>
>>> >>>>>>>>> Post alpha-4, we'll have to hunt down our downstreamers and
>>> help
>>> >>>>>>>>> them
>>> >>>>>>>>> test
>>> >>>>>>>>> on top of alpha-4 so rolling into beta-1, we have confidence
>>> our
>>> >>>>>>>>> downstreamers know what to expect (or we discover what we
>>> missed
>>> >>>>>>>>> BEFORE
>>> >>>>>>>>> we
>>> >>>>>>>>> beta-1).
>>> >>>>>>>>>
>>> >>>>>>>>> Thanks for time,
>>> >>>>>>>>> S
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>> On Fri, Sep 8, 2017 at 2:04 PM, Stack <st...@duboce.net>
>>> wrote:
>>> >>>>>>>>>
>>> >>>>>>>>> I'll put up an alpha3 RC Monday, probably Monday night. That
>>> should
>>> >>>>>>>>> be
>>> >>>>>>>>>
>>> >>>>>>>>> time, if we all sprint, for the public-facing API fixes to be
>>> done.
>>> >>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>> I had a bunch of Coprocessor refactor and fixup scheduled for
>>> >>>>>>>>>> alpha3
>>> >>>>>>>>>> but
>>> >>>>>>>>>> it is plain that more time is needed (in spite of valiant
>>> effort
>>> >>>>>>>>>> so
>>> >>>>>>>>>> far
>>> >>>>>>>>>> by
>>> >>>>>>>>>> Anoop, Duo, Appy, etc.). Therefore, lets run a 2.0.0-alpha-4
>>> whose
>>> >>>>>>>>>> theme is
>>> >>>>>>>>>> "Coprocessor Fixup". Hopefully we can put an alpha-4 up by the
>>> >>>>>>>>>> following
>>> >>>>>>>>>> week.
>>> >>>>>>>>>>
>>> >>>>>>>>>> We should then be ready for beta (beta == no new features, no
>>> API
>>> >>>>>>>>>> changes,
>>> >>>>>>>>>> just fixes).
>>> >>>>>>>>>>
>>> >>>>>>>>>> Thanks,
>>> >>>>>>>>>> St.Ack
>>> >>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>> On Thu, Aug 17, 2017 at 12:35 PM, Stack <st...@duboce.net>
>>> wrote:
>>> >>>>>>>>>>
>>> >>>>>>>>>> I put up the hbase-2.0.0-alpha2 release candidate. Please
>>> vote on
>>> >>>>>>>>>> it.
>>> >>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>> For hbase-2.0.0-alpha3, the theme is solidifying API. I hope
>>> to
>>> >>>>>>>>>>> get a
>>> >>>>>>>>>>> release out in the next week or so.
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> I did a weeding of 2.0.0 issues over the last day. If folks
>>> are
>>> >>>>>>>>>>> interested in helping out, below are the items I think we
>>> need
>>> >>>>>>>>>>> done
>>> >>>>>>>>>>> for
>>> >>>>>>>>>>> alpha3 (below are at least 'Critical' status, are API
>>> possibly
>>> >>>>>>>>>>> altering
>>> >>>>>>>>>>> items, and are absent those JIRAs that are making active
>>> >>>>>>>>>>> progress,
>>> >>>>>>>>>>> i.e. the
>>> >>>>>>>>>>> HTD/HCD revamp by Chia-Ping Tsai). A project NOT listed that
>>> >>>>>>>>>>> needs
>>> >>>>>>>>>>> doing is
>>> >>>>>>>>>>> what Andrew did comparing 1.3. and 1.4 APIs
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> * HBASE-18622 Mitigate compatibility concerns between
>>> branch-1
>>> >>>>>>>>>>> and
>>> >>>>>>>>>>> branch-2
>>> >>>>>>>>>>> This is to do what Andrew did between 1.3 and 1.4 branches
>>> only
>>> >>>>>>>>>>> do
>>> >>>>>>>>>>> it
>>> >>>>>>>>>>> between branch-1 and branch-2.
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> * HBASE-10462 Recategorize some of the client facing Public /
>>> >>>>>>>>>>> Private
>>> >>>>>>>>>>> interfaces
>>> >>>>>>>>>>> This one is almost done. It could do with a finish,
>>> attention to
>>> >>>>>>>>>>> the
>>> >>>>>>>>>>> items in last comment, and then our codebase could do with
>>> >>>>>>>>>>> another
>>> >>>>>>>>>>> sweep
>>> >>>>>>>>>>> after the spirit of this issue since a bunch has gone in
>>> since
>>> >>>>>>>>>>> the
>>> >>>>>>>>>>> pass
>>> >>>>>>>>>>> that was the basis of this issue.
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> * HBASE-10504 Define Replication Interface
>>> >>>>>>>>>>> I was going to take a crack at this as part of the revamp
>>> forced
>>> >>>>>>>>>>> by
>>> >>>>>>>>>>> 'HBASE-15982 Interface ReplicationEndpoint extends Guava's
>>> >>>>>>>>>>> Service'
>>> >>>>>>>>>>> but if
>>> >>>>>>>>>>> anyone else is interested, be my guest.
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> * HBASE-14996 Some more API cleanup for 2.0
>>> >>>>>>>>>>> Has a bunch of subtasks, some of which are being worked on.
>>> Needs
>>> >>>>>>>>>>> finishing.
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> * HBASE-14998 Unify synchronous and asynchronous methods in
>>> Admin
>>> >>>>>>>>>>> and
>>> >>>>>>>>>>> cleanup
>>> >>>>>>>>>>> Needs a pass. Small issue I think. Could also look at new
>>> >>>>>>>>>>> AsyncClient
>>> >>>>>>>>>>> and
>>> >>>>>>>>>>> make sure symmetry.
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> * HBASE-15607 Remove PB references from Admin for 2.0
>>> >>>>>>>>>>> Predicated on result of an ongoing DISCUSSION thread but
>>> needs to
>>> >>>>>>>>>>> be
>>> >>>>>>>>>>> done.
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> Rolling upgrade will have implications for our API. Would be
>>> good
>>> >>>>>>>>>>> to
>>> >>>>>>>>>>> try
>>> >>>>>>>>>>> it and figure what needs fixup (as said above, according to
>>> trial
>>> >>>>>>>>>>> by
>>> >>>>>>>>>>> Sean,
>>> >>>>>>>>>>> we might not be too bad here):
>>> >>>>>>>>>>> * HBASE-16060 1.x clients cannot access table state talking
>>> to
>>> >>>>>>>>>>> 2.0
>>> >>>>>>>>>>> cluster
>>> >>>>>>>>>>> * HBASE-16550 Procedure v2 - Add AM compatibility for 2.x
>>> Master
>>> >>>>>>>>>>> and
>>> >>>>>>>>>>> 1.x
>>> >>>>>>>>>>> RSs; i.e. support Rolling Upgrade from hbase-1 to -2.
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> * HBASE-17442 Move most of the replication related classes to
>>> >>>>>>>>>>> hbase-server package
>>> >>>>>>>>>>> The above would be good to do generally but it may make for
>>> >>>>>>>>>>> ripples
>>> >>>>>>>>>>> in
>>> >>>>>>>>>>> API so would be good to do now.
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> * HBASE-18106 Redo ProcedureInfo and LockInfo
>>> >>>>>>>>>>> Balazs is working on this. The idea is that we avoid adding
>>> two
>>> >>>>>>>>>>> new
>>> >>>>>>>>>>> types
>>> >>>>>>>>>>> to our API, two types that are nought but curtailed,
>>> read-only
>>> >>>>>>>>>>> views
>>> >>>>>>>>>>> on
>>> >>>>>>>>>>> internals. Input if you have time appreciated.
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> * HBASE-18596 A hbase1 cluster should be able to replicate
>>> to a
>>> >>>>>>>>>>> hbase2
>>> >>>>>>>>>>> cluster; verify
>>> >>>>>>>>>>> Esteban is looking at this one
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> * HBASE-9417 SecureBulkLoadEndpoint should be folded in core
>>> >>>>>>>>>>> * HBASE-17143 Scan improvement
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> Our Coprocessor Interface needs a tough edit. It exposes
>>> >>>>>>>>>>> implementations
>>> >>>>>>>>>>> marked audience Private and returns implementations rather
>>> than
>>> >>>>>>>>>>> Interfaces.
>>> >>>>>>>>>>> In a few locations, we allow returning an alternate
>>> >>>>>>>>>>> implementation
>>> >>>>>>>>>>> altogether which is probably something we don't want a CP
>>> doing.
>>> >>>>>>>>>>> To
>>> >>>>>>>>>>> that
>>> >>>>>>>>>>> end, the following issues started by Duo and Anoop need to be
>>> >>>>>>>>>>> taken
>>> >>>>>>>>>>> to
>>> >>>>>>>>>>> the
>>> >>>>>>>>>>> finish line; ideally they'd have an owner:
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> * HBASE-18169 Coprocessor fix and cleanup before 2.0.0
>>> release <=
>>> >>>>>>>>>>> The
>>> >>>>>>>>>>> umbrella issue.
>>> >>>>>>>>>>> * HBASE-18298 RegionServerServices Interface cleanup for CP
>>> >>>>>>>>>>> expose
>>> >>>>>>>>>>> * HBASE-16769 Deprecate/remove PB references from
>>> MasterObserver
>>> >>>>>>>>>>> and
>>> >>>>>>>>>>> RegionServerObserver
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> Nice-to-haves:
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> * HBASE-15284 Make TimeRange constructors IA.Private and
>>> remove
>>> >>>>>>>>>>> unused
>>> >>>>>>>>>>> TimeRange constructors
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> * HBASE-10944 Remove all kv.getBuffer() and kv.getRow()
>>> >>>>>>>>>>> references
>>> >>>>>>>>>>> existing in the code
>>> >>>>>>>>>>> This is the end of an old long-running project moving up on
>>> to
>>> >>>>>>>>>>> Cell
>>> >>>>>>>>>>> Interface. We think it is done but for a few little items
>>> >>>>>>>>>>> (deprecate
>>> >>>>>>>>>>> KV
>>> >>>>>>>>>>> methods in MR and provide Cell versions instead...)
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> * HBASE-13271 Table#puts(List<Put>) operation is
>>> indeterminate;
>>> >>>>>>>>>>> needs
>>> >>>>>>>>>>> fixing
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> * HBASE-13346 Clean up Filter package for post 1.0
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> * HBASE-14255 Simplify Cell creation post 1.0
>>> >>>>>>>>>>> * HBASE-14997
>>> >>>>>>>>>>> Move compareOp and Comparators out of filter to client
>>> package
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> * HBASE-13740 Stop using Hadoop private interfaces
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> What about:
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> * HBASE-18601 Remove Htrace 3.2
>>> >>>>>>>>>>> As has been noted, the HTrace API is our 'trace' API.
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> If interested in any of the above and you need a legup, just
>>> ask
>>> >>>>>>>>>>> in
>>> >>>>>>>>>>> the
>>> >>>>>>>>>>> issue and I'll be by....
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> Thanks,
>>> >>>>>>>>>>> St.Ack
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> On Mon, Aug 14, 2017 at 10:54 AM, Stack <st...@duboce.net>
>>> >>>>>>>>>>> wrote:
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> Heads-up:
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> I'm about to put up an hbase-2.0.0-alpha2 Release Candidate.
>>> >>>>>>>>>>>> Theme
>>> >>>>>>>>>>>> is
>>> >>>>>>>>>>>> updated dependencies, reliance on relocated popular libs
>>> (guava,
>>> >>>>>>>>>>>> netty,
>>> >>>>>>>>>>>> protobuf), purge of checked-in generated src, and
>>> >>>>>>>>>>>> master-carries-no-regions
>>> >>>>>>>>>>>> by default.
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> alpha3 I hope will follow soon after (end-of-August?). Its
>>> theme
>>> >>>>>>>>>>>> will
>>> >>>>>>>>>>>> be
>>> >>>>>>>>>>>> settling the APIs and compatibility (At first blush, we are
>>> not
>>> >>>>>>>>>>>> looking too
>>> >>>>>>>>>>>> bad; our Sean ran some tests over weekend that have hbase-1
>>> >>>>>>>>>>>> client
>>> >>>>>>>>>>>> running
>>> >>>>>>>>>>>> against an hbase-2 cluster....). The Coprocessor Interface
>>> >>>>>>>>>>>> revamp
>>> >>>>>>>>>>>> should be
>>> >>>>>>>>>>>> done by alpha3 (i.e. returning Interfaces rather than
>>> >>>>>>>>>>>> Implementations, and
>>> >>>>>>>>>>>> our shutdown of CPs accessing classes in hbase marked
>>> >>>>>>>>>>>> InterfaceAudience).
>>> >>>>>>>>>>>> We'll also have purged thirdparty classes from our API; e.g.
>>> >>>>>>>>>>>> guava
>>> >>>>>>>>>>>> 0.12
>>> >>>>>>>>>>>> Service showing through in our replication API and
>>> protobufs in
>>> >>>>>>>>>>>> Admin
>>> >>>>>>>>>>>> Interface. On alpha3, we will have to do a bunch of
>>> outreach to
>>> >>>>>>>>>>>> make
>>> >>>>>>>>>>>> sure
>>> >>>>>>>>>>>> our downstreamers are up on what is coming down the pipe.
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> Beta1 in mid-September?
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> I encourage you to check out the items marked for hbase2:
>>> >>>>>>>>>>>> https://issues.apache.org/jira
>>> /projects/HBASE/versions/12327188
>>> >>>>>>>>>>>> Edit
>>> >>>>>>>>>>>> as
>>> >>>>>>>>>>>> you see appropriate. Punt if you know the JIRA will not get
>>> any
>>> >>>>>>>>>>>> attention
>>> >>>>>>>>>>>> in next month or so.
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> A bunch of issues marked blocker are unassigned. I'll leave
>>> them
>>> >>>>>>>>>>>> as
>>> >>>>>>>>>>>> is
>>> >>>>>>>>>>>> another while but I'll boot them soon.
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> While I have your attention:
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> + I think we should leave thrift version at 0.9.3. Moving
>>> hbase
>>> >>>>>>>>>>>> thrift
>>> >>>>>>>>>>>> to 0.10.0 will break existing clients. The change is easy
>>> enough
>>> >>>>>>>>>>>> if
>>> >>>>>>>>>>>> folks
>>> >>>>>>>>>>>> need to upgrade their hbase thrift. See HBASE-18591.
>>> >>>>>>>>>>>> + Upgrade from 0.94 is disallowed. You have to get to 1.0
>>> first
>>> >>>>>>>>>>>> (0.98?).
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> St.Ack
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> On Wed, Aug 2, 2017 at 9:43 AM, Stack <st...@duboce.net>
>>> wrote:
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> On Tue, Aug 1, 2017 at 2:06 PM, Josh Elser <
>>> elserj@apache.org>
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> On 7/31/17 9:00 AM, Stack wrote:
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> On Mon, Jul 24, 2017 at 12:25 PM, Josh Elser<
>>> >>>>>>>>>>>>>> elserj@apache.org>
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> ...
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> I like the idea of this also hitting 2.0 as it would
>>> make the
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> feature a
>>> >>>>>>>>>>>>>>>> bit more "real", but am obviously a little nervous (I
>>> have
>>> >>>>>>>>>>>>>>>> no
>>> >>>>>>>>>>>>>>>> reason
>>> >>>>>>>>>>>>>>>> to be
>>> >>>>>>>>>>>>>>>> nervous though). I am pretty happy with the feature in
>>> terms
>>> >>>>>>>>>>>>>>>> of
>>> >>>>>>>>>>>>>>>> how
>>> >>>>>>>>>>>>>>>> much it
>>> >>>>>>>>>>>>>>>> is covered via testing.
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/HBASE-17748
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> Sounds good to me. Whats involved? Backport? If so, +1
>>> Josh.
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> Last think on space quota says that need doc too. See
>>> 'Space
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> Quota' in
>>> >>>>>>>>>>>>>>> here:
>>> >>>>>>>>>>>>>>> https://docs.google.com/docume
>>> nt/d/1WCsVlnHjJeKUcl7wHwqb4z9i
>>> >>>>>>>>>>>>>>> Eu_ktczrlKHK8N4SZzs/edit#heading=h.wuw3a6jukzo5
>>> >>>>>>>>>>>>>>> Does this little section need an update Josh?
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> Thanks,
>>> >>>>>>>>>>>>>>> S
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> Yep, just a couple of cherry-picks. Good test coverage
>>> and
>>> >>>>>>>>>>>>>>> some
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> docs
>>> >>>>>>>>>>>>>> already included for 17748.  Happy to put that on my
>>> plate if
>>> >>>>>>>>>>>>>> you're good
>>> >>>>>>>>>>>>>> with it. I can reasonably assume that no one is against
>>> it :)
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> I think I had knocked out docs for the "phase 1" stuff
>>> before
>>> >>>>>>>>>>>>>> we
>>> >>>>>>>>>>>>>> merged it in from the original feature branch. I'll double
>>> >>>>>>>>>>>>>> check
>>> >>>>>>>>>>>>>> and update
>>> >>>>>>>>>>>>>> the gdoc. Perhaps this was just a timing thing.
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> Thanks Josh,
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> S
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>
>>> >>>>>>
>>> >>>>
>>> >>
>>>
>>
>>
>

Re: Moving 2.0 forward

[Stack <st...@duboce.net>]

On Thu, Nov 16, 2017 at 8:08 PM, ramkrishna vasudevan <
ramkrishna.s.vasudevan@gmail.com> wrote:

> HBASE-18946 Stochastic load balancer assigns replica regions to the same RS
>
> Am on this today and probably will put up another patch.
>
> Regards
> Ram
>
> Thanks Ram,
St.Ack



> On Fri, Nov 17, 2017 at 5:18 AM, Stack <st...@duboce.net> wrote:
>
> > hbase-2.0.0-beta-1 update (Reminder, beta-1 is where we finish last
> > remaining features and apply final polish to API. There will be a beta-2
> > but it is about upgrade/rolling-upgrade and bug fixes ONLY).
> >
> > Myself and Mr. Drob did a pass over the outstanding hbase-2.0.0-beta-1
> list
> > this morning. See here [1].
> >
> > There are about ~12 issues in progress with most of these about to land.
> > There are 37 TODO. Many of these are tests we need to run, some are
> related
> > to the backup/restore, but a good few are meaty w/o assignees.
> >
> > The awkward outstanding ones as I see it are the below:
> >
> > HBASE-18946 Stochastic load balancer assigns replica regions to the same
> RS
> > HBASE-17204 Make L2 off heap cache default ON
> > HBASE-19112 Suspect methods on Cell to be deprecated
> > HBASE-19147 All branch-2 unit tests pass
> >
> > We need to make progress on the above or punt on them (can't punt on the
> > last one though).
> >
> > Any ideas on what configs we should update in hbase2? Dump ideas into:
> > HBASE-19148 Edit of default configuration
> >
> > Still hoping for an early December beta-1 RC. beta-2 hopefully will be
> > close behind.
> >
> > Comments? Thoughts?
> >
> > Thanks all,
> > S
> >
> > 1. https://issues.apache.org/jira/projects/HBASE/versions/12340861
> >
> > On Tue, Oct 31, 2017 at 2:56 PM, Stack <st...@duboce.net> wrote:
> >
> > >
> > >
> > > On Tue, Oct 31, 2017 at 2:31 PM, Mike Drob <md...@apache.org> wrote:
> > >
> > >> Hoping to keep momentum going from our Stack working on alpha4, I
> tried
> > to
> > >> take a stab at triaging some of the open beta-1 issues.
> > >>
> > >> I moved some docs stuff out form beta-1 to 2.0-GA, if it gets done
> > sooner
> > >> then I'm happy to see it pulled back in. Trying to balance optimism
> with
> > >> realism here, and knowing that documentation unfortunately often gets
> > >> pushed to the back-burner.
> > >>
> > >> Also, I poked some folks on unassigned issues that they've filed for
> > >> beta-1, especially in the last few days. If issues don't have an owner
> > >> they
> > >> are unlikely to get worked. I chatted with stack and he agreed to take
> > on
> > >> some of the tasks, but there's a lot of surface area to cover.
> > >>
> > >> If you you're working on issues that are critical for beta-1, please
> > mark
> > >> them as such. Then the rest of the community will know to help
> > prioritize
> > >> feedback and reviews there.
> > >>
> > >> Do we have a general theme for the betas like we did with the alphas?
> > >>
> > >> Beta1 is upgrades work from branch1, beta2 is rolling upgrades work as
> > >> well? Continue to work on tests throughout?
> > >>
> > >>
> > > Thanks Mike for helping to kick off the beta-1 train.
> > >
> > > Regards a theme for beta-1, I'd like it to be 'finish'; beta-1 is what
> we
> > > are going to ship (beta-2 is rolling upgrade and any minor items turned
> > up
> > > in testing/burn-in).
> > >
> > > Thanks,
> > > S
> > >
> > >
> > >
> > >> Mike
> > >>
> > >> On Tue, Oct 31, 2017 at 10:04 AM, Josh Elser <el...@apache.org>
> wrote:
> > >>
> > >> > +1 go from my POV.
> > >> >
> > >> >
> > >> > On 10/31/17 10:07 AM, Stack wrote:
> > >> >
> > >> >> I want to push an alpha-4 today. A few items didn't make it
> > >> (HBASE-19092).
> > >> >> They need more time. We'll pull them in for beta-1. CP API is
> > basically
> > >> >> done. There may be some changes for beta-1 but hopefully only
> changes
> > >> >> informed by experience trying to port an existing Coprocessor to
> > >> hbase2.
> > >> >>
> > >> >> Shout if there is anything that needs to make alpha-4.
> > >> >>
> > >> >> Thanks,
> > >> >> St.Ack
> > >> >>
> > >> >>
> > >> >> On Sat, Oct 28, 2017 at 2:48 PM, Josh Elser <jo...@gmail.com>
> > >> wrote:
> > >> >>
> > >> >> Yup, that was going to be my plan, Mike!
> > >> >>>
> > >> >>> Making a pass now, and will check back later tonight again. I see
> > >> others
> > >> >>> have already done some work today on this front.
> > >> >>>
> > >> >>>
> > >> >>> On 10/27/17 11:38 PM, Mike Drob wrote:
> > >> >>>
> > >> >>> Josh - Do you want to kick off a bunch of QA runs? (Do you know
> how
> > >> to do
> > >> >>>> it directly on the jenkins job, so you don't have to bother with
> > JIRA
> > >> >>>> uploads)
> > >> >>>>
> > >> >>>> If you're busy, then I can make time tomorrow or Sunday to kick
> off
> > >> >>>> jobs,
> > >> >>>> but I want to make sure we're not duplicating effort and jenkins
> > >> cycles.
> > >> >>>>
> > >> >>>> On Fri, Oct 27, 2017 at 7:10 PM, Josh Elser <el...@apache.org>
> > >> wrote:
> > >> >>>>
> > >> >>>> My turn to bump ;)
> > >> >>>>
> > >> >>>>>
> > >> >>>>> By my take: HBASE-18770 and HBASE-19092 are the only issues that
> > >> remain
> > >> >>>>> needing some more work. The rest are just awaiting a good QA
> run.
> > >> >>>>>
> > >> >>>>> Unless I hear otherwise, I'll try to keep an eye on things over
> > the
> > >> >>>>> weekend, bump them along as necessary, and get them committed.
> > >> Would be
> > >> >>>>> great to be able get a vote up on Monday.
> > >> >>>>>
> > >> >>>>>
> > >> >>>>> On 10/24/17 6:03 PM, Stack wrote:
> > >> >>>>>
> > >> >>>>> Chatting with my coworker Mr. Mike Drob, we were batting back
> and
> > >> forth
> > >> >>>>>
> > >> >>>>>> what remains to be done. Surfacing our thoughts here so you all
> > >> clued
> > >> >>>>>> in....Or if you think otherwise, please speak up.
> > >> >>>>>>
> > >> >>>>>> We have ~13 issues to land:
> > >> >>>>>> https://issues.apache.org/jira/projects/HBASE/versions/
> 12341594
> > >> About
> > >> >>>>>> two
> > >> >>>>>> are meta-issues that are about process which leaves 11.
> > >> >>>>>>
> > >> >>>>>> Duo and Zheng Hu are to merge the FilterList fixes improvements
> > >> >>>>>> (HBASE-19057, HBASE-18410 et al.). These are blocker because
> some
> > >> >>>>>> changed
> > >> >>>>>> API/semantic that we need to get out earlier rather than later.
> > >> >>>>>>
> > >> >>>>>> Once the above is merged, HBASE-13346, change of Filter method
> > >> names
> > >> >>>>>> to
> > >> >>>>>> mention Cell instead of KeyValue can land.
> > >> >>>>>>
> > >> >>>>>> HBASE-199048 needs a review (Anoop will probably do it),
> removing
> > >> >>>>>> IA.Private objects as params to MasterObserver... Hopefully
> this
> > >> goes
> > >> >>>>>> in
> > >> >>>>>> soon.
> > >> >>>>>>
> > >> >>>>>> Duo is hard at work on trackers for flush and compaction for
> CPs
> > >> >>>>>> (HBASE-18905). How is HBASE-19033 looking Duo (facility for
> > >> Tephra)?
> > >> >>>>>>
> > >> >>>>>> I think HBASE-18906 (Phoenix Region#waitFor...) will evaporate
> > >> after
> > >> >>>>>> Duo
> > >> >>>>>> is
> > >> >>>>>> done w/ his work above.
> > >> >>>>>>
> > >> >>>>>> I'm on HBASE-18770 bypass and HBASE-19077 restore some parity
> > after
> > >> >>>>>> all
> > >> >>>>>> the
> > >> >>>>>> purges allowing CPs do direct calls against Regions in same
> Host.
> > >> >>>>>>
> > >> >>>>>> Anoop is on HBASE-19047 (Fixes) and Ram on cleanup of CellUtil.
> > >> >>>>>>
> > >> >>>>>> Another day or two?
> > >> >>>>>>
> > >> >>>>>> St.Ack
> > >> >>>>>>
> > >> >>>>>>
> > >> >>>>>>
> > >> >>>>>> On Mon, Oct 23, 2017 at 2:52 PM, Stack <st...@duboce.net>
> wrote:
> > >> >>>>>>
> > >> >>>>>>
> > >> >>>>>> On Mon, Oct 23, 2017 at 11:59 AM, Josh Elser <
> elserj@apache.org>
> > >> >>>>>> wrote:
> > >> >>>>>>
> > >> >>>>>>>
> > >> >>>>>>> +1
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>> I was trying to work on helping out on the outstanding
> alpha-4
> > >> stuff
> > >> >>>>>>>> last
> > >> >>>>>>>> week -- will be continuing to try to do the same this week.
> > >> >>>>>>>>
> > >> >>>>>>>> If you need any help, Stack, or if others need reviews where
> I
> > >> >>>>>>>> haven't
> > >> >>>>>>>> noticed on my own: feel free to @mention me.
> > >> >>>>>>>>
> > >> >>>>>>>>
> > >> >>>>>>>> Thanks for the offer Josh. All items seem assigned and are
> > being
> > >> >>>>>>>>
> > >> >>>>>>>> actively
> > >> >>>>>>> worked on. If you get a moment, reviews by you (or anyone
> else)
> > >> helps
> > >> >>>>>>> move
> > >> >>>>>>> the process along.
> > >> >>>>>>>
> > >> >>>>>>> We need to merge in HBASE-18410 branch to pick up Filter
> > >> >>>>>>> improvements.
> > >> >>>>>>> Then HBASE-13346 can go in.
> > >> >>>>>>>
> > >> >>>>>>> You are already helping out on HBASE-18906, thanks. Looks like
> > >> that
> > >> >>>>>>> will
> > >> >>>>>>> be addressed by other alpha-4s about to land.
> > >> >>>>>>>
> > >> >>>>>>> St.Ack
> > >> >>>>>>> TODOs: https://issues.apache.org/jira
> > >> /projects/HBASE/versions/12341
> > >> >>>>>>> 594
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> On 10/23/17 12:53 PM, Stack wrote:
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>> (Reviving this thread)
> > >> >>>>>>>>
> > >> >>>>>>>>
> > >> >>>>>>>>> Lets push out alpha-4 this week. Alpha-4 is the release that
> > has
> > >> >>>>>>>>> the
> > >> >>>>>>>>> refactor of the Coprocessor API shutting down access to
> > >> internals
> > >> >>>>>>>>> marked
> > >> >>>>>>>>> InterfaceAudience.Private.
> > >> >>>>>>>>>
> > >> >>>>>>>>> The outstanding list is here:
> > >> >>>>>>>>> https://issues.apache.org/jira/projects/HBASE/versions/
> > 12341594
> > >> >>>>>>>>>
> > >> >>>>>>>>> Please push in anything marked alpha-4 that belongs to you.
> > >> >>>>>>>>>
> > >> >>>>>>>>> If issue, talk out loud on this thread. If you need a review
> > to
> > >> >>>>>>>>> land
> > >> >>>>>>>>> an
> > >> >>>>>>>>> item, shout on the issue and here; we'll help you out.
> > >> >>>>>>>>>
> > >> >>>>>>>>> As is, items are coming along nicely I'd say. We need to
> merge
> > >> the
> > >> >>>>>>>>> filter
> > >> >>>>>>>>> branch -- HBASE-18410 -- so APIs are finished for hbase2.
> > >> >>>>>>>>>
> > >> >>>>>>>>> Post alpha-4, we'll have to hunt down our downstreamers and
> > help
> > >> >>>>>>>>> them
> > >> >>>>>>>>> test
> > >> >>>>>>>>> on top of alpha-4 so rolling into beta-1, we have confidence
> > our
> > >> >>>>>>>>> downstreamers know what to expect (or we discover what we
> > missed
> > >> >>>>>>>>> BEFORE
> > >> >>>>>>>>> we
> > >> >>>>>>>>> beta-1).
> > >> >>>>>>>>>
> > >> >>>>>>>>> Thanks for time,
> > >> >>>>>>>>> S
> > >> >>>>>>>>>
> > >> >>>>>>>>>
> > >> >>>>>>>>>
> > >> >>>>>>>>>
> > >> >>>>>>>>>
> > >> >>>>>>>>> On Fri, Sep 8, 2017 at 2:04 PM, Stack <st...@duboce.net>
> > wrote:
> > >> >>>>>>>>>
> > >> >>>>>>>>> I'll put up an alpha3 RC Monday, probably Monday night. That
> > >> should
> > >> >>>>>>>>> be
> > >> >>>>>>>>>
> > >> >>>>>>>>> time, if we all sprint, for the public-facing API fixes to
> be
> > >> done.
> > >> >>>>>>>>>
> > >> >>>>>>>>>>
> > >> >>>>>>>>>> I had a bunch of Coprocessor refactor and fixup scheduled
> for
> > >> >>>>>>>>>> alpha3
> > >> >>>>>>>>>> but
> > >> >>>>>>>>>> it is plain that more time is needed (in spite of valiant
> > >> effort
> > >> >>>>>>>>>> so
> > >> >>>>>>>>>> far
> > >> >>>>>>>>>> by
> > >> >>>>>>>>>> Anoop, Duo, Appy, etc.). Therefore, lets run a
> 2.0.0-alpha-4
> > >> whose
> > >> >>>>>>>>>> theme is
> > >> >>>>>>>>>> "Coprocessor Fixup". Hopefully we can put an alpha-4 up by
> > the
> > >> >>>>>>>>>> following
> > >> >>>>>>>>>> week.
> > >> >>>>>>>>>>
> > >> >>>>>>>>>> We should then be ready for beta (beta == no new features,
> no
> > >> API
> > >> >>>>>>>>>> changes,
> > >> >>>>>>>>>> just fixes).
> > >> >>>>>>>>>>
> > >> >>>>>>>>>> Thanks,
> > >> >>>>>>>>>> St.Ack
> > >> >>>>>>>>>>
> > >> >>>>>>>>>>
> > >> >>>>>>>>>> On Thu, Aug 17, 2017 at 12:35 PM, Stack <st...@duboce.net>
> > >> wrote:
> > >> >>>>>>>>>>
> > >> >>>>>>>>>> I put up the hbase-2.0.0-alpha2 release candidate. Please
> > vote
> > >> on
> > >> >>>>>>>>>> it.
> > >> >>>>>>>>>>
> > >> >>>>>>>>>>
> > >> >>>>>>>>>> For hbase-2.0.0-alpha3, the theme is solidifying API. I
> hope
> > to
> > >> >>>>>>>>>>> get a
> > >> >>>>>>>>>>> release out in the next week or so.
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> I did a weeding of 2.0.0 issues over the last day. If
> folks
> > >> are
> > >> >>>>>>>>>>> interested in helping out, below are the items I think we
> > need
> > >> >>>>>>>>>>> done
> > >> >>>>>>>>>>> for
> > >> >>>>>>>>>>> alpha3 (below are at least 'Critical' status, are API
> > possibly
> > >> >>>>>>>>>>> altering
> > >> >>>>>>>>>>> items, and are absent those JIRAs that are making active
> > >> >>>>>>>>>>> progress,
> > >> >>>>>>>>>>> i.e. the
> > >> >>>>>>>>>>> HTD/HCD revamp by Chia-Ping Tsai). A project NOT listed
> that
> > >> >>>>>>>>>>> needs
> > >> >>>>>>>>>>> doing is
> > >> >>>>>>>>>>> what Andrew did comparing 1.3. and 1.4 APIs
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> * HBASE-18622 Mitigate compatibility concerns between
> > branch-1
> > >> >>>>>>>>>>> and
> > >> >>>>>>>>>>> branch-2
> > >> >>>>>>>>>>> This is to do what Andrew did between 1.3 and 1.4 branches
> > >> only
> > >> >>>>>>>>>>> do
> > >> >>>>>>>>>>> it
> > >> >>>>>>>>>>> between branch-1 and branch-2.
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> * HBASE-10462 Recategorize some of the client facing
> Public
> > /
> > >> >>>>>>>>>>> Private
> > >> >>>>>>>>>>> interfaces
> > >> >>>>>>>>>>> This one is almost done. It could do with a finish,
> > attention
> > >> to
> > >> >>>>>>>>>>> the
> > >> >>>>>>>>>>> items in last comment, and then our codebase could do with
> > >> >>>>>>>>>>> another
> > >> >>>>>>>>>>> sweep
> > >> >>>>>>>>>>> after the spirit of this issue since a bunch has gone in
> > since
> > >> >>>>>>>>>>> the
> > >> >>>>>>>>>>> pass
> > >> >>>>>>>>>>> that was the basis of this issue.
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> * HBASE-10504 Define Replication Interface
> > >> >>>>>>>>>>> I was going to take a crack at this as part of the revamp
> > >> forced
> > >> >>>>>>>>>>> by
> > >> >>>>>>>>>>> 'HBASE-15982 Interface ReplicationEndpoint extends Guava's
> > >> >>>>>>>>>>> Service'
> > >> >>>>>>>>>>> but if
> > >> >>>>>>>>>>> anyone else is interested, be my guest.
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> * HBASE-14996 Some more API cleanup for 2.0
> > >> >>>>>>>>>>> Has a bunch of subtasks, some of which are being worked
> on.
> > >> Needs
> > >> >>>>>>>>>>> finishing.
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> * HBASE-14998 Unify synchronous and asynchronous methods
> in
> > >> Admin
> > >> >>>>>>>>>>> and
> > >> >>>>>>>>>>> cleanup
> > >> >>>>>>>>>>> Needs a pass. Small issue I think. Could also look at new
> > >> >>>>>>>>>>> AsyncClient
> > >> >>>>>>>>>>> and
> > >> >>>>>>>>>>> make sure symmetry.
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> * HBASE-15607 Remove PB references from Admin for 2.0
> > >> >>>>>>>>>>> Predicated on result of an ongoing DISCUSSION thread but
> > >> needs to
> > >> >>>>>>>>>>> be
> > >> >>>>>>>>>>> done.
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> Rolling upgrade will have implications for our API. Would
> be
> > >> good
> > >> >>>>>>>>>>> to
> > >> >>>>>>>>>>> try
> > >> >>>>>>>>>>> it and figure what needs fixup (as said above, according
> to
> > >> trial
> > >> >>>>>>>>>>> by
> > >> >>>>>>>>>>> Sean,
> > >> >>>>>>>>>>> we might not be too bad here):
> > >> >>>>>>>>>>> * HBASE-16060 1.x clients cannot access table state
> talking
> > to
> > >> >>>>>>>>>>> 2.0
> > >> >>>>>>>>>>> cluster
> > >> >>>>>>>>>>> * HBASE-16550 Procedure v2 - Add AM compatibility for 2.x
> > >> Master
> > >> >>>>>>>>>>> and
> > >> >>>>>>>>>>> 1.x
> > >> >>>>>>>>>>> RSs; i.e. support Rolling Upgrade from hbase-1 to -2.
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> * HBASE-17442 Move most of the replication related classes
> > to
> > >> >>>>>>>>>>> hbase-server package
> > >> >>>>>>>>>>> The above would be good to do generally but it may make
> for
> > >> >>>>>>>>>>> ripples
> > >> >>>>>>>>>>> in
> > >> >>>>>>>>>>> API so would be good to do now.
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> * HBASE-18106 Redo ProcedureInfo and LockInfo
> > >> >>>>>>>>>>> Balazs is working on this. The idea is that we avoid
> adding
> > >> two
> > >> >>>>>>>>>>> new
> > >> >>>>>>>>>>> types
> > >> >>>>>>>>>>> to our API, two types that are nought but curtailed,
> > read-only
> > >> >>>>>>>>>>> views
> > >> >>>>>>>>>>> on
> > >> >>>>>>>>>>> internals. Input if you have time appreciated.
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> * HBASE-18596 A hbase1 cluster should be able to replicate
> > to
> > >> a
> > >> >>>>>>>>>>> hbase2
> > >> >>>>>>>>>>> cluster; verify
> > >> >>>>>>>>>>> Esteban is looking at this one
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> * HBASE-9417 SecureBulkLoadEndpoint should be folded in
> core
> > >> >>>>>>>>>>> * HBASE-17143 Scan improvement
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> Our Coprocessor Interface needs a tough edit. It exposes
> > >> >>>>>>>>>>> implementations
> > >> >>>>>>>>>>> marked audience Private and returns implementations rather
> > >> than
> > >> >>>>>>>>>>> Interfaces.
> > >> >>>>>>>>>>> In a few locations, we allow returning an alternate
> > >> >>>>>>>>>>> implementation
> > >> >>>>>>>>>>> altogether which is probably something we don't want a CP
> > >> doing.
> > >> >>>>>>>>>>> To
> > >> >>>>>>>>>>> that
> > >> >>>>>>>>>>> end, the following issues started by Duo and Anoop need to
> > be
> > >> >>>>>>>>>>> taken
> > >> >>>>>>>>>>> to
> > >> >>>>>>>>>>> the
> > >> >>>>>>>>>>> finish line; ideally they'd have an owner:
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> * HBASE-18169 Coprocessor fix and cleanup before 2.0.0
> > >> release <=
> > >> >>>>>>>>>>> The
> > >> >>>>>>>>>>> umbrella issue.
> > >> >>>>>>>>>>> * HBASE-18298 RegionServerServices Interface cleanup for
> CP
> > >> >>>>>>>>>>> expose
> > >> >>>>>>>>>>> * HBASE-16769 Deprecate/remove PB references from
> > >> MasterObserver
> > >> >>>>>>>>>>> and
> > >> >>>>>>>>>>> RegionServerObserver
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> Nice-to-haves:
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> * HBASE-15284 Make TimeRange constructors IA.Private and
> > >> remove
> > >> >>>>>>>>>>> unused
> > >> >>>>>>>>>>> TimeRange constructors
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> * HBASE-10944 Remove all kv.getBuffer() and kv.getRow()
> > >> >>>>>>>>>>> references
> > >> >>>>>>>>>>> existing in the code
> > >> >>>>>>>>>>> This is the end of an old long-running project moving up
> on
> > to
> > >> >>>>>>>>>>> Cell
> > >> >>>>>>>>>>> Interface. We think it is done but for a few little items
> > >> >>>>>>>>>>> (deprecate
> > >> >>>>>>>>>>> KV
> > >> >>>>>>>>>>> methods in MR and provide Cell versions instead...)
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> * HBASE-13271 Table#puts(List<Put>) operation is
> > >> indeterminate;
> > >> >>>>>>>>>>> needs
> > >> >>>>>>>>>>> fixing
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> * HBASE-13346 Clean up Filter package for post 1.0
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> * HBASE-14255 Simplify Cell creation post 1.0
> > >> >>>>>>>>>>> * HBASE-14997
> > >> >>>>>>>>>>> Move compareOp and Comparators out of filter to client
> > package
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> * HBASE-13740 Stop using Hadoop private interfaces
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> What about:
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> * HBASE-18601 Remove Htrace 3.2
> > >> >>>>>>>>>>> As has been noted, the HTrace API is our 'trace' API.
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> If interested in any of the above and you need a legup,
> just
> > >> ask
> > >> >>>>>>>>>>> in
> > >> >>>>>>>>>>> the
> > >> >>>>>>>>>>> issue and I'll be by....
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> Thanks,
> > >> >>>>>>>>>>> St.Ack
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> On Mon, Aug 14, 2017 at 10:54 AM, Stack <stack@duboce.net
> >
> > >> >>>>>>>>>>> wrote:
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> Heads-up:
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> I'm about to put up an hbase-2.0.0-alpha2 Release
> Candidate.
> > >> >>>>>>>>>>>> Theme
> > >> >>>>>>>>>>>> is
> > >> >>>>>>>>>>>> updated dependencies, reliance on relocated popular libs
> > >> (guava,
> > >> >>>>>>>>>>>> netty,
> > >> >>>>>>>>>>>> protobuf), purge of checked-in generated src, and
> > >> >>>>>>>>>>>> master-carries-no-regions
> > >> >>>>>>>>>>>> by default.
> > >> >>>>>>>>>>>>
> > >> >>>>>>>>>>>> alpha3 I hope will follow soon after (end-of-August?).
> Its
> > >> theme
> > >> >>>>>>>>>>>> will
> > >> >>>>>>>>>>>> be
> > >> >>>>>>>>>>>> settling the APIs and compatibility (At first blush, we
> are
> > >> not
> > >> >>>>>>>>>>>> looking too
> > >> >>>>>>>>>>>> bad; our Sean ran some tests over weekend that have
> hbase-1
> > >> >>>>>>>>>>>> client
> > >> >>>>>>>>>>>> running
> > >> >>>>>>>>>>>> against an hbase-2 cluster....). The Coprocessor
> Interface
> > >> >>>>>>>>>>>> revamp
> > >> >>>>>>>>>>>> should be
> > >> >>>>>>>>>>>> done by alpha3 (i.e. returning Interfaces rather than
> > >> >>>>>>>>>>>> Implementations, and
> > >> >>>>>>>>>>>> our shutdown of CPs accessing classes in hbase marked
> > >> >>>>>>>>>>>> InterfaceAudience).
> > >> >>>>>>>>>>>> We'll also have purged thirdparty classes from our API;
> > e.g.
> > >> >>>>>>>>>>>> guava
> > >> >>>>>>>>>>>> 0.12
> > >> >>>>>>>>>>>> Service showing through in our replication API and
> > protobufs
> > >> in
> > >> >>>>>>>>>>>> Admin
> > >> >>>>>>>>>>>> Interface. On alpha3, we will have to do a bunch of
> > outreach
> > >> to
> > >> >>>>>>>>>>>> make
> > >> >>>>>>>>>>>> sure
> > >> >>>>>>>>>>>> our downstreamers are up on what is coming down the pipe.
> > >> >>>>>>>>>>>>
> > >> >>>>>>>>>>>> Beta1 in mid-September?
> > >> >>>>>>>>>>>>
> > >> >>>>>>>>>>>> I encourage you to check out the items marked for hbase2:
> > >> >>>>>>>>>>>> https://issues.apache.org/jira/projects/HBASE/versions/
> > 12327
> > >> 188
> > >> >>>>>>>>>>>> Edit
> > >> >>>>>>>>>>>> as
> > >> >>>>>>>>>>>> you see appropriate. Punt if you know the JIRA will not
> get
> > >> any
> > >> >>>>>>>>>>>> attention
> > >> >>>>>>>>>>>> in next month or so.
> > >> >>>>>>>>>>>>
> > >> >>>>>>>>>>>> A bunch of issues marked blocker are unassigned. I'll
> leave
> > >> them
> > >> >>>>>>>>>>>> as
> > >> >>>>>>>>>>>> is
> > >> >>>>>>>>>>>> another while but I'll boot them soon.
> > >> >>>>>>>>>>>>
> > >> >>>>>>>>>>>> While I have your attention:
> > >> >>>>>>>>>>>>
> > >> >>>>>>>>>>>> + I think we should leave thrift version at 0.9.3. Moving
> > >> hbase
> > >> >>>>>>>>>>>> thrift
> > >> >>>>>>>>>>>> to 0.10.0 will break existing clients. The change is easy
> > >> enough
> > >> >>>>>>>>>>>> if
> > >> >>>>>>>>>>>> folks
> > >> >>>>>>>>>>>> need to upgrade their hbase thrift. See HBASE-18591.
> > >> >>>>>>>>>>>> + Upgrade from 0.94 is disallowed. You have to get to 1.0
> > >> first
> > >> >>>>>>>>>>>> (0.98?).
> > >> >>>>>>>>>>>>
> > >> >>>>>>>>>>>> St.Ack
> > >> >>>>>>>>>>>>
> > >> >>>>>>>>>>>>
> > >> >>>>>>>>>>>>
> > >> >>>>>>>>>>>> On Wed, Aug 2, 2017 at 9:43 AM, Stack <st...@duboce.net>
> > >> wrote:
> > >> >>>>>>>>>>>>
> > >> >>>>>>>>>>>>
> > >> >>>>>>>>>>>>
> > >> >>>>>>>>>>>> On Tue, Aug 1, 2017 at 2:06 PM, Josh Elser <
> > >> elserj@apache.org>
> > >> >>>>>>>>>>>>
> > >> >>>>>>>>>>>>> wrote:
> > >> >>>>>>>>>>>>>
> > >> >>>>>>>>>>>>>
> > >> >>>>>>>>>>>>>
> > >> >>>>>>>>>>>>> On 7/31/17 9:00 AM, Stack wrote:
> > >> >>>>>>>>>>>>>
> > >> >>>>>>>>>>>>>>
> > >> >>>>>>>>>>>>>> On Mon, Jul 24, 2017 at 12:25 PM, Josh Elser<
> > >> >>>>>>>>>>>>>> elserj@apache.org>
> > >> >>>>>>>>>>>>>>
> > >> >>>>>>>>>>>>>> wrote:
> > >> >>>>>>>>>>>>>>
> > >> >>>>>>>>>>>>>>>
> > >> >>>>>>>>>>>>>>> ...
> > >> >>>>>>>>>>>>>>>
> > >> >>>>>>>>>>>>>>>
> > >> >>>>>>>>>>>>>>> I like the idea of this also hitting 2.0 as it would
> > make
> > >> the
> > >> >>>>>>>>>>>>>>>
> > >> >>>>>>>>>>>>>>>> feature a
> > >> >>>>>>>>>>>>>>>> bit more "real", but am obviously a little nervous (I
> > >> have
> > >> >>>>>>>>>>>>>>>> no
> > >> >>>>>>>>>>>>>>>> reason
> > >> >>>>>>>>>>>>>>>> to be
> > >> >>>>>>>>>>>>>>>> nervous though). I am pretty happy with the feature
> in
> > >> terms
> > >> >>>>>>>>>>>>>>>> of
> > >> >>>>>>>>>>>>>>>> how
> > >> >>>>>>>>>>>>>>>> much it
> > >> >>>>>>>>>>>>>>>> is covered via testing.
> > >> >>>>>>>>>>>>>>>>
> > >> >>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/HBASE-17748
> > >> >>>>>>>>>>>>>>>>
> > >> >>>>>>>>>>>>>>>>
> > >> >>>>>>>>>>>>>>>>
> > >> >>>>>>>>>>>>>>>> Sounds good to me. Whats involved? Backport? If so,
> +1
> > >> Josh.
> > >> >>>>>>>>>>>>>>>>
> > >> >>>>>>>>>>>>>>>>
> > >> >>>>>>>>>>>>>>>> Last think on space quota says that need doc too. See
> > >> 'Space
> > >> >>>>>>>>>>>>>>>>
> > >> >>>>>>>>>>>>>>> Quota' in
> > >> >>>>>>>>>>>>>>> here:
> > >> >>>>>>>>>>>>>>> https://docs.google.com/docume
> > >> nt/d/1WCsVlnHjJeKUcl7wHwqb4z9i
> > >> >>>>>>>>>>>>>>> Eu_ktczrlKHK8N4SZzs/edit#heading=h.wuw3a6jukzo5
> > >> >>>>>>>>>>>>>>> Does this little section need an update Josh?
> > >> >>>>>>>>>>>>>>>
> > >> >>>>>>>>>>>>>>> Thanks,
> > >> >>>>>>>>>>>>>>> S
> > >> >>>>>>>>>>>>>>>
> > >> >>>>>>>>>>>>>>>
> > >> >>>>>>>>>>>>>>> Yep, just a couple of cherry-picks. Good test coverage
> > and
> > >> >>>>>>>>>>>>>>> some
> > >> >>>>>>>>>>>>>>>
> > >> >>>>>>>>>>>>>>> docs
> > >> >>>>>>>>>>>>>> already included for 17748.  Happy to put that on my
> > plate
> > >> if
> > >> >>>>>>>>>>>>>> you're good
> > >> >>>>>>>>>>>>>> with it. I can reasonably assume that no one is against
> > it
> > >> :)
> > >> >>>>>>>>>>>>>>
> > >> >>>>>>>>>>>>>> I think I had knocked out docs for the "phase 1" stuff
> > >> before
> > >> >>>>>>>>>>>>>> we
> > >> >>>>>>>>>>>>>> merged it in from the original feature branch. I'll
> > double
> > >> >>>>>>>>>>>>>> check
> > >> >>>>>>>>>>>>>> and update
> > >> >>>>>>>>>>>>>> the gdoc. Perhaps this was just a timing thing.
> > >> >>>>>>>>>>>>>>
> > >> >>>>>>>>>>>>>>
> > >> >>>>>>>>>>>>>> Thanks Josh,
> > >> >>>>>>>>>>>>>>
> > >> >>>>>>>>>>>>>> S
> > >> >>>>>>>>>>>>>
> > >> >>>>>>>>>>>>>
> > >> >>>>>>>>>>>>>
> > >> >>>>>>>>>>>>>
> > >> >>>>>>>>>>>>>
> > >> >>>>>>>>>>>>
> > >> >>>>>>>>>>>>
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>
> > >> >>>>>>>>>
> > >> >>>>>>>
> > >> >>>>>>
> > >> >>>>
> > >> >>
> > >>
> > >
> > >
> >
>

Re: Moving 2.0 forward

[ramkrishna vasudevan <ra...@gmail.com>]

HBASE-18946 Stochastic load balancer assigns replica regions to the same RS

Am on this today and probably will put up another patch.

Regards
Ram

On Fri, Nov 17, 2017 at 5:18 AM, Stack <st...@duboce.net> wrote:

> hbase-2.0.0-beta-1 update (Reminder, beta-1 is where we finish last
> remaining features and apply final polish to API. There will be a beta-2
> but it is about upgrade/rolling-upgrade and bug fixes ONLY).
>
> Myself and Mr. Drob did a pass over the outstanding hbase-2.0.0-beta-1 list
> this morning. See here [1].
>
> There are about ~12 issues in progress with most of these about to land.
> There are 37 TODO. Many of these are tests we need to run, some are related
> to the backup/restore, but a good few are meaty w/o assignees.
>
> The awkward outstanding ones as I see it are the below:
>
> HBASE-18946 Stochastic load balancer assigns replica regions to the same RS
> HBASE-17204 Make L2 off heap cache default ON
> HBASE-19112 Suspect methods on Cell to be deprecated
> HBASE-19147 All branch-2 unit tests pass
>
> We need to make progress on the above or punt on them (can't punt on the
> last one though).
>
> Any ideas on what configs we should update in hbase2? Dump ideas into:
> HBASE-19148 Edit of default configuration
>
> Still hoping for an early December beta-1 RC. beta-2 hopefully will be
> close behind.
>
> Comments? Thoughts?
>
> Thanks all,
> S
>
> 1. https://issues.apache.org/jira/projects/HBASE/versions/12340861
>
> On Tue, Oct 31, 2017 at 2:56 PM, Stack <st...@duboce.net> wrote:
>
> >
> >
> > On Tue, Oct 31, 2017 at 2:31 PM, Mike Drob <md...@apache.org> wrote:
> >
> >> Hoping to keep momentum going from our Stack working on alpha4, I tried
> to
> >> take a stab at triaging some of the open beta-1 issues.
> >>
> >> I moved some docs stuff out form beta-1 to 2.0-GA, if it gets done
> sooner
> >> then I'm happy to see it pulled back in. Trying to balance optimism with
> >> realism here, and knowing that documentation unfortunately often gets
> >> pushed to the back-burner.
> >>
> >> Also, I poked some folks on unassigned issues that they've filed for
> >> beta-1, especially in the last few days. If issues don't have an owner
> >> they
> >> are unlikely to get worked. I chatted with stack and he agreed to take
> on
> >> some of the tasks, but there's a lot of surface area to cover.
> >>
> >> If you you're working on issues that are critical for beta-1, please
> mark
> >> them as such. Then the rest of the community will know to help
> prioritize
> >> feedback and reviews there.
> >>
> >> Do we have a general theme for the betas like we did with the alphas?
> >>
> >> Beta1 is upgrades work from branch1, beta2 is rolling upgrades work as
> >> well? Continue to work on tests throughout?
> >>
> >>
> > Thanks Mike for helping to kick off the beta-1 train.
> >
> > Regards a theme for beta-1, I'd like it to be 'finish'; beta-1 is what we
> > are going to ship (beta-2 is rolling upgrade and any minor items turned
> up
> > in testing/burn-in).
> >
> > Thanks,
> > S
> >
> >
> >
> >> Mike
> >>
> >> On Tue, Oct 31, 2017 at 10:04 AM, Josh Elser <el...@apache.org> wrote:
> >>
> >> > +1 go from my POV.
> >> >
> >> >
> >> > On 10/31/17 10:07 AM, Stack wrote:
> >> >
> >> >> I want to push an alpha-4 today. A few items didn't make it
> >> (HBASE-19092).
> >> >> They need more time. We'll pull them in for beta-1. CP API is
> basically
> >> >> done. There may be some changes for beta-1 but hopefully only changes
> >> >> informed by experience trying to port an existing Coprocessor to
> >> hbase2.
> >> >>
> >> >> Shout if there is anything that needs to make alpha-4.
> >> >>
> >> >> Thanks,
> >> >> St.Ack
> >> >>
> >> >>
> >> >> On Sat, Oct 28, 2017 at 2:48 PM, Josh Elser <jo...@gmail.com>
> >> wrote:
> >> >>
> >> >> Yup, that was going to be my plan, Mike!
> >> >>>
> >> >>> Making a pass now, and will check back later tonight again. I see
> >> others
> >> >>> have already done some work today on this front.
> >> >>>
> >> >>>
> >> >>> On 10/27/17 11:38 PM, Mike Drob wrote:
> >> >>>
> >> >>> Josh - Do you want to kick off a bunch of QA runs? (Do you know how
> >> to do
> >> >>>> it directly on the jenkins job, so you don't have to bother with
> JIRA
> >> >>>> uploads)
> >> >>>>
> >> >>>> If you're busy, then I can make time tomorrow or Sunday to kick off
> >> >>>> jobs,
> >> >>>> but I want to make sure we're not duplicating effort and jenkins
> >> cycles.
> >> >>>>
> >> >>>> On Fri, Oct 27, 2017 at 7:10 PM, Josh Elser <el...@apache.org>
> >> wrote:
> >> >>>>
> >> >>>> My turn to bump ;)
> >> >>>>
> >> >>>>>
> >> >>>>> By my take: HBASE-18770 and HBASE-19092 are the only issues that
> >> remain
> >> >>>>> needing some more work. The rest are just awaiting a good QA run.
> >> >>>>>
> >> >>>>> Unless I hear otherwise, I'll try to keep an eye on things over
> the
> >> >>>>> weekend, bump them along as necessary, and get them committed.
> >> Would be
> >> >>>>> great to be able get a vote up on Monday.
> >> >>>>>
> >> >>>>>
> >> >>>>> On 10/24/17 6:03 PM, Stack wrote:
> >> >>>>>
> >> >>>>> Chatting with my coworker Mr. Mike Drob, we were batting back and
> >> forth
> >> >>>>>
> >> >>>>>> what remains to be done. Surfacing our thoughts here so you all
> >> clued
> >> >>>>>> in....Or if you think otherwise, please speak up.
> >> >>>>>>
> >> >>>>>> We have ~13 issues to land:
> >> >>>>>> https://issues.apache.org/jira/projects/HBASE/versions/12341594
> >> About
> >> >>>>>> two
> >> >>>>>> are meta-issues that are about process which leaves 11.
> >> >>>>>>
> >> >>>>>> Duo and Zheng Hu are to merge the FilterList fixes improvements
> >> >>>>>> (HBASE-19057, HBASE-18410 et al.). These are blocker because some
> >> >>>>>> changed
> >> >>>>>> API/semantic that we need to get out earlier rather than later.
> >> >>>>>>
> >> >>>>>> Once the above is merged, HBASE-13346, change of Filter method
> >> names
> >> >>>>>> to
> >> >>>>>> mention Cell instead of KeyValue can land.
> >> >>>>>>
> >> >>>>>> HBASE-199048 needs a review (Anoop will probably do it), removing
> >> >>>>>> IA.Private objects as params to MasterObserver... Hopefully this
> >> goes
> >> >>>>>> in
> >> >>>>>> soon.
> >> >>>>>>
> >> >>>>>> Duo is hard at work on trackers for flush and compaction for CPs
> >> >>>>>> (HBASE-18905). How is HBASE-19033 looking Duo (facility for
> >> Tephra)?
> >> >>>>>>
> >> >>>>>> I think HBASE-18906 (Phoenix Region#waitFor...) will evaporate
> >> after
> >> >>>>>> Duo
> >> >>>>>> is
> >> >>>>>> done w/ his work above.
> >> >>>>>>
> >> >>>>>> I'm on HBASE-18770 bypass and HBASE-19077 restore some parity
> after
> >> >>>>>> all
> >> >>>>>> the
> >> >>>>>> purges allowing CPs do direct calls against Regions in same Host.
> >> >>>>>>
> >> >>>>>> Anoop is on HBASE-19047 (Fixes) and Ram on cleanup of CellUtil.
> >> >>>>>>
> >> >>>>>> Another day or two?
> >> >>>>>>
> >> >>>>>> St.Ack
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>>> On Mon, Oct 23, 2017 at 2:52 PM, Stack <st...@duboce.net> wrote:
> >> >>>>>>
> >> >>>>>>
> >> >>>>>> On Mon, Oct 23, 2017 at 11:59 AM, Josh Elser <el...@apache.org>
> >> >>>>>> wrote:
> >> >>>>>>
> >> >>>>>>>
> >> >>>>>>> +1
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>> I was trying to work on helping out on the outstanding alpha-4
> >> stuff
> >> >>>>>>>> last
> >> >>>>>>>> week -- will be continuing to try to do the same this week.
> >> >>>>>>>>
> >> >>>>>>>> If you need any help, Stack, or if others need reviews where I
> >> >>>>>>>> haven't
> >> >>>>>>>> noticed on my own: feel free to @mention me.
> >> >>>>>>>>
> >> >>>>>>>>
> >> >>>>>>>> Thanks for the offer Josh. All items seem assigned and are
> being
> >> >>>>>>>>
> >> >>>>>>>> actively
> >> >>>>>>> worked on. If you get a moment, reviews by you (or anyone else)
> >> helps
> >> >>>>>>> move
> >> >>>>>>> the process along.
> >> >>>>>>>
> >> >>>>>>> We need to merge in HBASE-18410 branch to pick up Filter
> >> >>>>>>> improvements.
> >> >>>>>>> Then HBASE-13346 can go in.
> >> >>>>>>>
> >> >>>>>>> You are already helping out on HBASE-18906, thanks. Looks like
> >> that
> >> >>>>>>> will
> >> >>>>>>> be addressed by other alpha-4s about to land.
> >> >>>>>>>
> >> >>>>>>> St.Ack
> >> >>>>>>> TODOs: https://issues.apache.org/jira
> >> /projects/HBASE/versions/12341
> >> >>>>>>> 594
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> On 10/23/17 12:53 PM, Stack wrote:
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>> (Reviving this thread)
> >> >>>>>>>>
> >> >>>>>>>>
> >> >>>>>>>>> Lets push out alpha-4 this week. Alpha-4 is the release that
> has
> >> >>>>>>>>> the
> >> >>>>>>>>> refactor of the Coprocessor API shutting down access to
> >> internals
> >> >>>>>>>>> marked
> >> >>>>>>>>> InterfaceAudience.Private.
> >> >>>>>>>>>
> >> >>>>>>>>> The outstanding list is here:
> >> >>>>>>>>> https://issues.apache.org/jira/projects/HBASE/versions/
> 12341594
> >> >>>>>>>>>
> >> >>>>>>>>> Please push in anything marked alpha-4 that belongs to you.
> >> >>>>>>>>>
> >> >>>>>>>>> If issue, talk out loud on this thread. If you need a review
> to
> >> >>>>>>>>> land
> >> >>>>>>>>> an
> >> >>>>>>>>> item, shout on the issue and here; we'll help you out.
> >> >>>>>>>>>
> >> >>>>>>>>> As is, items are coming along nicely I'd say. We need to merge
> >> the
> >> >>>>>>>>> filter
> >> >>>>>>>>> branch -- HBASE-18410 -- so APIs are finished for hbase2.
> >> >>>>>>>>>
> >> >>>>>>>>> Post alpha-4, we'll have to hunt down our downstreamers and
> help
> >> >>>>>>>>> them
> >> >>>>>>>>> test
> >> >>>>>>>>> on top of alpha-4 so rolling into beta-1, we have confidence
> our
> >> >>>>>>>>> downstreamers know what to expect (or we discover what we
> missed
> >> >>>>>>>>> BEFORE
> >> >>>>>>>>> we
> >> >>>>>>>>> beta-1).
> >> >>>>>>>>>
> >> >>>>>>>>> Thanks for time,
> >> >>>>>>>>> S
> >> >>>>>>>>>
> >> >>>>>>>>>
> >> >>>>>>>>>
> >> >>>>>>>>>
> >> >>>>>>>>>
> >> >>>>>>>>> On Fri, Sep 8, 2017 at 2:04 PM, Stack <st...@duboce.net>
> wrote:
> >> >>>>>>>>>
> >> >>>>>>>>> I'll put up an alpha3 RC Monday, probably Monday night. That
> >> should
> >> >>>>>>>>> be
> >> >>>>>>>>>
> >> >>>>>>>>> time, if we all sprint, for the public-facing API fixes to be
> >> done.
> >> >>>>>>>>>
> >> >>>>>>>>>>
> >> >>>>>>>>>> I had a bunch of Coprocessor refactor and fixup scheduled for
> >> >>>>>>>>>> alpha3
> >> >>>>>>>>>> but
> >> >>>>>>>>>> it is plain that more time is needed (in spite of valiant
> >> effort
> >> >>>>>>>>>> so
> >> >>>>>>>>>> far
> >> >>>>>>>>>> by
> >> >>>>>>>>>> Anoop, Duo, Appy, etc.). Therefore, lets run a 2.0.0-alpha-4
> >> whose
> >> >>>>>>>>>> theme is
> >> >>>>>>>>>> "Coprocessor Fixup". Hopefully we can put an alpha-4 up by
> the
> >> >>>>>>>>>> following
> >> >>>>>>>>>> week.
> >> >>>>>>>>>>
> >> >>>>>>>>>> We should then be ready for beta (beta == no new features, no
> >> API
> >> >>>>>>>>>> changes,
> >> >>>>>>>>>> just fixes).
> >> >>>>>>>>>>
> >> >>>>>>>>>> Thanks,
> >> >>>>>>>>>> St.Ack
> >> >>>>>>>>>>
> >> >>>>>>>>>>
> >> >>>>>>>>>> On Thu, Aug 17, 2017 at 12:35 PM, Stack <st...@duboce.net>
> >> wrote:
> >> >>>>>>>>>>
> >> >>>>>>>>>> I put up the hbase-2.0.0-alpha2 release candidate. Please
> vote
> >> on
> >> >>>>>>>>>> it.
> >> >>>>>>>>>>
> >> >>>>>>>>>>
> >> >>>>>>>>>> For hbase-2.0.0-alpha3, the theme is solidifying API. I hope
> to
> >> >>>>>>>>>>> get a
> >> >>>>>>>>>>> release out in the next week or so.
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> I did a weeding of 2.0.0 issues over the last day. If folks
> >> are
> >> >>>>>>>>>>> interested in helping out, below are the items I think we
> need
> >> >>>>>>>>>>> done
> >> >>>>>>>>>>> for
> >> >>>>>>>>>>> alpha3 (below are at least 'Critical' status, are API
> possibly
> >> >>>>>>>>>>> altering
> >> >>>>>>>>>>> items, and are absent those JIRAs that are making active
> >> >>>>>>>>>>> progress,
> >> >>>>>>>>>>> i.e. the
> >> >>>>>>>>>>> HTD/HCD revamp by Chia-Ping Tsai). A project NOT listed that
> >> >>>>>>>>>>> needs
> >> >>>>>>>>>>> doing is
> >> >>>>>>>>>>> what Andrew did comparing 1.3. and 1.4 APIs
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> * HBASE-18622 Mitigate compatibility concerns between
> branch-1
> >> >>>>>>>>>>> and
> >> >>>>>>>>>>> branch-2
> >> >>>>>>>>>>> This is to do what Andrew did between 1.3 and 1.4 branches
> >> only
> >> >>>>>>>>>>> do
> >> >>>>>>>>>>> it
> >> >>>>>>>>>>> between branch-1 and branch-2.
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> * HBASE-10462 Recategorize some of the client facing Public
> /
> >> >>>>>>>>>>> Private
> >> >>>>>>>>>>> interfaces
> >> >>>>>>>>>>> This one is almost done. It could do with a finish,
> attention
> >> to
> >> >>>>>>>>>>> the
> >> >>>>>>>>>>> items in last comment, and then our codebase could do with
> >> >>>>>>>>>>> another
> >> >>>>>>>>>>> sweep
> >> >>>>>>>>>>> after the spirit of this issue since a bunch has gone in
> since
> >> >>>>>>>>>>> the
> >> >>>>>>>>>>> pass
> >> >>>>>>>>>>> that was the basis of this issue.
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> * HBASE-10504 Define Replication Interface
> >> >>>>>>>>>>> I was going to take a crack at this as part of the revamp
> >> forced
> >> >>>>>>>>>>> by
> >> >>>>>>>>>>> 'HBASE-15982 Interface ReplicationEndpoint extends Guava's
> >> >>>>>>>>>>> Service'
> >> >>>>>>>>>>> but if
> >> >>>>>>>>>>> anyone else is interested, be my guest.
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> * HBASE-14996 Some more API cleanup for 2.0
> >> >>>>>>>>>>> Has a bunch of subtasks, some of which are being worked on.
> >> Needs
> >> >>>>>>>>>>> finishing.
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> * HBASE-14998 Unify synchronous and asynchronous methods in
> >> Admin
> >> >>>>>>>>>>> and
> >> >>>>>>>>>>> cleanup
> >> >>>>>>>>>>> Needs a pass. Small issue I think. Could also look at new
> >> >>>>>>>>>>> AsyncClient
> >> >>>>>>>>>>> and
> >> >>>>>>>>>>> make sure symmetry.
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> * HBASE-15607 Remove PB references from Admin for 2.0
> >> >>>>>>>>>>> Predicated on result of an ongoing DISCUSSION thread but
> >> needs to
> >> >>>>>>>>>>> be
> >> >>>>>>>>>>> done.
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> Rolling upgrade will have implications for our API. Would be
> >> good
> >> >>>>>>>>>>> to
> >> >>>>>>>>>>> try
> >> >>>>>>>>>>> it and figure what needs fixup (as said above, according to
> >> trial
> >> >>>>>>>>>>> by
> >> >>>>>>>>>>> Sean,
> >> >>>>>>>>>>> we might not be too bad here):
> >> >>>>>>>>>>> * HBASE-16060 1.x clients cannot access table state talking
> to
> >> >>>>>>>>>>> 2.0
> >> >>>>>>>>>>> cluster
> >> >>>>>>>>>>> * HBASE-16550 Procedure v2 - Add AM compatibility for 2.x
> >> Master
> >> >>>>>>>>>>> and
> >> >>>>>>>>>>> 1.x
> >> >>>>>>>>>>> RSs; i.e. support Rolling Upgrade from hbase-1 to -2.
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> * HBASE-17442 Move most of the replication related classes
> to
> >> >>>>>>>>>>> hbase-server package
> >> >>>>>>>>>>> The above would be good to do generally but it may make for
> >> >>>>>>>>>>> ripples
> >> >>>>>>>>>>> in
> >> >>>>>>>>>>> API so would be good to do now.
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> * HBASE-18106 Redo ProcedureInfo and LockInfo
> >> >>>>>>>>>>> Balazs is working on this. The idea is that we avoid adding
> >> two
> >> >>>>>>>>>>> new
> >> >>>>>>>>>>> types
> >> >>>>>>>>>>> to our API, two types that are nought but curtailed,
> read-only
> >> >>>>>>>>>>> views
> >> >>>>>>>>>>> on
> >> >>>>>>>>>>> internals. Input if you have time appreciated.
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> * HBASE-18596 A hbase1 cluster should be able to replicate
> to
> >> a
> >> >>>>>>>>>>> hbase2
> >> >>>>>>>>>>> cluster; verify
> >> >>>>>>>>>>> Esteban is looking at this one
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> * HBASE-9417 SecureBulkLoadEndpoint should be folded in core
> >> >>>>>>>>>>> * HBASE-17143 Scan improvement
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> Our Coprocessor Interface needs a tough edit. It exposes
> >> >>>>>>>>>>> implementations
> >> >>>>>>>>>>> marked audience Private and returns implementations rather
> >> than
> >> >>>>>>>>>>> Interfaces.
> >> >>>>>>>>>>> In a few locations, we allow returning an alternate
> >> >>>>>>>>>>> implementation
> >> >>>>>>>>>>> altogether which is probably something we don't want a CP
> >> doing.
> >> >>>>>>>>>>> To
> >> >>>>>>>>>>> that
> >> >>>>>>>>>>> end, the following issues started by Duo and Anoop need to
> be
> >> >>>>>>>>>>> taken
> >> >>>>>>>>>>> to
> >> >>>>>>>>>>> the
> >> >>>>>>>>>>> finish line; ideally they'd have an owner:
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> * HBASE-18169 Coprocessor fix and cleanup before 2.0.0
> >> release <=
> >> >>>>>>>>>>> The
> >> >>>>>>>>>>> umbrella issue.
> >> >>>>>>>>>>> * HBASE-18298 RegionServerServices Interface cleanup for CP
> >> >>>>>>>>>>> expose
> >> >>>>>>>>>>> * HBASE-16769 Deprecate/remove PB references from
> >> MasterObserver
> >> >>>>>>>>>>> and
> >> >>>>>>>>>>> RegionServerObserver
> >> >>>>>>>>>>>
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> Nice-to-haves:
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> * HBASE-15284 Make TimeRange constructors IA.Private and
> >> remove
> >> >>>>>>>>>>> unused
> >> >>>>>>>>>>> TimeRange constructors
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> * HBASE-10944 Remove all kv.getBuffer() and kv.getRow()
> >> >>>>>>>>>>> references
> >> >>>>>>>>>>> existing in the code
> >> >>>>>>>>>>> This is the end of an old long-running project moving up on
> to
> >> >>>>>>>>>>> Cell
> >> >>>>>>>>>>> Interface. We think it is done but for a few little items
> >> >>>>>>>>>>> (deprecate
> >> >>>>>>>>>>> KV
> >> >>>>>>>>>>> methods in MR and provide Cell versions instead...)
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> * HBASE-13271 Table#puts(List<Put>) operation is
> >> indeterminate;
> >> >>>>>>>>>>> needs
> >> >>>>>>>>>>> fixing
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> * HBASE-13346 Clean up Filter package for post 1.0
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> * HBASE-14255 Simplify Cell creation post 1.0
> >> >>>>>>>>>>> * HBASE-14997
> >> >>>>>>>>>>> Move compareOp and Comparators out of filter to client
> package
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> * HBASE-13740 Stop using Hadoop private interfaces
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> What about:
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> * HBASE-18601 Remove Htrace 3.2
> >> >>>>>>>>>>> As has been noted, the HTrace API is our 'trace' API.
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> If interested in any of the above and you need a legup, just
> >> ask
> >> >>>>>>>>>>> in
> >> >>>>>>>>>>> the
> >> >>>>>>>>>>> issue and I'll be by....
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> Thanks,
> >> >>>>>>>>>>> St.Ack
> >> >>>>>>>>>>>
> >> >>>>>>>>>>>
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> On Mon, Aug 14, 2017 at 10:54 AM, Stack <st...@duboce.net>
> >> >>>>>>>>>>> wrote:
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> Heads-up:
> >> >>>>>>>>>>>
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> I'm about to put up an hbase-2.0.0-alpha2 Release Candidate.
> >> >>>>>>>>>>>> Theme
> >> >>>>>>>>>>>> is
> >> >>>>>>>>>>>> updated dependencies, reliance on relocated popular libs
> >> (guava,
> >> >>>>>>>>>>>> netty,
> >> >>>>>>>>>>>> protobuf), purge of checked-in generated src, and
> >> >>>>>>>>>>>> master-carries-no-regions
> >> >>>>>>>>>>>> by default.
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>> alpha3 I hope will follow soon after (end-of-August?). Its
> >> theme
> >> >>>>>>>>>>>> will
> >> >>>>>>>>>>>> be
> >> >>>>>>>>>>>> settling the APIs and compatibility (At first blush, we are
> >> not
> >> >>>>>>>>>>>> looking too
> >> >>>>>>>>>>>> bad; our Sean ran some tests over weekend that have hbase-1
> >> >>>>>>>>>>>> client
> >> >>>>>>>>>>>> running
> >> >>>>>>>>>>>> against an hbase-2 cluster....). The Coprocessor Interface
> >> >>>>>>>>>>>> revamp
> >> >>>>>>>>>>>> should be
> >> >>>>>>>>>>>> done by alpha3 (i.e. returning Interfaces rather than
> >> >>>>>>>>>>>> Implementations, and
> >> >>>>>>>>>>>> our shutdown of CPs accessing classes in hbase marked
> >> >>>>>>>>>>>> InterfaceAudience).
> >> >>>>>>>>>>>> We'll also have purged thirdparty classes from our API;
> e.g.
> >> >>>>>>>>>>>> guava
> >> >>>>>>>>>>>> 0.12
> >> >>>>>>>>>>>> Service showing through in our replication API and
> protobufs
> >> in
> >> >>>>>>>>>>>> Admin
> >> >>>>>>>>>>>> Interface. On alpha3, we will have to do a bunch of
> outreach
> >> to
> >> >>>>>>>>>>>> make
> >> >>>>>>>>>>>> sure
> >> >>>>>>>>>>>> our downstreamers are up on what is coming down the pipe.
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>> Beta1 in mid-September?
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>> I encourage you to check out the items marked for hbase2:
> >> >>>>>>>>>>>> https://issues.apache.org/jira/projects/HBASE/versions/
> 12327
> >> 188
> >> >>>>>>>>>>>> Edit
> >> >>>>>>>>>>>> as
> >> >>>>>>>>>>>> you see appropriate. Punt if you know the JIRA will not get
> >> any
> >> >>>>>>>>>>>> attention
> >> >>>>>>>>>>>> in next month or so.
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>> A bunch of issues marked blocker are unassigned. I'll leave
> >> them
> >> >>>>>>>>>>>> as
> >> >>>>>>>>>>>> is
> >> >>>>>>>>>>>> another while but I'll boot them soon.
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>> While I have your attention:
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>> + I think we should leave thrift version at 0.9.3. Moving
> >> hbase
> >> >>>>>>>>>>>> thrift
> >> >>>>>>>>>>>> to 0.10.0 will break existing clients. The change is easy
> >> enough
> >> >>>>>>>>>>>> if
> >> >>>>>>>>>>>> folks
> >> >>>>>>>>>>>> need to upgrade their hbase thrift. See HBASE-18591.
> >> >>>>>>>>>>>> + Upgrade from 0.94 is disallowed. You have to get to 1.0
> >> first
> >> >>>>>>>>>>>> (0.98?).
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>> St.Ack
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>> On Wed, Aug 2, 2017 at 9:43 AM, Stack <st...@duboce.net>
> >> wrote:
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>> On Tue, Aug 1, 2017 at 2:06 PM, Josh Elser <
> >> elserj@apache.org>
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>>> wrote:
> >> >>>>>>>>>>>>>
> >> >>>>>>>>>>>>>
> >> >>>>>>>>>>>>>
> >> >>>>>>>>>>>>> On 7/31/17 9:00 AM, Stack wrote:
> >> >>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>> On Mon, Jul 24, 2017 at 12:25 PM, Josh Elser<
> >> >>>>>>>>>>>>>> elserj@apache.org>
> >> >>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>> wrote:
> >> >>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>> ...
> >> >>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>> I like the idea of this also hitting 2.0 as it would
> make
> >> the
> >> >>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>> feature a
> >> >>>>>>>>>>>>>>>> bit more "real", but am obviously a little nervous (I
> >> have
> >> >>>>>>>>>>>>>>>> no
> >> >>>>>>>>>>>>>>>> reason
> >> >>>>>>>>>>>>>>>> to be
> >> >>>>>>>>>>>>>>>> nervous though). I am pretty happy with the feature in
> >> terms
> >> >>>>>>>>>>>>>>>> of
> >> >>>>>>>>>>>>>>>> how
> >> >>>>>>>>>>>>>>>> much it
> >> >>>>>>>>>>>>>>>> is covered via testing.
> >> >>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/HBASE-17748
> >> >>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>> Sounds good to me. Whats involved? Backport? If so, +1
> >> Josh.
> >> >>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>> Last think on space quota says that need doc too. See
> >> 'Space
> >> >>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>> Quota' in
> >> >>>>>>>>>>>>>>> here:
> >> >>>>>>>>>>>>>>> https://docs.google.com/docume
> >> nt/d/1WCsVlnHjJeKUcl7wHwqb4z9i
> >> >>>>>>>>>>>>>>> Eu_ktczrlKHK8N4SZzs/edit#heading=h.wuw3a6jukzo5
> >> >>>>>>>>>>>>>>> Does this little section need an update Josh?
> >> >>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>> Thanks,
> >> >>>>>>>>>>>>>>> S
> >> >>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>> Yep, just a couple of cherry-picks. Good test coverage
> and
> >> >>>>>>>>>>>>>>> some
> >> >>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>> docs
> >> >>>>>>>>>>>>>> already included for 17748.  Happy to put that on my
> plate
> >> if
> >> >>>>>>>>>>>>>> you're good
> >> >>>>>>>>>>>>>> with it. I can reasonably assume that no one is against
> it
> >> :)
> >> >>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>> I think I had knocked out docs for the "phase 1" stuff
> >> before
> >> >>>>>>>>>>>>>> we
> >> >>>>>>>>>>>>>> merged it in from the original feature branch. I'll
> double
> >> >>>>>>>>>>>>>> check
> >> >>>>>>>>>>>>>> and update
> >> >>>>>>>>>>>>>> the gdoc. Perhaps this was just a timing thing.
> >> >>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>> Thanks Josh,
> >> >>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>> S
> >> >>>>>>>>>>>>>
> >> >>>>>>>>>>>>>
> >> >>>>>>>>>>>>>
> >> >>>>>>>>>>>>>
> >> >>>>>>>>>>>>>
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>
> >> >>>>>>>>>>
> >> >>>>>>>>>
> >> >>>>>>>
> >> >>>>>>
> >> >>>>
> >> >>
> >>
> >
> >
>