Re: [DISCUSS] hbase-2.0.0 compatibility expectations

[stack <sa...@gmail.com>]

On Thu, Aug 24, 2017 at 9:13 AM, stack <sa...@gmail.com> wrote:

> On Thu, Aug 3, 2017 at 11:05 PM, stack <sa...@gmail.com> wrote:
>
>> Thanks Zach for clarification. Let me work up a list and then come back
>> to this thread.  Jira needs an edit pass to.
>>
>>
> JIRA editing reflecting incompatibles has been coming along. It is still a
> work-in-progress though. Meantime, let me air out here the incompatibles as
> I encounter them. The complete list may never be known (smile) and the best
> knowledge will only be had on the day before release which will be giving
> notice too late for folks to do much about it.
>
> Here is a known breaking incompatible that just went in: HBASE-15982
> "Interface ReplicationEndpoint extends Guava's Service". See the release
> note for why the breakage unavoidable. Anyone who has implemented our
> Replication Endpoint will have a refactor on their hands deploying their
> replication machine atop hbase2 (Side note: Estebans' initial work has it
> that hbase native replication seems to work going from hbase1 to hbase2 and
> even the other way around; more to do....). Upside, if there is the will,
> now is the time for HBASE-10504 "Define Replication Interface". It is late
> in the game but am up for helping out if interest.
>
> Here is a breaking compatible we *could* (maybe) avoid but the amount of
> work involved in the workaround and how the patch-up would muddy our
> messaging around protobufs in the HBase public API is not worth the price
> IMO. What do you all think? The issue is HBASE-15607 "Remove PB references
> from Admin for 2.0". The plan is to (re-)commit Ram's change. We'd then
> tell users that you cannot administer an hbase2 cluster using an hbase1
> client.
>
>
Ok. The "could" above has changed to a "can't" on review of changes in
Admin Interface since hbase1.0.0.  The amount of work involved would be too
much and the methods we'd be restoring compatibility on are wonky in the
first place; i.e. async calls that gave you no means of querying state of
the call and protobuf Messages that would require 'translation' from our
internal protobuf to protobuf 2.5. Admin Interface in 2.0 will be
incompatible with hbase 1.x Admin.

I've a WIP incompatibles list as a messy subsection of the hbase2 doc [1].

St.Ack


1.
https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9iEu_ktczrlKHK8N4SZzs/edit#heading=h.723jjn18p2jr




> Coprocessors will require a refactor to deploy on hbase2. More detail to
> follow.
>
> St.Ack
>
>
>
>
>
>
>> S
>>
>> On Aug 3, 2017 23:54, "Zach York" <zy...@gmail.com> wrote:
>>
>> This kinda helps, but these seem more like expectations. I was going more
>> for things like HFile format changed, meta table structure changed,
>> coprocessor implementations changed (these are just examples, I don't know
>> if any of these actually changed).
>>
>> More technical differences between branch-1 and branch-2 which then can
>> help us get the right expectations for compatibility.
>>
>> On Wed, Aug 2, 2017 at 6:34 PM, Stack <st...@duboce.net> wrote:
>>
>> > On Wed, Aug 2, 2017 at 5:25 PM, Zach York <zyork.contribution@gmail.com
>> >
>> > wrote:
>> >
>> > > Do we know what the major pain points for migration are? Can we
>> discuss
>> > > that/get a list going?
>> > >
>> > >
>> > Here's a few in outline:
>> >
>> > + There is issue of formats, of hbase-2.x being able to read hbase-1.x
>> data
>> > whether from HDFS or ZooKeeper or off the wire.
>> > + An hbase-1.x client should be able to Get/Put and Scan an hbase-2.x
>> > cluster; no holes in the API or unintelligible serializations.
>> > + There is then the little dance that has us rolling restart from an
>> > hbase-1.x cluster to hbase-2.x; i.e. upgrade master first and then it
>> will
>> > assign regions to the new hbase-2.x regionservers as they come on line.
>> > TBD.
>> >
>> > Is this what you mean sir?
>> >
>> > S
>> >
>> >
>> > > I think without that knowledge it is hard (for me at least :) ) to
>> > > determine where we should set our sights in terms of migration.
>> > >
>> > > Thanks,
>> > > Zach
>> > >
>> > > On Wed, Aug 2, 2017 at 4:38 PM, Stack <st...@duboce.net> wrote:
>> > >
>> > > > What are our expectations regards compatibility between hbase1 and
>> > > hbase2?
>> > > >
>> > > > Lets have a chat about it. Here are some goal posts.
>> > > >
>> > > > + You have to upgrade to hbase-1.x before you can migrate to
>> hbase-2.
>> > No
>> > > > migration from < hbase-1 (Is this too onerous? Should we support
>> 0.98
>> > =>
>> > > > 2.0?).
>> > > > + You do NOT have to upgrade to the latest release of hbase1 to
>> migrate
>> > > to
>> > > > hbase2; being up on hbase-1.0.0+ will be sufficient.
>> > > > + You'll have to update your hbase1 coprocessors to deploy them on
>> > > hbase2.
>> > > > A bunch of CP API has/will change by the time hbase2 comes out; e.g.
>> > > > watching for region split on RegionServer no longer makes sense
>> given
>> > > > Master runs all splits now.
>> > > > + An hbase1 client can run against an hbase2 cluster but it will
>> only
>> > be
>> > > > able to do DML (Get/Put/Scan, etc.). We do not allow being able to
>> do
>> > > admin
>> > > > ops using an hbase1 Admin client against an hbase2 cluster. We have
>> > some
>> > > > egregious API violations in branch-1; e.g. we have protobuf in our
>> API
>> > > (See
>> > > > HBASE-15607). The notion is that we can't afford a deprecation cycle
>> > > > purging this stuff from our Admin API.
>> > > >
>> > > > What you all think?
>> > > >
>> > > > St.Ack
>> > > >
>> > >
>> >
>>
>>
>>
>