Note breaking change on RegionObserver in hbase-2.0.0 (HBASE-15296 nice cleanup/refactor)

[Stack <st...@duboce.net>]

We cool w/ this?

(I know we keep saying it over and over again that its fine to break CPs
w/o deprecation but still uneasy doing the actual breakage.... hence the
note here.)

St.Ack

Re: Note breaking change on RegionObserver in hbase-2.0.0 (HBASE-15296 nice cleanup/refactor)

[Ted Yu <yu...@gmail.com>]

+1 to what Stephen said. 

> On Apr 14, 2016, at 3:37 PM, Stephen Jiang <sy...@gmail.com> wrote:
> 
> In the past, both HBASE-11425 and HBASE013645 used deprecation model,
> HBASE-15296 now uses replacement model.
> 
> Even we don't guarantee the compatibility of upgrading to 2.0.  the
> replacement model would make the upgrade more challenge and I am sure
> enterprise customer would stay in 1.x line longer.
> 
> My 2-cent is unless it is necessary (eg. no way to maintain it), we should
> try hard to make upgrade easier and use the deprecation model.
> 
> Thanks
> Stephen
> 
> On Thu, Apr 14, 2016 at 12:50 PM, Andrew Purtell <an...@gmail.com>
> wrote:
> 
>> Great idea.
>> 
>>> On Apr 14, 2016, at 12:35 PM, Sean Busbey <bu...@apache.org> wrote:
>>> 
>>> We could also set up a job to run through the compat checking script
>>> we have against Public and LimitedPrivate API nightly.
>>> 
>>>> On Thu, Apr 14, 2016 at 10:59 AM, Andrew Purtell <ap...@apache.org>
>> wrote:
>>>> I think a major version increment is when we've allowed ourselves
>> leeway to
>>>> make breaking changes. If we were to do this though I'd like to see us
>> roll
>>>> in as many as we can at once.
>>>> 
>>>> By the way, we are still sometimes breaking CPs without meaning to. I
>> think
>>>> we messed up the RpcScheduler LimitedPrivate interface in 1.2 with
>>>> HBASE-15146, which added a return type to RpcScheduler#dispatch, and
>> breaks
>>>> Phoenix. Would you lot be interested in setting up a Jenkins job that
>> uses
>>>> Phoenix to watch for accidental breakage? It's not comprehensive of
>> course
>>>> but might be the closest available thing to it.
>>>> 
>>>> 
>>>> 
>>>>> On Thu, Apr 14, 2016 at 8:50 AM, Stack <st...@duboce.net> wrote:
>>>>> 
>>>>> We cool w/ this?
>>>>> 
>>>>> (I know we keep saying it over and over again that its fine to break
>> CPs
>>>>> w/o deprecation but still uneasy doing the actual breakage.... hence
>> the
>>>>> note here.)
>>>>> 
>>>>> St.Ack
>>>> 
>>>> 
>>>> 
>>>> --
>>>> Best regards,
>>>> 
>>>>  - Andy
>>>> 
>>>> Problems worthy of attack prove their worth by hitting back. - Piet Hein
>>>> (via Tom White)
>> 

Re: Note breaking change on RegionObserver in hbase-2.0.0 (HBASE-15296 nice cleanup/refactor)

[Stack <st...@duboce.net>]

On Thu, Apr 14, 2016 at 3:37 PM, Stephen Jiang <sy...@gmail.com>
wrote:

> In the past, both HBASE-11425 and HBASE013645 used deprecation model,
> HBASE-15296 now uses replacement model.
>
> Even we don't guarantee the compatibility of upgrading to 2.0.  the
> replacement model would make the upgrade more challenge and I am sure
> enterprise customer would stay in 1.x line longer.
>
> My 2-cent is unless it is necessary (eg. no way to maintain it), we should
> try hard to make upgrade easier and use the deprecation model.
>
>
I've reopened HBASE-15296 to ask if the change can be made in a
non-breaking way (I think not since we'd have to retain removed types). I
could back out HBASE-15296 until there is more justification than 'cleanup'
for breaking CP APIs. My sense is that by the time of 2.0, the breakage
that HBASE-15296 does to the CP API will be minor in the scheme of things.

St.Ack




> Thanks
> Stephen
>
> On Thu, Apr 14, 2016 at 12:50 PM, Andrew Purtell <andrew.purtell@gmail.com
> >
> wrote:
>
> > Great idea.
> >
> > > On Apr 14, 2016, at 12:35 PM, Sean Busbey <bu...@apache.org> wrote:
> > >
> > > We could also set up a job to run through the compat checking script
> > > we have against Public and LimitedPrivate API nightly.
> > >
> > >> On Thu, Apr 14, 2016 at 10:59 AM, Andrew Purtell <apurtell@apache.org
> >
> > wrote:
> > >> I think a major version increment is when we've allowed ourselves
> > leeway to
> > >> make breaking changes. If we were to do this though I'd like to see us
> > roll
> > >> in as many as we can at once.
> > >>
> > >> By the way, we are still sometimes breaking CPs without meaning to. I
> > think
> > >> we messed up the RpcScheduler LimitedPrivate interface in 1.2 with
> > >> HBASE-15146, which added a return type to RpcScheduler#dispatch, and
> > breaks
> > >> Phoenix. Would you lot be interested in setting up a Jenkins job that
> > uses
> > >> Phoenix to watch for accidental breakage? It's not comprehensive of
> > course
> > >> but might be the closest available thing to it.
> > >>
> > >>
> > >>
> > >>> On Thu, Apr 14, 2016 at 8:50 AM, Stack <st...@duboce.net> wrote:
> > >>>
> > >>> We cool w/ this?
> > >>>
> > >>> (I know we keep saying it over and over again that its fine to break
> > CPs
> > >>> w/o deprecation but still uneasy doing the actual breakage.... hence
> > the
> > >>> note here.)
> > >>>
> > >>> St.Ack
> > >>
> > >>
> > >>
> > >> --
> > >> Best regards,
> > >>
> > >>   - Andy
> > >>
> > >> Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > >> (via Tom White)
> >
>

Re: Note breaking change on RegionObserver in hbase-2.0.0 (HBASE-15296 nice cleanup/refactor)

[Stephen Jiang <sy...@gmail.com>]

In the past, both HBASE-11425 and HBASE013645 used deprecation model,
HBASE-15296 now uses replacement model.

Even we don't guarantee the compatibility of upgrading to 2.0.  the
replacement model would make the upgrade more challenge and I am sure
enterprise customer would stay in 1.x line longer.

My 2-cent is unless it is necessary (eg. no way to maintain it), we should
try hard to make upgrade easier and use the deprecation model.

Thanks
Stephen

On Thu, Apr 14, 2016 at 12:50 PM, Andrew Purtell <an...@gmail.com>
wrote:

> Great idea.
>
> > On Apr 14, 2016, at 12:35 PM, Sean Busbey <bu...@apache.org> wrote:
> >
> > We could also set up a job to run through the compat checking script
> > we have against Public and LimitedPrivate API nightly.
> >
> >> On Thu, Apr 14, 2016 at 10:59 AM, Andrew Purtell <ap...@apache.org>
> wrote:
> >> I think a major version increment is when we've allowed ourselves
> leeway to
> >> make breaking changes. If we were to do this though I'd like to see us
> roll
> >> in as many as we can at once.
> >>
> >> By the way, we are still sometimes breaking CPs without meaning to. I
> think
> >> we messed up the RpcScheduler LimitedPrivate interface in 1.2 with
> >> HBASE-15146, which added a return type to RpcScheduler#dispatch, and
> breaks
> >> Phoenix. Would you lot be interested in setting up a Jenkins job that
> uses
> >> Phoenix to watch for accidental breakage? It's not comprehensive of
> course
> >> but might be the closest available thing to it.
> >>
> >>
> >>
> >>> On Thu, Apr 14, 2016 at 8:50 AM, Stack <st...@duboce.net> wrote:
> >>>
> >>> We cool w/ this?
> >>>
> >>> (I know we keep saying it over and over again that its fine to break
> CPs
> >>> w/o deprecation but still uneasy doing the actual breakage.... hence
> the
> >>> note here.)
> >>>
> >>> St.Ack
> >>
> >>
> >>
> >> --
> >> Best regards,
> >>
> >>   - Andy
> >>
> >> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> >> (via Tom White)
>

Re: Note breaking change on RegionObserver in hbase-2.0.0 (HBASE-15296 nice cleanup/refactor)

[Andrew Purtell <an...@gmail.com>]

Great idea. 

> On Apr 14, 2016, at 12:35 PM, Sean Busbey <bu...@apache.org> wrote:
> 
> We could also set up a job to run through the compat checking script
> we have against Public and LimitedPrivate API nightly.
> 
>> On Thu, Apr 14, 2016 at 10:59 AM, Andrew Purtell <ap...@apache.org> wrote:
>> I think a major version increment is when we've allowed ourselves leeway to
>> make breaking changes. If we were to do this though I'd like to see us roll
>> in as many as we can at once.
>> 
>> By the way, we are still sometimes breaking CPs without meaning to. I think
>> we messed up the RpcScheduler LimitedPrivate interface in 1.2 with
>> HBASE-15146, which added a return type to RpcScheduler#dispatch, and breaks
>> Phoenix. Would you lot be interested in setting up a Jenkins job that uses
>> Phoenix to watch for accidental breakage? It's not comprehensive of course
>> but might be the closest available thing to it.
>> 
>> 
>> 
>>> On Thu, Apr 14, 2016 at 8:50 AM, Stack <st...@duboce.net> wrote:
>>> 
>>> We cool w/ this?
>>> 
>>> (I know we keep saying it over and over again that its fine to break CPs
>>> w/o deprecation but still uneasy doing the actual breakage.... hence the
>>> note here.)
>>> 
>>> St.Ack
>> 
>> 
>> 
>> --
>> Best regards,
>> 
>>   - Andy
>> 
>> Problems worthy of attack prove their worth by hitting back. - Piet Hein
>> (via Tom White)

Re: Note breaking change on RegionObserver in hbase-2.0.0 (HBASE-15296 nice cleanup/refactor)

[Sean Busbey <bu...@apache.org>]

We could also set up a job to run through the compat checking script
we have against Public and LimitedPrivate API nightly.

On Thu, Apr 14, 2016 at 10:59 AM, Andrew Purtell <ap...@apache.org> wrote:
> I think a major version increment is when we've allowed ourselves leeway to
> make breaking changes. If we were to do this though I'd like to see us roll
> in as many as we can at once.
>
> By the way, we are still sometimes breaking CPs without meaning to. I think
> we messed up the RpcScheduler LimitedPrivate interface in 1.2 with
> HBASE-15146, which added a return type to RpcScheduler#dispatch, and breaks
> Phoenix. Would you lot be interested in setting up a Jenkins job that uses
> Phoenix to watch for accidental breakage? It's not comprehensive of course
> but might be the closest available thing to it.
>
>
>
> On Thu, Apr 14, 2016 at 8:50 AM, Stack <st...@duboce.net> wrote:
>
>> We cool w/ this?
>>
>> (I know we keep saying it over and over again that its fine to break CPs
>> w/o deprecation but still uneasy doing the actual breakage.... hence the
>> note here.)
>>
>> St.Ack
>>
>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)

Re: Note breaking change on RegionObserver in hbase-2.0.0 (HBASE-15296 nice cleanup/refactor)

[Anoop John <an...@gmail.com>]

Actually while working on HBSE-11425 (Off heaping read path), came in
a need for a change in a CP hook and I went with deprecation model
there. The old one is deprecated and new one is added. I think the
deprecated one can be removed now..
postScannerFilterRow(ObserverContext<RegionCoprocessorEnvironment>,
InternalScanner, byte[], int, short, boolean)   - Deprecated
postScannerFilterRow(ObserverContext<RegionCoprocessorEnvironment>,
InternalScanner, Cell, boolean)

Will do

-Anoop-

On Thu, Apr 14, 2016 at 11:18 PM, Andrew Purtell <ap...@apache.org> wrote:
> On Thu, Apr 14, 2016 at 10:25 AM, Stack <st...@duboce.net> wrote:
>
>> On Thu, Apr 14, 2016 at 8:59 AM, Andrew Purtell <ap...@apache.org>
>> wrote:
>>
>> > I think a major version increment is when we've allowed ourselves leeway
>> to
>> > make breaking changes. If we were to do this though I'd like to see us
>> roll
>> > in as many as we can at once.
>> >
>> >
>> Agreed.
>>
>> I suppose I'm opening the flood gates so bring on your CP changes in time
>> for 2.0!
>>
>>
>>
>> > By the way, we are still sometimes breaking CPs without meaning to. I
>> think
>> > we messed up the RpcScheduler LimitedPrivate interface in 1.2 with
>> > HBASE-15146, which added a return type to RpcScheduler#dispatch, and
>> breaks
>> > Phoenix. Would you lot be interested in setting up a Jenkins job that
>> uses
>> > Phoenix to watch for accidental breakage? It's not comprehensive of
>> course
>> > but might be the closest available thing to it.
>> >
>> >
>> Probably no harm. Phoenix would be the canary. As long as someone looks at
>> it though? It'd be branch-1 job?
>>
>>
> Yeah, it would run against branch-1.
>
> Let me ask over on dev@phoenix if they want to set up and tend the job.
> Failure mails would come over to builds@hbase though. I'd ask them to copy
> me because I can't handle builds@ traffic but would be specifically
> interested in Phoenix breakage. Sound reasonable?
>
>
>
>
>> St.Ack
>>
>>
>> >
>> >
>> > On Thu, Apr 14, 2016 at 8:50 AM, Stack <st...@duboce.net> wrote:
>> >
>> > > We cool w/ this?
>> > >
>> > > (I know we keep saying it over and over again that its fine to break
>> CPs
>> > > w/o deprecation but still uneasy doing the actual breakage.... hence
>> the
>> > > note here.)
>> > >
>> > > St.Ack
>> > >
>>

Re: Note breaking change on RegionObserver in hbase-2.0.0 (HBASE-15296 nice cleanup/refactor)

[Andrew Purtell <ap...@apache.org>]

On Thu, Apr 14, 2016 at 10:25 AM, Stack <st...@duboce.net> wrote:

> On Thu, Apr 14, 2016 at 8:59 AM, Andrew Purtell <ap...@apache.org>
> wrote:
>
> > I think a major version increment is when we've allowed ourselves leeway
> to
> > make breaking changes. If we were to do this though I'd like to see us
> roll
> > in as many as we can at once.
> >
> >
> Agreed.
>
> I suppose I'm opening the flood gates so bring on your CP changes in time
> for 2.0!
>
>
>
> > By the way, we are still sometimes breaking CPs without meaning to. I
> think
> > we messed up the RpcScheduler LimitedPrivate interface in 1.2 with
> > HBASE-15146, which added a return type to RpcScheduler#dispatch, and
> breaks
> > Phoenix. Would you lot be interested in setting up a Jenkins job that
> uses
> > Phoenix to watch for accidental breakage? It's not comprehensive of
> course
> > but might be the closest available thing to it.
> >
> >
> Probably no harm. Phoenix would be the canary. As long as someone looks at
> it though? It'd be branch-1 job?
>
>
​Yeah, it would run against branch-1.
​
Let me ask over on dev@phoenix if they want to set up and tend the job.
Failure mails would come over to builds@hbase though. I'd ask them to copy
me because I can't handle builds@ traffic but would be specifically
interested in Phoenix breakage. Sound reasonable?




> St.Ack
>
>
> >
> >
> > On Thu, Apr 14, 2016 at 8:50 AM, Stack <st...@duboce.net> wrote:
> >
> > > We cool w/ this?
> > >
> > > (I know we keep saying it over and over again that its fine to break
> CPs
> > > w/o deprecation but still uneasy doing the actual breakage.... hence
> the
> > > note here.)
> > >
> > > St.Ack
> > >
>

Re: Note breaking change on RegionObserver in hbase-2.0.0 (HBASE-15296 nice cleanup/refactor)

[Andrew Purtell <ap...@apache.org>]

On Thu, Apr 14, 2016 at 10:43 AM, Matteo Bertozzi <th...@gmail.com>
wrote:

> If we change the coprocessor API we should consider a better
> naming/semantic. see HBASE-6992
>
>
​I also put a bunch of BS up on
https://issues.apache.org/jira/browse/HBASE-11125 once. ​It went off the
rails toward the end but I do like the idea of having a set of
occurrances/events upon which a CP user can register intent to observe
success or failure or intercept the operation.



> pre/postOperation doesn't do what the user want because the post is not
> really post.
> our execution in genereal is:
>
>    - pre operation (rpc thread)
>    - submit operation
>    - post operation (rpc thread)
>    - [pre operation (handler thread)]
>    - [post operation (handler thread)]
>
> for the user the pre and post should probably be the actual pre/post.
> which means pre (rpc thread) and post (handler thread).
>
> also, today on failure we don't have a post operation.
> which means that if someone does something in the pre operation there is no
> way to rollback that on failure.
> so, we should also add something like a "post operation failure".
>
>
>
> Matteo
>
>
> On Thu, Apr 14, 2016 at 10:25 AM, Stack <st...@duboce.net> wrote:
>
> > On Thu, Apr 14, 2016 at 8:59 AM, Andrew Purtell <ap...@apache.org>
> > wrote:
> >
> > > I think a major version increment is when we've allowed ourselves
> leeway
> > to
> > > make breaking changes. If we were to do this though I'd like to see us
> > roll
> > > in as many as we can at once.
> > >
> > >
> > Agreed.
> >
> > I suppose I'm opening the flood gates so bring on your CP changes in time
> > for 2.0!
> >
> >
> >
> > > By the way, we are still sometimes breaking CPs without meaning to. I
> > think
> > > we messed up the RpcScheduler LimitedPrivate interface in 1.2 with
> > > HBASE-15146, which added a return type to RpcScheduler#dispatch, and
> > breaks
> > > Phoenix. Would you lot be interested in setting up a Jenkins job that
> > uses
> > > Phoenix to watch for accidental breakage? It's not comprehensive of
> > course
> > > but might be the closest available thing to it.
> > >
> > >
> > Probably no harm. Phoenix would be the canary. As long as someone looks
> at
> > it though? It'd be branch-1 job?
> >
> > St.Ack
> >
> >
> > >
> > >
> > > On Thu, Apr 14, 2016 at 8:50 AM, Stack <st...@duboce.net> wrote:
> > >
> > > > We cool w/ this?
> > > >
> > > > (I know we keep saying it over and over again that its fine to break
> > CPs
> > > > w/o deprecation but still uneasy doing the actual breakage.... hence
> > the
> > > > note here.)
> > > >
> > > > St.Ack
> > > >
> > >
> > >
> > >
> > > --
> > > Best regards,
> > >
> > >    - Andy
> > >
> > > Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > > (via Tom White)
> > >
> >
>



-- 
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Re: Note breaking change on RegionObserver in hbase-2.0.0 (HBASE-15296 nice cleanup/refactor)

[Matteo Bertozzi <th...@gmail.com>]

If we change the coprocessor API we should consider a better
naming/semantic. see HBASE-6992

pre/postOperation doesn't do what the user want because the post is not
really post.
our execution in genereal is:

   - pre operation (rpc thread)
   - submit operation
   - post operation (rpc thread)
   - [pre operation (handler thread)]
   - [post operation (handler thread)]

for the user the pre and post should probably be the actual pre/post.
which means pre (rpc thread) and post (handler thread).

also, today on failure we don't have a post operation.
which means that if someone does something in the pre operation there is no
way to rollback that on failure.
so, we should also add something like a "post operation failure".



Matteo


On Thu, Apr 14, 2016 at 10:25 AM, Stack <st...@duboce.net> wrote:

> On Thu, Apr 14, 2016 at 8:59 AM, Andrew Purtell <ap...@apache.org>
> wrote:
>
> > I think a major version increment is when we've allowed ourselves leeway
> to
> > make breaking changes. If we were to do this though I'd like to see us
> roll
> > in as many as we can at once.
> >
> >
> Agreed.
>
> I suppose I'm opening the flood gates so bring on your CP changes in time
> for 2.0!
>
>
>
> > By the way, we are still sometimes breaking CPs without meaning to. I
> think
> > we messed up the RpcScheduler LimitedPrivate interface in 1.2 with
> > HBASE-15146, which added a return type to RpcScheduler#dispatch, and
> breaks
> > Phoenix. Would you lot be interested in setting up a Jenkins job that
> uses
> > Phoenix to watch for accidental breakage? It's not comprehensive of
> course
> > but might be the closest available thing to it.
> >
> >
> Probably no harm. Phoenix would be the canary. As long as someone looks at
> it though? It'd be branch-1 job?
>
> St.Ack
>
>
> >
> >
> > On Thu, Apr 14, 2016 at 8:50 AM, Stack <st...@duboce.net> wrote:
> >
> > > We cool w/ this?
> > >
> > > (I know we keep saying it over and over again that its fine to break
> CPs
> > > w/o deprecation but still uneasy doing the actual breakage.... hence
> the
> > > note here.)
> > >
> > > St.Ack
> > >
> >
> >
> >
> > --
> > Best regards,
> >
> >    - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
> >
>

Re: Note breaking change on RegionObserver in hbase-2.0.0 (HBASE-15296 nice cleanup/refactor)

[Stack <st...@duboce.net>]

On Thu, Apr 14, 2016 at 8:59 AM, Andrew Purtell <ap...@apache.org> wrote:

> I think a major version increment is when we've allowed ourselves leeway to
> make breaking changes. If we were to do this though I'd like to see us roll
> in as many as we can at once.
>
>
Agreed.

I suppose I'm opening the flood gates so bring on your CP changes in time
for 2.0!



> By the way, we are still sometimes breaking CPs without meaning to. I think
> we messed up the RpcScheduler LimitedPrivate interface in 1.2 with
> HBASE-15146, which added a return type to RpcScheduler#dispatch, and breaks
> Phoenix. Would you lot be interested in setting up a Jenkins job that uses
> Phoenix to watch for accidental breakage? It's not comprehensive of course
> but might be the closest available thing to it.
>
>
Probably no harm. Phoenix would be the canary. As long as someone looks at
it though? It'd be branch-1 job?

St.Ack


>
>
> On Thu, Apr 14, 2016 at 8:50 AM, Stack <st...@duboce.net> wrote:
>
> > We cool w/ this?
> >
> > (I know we keep saying it over and over again that its fine to break CPs
> > w/o deprecation but still uneasy doing the actual breakage.... hence the
> > note here.)
> >
> > St.Ack
> >
>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>

Re: Note breaking change on RegionObserver in hbase-2.0.0 (HBASE-15296 nice cleanup/refactor)

[Andrew Purtell <ap...@apache.org>]

I think a major version increment is when we've allowed ourselves leeway to
make breaking changes. If we were to do this though I'd like to see us roll
in as many as we can at once.

By the way, we are still sometimes breaking CPs without meaning to. I think
we messed up the RpcScheduler LimitedPrivate interface in 1.2 with
HBASE-15146, which added a return type to RpcScheduler#dispatch, and breaks
Phoenix. Would you lot be interested in setting up a Jenkins job that uses
Phoenix to watch for accidental breakage? It's not comprehensive of course
but might be the closest available thing to it.



On Thu, Apr 14, 2016 at 8:50 AM, Stack <st...@duboce.net> wrote:

> We cool w/ this?
>
> (I know we keep saying it over and over again that its fine to break CPs
> w/o deprecation but still uneasy doing the actual breakage.... hence the
> note here.)
>
> St.Ack
>



-- 
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)