You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@phoenix.apache.org by Jeffrey Zhong <jz...@hortonworks.com> on 2014/02/10 22:38:57 UTC
Branch 4.0.0 is created for Phoenix on HBase0.98
Hey,
I created branch 4.0.0 from master branch for Phoenix on HBase0.98. The
code can work on tip of HBase0.98 & trunk branch but not HBase0.96(only
need trivial changes to make it work though) because some "private" Hbase
interface changes from release to release. We need a good way to remove
all private interface dependencies or convince Hbase folks to make those
interface backward compatible.
We thought about to use a shim layer to load either Hbase0.94 or Hbase0.98
dependencies upon different hbase deployment so that we can put this code
in master branch. The difficulty is that Phoenix isn't a normal Hbase
application. It extensively uses Hbase internals: Coprocessors, WAL,
Filters, KeyValue etc which have most incompatible changes between 0.94 &
0.96+. In addition, if more & more people are moving to the modern Hbase
code, it's not worth to put significant effort to create such a shim layer.
So far all unit tests are passed in 4.0.0 branch. I'll do one more merge
to get checkins between 2/8 and today. Moving forward please merge your
changes to 4.0.0 in order for it to be used in hbase96+(the bright
future). During merge, please don't use deprecated APIs because those APIs
mostly internally create a new copy(e.g. Mutation#getFamilyMap) and cause
unexpected behaviors if you're thinking to change internal state of an
object.
Thanks,
-Jeffrey
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Branch 4.0.0 is created for Phoenix on HBase0.98
Posted by Jesse Yates <je...@gmail.com>.
Branches make sense where you are going to be making more changes along
that same line. A branch for 4.0.0 implies that you are going to release a
4.0.0.1, 4.0.0.2, etc. (at least to me).
By having a 4.0 branch, you *more* lean towards semantic versioning because
along that same branch you can easily release 4.0.1, 4.0.2, etc. and just
mark them with tags.
When you are ready for the 4.1 jump (enough big, but not breaking,
features), you can cut a new branch off the 4.0 series as 4.1* and continue
to tag the point releases in each line as needed (say the 4.0.X series
needed a point release, this allows you to easily track that state).
--j
-------------------
Jesse Yates
@jesse_yates
jyates.github.com
On Mon, Feb 10, 2014 at 4:03 PM, lars hofhansl <la...@apache.org> wrote:
> Personally I'd be in favor of 4.0.0 as this conveys the fact that we do
> semantic versioning.
> As usually I do not feel strongly about this.
>
> -- Lars
>
>
> ________________________________
> From: James Taylor <ja...@apache.org>
> To: "dev@phoenix.incubator.apache.org" <de...@phoenix.incubator.apache.org>
> Sent: Monday, February 10, 2014 3:50 PM
> Subject: Re: Branch 4.0.0 is created for Phoenix on HBase0.98
>
>
> Sure, 4.0 is fine. I think we can just tag our minor and point releases
> instead of creating branches for them.
>
>
>
> On Mon, Feb 10, 2014 at 3:41 PM, Jeffrey Zhong <jzhong@hortonworks.com
> >wrote:
>
> >
> > James, how do you think the branch name 4.0 over 4.0.0? It mostly depends
> > on the naming convention. 4.0 seems better
> > unless in the future we have branch like 4.0.1(or 4.0.##)
> >
> > > If you set up a job building against HBase trunk snapshots (and
> > HBase remembers to publish those then
> > > Jenkins can catch any incompatibilities accidentally introduced
> > going forward.
> >
> >
> > I like the above idea so that we can catch interface changes as earlier
> as
> > possible.
> >
> >
> > On 2/10/14 2:33 PM, "James Taylor" <ja...@apache.org> wrote:
> >
> > >+1. That's a very good idea, Enis - for HBase 1.0 if possible.
> > >
> > >
> > >On Mon, Feb 10, 2014 at 2:28 PM, Enis Söztutar <en...@gmail.com>
> > wrote:
> > >
> > >> Forgot to mention, there was some discussion about filter /
> coprocessors
> > >> versus plugins. It seems that coprocessors are doing both more
> > >>user-facing
> > >> database trigger kind of functionality + some plugging into Hbase
> > >>internals
> > >> functionality. I think longer term, we should define different
> > >>interfaces
> > >> for some pluggable functionality (see recent StorageEngine,
> Compaction,
> > >> Flush, HLog, etc) and keep coprocessors more simple. This was plugins
> > >>would
> > >> be the supported way to extend HBase, and the interfaces would be more
> > >> stable.
> > >>
> > >>
> > >> On Mon, Feb 10, 2014 at 2:23 PM, Enis Söztutar <en...@gmail.com>
> > >>wrote:
> > >>
> > >> > Awesome work Jeffrey.
> > >> >
> > >> > Should we name the branch 4.0, instead of 4.0.0?
> > >> >
> > >> > It would be good to have a list of methods used by Phoenix to be
> > >> > identified and tagged in HBase so that at least future changes are
> not
> > >> > completely random. We might start with annotation
> > >> > InterfaceAudience.LimitedPrivate, so that HBase devs will be more
> > >> careful.
> > >> > However, we should also try to make Phoenix not so far-reaching into
> > >> HBase
> > >> > internals as well.
> > >> >
> > >> > Enis
> > >> >
> > >> >
> > >> > On Mon, Feb 10, 2014 at 2:16 PM, Andrew Purtell <
> apurtell@apache.org
> > >> >wrote:
> > >> >
> > >> >> On Mon, Feb 10, 2014 at 2:07 PM, James Taylor <
> > jamestaylor@apache.org
> > >> >> >wrote:
> > >> >>
> > >> >> > Mujtaba - would it be
> > >> >> > feasible to setup Jenkins builds for this branch as well?
> > >> >> >
> > >> >>
> > >> >> Or, HBase 0.98 is really close to HBase trunk still. If you set up
> a
> > >>job
> > >> >> building against HBase trunk snapshots (and HBase remembers to
> > >>publish
> > >> >> those...) then Jenkins can catch any incompatibilities accidentally
> > >> >> introduced going forward.
> > >> >>
> > >> >> --
> > >> >> Best regards,
> > >> >>
> > >> >> - Andy
> > >> >>
> > >> >> Problems worthy of attack prove their worth by hitting back. - Piet
> > >>Hein
> > >> >> (via Tom White)
> > >> >>
> > >> >
> > >> >
> > >>
> >
> >
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
> >
>
Re: Branch 4.0.0 is created for Phoenix on HBase0.98
Posted by lars hofhansl <la...@apache.org>.
Personally I'd be in favor of 4.0.0 as this conveys the fact that we do semantic versioning.
As usually I do not feel strongly about this.
-- Lars
________________________________
From: James Taylor <ja...@apache.org>
To: "dev@phoenix.incubator.apache.org" <de...@phoenix.incubator.apache.org>
Sent: Monday, February 10, 2014 3:50 PM
Subject: Re: Branch 4.0.0 is created for Phoenix on HBase0.98
Sure, 4.0 is fine. I think we can just tag our minor and point releases
instead of creating branches for them.
On Mon, Feb 10, 2014 at 3:41 PM, Jeffrey Zhong <jz...@hortonworks.com>wrote:
>
> James, how do you think the branch name 4.0 over 4.0.0? It mostly depends
> on the naming convention. 4.0 seems better
> unless in the future we have branch like 4.0.1(or 4.0.##)
>
> > If you set up a job building against HBase trunk snapshots (and
> HBase remembers to publish those then
> > Jenkins can catch any incompatibilities accidentally introduced
> going forward.
>
>
> I like the above idea so that we can catch interface changes as earlier as
> possible.
>
>
> On 2/10/14 2:33 PM, "James Taylor" <ja...@apache.org> wrote:
>
> >+1. That's a very good idea, Enis - for HBase 1.0 if possible.
> >
> >
> >On Mon, Feb 10, 2014 at 2:28 PM, Enis Söztutar <en...@gmail.com>
> wrote:
> >
> >> Forgot to mention, there was some discussion about filter / coprocessors
> >> versus plugins. It seems that coprocessors are doing both more
> >>user-facing
> >> database trigger kind of functionality + some plugging into Hbase
> >>internals
> >> functionality. I think longer term, we should define different
> >>interfaces
> >> for some pluggable functionality (see recent StorageEngine, Compaction,
> >> Flush, HLog, etc) and keep coprocessors more simple. This was plugins
> >>would
> >> be the supported way to extend HBase, and the interfaces would be more
> >> stable.
> >>
> >>
> >> On Mon, Feb 10, 2014 at 2:23 PM, Enis Söztutar <en...@gmail.com>
> >>wrote:
> >>
> >> > Awesome work Jeffrey.
> >> >
> >> > Should we name the branch 4.0, instead of 4.0.0?
> >> >
> >> > It would be good to have a list of methods used by Phoenix to be
> >> > identified and tagged in HBase so that at least future changes are not
> >> > completely random. We might start with annotation
> >> > InterfaceAudience.LimitedPrivate, so that HBase devs will be more
> >> careful.
> >> > However, we should also try to make Phoenix not so far-reaching into
> >> HBase
> >> > internals as well.
> >> >
> >> > Enis
> >> >
> >> >
> >> > On Mon, Feb 10, 2014 at 2:16 PM, Andrew Purtell <apurtell@apache.org
> >> >wrote:
> >> >
> >> >> On Mon, Feb 10, 2014 at 2:07 PM, James Taylor <
> jamestaylor@apache.org
> >> >> >wrote:
> >> >>
> >> >> > Mujtaba - would it be
> >> >> > feasible to setup Jenkins builds for this branch as well?
> >> >> >
> >> >>
> >> >> Or, HBase 0.98 is really close to HBase trunk still. If you set up a
> >>job
> >> >> building against HBase trunk snapshots (and HBase remembers to
> >>publish
> >> >> those...) then Jenkins can catch any incompatibilities accidentally
> >> >> introduced going forward.
> >> >>
> >> >> --
> >> >> Best regards,
> >> >>
> >> >> - Andy
> >> >>
> >> >> Problems worthy of attack prove their worth by hitting back. - Piet
> >>Hein
> >> >> (via Tom White)
> >> >>
> >> >
> >> >
> >>
>
>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
Re: Branch 4.0.0 is created for Phoenix on HBase0.98
Posted by Jesse Yates <je...@gmail.com>.
+1
Caveat that that 4.1, 4.2 lines should probably get their own branches.
-------------------
Jesse Yates
@jesse_yates
jyates.github.com
On Mon, Feb 10, 2014 at 3:50 PM, James Taylor <ja...@apache.org>wrote:
> Sure, 4.0 is fine. I think we can just tag our minor and point releases
> instead of creating branches for them.
>
>
> On Mon, Feb 10, 2014 at 3:41 PM, Jeffrey Zhong <jzhong@hortonworks.com
> >wrote:
>
> >
> > James, how do you think the branch name 4.0 over 4.0.0? It mostly depends
> > on the naming convention. 4.0 seems better
> > unless in the future we have branch like 4.0.1(or 4.0.##)
> >
> > > If you set up a job building against HBase trunk snapshots (and
> > HBase remembers to publish those then
> > > Jenkins can catch any incompatibilities accidentally introduced
> > going forward.
> >
> >
> > I like the above idea so that we can catch interface changes as earlier
> as
> > possible.
> >
> >
> > On 2/10/14 2:33 PM, "James Taylor" <ja...@apache.org> wrote:
> >
> > >+1. That's a very good idea, Enis - for HBase 1.0 if possible.
> > >
> > >
> > >On Mon, Feb 10, 2014 at 2:28 PM, Enis Söztutar <en...@gmail.com>
> > wrote:
> > >
> > >> Forgot to mention, there was some discussion about filter /
> coprocessors
> > >> versus plugins. It seems that coprocessors are doing both more
> > >>user-facing
> > >> database trigger kind of functionality + some plugging into Hbase
> > >>internals
> > >> functionality. I think longer term, we should define different
> > >>interfaces
> > >> for some pluggable functionality (see recent StorageEngine,
> Compaction,
> > >> Flush, HLog, etc) and keep coprocessors more simple. This was plugins
> > >>would
> > >> be the supported way to extend HBase, and the interfaces would be more
> > >> stable.
> > >>
> > >>
> > >> On Mon, Feb 10, 2014 at 2:23 PM, Enis Söztutar <en...@gmail.com>
> > >>wrote:
> > >>
> > >> > Awesome work Jeffrey.
> > >> >
> > >> > Should we name the branch 4.0, instead of 4.0.0?
> > >> >
> > >> > It would be good to have a list of methods used by Phoenix to be
> > >> > identified and tagged in HBase so that at least future changes are
> not
> > >> > completely random. We might start with annotation
> > >> > InterfaceAudience.LimitedPrivate, so that HBase devs will be more
> > >> careful.
> > >> > However, we should also try to make Phoenix not so far-reaching into
> > >> HBase
> > >> > internals as well.
> > >> >
> > >> > Enis
> > >> >
> > >> >
> > >> > On Mon, Feb 10, 2014 at 2:16 PM, Andrew Purtell <
> apurtell@apache.org
> > >> >wrote:
> > >> >
> > >> >> On Mon, Feb 10, 2014 at 2:07 PM, James Taylor <
> > jamestaylor@apache.org
> > >> >> >wrote:
> > >> >>
> > >> >> > Mujtaba - would it be
> > >> >> > feasible to setup Jenkins builds for this branch as well?
> > >> >> >
> > >> >>
> > >> >> Or, HBase 0.98 is really close to HBase trunk still. If you set up
> a
> > >>job
> > >> >> building against HBase trunk snapshots (and HBase remembers to
> > >>publish
> > >> >> those...) then Jenkins can catch any incompatibilities accidentally
> > >> >> introduced going forward.
> > >> >>
> > >> >> --
> > >> >> Best regards,
> > >> >>
> > >> >> - Andy
> > >> >>
> > >> >> Problems worthy of attack prove their worth by hitting back. - Piet
> > >>Hein
> > >> >> (via Tom White)
> > >> >>
> > >> >
> > >> >
> > >>
> >
> >
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
> >
>
Re: Branch 4.0.0 is created for Phoenix on HBase0.98
Posted by James Taylor <ja...@apache.org>.
Sure, 4.0 is fine. I think we can just tag our minor and point releases
instead of creating branches for them.
On Mon, Feb 10, 2014 at 3:41 PM, Jeffrey Zhong <jz...@hortonworks.com>wrote:
>
> James, how do you think the branch name 4.0 over 4.0.0? It mostly depends
> on the naming convention. 4.0 seems better
> unless in the future we have branch like 4.0.1(or 4.0.##)
>
> > If you set up a job building against HBase trunk snapshots (and
> HBase remembers to publish those then
> > Jenkins can catch any incompatibilities accidentally introduced
> going forward.
>
>
> I like the above idea so that we can catch interface changes as earlier as
> possible.
>
>
> On 2/10/14 2:33 PM, "James Taylor" <ja...@apache.org> wrote:
>
> >+1. That's a very good idea, Enis - for HBase 1.0 if possible.
> >
> >
> >On Mon, Feb 10, 2014 at 2:28 PM, Enis Söztutar <en...@gmail.com>
> wrote:
> >
> >> Forgot to mention, there was some discussion about filter / coprocessors
> >> versus plugins. It seems that coprocessors are doing both more
> >>user-facing
> >> database trigger kind of functionality + some plugging into Hbase
> >>internals
> >> functionality. I think longer term, we should define different
> >>interfaces
> >> for some pluggable functionality (see recent StorageEngine, Compaction,
> >> Flush, HLog, etc) and keep coprocessors more simple. This was plugins
> >>would
> >> be the supported way to extend HBase, and the interfaces would be more
> >> stable.
> >>
> >>
> >> On Mon, Feb 10, 2014 at 2:23 PM, Enis Söztutar <en...@gmail.com>
> >>wrote:
> >>
> >> > Awesome work Jeffrey.
> >> >
> >> > Should we name the branch 4.0, instead of 4.0.0?
> >> >
> >> > It would be good to have a list of methods used by Phoenix to be
> >> > identified and tagged in HBase so that at least future changes are not
> >> > completely random. We might start with annotation
> >> > InterfaceAudience.LimitedPrivate, so that HBase devs will be more
> >> careful.
> >> > However, we should also try to make Phoenix not so far-reaching into
> >> HBase
> >> > internals as well.
> >> >
> >> > Enis
> >> >
> >> >
> >> > On Mon, Feb 10, 2014 at 2:16 PM, Andrew Purtell <apurtell@apache.org
> >> >wrote:
> >> >
> >> >> On Mon, Feb 10, 2014 at 2:07 PM, James Taylor <
> jamestaylor@apache.org
> >> >> >wrote:
> >> >>
> >> >> > Mujtaba - would it be
> >> >> > feasible to setup Jenkins builds for this branch as well?
> >> >> >
> >> >>
> >> >> Or, HBase 0.98 is really close to HBase trunk still. If you set up a
> >>job
> >> >> building against HBase trunk snapshots (and HBase remembers to
> >>publish
> >> >> those...) then Jenkins can catch any incompatibilities accidentally
> >> >> introduced going forward.
> >> >>
> >> >> --
> >> >> Best regards,
> >> >>
> >> >> - Andy
> >> >>
> >> >> Problems worthy of attack prove their worth by hitting back. - Piet
> >>Hein
> >> >> (via Tom White)
> >> >>
> >> >
> >> >
> >>
>
>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
Re: Branch 4.0.0 is created for Phoenix on HBase0.98
Posted by Andrew Purtell <ap...@apache.org>.
Unlikely if 1.0 is going to be in a couple of months, unless volunteers
step forward to do that big refactor tomorrow. Also, discussion of a change
of that magnitude should definitely surface to dev@hbase.
On Mon, Feb 10, 2014 at 2:33 PM, James Taylor <ja...@apache.org>wrote:
> +1. That's a very good idea, Enis - for HBase 1.0 if possible.
>
>
> On Mon, Feb 10, 2014 at 2:28 PM, Enis Söztutar <en...@gmail.com> wrote:
>
> > Forgot to mention, there was some discussion about filter / coprocessors
> > versus plugins. It seems that coprocessors are doing both more
> user-facing
> > database trigger kind of functionality + some plugging into Hbase
> internals
> > functionality. I think longer term, we should define different interfaces
> > for some pluggable functionality (see recent StorageEngine, Compaction,
> > Flush, HLog, etc) and keep coprocessors more simple. This was plugins
> would
> > be the supported way to extend HBase, and the interfaces would be more
> > stable.
> >
> >
> > On Mon, Feb 10, 2014 at 2:23 PM, Enis Söztutar <en...@gmail.com>
> wrote:
> >
> > > Awesome work Jeffrey.
> > >
> > > Should we name the branch 4.0, instead of 4.0.0?
> > >
> > > It would be good to have a list of methods used by Phoenix to be
> > > identified and tagged in HBase so that at least future changes are not
> > > completely random. We might start with annotation
> > > InterfaceAudience.LimitedPrivate, so that HBase devs will be more
> > careful.
> > > However, we should also try to make Phoenix not so far-reaching into
> > HBase
> > > internals as well.
> > >
> > > Enis
> > >
> > >
> > > On Mon, Feb 10, 2014 at 2:16 PM, Andrew Purtell <apurtell@apache.org
> > >wrote:
> > >
> > >> On Mon, Feb 10, 2014 at 2:07 PM, James Taylor <jamestaylor@apache.org
> > >> >wrote:
> > >>
> > >> > Mujtaba - would it be
> > >> > feasible to setup Jenkins builds for this branch as well?
> > >> >
> > >>
> > >> Or, HBase 0.98 is really close to HBase trunk still. If you set up a
> job
> > >> building against HBase trunk snapshots (and HBase remembers to publish
> > >> those...) then Jenkins can catch any incompatibilities accidentally
> > >> introduced going forward.
> > >>
> > >> --
> > >> Best regards,
> > >>
> > >> - Andy
> > >>
> > >> Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > >> (via Tom White)
> > >>
> > >
> > >
> >
>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: Branch 4.0.0 is created for Phoenix on HBase0.98
Posted by Jeffrey Zhong <jz...@hortonworks.com>.
James, how do you think the branch name 4.0 over 4.0.0? It mostly depends
on the naming convention. 4.0 seems better
unless in the future we have branch like 4.0.1(or 4.0.##)
> If you set up a job building against HBase trunk snapshots (and
HBase remembers to publish those then
> Jenkins can catch any incompatibilities accidentally introduced
going forward.
I like the above idea so that we can catch interface changes as earlier as
possible.
On 2/10/14 2:33 PM, "James Taylor" <ja...@apache.org> wrote:
>+1. That's a very good idea, Enis - for HBase 1.0 if possible.
>
>
>On Mon, Feb 10, 2014 at 2:28 PM, Enis Söztutar <en...@gmail.com> wrote:
>
>> Forgot to mention, there was some discussion about filter / coprocessors
>> versus plugins. It seems that coprocessors are doing both more
>>user-facing
>> database trigger kind of functionality + some plugging into Hbase
>>internals
>> functionality. I think longer term, we should define different
>>interfaces
>> for some pluggable functionality (see recent StorageEngine, Compaction,
>> Flush, HLog, etc) and keep coprocessors more simple. This was plugins
>>would
>> be the supported way to extend HBase, and the interfaces would be more
>> stable.
>>
>>
>> On Mon, Feb 10, 2014 at 2:23 PM, Enis Söztutar <en...@gmail.com>
>>wrote:
>>
>> > Awesome work Jeffrey.
>> >
>> > Should we name the branch 4.0, instead of 4.0.0?
>> >
>> > It would be good to have a list of methods used by Phoenix to be
>> > identified and tagged in HBase so that at least future changes are not
>> > completely random. We might start with annotation
>> > InterfaceAudience.LimitedPrivate, so that HBase devs will be more
>> careful.
>> > However, we should also try to make Phoenix not so far-reaching into
>> HBase
>> > internals as well.
>> >
>> > Enis
>> >
>> >
>> > On Mon, Feb 10, 2014 at 2:16 PM, Andrew Purtell <apurtell@apache.org
>> >wrote:
>> >
>> >> On Mon, Feb 10, 2014 at 2:07 PM, James Taylor <jamestaylor@apache.org
>> >> >wrote:
>> >>
>> >> > Mujtaba - would it be
>> >> > feasible to setup Jenkins builds for this branch as well?
>> >> >
>> >>
>> >> Or, HBase 0.98 is really close to HBase trunk still. If you set up a
>>job
>> >> building against HBase trunk snapshots (and HBase remembers to
>>publish
>> >> those...) then Jenkins can catch any incompatibilities accidentally
>> >> introduced going forward.
>> >>
>> >> --
>> >> Best regards,
>> >>
>> >> - Andy
>> >>
>> >> Problems worthy of attack prove their worth by hitting back. - Piet
>>Hein
>> >> (via Tom White)
>> >>
>> >
>> >
>>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Branch 4.0.0 is created for Phoenix on HBase0.98
Posted by James Taylor <ja...@apache.org>.
+1. That's a very good idea, Enis - for HBase 1.0 if possible.
On Mon, Feb 10, 2014 at 2:28 PM, Enis Söztutar <en...@gmail.com> wrote:
> Forgot to mention, there was some discussion about filter / coprocessors
> versus plugins. It seems that coprocessors are doing both more user-facing
> database trigger kind of functionality + some plugging into Hbase internals
> functionality. I think longer term, we should define different interfaces
> for some pluggable functionality (see recent StorageEngine, Compaction,
> Flush, HLog, etc) and keep coprocessors more simple. This was plugins would
> be the supported way to extend HBase, and the interfaces would be more
> stable.
>
>
> On Mon, Feb 10, 2014 at 2:23 PM, Enis Söztutar <en...@gmail.com> wrote:
>
> > Awesome work Jeffrey.
> >
> > Should we name the branch 4.0, instead of 4.0.0?
> >
> > It would be good to have a list of methods used by Phoenix to be
> > identified and tagged in HBase so that at least future changes are not
> > completely random. We might start with annotation
> > InterfaceAudience.LimitedPrivate, so that HBase devs will be more
> careful.
> > However, we should also try to make Phoenix not so far-reaching into
> HBase
> > internals as well.
> >
> > Enis
> >
> >
> > On Mon, Feb 10, 2014 at 2:16 PM, Andrew Purtell <apurtell@apache.org
> >wrote:
> >
> >> On Mon, Feb 10, 2014 at 2:07 PM, James Taylor <jamestaylor@apache.org
> >> >wrote:
> >>
> >> > Mujtaba - would it be
> >> > feasible to setup Jenkins builds for this branch as well?
> >> >
> >>
> >> Or, HBase 0.98 is really close to HBase trunk still. If you set up a job
> >> building against HBase trunk snapshots (and HBase remembers to publish
> >> those...) then Jenkins can catch any incompatibilities accidentally
> >> introduced going forward.
> >>
> >> --
> >> Best regards,
> >>
> >> - Andy
> >>
> >> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> >> (via Tom White)
> >>
> >
> >
>
Re: Branch 4.0.0 is created for Phoenix on HBase0.98
Posted by Enis Söztutar <en...@gmail.com>.
Forgot to mention, there was some discussion about filter / coprocessors
versus plugins. It seems that coprocessors are doing both more user-facing
database trigger kind of functionality + some plugging into Hbase internals
functionality. I think longer term, we should define different interfaces
for some pluggable functionality (see recent StorageEngine, Compaction,
Flush, HLog, etc) and keep coprocessors more simple. This was plugins would
be the supported way to extend HBase, and the interfaces would be more
stable.
On Mon, Feb 10, 2014 at 2:23 PM, Enis Söztutar <en...@gmail.com> wrote:
> Awesome work Jeffrey.
>
> Should we name the branch 4.0, instead of 4.0.0?
>
> It would be good to have a list of methods used by Phoenix to be
> identified and tagged in HBase so that at least future changes are not
> completely random. We might start with annotation
> InterfaceAudience.LimitedPrivate, so that HBase devs will be more careful.
> However, we should also try to make Phoenix not so far-reaching into HBase
> internals as well.
>
> Enis
>
>
> On Mon, Feb 10, 2014 at 2:16 PM, Andrew Purtell <ap...@apache.org>wrote:
>
>> On Mon, Feb 10, 2014 at 2:07 PM, James Taylor <jamestaylor@apache.org
>> >wrote:
>>
>> > Mujtaba - would it be
>> > feasible to setup Jenkins builds for this branch as well?
>> >
>>
>> Or, HBase 0.98 is really close to HBase trunk still. If you set up a job
>> building against HBase trunk snapshots (and HBase remembers to publish
>> those...) then Jenkins can catch any incompatibilities accidentally
>> introduced going forward.
>>
>> --
>> Best regards,
>>
>> - Andy
>>
>> Problems worthy of attack prove their worth by hitting back. - Piet Hein
>> (via Tom White)
>>
>
>
Re: Branch 4.0.0 is created for Phoenix on HBase0.98
Posted by Enis Söztutar <en...@gmail.com>.
Awesome work Jeffrey.
Should we name the branch 4.0, instead of 4.0.0?
It would be good to have a list of methods used by Phoenix to be identified
and tagged in HBase so that at least future changes are not completely
random. We might start with annotation InterfaceAudience.LimitedPrivate, so
that HBase devs will be more careful. However, we should also try to make
Phoenix not so far-reaching into HBase internals as well.
Enis
On Mon, Feb 10, 2014 at 2:16 PM, Andrew Purtell <ap...@apache.org> wrote:
> On Mon, Feb 10, 2014 at 2:07 PM, James Taylor <jamestaylor@apache.org
> >wrote:
>
> > Mujtaba - would it be
> > feasible to setup Jenkins builds for this branch as well?
> >
>
> Or, HBase 0.98 is really close to HBase trunk still. If you set up a job
> building against HBase trunk snapshots (and HBase remembers to publish
> those...) then Jenkins can catch any incompatibilities accidentally
> introduced going forward.
>
> --
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>
Re: Branch 4.0.0 is created for Phoenix on HBase0.98
Posted by Andrew Purtell <ap...@apache.org>.
On Mon, Feb 10, 2014 at 2:07 PM, James Taylor <ja...@apache.org>wrote:
> Mujtaba - would it be
> feasible to setup Jenkins builds for this branch as well?
>
Or, HBase 0.98 is really close to HBase trunk still. If you set up a job
building against HBase trunk snapshots (and HBase remembers to publish
those...) then Jenkins can catch any incompatibilities accidentally
introduced going forward.
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: Branch 4.0.0 is created for Phoenix on HBase0.98
Posted by Jeffrey Zhong <jz...@hortonworks.com>.
All tests were run in both cases?
Yes, both cases have same test runs in the result.
On 2/11/14 10:42 AM, "Nick Dimiduk" <nd...@gmail.com> wrote:
>On Tue, Feb 11, 2014 at 11:09 AM, Jeffrey Zhong
><jz...@hortonworks.com>wrote:
>
>>
>> That's really great! The build box is powerful and the same tests took
>>me
>> 1 hr+ on my local dev box.
>>
>
>That sounds suspicious. All tests were run in both cases?
>
> On 2/11/14 9:39 AM, "Mujtaba Chohan" <mu...@apache.org> wrote:
>>
>> >Jenkins build for 4.0 branch is up:
>> >https://builds.apache.org/view/H-L/view/Phoenix/job/Phoenix-4.0/
>> >
>> >//mujtaba
>> >
>> >
>> >On Mon, Feb 10, 2014 at 2:07 PM, James Taylor
>> ><ja...@apache.org>wrote:
>> >
>> >> Awesome job, Jeffrey! This is pretty exciting! Mujtaba - would it be
>> >> feasible to setup Jenkins builds for this branch as well?
>> >>
>> >> Thanks,
>> >>
>> >> James
>> >>
>> >>
>> >> On Mon, Feb 10, 2014 at 1:38 PM, Jeffrey Zhong
>><jzhong@hortonworks.com
>> >> >wrote:
>> >>
>> >> > Hey,
>> >> >
>> >> > I created branch 4.0.0 from master branch for Phoenix on HBase0.98.
>> >>The
>> >> > code can work on tip of HBase0.98 & trunk branch but not
>> >>HBase0.96(only
>> >> > need trivial changes to make it work though) because some "private"
>> >>Hbase
>> >> > interface changes from release to release. We need a good way to
>> >>remove
>> >> > all private interface dependencies or convince Hbase folks to make
>> >>those
>> >> > interface backward compatible.
>> >> >
>> >> > We thought about to use a shim layer to load either Hbase0.94 or
>> >> Hbase0.98
>> >> > dependencies upon different hbase deployment so that we can put
>>this
>> >>code
>> >> > in master branch. The difficulty is that Phoenix isn't a normal
>>Hbase
>> >> > application. It extensively uses Hbase internals: Coprocessors,
>>WAL,
>> >> > Filters, KeyValue etc which have most incompatible changes between
>> >>0.94 &
>> >> > 0.96+. In addition, if more & more people are moving to the modern
>> >>Hbase
>> >> > code, it's not worth to put significant effort to create such a
>>shim
>> >> layer.
>> >> >
>> >> > So far all unit tests are passed in 4.0.0 branch. I'll do one more
>> >>merge
>> >> > to get checkins between 2/8 and today. Moving forward please merge
>> >>your
>> >> > changes to 4.0.0 in order for it to be used in hbase96+(the bright
>> >> > future). During merge, please don't use deprecated APIs because
>>those
>> >> APIs
>> >> > mostly internally create a new copy(e.g. Mutation#getFamilyMap) and
>> >>cause
>> >> > unexpected behaviors if you're thinking to change internal state
>>of an
>> >> > object.
>> >> >
>> >> > Thanks,
>> >> > -Jeffrey
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > CONFIDENTIALITY NOTICE
>> >> > NOTICE: This message is intended for the use of the individual or
>> >>entity
>> >> to
>> >> > which it is addressed and may contain information that is
>> >>confidential,
>> >> > privileged and exempt from disclosure under applicable law. If the
>> >>reader
>> >> > of this message is not the intended recipient, you are hereby
>>notified
>> >> that
>> >> > any printing, copying, dissemination, distribution, disclosure or
>> >> > forwarding of this communication is strictly prohibited. If you
>>have
>> >> > received this communication in error, please contact the sender
>> >> immediately
>> >> > and delete it from your system. Thank You.
>> >> >
>> >>
>>
>>
>>
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or
>>entity to
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the
>>reader
>> of this message is not the intended recipient, you are hereby notified
>>that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender
>>immediately
>> and delete it from your system. Thank You.
>>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Branch 4.0.0 is created for Phoenix on HBase0.98
Posted by Nick Dimiduk <nd...@gmail.com>.
On Tue, Feb 11, 2014 at 11:09 AM, Jeffrey Zhong <jz...@hortonworks.com>wrote:
>
> That's really great! The build box is powerful and the same tests took me
> 1 hr+ on my local dev box.
>
That sounds suspicious. All tests were run in both cases?
On 2/11/14 9:39 AM, "Mujtaba Chohan" <mu...@apache.org> wrote:
>
> >Jenkins build for 4.0 branch is up:
> >https://builds.apache.org/view/H-L/view/Phoenix/job/Phoenix-4.0/
> >
> >//mujtaba
> >
> >
> >On Mon, Feb 10, 2014 at 2:07 PM, James Taylor
> ><ja...@apache.org>wrote:
> >
> >> Awesome job, Jeffrey! This is pretty exciting! Mujtaba - would it be
> >> feasible to setup Jenkins builds for this branch as well?
> >>
> >> Thanks,
> >>
> >> James
> >>
> >>
> >> On Mon, Feb 10, 2014 at 1:38 PM, Jeffrey Zhong <jzhong@hortonworks.com
> >> >wrote:
> >>
> >> > Hey,
> >> >
> >> > I created branch 4.0.0 from master branch for Phoenix on HBase0.98.
> >>The
> >> > code can work on tip of HBase0.98 & trunk branch but not
> >>HBase0.96(only
> >> > need trivial changes to make it work though) because some "private"
> >>Hbase
> >> > interface changes from release to release. We need a good way to
> >>remove
> >> > all private interface dependencies or convince Hbase folks to make
> >>those
> >> > interface backward compatible.
> >> >
> >> > We thought about to use a shim layer to load either Hbase0.94 or
> >> Hbase0.98
> >> > dependencies upon different hbase deployment so that we can put this
> >>code
> >> > in master branch. The difficulty is that Phoenix isn't a normal Hbase
> >> > application. It extensively uses Hbase internals: Coprocessors, WAL,
> >> > Filters, KeyValue etc which have most incompatible changes between
> >>0.94 &
> >> > 0.96+. In addition, if more & more people are moving to the modern
> >>Hbase
> >> > code, it's not worth to put significant effort to create such a shim
> >> layer.
> >> >
> >> > So far all unit tests are passed in 4.0.0 branch. I'll do one more
> >>merge
> >> > to get checkins between 2/8 and today. Moving forward please merge
> >>your
> >> > changes to 4.0.0 in order for it to be used in hbase96+(the bright
> >> > future). During merge, please don't use deprecated APIs because those
> >> APIs
> >> > mostly internally create a new copy(e.g. Mutation#getFamilyMap) and
> >>cause
> >> > unexpected behaviors if you're thinking to change internal state of an
> >> > object.
> >> >
> >> > Thanks,
> >> > -Jeffrey
> >> >
> >> >
> >> >
> >> > --
> >> > CONFIDENTIALITY NOTICE
> >> > NOTICE: This message is intended for the use of the individual or
> >>entity
> >> to
> >> > which it is addressed and may contain information that is
> >>confidential,
> >> > privileged and exempt from disclosure under applicable law. If the
> >>reader
> >> > of this message is not the intended recipient, you are hereby notified
> >> that
> >> > any printing, copying, dissemination, distribution, disclosure or
> >> > forwarding of this communication is strictly prohibited. If you have
> >> > received this communication in error, please contact the sender
> >> immediately
> >> > and delete it from your system. Thank You.
> >> >
> >>
>
>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
Re: Branch 4.0.0 is created for Phoenix on HBase0.98
Posted by Jeffrey Zhong <jz...@hortonworks.com>.
That's really great! The build box is powerful and the same tests took me
1 hr+ on my local dev box.
I saw the following test runs and not sure if we can use them as a
preliminary perf comparison. But master branch seems quite slow
Branch4.0 took 15 mins run 1675 tests
Branch2.2.3 took 18 mins run 1444 tests
Master branch took 37 mins run 1700 tests
On 2/11/14 9:39 AM, "Mujtaba Chohan" <mu...@apache.org> wrote:
>Jenkins build for 4.0 branch is up:
>https://builds.apache.org/view/H-L/view/Phoenix/job/Phoenix-4.0/
>
>//mujtaba
>
>
>On Mon, Feb 10, 2014 at 2:07 PM, James Taylor
><ja...@apache.org>wrote:
>
>> Awesome job, Jeffrey! This is pretty exciting! Mujtaba - would it be
>> feasible to setup Jenkins builds for this branch as well?
>>
>> Thanks,
>>
>> James
>>
>>
>> On Mon, Feb 10, 2014 at 1:38 PM, Jeffrey Zhong <jzhong@hortonworks.com
>> >wrote:
>>
>> > Hey,
>> >
>> > I created branch 4.0.0 from master branch for Phoenix on HBase0.98.
>>The
>> > code can work on tip of HBase0.98 & trunk branch but not
>>HBase0.96(only
>> > need trivial changes to make it work though) because some "private"
>>Hbase
>> > interface changes from release to release. We need a good way to
>>remove
>> > all private interface dependencies or convince Hbase folks to make
>>those
>> > interface backward compatible.
>> >
>> > We thought about to use a shim layer to load either Hbase0.94 or
>> Hbase0.98
>> > dependencies upon different hbase deployment so that we can put this
>>code
>> > in master branch. The difficulty is that Phoenix isn't a normal Hbase
>> > application. It extensively uses Hbase internals: Coprocessors, WAL,
>> > Filters, KeyValue etc which have most incompatible changes between
>>0.94 &
>> > 0.96+. In addition, if more & more people are moving to the modern
>>Hbase
>> > code, it's not worth to put significant effort to create such a shim
>> layer.
>> >
>> > So far all unit tests are passed in 4.0.0 branch. I'll do one more
>>merge
>> > to get checkins between 2/8 and today. Moving forward please merge
>>your
>> > changes to 4.0.0 in order for it to be used in hbase96+(the bright
>> > future). During merge, please don't use deprecated APIs because those
>> APIs
>> > mostly internally create a new copy(e.g. Mutation#getFamilyMap) and
>>cause
>> > unexpected behaviors if you're thinking to change internal state of an
>> > object.
>> >
>> > Thanks,
>> > -Jeffrey
>> >
>> >
>> >
>> > --
>> > CONFIDENTIALITY NOTICE
>> > NOTICE: This message is intended for the use of the individual or
>>entity
>> to
>> > which it is addressed and may contain information that is
>>confidential,
>> > privileged and exempt from disclosure under applicable law. If the
>>reader
>> > of this message is not the intended recipient, you are hereby notified
>> that
>> > any printing, copying, dissemination, distribution, disclosure or
>> > forwarding of this communication is strictly prohibited. If you have
>> > received this communication in error, please contact the sender
>> immediately
>> > and delete it from your system. Thank You.
>> >
>>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Branch 4.0.0 is created for Phoenix on HBase0.98
Posted by Mujtaba Chohan <mu...@apache.org>.
Jenkins build for 4.0 branch is up:
https://builds.apache.org/view/H-L/view/Phoenix/job/Phoenix-4.0/
//mujtaba
On Mon, Feb 10, 2014 at 2:07 PM, James Taylor <ja...@apache.org>wrote:
> Awesome job, Jeffrey! This is pretty exciting! Mujtaba - would it be
> feasible to setup Jenkins builds for this branch as well?
>
> Thanks,
>
> James
>
>
> On Mon, Feb 10, 2014 at 1:38 PM, Jeffrey Zhong <jzhong@hortonworks.com
> >wrote:
>
> > Hey,
> >
> > I created branch 4.0.0 from master branch for Phoenix on HBase0.98. The
> > code can work on tip of HBase0.98 & trunk branch but not HBase0.96(only
> > need trivial changes to make it work though) because some "private" Hbase
> > interface changes from release to release. We need a good way to remove
> > all private interface dependencies or convince Hbase folks to make those
> > interface backward compatible.
> >
> > We thought about to use a shim layer to load either Hbase0.94 or
> Hbase0.98
> > dependencies upon different hbase deployment so that we can put this code
> > in master branch. The difficulty is that Phoenix isn't a normal Hbase
> > application. It extensively uses Hbase internals: Coprocessors, WAL,
> > Filters, KeyValue etc which have most incompatible changes between 0.94 &
> > 0.96+. In addition, if more & more people are moving to the modern Hbase
> > code, it's not worth to put significant effort to create such a shim
> layer.
> >
> > So far all unit tests are passed in 4.0.0 branch. I'll do one more merge
> > to get checkins between 2/8 and today. Moving forward please merge your
> > changes to 4.0.0 in order for it to be used in hbase96+(the bright
> > future). During merge, please don't use deprecated APIs because those
> APIs
> > mostly internally create a new copy(e.g. Mutation#getFamilyMap) and cause
> > unexpected behaviors if you're thinking to change internal state of an
> > object.
> >
> > Thanks,
> > -Jeffrey
> >
> >
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
> >
>
Re: Branch 4.0.0 is created for Phoenix on HBase0.98
Posted by James Taylor <ja...@apache.org>.
Awesome job, Jeffrey! This is pretty exciting! Mujtaba - would it be
feasible to setup Jenkins builds for this branch as well?
Thanks,
James
On Mon, Feb 10, 2014 at 1:38 PM, Jeffrey Zhong <jz...@hortonworks.com>wrote:
> Hey,
>
> I created branch 4.0.0 from master branch for Phoenix on HBase0.98. The
> code can work on tip of HBase0.98 & trunk branch but not HBase0.96(only
> need trivial changes to make it work though) because some "private" Hbase
> interface changes from release to release. We need a good way to remove
> all private interface dependencies or convince Hbase folks to make those
> interface backward compatible.
>
> We thought about to use a shim layer to load either Hbase0.94 or Hbase0.98
> dependencies upon different hbase deployment so that we can put this code
> in master branch. The difficulty is that Phoenix isn't a normal Hbase
> application. It extensively uses Hbase internals: Coprocessors, WAL,
> Filters, KeyValue etc which have most incompatible changes between 0.94 &
> 0.96+. In addition, if more & more people are moving to the modern Hbase
> code, it's not worth to put significant effort to create such a shim layer.
>
> So far all unit tests are passed in 4.0.0 branch. I'll do one more merge
> to get checkins between 2/8 and today. Moving forward please merge your
> changes to 4.0.0 in order for it to be used in hbase96+(the bright
> future). During merge, please don't use deprecated APIs because those APIs
> mostly internally create a new copy(e.g. Mutation#getFamilyMap) and cause
> unexpected behaviors if you're thinking to change internal state of an
> object.
>
> Thanks,
> -Jeffrey
>
>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>