You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Stack <st...@duboce.net> on 2014/02/10 18:24:26 UTC
Re: DISCUSSION: 1.0.0
Suggestions for 1.0.0 if if it is to come out in next month or so:
+ Update included libs (e.g. move to log4j2)
+ Enable distributed log replay as default (fix bugs)
+ Enable hfilev3 as default.
+ Ship with default logging level set to INFO and content of the logs still
makes sense
What else?
+ Enable dynamic config and schema by default.
St.Ack
On Mon, Jan 20, 2014 at 4:09 PM, Stack <st...@duboce.net> wrote:
> On Mon, Jan 20, 2014 at 1:52 PM, lars hofhansl <la...@apache.org> wrote:
>
>> I'm happy to volunteer. Happy if Enis does it, too.
>>
>>
> I'd be happy to do it too but my thinking is that it is good to spread the
> role around.
> St.Ack
>
>
>
>
>> ------------------------------
>> *From:* Stack <st...@duboce.net>
>> *To:* HBase Dev List <de...@hbase.apache.org>
>> *Cc:* lars hofhansl <la...@apache.org>
>> *Sent:* Monday, January 20, 2014 1:43 PM
>> *Subject:* Re: DISCUSSION: 1.0.0
>>
>> On Thu, Jan 2, 2014 at 1:20 PM, Enis Söztutar <en...@apache.org> wrote:
>>
>> I think whether we will need a new RM will depend on the decision to
>> release 1.0 from 0.98 branches or 0.99 branches(current trunk).
>>
>>
>> I think it should have an RM regardless. We should probably try to put a
>> higher polish on a 1.0 than we would mayhaps on a lesser release. RM will
>> have enough work on their plate just keeping up state (IMO).
>>
>>
>>
>> We can do the previous practice of releasing 0.99.0, then turning 0.99.x
>> as
>> the 1.0.0. In that case, I can also volunteer as well.
>>
>>
>> Good by me. Anyone else interested in the job? Speak up if so.
>>
>> If not, you'd get it by default Enis. Else you and whoever will have to
>> dook it out.
>>
>> St.Ack
>>
>>
>>
>
Re: DISCUSSION: 1.0.0
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
wow. Wondering how I totally skipped this line :( Sorry.
So please disregard my initial email.
Thanks,
JM
2014-02-10 13:52 GMT-05:00 Stack <st...@duboce.net>:
> On Mon, Feb 10, 2014 at 9:31 AM, Jean-Marc Spaggiari <
> jean-marc@spaggiari.org> wrote:
>
> > What's about reducing the default log level?
> >
> >
> Is the above different from '...Ship with default logging level set to
> INFO'?
> Thanks JMS,
> St.Ack
>
>
>
> >
> > 2014-02-10 12:24 GMT-05:00 Stack <st...@duboce.net>:
> >
> > > Suggestions for 1.0.0 if if it is to come out in next month or so:
> > >
> > > + Update included libs (e.g. move to log4j2)
> > >
> > > + Enable distributed log replay as default (fix bugs)
> > > + Enable hfilev3 as default.
> > > + Ship with default logging level set to INFO and content of the logs
> > still
> > > makes sense
> > >
> > > What else?
> > >
> > > + Enable dynamic config and schema by default.
> > >
> > >
> > > St.Ack
> > >
> > >
> > >
> > >
> > > On Mon, Jan 20, 2014 at 4:09 PM, Stack <st...@duboce.net> wrote:
> > >
> > > > On Mon, Jan 20, 2014 at 1:52 PM, lars hofhansl <la...@apache.org>
> > wrote:
> > > >
> > > >> I'm happy to volunteer. Happy if Enis does it, too.
> > > >>
> > > >>
> > > > I'd be happy to do it too but my thinking is that it is good to
> spread
> > > the
> > > > role around.
> > > > St.Ack
> > > >
> > > >
> > > >
> > > >
> > > >> ------------------------------
> > > >> *From:* Stack <st...@duboce.net>
> > > >> *To:* HBase Dev List <de...@hbase.apache.org>
> > > >> *Cc:* lars hofhansl <la...@apache.org>
> > > >> *Sent:* Monday, January 20, 2014 1:43 PM
> > > >> *Subject:* Re: DISCUSSION: 1.0.0
> > > >>
> > > >> On Thu, Jan 2, 2014 at 1:20 PM, Enis Söztutar <en...@apache.org>
> > wrote:
> > > >>
> > > >> I think whether we will need a new RM will depend on the decision to
> > > >> release 1.0 from 0.98 branches or 0.99 branches(current trunk).
> > > >>
> > > >>
> > > >> I think it should have an RM regardless. We should probably try to
> > put
> > > a
> > > >> higher polish on a 1.0 than we would mayhaps on a lesser release.
> RM
> > > will
> > > >> have enough work on their plate just keeping up state (IMO).
> > > >>
> > > >>
> > > >>
> > > >> We can do the previous practice of releasing 0.99.0, then turning
> > 0.99.x
> > > >> as
> > > >> the 1.0.0. In that case, I can also volunteer as well.
> > > >>
> > > >>
> > > >> Good by me. Anyone else interested in the job? Speak up if so.
> > > >>
> > > >> If not, you'd get it by default Enis. Else you and whoever will
> have
> > to
> > > >> dook it out.
> > > >>
> > > >> St.Ack
> > > >>
> > > >>
> > > >>
> > > >
> > >
> >
>
Re: DISCUSSION: 1.0.0
Posted by Stack <st...@duboce.net>.
On Mon, Feb 10, 2014 at 9:31 AM, Jean-Marc Spaggiari <
jean-marc@spaggiari.org> wrote:
> What's about reducing the default log level?
>
>
Is the above different from '...Ship with default logging level set to
INFO'?
Thanks JMS,
St.Ack
>
> 2014-02-10 12:24 GMT-05:00 Stack <st...@duboce.net>:
>
> > Suggestions for 1.0.0 if if it is to come out in next month or so:
> >
> > + Update included libs (e.g. move to log4j2)
> >
> > + Enable distributed log replay as default (fix bugs)
> > + Enable hfilev3 as default.
> > + Ship with default logging level set to INFO and content of the logs
> still
> > makes sense
> >
> > What else?
> >
> > + Enable dynamic config and schema by default.
> >
> >
> > St.Ack
> >
> >
> >
> >
> > On Mon, Jan 20, 2014 at 4:09 PM, Stack <st...@duboce.net> wrote:
> >
> > > On Mon, Jan 20, 2014 at 1:52 PM, lars hofhansl <la...@apache.org>
> wrote:
> > >
> > >> I'm happy to volunteer. Happy if Enis does it, too.
> > >>
> > >>
> > > I'd be happy to do it too but my thinking is that it is good to spread
> > the
> > > role around.
> > > St.Ack
> > >
> > >
> > >
> > >
> > >> ------------------------------
> > >> *From:* Stack <st...@duboce.net>
> > >> *To:* HBase Dev List <de...@hbase.apache.org>
> > >> *Cc:* lars hofhansl <la...@apache.org>
> > >> *Sent:* Monday, January 20, 2014 1:43 PM
> > >> *Subject:* Re: DISCUSSION: 1.0.0
> > >>
> > >> On Thu, Jan 2, 2014 at 1:20 PM, Enis Söztutar <en...@apache.org>
> wrote:
> > >>
> > >> I think whether we will need a new RM will depend on the decision to
> > >> release 1.0 from 0.98 branches or 0.99 branches(current trunk).
> > >>
> > >>
> > >> I think it should have an RM regardless. We should probably try to
> put
> > a
> > >> higher polish on a 1.0 than we would mayhaps on a lesser release. RM
> > will
> > >> have enough work on their plate just keeping up state (IMO).
> > >>
> > >>
> > >>
> > >> We can do the previous practice of releasing 0.99.0, then turning
> 0.99.x
> > >> as
> > >> the 1.0.0. In that case, I can also volunteer as well.
> > >>
> > >>
> > >> Good by me. Anyone else interested in the job? Speak up if so.
> > >>
> > >> If not, you'd get it by default Enis. Else you and whoever will have
> to
> > >> dook it out.
> > >>
> > >> St.Ack
> > >>
> > >>
> > >>
> > >
> >
>
Re: DISCUSSION: 1.0.0
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
What's about reducing the default log level?
2014-02-10 12:24 GMT-05:00 Stack <st...@duboce.net>:
> Suggestions for 1.0.0 if if it is to come out in next month or so:
>
> + Update included libs (e.g. move to log4j2)
>
> + Enable distributed log replay as default (fix bugs)
> + Enable hfilev3 as default.
> + Ship with default logging level set to INFO and content of the logs still
> makes sense
>
> What else?
>
> + Enable dynamic config and schema by default.
>
>
> St.Ack
>
>
>
>
> On Mon, Jan 20, 2014 at 4:09 PM, Stack <st...@duboce.net> wrote:
>
> > On Mon, Jan 20, 2014 at 1:52 PM, lars hofhansl <la...@apache.org> wrote:
> >
> >> I'm happy to volunteer. Happy if Enis does it, too.
> >>
> >>
> > I'd be happy to do it too but my thinking is that it is good to spread
> the
> > role around.
> > St.Ack
> >
> >
> >
> >
> >> ------------------------------
> >> *From:* Stack <st...@duboce.net>
> >> *To:* HBase Dev List <de...@hbase.apache.org>
> >> *Cc:* lars hofhansl <la...@apache.org>
> >> *Sent:* Monday, January 20, 2014 1:43 PM
> >> *Subject:* Re: DISCUSSION: 1.0.0
> >>
> >> On Thu, Jan 2, 2014 at 1:20 PM, Enis Söztutar <en...@apache.org> wrote:
> >>
> >> I think whether we will need a new RM will depend on the decision to
> >> release 1.0 from 0.98 branches or 0.99 branches(current trunk).
> >>
> >>
> >> I think it should have an RM regardless. We should probably try to put
> a
> >> higher polish on a 1.0 than we would mayhaps on a lesser release. RM
> will
> >> have enough work on their plate just keeping up state (IMO).
> >>
> >>
> >>
> >> We can do the previous practice of releasing 0.99.0, then turning 0.99.x
> >> as
> >> the 1.0.0. In that case, I can also volunteer as well.
> >>
> >>
> >> Good by me. Anyone else interested in the job? Speak up if so.
> >>
> >> If not, you'd get it by default Enis. Else you and whoever will have to
> >> dook it out.
> >>
> >> St.Ack
> >>
> >>
> >>
> >
>
Re: DISCUSSION: 1.0.0
Posted by Ted Yu <yu...@gmail.com>.
It is desirable if HBASE-9203, the umbrella JIRA for secondary indexing, is
given some reviews.
Cheers
On Mon, Feb 10, 2014 at 5:41 PM, Jeffrey Zhong <jz...@hortonworks.com>wrote:
>
> + HBASE-8763 Combine MVCC and SeqId
> Currently replication is broken on same version updates in the following
> scenarios: when a region move or RS get restarted in the source cluster,
> replicated changes may come out of order to the receiving RS. There are
> other cases we need to keep the ordering of puts. This JIRA can be used to
> fix those above issues.
>
> + Replicated Meta Regions for Distributed Region Location Lookup(I haven't
> cut a JIRA for it but I've been thinking this for a while)
>
>
> On 2/10/14 4:42 PM, "lars hofhansl" <la...@apache.org> wrote:
>
> >+ a clearly defined compatibility strategy between patch, minor, and
> >major versions
> >+ known coprocessor interfaces. For example right now a coprocessor can
> >use everything on HRegion. We need to distill what's useful into an
> >interface.
> >+ (wishes for ponies and unicorns, probably not going to happen) better
> >ops supports, such as sample puppet/chef/etc scripts
> >
> >
> >
> >________________________________
> > From: Stack <st...@duboce.net>
> >To: HBase Dev List <de...@hbase.apache.org>
> >Sent: Monday, February 10, 2014 4:29 PM
> >Subject: Re: DISCUSSION: 1.0.0
> >
> >
> >Keep them coming and I'll file issues against 1.0 version.
> >Thanks lads,
> >St.Ack
> >
> >
> >
> >On Mon, Feb 10, 2014 at 11:38 AM, Enis Söztutar <en...@gmail.com>
> >wrote:
> >
> >> + fix interface audience annotations (HBASE-10462).
> >> + Continue on limiting some broad reaching interfaces (HCM, HCI, HTable)
> >> See recent HBASE-10479 discussions)
> >> + Promote HTableInterface vs HTable, getting connections from HCM and
> >> getting tables there.
> >>
> >>
> >>
> >> On Mon, Feb 10, 2014 at 11:20 AM, Stack <st...@duboce.net> wrote:
> >>
> >> > On Mon, Feb 10, 2014 at 11:15 AM, Steve Loughran
> >><stevel@hortonworks.com
> >> > >wrote:
> >> >
> >> > > On 10 February 2014 09:24, Stack <st...@duboce.net> wrote:
> >> > >
> >> > > > Suggestions for 1.0.0 if if it is to come out in next month or so:
> >> > > >
> >> > > > + Update included libs (e.g. move to log4j2)
> >> > > >
> >> > >
> >> > > love to see how that goes on -it looks good but I've not seen it in
> >> > > production yet. I do note it has a NoSQL outputter, so there are
> >>good
> >> > > opportunities for recursion
> >> > >
> >> > >
> >> > Smile.
> >> >
>
>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
Re: DISCUSSION: 1.0.0
Posted by Jeffrey Zhong <jz...@hortonworks.com>.
+ HBASE-8763 Combine MVCC and SeqId
Currently replication is broken on same version updates in the following
scenarios: when a region move or RS get restarted in the source cluster,
replicated changes may come out of order to the receiving RS. There are
other cases we need to keep the ordering of puts. This JIRA can be used to
fix those above issues.
+ Replicated Meta Regions for Distributed Region Location Lookup(I haven't
cut a JIRA for it but I've been thinking this for a while)
On 2/10/14 4:42 PM, "lars hofhansl" <la...@apache.org> wrote:
>+ a clearly defined compatibility strategy between patch, minor, and
>major versions
>+ known coprocessor interfaces. For example right now a coprocessor can
>use everything on HRegion. We need to distill what's useful into an
>interface.
>+ (wishes for ponies and unicorns, probably not going to happen) better
>ops supports, such as sample puppet/chef/etc scripts
>
>
>
>________________________________
> From: Stack <st...@duboce.net>
>To: HBase Dev List <de...@hbase.apache.org>
>Sent: Monday, February 10, 2014 4:29 PM
>Subject: Re: DISCUSSION: 1.0.0
>
>
>Keep them coming and I'll file issues against 1.0 version.
>Thanks lads,
>St.Ack
>
>
>
>On Mon, Feb 10, 2014 at 11:38 AM, Enis Söztutar <en...@gmail.com>
>wrote:
>
>> + fix interface audience annotations (HBASE-10462).
>> + Continue on limiting some broad reaching interfaces (HCM, HCI, HTable)
>> See recent HBASE-10479 discussions)
>> + Promote HTableInterface vs HTable, getting connections from HCM and
>> getting tables there.
>>
>>
>>
>> On Mon, Feb 10, 2014 at 11:20 AM, Stack <st...@duboce.net> wrote:
>>
>> > On Mon, Feb 10, 2014 at 11:15 AM, Steve Loughran
>><stevel@hortonworks.com
>> > >wrote:
>> >
>> > > On 10 February 2014 09:24, Stack <st...@duboce.net> wrote:
>> > >
>> > > > Suggestions for 1.0.0 if if it is to come out in next month or so:
>> > > >
>> > > > + Update included libs (e.g. move to log4j2)
>> > > >
>> > >
>> > > love to see how that goes on -it looks good but I've not seen it in
>> > > production yet. I do note it has a NoSQL outputter, so there are
>>good
>> > > opportunities for recursion
>> > >
>> > >
>> > Smile.
>> >
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: DISCUSSION: 1.0.0
Posted by Enis Söztutar <en...@gmail.com>.
Thanks Ted.
It makes sense to fix it before 1.0 if we can.
Enis
On Sun, Mar 2, 2014 at 8:54 PM, Ted Yu <yu...@gmail.com> wrote:
> The following thread brought attention to HBASE-9206 'namespace
> permissions'
> :
>
>
> http://search-hadoop.com/m/DHED49RMDk/enable%252Fdisable+table+permission&subj=enable+disable+table+permission
>
> This would help user implement multi-tenancy in a shared cluster.
>
> Cheers
>
>
> On Tue, Feb 11, 2014 at 12:43 PM, Jonathan Hsieh <jo...@cloudera.com> wrote:
>
> > Lily's indexer uses hbase private apis in the replication portion of
> code.
> > we may want to expose and make promises on the replication api/wire to
> > make their lives easier and to enable cross version replication for
> future
> > hbases.
> >
> > Jon.
> >
> >
> > On Tue, Feb 11, 2014 at 11:21 AM, Nick Dimiduk <nd...@gmail.com>
> wrote:
> >
> > > FYI, Lily's Indexer [0] also makes use of this level of integration
> into
> > > HBase. Others?
> > >
> > > [0]: https://github.com/NGDATA/hbase-indexer
> > >
> > >
> > > On Tue, Feb 11, 2014 at 12:09 PM, Enis Söztutar <en...@gmail.com>
> > > wrote:
> > >
> > > > This is good discussion,
> > > >
> > > > I mentioned this in phoenix-dev that we need to repurpose
> coprocessors,
> > > and
> > > > just give them put / get kind of injection points, but not allow
> > > > co-processors to tap into log / compaction, etc. Those will be better
> > > > served by explicitly defined plugins (like the recent StorageEngine,
> > > HLog,
> > > > etc) that we explicitly allow injecting code with some API stability
> > > > (though probably still evolving at a fast pace).
> > > >
> > > > It would be good to have an idea of what classes / methods, phoenix
> and
> > > > similar depend on HBase right now.
> > > >
> > > > Enis
> > > >
> > > >
> > > > On Mon, Feb 10, 2014 at 6:04 PM, Jeffrey Zhong <
> jzhong@hortonworks.com
> > > > >wrote:
> > > >
> > > > >
> > > > > We need to revisit which interfaces should be marked as real
> > "private"
> > > > > because coprocessors allow custom applications to accesses internal
> > > > > states. Current private interface such as KeyValue, WALEdit,
> Hregion,
> > > > > HLogKey, Store etc is may used by custom coprocessor applications
> so
> > > any
> > > > > public method signature change in those "private" interfaces
> > possibliy
> > > > > breaks custom coprocessor implementations.
> > > > >
> > > > > -Jeffrey
> > > > >
> > > > > On 2/10/14 4:59 PM, "lars hofhansl" <la...@apache.org> wrote:
> > > > >
> > > > > >True. But we can clearly annotate what is public and what is not.
> > > > > >In HRegion we'd have to do it per method. It's probably easier to
> > > > extract
> > > > > >the public interface out into a Interface. (similar to Store and
> > > HStore)
> > > > > >HBASE-4207 solves a slightly different problem. Here I want to
> have
> > > > > >access to some HBase internals for performance, but I want to know
> > > what
> > > > > >is public and stable and what I should not touch (thinking about
> > > things
> > > > > >like Phoenix).
> > > > > >
> > > > > >
> > > > > >-- Lars
> > > > > >
> > > > > >
> > > > > >
> > > > > >________________________________
> > > > > > From: Andrew Purtell <ap...@apache.org>
> > > > > >To: "dev@hbase.apache.org" <de...@hbase.apache.org>
> > > > > >Sent: Monday, February 10, 2014 4:46 PM
> > > > > >Subject: Re: DISCUSSION: 1.0.0
> > > > > >
> > > > > >
> > > > > >On Mon, Feb 10, 2014 at 4:42 PM, lars hofhansl <la...@apache.org>
> > > > wrote:
> > > > > >
> > > > > >
> > > > > >> + known coprocessor interfaces. For example right now a
> > coprocessor
> > > > can
> > > > > >> use everything on HRegion. We need to distill what's useful into
> > an
> > > > > >> interface.
> > > > > >
> > > > > >
> > > > > >We started this with Environment and the HTable wrapper as an
> > example
> > > of
> > > > > >how to wrap an interface for CPs. At the end of the day we can't
> > stop
> > > a
> > > > > >coprocessor from accessing internal objects and calling their
> > methods
> > > > > >directly until HBASE-4047. (Maybe that makes the 1.0 list?)
> > > > > >
> > > > > >Related, paring down the coprocessor interfaces to only intercept
> > RPC
> > > > > >based
> > > > > >actions and move everything else to plugins.
> > > > > >
> > > > > >
> > > > > >
> > > > > >--
> > > > > >Best regards,
> > > > > >
> > > > > > - Andy
> > > > > >
> > > > > >Problems worthy of attack prove their worth by hitting back. -
> Piet
> > > Hein
> > > > > >(via Tom White)
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > CONFIDENTIALITY NOTICE
> > > > > NOTICE: This message is intended for the use of the individual or
> > > entity
> > > > to
> > > > > which it is addressed and may contain information that is
> > confidential,
> > > > > privileged and exempt from disclosure under applicable law. If the
> > > reader
> > > > > of this message is not the intended recipient, you are hereby
> > notified
> > > > that
> > > > > any printing, copying, dissemination, distribution, disclosure or
> > > > > forwarding of this communication is strictly prohibited. If you
> have
> > > > > received this communication in error, please contact the sender
> > > > immediately
> > > > > and delete it from your system. Thank You.
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > // Jonathan Hsieh (shay)
> > // HBase Tech Lead, Software Engineer, Cloudera
> > // jon@cloudera.com // @jmhsieh
> >
>
Re: DISCUSSION: 1.0.0
Posted by Ted Yu <yu...@gmail.com>.
The following thread brought attention to HBASE-9206 'namespace permissions'
:
http://search-hadoop.com/m/DHED49RMDk/enable%252Fdisable+table+permission&subj=enable+disable+table+permission
This would help user implement multi-tenancy in a shared cluster.
Cheers
On Tue, Feb 11, 2014 at 12:43 PM, Jonathan Hsieh <jo...@cloudera.com> wrote:
> Lily's indexer uses hbase private apis in the replication portion of code.
> we may want to expose and make promises on the replication api/wire to
> make their lives easier and to enable cross version replication for future
> hbases.
>
> Jon.
>
>
> On Tue, Feb 11, 2014 at 11:21 AM, Nick Dimiduk <nd...@gmail.com> wrote:
>
> > FYI, Lily's Indexer [0] also makes use of this level of integration into
> > HBase. Others?
> >
> > [0]: https://github.com/NGDATA/hbase-indexer
> >
> >
> > On Tue, Feb 11, 2014 at 12:09 PM, Enis Söztutar <en...@gmail.com>
> > wrote:
> >
> > > This is good discussion,
> > >
> > > I mentioned this in phoenix-dev that we need to repurpose coprocessors,
> > and
> > > just give them put / get kind of injection points, but not allow
> > > co-processors to tap into log / compaction, etc. Those will be better
> > > served by explicitly defined plugins (like the recent StorageEngine,
> > HLog,
> > > etc) that we explicitly allow injecting code with some API stability
> > > (though probably still evolving at a fast pace).
> > >
> > > It would be good to have an idea of what classes / methods, phoenix and
> > > similar depend on HBase right now.
> > >
> > > Enis
> > >
> > >
> > > On Mon, Feb 10, 2014 at 6:04 PM, Jeffrey Zhong <jzhong@hortonworks.com
> > > >wrote:
> > >
> > > >
> > > > We need to revisit which interfaces should be marked as real
> "private"
> > > > because coprocessors allow custom applications to accesses internal
> > > > states. Current private interface such as KeyValue, WALEdit, Hregion,
> > > > HLogKey, Store etc is may used by custom coprocessor applications so
> > any
> > > > public method signature change in those "private" interfaces
> possibliy
> > > > breaks custom coprocessor implementations.
> > > >
> > > > -Jeffrey
> > > >
> > > > On 2/10/14 4:59 PM, "lars hofhansl" <la...@apache.org> wrote:
> > > >
> > > > >True. But we can clearly annotate what is public and what is not.
> > > > >In HRegion we'd have to do it per method. It's probably easier to
> > > extract
> > > > >the public interface out into a Interface. (similar to Store and
> > HStore)
> > > > >HBASE-4207 solves a slightly different problem. Here I want to have
> > > > >access to some HBase internals for performance, but I want to know
> > what
> > > > >is public and stable and what I should not touch (thinking about
> > things
> > > > >like Phoenix).
> > > > >
> > > > >
> > > > >-- Lars
> > > > >
> > > > >
> > > > >
> > > > >________________________________
> > > > > From: Andrew Purtell <ap...@apache.org>
> > > > >To: "dev@hbase.apache.org" <de...@hbase.apache.org>
> > > > >Sent: Monday, February 10, 2014 4:46 PM
> > > > >Subject: Re: DISCUSSION: 1.0.0
> > > > >
> > > > >
> > > > >On Mon, Feb 10, 2014 at 4:42 PM, lars hofhansl <la...@apache.org>
> > > wrote:
> > > > >
> > > > >
> > > > >> + known coprocessor interfaces. For example right now a
> coprocessor
> > > can
> > > > >> use everything on HRegion. We need to distill what's useful into
> an
> > > > >> interface.
> > > > >
> > > > >
> > > > >We started this with Environment and the HTable wrapper as an
> example
> > of
> > > > >how to wrap an interface for CPs. At the end of the day we can't
> stop
> > a
> > > > >coprocessor from accessing internal objects and calling their
> methods
> > > > >directly until HBASE-4047. (Maybe that makes the 1.0 list?)
> > > > >
> > > > >Related, paring down the coprocessor interfaces to only intercept
> RPC
> > > > >based
> > > > >actions and move everything else to plugins.
> > > > >
> > > > >
> > > > >
> > > > >--
> > > > >Best regards,
> > > > >
> > > > > - Andy
> > > > >
> > > > >Problems worthy of attack prove their worth by hitting back. - Piet
> > Hein
> > > > >(via Tom White)
> > > >
> > > >
> > > >
> > > > --
> > > > CONFIDENTIALITY NOTICE
> > > > NOTICE: This message is intended for the use of the individual or
> > entity
> > > to
> > > > which it is addressed and may contain information that is
> confidential,
> > > > privileged and exempt from disclosure under applicable law. If the
> > reader
> > > > of this message is not the intended recipient, you are hereby
> notified
> > > that
> > > > any printing, copying, dissemination, distribution, disclosure or
> > > > forwarding of this communication is strictly prohibited. If you have
> > > > received this communication in error, please contact the sender
> > > immediately
> > > > and delete it from your system. Thank You.
> > > >
> > >
> >
>
>
>
> --
> // Jonathan Hsieh (shay)
> // HBase Tech Lead, Software Engineer, Cloudera
> // jon@cloudera.com // @jmhsieh
>
Re: DISCUSSION: 1.0.0
Posted by Jonathan Hsieh <jo...@cloudera.com>.
Lily's indexer uses hbase private apis in the replication portion of code.
we may want to expose and make promises on the replication api/wire to
make their lives easier and to enable cross version replication for future
hbases.
Jon.
On Tue, Feb 11, 2014 at 11:21 AM, Nick Dimiduk <nd...@gmail.com> wrote:
> FYI, Lily's Indexer [0] also makes use of this level of integration into
> HBase. Others?
>
> [0]: https://github.com/NGDATA/hbase-indexer
>
>
> On Tue, Feb 11, 2014 at 12:09 PM, Enis Söztutar <en...@gmail.com>
> wrote:
>
> > This is good discussion,
> >
> > I mentioned this in phoenix-dev that we need to repurpose coprocessors,
> and
> > just give them put / get kind of injection points, but not allow
> > co-processors to tap into log / compaction, etc. Those will be better
> > served by explicitly defined plugins (like the recent StorageEngine,
> HLog,
> > etc) that we explicitly allow injecting code with some API stability
> > (though probably still evolving at a fast pace).
> >
> > It would be good to have an idea of what classes / methods, phoenix and
> > similar depend on HBase right now.
> >
> > Enis
> >
> >
> > On Mon, Feb 10, 2014 at 6:04 PM, Jeffrey Zhong <jzhong@hortonworks.com
> > >wrote:
> >
> > >
> > > We need to revisit which interfaces should be marked as real "private"
> > > because coprocessors allow custom applications to accesses internal
> > > states. Current private interface such as KeyValue, WALEdit, Hregion,
> > > HLogKey, Store etc is may used by custom coprocessor applications so
> any
> > > public method signature change in those "private" interfaces possibliy
> > > breaks custom coprocessor implementations.
> > >
> > > -Jeffrey
> > >
> > > On 2/10/14 4:59 PM, "lars hofhansl" <la...@apache.org> wrote:
> > >
> > > >True. But we can clearly annotate what is public and what is not.
> > > >In HRegion we'd have to do it per method. It's probably easier to
> > extract
> > > >the public interface out into a Interface. (similar to Store and
> HStore)
> > > >HBASE-4207 solves a slightly different problem. Here I want to have
> > > >access to some HBase internals for performance, but I want to know
> what
> > > >is public and stable and what I should not touch (thinking about
> things
> > > >like Phoenix).
> > > >
> > > >
> > > >-- Lars
> > > >
> > > >
> > > >
> > > >________________________________
> > > > From: Andrew Purtell <ap...@apache.org>
> > > >To: "dev@hbase.apache.org" <de...@hbase.apache.org>
> > > >Sent: Monday, February 10, 2014 4:46 PM
> > > >Subject: Re: DISCUSSION: 1.0.0
> > > >
> > > >
> > > >On Mon, Feb 10, 2014 at 4:42 PM, lars hofhansl <la...@apache.org>
> > wrote:
> > > >
> > > >
> > > >> + known coprocessor interfaces. For example right now a coprocessor
> > can
> > > >> use everything on HRegion. We need to distill what's useful into an
> > > >> interface.
> > > >
> > > >
> > > >We started this with Environment and the HTable wrapper as an example
> of
> > > >how to wrap an interface for CPs. At the end of the day we can't stop
> a
> > > >coprocessor from accessing internal objects and calling their methods
> > > >directly until HBASE-4047. (Maybe that makes the 1.0 list?)
> > > >
> > > >Related, paring down the coprocessor interfaces to only intercept RPC
> > > >based
> > > >actions and move everything else to plugins.
> > > >
> > > >
> > > >
> > > >--
> > > >Best regards,
> > > >
> > > > - Andy
> > > >
> > > >Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > > >(via Tom White)
> > >
> > >
> > >
> > > --
> > > CONFIDENTIALITY NOTICE
> > > NOTICE: This message is intended for the use of the individual or
> entity
> > to
> > > which it is addressed and may contain information that is confidential,
> > > privileged and exempt from disclosure under applicable law. If the
> reader
> > > of this message is not the intended recipient, you are hereby notified
> > that
> > > any printing, copying, dissemination, distribution, disclosure or
> > > forwarding of this communication is strictly prohibited. If you have
> > > received this communication in error, please contact the sender
> > immediately
> > > and delete it from your system. Thank You.
> > >
> >
>
--
// Jonathan Hsieh (shay)
// HBase Tech Lead, Software Engineer, Cloudera
// jon@cloudera.com // @jmhsieh
Re: DISCUSSION: 1.0.0
Posted by Nick Dimiduk <nd...@gmail.com>.
On Tue, Feb 11, 2014 at 1:46 PM, Stack <st...@duboce.net> wrote:
> A few us have gotten the "We need a Replication Interface defined" along
> another channel.
>
I've set HBASE-9507 to 0.99.0. Is there a better way to mark it (and these
1.0 issues in general) for further discussion?
On Tue, Feb 11, 2014 at 11:21 AM, Nick Dimiduk <nd...@gmail.com> wrote:
>
> > FYI, Lily's Indexer [0] also makes use of this level of integration into
> > HBase. Others?
> >
> > [0]: https://github.com/NGDATA/hbase-indexer
> >
> >
> > On Tue, Feb 11, 2014 at 12:09 PM, Enis Söztutar <en...@gmail.com>
> > wrote:
> >
> > > This is good discussion,
> > >
> > > I mentioned this in phoenix-dev that we need to repurpose coprocessors,
> > and
> > > just give them put / get kind of injection points, but not allow
> > > co-processors to tap into log / compaction, etc. Those will be better
> > > served by explicitly defined plugins (like the recent StorageEngine,
> > HLog,
> > > etc) that we explicitly allow injecting code with some API stability
> > > (though probably still evolving at a fast pace).
> > >
> > > It would be good to have an idea of what classes / methods, phoenix and
> > > similar depend on HBase right now.
> > >
> > > Enis
> > >
> > >
> > > On Mon, Feb 10, 2014 at 6:04 PM, Jeffrey Zhong <jzhong@hortonworks.com
> > > >wrote:
> > >
> > > >
> > > > We need to revisit which interfaces should be marked as real
> "private"
> > > > because coprocessors allow custom applications to accesses internal
> > > > states. Current private interface such as KeyValue, WALEdit, Hregion,
> > > > HLogKey, Store etc is may used by custom coprocessor applications so
> > any
> > > > public method signature change in those "private" interfaces
> possibliy
> > > > breaks custom coprocessor implementations.
> > > >
> > > > -Jeffrey
> > > >
> > > > On 2/10/14 4:59 PM, "lars hofhansl" <la...@apache.org> wrote:
> > > >
> > > > >True. But we can clearly annotate what is public and what is not.
> > > > >In HRegion we'd have to do it per method. It's probably easier to
> > > extract
> > > > >the public interface out into a Interface. (similar to Store and
> > HStore)
> > > > >HBASE-4207 solves a slightly different problem. Here I want to have
> > > > >access to some HBase internals for performance, but I want to know
> > what
> > > > >is public and stable and what I should not touch (thinking about
> > things
> > > > >like Phoenix).
> > > > >
> > > > >
> > > > >-- Lars
> > > > >
> > > > >
> > > > >
> > > > >________________________________
> > > > > From: Andrew Purtell <ap...@apache.org>
> > > > >To: "dev@hbase.apache.org" <de...@hbase.apache.org>
> > > > >Sent: Monday, February 10, 2014 4:46 PM
> > > > >Subject: Re: DISCUSSION: 1.0.0
> > > > >
> > > > >
> > > > >On Mon, Feb 10, 2014 at 4:42 PM, lars hofhansl <la...@apache.org>
> > > wrote:
> > > > >
> > > > >
> > > > >> + known coprocessor interfaces. For example right now a
> coprocessor
> > > can
> > > > >> use everything on HRegion. We need to distill what's useful into
> an
> > > > >> interface.
> > > > >
> > > > >
> > > > >We started this with Environment and the HTable wrapper as an
> example
> > of
> > > > >how to wrap an interface for CPs. At the end of the day we can't
> stop
> > a
> > > > >coprocessor from accessing internal objects and calling their
> methods
> > > > >directly until HBASE-4047. (Maybe that makes the 1.0 list?)
> > > > >
> > > > >Related, paring down the coprocessor interfaces to only intercept
> RPC
> > > > >based
> > > > >actions and move everything else to plugins.
> > > > >
> > > > >
> > > > >
> > > > >--
> > > > >Best regards,
> > > > >
> > > > > - Andy
> > > > >
> > > > >Problems worthy of attack prove their worth by hitting back. - Piet
> > Hein
> > > > >(via Tom White)
> > > >
> > > >
> > > >
> > > > --
> > > > CONFIDENTIALITY NOTICE
> > > > NOTICE: This message is intended for the use of the individual or
> > entity
> > > to
> > > > which it is addressed and may contain information that is
> confidential,
> > > > privileged and exempt from disclosure under applicable law. If the
> > reader
> > > > of this message is not the intended recipient, you are hereby
> notified
> > > that
> > > > any printing, copying, dissemination, distribution, disclosure or
> > > > forwarding of this communication is strictly prohibited. If you have
> > > > received this communication in error, please contact the sender
> > > immediately
> > > > and delete it from your system. Thank You.
> > > >
> > >
> >
>
Re: DISCUSSION: 1.0.0
Posted by Stack <st...@duboce.net>.
On Tue, Feb 11, 2014 at 2:40 PM, Enis Söztutar <en...@gmail.com> wrote:
> >
> >
> > I think it'd be great if you did it Enis. It looks like the old-guard
> RMs
> > are up for helping out too (going by comments above).
> >
>
> Since this is 1.0, we would need help from everybody!
>
>
Amen.
>
> >
> > Do we have to vote on it? Unless someone else wants to do it (speak
> up!),
> > I'd think not.
> >
>
> Yeah, I was not sure whether we should call a vote for it. If any there are
> other volunteers, I can also do RM duty for a 1.1 or so release. Otherwise
> consider myself the "official" RM :)
>
>
Let me put it up on its own thread at least....
St.Ack
Re: DISCUSSION: 1.0.0
Posted by Enis Söztutar <en...@gmail.com>.
>
>
> I think it'd be great if you did it Enis. It looks like the old-guard RMs
> are up for helping out too (going by comments above).
>
Since this is 1.0, we would need help from everybody!
>
> Do we have to vote on it? Unless someone else wants to do it (speak up!),
> I'd think not.
>
Yeah, I was not sure whether we should call a vote for it. If any there are
other volunteers, I can also do RM duty for a 1.1 or so release. Otherwise
consider myself the "official" RM :)
>
> Yeah, would be sweet if it arrived in time for hbasecon. It'd be good to
> hit our 3/4 month cadence two releases in a row. 0.99 dev release sounds
> right too.
>
> Good stuff,
> St.Ack
>
Re: DISCUSSION: 1.0.0
Posted by Stack <st...@duboce.net>.
On Tue, Feb 11, 2014 at 1:06 PM, Enis Söztutar <en...@gmail.com> wrote:
> > (Sigh....If only we had a RM for 1.0.0...)
>
> Thanks Stack for bringing this up. I actually want to volunteer as an RM
> for 1.0 release. We can do a 0.99 release first as a dev release, then turn
> 0.99.x into 1.0 as we did in 0.95 -> 0.96 releases.
>
> As for timing, we can start by filing some jiras for the discussion items
> and setting the target versions for those to see where we are at. 0.98 will
> be out soon, and we want to make 1.0 sooner rather than later. Also if we
> can nail the release before HBaseCon that would be good with users.
>
I think it'd be great if you did it Enis. It looks like the old-guard RMs
are up for helping out too (going by comments above).
Do we have to vote on it? Unless someone else wants to do it (speak up!),
I'd think not.
Yeah, would be sweet if it arrived in time for hbasecon. It'd be good to
hit our 3/4 month cadence two releases in a row. 0.99 dev release sounds
right too.
Good stuff,
St.Ack
Re: DISCUSSION: 1.0.0
Posted by Enis Söztutar <en...@gmail.com>.
> (Sigh....If only we had a RM for 1.0.0...)
Thanks Stack for bringing this up. I actually want to volunteer as an RM
for 1.0 release. We can do a 0.99 release first as a dev release, then turn
0.99.x into 1.0 as we did in 0.95 -> 0.96 releases.
As for timing, we can start by filing some jiras for the discussion items
and setting the target versions for those to see where we are at. 0.98 will
be out soon, and we want to make 1.0 sooner rather than later. Also if we
can nail the release before HBaseCon that would be good with users.
Enis
On Tue, Feb 11, 2014 at 12:46 PM, Stack <st...@duboce.net> wrote:
> A few us have gotten the "We need a Replication Interface defined" along
> another channel.
>
> Agree this is a good discussion. I'll file issues to catch the above
> against the 0.99 label.
>
> (Sigh....If only we had a RM for 1.0.0...)
>
> St.Ack
>
>
> On Tue, Feb 11, 2014 at 11:21 AM, Nick Dimiduk <nd...@gmail.com> wrote:
>
> > FYI, Lily's Indexer [0] also makes use of this level of integration into
> > HBase. Others?
> >
> > [0]: https://github.com/NGDATA/hbase-indexer
> >
> >
> > On Tue, Feb 11, 2014 at 12:09 PM, Enis Söztutar <en...@gmail.com>
> > wrote:
> >
> > > This is good discussion,
> > >
> > > I mentioned this in phoenix-dev that we need to repurpose coprocessors,
> > and
> > > just give them put / get kind of injection points, but not allow
> > > co-processors to tap into log / compaction, etc. Those will be better
> > > served by explicitly defined plugins (like the recent StorageEngine,
> > HLog,
> > > etc) that we explicitly allow injecting code with some API stability
> > > (though probably still evolving at a fast pace).
> > >
> > > It would be good to have an idea of what classes / methods, phoenix and
> > > similar depend on HBase right now.
> > >
> > > Enis
> > >
> > >
> > > On Mon, Feb 10, 2014 at 6:04 PM, Jeffrey Zhong <jzhong@hortonworks.com
> > > >wrote:
> > >
> > > >
> > > > We need to revisit which interfaces should be marked as real
> "private"
> > > > because coprocessors allow custom applications to accesses internal
> > > > states. Current private interface such as KeyValue, WALEdit, Hregion,
> > > > HLogKey, Store etc is may used by custom coprocessor applications so
> > any
> > > > public method signature change in those "private" interfaces
> possibliy
> > > > breaks custom coprocessor implementations.
> > > >
> > > > -Jeffrey
> > > >
> > > > On 2/10/14 4:59 PM, "lars hofhansl" <la...@apache.org> wrote:
> > > >
> > > > >True. But we can clearly annotate what is public and what is not.
> > > > >In HRegion we'd have to do it per method. It's probably easier to
> > > extract
> > > > >the public interface out into a Interface. (similar to Store and
> > HStore)
> > > > >HBASE-4207 solves a slightly different problem. Here I want to have
> > > > >access to some HBase internals for performance, but I want to know
> > what
> > > > >is public and stable and what I should not touch (thinking about
> > things
> > > > >like Phoenix).
> > > > >
> > > > >
> > > > >-- Lars
> > > > >
> > > > >
> > > > >
> > > > >________________________________
> > > > > From: Andrew Purtell <ap...@apache.org>
> > > > >To: "dev@hbase.apache.org" <de...@hbase.apache.org>
> > > > >Sent: Monday, February 10, 2014 4:46 PM
> > > > >Subject: Re: DISCUSSION: 1.0.0
> > > > >
> > > > >
> > > > >On Mon, Feb 10, 2014 at 4:42 PM, lars hofhansl <la...@apache.org>
> > > wrote:
> > > > >
> > > > >
> > > > >> + known coprocessor interfaces. For example right now a
> coprocessor
> > > can
> > > > >> use everything on HRegion. We need to distill what's useful into
> an
> > > > >> interface.
> > > > >
> > > > >
> > > > >We started this with Environment and the HTable wrapper as an
> example
> > of
> > > > >how to wrap an interface for CPs. At the end of the day we can't
> stop
> > a
> > > > >coprocessor from accessing internal objects and calling their
> methods
> > > > >directly until HBASE-4047. (Maybe that makes the 1.0 list?)
> > > > >
> > > > >Related, paring down the coprocessor interfaces to only intercept
> RPC
> > > > >based
> > > > >actions and move everything else to plugins.
> > > > >
> > > > >
> > > > >
> > > > >--
> > > > >Best regards,
> > > > >
> > > > > - Andy
> > > > >
> > > > >Problems worthy of attack prove their worth by hitting back. - Piet
> > Hein
> > > > >(via Tom White)
> > > >
> > > >
> > > >
> > > > --
> > > > CONFIDENTIALITY NOTICE
> > > > NOTICE: This message is intended for the use of the individual or
> > entity
> > > to
> > > > which it is addressed and may contain information that is
> confidential,
> > > > privileged and exempt from disclosure under applicable law. If the
> > reader
> > > > of this message is not the intended recipient, you are hereby
> notified
> > > that
> > > > any printing, copying, dissemination, distribution, disclosure or
> > > > forwarding of this communication is strictly prohibited. If you have
> > > > received this communication in error, please contact the sender
> > > immediately
> > > > and delete it from your system. Thank You.
> > > >
> > >
> >
>
Re: DISCUSSION: 1.0.0
Posted by Stack <st...@duboce.net>.
A few us have gotten the "We need a Replication Interface defined" along
another channel.
Agree this is a good discussion. I'll file issues to catch the above
against the 0.99 label.
(Sigh....If only we had a RM for 1.0.0...)
St.Ack
On Tue, Feb 11, 2014 at 11:21 AM, Nick Dimiduk <nd...@gmail.com> wrote:
> FYI, Lily's Indexer [0] also makes use of this level of integration into
> HBase. Others?
>
> [0]: https://github.com/NGDATA/hbase-indexer
>
>
> On Tue, Feb 11, 2014 at 12:09 PM, Enis Söztutar <en...@gmail.com>
> wrote:
>
> > This is good discussion,
> >
> > I mentioned this in phoenix-dev that we need to repurpose coprocessors,
> and
> > just give them put / get kind of injection points, but not allow
> > co-processors to tap into log / compaction, etc. Those will be better
> > served by explicitly defined plugins (like the recent StorageEngine,
> HLog,
> > etc) that we explicitly allow injecting code with some API stability
> > (though probably still evolving at a fast pace).
> >
> > It would be good to have an idea of what classes / methods, phoenix and
> > similar depend on HBase right now.
> >
> > Enis
> >
> >
> > On Mon, Feb 10, 2014 at 6:04 PM, Jeffrey Zhong <jzhong@hortonworks.com
> > >wrote:
> >
> > >
> > > We need to revisit which interfaces should be marked as real "private"
> > > because coprocessors allow custom applications to accesses internal
> > > states. Current private interface such as KeyValue, WALEdit, Hregion,
> > > HLogKey, Store etc is may used by custom coprocessor applications so
> any
> > > public method signature change in those "private" interfaces possibliy
> > > breaks custom coprocessor implementations.
> > >
> > > -Jeffrey
> > >
> > > On 2/10/14 4:59 PM, "lars hofhansl" <la...@apache.org> wrote:
> > >
> > > >True. But we can clearly annotate what is public and what is not.
> > > >In HRegion we'd have to do it per method. It's probably easier to
> > extract
> > > >the public interface out into a Interface. (similar to Store and
> HStore)
> > > >HBASE-4207 solves a slightly different problem. Here I want to have
> > > >access to some HBase internals for performance, but I want to know
> what
> > > >is public and stable and what I should not touch (thinking about
> things
> > > >like Phoenix).
> > > >
> > > >
> > > >-- Lars
> > > >
> > > >
> > > >
> > > >________________________________
> > > > From: Andrew Purtell <ap...@apache.org>
> > > >To: "dev@hbase.apache.org" <de...@hbase.apache.org>
> > > >Sent: Monday, February 10, 2014 4:46 PM
> > > >Subject: Re: DISCUSSION: 1.0.0
> > > >
> > > >
> > > >On Mon, Feb 10, 2014 at 4:42 PM, lars hofhansl <la...@apache.org>
> > wrote:
> > > >
> > > >
> > > >> + known coprocessor interfaces. For example right now a coprocessor
> > can
> > > >> use everything on HRegion. We need to distill what's useful into an
> > > >> interface.
> > > >
> > > >
> > > >We started this with Environment and the HTable wrapper as an example
> of
> > > >how to wrap an interface for CPs. At the end of the day we can't stop
> a
> > > >coprocessor from accessing internal objects and calling their methods
> > > >directly until HBASE-4047. (Maybe that makes the 1.0 list?)
> > > >
> > > >Related, paring down the coprocessor interfaces to only intercept RPC
> > > >based
> > > >actions and move everything else to plugins.
> > > >
> > > >
> > > >
> > > >--
> > > >Best regards,
> > > >
> > > > - Andy
> > > >
> > > >Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > > >(via Tom White)
> > >
> > >
> > >
> > > --
> > > CONFIDENTIALITY NOTICE
> > > NOTICE: This message is intended for the use of the individual or
> entity
> > to
> > > which it is addressed and may contain information that is confidential,
> > > privileged and exempt from disclosure under applicable law. If the
> reader
> > > of this message is not the intended recipient, you are hereby notified
> > that
> > > any printing, copying, dissemination, distribution, disclosure or
> > > forwarding of this communication is strictly prohibited. If you have
> > > received this communication in error, please contact the sender
> > immediately
> > > and delete it from your system. Thank You.
> > >
> >
>
Re: DISCUSSION: 1.0.0
Posted by Nick Dimiduk <nd...@gmail.com>.
FYI, Lily's Indexer [0] also makes use of this level of integration into
HBase. Others?
[0]: https://github.com/NGDATA/hbase-indexer
On Tue, Feb 11, 2014 at 12:09 PM, Enis Söztutar <en...@gmail.com> wrote:
> This is good discussion,
>
> I mentioned this in phoenix-dev that we need to repurpose coprocessors, and
> just give them put / get kind of injection points, but not allow
> co-processors to tap into log / compaction, etc. Those will be better
> served by explicitly defined plugins (like the recent StorageEngine, HLog,
> etc) that we explicitly allow injecting code with some API stability
> (though probably still evolving at a fast pace).
>
> It would be good to have an idea of what classes / methods, phoenix and
> similar depend on HBase right now.
>
> Enis
>
>
> On Mon, Feb 10, 2014 at 6:04 PM, Jeffrey Zhong <jzhong@hortonworks.com
> >wrote:
>
> >
> > We need to revisit which interfaces should be marked as real "private"
> > because coprocessors allow custom applications to accesses internal
> > states. Current private interface such as KeyValue, WALEdit, Hregion,
> > HLogKey, Store etc is may used by custom coprocessor applications so any
> > public method signature change in those "private" interfaces possibliy
> > breaks custom coprocessor implementations.
> >
> > -Jeffrey
> >
> > On 2/10/14 4:59 PM, "lars hofhansl" <la...@apache.org> wrote:
> >
> > >True. But we can clearly annotate what is public and what is not.
> > >In HRegion we'd have to do it per method. It's probably easier to
> extract
> > >the public interface out into a Interface. (similar to Store and HStore)
> > >HBASE-4207 solves a slightly different problem. Here I want to have
> > >access to some HBase internals for performance, but I want to know what
> > >is public and stable and what I should not touch (thinking about things
> > >like Phoenix).
> > >
> > >
> > >-- Lars
> > >
> > >
> > >
> > >________________________________
> > > From: Andrew Purtell <ap...@apache.org>
> > >To: "dev@hbase.apache.org" <de...@hbase.apache.org>
> > >Sent: Monday, February 10, 2014 4:46 PM
> > >Subject: Re: DISCUSSION: 1.0.0
> > >
> > >
> > >On Mon, Feb 10, 2014 at 4:42 PM, lars hofhansl <la...@apache.org>
> wrote:
> > >
> > >
> > >> + known coprocessor interfaces. For example right now a coprocessor
> can
> > >> use everything on HRegion. We need to distill what's useful into an
> > >> interface.
> > >
> > >
> > >We started this with Environment and the HTable wrapper as an example of
> > >how to wrap an interface for CPs. At the end of the day we can't stop a
> > >coprocessor from accessing internal objects and calling their methods
> > >directly until HBASE-4047. (Maybe that makes the 1.0 list?)
> > >
> > >Related, paring down the coprocessor interfaces to only intercept RPC
> > >based
> > >actions and move everything else to plugins.
> > >
> > >
> > >
> > >--
> > >Best regards,
> > >
> > > - Andy
> > >
> > >Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > >(via Tom White)
> >
> >
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
> >
>
Re: DISCUSSION: 1.0.0
Posted by Andrew Purtell <ap...@apache.org>.
This is no coincidence, Linux kernel modules (specifically the security
ones, and what enables them) were some direct inspiration. :-)
I may have said this before but I think stable interfaces for coprocessors
comes into play only once we are doing HBASE-4047, and even then I would
expect an eventing model where the vocabulary evolves from release to
release.
On Wed, Feb 12, 2014 at 1:22 PM, Todd Lipcon <to...@cloudera.com> wrote:
> On Wed, Feb 12, 2014 at 12:46 PM, Gary Helmling <gh...@gmail.com>
> wrote:
>
> > >
> > > 'Repurpose' might not be the way I would put it.
> > >
> > > Coprocessors were and are a means for internal server extension by
> mixin.
> > > The original problem we solved was needing to subclass HRegionServer
> and
> > > other classes to extend core HBase functions, but having more than one
> > > otherwise orthogonal extension that users want to use. Now we can mix
> in
> > > multiple extensions with a framework that has some simple rules for
> > > cooperation between the extensions.
> > >
> > > We return to the earlier state of affairs with modules. Sure, we can
> plug
> > > in an alternate behavior with a module that subclasses and extends the
> > > default, say flush strategy, but we can't then instantiate multiple
> > modules
> > > into the same slot, both subclassing the same base but doing different
> > > things.
> > >
> > >
> >
> > I agree the ability to compose coprocessors in order to extend behavior
> is
> > a key capability that we should not throw out.
> >
> > I think the current Observer APIs could probably do with a bit of
> > reorganization to make them a little more accessible and comprehensible.
> I
> > think there is also an emerging need to see if we can define some subset
> of
> > these APIs that we can stabilize for easier public consumption, while
> > keeping the rest of the APIs free to evolve as needed as HBase internals
> > change (since these are an extension mechanism for internal behaviors).
> > I'm not sure we've really seen enough commonality emerge yet to say what
> > those APIs are though. We could try to define the public subset as those
> > involved in client requests, but flush and compaction, for example, can
> > also be triggered by client requests. And my own use of coprocessor APIs
> > lately has been focused on overriding the flush and compaction behaviors,
> > not on client requests.
> >
> > I think the best place to start is by breaking up some of the current
> APIs,
> > grouping them around behaviors or areas of functionality. Whether we
> call
> > some of these "coprocessors" and others "plugins" is a question of
> > branding. I do think it's important to figure out which we can stabilize
> > and offer longer term contracts for. But whatever we call them, I
> strongly
> > agree that we should maintain the "mixin" / composition approach and not
> > return to a simple fixed inheritance scheme.
> >
>
> I've always considered coprocessors to be the "kernel modules" of the HBase
> world. They give you way more power than user-space programming, but come
> with the cautions that if you make a mistake, you'll crash your whole
> system or trigger unexpected behavior.
>
> Given that, I don't think we should really be spending too much effort on
> coprocessor API stability. If we make this a requirement, it can hamper the
> ability of the HBase core developers to make good changes which really
> improve the system. I don't think we're at the level of maturity as a
> project where this is the right tradeoff, as of yet.
>
> For what it's worth, the Linux kernel module API is also not
> stable/compatible between versions. This document is a good read:
>
> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/Documentation/stable_api_nonsense.txt
>
> I do think we should seek to keep the interfaces stable through *patch*
> level releases -- a bug fix shouldn't break a coprocessor API. But between
> minor releases that add new features, it seems like an unnecessary
> restriction.
>
> -Todd
> --
> Todd Lipcon
> Software Engineer, Cloudera
>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: DISCUSSION: 1.0.0
Posted by Todd Lipcon <to...@cloudera.com>.
On Wed, Feb 12, 2014 at 12:46 PM, Gary Helmling <gh...@gmail.com> wrote:
> >
> > 'Repurpose' might not be the way I would put it.
> >
> > Coprocessors were and are a means for internal server extension by mixin.
> > The original problem we solved was needing to subclass HRegionServer and
> > other classes to extend core HBase functions, but having more than one
> > otherwise orthogonal extension that users want to use. Now we can mix in
> > multiple extensions with a framework that has some simple rules for
> > cooperation between the extensions.
> >
> > We return to the earlier state of affairs with modules. Sure, we can plug
> > in an alternate behavior with a module that subclasses and extends the
> > default, say flush strategy, but we can't then instantiate multiple
> modules
> > into the same slot, both subclassing the same base but doing different
> > things.
> >
> >
>
> I agree the ability to compose coprocessors in order to extend behavior is
> a key capability that we should not throw out.
>
> I think the current Observer APIs could probably do with a bit of
> reorganization to make them a little more accessible and comprehensible. I
> think there is also an emerging need to see if we can define some subset of
> these APIs that we can stabilize for easier public consumption, while
> keeping the rest of the APIs free to evolve as needed as HBase internals
> change (since these are an extension mechanism for internal behaviors).
> I'm not sure we've really seen enough commonality emerge yet to say what
> those APIs are though. We could try to define the public subset as those
> involved in client requests, but flush and compaction, for example, can
> also be triggered by client requests. And my own use of coprocessor APIs
> lately has been focused on overriding the flush and compaction behaviors,
> not on client requests.
>
> I think the best place to start is by breaking up some of the current APIs,
> grouping them around behaviors or areas of functionality. Whether we call
> some of these "coprocessors" and others "plugins" is a question of
> branding. I do think it's important to figure out which we can stabilize
> and offer longer term contracts for. But whatever we call them, I strongly
> agree that we should maintain the "mixin" / composition approach and not
> return to a simple fixed inheritance scheme.
>
I've always considered coprocessors to be the "kernel modules" of the HBase
world. They give you way more power than user-space programming, but come
with the cautions that if you make a mistake, you'll crash your whole
system or trigger unexpected behavior.
Given that, I don't think we should really be spending too much effort on
coprocessor API stability. If we make this a requirement, it can hamper the
ability of the HBase core developers to make good changes which really
improve the system. I don't think we're at the level of maturity as a
project where this is the right tradeoff, as of yet.
For what it's worth, the Linux kernel module API is also not
stable/compatible between versions. This document is a good read:
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/Documentation/stable_api_nonsense.txt
I do think we should seek to keep the interfaces stable through *patch*
level releases -- a bug fix shouldn't break a coprocessor API. But between
minor releases that add new features, it seems like an unnecessary
restriction.
-Todd
--
Todd Lipcon
Software Engineer, Cloudera
Re: DISCUSSION: 1.0.0
Posted by Gary Helmling <gh...@gmail.com>.
>
> 'Repurpose' might not be the way I would put it.
>
> Coprocessors were and are a means for internal server extension by mixin.
> The original problem we solved was needing to subclass HRegionServer and
> other classes to extend core HBase functions, but having more than one
> otherwise orthogonal extension that users want to use. Now we can mix in
> multiple extensions with a framework that has some simple rules for
> cooperation between the extensions.
>
> We return to the earlier state of affairs with modules. Sure, we can plug
> in an alternate behavior with a module that subclasses and extends the
> default, say flush strategy, but we can't then instantiate multiple modules
> into the same slot, both subclassing the same base but doing different
> things.
>
>
I agree the ability to compose coprocessors in order to extend behavior is
a key capability that we should not throw out.
I think the current Observer APIs could probably do with a bit of
reorganization to make them a little more accessible and comprehensible. I
think there is also an emerging need to see if we can define some subset of
these APIs that we can stabilize for easier public consumption, while
keeping the rest of the APIs free to evolve as needed as HBase internals
change (since these are an extension mechanism for internal behaviors).
I'm not sure we've really seen enough commonality emerge yet to say what
those APIs are though. We could try to define the public subset as those
involved in client requests, but flush and compaction, for example, can
also be triggered by client requests. And my own use of coprocessor APIs
lately has been focused on overriding the flush and compaction behaviors,
not on client requests.
I think the best place to start is by breaking up some of the current APIs,
grouping them around behaviors or areas of functionality. Whether we call
some of these "coprocessors" and others "plugins" is a question of
branding. I do think it's important to figure out which we can stabilize
and offer longer term contracts for. But whatever we call them, I strongly
agree that we should maintain the "mixin" / composition approach and not
return to a simple fixed inheritance scheme.
Re: DISCUSSION: 1.0.0
Posted by Andrew Purtell <an...@gmail.com>.
'Repurpose' might not be the way I would put it.
Coprocessors were and are a means for internal server extension by mixin. The original problem we solved was needing to subclass HRegionServer and other classes to extend core HBase functions, but having more than one otherwise orthogonal extension that users want to use. Now we can mix in multiple extensions with a framework that has some simple rules for cooperation between the extensions.
We return to the earlier state of affairs with modules. Sure, we can plug in an alternate behavior with a module that subclasses and extends the default, say flush strategy, but we can't then instantiate multiple modules into the same slot, both subclassing the same base but doing different things.
> On Feb 11, 2014, at 11:09 AM, Enis Söztutar <en...@gmail.com> wrote:
>
> This is good discussion,
>
> I mentioned this in phoenix-dev that we need to repurpose coprocessors, and
> just give them put / get kind of injection points, but not allow
> co-processors to tap into log / compaction, etc. Those will be better
> served by explicitly defined plugins (like the recent StorageEngine, HLog,
> etc) that we explicitly allow injecting code with some API stability
> (though probably still evolving at a fast pace).
>
> It would be good to have an idea of what classes / methods, phoenix and
> similar depend on HBase right now.
>
> Enis
>
>
> On Mon, Feb 10, 2014 at 6:04 PM, Jeffrey Zhong <jz...@hortonworks.com>wrote:
>
>>
>> We need to revisit which interfaces should be marked as real "private"
>> because coprocessors allow custom applications to accesses internal
>> states. Current private interface such as KeyValue, WALEdit, Hregion,
>> HLogKey, Store etc is may used by custom coprocessor applications so any
>> public method signature change in those "private" interfaces possibliy
>> breaks custom coprocessor implementations.
>>
>> -Jeffrey
>>
>>> On 2/10/14 4:59 PM, "lars hofhansl" <la...@apache.org> wrote:
>>>
>>> True. But we can clearly annotate what is public and what is not.
>>> In HRegion we'd have to do it per method. It's probably easier to extract
>>> the public interface out into a Interface. (similar to Store and HStore)
>>> HBASE-4207 solves a slightly different problem. Here I want to have
>>> access to some HBase internals for performance, but I want to know what
>>> is public and stable and what I should not touch (thinking about things
>>> like Phoenix).
>>>
>>>
>>> -- Lars
>>>
>>>
>>>
>>> ________________________________
>>> From: Andrew Purtell <ap...@apache.org>
>>> To: "dev@hbase.apache.org" <de...@hbase.apache.org>
>>> Sent: Monday, February 10, 2014 4:46 PM
>>> Subject: Re: DISCUSSION: 1.0.0
>>>
>>>
>>> On Mon, Feb 10, 2014 at 4:42 PM, lars hofhansl <la...@apache.org> wrote:
>>>
>>>
>>>> + known coprocessor interfaces. For example right now a coprocessor can
>>>> use everything on HRegion. We need to distill what's useful into an
>>>> interface.
>>>
>>>
>>> We started this with Environment and the HTable wrapper as an example of
>>> how to wrap an interface for CPs. At the end of the day we can't stop a
>>> coprocessor from accessing internal objects and calling their methods
>>> directly until HBASE-4047. (Maybe that makes the 1.0 list?)
>>>
>>> Related, paring down the coprocessor interfaces to only intercept RPC
>>> based
>>> actions and move everything else to plugins.
>>>
>>>
>>>
>>> --
>>> Best regards,
>>>
>>> - Andy
>>>
>>> Problems worthy of attack prove their worth by hitting back. - Piet Hein
>>> (via Tom White)
>>
>>
>>
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity to
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>>
Re: DISCUSSION: 1.0.0
Posted by Enis Söztutar <en...@gmail.com>.
This is good discussion,
I mentioned this in phoenix-dev that we need to repurpose coprocessors, and
just give them put / get kind of injection points, but not allow
co-processors to tap into log / compaction, etc. Those will be better
served by explicitly defined plugins (like the recent StorageEngine, HLog,
etc) that we explicitly allow injecting code with some API stability
(though probably still evolving at a fast pace).
It would be good to have an idea of what classes / methods, phoenix and
similar depend on HBase right now.
Enis
On Mon, Feb 10, 2014 at 6:04 PM, Jeffrey Zhong <jz...@hortonworks.com>wrote:
>
> We need to revisit which interfaces should be marked as real "private"
> because coprocessors allow custom applications to accesses internal
> states. Current private interface such as KeyValue, WALEdit, Hregion,
> HLogKey, Store etc is may used by custom coprocessor applications so any
> public method signature change in those "private" interfaces possibliy
> breaks custom coprocessor implementations.
>
> -Jeffrey
>
> On 2/10/14 4:59 PM, "lars hofhansl" <la...@apache.org> wrote:
>
> >True. But we can clearly annotate what is public and what is not.
> >In HRegion we'd have to do it per method. It's probably easier to extract
> >the public interface out into a Interface. (similar to Store and HStore)
> >HBASE-4207 solves a slightly different problem. Here I want to have
> >access to some HBase internals for performance, but I want to know what
> >is public and stable and what I should not touch (thinking about things
> >like Phoenix).
> >
> >
> >-- Lars
> >
> >
> >
> >________________________________
> > From: Andrew Purtell <ap...@apache.org>
> >To: "dev@hbase.apache.org" <de...@hbase.apache.org>
> >Sent: Monday, February 10, 2014 4:46 PM
> >Subject: Re: DISCUSSION: 1.0.0
> >
> >
> >On Mon, Feb 10, 2014 at 4:42 PM, lars hofhansl <la...@apache.org> wrote:
> >
> >
> >> + known coprocessor interfaces. For example right now a coprocessor can
> >> use everything on HRegion. We need to distill what's useful into an
> >> interface.
> >
> >
> >We started this with Environment and the HTable wrapper as an example of
> >how to wrap an interface for CPs. At the end of the day we can't stop a
> >coprocessor from accessing internal objects and calling their methods
> >directly until HBASE-4047. (Maybe that makes the 1.0 list?)
> >
> >Related, paring down the coprocessor interfaces to only intercept RPC
> >based
> >actions and move everything else to plugins.
> >
> >
> >
> >--
> >Best regards,
> >
> > - Andy
> >
> >Problems worthy of attack prove their worth by hitting back. - Piet Hein
> >(via Tom White)
>
>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
Re: DISCUSSION: 1.0.0
Posted by Jeffrey Zhong <jz...@hortonworks.com>.
We need to revisit which interfaces should be marked as real "private"
because coprocessors allow custom applications to accesses internal
states. Current private interface such as KeyValue, WALEdit, Hregion,
HLogKey, Store etc is may used by custom coprocessor applications so any
public method signature change in those "private" interfaces possibliy
breaks custom coprocessor implementations.
-Jeffrey
On 2/10/14 4:59 PM, "lars hofhansl" <la...@apache.org> wrote:
>True. But we can clearly annotate what is public and what is not.
>In HRegion we'd have to do it per method. It's probably easier to extract
>the public interface out into a Interface. (similar to Store and HStore)
>HBASE-4207 solves a slightly different problem. Here I want to have
>access to some HBase internals for performance, but I want to know what
>is public and stable and what I should not touch (thinking about things
>like Phoenix).
>
>
>-- Lars
>
>
>
>________________________________
> From: Andrew Purtell <ap...@apache.org>
>To: "dev@hbase.apache.org" <de...@hbase.apache.org>
>Sent: Monday, February 10, 2014 4:46 PM
>Subject: Re: DISCUSSION: 1.0.0
>
>
>On Mon, Feb 10, 2014 at 4:42 PM, lars hofhansl <la...@apache.org> wrote:
>
>
>> + known coprocessor interfaces. For example right now a coprocessor can
>> use everything on HRegion. We need to distill what's useful into an
>> interface.
>
>
>We started this with Environment and the HTable wrapper as an example of
>how to wrap an interface for CPs. At the end of the day we can't stop a
>coprocessor from accessing internal objects and calling their methods
>directly until HBASE-4047. (Maybe that makes the 1.0 list?)
>
>Related, paring down the coprocessor interfaces to only intercept RPC
>based
>actions and move everything else to plugins.
>
>
>
>--
>Best regards,
>
> - Andy
>
>Problems worthy of attack prove their worth by hitting back. - Piet Hein
>(via Tom White)
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: DISCUSSION: 1.0.0
Posted by lars hofhansl <la...@apache.org>.
True. But we can clearly annotate what is public and what is not.
In HRegion we'd have to do it per method. It's probably easier to extract the public interface out into a Interface. (similar to Store and HStore)
HBASE-4207 solves a slightly different problem. Here I want to have access to some HBase internals for performance, but I want to know what is public and stable and what I should not touch (thinking about things like Phoenix).
-- Lars
________________________________
From: Andrew Purtell <ap...@apache.org>
To: "dev@hbase.apache.org" <de...@hbase.apache.org>
Sent: Monday, February 10, 2014 4:46 PM
Subject: Re: DISCUSSION: 1.0.0
On Mon, Feb 10, 2014 at 4:42 PM, lars hofhansl <la...@apache.org> wrote:
> + known coprocessor interfaces. For example right now a coprocessor can
> use everything on HRegion. We need to distill what's useful into an
> interface.
We started this with Environment and the HTable wrapper as an example of
how to wrap an interface for CPs. At the end of the day we can't stop a
coprocessor from accessing internal objects and calling their methods
directly until HBASE-4047. (Maybe that makes the 1.0 list?)
Related, paring down the coprocessor interfaces to only intercept RPC based
actions and move everything else to plugins.
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: DISCUSSION: 1.0.0
Posted by Andrew Purtell <ap...@apache.org>.
On Mon, Feb 10, 2014 at 4:42 PM, lars hofhansl <la...@apache.org> wrote:
> + known coprocessor interfaces. For example right now a coprocessor can
> use everything on HRegion. We need to distill what's useful into an
> interface.
We started this with Environment and the HTable wrapper as an example of
how to wrap an interface for CPs. At the end of the day we can't stop a
coprocessor from accessing internal objects and calling their methods
directly until HBASE-4047. (Maybe that makes the 1.0 list?)
Related, paring down the coprocessor interfaces to only intercept RPC based
actions and move everything else to plugins.
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: DISCUSSION: 1.0.0
Posted by lars hofhansl <la...@apache.org>.
+ a clearly defined compatibility strategy between patch, minor, and major versions
+ known coprocessor interfaces. For example right now a coprocessor can use everything on HRegion. We need to distill what's useful into an interface.
+ (wishes for ponies and unicorns, probably not going to happen) better ops supports, such as sample puppet/chef/etc scripts
________________________________
From: Stack <st...@duboce.net>
To: HBase Dev List <de...@hbase.apache.org>
Sent: Monday, February 10, 2014 4:29 PM
Subject: Re: DISCUSSION: 1.0.0
Keep them coming and I'll file issues against 1.0 version.
Thanks lads,
St.Ack
On Mon, Feb 10, 2014 at 11:38 AM, Enis Söztutar <en...@gmail.com> wrote:
> + fix interface audience annotations (HBASE-10462).
> + Continue on limiting some broad reaching interfaces (HCM, HCI, HTable)
> See recent HBASE-10479 discussions)
> + Promote HTableInterface vs HTable, getting connections from HCM and
> getting tables there.
>
>
>
> On Mon, Feb 10, 2014 at 11:20 AM, Stack <st...@duboce.net> wrote:
>
> > On Mon, Feb 10, 2014 at 11:15 AM, Steve Loughran <stevel@hortonworks.com
> > >wrote:
> >
> > > On 10 February 2014 09:24, Stack <st...@duboce.net> wrote:
> > >
> > > > Suggestions for 1.0.0 if if it is to come out in next month or so:
> > > >
> > > > + Update included libs (e.g. move to log4j2)
> > > >
> > >
> > > love to see how that goes on -it looks good but I've not seen it in
> > > production yet. I do note it has a NoSQL outputter, so there are good
> > > opportunities for recursion
> > >
> > >
> > Smile.
> >
>
Re: DISCUSSION: 1.0.0
Posted by Stack <st...@duboce.net>.
Keep them coming and I'll file issues against 1.0 version.
Thanks lads,
St.Ack
On Mon, Feb 10, 2014 at 11:38 AM, Enis Söztutar <en...@gmail.com> wrote:
> + fix interface audience annotations (HBASE-10462).
> + Continue on limiting some broad reaching interfaces (HCM, HCI, HTable)
> See recent HBASE-10479 discussions)
> + Promote HTableInterface vs HTable, getting connections from HCM and
> getting tables there.
>
>
>
> On Mon, Feb 10, 2014 at 11:20 AM, Stack <st...@duboce.net> wrote:
>
> > On Mon, Feb 10, 2014 at 11:15 AM, Steve Loughran <stevel@hortonworks.com
> > >wrote:
> >
> > > On 10 February 2014 09:24, Stack <st...@duboce.net> wrote:
> > >
> > > > Suggestions for 1.0.0 if if it is to come out in next month or so:
> > > >
> > > > + Update included libs (e.g. move to log4j2)
> > > >
> > >
> > > love to see how that goes on -it looks good but I've not seen it in
> > > production yet. I do note it has a NoSQL outputter, so there are good
> > > opportunities for recursion
> > >
> > >
> > Smile.
> >
>
Re: DISCUSSION: 1.0.0
Posted by Enis Söztutar <en...@gmail.com>.
+ fix interface audience annotations (HBASE-10462).
+ Continue on limiting some broad reaching interfaces (HCM, HCI, HTable)
See recent HBASE-10479 discussions)
+ Promote HTableInterface vs HTable, getting connections from HCM and
getting tables there.
On Mon, Feb 10, 2014 at 11:20 AM, Stack <st...@duboce.net> wrote:
> On Mon, Feb 10, 2014 at 11:15 AM, Steve Loughran <stevel@hortonworks.com
> >wrote:
>
> > On 10 February 2014 09:24, Stack <st...@duboce.net> wrote:
> >
> > > Suggestions for 1.0.0 if if it is to come out in next month or so:
> > >
> > > + Update included libs (e.g. move to log4j2)
> > >
> >
> > love to see how that goes on -it looks good but I've not seen it in
> > production yet. I do note it has a NoSQL outputter, so there are good
> > opportunities for recursion
> >
> >
> Smile.
>
Re: DISCUSSION: 1.0.0
Posted by Stack <st...@duboce.net>.
On Mon, Feb 10, 2014 at 11:15 AM, Steve Loughran <st...@hortonworks.com>wrote:
> On 10 February 2014 09:24, Stack <st...@duboce.net> wrote:
>
> > Suggestions for 1.0.0 if if it is to come out in next month or so:
> >
> > + Update included libs (e.g. move to log4j2)
> >
>
> love to see how that goes on -it looks good but I've not seen it in
> production yet. I do note it has a NoSQL outputter, so there are good
> opportunities for recursion
>
>
Smile.
Re: DISCUSSION: 1.0.0
Posted by Steve Loughran <st...@hortonworks.com>.
On 10 February 2014 09:24, Stack <st...@duboce.net> wrote:
> Suggestions for 1.0.0 if if it is to come out in next month or so:
>
> + Update included libs (e.g. move to log4j2)
>
love to see how that goes on -it looks good but I've not seen it in
production yet. I do note it has a NoSQL outputter, so there are good
opportunities for recursion
>
> + Enable distributed log replay as default (fix bugs)
> + Enable hfilev3 as default.
> + Ship with default logging level set to INFO and content of the logs still
> makes sense
>
> What else?
>
>
+ maybe make the hconnection output a bit less verbose when ZK isn't live
or HBase still coming up; it does generate a fair amount of log data
+ light pastel purple colour scheme.
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: DISCUSSION: 1.0.0
Posted by Nick Dimiduk <nd...@gmail.com>.
I'd like to see a bunch of our @Deprecated APIs cleaned up. Delete stale
methods or remove the annotations.
On Mon, Feb 10, 2014 at 10:24 AM, Stack <st...@duboce.net> wrote:
> Suggestions for 1.0.0 if if it is to come out in next month or so:
>
> + Update included libs (e.g. move to log4j2)
>
> + Enable distributed log replay as default (fix bugs)
> + Enable hfilev3 as default.
> + Ship with default logging level set to INFO and content of the logs still
> makes sense
>
> What else?
>
> + Enable dynamic config and schema by default.
>
>
> St.Ack
>
>
>
>
> On Mon, Jan 20, 2014 at 4:09 PM, Stack <st...@duboce.net> wrote:
>
> > On Mon, Jan 20, 2014 at 1:52 PM, lars hofhansl <la...@apache.org> wrote:
> >
> >> I'm happy to volunteer. Happy if Enis does it, too.
> >>
> >>
> > I'd be happy to do it too but my thinking is that it is good to spread
> the
> > role around.
> > St.Ack
> >
> >
> >
> >
> >> ------------------------------
> >> *From:* Stack <st...@duboce.net>
> >> *To:* HBase Dev List <de...@hbase.apache.org>
> >> *Cc:* lars hofhansl <la...@apache.org>
> >> *Sent:* Monday, January 20, 2014 1:43 PM
> >> *Subject:* Re: DISCUSSION: 1.0.0
> >>
> >> On Thu, Jan 2, 2014 at 1:20 PM, Enis Söztutar <en...@apache.org> wrote:
> >>
> >> I think whether we will need a new RM will depend on the decision to
> >> release 1.0 from 0.98 branches or 0.99 branches(current trunk).
> >>
> >>
> >> I think it should have an RM regardless. We should probably try to put
> a
> >> higher polish on a 1.0 than we would mayhaps on a lesser release. RM
> will
> >> have enough work on their plate just keeping up state (IMO).
> >>
> >>
> >>
> >> We can do the previous practice of releasing 0.99.0, then turning 0.99.x
> >> as
> >> the 1.0.0. In that case, I can also volunteer as well.
> >>
> >>
> >> Good by me. Anyone else interested in the job? Speak up if so.
> >>
> >> If not, you'd get it by default Enis. Else you and whoever will have to
> >> dook it out.
> >>
> >> St.Ack
> >>
> >>
> >>
> >
>