[DISCUSS] HBase 3

[Sean Busbey <bu...@apache.org>]

Hi folks!

We're about a year from when HBase 2.0.0 went GA. I'd like to start
cutting alpha releases of 3.0.0 off of the master branch soon.
(expressly I do not want to create another branch, so for now anything
landing in master would be "in" for 3.0.0. hence the "alpha"
designation.)

I was going to wait for one of the HBase 2 releases to get the stable
label. But lately I don't have a good sense of if that will happen
within a month or within six months, so I'm leaning away from waiting.

I personally don't have a particular feature I'm trying to get out the
door. I just think we're at risk of another waiting-too-long for a
major release and want to get started on the work of quantifying
what's changed and figuring out how downstream projects are impacted.
I find that all much easier to do when there's a release artifact to
reference.

I'm happy to just cut alpha releases until someone shows up with a
specific feature need. At that point we can come up with criteria for
entering and exiting beta releases.

What do folks plan to work on getting ready that needs happen in a
major release?

Re: [DISCUSS] HBase 3

[Sean Busbey <bu...@apache.org>]

I agree with Duo, neither of those is close to done enough for 3.0.

On Mon, May 6, 2019, 06:48 张铎(Duo Zhang) <pa...@gmail.com> wrote:

> I do not think these two features can be done in 3.0...
>
> Anoop John <an...@gmail.com>于2019年5月6日 周一14:48写道：
>
> > For the cloud usages, we should include the new FS abstraction issue also
> > targeted ?  What abt the WAL on Ratis?  We can fix the features what we
> > would like to see in 3.0 and then have feature freeze. So that 3.0 wont
> > become so huge changes.
> >
> > Anoop
> >
> > On Mon, May 6, 2019 at 9:52 AM Yu Li <ca...@gmail.com> wrote:
> >
> > > TL;DR: Maybe firstly we should figure out what direction we should
> focus
> > > for 3.0? It seems to me current performance is good enough for most use
> > > case and no longer a big concern, so more requirements are about
> > stability
> > > and elasticity (in cloud)? Or if anyone do have performance concern,
> > please
> > > shout and let us know, thanks.
> > >
> > > All below items are performance oriented, just list out for reference
> > (and
> > > not sure about priority from community perspective):
> > > 1. CCSMap (HBASE-20312) (delayed due to job priority, sorry, but we
> could
> > > get it done if required).
> > > 2. Maybe we should also target at making in-memory-flush/compaction
> from
> > > experimental to production ready?
> > > 3. Server-side asynchronous (especially the write pipeline) is another
> > > left-over job I ever promised to upstream but failed because the
> business
> > > growth on my side slows down quite a bit so it's not fully verified
> > online
> > > yet.
> > >
> > > Best Regards,
> > > Yu
> > >
> > >
> > > On Mon, 6 May 2019 at 06:07, Andrew Purtell <ap...@apache.org>
> wrote:
> > >
> > > > Definitely there is value in starting this so trunk doesn't sink
> into a
> > > > difficult to release state. Alpha releases seem fine if you want to
> > make
> > > > them. This assumes you'll have at least a small amount of bandwidth
> to
> > > run
> > > > tests and triage any issues, though, or else any effort would be
> better
> > > > spent completing the work needed to move the stable pointer to 2.x.
> > Just
> > > my
> > > > random advice.
> > > >
> > > >
> > > > On Thu, Apr 25, 2019 at 7:55 AM Sean Busbey <bu...@apache.org>
> wrote:
> > > >
> > > > > Hi folks!
> > > > >
> > > > > We're about a year from when HBase 2.0.0 went GA. I'd like to start
> > > > > cutting alpha releases of 3.0.0 off of the master branch soon.
> > > > > (expressly I do not want to create another branch, so for now
> > anything
> > > > > landing in master would be "in" for 3.0.0. hence the "alpha"
> > > > > designation.)
> > > > >
> > > > > I was going to wait for one of the HBase 2 releases to get the
> stable
> > > > > label. But lately I don't have a good sense of if that will happen
> > > > > within a month or within six months, so I'm leaning away from
> > waiting.
> > > > >
> > > > > I personally don't have a particular feature I'm trying to get out
> > the
> > > > > door. I just think we're at risk of another waiting-too-long for a
> > > > > major release and want to get started on the work of quantifying
> > > > > what's changed and figuring out how downstream projects are
> impacted.
> > > > > I find that all much easier to do when there's a release artifact
> to
> > > > > reference.
> > > > >
> > > > > I'm happy to just cut alpha releases until someone shows up with a
> > > > > specific feature need. At that point we can come up with criteria
> for
> > > > > entering and exiting beta releases.
> > > > >
> > > > > What do folks plan to work on getting ready that needs happen in a
> > > > > major release?
> > > > >
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > > Andrew
> > > >
> > > > Words like orphans lost among the crosstalk, meaning torn from
> truth's
> > > > decrepit hands
> > > >    - A23, Crosstalk
> > > >
> > >
> >
>

Re: [DISCUSS] HBase 3

["张铎 (Duo Zhang)" <pa...@gmail.com>]

I do not think these two features can be done in 3.0...

Anoop John <an...@gmail.com>于2019年5月6日 周一14:48写道：

> For the cloud usages, we should include the new FS abstraction issue also
> targeted ?  What abt the WAL on Ratis?  We can fix the features what we
> would like to see in 3.0 and then have feature freeze. So that 3.0 wont
> become so huge changes.
>
> Anoop
>
> On Mon, May 6, 2019 at 9:52 AM Yu Li <ca...@gmail.com> wrote:
>
> > TL;DR: Maybe firstly we should figure out what direction we should focus
> > for 3.0? It seems to me current performance is good enough for most use
> > case and no longer a big concern, so more requirements are about
> stability
> > and elasticity (in cloud)? Or if anyone do have performance concern,
> please
> > shout and let us know, thanks.
> >
> > All below items are performance oriented, just list out for reference
> (and
> > not sure about priority from community perspective):
> > 1. CCSMap (HBASE-20312) (delayed due to job priority, sorry, but we could
> > get it done if required).
> > 2. Maybe we should also target at making in-memory-flush/compaction from
> > experimental to production ready?
> > 3. Server-side asynchronous (especially the write pipeline) is another
> > left-over job I ever promised to upstream but failed because the business
> > growth on my side slows down quite a bit so it's not fully verified
> online
> > yet.
> >
> > Best Regards,
> > Yu
> >
> >
> > On Mon, 6 May 2019 at 06:07, Andrew Purtell <ap...@apache.org> wrote:
> >
> > > Definitely there is value in starting this so trunk doesn't sink into a
> > > difficult to release state. Alpha releases seem fine if you want to
> make
> > > them. This assumes you'll have at least a small amount of bandwidth to
> > run
> > > tests and triage any issues, though, or else any effort would be better
> > > spent completing the work needed to move the stable pointer to 2.x.
> Just
> > my
> > > random advice.
> > >
> > >
> > > On Thu, Apr 25, 2019 at 7:55 AM Sean Busbey <bu...@apache.org> wrote:
> > >
> > > > Hi folks!
> > > >
> > > > We're about a year from when HBase 2.0.0 went GA. I'd like to start
> > > > cutting alpha releases of 3.0.0 off of the master branch soon.
> > > > (expressly I do not want to create another branch, so for now
> anything
> > > > landing in master would be "in" for 3.0.0. hence the "alpha"
> > > > designation.)
> > > >
> > > > I was going to wait for one of the HBase 2 releases to get the stable
> > > > label. But lately I don't have a good sense of if that will happen
> > > > within a month or within six months, so I'm leaning away from
> waiting.
> > > >
> > > > I personally don't have a particular feature I'm trying to get out
> the
> > > > door. I just think we're at risk of another waiting-too-long for a
> > > > major release and want to get started on the work of quantifying
> > > > what's changed and figuring out how downstream projects are impacted.
> > > > I find that all much easier to do when there's a release artifact to
> > > > reference.
> > > >
> > > > I'm happy to just cut alpha releases until someone shows up with a
> > > > specific feature need. At that point we can come up with criteria for
> > > > entering and exiting beta releases.
> > > >
> > > > What do folks plan to work on getting ready that needs happen in a
> > > > major release?
> > > >
> > >
> > >
> > > --
> > > Best regards,
> > > Andrew
> > >
> > > Words like orphans lost among the crosstalk, meaning torn from truth's
> > > decrepit hands
> > >    - A23, Crosstalk
> > >
> >
>

Re: [DISCUSS] HBase 3

[Anoop John <an...@gmail.com>]

For the cloud usages, we should include the new FS abstraction issue also
targeted ?  What abt the WAL on Ratis?  We can fix the features what we
would like to see in 3.0 and then have feature freeze. So that 3.0 wont
become so huge changes.

Anoop

On Mon, May 6, 2019 at 9:52 AM Yu Li <ca...@gmail.com> wrote:

> TL;DR: Maybe firstly we should figure out what direction we should focus
> for 3.0? It seems to me current performance is good enough for most use
> case and no longer a big concern, so more requirements are about stability
> and elasticity (in cloud)? Or if anyone do have performance concern, please
> shout and let us know, thanks.
>
> All below items are performance oriented, just list out for reference (and
> not sure about priority from community perspective):
> 1. CCSMap (HBASE-20312) (delayed due to job priority, sorry, but we could
> get it done if required).
> 2. Maybe we should also target at making in-memory-flush/compaction from
> experimental to production ready?
> 3. Server-side asynchronous (especially the write pipeline) is another
> left-over job I ever promised to upstream but failed because the business
> growth on my side slows down quite a bit so it's not fully verified online
> yet.
>
> Best Regards,
> Yu
>
>
> On Mon, 6 May 2019 at 06:07, Andrew Purtell <ap...@apache.org> wrote:
>
> > Definitely there is value in starting this so trunk doesn't sink into a
> > difficult to release state. Alpha releases seem fine if you want to make
> > them. This assumes you'll have at least a small amount of bandwidth to
> run
> > tests and triage any issues, though, or else any effort would be better
> > spent completing the work needed to move the stable pointer to 2.x. Just
> my
> > random advice.
> >
> >
> > On Thu, Apr 25, 2019 at 7:55 AM Sean Busbey <bu...@apache.org> wrote:
> >
> > > Hi folks!
> > >
> > > We're about a year from when HBase 2.0.0 went GA. I'd like to start
> > > cutting alpha releases of 3.0.0 off of the master branch soon.
> > > (expressly I do not want to create another branch, so for now anything
> > > landing in master would be "in" for 3.0.0. hence the "alpha"
> > > designation.)
> > >
> > > I was going to wait for one of the HBase 2 releases to get the stable
> > > label. But lately I don't have a good sense of if that will happen
> > > within a month or within six months, so I'm leaning away from waiting.
> > >
> > > I personally don't have a particular feature I'm trying to get out the
> > > door. I just think we're at risk of another waiting-too-long for a
> > > major release and want to get started on the work of quantifying
> > > what's changed and figuring out how downstream projects are impacted.
> > > I find that all much easier to do when there's a release artifact to
> > > reference.
> > >
> > > I'm happy to just cut alpha releases until someone shows up with a
> > > specific feature need. At that point we can come up with criteria for
> > > entering and exiting beta releases.
> > >
> > > What do folks plan to work on getting ready that needs happen in a
> > > major release?
> > >
> >
> >
> > --
> > Best regards,
> > Andrew
> >
> > Words like orphans lost among the crosstalk, meaning torn from truth's
> > decrepit hands
> >    - A23, Crosstalk
> >
>

Re: [DISCUSS] HBase 3

[Yu Li <ca...@gmail.com>]

TL;DR: Maybe firstly we should figure out what direction we should focus
for 3.0? It seems to me current performance is good enough for most use
case and no longer a big concern, so more requirements are about stability
and elasticity (in cloud)? Or if anyone do have performance concern, please
shout and let us know, thanks.

All below items are performance oriented, just list out for reference (and
not sure about priority from community perspective):
1. CCSMap (HBASE-20312) (delayed due to job priority, sorry, but we could
get it done if required).
2. Maybe we should also target at making in-memory-flush/compaction from
experimental to production ready?
3. Server-side asynchronous (especially the write pipeline) is another
left-over job I ever promised to upstream but failed because the business
growth on my side slows down quite a bit so it's not fully verified online
yet.

Best Regards,
Yu


On Mon, 6 May 2019 at 06:07, Andrew Purtell <ap...@apache.org> wrote:

> Definitely there is value in starting this so trunk doesn't sink into a
> difficult to release state. Alpha releases seem fine if you want to make
> them. This assumes you'll have at least a small amount of bandwidth to run
> tests and triage any issues, though, or else any effort would be better
> spent completing the work needed to move the stable pointer to 2.x. Just my
> random advice.
>
>
> On Thu, Apr 25, 2019 at 7:55 AM Sean Busbey <bu...@apache.org> wrote:
>
> > Hi folks!
> >
> > We're about a year from when HBase 2.0.0 went GA. I'd like to start
> > cutting alpha releases of 3.0.0 off of the master branch soon.
> > (expressly I do not want to create another branch, so for now anything
> > landing in master would be "in" for 3.0.0. hence the "alpha"
> > designation.)
> >
> > I was going to wait for one of the HBase 2 releases to get the stable
> > label. But lately I don't have a good sense of if that will happen
> > within a month or within six months, so I'm leaning away from waiting.
> >
> > I personally don't have a particular feature I'm trying to get out the
> > door. I just think we're at risk of another waiting-too-long for a
> > major release and want to get started on the work of quantifying
> > what's changed and figuring out how downstream projects are impacted.
> > I find that all much easier to do when there's a release artifact to
> > reference.
> >
> > I'm happy to just cut alpha releases until someone shows up with a
> > specific feature need. At that point we can come up with criteria for
> > entering and exiting beta releases.
> >
> > What do folks plan to work on getting ready that needs happen in a
> > major release?
> >
>
>
> --
> Best regards,
> Andrew
>
> Words like orphans lost among the crosstalk, meaning torn from truth's
> decrepit hands
>    - A23, Crosstalk
>

Re: [DISCUSS] HBase 3

[Andrew Purtell <ap...@apache.org>]

Definitely there is value in starting this so trunk doesn't sink into a
difficult to release state. Alpha releases seem fine if you want to make
them. This assumes you'll have at least a small amount of bandwidth to run
tests and triage any issues, though, or else any effort would be better
spent completing the work needed to move the stable pointer to 2.x. Just my
random advice.


On Thu, Apr 25, 2019 at 7:55 AM Sean Busbey <bu...@apache.org> wrote:

> Hi folks!
>
> We're about a year from when HBase 2.0.0 went GA. I'd like to start
> cutting alpha releases of 3.0.0 off of the master branch soon.
> (expressly I do not want to create another branch, so for now anything
> landing in master would be "in" for 3.0.0. hence the "alpha"
> designation.)
>
> I was going to wait for one of the HBase 2 releases to get the stable
> label. But lately I don't have a good sense of if that will happen
> within a month or within six months, so I'm leaning away from waiting.
>
> I personally don't have a particular feature I'm trying to get out the
> door. I just think we're at risk of another waiting-too-long for a
> major release and want to get started on the work of quantifying
> what's changed and figuring out how downstream projects are impacted.
> I find that all much easier to do when there's a release artifact to
> reference.
>
> I'm happy to just cut alpha releases until someone shows up with a
> specific feature need. At that point we can come up with criteria for
> entering and exiting beta releases.
>
> What do folks plan to work on getting ready that needs happen in a
> major release?
>


-- 
Best regards,
Andrew

Words like orphans lost among the crosstalk, meaning torn from truth's
decrepit hands
   - A23, Crosstalk

Re: [DISCUSS] HBase 3

["张铎 (Duo Zhang)" <pa...@gmail.com>]

I think sync replication is a big enough feature for a major release, but I
do not think it is production ready... We have back ported the feature to
our internal branch which is based on branch-2, but haven't used it yet...

And I also want to land HBASE-21512, which can greatly reduce our client
code base, in HBASE-21723 we can remove more than 20k lines of code.

And maybe HBASE-21879? Which is good for off heap handling and performance.
And also CCSMap?

Sean Busbey <bu...@apache.org> 于2019年4月25日周四 下午10:55写道：

> Hi folks!
>
> We're about a year from when HBase 2.0.0 went GA. I'd like to start
> cutting alpha releases of 3.0.0 off of the master branch soon.
> (expressly I do not want to create another branch, so for now anything
> landing in master would be "in" for 3.0.0. hence the "alpha"
> designation.)
>
> I was going to wait for one of the HBase 2 releases to get the stable
> label. But lately I don't have a good sense of if that will happen
> within a month or within six months, so I'm leaning away from waiting.
>
> I personally don't have a particular feature I'm trying to get out the
> door. I just think we're at risk of another waiting-too-long for a
> major release and want to get started on the work of quantifying
> what's changed and figuring out how downstream projects are impacted.
> I find that all much easier to do when there's a release artifact to
> reference.
>
> I'm happy to just cut alpha releases until someone shows up with a
> specific feature need. At that point we can come up with criteria for
> entering and exiting beta releases.
>
> What do folks plan to work on getting ready that needs happen in a
> major release?
>