You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Mikhail Antonov <ol...@gmail.com> on 2016/03/21 09:35:01 UTC
Re: Branch for 1.3
Hi folks,
bringing this topic up again. I'm planning to start spinning 1.3 builds and
see if/where they break in a week or two, and (depending on how it does)
start preparing RCs in a month or maybe two. So, let's see where we are.
Big items first. There were long debates around three big items - MOBs,
Spark connector and RS groups, whether we should have them or not.
- MOBs
I believe we decided that they aren't going to go in branch-1, and hence
not in branch-1.3 for sure. We might get back to that debate and
re-consider MOBs for branch-1 if 2.0 is delayed, but almost certainly they
won't make it in 1.3 timeframe anyway as I feel.
- Spark connector - HBASE-14160 it looks like it has 3 subtasks open, one
of which is big one (HBASE-14375) - define public API for Spark
integration. From the Jira looks like active work is happening on other
subtasks, but not on this one. So how's public API going? How stable it is?
Who would want to have Spark in 1.3 and willing to help with this? OTOH -
who has objections about back-porting it? Has anyone been using it in some
real environment?
- RS groups - there was recently a thread about them, I'd like to bring it
up again and get to some conclusion.
Other features which we had in flight a month ago -
- HBASE-15181 - date based tiered compactions has landed
- HBASE-11290 - unlock RegionStates. I'm afraid the codebase has moved
forward quite a bit since the benchmark was run on this change :( - Francis
- are you using it now? If we could have some benchmarks on the latest
rebase that I think would be great.
- HBASE-13557 - special handling for system tables WALs - should we still
keep it targeted for 1.3?
- HBASE-13017 - keep table state in meta, this one doesn't look like it's
going to make it in
As a new item on my list - I'm looking forward to see more of HBASE-15492
(memory optimizations) subtasks to go in branch-1.
Thanks!
Mikhail
On Sun, Feb 28, 2016 at 10:53 AM, Andrew Purtell <an...@gmail.com>
wrote:
> I'm starting a push at work to get us up on 1.2. Assuming that happens
> later this year I think that will be the end of my close attention to 0.98.
>
> > On Feb 26, 2016, at 1:54 PM, Nick Dimiduk <nd...@gmail.com> wrote:
> >
> > On Fri, Feb 26, 2016 at 12:23 PM, Andrew Purtell <ap...@apache.org>
> > wrote:
> >
> >> In the meantime those of us running HBase in production would benefit
> from
> >> fairly frequent minor releases.
> >
> > +1. Having to look back to 0.98 to get some new feature is problematic.
> >
> >> On Fri, Feb 26, 2016 at 9:50 AM, Elliott Clark <ec...@apache.org>
> wrote:
> >>
> >>> I disagree. We have agreed that 2.0 will have a new assignement
> manager.
> >>> There's a lot of work that has been done on getting that in, so far
> there
> >>> are no benefits to the end user from all that work. We should stick
> with
> >>> the plan and release 2.0 when it's ready.
> >>>
> >>> On Fri, Feb 26, 2016 at 9:39 AM, Stephen Jiang <
> syuanjiangdev@gmail.com>
> >>> wrote:
> >>>
> >>>> Thanks for Mikhail for taking the 1.3 RM role. Looks like we have a
> >> lot
> >>> of
> >>>> new things in 1.3 release.
> >>>>
> >>>> Based on the experience of 1.1 and 1.2 release, it takes a lot of
> >> efforts
> >>>> to get a stable minor release out. From this, I have my own 2-cents
> on
> >>> 1.4
> >>>> release. The plan is to have 2.0 release during summer time of this
> >> year
> >>>> (yeah, *this year). * Given the limited time and resource, after 1.3
> >>>> release, instead of spending effort on 1.4 release, the community
> >> should
> >>>> focus on stabilizing master (or branch-2, not exist as of now) branch
> >> and
> >>>> make 2.0 release a priority. 2.0 release would bring more values to
> >>>> customer & move towards maturity of HBASE product.
> >>>>
> >>>> Thanks
> >>>> Stephen
> >>>>
> >>>> On Fri, Feb 26, 2016 at 1:49 AM, Mikhail Antonov <
> olorinbant@gmail.com
> >>>
> >>>> wrote:
> >>>>
> >>>>> Created an umbrella jira for 1.3 release - HBASE-15341
> >>>>>
> >>>>> So it looks like we may have 1.4 release before 2.0 is out? I tried
> >> to
> >>>> add
> >>>>> 1.4 version in jira so we can keep it in branch-1 poms but I
> >> couldn't -
> >>>>> looks like I don't have permissions?
> >>>>>
> >>>>> -Mikhail
> >>>>>
> >>>>> On Thu, Feb 25, 2016 at 3:52 PM, Andrew Purtell <apurtell@apache.org
> >>>
> >>>>> wrote:
> >>>>>
> >>>>>> The guy we had looking at streaming replication moved on and
> >> there's
> >>> no
> >>>>>> immediate plans to take on the work, FWIW
> >>>>>>
> >>>>>> On Thu, Feb 25, 2016 at 3:33 PM, Matteo Bertozzi <
> >>>>> theo.bertozzi@gmail.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> I was shooting for summer for hbase 2.0, the main problem is that
> >>>> there
> >>>>>> is
> >>>>>>> still no code for the new AM or for fs changes, which are the two
> >>>> that
> >>>>>> may
> >>>>>>> impact compatibility (working slowly on that). Streaming
> >>> replication
> >>>>> and
> >>>>>>> others seems compatible enough but no code there too.
> >>>>>>>
> >>>>>>> Matteo
> >>>>>>>
> >>>>>>>
> >>>>>>> On Thu, Feb 25, 2016 at 3:29 PM, Mikhail Antonov <
> >>>> olorinbant@gmail.com
> >>>>>>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Agreed. I just meant - readiness of 2.0 is something affecting
> >>>>>> decisions
> >>>>>>> on
> >>>>>>>> whether or not to backport mobs to branch-1 (which is itself
> >>>> separate
> >>>>>>>> thread).
> >>>>>>>>
> >>>>>>>> -Mikhail
> >>>>>>>>
> >>>>>>>> On Thu, Feb 25, 2016 at 3:27 PM, Sean Busbey <
> >>> busbey@cloudera.com>
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> On Thu, Feb 25, 2016 at 5:21 PM, Mikhail Antonov <
> >>>>>> olorinbant@gmail.com
> >>>>>>>>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> - "Shouldn't we rather try to get 2.0 release out and have
> >>>> mobs
> >>>>>>>> there".
> >>>>>>>>> -
> >>>>>>>>>> So how far do we feel 2.0 release is?
> >>>>>>>>>
> >>>>>>>>> 2.0 readiness probably deserves its own [DISCUSS] thread, but
> >>>> we're
> >>>>>> now
> >>>>>>>>> past a year since the HBase 1.0.0 release, so I hope it's
> >> soon.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> Sean
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> Thanks,
> >>>>>>>> Michael Antonov
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Best regards,
> >>>>>>
> >>>>>> - Andy
> >>>>>>
> >>>>>> Problems worthy of attack prove their worth by hitting back. - Piet
> >>>> Hein
> >>>>>> (via Tom White)
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> Thanks,
> >>>>> Michael Antonov
> >>
> >>
> >>
> >> --
> >> Best regards,
> >>
> >> - Andy
> >>
> >> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> >> (via Tom White)
> >>
>
--
Thanks,
Michael Antonov
Re: Branch for 1.3
Posted by Sean Busbey <bu...@cloudera.com>.
On Mon, Mar 21, 2016 at 2:05 PM, Mikhail Antonov <ol...@gmail.com>
wrote:
> At this point branch-1.3 is very close (if different from at all) to
> branch-1, so that's probably the same discussion.
>
>
Fair enough. Is the previous discussion and documented gates (along with
Andrew's assertion of users) sufficient?
--
busbey
Re: Branch for 1.3
Posted by Andrew Purtell <ap...@apache.org>.
FWIW, I'd like to see the Spark connector get in. We have users who will be
interested in it.
On Mon, Mar 21, 2016 at 12:05 PM, Mikhail Antonov <ol...@gmail.com>
wrote:
> At this point branch-1.3 is very close (if different from at all) to
> branch-1, so that's probably the same discussion.
>
> -Mikhail
>
> On Mon, Mar 21, 2016 at 11:47 AM, Sean Busbey <bu...@cloudera.com> wrote:
>
> > Is the spark connector thread specifically about 1.3? or branch-1?
> because
> > we already had the branch-1 conversation. the specific gates were tracked
> > in the umbrella jira.
> >
> > -Sean
> >
> > On Mon, Mar 21, 2016 at 1:33 PM, Mikhail Antonov <ol...@gmail.com>
> > wrote:
> >
> > > Yeah, we probably should start discussion thread about Spark connector.
> > > Anyone wants to start the thread and push it forward?
> > >
> > > Regarding date-tiered compactions - since first impl already went in
> 1.3,
> > > would be good to get any possible improvements in 1.3 as well, as long
> as
> > > they are stable, IMO.
> > >
> > > -Mikhail
> > >
> > > On Mon, Mar 21, 2016 at 11:03 AM, Enis Söztutar <en...@gmail.com>
> > > wrote:
> > >
> > > > You may want to track
> > https://issues.apache.org/jira/browse/HBASE-15339
> > > as
> > > > a parent for date-tiered compaction improvements. Current compaction
> > > policy
> > > > is useful in its own, but handling existing data, bulk loading etc
> will
> > > be
> > > > improved with these subtasks. I think the patches can land before the
> > 1.3
> > > > timeframe, but of course open for discussion for inclusion. My vote
> > would
> > > > be to include all the improvements since it would be easier to tell
> the
> > > > story to users.
> > > >
> > > > Enis
> > > >
> > > > On Mon, Mar 21, 2016 at 3:32 AM, Ted Yu <yu...@gmail.com> wrote:
> > > >
> > > > > For Spark connector, we should start a separate discussion thread
> > about
> > > > > backporting to branch-1.
> > > > >
> > > > > Zhan has a bug fix coming this week which deals with how negative
> > > numbers
> > > > > are handled in comparison.
> > > > >
> > > > > FYI
> > > > >
> > > > > On Mon, Mar 21, 2016 at 1:35 AM, Mikhail Antonov <
> > olorinbant@gmail.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > Hi folks,
> > > > > >
> > > > > > bringing this topic up again. I'm planning to start spinning 1.3
> > > builds
> > > > > and
> > > > > > see if/where they break in a week or two, and (depending on how
> it
> > > > does)
> > > > > > start preparing RCs in a month or maybe two. So, let's see where
> we
> > > > are.
> > > > > >
> > > > > > Big items first. There were long debates around three big items -
> > > MOBs,
> > > > > > Spark connector and RS groups, whether we should have them or
> not.
> > > > > >
> > > > > > - MOBs
> > > > > > I believe we decided that they aren't going to go in branch-1,
> and
> > > > hence
> > > > > > not in branch-1.3 for sure. We might get back to that debate and
> > > > > > re-consider MOBs for branch-1 if 2.0 is delayed, but almost
> > certainly
> > > > > they
> > > > > > won't make it in 1.3 timeframe anyway as I feel.
> > > > > >
> > > > > > - Spark connector - HBASE-14160 it looks like it has 3 subtasks
> > open,
> > > > one
> > > > > > of which is big one (HBASE-14375) - define public API for Spark
> > > > > > integration. From the Jira looks like active work is happening on
> > > other
> > > > > > subtasks, but not on this one. So how's public API going? How
> > stable
> > > it
> > > > > is?
> > > > > > Who would want to have Spark in 1.3 and willing to help with
> this?
> > > > OTOH -
> > > > > > who has objections about back-porting it? Has anyone been using
> it
> > in
> > > > > some
> > > > > > real environment?
> > > > > >
> > > > > > - RS groups - there was recently a thread about them, I'd like
> to
> > > > bring
> > > > > it
> > > > > > up again and get to some conclusion.
> > > > > >
> > > > > > Other features which we had in flight a month ago -
> > > > > >
> > > > > > - HBASE-15181 - date based tiered compactions has landed
> > > > > > - HBASE-11290 - unlock RegionStates. I'm afraid the codebase has
> > > moved
> > > > > > forward quite a bit since the benchmark was run on this change
> :( -
> > > > > Francis
> > > > > > - are you using it now? If we could have some benchmarks on the
> > > latest
> > > > > > rebase that I think would be great.
> > > > > >
> > > > > >
> > > > > > - HBASE-13557 - special handling for system tables WALs - should
> > we
> > > > > still
> > > > > > keep it targeted for 1.3?
> > > > > > - HBASE-13017 - keep table state in meta, this one doesn't look
> > like
> > > > > it's
> > > > > > going to make it in
> > > > > >
> > > > > > As a new item on my list - I'm looking forward to see more of
> > > > HBASE-15492
> > > > > > (memory optimizations) subtasks to go in branch-1.
> > > > > >
> > > > > > Thanks!
> > > > > > Mikhail
> > > > > >
> > > > > > On Sun, Feb 28, 2016 at 10:53 AM, Andrew Purtell <
> > > > > andrew.purtell@gmail.com
> > > > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > I'm starting a push at work to get us up on 1.2. Assuming that
> > > > happens
> > > > > > > later this year I think that will be the end of my close
> > attention
> > > to
> > > > > > 0.98.
> > > > > > >
> > > > > > > > On Feb 26, 2016, at 1:54 PM, Nick Dimiduk <
> ndimiduk@gmail.com>
> > > > > wrote:
> > > > > > > >
> > > > > > > > On Fri, Feb 26, 2016 at 12:23 PM, Andrew Purtell <
> > > > > apurtell@apache.org>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > >> In the meantime those of us running HBase in production
> would
> > > > > benefit
> > > > > > > from
> > > > > > > >> fairly frequent minor releases.
> > > > > > > >
> > > > > > > > +1. Having to look back to 0.98 to get some new feature is
> > > > > problematic.
> > > > > > > >
> > > > > > > >> On Fri, Feb 26, 2016 at 9:50 AM, Elliott Clark <
> > > eclark@apache.org
> > > > >
> > > > > > > wrote:
> > > > > > > >>
> > > > > > > >>> I disagree. We have agreed that 2.0 will have a new
> > assignement
> > > > > > > manager.
> > > > > > > >>> There's a lot of work that has been done on getting that
> in,
> > so
> > > > far
> > > > > > > there
> > > > > > > >>> are no benefits to the end user from all that work. We
> should
> > > > stick
> > > > > > > with
> > > > > > > >>> the plan and release 2.0 when it's ready.
> > > > > > > >>>
> > > > > > > >>> On Fri, Feb 26, 2016 at 9:39 AM, Stephen Jiang <
> > > > > > > syuanjiangdev@gmail.com>
> > > > > > > >>> wrote:
> > > > > > > >>>
> > > > > > > >>>> Thanks for Mikhail for taking the 1.3 RM role. Looks like
> > we
> > > > > have a
> > > > > > > >> lot
> > > > > > > >>> of
> > > > > > > >>>> new things in 1.3 release.
> > > > > > > >>>>
> > > > > > > >>>> Based on the experience of 1.1 and 1.2 release, it takes a
> > lot
> > > > of
> > > > > > > >> efforts
> > > > > > > >>>> to get a stable minor release out. From this, I have my
> own
> > > > > 2-cents
> > > > > > > on
> > > > > > > >>> 1.4
> > > > > > > >>>> release. The plan is to have 2.0 release during summer
> time
> > > of
> > > > > this
> > > > > > > >> year
> > > > > > > >>>> (yeah, *this year). * Given the limited time and
> resource,
> > > > after
> > > > > > 1.3
> > > > > > > >>>> release, instead of spending effort on 1.4 release, the
> > > > community
> > > > > > > >> should
> > > > > > > >>>> focus on stabilizing master (or branch-2, not exist as of
> > now)
> > > > > > branch
> > > > > > > >> and
> > > > > > > >>>> make 2.0 release a priority. 2.0 release would bring more
> > > > values
> > > > > to
> > > > > > > >>>> customer & move towards maturity of HBASE product.
> > > > > > > >>>>
> > > > > > > >>>> Thanks
> > > > > > > >>>> Stephen
> > > > > > > >>>>
> > > > > > > >>>> On Fri, Feb 26, 2016 at 1:49 AM, Mikhail Antonov <
> > > > > > > olorinbant@gmail.com
> > > > > > > >>>
> > > > > > > >>>> wrote:
> > > > > > > >>>>
> > > > > > > >>>>> Created an umbrella jira for 1.3 release - HBASE-15341
> > > > > > > >>>>>
> > > > > > > >>>>> So it looks like we may have 1.4 release before 2.0 is
> > out? I
> > > > > tried
> > > > > > > >> to
> > > > > > > >>>> add
> > > > > > > >>>>> 1.4 version in jira so we can keep it in branch-1 poms
> but
> > I
> > > > > > > >> couldn't -
> > > > > > > >>>>> looks like I don't have permissions?
> > > > > > > >>>>>
> > > > > > > >>>>> -Mikhail
> > > > > > > >>>>>
> > > > > > > >>>>> On Thu, Feb 25, 2016 at 3:52 PM, Andrew Purtell <
> > > > > > apurtell@apache.org
> > > > > > > >>>
> > > > > > > >>>>> wrote:
> > > > > > > >>>>>
> > > > > > > >>>>>> The guy we had looking at streaming replication moved on
> > and
> > > > > > > >> there's
> > > > > > > >>> no
> > > > > > > >>>>>> immediate plans to take on the work, FWIW
> > > > > > > >>>>>>
> > > > > > > >>>>>> On Thu, Feb 25, 2016 at 3:33 PM, Matteo Bertozzi <
> > > > > > > >>>>> theo.bertozzi@gmail.com>
> > > > > > > >>>>>> wrote:
> > > > > > > >>>>>>
> > > > > > > >>>>>>> I was shooting for summer for hbase 2.0, the main
> problem
> > > is
> > > > > that
> > > > > > > >>>> there
> > > > > > > >>>>>> is
> > > > > > > >>>>>>> still no code for the new AM or for fs changes, which
> are
> > > the
> > > > > two
> > > > > > > >>>> that
> > > > > > > >>>>>> may
> > > > > > > >>>>>>> impact compatibility (working slowly on that).
> Streaming
> > > > > > > >>> replication
> > > > > > > >>>>> and
> > > > > > > >>>>>>> others seems compatible enough but no code there too.
> > > > > > > >>>>>>>
> > > > > > > >>>>>>> Matteo
> > > > > > > >>>>>>>
> > > > > > > >>>>>>>
> > > > > > > >>>>>>> On Thu, Feb 25, 2016 at 3:29 PM, Mikhail Antonov <
> > > > > > > >>>> olorinbant@gmail.com
> > > > > > > >>>>>>
> > > > > > > >>>>>>> wrote:
> > > > > > > >>>>>>>
> > > > > > > >>>>>>>> Agreed. I just meant - readiness of 2.0 is something
> > > > affecting
> > > > > > > >>>>>> decisions
> > > > > > > >>>>>>> on
> > > > > > > >>>>>>>> whether or not to backport mobs to branch-1 (which is
> > > itself
> > > > > > > >>>> separate
> > > > > > > >>>>>>>> thread).
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>> -Mikhail
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>> On Thu, Feb 25, 2016 at 3:27 PM, Sean Busbey <
> > > > > > > >>> busbey@cloudera.com>
> > > > > > > >>>>>>> wrote:
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>>> On Thu, Feb 25, 2016 at 5:21 PM, Mikhail Antonov <
> > > > > > > >>>>>> olorinbant@gmail.com
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>>> wrote:
> > > > > > > >>>>>>>>>
> > > > > > > >>>>>>>>>>
> > > > > > > >>>>>>>>>> - "Shouldn't we rather try to get 2.0 release out
> and
> > > have
> > > > > > > >>>> mobs
> > > > > > > >>>>>>>> there".
> > > > > > > >>>>>>>>> -
> > > > > > > >>>>>>>>>> So how far do we feel 2.0 release is?
> > > > > > > >>>>>>>>>
> > > > > > > >>>>>>>>> 2.0 readiness probably deserves its own [DISCUSS]
> > thread,
> > > > but
> > > > > > > >>>> we're
> > > > > > > >>>>>> now
> > > > > > > >>>>>>>>> past a year since the HBase 1.0.0 release, so I hope
> > it's
> > > > > > > >> soon.
> > > > > > > >>>>>>>>>
> > > > > > > >>>>>>>>>
> > > > > > > >>>>>>>>>
> > > > > > > >>>>>>>>> --
> > > > > > > >>>>>>>>> Sean
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>> --
> > > > > > > >>>>>>>> Thanks,
> > > > > > > >>>>>>>> Michael Antonov
> > > > > > > >>>>>>
> > > > > > > >>>>>>
> > > > > > > >>>>>>
> > > > > > > >>>>>> --
> > > > > > > >>>>>> Best regards,
> > > > > > > >>>>>>
> > > > > > > >>>>>> - Andy
> > > > > > > >>>>>>
> > > > > > > >>>>>> Problems worthy of attack prove their worth by hitting
> > > back. -
> > > > > > Piet
> > > > > > > >>>> Hein
> > > > > > > >>>>>> (via Tom White)
> > > > > > > >>>>>
> > > > > > > >>>>>
> > > > > > > >>>>>
> > > > > > > >>>>> --
> > > > > > > >>>>> Thanks,
> > > > > > > >>>>> Michael Antonov
> > > > > > > >>
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> --
> > > > > > > >> Best regards,
> > > > > > > >>
> > > > > > > >> - Andy
> > > > > > > >>
> > > > > > > >> Problems worthy of attack prove their worth by hitting
> back. -
> > > > Piet
> > > > > > Hein
> > > > > > > >> (via Tom White)
> > > > > > > >>
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Thanks,
> > > > > > Michael Antonov
> > > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Thanks,
> > > Michael Antonov
> > >
> >
> >
> >
> > --
> > busbey
> >
>
>
>
> --
> Thanks,
> Michael Antonov
>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: Branch for 1.3
Posted by Mikhail Antonov <ol...@gmail.com>.
At this point branch-1.3 is very close (if different from at all) to
branch-1, so that's probably the same discussion.
-Mikhail
On Mon, Mar 21, 2016 at 11:47 AM, Sean Busbey <bu...@cloudera.com> wrote:
> Is the spark connector thread specifically about 1.3? or branch-1? because
> we already had the branch-1 conversation. the specific gates were tracked
> in the umbrella jira.
>
> -Sean
>
> On Mon, Mar 21, 2016 at 1:33 PM, Mikhail Antonov <ol...@gmail.com>
> wrote:
>
> > Yeah, we probably should start discussion thread about Spark connector.
> > Anyone wants to start the thread and push it forward?
> >
> > Regarding date-tiered compactions - since first impl already went in 1.3,
> > would be good to get any possible improvements in 1.3 as well, as long as
> > they are stable, IMO.
> >
> > -Mikhail
> >
> > On Mon, Mar 21, 2016 at 11:03 AM, Enis Söztutar <en...@gmail.com>
> > wrote:
> >
> > > You may want to track
> https://issues.apache.org/jira/browse/HBASE-15339
> > as
> > > a parent for date-tiered compaction improvements. Current compaction
> > policy
> > > is useful in its own, but handling existing data, bulk loading etc will
> > be
> > > improved with these subtasks. I think the patches can land before the
> 1.3
> > > timeframe, but of course open for discussion for inclusion. My vote
> would
> > > be to include all the improvements since it would be easier to tell the
> > > story to users.
> > >
> > > Enis
> > >
> > > On Mon, Mar 21, 2016 at 3:32 AM, Ted Yu <yu...@gmail.com> wrote:
> > >
> > > > For Spark connector, we should start a separate discussion thread
> about
> > > > backporting to branch-1.
> > > >
> > > > Zhan has a bug fix coming this week which deals with how negative
> > numbers
> > > > are handled in comparison.
> > > >
> > > > FYI
> > > >
> > > > On Mon, Mar 21, 2016 at 1:35 AM, Mikhail Antonov <
> olorinbant@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > Hi folks,
> > > > >
> > > > > bringing this topic up again. I'm planning to start spinning 1.3
> > builds
> > > > and
> > > > > see if/where they break in a week or two, and (depending on how it
> > > does)
> > > > > start preparing RCs in a month or maybe two. So, let's see where we
> > > are.
> > > > >
> > > > > Big items first. There were long debates around three big items -
> > MOBs,
> > > > > Spark connector and RS groups, whether we should have them or not.
> > > > >
> > > > > - MOBs
> > > > > I believe we decided that they aren't going to go in branch-1, and
> > > hence
> > > > > not in branch-1.3 for sure. We might get back to that debate and
> > > > > re-consider MOBs for branch-1 if 2.0 is delayed, but almost
> certainly
> > > > they
> > > > > won't make it in 1.3 timeframe anyway as I feel.
> > > > >
> > > > > - Spark connector - HBASE-14160 it looks like it has 3 subtasks
> open,
> > > one
> > > > > of which is big one (HBASE-14375) - define public API for Spark
> > > > > integration. From the Jira looks like active work is happening on
> > other
> > > > > subtasks, but not on this one. So how's public API going? How
> stable
> > it
> > > > is?
> > > > > Who would want to have Spark in 1.3 and willing to help with this?
> > > OTOH -
> > > > > who has objections about back-porting it? Has anyone been using it
> in
> > > > some
> > > > > real environment?
> > > > >
> > > > > - RS groups - there was recently a thread about them, I'd like to
> > > bring
> > > > it
> > > > > up again and get to some conclusion.
> > > > >
> > > > > Other features which we had in flight a month ago -
> > > > >
> > > > > - HBASE-15181 - date based tiered compactions has landed
> > > > > - HBASE-11290 - unlock RegionStates. I'm afraid the codebase has
> > moved
> > > > > forward quite a bit since the benchmark was run on this change :( -
> > > > Francis
> > > > > - are you using it now? If we could have some benchmarks on the
> > latest
> > > > > rebase that I think would be great.
> > > > >
> > > > >
> > > > > - HBASE-13557 - special handling for system tables WALs - should
> we
> > > > still
> > > > > keep it targeted for 1.3?
> > > > > - HBASE-13017 - keep table state in meta, this one doesn't look
> like
> > > > it's
> > > > > going to make it in
> > > > >
> > > > > As a new item on my list - I'm looking forward to see more of
> > > HBASE-15492
> > > > > (memory optimizations) subtasks to go in branch-1.
> > > > >
> > > > > Thanks!
> > > > > Mikhail
> > > > >
> > > > > On Sun, Feb 28, 2016 at 10:53 AM, Andrew Purtell <
> > > > andrew.purtell@gmail.com
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > I'm starting a push at work to get us up on 1.2. Assuming that
> > > happens
> > > > > > later this year I think that will be the end of my close
> attention
> > to
> > > > > 0.98.
> > > > > >
> > > > > > > On Feb 26, 2016, at 1:54 PM, Nick Dimiduk <nd...@gmail.com>
> > > > wrote:
> > > > > > >
> > > > > > > On Fri, Feb 26, 2016 at 12:23 PM, Andrew Purtell <
> > > > apurtell@apache.org>
> > > > > > > wrote:
> > > > > > >
> > > > > > >> In the meantime those of us running HBase in production would
> > > > benefit
> > > > > > from
> > > > > > >> fairly frequent minor releases.
> > > > > > >
> > > > > > > +1. Having to look back to 0.98 to get some new feature is
> > > > problematic.
> > > > > > >
> > > > > > >> On Fri, Feb 26, 2016 at 9:50 AM, Elliott Clark <
> > eclark@apache.org
> > > >
> > > > > > wrote:
> > > > > > >>
> > > > > > >>> I disagree. We have agreed that 2.0 will have a new
> assignement
> > > > > > manager.
> > > > > > >>> There's a lot of work that has been done on getting that in,
> so
> > > far
> > > > > > there
> > > > > > >>> are no benefits to the end user from all that work. We should
> > > stick
> > > > > > with
> > > > > > >>> the plan and release 2.0 when it's ready.
> > > > > > >>>
> > > > > > >>> On Fri, Feb 26, 2016 at 9:39 AM, Stephen Jiang <
> > > > > > syuanjiangdev@gmail.com>
> > > > > > >>> wrote:
> > > > > > >>>
> > > > > > >>>> Thanks for Mikhail for taking the 1.3 RM role. Looks like
> we
> > > > have a
> > > > > > >> lot
> > > > > > >>> of
> > > > > > >>>> new things in 1.3 release.
> > > > > > >>>>
> > > > > > >>>> Based on the experience of 1.1 and 1.2 release, it takes a
> lot
> > > of
> > > > > > >> efforts
> > > > > > >>>> to get a stable minor release out. From this, I have my own
> > > > 2-cents
> > > > > > on
> > > > > > >>> 1.4
> > > > > > >>>> release. The plan is to have 2.0 release during summer time
> > of
> > > > this
> > > > > > >> year
> > > > > > >>>> (yeah, *this year). * Given the limited time and resource,
> > > after
> > > > > 1.3
> > > > > > >>>> release, instead of spending effort on 1.4 release, the
> > > community
> > > > > > >> should
> > > > > > >>>> focus on stabilizing master (or branch-2, not exist as of
> now)
> > > > > branch
> > > > > > >> and
> > > > > > >>>> make 2.0 release a priority. 2.0 release would bring more
> > > values
> > > > to
> > > > > > >>>> customer & move towards maturity of HBASE product.
> > > > > > >>>>
> > > > > > >>>> Thanks
> > > > > > >>>> Stephen
> > > > > > >>>>
> > > > > > >>>> On Fri, Feb 26, 2016 at 1:49 AM, Mikhail Antonov <
> > > > > > olorinbant@gmail.com
> > > > > > >>>
> > > > > > >>>> wrote:
> > > > > > >>>>
> > > > > > >>>>> Created an umbrella jira for 1.3 release - HBASE-15341
> > > > > > >>>>>
> > > > > > >>>>> So it looks like we may have 1.4 release before 2.0 is
> out? I
> > > > tried
> > > > > > >> to
> > > > > > >>>> add
> > > > > > >>>>> 1.4 version in jira so we can keep it in branch-1 poms but
> I
> > > > > > >> couldn't -
> > > > > > >>>>> looks like I don't have permissions?
> > > > > > >>>>>
> > > > > > >>>>> -Mikhail
> > > > > > >>>>>
> > > > > > >>>>> On Thu, Feb 25, 2016 at 3:52 PM, Andrew Purtell <
> > > > > apurtell@apache.org
> > > > > > >>>
> > > > > > >>>>> wrote:
> > > > > > >>>>>
> > > > > > >>>>>> The guy we had looking at streaming replication moved on
> and
> > > > > > >> there's
> > > > > > >>> no
> > > > > > >>>>>> immediate plans to take on the work, FWIW
> > > > > > >>>>>>
> > > > > > >>>>>> On Thu, Feb 25, 2016 at 3:33 PM, Matteo Bertozzi <
> > > > > > >>>>> theo.bertozzi@gmail.com>
> > > > > > >>>>>> wrote:
> > > > > > >>>>>>
> > > > > > >>>>>>> I was shooting for summer for hbase 2.0, the main problem
> > is
> > > > that
> > > > > > >>>> there
> > > > > > >>>>>> is
> > > > > > >>>>>>> still no code for the new AM or for fs changes, which are
> > the
> > > > two
> > > > > > >>>> that
> > > > > > >>>>>> may
> > > > > > >>>>>>> impact compatibility (working slowly on that). Streaming
> > > > > > >>> replication
> > > > > > >>>>> and
> > > > > > >>>>>>> others seems compatible enough but no code there too.
> > > > > > >>>>>>>
> > > > > > >>>>>>> Matteo
> > > > > > >>>>>>>
> > > > > > >>>>>>>
> > > > > > >>>>>>> On Thu, Feb 25, 2016 at 3:29 PM, Mikhail Antonov <
> > > > > > >>>> olorinbant@gmail.com
> > > > > > >>>>>>
> > > > > > >>>>>>> wrote:
> > > > > > >>>>>>>
> > > > > > >>>>>>>> Agreed. I just meant - readiness of 2.0 is something
> > > affecting
> > > > > > >>>>>> decisions
> > > > > > >>>>>>> on
> > > > > > >>>>>>>> whether or not to backport mobs to branch-1 (which is
> > itself
> > > > > > >>>> separate
> > > > > > >>>>>>>> thread).
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> -Mikhail
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> On Thu, Feb 25, 2016 at 3:27 PM, Sean Busbey <
> > > > > > >>> busbey@cloudera.com>
> > > > > > >>>>>>> wrote:
> > > > > > >>>>>>>>
> > > > > > >>>>>>>>> On Thu, Feb 25, 2016 at 5:21 PM, Mikhail Antonov <
> > > > > > >>>>>> olorinbant@gmail.com
> > > > > > >>>>>>>>
> > > > > > >>>>>>>>> wrote:
> > > > > > >>>>>>>>>
> > > > > > >>>>>>>>>>
> > > > > > >>>>>>>>>> - "Shouldn't we rather try to get 2.0 release out and
> > have
> > > > > > >>>> mobs
> > > > > > >>>>>>>> there".
> > > > > > >>>>>>>>> -
> > > > > > >>>>>>>>>> So how far do we feel 2.0 release is?
> > > > > > >>>>>>>>>
> > > > > > >>>>>>>>> 2.0 readiness probably deserves its own [DISCUSS]
> thread,
> > > but
> > > > > > >>>> we're
> > > > > > >>>>>> now
> > > > > > >>>>>>>>> past a year since the HBase 1.0.0 release, so I hope
> it's
> > > > > > >> soon.
> > > > > > >>>>>>>>>
> > > > > > >>>>>>>>>
> > > > > > >>>>>>>>>
> > > > > > >>>>>>>>> --
> > > > > > >>>>>>>>> Sean
> > > > > > >>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> --
> > > > > > >>>>>>>> Thanks,
> > > > > > >>>>>>>> Michael Antonov
> > > > > > >>>>>>
> > > > > > >>>>>>
> > > > > > >>>>>>
> > > > > > >>>>>> --
> > > > > > >>>>>> Best regards,
> > > > > > >>>>>>
> > > > > > >>>>>> - Andy
> > > > > > >>>>>>
> > > > > > >>>>>> Problems worthy of attack prove their worth by hitting
> > back. -
> > > > > Piet
> > > > > > >>>> Hein
> > > > > > >>>>>> (via Tom White)
> > > > > > >>>>>
> > > > > > >>>>>
> > > > > > >>>>>
> > > > > > >>>>> --
> > > > > > >>>>> Thanks,
> > > > > > >>>>> Michael Antonov
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >> --
> > > > > > >> Best regards,
> > > > > > >>
> > > > > > >> - Andy
> > > > > > >>
> > > > > > >> Problems worthy of attack prove their worth by hitting back. -
> > > Piet
> > > > > Hein
> > > > > > >> (via Tom White)
> > > > > > >>
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Thanks,
> > > > > Michael Antonov
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > Thanks,
> > Michael Antonov
> >
>
>
>
> --
> busbey
>
--
Thanks,
Michael Antonov
Re: Branch for 1.3
Posted by Sean Busbey <bu...@cloudera.com>.
Is the spark connector thread specifically about 1.3? or branch-1? because
we already had the branch-1 conversation. the specific gates were tracked
in the umbrella jira.
-Sean
On Mon, Mar 21, 2016 at 1:33 PM, Mikhail Antonov <ol...@gmail.com>
wrote:
> Yeah, we probably should start discussion thread about Spark connector.
> Anyone wants to start the thread and push it forward?
>
> Regarding date-tiered compactions - since first impl already went in 1.3,
> would be good to get any possible improvements in 1.3 as well, as long as
> they are stable, IMO.
>
> -Mikhail
>
> On Mon, Mar 21, 2016 at 11:03 AM, Enis Söztutar <en...@gmail.com>
> wrote:
>
> > You may want to track https://issues.apache.org/jira/browse/HBASE-15339
> as
> > a parent for date-tiered compaction improvements. Current compaction
> policy
> > is useful in its own, but handling existing data, bulk loading etc will
> be
> > improved with these subtasks. I think the patches can land before the 1.3
> > timeframe, but of course open for discussion for inclusion. My vote would
> > be to include all the improvements since it would be easier to tell the
> > story to users.
> >
> > Enis
> >
> > On Mon, Mar 21, 2016 at 3:32 AM, Ted Yu <yu...@gmail.com> wrote:
> >
> > > For Spark connector, we should start a separate discussion thread about
> > > backporting to branch-1.
> > >
> > > Zhan has a bug fix coming this week which deals with how negative
> numbers
> > > are handled in comparison.
> > >
> > > FYI
> > >
> > > On Mon, Mar 21, 2016 at 1:35 AM, Mikhail Antonov <olorinbant@gmail.com
> >
> > > wrote:
> > >
> > > > Hi folks,
> > > >
> > > > bringing this topic up again. I'm planning to start spinning 1.3
> builds
> > > and
> > > > see if/where they break in a week or two, and (depending on how it
> > does)
> > > > start preparing RCs in a month or maybe two. So, let's see where we
> > are.
> > > >
> > > > Big items first. There were long debates around three big items -
> MOBs,
> > > > Spark connector and RS groups, whether we should have them or not.
> > > >
> > > > - MOBs
> > > > I believe we decided that they aren't going to go in branch-1, and
> > hence
> > > > not in branch-1.3 for sure. We might get back to that debate and
> > > > re-consider MOBs for branch-1 if 2.0 is delayed, but almost certainly
> > > they
> > > > won't make it in 1.3 timeframe anyway as I feel.
> > > >
> > > > - Spark connector - HBASE-14160 it looks like it has 3 subtasks open,
> > one
> > > > of which is big one (HBASE-14375) - define public API for Spark
> > > > integration. From the Jira looks like active work is happening on
> other
> > > > subtasks, but not on this one. So how's public API going? How stable
> it
> > > is?
> > > > Who would want to have Spark in 1.3 and willing to help with this?
> > OTOH -
> > > > who has objections about back-porting it? Has anyone been using it in
> > > some
> > > > real environment?
> > > >
> > > > - RS groups - there was recently a thread about them, I'd like to
> > bring
> > > it
> > > > up again and get to some conclusion.
> > > >
> > > > Other features which we had in flight a month ago -
> > > >
> > > > - HBASE-15181 - date based tiered compactions has landed
> > > > - HBASE-11290 - unlock RegionStates. I'm afraid the codebase has
> moved
> > > > forward quite a bit since the benchmark was run on this change :( -
> > > Francis
> > > > - are you using it now? If we could have some benchmarks on the
> latest
> > > > rebase that I think would be great.
> > > >
> > > >
> > > > - HBASE-13557 - special handling for system tables WALs - should we
> > > still
> > > > keep it targeted for 1.3?
> > > > - HBASE-13017 - keep table state in meta, this one doesn't look like
> > > it's
> > > > going to make it in
> > > >
> > > > As a new item on my list - I'm looking forward to see more of
> > HBASE-15492
> > > > (memory optimizations) subtasks to go in branch-1.
> > > >
> > > > Thanks!
> > > > Mikhail
> > > >
> > > > On Sun, Feb 28, 2016 at 10:53 AM, Andrew Purtell <
> > > andrew.purtell@gmail.com
> > > > >
> > > > wrote:
> > > >
> > > > > I'm starting a push at work to get us up on 1.2. Assuming that
> > happens
> > > > > later this year I think that will be the end of my close attention
> to
> > > > 0.98.
> > > > >
> > > > > > On Feb 26, 2016, at 1:54 PM, Nick Dimiduk <nd...@gmail.com>
> > > wrote:
> > > > > >
> > > > > > On Fri, Feb 26, 2016 at 12:23 PM, Andrew Purtell <
> > > apurtell@apache.org>
> > > > > > wrote:
> > > > > >
> > > > > >> In the meantime those of us running HBase in production would
> > > benefit
> > > > > from
> > > > > >> fairly frequent minor releases.
> > > > > >
> > > > > > +1. Having to look back to 0.98 to get some new feature is
> > > problematic.
> > > > > >
> > > > > >> On Fri, Feb 26, 2016 at 9:50 AM, Elliott Clark <
> eclark@apache.org
> > >
> > > > > wrote:
> > > > > >>
> > > > > >>> I disagree. We have agreed that 2.0 will have a new assignement
> > > > > manager.
> > > > > >>> There's a lot of work that has been done on getting that in, so
> > far
> > > > > there
> > > > > >>> are no benefits to the end user from all that work. We should
> > stick
> > > > > with
> > > > > >>> the plan and release 2.0 when it's ready.
> > > > > >>>
> > > > > >>> On Fri, Feb 26, 2016 at 9:39 AM, Stephen Jiang <
> > > > > syuanjiangdev@gmail.com>
> > > > > >>> wrote:
> > > > > >>>
> > > > > >>>> Thanks for Mikhail for taking the 1.3 RM role. Looks like we
> > > have a
> > > > > >> lot
> > > > > >>> of
> > > > > >>>> new things in 1.3 release.
> > > > > >>>>
> > > > > >>>> Based on the experience of 1.1 and 1.2 release, it takes a lot
> > of
> > > > > >> efforts
> > > > > >>>> to get a stable minor release out. From this, I have my own
> > > 2-cents
> > > > > on
> > > > > >>> 1.4
> > > > > >>>> release. The plan is to have 2.0 release during summer time
> of
> > > this
> > > > > >> year
> > > > > >>>> (yeah, *this year). * Given the limited time and resource,
> > after
> > > > 1.3
> > > > > >>>> release, instead of spending effort on 1.4 release, the
> > community
> > > > > >> should
> > > > > >>>> focus on stabilizing master (or branch-2, not exist as of now)
> > > > branch
> > > > > >> and
> > > > > >>>> make 2.0 release a priority. 2.0 release would bring more
> > values
> > > to
> > > > > >>>> customer & move towards maturity of HBASE product.
> > > > > >>>>
> > > > > >>>> Thanks
> > > > > >>>> Stephen
> > > > > >>>>
> > > > > >>>> On Fri, Feb 26, 2016 at 1:49 AM, Mikhail Antonov <
> > > > > olorinbant@gmail.com
> > > > > >>>
> > > > > >>>> wrote:
> > > > > >>>>
> > > > > >>>>> Created an umbrella jira for 1.3 release - HBASE-15341
> > > > > >>>>>
> > > > > >>>>> So it looks like we may have 1.4 release before 2.0 is out? I
> > > tried
> > > > > >> to
> > > > > >>>> add
> > > > > >>>>> 1.4 version in jira so we can keep it in branch-1 poms but I
> > > > > >> couldn't -
> > > > > >>>>> looks like I don't have permissions?
> > > > > >>>>>
> > > > > >>>>> -Mikhail
> > > > > >>>>>
> > > > > >>>>> On Thu, Feb 25, 2016 at 3:52 PM, Andrew Purtell <
> > > > apurtell@apache.org
> > > > > >>>
> > > > > >>>>> wrote:
> > > > > >>>>>
> > > > > >>>>>> The guy we had looking at streaming replication moved on and
> > > > > >> there's
> > > > > >>> no
> > > > > >>>>>> immediate plans to take on the work, FWIW
> > > > > >>>>>>
> > > > > >>>>>> On Thu, Feb 25, 2016 at 3:33 PM, Matteo Bertozzi <
> > > > > >>>>> theo.bertozzi@gmail.com>
> > > > > >>>>>> wrote:
> > > > > >>>>>>
> > > > > >>>>>>> I was shooting for summer for hbase 2.0, the main problem
> is
> > > that
> > > > > >>>> there
> > > > > >>>>>> is
> > > > > >>>>>>> still no code for the new AM or for fs changes, which are
> the
> > > two
> > > > > >>>> that
> > > > > >>>>>> may
> > > > > >>>>>>> impact compatibility (working slowly on that). Streaming
> > > > > >>> replication
> > > > > >>>>> and
> > > > > >>>>>>> others seems compatible enough but no code there too.
> > > > > >>>>>>>
> > > > > >>>>>>> Matteo
> > > > > >>>>>>>
> > > > > >>>>>>>
> > > > > >>>>>>> On Thu, Feb 25, 2016 at 3:29 PM, Mikhail Antonov <
> > > > > >>>> olorinbant@gmail.com
> > > > > >>>>>>
> > > > > >>>>>>> wrote:
> > > > > >>>>>>>
> > > > > >>>>>>>> Agreed. I just meant - readiness of 2.0 is something
> > affecting
> > > > > >>>>>> decisions
> > > > > >>>>>>> on
> > > > > >>>>>>>> whether or not to backport mobs to branch-1 (which is
> itself
> > > > > >>>> separate
> > > > > >>>>>>>> thread).
> > > > > >>>>>>>>
> > > > > >>>>>>>> -Mikhail
> > > > > >>>>>>>>
> > > > > >>>>>>>> On Thu, Feb 25, 2016 at 3:27 PM, Sean Busbey <
> > > > > >>> busbey@cloudera.com>
> > > > > >>>>>>> wrote:
> > > > > >>>>>>>>
> > > > > >>>>>>>>> On Thu, Feb 25, 2016 at 5:21 PM, Mikhail Antonov <
> > > > > >>>>>> olorinbant@gmail.com
> > > > > >>>>>>>>
> > > > > >>>>>>>>> wrote:
> > > > > >>>>>>>>>
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>> - "Shouldn't we rather try to get 2.0 release out and
> have
> > > > > >>>> mobs
> > > > > >>>>>>>> there".
> > > > > >>>>>>>>> -
> > > > > >>>>>>>>>> So how far do we feel 2.0 release is?
> > > > > >>>>>>>>>
> > > > > >>>>>>>>> 2.0 readiness probably deserves its own [DISCUSS] thread,
> > but
> > > > > >>>> we're
> > > > > >>>>>> now
> > > > > >>>>>>>>> past a year since the HBase 1.0.0 release, so I hope it's
> > > > > >> soon.
> > > > > >>>>>>>>>
> > > > > >>>>>>>>>
> > > > > >>>>>>>>>
> > > > > >>>>>>>>> --
> > > > > >>>>>>>>> Sean
> > > > > >>>>>>>>
> > > > > >>>>>>>>
> > > > > >>>>>>>>
> > > > > >>>>>>>> --
> > > > > >>>>>>>> Thanks,
> > > > > >>>>>>>> Michael Antonov
> > > > > >>>>>>
> > > > > >>>>>>
> > > > > >>>>>>
> > > > > >>>>>> --
> > > > > >>>>>> Best regards,
> > > > > >>>>>>
> > > > > >>>>>> - Andy
> > > > > >>>>>>
> > > > > >>>>>> Problems worthy of attack prove their worth by hitting
> back. -
> > > > Piet
> > > > > >>>> Hein
> > > > > >>>>>> (via Tom White)
> > > > > >>>>>
> > > > > >>>>>
> > > > > >>>>>
> > > > > >>>>> --
> > > > > >>>>> Thanks,
> > > > > >>>>> Michael Antonov
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> --
> > > > > >> Best regards,
> > > > > >>
> > > > > >> - Andy
> > > > > >>
> > > > > >> Problems worthy of attack prove their worth by hitting back. -
> > Piet
> > > > Hein
> > > > > >> (via Tom White)
> > > > > >>
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Thanks,
> > > > Michael Antonov
> > > >
> > >
> >
>
>
>
> --
> Thanks,
> Michael Antonov
>
--
busbey
Re: Branch for 1.3
Posted by Mikhail Antonov <ol...@gmail.com>.
Yeah, we probably should start discussion thread about Spark connector.
Anyone wants to start the thread and push it forward?
Regarding date-tiered compactions - since first impl already went in 1.3,
would be good to get any possible improvements in 1.3 as well, as long as
they are stable, IMO.
-Mikhail
On Mon, Mar 21, 2016 at 11:03 AM, Enis Söztutar <en...@gmail.com> wrote:
> You may want to track https://issues.apache.org/jira/browse/HBASE-15339 as
> a parent for date-tiered compaction improvements. Current compaction policy
> is useful in its own, but handling existing data, bulk loading etc will be
> improved with these subtasks. I think the patches can land before the 1.3
> timeframe, but of course open for discussion for inclusion. My vote would
> be to include all the improvements since it would be easier to tell the
> story to users.
>
> Enis
>
> On Mon, Mar 21, 2016 at 3:32 AM, Ted Yu <yu...@gmail.com> wrote:
>
> > For Spark connector, we should start a separate discussion thread about
> > backporting to branch-1.
> >
> > Zhan has a bug fix coming this week which deals with how negative numbers
> > are handled in comparison.
> >
> > FYI
> >
> > On Mon, Mar 21, 2016 at 1:35 AM, Mikhail Antonov <ol...@gmail.com>
> > wrote:
> >
> > > Hi folks,
> > >
> > > bringing this topic up again. I'm planning to start spinning 1.3 builds
> > and
> > > see if/where they break in a week or two, and (depending on how it
> does)
> > > start preparing RCs in a month or maybe two. So, let's see where we
> are.
> > >
> > > Big items first. There were long debates around three big items - MOBs,
> > > Spark connector and RS groups, whether we should have them or not.
> > >
> > > - MOBs
> > > I believe we decided that they aren't going to go in branch-1, and
> hence
> > > not in branch-1.3 for sure. We might get back to that debate and
> > > re-consider MOBs for branch-1 if 2.0 is delayed, but almost certainly
> > they
> > > won't make it in 1.3 timeframe anyway as I feel.
> > >
> > > - Spark connector - HBASE-14160 it looks like it has 3 subtasks open,
> one
> > > of which is big one (HBASE-14375) - define public API for Spark
> > > integration. From the Jira looks like active work is happening on other
> > > subtasks, but not on this one. So how's public API going? How stable it
> > is?
> > > Who would want to have Spark in 1.3 and willing to help with this?
> OTOH -
> > > who has objections about back-porting it? Has anyone been using it in
> > some
> > > real environment?
> > >
> > > - RS groups - there was recently a thread about them, I'd like to
> bring
> > it
> > > up again and get to some conclusion.
> > >
> > > Other features which we had in flight a month ago -
> > >
> > > - HBASE-15181 - date based tiered compactions has landed
> > > - HBASE-11290 - unlock RegionStates. I'm afraid the codebase has moved
> > > forward quite a bit since the benchmark was run on this change :( -
> > Francis
> > > - are you using it now? If we could have some benchmarks on the latest
> > > rebase that I think would be great.
> > >
> > >
> > > - HBASE-13557 - special handling for system tables WALs - should we
> > still
> > > keep it targeted for 1.3?
> > > - HBASE-13017 - keep table state in meta, this one doesn't look like
> > it's
> > > going to make it in
> > >
> > > As a new item on my list - I'm looking forward to see more of
> HBASE-15492
> > > (memory optimizations) subtasks to go in branch-1.
> > >
> > > Thanks!
> > > Mikhail
> > >
> > > On Sun, Feb 28, 2016 at 10:53 AM, Andrew Purtell <
> > andrew.purtell@gmail.com
> > > >
> > > wrote:
> > >
> > > > I'm starting a push at work to get us up on 1.2. Assuming that
> happens
> > > > later this year I think that will be the end of my close attention to
> > > 0.98.
> > > >
> > > > > On Feb 26, 2016, at 1:54 PM, Nick Dimiduk <nd...@gmail.com>
> > wrote:
> > > > >
> > > > > On Fri, Feb 26, 2016 at 12:23 PM, Andrew Purtell <
> > apurtell@apache.org>
> > > > > wrote:
> > > > >
> > > > >> In the meantime those of us running HBase in production would
> > benefit
> > > > from
> > > > >> fairly frequent minor releases.
> > > > >
> > > > > +1. Having to look back to 0.98 to get some new feature is
> > problematic.
> > > > >
> > > > >> On Fri, Feb 26, 2016 at 9:50 AM, Elliott Clark <eclark@apache.org
> >
> > > > wrote:
> > > > >>
> > > > >>> I disagree. We have agreed that 2.0 will have a new assignement
> > > > manager.
> > > > >>> There's a lot of work that has been done on getting that in, so
> far
> > > > there
> > > > >>> are no benefits to the end user from all that work. We should
> stick
> > > > with
> > > > >>> the plan and release 2.0 when it's ready.
> > > > >>>
> > > > >>> On Fri, Feb 26, 2016 at 9:39 AM, Stephen Jiang <
> > > > syuanjiangdev@gmail.com>
> > > > >>> wrote:
> > > > >>>
> > > > >>>> Thanks for Mikhail for taking the 1.3 RM role. Looks like we
> > have a
> > > > >> lot
> > > > >>> of
> > > > >>>> new things in 1.3 release.
> > > > >>>>
> > > > >>>> Based on the experience of 1.1 and 1.2 release, it takes a lot
> of
> > > > >> efforts
> > > > >>>> to get a stable minor release out. From this, I have my own
> > 2-cents
> > > > on
> > > > >>> 1.4
> > > > >>>> release. The plan is to have 2.0 release during summer time of
> > this
> > > > >> year
> > > > >>>> (yeah, *this year). * Given the limited time and resource,
> after
> > > 1.3
> > > > >>>> release, instead of spending effort on 1.4 release, the
> community
> > > > >> should
> > > > >>>> focus on stabilizing master (or branch-2, not exist as of now)
> > > branch
> > > > >> and
> > > > >>>> make 2.0 release a priority. 2.0 release would bring more
> values
> > to
> > > > >>>> customer & move towards maturity of HBASE product.
> > > > >>>>
> > > > >>>> Thanks
> > > > >>>> Stephen
> > > > >>>>
> > > > >>>> On Fri, Feb 26, 2016 at 1:49 AM, Mikhail Antonov <
> > > > olorinbant@gmail.com
> > > > >>>
> > > > >>>> wrote:
> > > > >>>>
> > > > >>>>> Created an umbrella jira for 1.3 release - HBASE-15341
> > > > >>>>>
> > > > >>>>> So it looks like we may have 1.4 release before 2.0 is out? I
> > tried
> > > > >> to
> > > > >>>> add
> > > > >>>>> 1.4 version in jira so we can keep it in branch-1 poms but I
> > > > >> couldn't -
> > > > >>>>> looks like I don't have permissions?
> > > > >>>>>
> > > > >>>>> -Mikhail
> > > > >>>>>
> > > > >>>>> On Thu, Feb 25, 2016 at 3:52 PM, Andrew Purtell <
> > > apurtell@apache.org
> > > > >>>
> > > > >>>>> wrote:
> > > > >>>>>
> > > > >>>>>> The guy we had looking at streaming replication moved on and
> > > > >> there's
> > > > >>> no
> > > > >>>>>> immediate plans to take on the work, FWIW
> > > > >>>>>>
> > > > >>>>>> On Thu, Feb 25, 2016 at 3:33 PM, Matteo Bertozzi <
> > > > >>>>> theo.bertozzi@gmail.com>
> > > > >>>>>> wrote:
> > > > >>>>>>
> > > > >>>>>>> I was shooting for summer for hbase 2.0, the main problem is
> > that
> > > > >>>> there
> > > > >>>>>> is
> > > > >>>>>>> still no code for the new AM or for fs changes, which are the
> > two
> > > > >>>> that
> > > > >>>>>> may
> > > > >>>>>>> impact compatibility (working slowly on that). Streaming
> > > > >>> replication
> > > > >>>>> and
> > > > >>>>>>> others seems compatible enough but no code there too.
> > > > >>>>>>>
> > > > >>>>>>> Matteo
> > > > >>>>>>>
> > > > >>>>>>>
> > > > >>>>>>> On Thu, Feb 25, 2016 at 3:29 PM, Mikhail Antonov <
> > > > >>>> olorinbant@gmail.com
> > > > >>>>>>
> > > > >>>>>>> wrote:
> > > > >>>>>>>
> > > > >>>>>>>> Agreed. I just meant - readiness of 2.0 is something
> affecting
> > > > >>>>>> decisions
> > > > >>>>>>> on
> > > > >>>>>>>> whether or not to backport mobs to branch-1 (which is itself
> > > > >>>> separate
> > > > >>>>>>>> thread).
> > > > >>>>>>>>
> > > > >>>>>>>> -Mikhail
> > > > >>>>>>>>
> > > > >>>>>>>> On Thu, Feb 25, 2016 at 3:27 PM, Sean Busbey <
> > > > >>> busbey@cloudera.com>
> > > > >>>>>>> wrote:
> > > > >>>>>>>>
> > > > >>>>>>>>> On Thu, Feb 25, 2016 at 5:21 PM, Mikhail Antonov <
> > > > >>>>>> olorinbant@gmail.com
> > > > >>>>>>>>
> > > > >>>>>>>>> wrote:
> > > > >>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> - "Shouldn't we rather try to get 2.0 release out and have
> > > > >>>> mobs
> > > > >>>>>>>> there".
> > > > >>>>>>>>> -
> > > > >>>>>>>>>> So how far do we feel 2.0 release is?
> > > > >>>>>>>>>
> > > > >>>>>>>>> 2.0 readiness probably deserves its own [DISCUSS] thread,
> but
> > > > >>>> we're
> > > > >>>>>> now
> > > > >>>>>>>>> past a year since the HBase 1.0.0 release, so I hope it's
> > > > >> soon.
> > > > >>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>> --
> > > > >>>>>>>>> Sean
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>> --
> > > > >>>>>>>> Thanks,
> > > > >>>>>>>> Michael Antonov
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>> --
> > > > >>>>>> Best regards,
> > > > >>>>>>
> > > > >>>>>> - Andy
> > > > >>>>>>
> > > > >>>>>> Problems worthy of attack prove their worth by hitting back. -
> > > Piet
> > > > >>>> Hein
> > > > >>>>>> (via Tom White)
> > > > >>>>>
> > > > >>>>>
> > > > >>>>>
> > > > >>>>> --
> > > > >>>>> Thanks,
> > > > >>>>> Michael Antonov
> > > > >>
> > > > >>
> > > > >>
> > > > >> --
> > > > >> Best regards,
> > > > >>
> > > > >> - Andy
> > > > >>
> > > > >> Problems worthy of attack prove their worth by hitting back. -
> Piet
> > > Hein
> > > > >> (via Tom White)
> > > > >>
> > > >
> > >
> > >
> > >
> > > --
> > > Thanks,
> > > Michael Antonov
> > >
> >
>
--
Thanks,
Michael Antonov
Re: Branch for 1.3
Posted by 张铎 <pa...@gmail.com>.
At least include HBASE-15400 in 1.3 please...
Otherwise these things can only be integrated in 1.4 since DTCP can not be
used together with DefaultStoreEngine after HBASE-15400 getting in.
2016-03-22 2:03 GMT+08:00 Enis Söztutar <en...@gmail.com>:
> You may want to track https://issues.apache.org/jira/browse/HBASE-15339 as
> a parent for date-tiered compaction improvements. Current compaction policy
> is useful in its own, but handling existing data, bulk loading etc will be
> improved with these subtasks. I think the patches can land before the 1.3
> timeframe, but of course open for discussion for inclusion. My vote would
> be to include all the improvements since it would be easier to tell the
> story to users.
>
> Enis
>
> On Mon, Mar 21, 2016 at 3:32 AM, Ted Yu <yu...@gmail.com> wrote:
>
> > For Spark connector, we should start a separate discussion thread about
> > backporting to branch-1.
> >
> > Zhan has a bug fix coming this week which deals with how negative numbers
> > are handled in comparison.
> >
> > FYI
> >
> > On Mon, Mar 21, 2016 at 1:35 AM, Mikhail Antonov <ol...@gmail.com>
> > wrote:
> >
> > > Hi folks,
> > >
> > > bringing this topic up again. I'm planning to start spinning 1.3 builds
> > and
> > > see if/where they break in a week or two, and (depending on how it
> does)
> > > start preparing RCs in a month or maybe two. So, let's see where we
> are.
> > >
> > > Big items first. There were long debates around three big items - MOBs,
> > > Spark connector and RS groups, whether we should have them or not.
> > >
> > > - MOBs
> > > I believe we decided that they aren't going to go in branch-1, and
> hence
> > > not in branch-1.3 for sure. We might get back to that debate and
> > > re-consider MOBs for branch-1 if 2.0 is delayed, but almost certainly
> > they
> > > won't make it in 1.3 timeframe anyway as I feel.
> > >
> > > - Spark connector - HBASE-14160 it looks like it has 3 subtasks open,
> one
> > > of which is big one (HBASE-14375) - define public API for Spark
> > > integration. From the Jira looks like active work is happening on other
> > > subtasks, but not on this one. So how's public API going? How stable it
> > is?
> > > Who would want to have Spark in 1.3 and willing to help with this?
> OTOH -
> > > who has objections about back-porting it? Has anyone been using it in
> > some
> > > real environment?
> > >
> > > - RS groups - there was recently a thread about them, I'd like to
> bring
> > it
> > > up again and get to some conclusion.
> > >
> > > Other features which we had in flight a month ago -
> > >
> > > - HBASE-15181 - date based tiered compactions has landed
> > > - HBASE-11290 - unlock RegionStates. I'm afraid the codebase has moved
> > > forward quite a bit since the benchmark was run on this change :( -
> > Francis
> > > - are you using it now? If we could have some benchmarks on the latest
> > > rebase that I think would be great.
> > >
> > >
> > > - HBASE-13557 - special handling for system tables WALs - should we
> > still
> > > keep it targeted for 1.3?
> > > - HBASE-13017 - keep table state in meta, this one doesn't look like
> > it's
> > > going to make it in
> > >
> > > As a new item on my list - I'm looking forward to see more of
> HBASE-15492
> > > (memory optimizations) subtasks to go in branch-1.
> > >
> > > Thanks!
> > > Mikhail
> > >
> > > On Sun, Feb 28, 2016 at 10:53 AM, Andrew Purtell <
> > andrew.purtell@gmail.com
> > > >
> > > wrote:
> > >
> > > > I'm starting a push at work to get us up on 1.2. Assuming that
> happens
> > > > later this year I think that will be the end of my close attention to
> > > 0.98.
> > > >
> > > > > On Feb 26, 2016, at 1:54 PM, Nick Dimiduk <nd...@gmail.com>
> > wrote:
> > > > >
> > > > > On Fri, Feb 26, 2016 at 12:23 PM, Andrew Purtell <
> > apurtell@apache.org>
> > > > > wrote:
> > > > >
> > > > >> In the meantime those of us running HBase in production would
> > benefit
> > > > from
> > > > >> fairly frequent minor releases.
> > > > >
> > > > > +1. Having to look back to 0.98 to get some new feature is
> > problematic.
> > > > >
> > > > >> On Fri, Feb 26, 2016 at 9:50 AM, Elliott Clark <eclark@apache.org
> >
> > > > wrote:
> > > > >>
> > > > >>> I disagree. We have agreed that 2.0 will have a new assignement
> > > > manager.
> > > > >>> There's a lot of work that has been done on getting that in, so
> far
> > > > there
> > > > >>> are no benefits to the end user from all that work. We should
> stick
> > > > with
> > > > >>> the plan and release 2.0 when it's ready.
> > > > >>>
> > > > >>> On Fri, Feb 26, 2016 at 9:39 AM, Stephen Jiang <
> > > > syuanjiangdev@gmail.com>
> > > > >>> wrote:
> > > > >>>
> > > > >>>> Thanks for Mikhail for taking the 1.3 RM role. Looks like we
> > have a
> > > > >> lot
> > > > >>> of
> > > > >>>> new things in 1.3 release.
> > > > >>>>
> > > > >>>> Based on the experience of 1.1 and 1.2 release, it takes a lot
> of
> > > > >> efforts
> > > > >>>> to get a stable minor release out. From this, I have my own
> > 2-cents
> > > > on
> > > > >>> 1.4
> > > > >>>> release. The plan is to have 2.0 release during summer time of
> > this
> > > > >> year
> > > > >>>> (yeah, *this year). * Given the limited time and resource,
> after
> > > 1.3
> > > > >>>> release, instead of spending effort on 1.4 release, the
> community
> > > > >> should
> > > > >>>> focus on stabilizing master (or branch-2, not exist as of now)
> > > branch
> > > > >> and
> > > > >>>> make 2.0 release a priority. 2.0 release would bring more
> values
> > to
> > > > >>>> customer & move towards maturity of HBASE product.
> > > > >>>>
> > > > >>>> Thanks
> > > > >>>> Stephen
> > > > >>>>
> > > > >>>> On Fri, Feb 26, 2016 at 1:49 AM, Mikhail Antonov <
> > > > olorinbant@gmail.com
> > > > >>>
> > > > >>>> wrote:
> > > > >>>>
> > > > >>>>> Created an umbrella jira for 1.3 release - HBASE-15341
> > > > >>>>>
> > > > >>>>> So it looks like we may have 1.4 release before 2.0 is out? I
> > tried
> > > > >> to
> > > > >>>> add
> > > > >>>>> 1.4 version in jira so we can keep it in branch-1 poms but I
> > > > >> couldn't -
> > > > >>>>> looks like I don't have permissions?
> > > > >>>>>
> > > > >>>>> -Mikhail
> > > > >>>>>
> > > > >>>>> On Thu, Feb 25, 2016 at 3:52 PM, Andrew Purtell <
> > > apurtell@apache.org
> > > > >>>
> > > > >>>>> wrote:
> > > > >>>>>
> > > > >>>>>> The guy we had looking at streaming replication moved on and
> > > > >> there's
> > > > >>> no
> > > > >>>>>> immediate plans to take on the work, FWIW
> > > > >>>>>>
> > > > >>>>>> On Thu, Feb 25, 2016 at 3:33 PM, Matteo Bertozzi <
> > > > >>>>> theo.bertozzi@gmail.com>
> > > > >>>>>> wrote:
> > > > >>>>>>
> > > > >>>>>>> I was shooting for summer for hbase 2.0, the main problem is
> > that
> > > > >>>> there
> > > > >>>>>> is
> > > > >>>>>>> still no code for the new AM or for fs changes, which are the
> > two
> > > > >>>> that
> > > > >>>>>> may
> > > > >>>>>>> impact compatibility (working slowly on that). Streaming
> > > > >>> replication
> > > > >>>>> and
> > > > >>>>>>> others seems compatible enough but no code there too.
> > > > >>>>>>>
> > > > >>>>>>> Matteo
> > > > >>>>>>>
> > > > >>>>>>>
> > > > >>>>>>> On Thu, Feb 25, 2016 at 3:29 PM, Mikhail Antonov <
> > > > >>>> olorinbant@gmail.com
> > > > >>>>>>
> > > > >>>>>>> wrote:
> > > > >>>>>>>
> > > > >>>>>>>> Agreed. I just meant - readiness of 2.0 is something
> affecting
> > > > >>>>>> decisions
> > > > >>>>>>> on
> > > > >>>>>>>> whether or not to backport mobs to branch-1 (which is itself
> > > > >>>> separate
> > > > >>>>>>>> thread).
> > > > >>>>>>>>
> > > > >>>>>>>> -Mikhail
> > > > >>>>>>>>
> > > > >>>>>>>> On Thu, Feb 25, 2016 at 3:27 PM, Sean Busbey <
> > > > >>> busbey@cloudera.com>
> > > > >>>>>>> wrote:
> > > > >>>>>>>>
> > > > >>>>>>>>> On Thu, Feb 25, 2016 at 5:21 PM, Mikhail Antonov <
> > > > >>>>>> olorinbant@gmail.com
> > > > >>>>>>>>
> > > > >>>>>>>>> wrote:
> > > > >>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> - "Shouldn't we rather try to get 2.0 release out and have
> > > > >>>> mobs
> > > > >>>>>>>> there".
> > > > >>>>>>>>> -
> > > > >>>>>>>>>> So how far do we feel 2.0 release is?
> > > > >>>>>>>>>
> > > > >>>>>>>>> 2.0 readiness probably deserves its own [DISCUSS] thread,
> but
> > > > >>>> we're
> > > > >>>>>> now
> > > > >>>>>>>>> past a year since the HBase 1.0.0 release, so I hope it's
> > > > >> soon.
> > > > >>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>> --
> > > > >>>>>>>>> Sean
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>> --
> > > > >>>>>>>> Thanks,
> > > > >>>>>>>> Michael Antonov
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>> --
> > > > >>>>>> Best regards,
> > > > >>>>>>
> > > > >>>>>> - Andy
> > > > >>>>>>
> > > > >>>>>> Problems worthy of attack prove their worth by hitting back. -
> > > Piet
> > > > >>>> Hein
> > > > >>>>>> (via Tom White)
> > > > >>>>>
> > > > >>>>>
> > > > >>>>>
> > > > >>>>> --
> > > > >>>>> Thanks,
> > > > >>>>> Michael Antonov
> > > > >>
> > > > >>
> > > > >>
> > > > >> --
> > > > >> Best regards,
> > > > >>
> > > > >> - Andy
> > > > >>
> > > > >> Problems worthy of attack prove their worth by hitting back. -
> Piet
> > > Hein
> > > > >> (via Tom White)
> > > > >>
> > > >
> > >
> > >
> > >
> > > --
> > > Thanks,
> > > Michael Antonov
> > >
> >
>
Re: Branch for 1.3
Posted by Enis Söztutar <en...@gmail.com>.
You may want to track https://issues.apache.org/jira/browse/HBASE-15339 as
a parent for date-tiered compaction improvements. Current compaction policy
is useful in its own, but handling existing data, bulk loading etc will be
improved with these subtasks. I think the patches can land before the 1.3
timeframe, but of course open for discussion for inclusion. My vote would
be to include all the improvements since it would be easier to tell the
story to users.
Enis
On Mon, Mar 21, 2016 at 3:32 AM, Ted Yu <yu...@gmail.com> wrote:
> For Spark connector, we should start a separate discussion thread about
> backporting to branch-1.
>
> Zhan has a bug fix coming this week which deals with how negative numbers
> are handled in comparison.
>
> FYI
>
> On Mon, Mar 21, 2016 at 1:35 AM, Mikhail Antonov <ol...@gmail.com>
> wrote:
>
> > Hi folks,
> >
> > bringing this topic up again. I'm planning to start spinning 1.3 builds
> and
> > see if/where they break in a week or two, and (depending on how it does)
> > start preparing RCs in a month or maybe two. So, let's see where we are.
> >
> > Big items first. There were long debates around three big items - MOBs,
> > Spark connector and RS groups, whether we should have them or not.
> >
> > - MOBs
> > I believe we decided that they aren't going to go in branch-1, and hence
> > not in branch-1.3 for sure. We might get back to that debate and
> > re-consider MOBs for branch-1 if 2.0 is delayed, but almost certainly
> they
> > won't make it in 1.3 timeframe anyway as I feel.
> >
> > - Spark connector - HBASE-14160 it looks like it has 3 subtasks open, one
> > of which is big one (HBASE-14375) - define public API for Spark
> > integration. From the Jira looks like active work is happening on other
> > subtasks, but not on this one. So how's public API going? How stable it
> is?
> > Who would want to have Spark in 1.3 and willing to help with this? OTOH -
> > who has objections about back-porting it? Has anyone been using it in
> some
> > real environment?
> >
> > - RS groups - there was recently a thread about them, I'd like to bring
> it
> > up again and get to some conclusion.
> >
> > Other features which we had in flight a month ago -
> >
> > - HBASE-15181 - date based tiered compactions has landed
> > - HBASE-11290 - unlock RegionStates. I'm afraid the codebase has moved
> > forward quite a bit since the benchmark was run on this change :( -
> Francis
> > - are you using it now? If we could have some benchmarks on the latest
> > rebase that I think would be great.
> >
> >
> > - HBASE-13557 - special handling for system tables WALs - should we
> still
> > keep it targeted for 1.3?
> > - HBASE-13017 - keep table state in meta, this one doesn't look like
> it's
> > going to make it in
> >
> > As a new item on my list - I'm looking forward to see more of HBASE-15492
> > (memory optimizations) subtasks to go in branch-1.
> >
> > Thanks!
> > Mikhail
> >
> > On Sun, Feb 28, 2016 at 10:53 AM, Andrew Purtell <
> andrew.purtell@gmail.com
> > >
> > wrote:
> >
> > > I'm starting a push at work to get us up on 1.2. Assuming that happens
> > > later this year I think that will be the end of my close attention to
> > 0.98.
> > >
> > > > On Feb 26, 2016, at 1:54 PM, Nick Dimiduk <nd...@gmail.com>
> wrote:
> > > >
> > > > On Fri, Feb 26, 2016 at 12:23 PM, Andrew Purtell <
> apurtell@apache.org>
> > > > wrote:
> > > >
> > > >> In the meantime those of us running HBase in production would
> benefit
> > > from
> > > >> fairly frequent minor releases.
> > > >
> > > > +1. Having to look back to 0.98 to get some new feature is
> problematic.
> > > >
> > > >> On Fri, Feb 26, 2016 at 9:50 AM, Elliott Clark <ec...@apache.org>
> > > wrote:
> > > >>
> > > >>> I disagree. We have agreed that 2.0 will have a new assignement
> > > manager.
> > > >>> There's a lot of work that has been done on getting that in, so far
> > > there
> > > >>> are no benefits to the end user from all that work. We should stick
> > > with
> > > >>> the plan and release 2.0 when it's ready.
> > > >>>
> > > >>> On Fri, Feb 26, 2016 at 9:39 AM, Stephen Jiang <
> > > syuanjiangdev@gmail.com>
> > > >>> wrote:
> > > >>>
> > > >>>> Thanks for Mikhail for taking the 1.3 RM role. Looks like we
> have a
> > > >> lot
> > > >>> of
> > > >>>> new things in 1.3 release.
> > > >>>>
> > > >>>> Based on the experience of 1.1 and 1.2 release, it takes a lot of
> > > >> efforts
> > > >>>> to get a stable minor release out. From this, I have my own
> 2-cents
> > > on
> > > >>> 1.4
> > > >>>> release. The plan is to have 2.0 release during summer time of
> this
> > > >> year
> > > >>>> (yeah, *this year). * Given the limited time and resource, after
> > 1.3
> > > >>>> release, instead of spending effort on 1.4 release, the community
> > > >> should
> > > >>>> focus on stabilizing master (or branch-2, not exist as of now)
> > branch
> > > >> and
> > > >>>> make 2.0 release a priority. 2.0 release would bring more values
> to
> > > >>>> customer & move towards maturity of HBASE product.
> > > >>>>
> > > >>>> Thanks
> > > >>>> Stephen
> > > >>>>
> > > >>>> On Fri, Feb 26, 2016 at 1:49 AM, Mikhail Antonov <
> > > olorinbant@gmail.com
> > > >>>
> > > >>>> wrote:
> > > >>>>
> > > >>>>> Created an umbrella jira for 1.3 release - HBASE-15341
> > > >>>>>
> > > >>>>> So it looks like we may have 1.4 release before 2.0 is out? I
> tried
> > > >> to
> > > >>>> add
> > > >>>>> 1.4 version in jira so we can keep it in branch-1 poms but I
> > > >> couldn't -
> > > >>>>> looks like I don't have permissions?
> > > >>>>>
> > > >>>>> -Mikhail
> > > >>>>>
> > > >>>>> On Thu, Feb 25, 2016 at 3:52 PM, Andrew Purtell <
> > apurtell@apache.org
> > > >>>
> > > >>>>> wrote:
> > > >>>>>
> > > >>>>>> The guy we had looking at streaming replication moved on and
> > > >> there's
> > > >>> no
> > > >>>>>> immediate plans to take on the work, FWIW
> > > >>>>>>
> > > >>>>>> On Thu, Feb 25, 2016 at 3:33 PM, Matteo Bertozzi <
> > > >>>>> theo.bertozzi@gmail.com>
> > > >>>>>> wrote:
> > > >>>>>>
> > > >>>>>>> I was shooting for summer for hbase 2.0, the main problem is
> that
> > > >>>> there
> > > >>>>>> is
> > > >>>>>>> still no code for the new AM or for fs changes, which are the
> two
> > > >>>> that
> > > >>>>>> may
> > > >>>>>>> impact compatibility (working slowly on that). Streaming
> > > >>> replication
> > > >>>>> and
> > > >>>>>>> others seems compatible enough but no code there too.
> > > >>>>>>>
> > > >>>>>>> Matteo
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> On Thu, Feb 25, 2016 at 3:29 PM, Mikhail Antonov <
> > > >>>> olorinbant@gmail.com
> > > >>>>>>
> > > >>>>>>> wrote:
> > > >>>>>>>
> > > >>>>>>>> Agreed. I just meant - readiness of 2.0 is something affecting
> > > >>>>>> decisions
> > > >>>>>>> on
> > > >>>>>>>> whether or not to backport mobs to branch-1 (which is itself
> > > >>>> separate
> > > >>>>>>>> thread).
> > > >>>>>>>>
> > > >>>>>>>> -Mikhail
> > > >>>>>>>>
> > > >>>>>>>> On Thu, Feb 25, 2016 at 3:27 PM, Sean Busbey <
> > > >>> busbey@cloudera.com>
> > > >>>>>>> wrote:
> > > >>>>>>>>
> > > >>>>>>>>> On Thu, Feb 25, 2016 at 5:21 PM, Mikhail Antonov <
> > > >>>>>> olorinbant@gmail.com
> > > >>>>>>>>
> > > >>>>>>>>> wrote:
> > > >>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>> - "Shouldn't we rather try to get 2.0 release out and have
> > > >>>> mobs
> > > >>>>>>>> there".
> > > >>>>>>>>> -
> > > >>>>>>>>>> So how far do we feel 2.0 release is?
> > > >>>>>>>>>
> > > >>>>>>>>> 2.0 readiness probably deserves its own [DISCUSS] thread, but
> > > >>>> we're
> > > >>>>>> now
> > > >>>>>>>>> past a year since the HBase 1.0.0 release, so I hope it's
> > > >> soon.
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>> --
> > > >>>>>>>>> Sean
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> --
> > > >>>>>>>> Thanks,
> > > >>>>>>>> Michael Antonov
> > > >>>>>>
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> --
> > > >>>>>> Best regards,
> > > >>>>>>
> > > >>>>>> - Andy
> > > >>>>>>
> > > >>>>>> Problems worthy of attack prove their worth by hitting back. -
> > Piet
> > > >>>> Hein
> > > >>>>>> (via Tom White)
> > > >>>>>
> > > >>>>>
> > > >>>>>
> > > >>>>> --
> > > >>>>> Thanks,
> > > >>>>> Michael Antonov
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> Best regards,
> > > >>
> > > >> - Andy
> > > >>
> > > >> Problems worthy of attack prove their worth by hitting back. - Piet
> > Hein
> > > >> (via Tom White)
> > > >>
> > >
> >
> >
> >
> > --
> > Thanks,
> > Michael Antonov
> >
>
Re: Branch for 1.3
Posted by Ted Yu <yu...@gmail.com>.
For Spark connector, we should start a separate discussion thread about
backporting to branch-1.
Zhan has a bug fix coming this week which deals with how negative numbers
are handled in comparison.
FYI
On Mon, Mar 21, 2016 at 1:35 AM, Mikhail Antonov <ol...@gmail.com>
wrote:
> Hi folks,
>
> bringing this topic up again. I'm planning to start spinning 1.3 builds and
> see if/where they break in a week or two, and (depending on how it does)
> start preparing RCs in a month or maybe two. So, let's see where we are.
>
> Big items first. There were long debates around three big items - MOBs,
> Spark connector and RS groups, whether we should have them or not.
>
> - MOBs
> I believe we decided that they aren't going to go in branch-1, and hence
> not in branch-1.3 for sure. We might get back to that debate and
> re-consider MOBs for branch-1 if 2.0 is delayed, but almost certainly they
> won't make it in 1.3 timeframe anyway as I feel.
>
> - Spark connector - HBASE-14160 it looks like it has 3 subtasks open, one
> of which is big one (HBASE-14375) - define public API for Spark
> integration. From the Jira looks like active work is happening on other
> subtasks, but not on this one. So how's public API going? How stable it is?
> Who would want to have Spark in 1.3 and willing to help with this? OTOH -
> who has objections about back-porting it? Has anyone been using it in some
> real environment?
>
> - RS groups - there was recently a thread about them, I'd like to bring it
> up again and get to some conclusion.
>
> Other features which we had in flight a month ago -
>
> - HBASE-15181 - date based tiered compactions has landed
> - HBASE-11290 - unlock RegionStates. I'm afraid the codebase has moved
> forward quite a bit since the benchmark was run on this change :( - Francis
> - are you using it now? If we could have some benchmarks on the latest
> rebase that I think would be great.
>
>
> - HBASE-13557 - special handling for system tables WALs - should we still
> keep it targeted for 1.3?
> - HBASE-13017 - keep table state in meta, this one doesn't look like it's
> going to make it in
>
> As a new item on my list - I'm looking forward to see more of HBASE-15492
> (memory optimizations) subtasks to go in branch-1.
>
> Thanks!
> Mikhail
>
> On Sun, Feb 28, 2016 at 10:53 AM, Andrew Purtell <andrew.purtell@gmail.com
> >
> wrote:
>
> > I'm starting a push at work to get us up on 1.2. Assuming that happens
> > later this year I think that will be the end of my close attention to
> 0.98.
> >
> > > On Feb 26, 2016, at 1:54 PM, Nick Dimiduk <nd...@gmail.com> wrote:
> > >
> > > On Fri, Feb 26, 2016 at 12:23 PM, Andrew Purtell <ap...@apache.org>
> > > wrote:
> > >
> > >> In the meantime those of us running HBase in production would benefit
> > from
> > >> fairly frequent minor releases.
> > >
> > > +1. Having to look back to 0.98 to get some new feature is problematic.
> > >
> > >> On Fri, Feb 26, 2016 at 9:50 AM, Elliott Clark <ec...@apache.org>
> > wrote:
> > >>
> > >>> I disagree. We have agreed that 2.0 will have a new assignement
> > manager.
> > >>> There's a lot of work that has been done on getting that in, so far
> > there
> > >>> are no benefits to the end user from all that work. We should stick
> > with
> > >>> the plan and release 2.0 when it's ready.
> > >>>
> > >>> On Fri, Feb 26, 2016 at 9:39 AM, Stephen Jiang <
> > syuanjiangdev@gmail.com>
> > >>> wrote:
> > >>>
> > >>>> Thanks for Mikhail for taking the 1.3 RM role. Looks like we have a
> > >> lot
> > >>> of
> > >>>> new things in 1.3 release.
> > >>>>
> > >>>> Based on the experience of 1.1 and 1.2 release, it takes a lot of
> > >> efforts
> > >>>> to get a stable minor release out. From this, I have my own 2-cents
> > on
> > >>> 1.4
> > >>>> release. The plan is to have 2.0 release during summer time of this
> > >> year
> > >>>> (yeah, *this year). * Given the limited time and resource, after
> 1.3
> > >>>> release, instead of spending effort on 1.4 release, the community
> > >> should
> > >>>> focus on stabilizing master (or branch-2, not exist as of now)
> branch
> > >> and
> > >>>> make 2.0 release a priority. 2.0 release would bring more values to
> > >>>> customer & move towards maturity of HBASE product.
> > >>>>
> > >>>> Thanks
> > >>>> Stephen
> > >>>>
> > >>>> On Fri, Feb 26, 2016 at 1:49 AM, Mikhail Antonov <
> > olorinbant@gmail.com
> > >>>
> > >>>> wrote:
> > >>>>
> > >>>>> Created an umbrella jira for 1.3 release - HBASE-15341
> > >>>>>
> > >>>>> So it looks like we may have 1.4 release before 2.0 is out? I tried
> > >> to
> > >>>> add
> > >>>>> 1.4 version in jira so we can keep it in branch-1 poms but I
> > >> couldn't -
> > >>>>> looks like I don't have permissions?
> > >>>>>
> > >>>>> -Mikhail
> > >>>>>
> > >>>>> On Thu, Feb 25, 2016 at 3:52 PM, Andrew Purtell <
> apurtell@apache.org
> > >>>
> > >>>>> wrote:
> > >>>>>
> > >>>>>> The guy we had looking at streaming replication moved on and
> > >> there's
> > >>> no
> > >>>>>> immediate plans to take on the work, FWIW
> > >>>>>>
> > >>>>>> On Thu, Feb 25, 2016 at 3:33 PM, Matteo Bertozzi <
> > >>>>> theo.bertozzi@gmail.com>
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>>> I was shooting for summer for hbase 2.0, the main problem is that
> > >>>> there
> > >>>>>> is
> > >>>>>>> still no code for the new AM or for fs changes, which are the two
> > >>>> that
> > >>>>>> may
> > >>>>>>> impact compatibility (working slowly on that). Streaming
> > >>> replication
> > >>>>> and
> > >>>>>>> others seems compatible enough but no code there too.
> > >>>>>>>
> > >>>>>>> Matteo
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> On Thu, Feb 25, 2016 at 3:29 PM, Mikhail Antonov <
> > >>>> olorinbant@gmail.com
> > >>>>>>
> > >>>>>>> wrote:
> > >>>>>>>
> > >>>>>>>> Agreed. I just meant - readiness of 2.0 is something affecting
> > >>>>>> decisions
> > >>>>>>> on
> > >>>>>>>> whether or not to backport mobs to branch-1 (which is itself
> > >>>> separate
> > >>>>>>>> thread).
> > >>>>>>>>
> > >>>>>>>> -Mikhail
> > >>>>>>>>
> > >>>>>>>> On Thu, Feb 25, 2016 at 3:27 PM, Sean Busbey <
> > >>> busbey@cloudera.com>
> > >>>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>>> On Thu, Feb 25, 2016 at 5:21 PM, Mikhail Antonov <
> > >>>>>> olorinbant@gmail.com
> > >>>>>>>>
> > >>>>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> - "Shouldn't we rather try to get 2.0 release out and have
> > >>>> mobs
> > >>>>>>>> there".
> > >>>>>>>>> -
> > >>>>>>>>>> So how far do we feel 2.0 release is?
> > >>>>>>>>>
> > >>>>>>>>> 2.0 readiness probably deserves its own [DISCUSS] thread, but
> > >>>> we're
> > >>>>>> now
> > >>>>>>>>> past a year since the HBase 1.0.0 release, so I hope it's
> > >> soon.
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> --
> > >>>>>>>>> Sean
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> --
> > >>>>>>>> Thanks,
> > >>>>>>>> Michael Antonov
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> --
> > >>>>>> Best regards,
> > >>>>>>
> > >>>>>> - Andy
> > >>>>>>
> > >>>>>> Problems worthy of attack prove their worth by hitting back. -
> Piet
> > >>>> Hein
> > >>>>>> (via Tom White)
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> --
> > >>>>> Thanks,
> > >>>>> Michael Antonov
> > >>
> > >>
> > >>
> > >> --
> > >> Best regards,
> > >>
> > >> - Andy
> > >>
> > >> Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > >> (via Tom White)
> > >>
> >
>
>
>
> --
> Thanks,
> Michael Antonov
>