You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Stack <st...@duboce.net> on 2013/06/25 05:45:40 UTC
[DISCUSSION] Finishing up 0.96 --> WAS Re: 0.95 and 0.96 remaining issues
(Changed the subject)
On Mon, Jun 24, 2013 at 4:04 PM, Nick Dimiduk <nd...@gmail.com> wrote:
> I want to see initial data type APIs ship out with 0.95.2. A patch for
> ordered byte serialization is up (HBASE-8201) and is nearing steady-state.
> However, sershe is the only person who's left feedback. I just posted an
> early patch for the data type API itself (HBASE-8693). It should get some
> eyes from all manor of interested parties, but I'll settle for folk from
> Phoenix for now.
>
>
It would be cool if Phoenix and Kiji fellows and any one else interested
would weigh in and take a look see.
This does not strike me as something we should hold up the release for
though. It looks like something that could go in at any time?
> Should these tasks be escalated to criticals in order to grab attention?
>
>
I don't think that works going by past experience (and I don't think this a
blocker on 0.96)
> Additional comments inline.
>
>
...
> Namespaces is the long pole and progress seems slow. Do we hold up the
> > release for them? How can we hurry this effort along? Swat team
> descends
> > on Y!?
> >
>
> It would be a shame to not get a decision on this in for 0.96.
>
>
Agree. We need to get 0.96 out though. It has been too long.
> + Is anyone testing? Integration tests fail on ec2 build from time to time
> > [2]. Our Elliott dug in on one of the failures a few days back and found
> > legit issue w/ no retry on admin tasks (I heart hbase-it tests). Our
> unit
> > test story is better [3] but there are still the odd failures.
> >
>
> With the creation of the new list, noticing these issues is going to push
> further back-burner. Nannying this stuff should retain focus. I'll
> volunteer to track on these issues as I see them.
>
>
Thank you Nick.
> Notice though that some of the more recent failures are caused by lack of
> disk space on the Jenkins build host.
>
>
Oh. Missed that. Let me dig in.
St.Ack
Re: [DISCUSSION] Finishing up 0.96 --> WAS Re: 0.95 and 0.96
remaining issues
Posted by Enis Söztutar <en...@gmail.com>.
> These are both criticals. Maybe they should be blocker altogether since
> can't ship w/o them.
>
Ok, did that.
>
> > https://issues.apache.org/jira/browse/HBASE-6929 (same for 0.94 and
> 0.96)
> >
> >
> This is a 0.94 issue. Could fall out from hbase-8224 work.
>
Agreed, as long as HBASE-8224 will enable us to publish 0.96-hadoop2 to
mvn, and at the time of release we do that, we are golden. 0.94-hadoop2 is
not a blocker.
>
>
>
> > For migration, there are some steps that the a user has to perform before
> > going to 0.96 (recompile with 0.96, select hadoop1 or hadoop2, some minor
> > class renamings, etc), we should document these steps in the book. (I
> don't
> > remember we already have an open jira for this.)
> >
> >
> HBASE-8348 Polish the migration to 0.96
>
>
> > For testing, IT tests are rapidly improving. We will also start running
> > those (or a subset of those) against 0.96.
> >
>
>
> Sweet.
>
> St.Ack
>
Re: [DISCUSSION] Finishing up 0.96 --> WAS Re: 0.95 and 0.96
remaining issues
Posted by Stack <st...@duboce.net>.
On Wed, Jul 17, 2013 at 4:21 PM, Enis Söztutar <en...@gmail.com> wrote:
> Thanks Stack for pushing us.
>
> For packaging, I think we should fix:
> https://issues.apache.org/jira/browse/HBASE-8224
> https://issues.apache.org/jira/browse/HBASE-8488
These are both criticals. Maybe they should be blocker altogether since
can't ship w/o them.
>
> https://issues.apache.org/jira/browse/HBASE-6929 (same for 0.94 and 0.96)
>
>
This is a 0.94 issue. Could fall out from hbase-8224 work.
> For migration, there are some steps that the a user has to perform before
> going to 0.96 (recompile with 0.96, select hadoop1 or hadoop2, some minor
> class renamings, etc), we should document these steps in the book. (I don't
> remember we already have an open jira for this.)
>
>
HBASE-8348 Polish the migration to 0.96
> For testing, IT tests are rapidly improving. We will also start running
> those (or a subset of those) against 0.96.
>
Sweet.
St.Ack
Re: [DISCUSSION] Finishing up 0.96 --> WAS Re: 0.95 and 0.96
remaining issues
Posted by Enis Söztutar <en...@gmail.com>.
Thanks Stack for pushing us.
For packaging, I think we should fix:
https://issues.apache.org/jira/browse/HBASE-8224
https://issues.apache.org/jira/browse/HBASE-8488
https://issues.apache.org/jira/browse/HBASE-6929 (same for 0.94 and 0.96)
For migration, there are some steps that the a user has to perform before
going to 0.96 (recompile with 0.96, select hadoop1 or hadoop2, some minor
class renamings, etc), we should document these steps in the book. (I don't
remember we already have an open jira for this.)
For testing, IT tests are rapidly improving. We will also start running
those (or a subset of those) against 0.96.
Enis
On Wed, Jul 17, 2013 at 3:21 PM, Andrew Purtell <ap...@apache.org> wrote:
> On Wed, Jul 17, 2013 at 2:58 PM, Stack <st...@duboce.net> wrote:
>
> > I love the idea of bigtop packaging. Should we do new issue for that?
> Or
> just use 8224.
>
> Ok, did both. See https://issues.apache.org/jira/browse/BIGTOP-1029, and
> linked to 8224, with comment there.
>
> > http://hbase.apache.org/book.html#upgrade0.96
>
> Let's see about scripting that for repeatable testing.
>
> > it unlikely that a *design* at this stage will make the cut given
> completed patches may not make it for want of review, etc., etc
>
> Patch, not design, didn't mean to imply otherwise.
>
>
> --
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>
Re: [DISCUSSION] Finishing up 0.96 --> WAS Re: 0.95 and 0.96
remaining issues
Posted by Stack <st...@duboce.net>.
On Wed, Jul 17, 2013 at 3:21 PM, Andrew Purtell <ap...@apache.org> wrote:
> On Wed, Jul 17, 2013 at 2:58 PM, Stack <st...@duboce.net> wrote:
>
> > I love the idea of bigtop packaging. Should we do new issue for that?
> Or
> just use 8224.
>
> Ok, did both. See https://issues.apache.org/jira/browse/BIGTOP-1029, and
> linked to 8224, with comment there.
>
>
Excellent.
> > http://hbase.apache.org/book.html#upgrade0.96
>
> Let's see about scripting that for repeatable testing.
>
>
Yeah. I can work on that.... do the old school tarball of 0.94 data, undo
it in a test, and run 0.96 over it.
> > it unlikely that a *design* at this stage will make the cut given
> completed patches may not make it for want of review, etc., etc
>
> Patch, not design, didn't mean to imply otherwise.
>
>
Ok. Patch has better chance than a design of making it in.
Good on you A,
St.Ack
Re: [DISCUSSION] Finishing up 0.96 --> WAS Re: 0.95 and 0.96
remaining issues
Posted by Andrew Purtell <ap...@apache.org>.
On Wed, Jul 17, 2013 at 2:58 PM, Stack <st...@duboce.net> wrote:
> I love the idea of bigtop packaging. Should we do new issue for that? Or
just use 8224.
Ok, did both. See https://issues.apache.org/jira/browse/BIGTOP-1029, and
linked to 8224, with comment there.
> http://hbase.apache.org/book.html#upgrade0.96
Let's see about scripting that for repeatable testing.
> it unlikely that a *design* at this stage will make the cut given
completed patches may not make it for want of review, etc., etc
Patch, not design, didn't mean to imply otherwise.
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: [DISCUSSION] Finishing up 0.96 --> WAS Re: 0.95 and 0.96
remaining issues
Posted by Stack <st...@duboce.net>.
On Wed, Jul 17, 2013 at 1:45 PM, Andrew Purtell <ap...@apache.org> wrote:
> > +Packaging
>
> I would like to help you on this. Can do packaging and testing using the
> Bigtop framework. I see HBASE-8187, but it's resolved. Elsewhere we
> can coordinate? Could open a Bigtop jira. Whatever you prefer.
>
>
I was most concerned about
https://issues.apache.org/jira/browse/HBASE-8224-- publishing to mvn.
That is what has me stumped at the moment.
I love the idea of bigtop packaging. Should we do new issue for that? Or
just use 8224. It has a few watchers -- downstreamers -- so it might be
better to hang there at least at first.
> > +Migration
>
> As part of the above. Test by starting with an 0.94 install on a small
> cluster, load up some data, shut down and upgrade packages, try to
> (re)launch...
>
>
Yes, following the (as-yet-incomplete) instruction here:
http://hbase.apache.org/book.html#upgrade0.96
> > What else?
>
> I beg your patience as RM. Some guys here have been working diligently for
> the past couple of months on something that is in our estimation 0.96able.
> Won't go into it here because they have been passing around a proposal for
> dev@ since the weekend, to make sure it says what they want. Will be out
> in
> the next day or so for the consideration of all.
>
>
Ain't no need to beg. Lets have a look at it before making a call. I'll
spare the rehearsal that it is very late in the game, etc., etc., and that
it unlikely that a *design* at this stage will make the cut given completed
patches may not make it for want of review, etc., etc .... But lets see.
Thanks Andrew,
St.Ack
Re: [DISCUSSION] Finishing up 0.96 --> WAS Re: 0.95 and 0.96
remaining issues
Posted by Andrew Purtell <ap...@apache.org>.
> +Packaging
I would like to help you on this. Can do packaging and testing using the
Bigtop framework. I see HBASE-8187, but it's resolved. Elsewhere we
can coordinate? Could open a Bigtop jira. Whatever you prefer.
> +Migration
As part of the above. Test by starting with an 0.94 install on a small
cluster, load up some data, shut down and upgrade packages, try to
(re)launch...
> What else?
I beg your patience as RM. Some guys here have been working diligently for
the past couple of months on something that is in our estimation 0.96able.
Won't go into it here because they have been passing around a proposal for
dev@ since the weekend, to make sure it says what they want. Will be out in
the next day or so for the consideration of all.
On Wed, Jul 17, 2013 at 12:12 PM, Stack <st...@duboce.net> wrote:
> STATUS
>
> +Build
>
> A few of us have been working on broke junit tests. Progress is slow but
> steady. Currently I am looking into zombie tests -- "all" tests complete
> but the build still fails. Andrew Purtell has set me up w/ a clone box
> from his ec2 rig which is helping figure the zombie phenomenon. I also
> just added our hadoopqa zombie detector up on apache trunk build to see if
> it yields clues at nkeywals' suggestion. I also need to fix
> https://issues.apache.org/jira/browse/HBASE-8764 so the ec2 integration
> test build passes (ec2 build runs integration tests which apache build does
> not)
>
> +Reviews
>
> Francis' namespaces is in need of review as are other 'Patch Available'
> criticals that we'd like committed such as Sergey's compaction changes. If
> any volunteers are game, see the 'patch available' issues here [1].
> Contrib appreciated.
>
> +Testing
> I know of at least one rig that is going up to do long-term hbase-it tests.
> Any other testing going on that folks would like to talk up?
>
> +Migration
>
> Testing and finish documentation on moving from 0.94 pedigree to 0.96.
> Still TODO.
>
> +Packaging
>
> Making it so we can publish an hbase 0.96 built against hadoop1 and
> hadoop2. Still TODO.
>
> What else?
>
> I am still heading for end of July for a complete '0.96'.
>
> Thanks,
> St.Ack
>
> 1.
>
> https://issues.apache.org/jira/browse/HBASE/fixforversion/12320040#selectedTab=com.atlassian.jira.plugin.system.project%3Aversion-issues-panel
>
>
>
>
>
> On Tue, Jul 9, 2013 at 1:14 PM, Stack <st...@duboce.net> wrote:
>
> > I am shooting for end of July for 0.96 being 'complete'. I would like
> to
> > make a 0.96 release in August. We have some criticals outstanding but I
> > think we could ship even if these are not fixed in time (excepting
> > migration polish and of course remaining build fixes). See [1.] for the
> > current list of issues. Please re-prioritize issues as you see fit (or
> > better, move issues out of 0.95.2 if you do not think they will be done
> in
> > time).
> >
> > What to do with namespaces -- the last 0.96 'feature' -- given the above
> > timeline? Currently it is a massive patch out on a branch. It is still
> > not done, in want of review, and the author is going on holidays for a
> few
> > weeks soon. My thinking as of now, going by the rate of change over the
> > last few weeks and estimating what is yet to be done, is that namespaces
> > will not make it. I am willing to be convinced otherwise but that is how
> > it looks to me currently.
> >
> > I am going to start just disabling flakey unit tests in 0.95 from here on
> > out. When folks get the itch, they can fix at leisure first on trunk and
> > then over in 0.95.
> >
> > What else?
> >
> > Thanks,
> > St.Ack
> >
> > 1.
> >
> https://issues.apache.org/jira/browse/HBASE/fixforversion/12320040#selectedTab=com.atlassian.jira.plugin.system.project%3Aversion-issues-panel
> >
> >
> >
> > On Mon, Jun 24, 2013 at 8:45 PM, Stack <st...@duboce.net> wrote:
> >
> >> (Changed the subject)
> >>
> >> On Mon, Jun 24, 2013 at 4:04 PM, Nick Dimiduk <nd...@gmail.com>
> wrote:
> >>
> >>> I want to see initial data type APIs ship out with 0.95.2. A patch for
> >>> ordered byte serialization is up (HBASE-8201) and is nearing
> >>> steady-state.
> >>> However, sershe is the only person who's left feedback. I just posted
> an
> >>> early patch for the data type API itself (HBASE-8693). It should get
> some
> >>> eyes from all manor of interested parties, but I'll settle for folk
> from
> >>> Phoenix for now.
> >>>
> >>>
> >> It would be cool if Phoenix and Kiji fellows and any one else interested
> >> would weigh in and take a look see.
> >>
> >> This does not strike me as something we should hold up the release for
> >> though. It looks like something that could go in at any time?
> >>
> >>
> >>
> >>> Should these tasks be escalated to criticals in order to grab
> attention?
> >>>
> >>>
> >> I don't think that works going by past experience (and I don't think
> this
> >> a blocker on 0.96)
> >>
> >>
> >>
> >>> Additional comments inline.
> >>>
> >>>
> >> ...
> >>
> >>
> >>
> >>> Namespaces is the long pole and progress seems slow. Do we hold up the
> >>> > release for them? How can we hurry this effort along? Swat team
> >>> descends
> >>> > on Y!?
> >>> >
> >>>
> >>> It would be a shame to not get a decision on this in for 0.96.
> >>>
> >>>
> >> Agree. We need to get 0.96 out though. It has been too long.
> >>
> >>
> >>
> >>> + Is anyone testing? Integration tests fail on ec2 build from time to
> >>> time
> >>> > [2]. Our Elliott dug in on one of the failures a few days back and
> >>> found
> >>> > legit issue w/ no retry on admin tasks (I heart hbase-it tests). Our
> >>> unit
> >>> > test story is better [3] but there are still the odd failures.
> >>> >
> >>>
> >>> With the creation of the new list, noticing these issues is going to
> push
> >>> further back-burner. Nannying this stuff should retain focus. I'll
> >>> volunteer to track on these issues as I see them.
> >>>
> >>>
> >> Thank you Nick.
> >>
> >>
> >>
> >>> Notice though that some of the more recent failures are caused by lack
> of
> >>> disk space on the Jenkins build host.
> >>>
> >>>
> >> Oh. Missed that. Let me dig in.
> >>
> >>
> >> St.Ack
> >>
> >
> >
>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: [DISCUSSION] Finishing up 0.96 --> WAS Re: 0.95 and 0.96
remaining issues
Posted by Stack <st...@duboce.net>.
On Thu, Jul 18, 2013 at 3:54 PM, Ted Yu <yu...@gmail.com> wrote:
> bq. Francis' namespaces is in need of review
>
> I plan to review this feature.
>
> The latest update to https://github.com/francisliu/hbase_namespace lagged
> the last comment on HBASE-8015
> I wonder if there are more significant changes to be made.
>
He moved his work to the github mentioned in the issue. Go there for his
latest work.
St.Ack
Re: [DISCUSSION] Finishing up 0.96 --> WAS Re: 0.95 and 0.96
remaining issues
Posted by Ted Yu <yu...@gmail.com>.
bq. Francis' namespaces is in need of review
I plan to review this feature.
The latest update to https://github.com/francisliu/hbase_namespace lagged
the last comment on HBASE-8015
I wonder if there are more significant changes to be made.
Cheers
On Wed, Jul 17, 2013 at 12:12 PM, Stack <st...@duboce.net> wrote:
> STATUS
>
> +Build
>
> A few of us have been working on broke junit tests. Progress is slow but
> steady. Currently I am looking into zombie tests -- "all" tests complete
> but the build still fails. Andrew Purtell has set me up w/ a clone box
> from his ec2 rig which is helping figure the zombie phenomenon. I also
> just added our hadoopqa zombie detector up on apache trunk build to see if
> it yields clues at nkeywals' suggestion. I also need to fix
> https://issues.apache.org/jira/browse/HBASE-8764 so the ec2 integration
> test build passes (ec2 build runs integration tests which apache build does
> not)
>
> +Reviews
>
> Francis' namespaces is in need of review as are other 'Patch Available'
> criticals that we'd like committed such as Sergey's compaction changes. If
> any volunteers are game, see the 'patch available' issues here [1].
> Contrib appreciated.
>
> +Testing
> I know of at least one rig that is going up to do long-term hbase-it tests.
> Any other testing going on that folks would like to talk up?
>
> +Migration
>
> Testing and finish documentation on moving from 0.94 pedigree to 0.96.
> Still TODO.
>
> +Packaging
>
> Making it so we can publish an hbase 0.96 built against hadoop1 and
> hadoop2. Still TODO.
>
> What else?
>
> I am still heading for end of July for a complete '0.96'.
>
> Thanks,
> St.Ack
>
> 1.
>
> https://issues.apache.org/jira/browse/HBASE/fixforversion/12320040#selectedTab=com.atlassian.jira.plugin.system.project%3Aversion-issues-panel
>
>
>
>
>
> On Tue, Jul 9, 2013 at 1:14 PM, Stack <st...@duboce.net> wrote:
>
> > I am shooting for end of July for 0.96 being 'complete'. I would like
> to
> > make a 0.96 release in August. We have some criticals outstanding but I
> > think we could ship even if these are not fixed in time (excepting
> > migration polish and of course remaining build fixes). See [1.] for the
> > current list of issues. Please re-prioritize issues as you see fit (or
> > better, move issues out of 0.95.2 if you do not think they will be done
> in
> > time).
> >
> > What to do with namespaces -- the last 0.96 'feature' -- given the above
> > timeline? Currently it is a massive patch out on a branch. It is still
> > not done, in want of review, and the author is going on holidays for a
> few
> > weeks soon. My thinking as of now, going by the rate of change over the
> > last few weeks and estimating what is yet to be done, is that namespaces
> > will not make it. I am willing to be convinced otherwise but that is how
> > it looks to me currently.
> >
> > I am going to start just disabling flakey unit tests in 0.95 from here on
> > out. When folks get the itch, they can fix at leisure first on trunk and
> > then over in 0.95.
> >
> > What else?
> >
> > Thanks,
> > St.Ack
> >
> > 1.
> >
> https://issues.apache.org/jira/browse/HBASE/fixforversion/12320040#selectedTab=com.atlassian.jira.plugin.system.project%3Aversion-issues-panel
> >
> >
> >
> > On Mon, Jun 24, 2013 at 8:45 PM, Stack <st...@duboce.net> wrote:
> >
> >> (Changed the subject)
> >>
> >> On Mon, Jun 24, 2013 at 4:04 PM, Nick Dimiduk <nd...@gmail.com>
> wrote:
> >>
> >>> I want to see initial data type APIs ship out with 0.95.2. A patch for
> >>> ordered byte serialization is up (HBASE-8201) and is nearing
> >>> steady-state.
> >>> However, sershe is the only person who's left feedback. I just posted
> an
> >>> early patch for the data type API itself (HBASE-8693). It should get
> some
> >>> eyes from all manor of interested parties, but I'll settle for folk
> from
> >>> Phoenix for now.
> >>>
> >>>
> >> It would be cool if Phoenix and Kiji fellows and any one else interested
> >> would weigh in and take a look see.
> >>
> >> This does not strike me as something we should hold up the release for
> >> though. It looks like something that could go in at any time?
> >>
> >>
> >>
> >>> Should these tasks be escalated to criticals in order to grab
> attention?
> >>>
> >>>
> >> I don't think that works going by past experience (and I don't think
> this
> >> a blocker on 0.96)
> >>
> >>
> >>
> >>> Additional comments inline.
> >>>
> >>>
> >> ...
> >>
> >>
> >>
> >>> Namespaces is the long pole and progress seems slow. Do we hold up the
> >>> > release for them? How can we hurry this effort along? Swat team
> >>> descends
> >>> > on Y!?
> >>> >
> >>>
> >>> It would be a shame to not get a decision on this in for 0.96.
> >>>
> >>>
> >> Agree. We need to get 0.96 out though. It has been too long.
> >>
> >>
> >>
> >>> + Is anyone testing? Integration tests fail on ec2 build from time to
> >>> time
> >>> > [2]. Our Elliott dug in on one of the failures a few days back and
> >>> found
> >>> > legit issue w/ no retry on admin tasks (I heart hbase-it tests). Our
> >>> unit
> >>> > test story is better [3] but there are still the odd failures.
> >>> >
> >>>
> >>> With the creation of the new list, noticing these issues is going to
> push
> >>> further back-burner. Nannying this stuff should retain focus. I'll
> >>> volunteer to track on these issues as I see them.
> >>>
> >>>
> >> Thank you Nick.
> >>
> >>
> >>
> >>> Notice though that some of the more recent failures are caused by lack
> of
> >>> disk space on the Jenkins build host.
> >>>
> >>>
> >> Oh. Missed that. Let me dig in.
> >>
> >>
> >> St.Ack
> >>
> >
> >
>
Re: [DISCUSSION] Finishing up 0.96 --> WAS Re: 0.95 and 0.96
remaining issues
Posted by Stack <st...@duboce.net>.
STATUS
+Build
A few of us have been working on broke junit tests. Progress is slow but
steady. Currently I am looking into zombie tests -- "all" tests complete
but the build still fails. Andrew Purtell has set me up w/ a clone box
from his ec2 rig which is helping figure the zombie phenomenon. I also
just added our hadoopqa zombie detector up on apache trunk build to see if
it yields clues at nkeywals' suggestion. I also need to fix
https://issues.apache.org/jira/browse/HBASE-8764 so the ec2 integration
test build passes (ec2 build runs integration tests which apache build does
not)
+Reviews
Francis' namespaces is in need of review as are other 'Patch Available'
criticals that we'd like committed such as Sergey's compaction changes. If
any volunteers are game, see the 'patch available' issues here [1].
Contrib appreciated.
+Testing
I know of at least one rig that is going up to do long-term hbase-it tests.
Any other testing going on that folks would like to talk up?
+Migration
Testing and finish documentation on moving from 0.94 pedigree to 0.96.
Still TODO.
+Packaging
Making it so we can publish an hbase 0.96 built against hadoop1 and
hadoop2. Still TODO.
What else?
I am still heading for end of July for a complete '0.96'.
Thanks,
St.Ack
1.
https://issues.apache.org/jira/browse/HBASE/fixforversion/12320040#selectedTab=com.atlassian.jira.plugin.system.project%3Aversion-issues-panel
On Tue, Jul 9, 2013 at 1:14 PM, Stack <st...@duboce.net> wrote:
> I am shooting for end of July for 0.96 being 'complete'. I would like to
> make a 0.96 release in August. We have some criticals outstanding but I
> think we could ship even if these are not fixed in time (excepting
> migration polish and of course remaining build fixes). See [1.] for the
> current list of issues. Please re-prioritize issues as you see fit (or
> better, move issues out of 0.95.2 if you do not think they will be done in
> time).
>
> What to do with namespaces -- the last 0.96 'feature' -- given the above
> timeline? Currently it is a massive patch out on a branch. It is still
> not done, in want of review, and the author is going on holidays for a few
> weeks soon. My thinking as of now, going by the rate of change over the
> last few weeks and estimating what is yet to be done, is that namespaces
> will not make it. I am willing to be convinced otherwise but that is how
> it looks to me currently.
>
> I am going to start just disabling flakey unit tests in 0.95 from here on
> out. When folks get the itch, they can fix at leisure first on trunk and
> then over in 0.95.
>
> What else?
>
> Thanks,
> St.Ack
>
> 1.
> https://issues.apache.org/jira/browse/HBASE/fixforversion/12320040#selectedTab=com.atlassian.jira.plugin.system.project%3Aversion-issues-panel
>
>
>
> On Mon, Jun 24, 2013 at 8:45 PM, Stack <st...@duboce.net> wrote:
>
>> (Changed the subject)
>>
>> On Mon, Jun 24, 2013 at 4:04 PM, Nick Dimiduk <nd...@gmail.com> wrote:
>>
>>> I want to see initial data type APIs ship out with 0.95.2. A patch for
>>> ordered byte serialization is up (HBASE-8201) and is nearing
>>> steady-state.
>>> However, sershe is the only person who's left feedback. I just posted an
>>> early patch for the data type API itself (HBASE-8693). It should get some
>>> eyes from all manor of interested parties, but I'll settle for folk from
>>> Phoenix for now.
>>>
>>>
>> It would be cool if Phoenix and Kiji fellows and any one else interested
>> would weigh in and take a look see.
>>
>> This does not strike me as something we should hold up the release for
>> though. It looks like something that could go in at any time?
>>
>>
>>
>>> Should these tasks be escalated to criticals in order to grab attention?
>>>
>>>
>> I don't think that works going by past experience (and I don't think this
>> a blocker on 0.96)
>>
>>
>>
>>> Additional comments inline.
>>>
>>>
>> ...
>>
>>
>>
>>> Namespaces is the long pole and progress seems slow. Do we hold up the
>>> > release for them? How can we hurry this effort along? Swat team
>>> descends
>>> > on Y!?
>>> >
>>>
>>> It would be a shame to not get a decision on this in for 0.96.
>>>
>>>
>> Agree. We need to get 0.96 out though. It has been too long.
>>
>>
>>
>>> + Is anyone testing? Integration tests fail on ec2 build from time to
>>> time
>>> > [2]. Our Elliott dug in on one of the failures a few days back and
>>> found
>>> > legit issue w/ no retry on admin tasks (I heart hbase-it tests). Our
>>> unit
>>> > test story is better [3] but there are still the odd failures.
>>> >
>>>
>>> With the creation of the new list, noticing these issues is going to push
>>> further back-burner. Nannying this stuff should retain focus. I'll
>>> volunteer to track on these issues as I see them.
>>>
>>>
>> Thank you Nick.
>>
>>
>>
>>> Notice though that some of the more recent failures are caused by lack of
>>> disk space on the Jenkins build host.
>>>
>>>
>> Oh. Missed that. Let me dig in.
>>
>>
>> St.Ack
>>
>
>
Re: [DISCUSSION] Finishing up 0.96 --> WAS Re: 0.95 and 0.96
remaining issues
Posted by Stack <st...@duboce.net>.
I am shooting for end of July for 0.96 being 'complete'. I would like to
make a 0.96 release in August. We have some criticals outstanding but I
think we could ship even if these are not fixed in time (excepting
migration polish and of course remaining build fixes). See [1.] for the
current list of issues. Please re-prioritize issues as you see fit (or
better, move issues out of 0.95.2 if you do not think they will be done in
time).
What to do with namespaces -- the last 0.96 'feature' -- given the above
timeline? Currently it is a massive patch out on a branch. It is still
not done, in want of review, and the author is going on holidays for a few
weeks soon. My thinking as of now, going by the rate of change over the
last few weeks and estimating what is yet to be done, is that namespaces
will not make it. I am willing to be convinced otherwise but that is how
it looks to me currently.
I am going to start just disabling flakey unit tests in 0.95 from here on
out. When folks get the itch, they can fix at leisure first on trunk and
then over in 0.95.
What else?
Thanks,
St.Ack
1.
https://issues.apache.org/jira/browse/HBASE/fixforversion/12320040#selectedTab=com.atlassian.jira.plugin.system.project%3Aversion-issues-panel
On Mon, Jun 24, 2013 at 8:45 PM, Stack <st...@duboce.net> wrote:
> (Changed the subject)
>
> On Mon, Jun 24, 2013 at 4:04 PM, Nick Dimiduk <nd...@gmail.com> wrote:
>
>> I want to see initial data type APIs ship out with 0.95.2. A patch for
>> ordered byte serialization is up (HBASE-8201) and is nearing steady-state.
>> However, sershe is the only person who's left feedback. I just posted an
>> early patch for the data type API itself (HBASE-8693). It should get some
>> eyes from all manor of interested parties, but I'll settle for folk from
>> Phoenix for now.
>>
>>
> It would be cool if Phoenix and Kiji fellows and any one else interested
> would weigh in and take a look see.
>
> This does not strike me as something we should hold up the release for
> though. It looks like something that could go in at any time?
>
>
>
>> Should these tasks be escalated to criticals in order to grab attention?
>>
>>
> I don't think that works going by past experience (and I don't think this
> a blocker on 0.96)
>
>
>
>> Additional comments inline.
>>
>>
> ...
>
>
>
>> Namespaces is the long pole and progress seems slow. Do we hold up the
>> > release for them? How can we hurry this effort along? Swat team
>> descends
>> > on Y!?
>> >
>>
>> It would be a shame to not get a decision on this in for 0.96.
>>
>>
> Agree. We need to get 0.96 out though. It has been too long.
>
>
>
>> + Is anyone testing? Integration tests fail on ec2 build from time to
>> time
>> > [2]. Our Elliott dug in on one of the failures a few days back and
>> found
>> > legit issue w/ no retry on admin tasks (I heart hbase-it tests). Our
>> unit
>> > test story is better [3] but there are still the odd failures.
>> >
>>
>> With the creation of the new list, noticing these issues is going to push
>> further back-burner. Nannying this stuff should retain focus. I'll
>> volunteer to track on these issues as I see them.
>>
>>
> Thank you Nick.
>
>
>
>> Notice though that some of the more recent failures are caused by lack of
>> disk space on the Jenkins build host.
>>
>>
> Oh. Missed that. Let me dig in.
>
>
> St.Ack
>
Re: [DISCUSSION] Finishing up 0.96 --> WAS Re: 0.95 and 0.96
remaining issues
Posted by Stack <st...@duboce.net>.
On Mon, Jul 1, 2013 at 1:40 PM, Devaraj Das <dd...@hortonworks.com> wrote:
> BTW I just wanted to bring to everyone's attention the work done over in
> HADOOP-9421. Sanjay Radia has been telling me that a corresponding work
> should be done in HBase as well. That will help keep the compatibility
> story straight when the work to do with pluggable authentication mechanisms
> happening elsewhere in the ecosystem takes shape..
>
> Any thoughts?
>
There is no description of what was finally decided on?
Looks like client initiates and then server and client can negotiate?
Can't we up the rpc version when we want to add support for negotiated
protocols? If so, this does not have to hold up 0.96?
St.Ack
Re: [DISCUSSION] Finishing up 0.96 --> WAS Re: 0.95 and 0.96
remaining issues
Posted by Andrew Purtell <ap...@apache.org>.
On Mon, Jul 1, 2013 at 2:54 PM, Devaraj Das <dd...@hortonworks.com> wrote:
> However doing the equivalent patch
> for HADOOP-9421 in HBase will probably be easier and doable for 0.96...
>
If someone shows up with a patch equivalent to HADOOP-9421 for HBase real
soon now let's look at it. Maybe that is good enough and we will be able to
finesse common authentication on top of two divergent RPC stacks.
Let me throw out there a third way: Hadoop and HBase gets together right
now in a "protobuf RPC wire format working group" and solves at least the
wire side of the problem before one or the other ships something that has
to be set going forward. On the HBase side, that's happening real soon now,
perhaps as early as the end of the month.
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: [DISCUSSION] Finishing up 0.96 --> WAS Re: 0.95 and 0.96
remaining issues
Posted by Devaraj Das <dd...@hortonworks.com>.
On Mon, Jul 1, 2013 at 3:34 PM, Stack <st...@duboce.net> wrote:
> On Mon, Jul 1, 2013 at 2:54 PM, Devaraj Das <dd...@hortonworks.com> wrote:
>
> > > so we might look at using Hadoop RPC directly again?
> >
> > Totally agree with that thought, Andrew. I must admit that I don't know
> > about all the context on why the HBase RPC forked in the first place :-)
> >
> >
> Bug fixes and speed optimizations
>
>
>
> > Your suggestion on using the Hadoop RPC work here makes sense. I am not
> too
> > sure whether that is practical in the short term though (we would need to
> > put in Tracing framework calls in Hadoop RPC and so on). IMO this will
> > further delay the 0.96 shipping as well. However doing the equivalent
> patch
> > for HADOOP-9421 in HBase will probably be easier and doable for 0.96...
> >
> >
> That patch won't go in to hbase. Can you sketch the back and forth finally
> decided upon in an issue? If you do this, I could help on how you might
> get it in.
>
>
I meant to say that using Hadoop RPC with changes needed for HBase to work
makes sense (and, of course, not the patch directly from HADOOP-9421 :-) )
But I'll write up the proposal implemented in HADOOP-9421 in a HBASE jira
and we can go from there.
> St.Ack
>
Re: [DISCUSSION] Finishing up 0.96 --> WAS Re: 0.95 and 0.96
remaining issues
Posted by Devaraj Das <dd...@hortonworks.com>.
On Mon, Jul 1, 2013 at 3:56 PM, Andrew Purtell <ap...@apache.org> wrote:
> On Mon, Jul 1, 2013 at 3:34 PM, Stack <st...@duboce.net> wrote:
>
> > That patch won't go in to hbase.
>
>
> What if we and Hadoop can come up with a common RPC connection setup, like
> PB-based SASL negotiation and a common version header? Then there are
> options post-singularity.
>
>
Hopefully, the HADOOP-9421 is a step in that direction. I am going to spend
some time on HADOOP-9421.
>
> --
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>
Re: [DISCUSSION] Finishing up 0.96 --> WAS Re: 0.95 and 0.96
remaining issues
Posted by Andrew Purtell <ap...@apache.org>.
On Mon, Jul 1, 2013 at 3:34 PM, Stack <st...@duboce.net> wrote:
> That patch won't go in to hbase.
What if we and Hadoop can come up with a common RPC connection setup, like
PB-based SASL negotiation and a common version header? Then there are
options post-singularity.
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: [DISCUSSION] Finishing up 0.96 --> WAS Re: 0.95 and 0.96
remaining issues
Posted by Stack <st...@duboce.net>.
On Mon, Jul 1, 2013 at 2:54 PM, Devaraj Das <dd...@hortonworks.com> wrote:
> > so we might look at using Hadoop RPC directly again?
>
> Totally agree with that thought, Andrew. I must admit that I don't know
> about all the context on why the HBase RPC forked in the first place :-)
>
>
Bug fixes and speed optimizations
> Your suggestion on using the Hadoop RPC work here makes sense. I am not too
> sure whether that is practical in the short term though (we would need to
> put in Tracing framework calls in Hadoop RPC and so on). IMO this will
> further delay the 0.96 shipping as well. However doing the equivalent patch
> for HADOOP-9421 in HBase will probably be easier and doable for 0.96...
>
>
That patch won't go in to hbase. Can you sketch the back and forth finally
decided upon in an issue? If you do this, I could help on how you might
get it in.
St.Ack
Re: [DISCUSSION] Finishing up 0.96 --> WAS Re: 0.95 and 0.96
remaining issues
Posted by Devaraj Das <dd...@hortonworks.com>.
> so we might look at using Hadoop RPC directly again?
Totally agree with that thought, Andrew. I must admit that I don't know
about all the context on why the HBase RPC forked in the first place :-)
Your suggestion on using the Hadoop RPC work here makes sense. I am not too
sure whether that is practical in the short term though (we would need to
put in Tracing framework calls in Hadoop RPC and so on). IMO this will
further delay the 0.96 shipping as well. However doing the equivalent patch
for HADOOP-9421 in HBase will probably be easier and doable for 0.96...
On Mon, Jul 1, 2013 at 1:54 PM, Andrew Purtell <ap...@apache.org> wrote:
> On Mon, Jul 1, 2013 at 1:40 PM, Devaraj Das <dd...@hortonworks.com> wrote:
>
> > HADOOP-9421
>
>
> How about turning that around? (smile) Any thoughts on removing the
> indirection in Hadoop RPC like Stack did with HBase PB RPC so we might look
> at using Hadoop RPC directly again? Seems if the goal is to support
> pluggable authentication mechanisms being developed in core then we should
> be taking that up through a core artifact, not duplicating the work over
> here?
>
>
> --
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>
Re: [DISCUSSION] Finishing up 0.96 --> WAS Re: 0.95 and 0.96
remaining issues
Posted by Andrew Purtell <ap...@apache.org>.
On Mon, Jul 1, 2013 at 1:40 PM, Devaraj Das <dd...@hortonworks.com> wrote:
> HADOOP-9421
How about turning that around? (smile) Any thoughts on removing the
indirection in Hadoop RPC like Stack did with HBase PB RPC so we might look
at using Hadoop RPC directly again? Seems if the goal is to support
pluggable authentication mechanisms being developed in core then we should
be taking that up through a core artifact, not duplicating the work over
here?
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: [DISCUSSION] Finishing up 0.96 --> WAS Re: 0.95 and 0.96
remaining issues
Posted by Devaraj Das <dd...@hortonworks.com>.
BTW I just wanted to bring to everyone's attention the work done over in
HADOOP-9421. Sanjay Radia has been telling me that a corresponding work
should be done in HBase as well. That will help keep the compatibility
story straight when the work to do with pluggable authentication mechanisms
happening elsewhere in the ecosystem takes shape..
Any thoughts?
Thanks
Devaraj
On Mon, Jun 24, 2013 at 8:45 PM, Stack <st...@duboce.net> wrote:
> (Changed the subject)
>
> On Mon, Jun 24, 2013 at 4:04 PM, Nick Dimiduk <nd...@gmail.com> wrote:
>
> > I want to see initial data type APIs ship out with 0.95.2. A patch for
> > ordered byte serialization is up (HBASE-8201) and is nearing
> steady-state.
> > However, sershe is the only person who's left feedback. I just posted an
> > early patch for the data type API itself (HBASE-8693). It should get some
> > eyes from all manor of interested parties, but I'll settle for folk from
> > Phoenix for now.
> >
> >
> It would be cool if Phoenix and Kiji fellows and any one else interested
> would weigh in and take a look see.
>
> This does not strike me as something we should hold up the release for
> though. It looks like something that could go in at any time?
>
>
>
> > Should these tasks be escalated to criticals in order to grab attention?
> >
> >
> I don't think that works going by past experience (and I don't think this a
> blocker on 0.96)
>
>
>
> > Additional comments inline.
> >
> >
> ...
>
>
>
> > Namespaces is the long pole and progress seems slow. Do we hold up the
> > > release for them? How can we hurry this effort along? Swat team
> > descends
> > > on Y!?
> > >
> >
> > It would be a shame to not get a decision on this in for 0.96.
> >
> >
> Agree. We need to get 0.96 out though. It has been too long.
>
>
>
> > + Is anyone testing? Integration tests fail on ec2 build from time to
> time
> > > [2]. Our Elliott dug in on one of the failures a few days back and
> found
> > > legit issue w/ no retry on admin tasks (I heart hbase-it tests). Our
> > unit
> > > test story is better [3] but there are still the odd failures.
> > >
> >
> > With the creation of the new list, noticing these issues is going to push
> > further back-burner. Nannying this stuff should retain focus. I'll
> > volunteer to track on these issues as I see them.
> >
> >
> Thank you Nick.
>
>
>
> > Notice though that some of the more recent failures are caused by lack of
> > disk space on the Jenkins build host.
> >
> >
> Oh. Missed that. Let me dig in.
>
>
> St.Ack
>