[DISCUSSION] Sorting out issues for 0.96 for (eventual) release

[Andrew Purtell <ap...@apache.org>]

There are 118 JIRAs targeting 0.96 specifically. 3 blockers, 10 criticals.
Snapshots will be merged in soon. PBing RPC and persistence is well along
but there are 5945, 6521, 7233, 7448 and others due before release could be
considered. The work is important and as a consequence 0.96 will still need
a few months to come together.

That said, I think we should make up a list of what people want to see in,
and what could/should be excluded after we find that consensus.

I wonder if we might be able to start fielding RCs in the second quarter of
2013 sometime. I think it will still take some time to bake beyond that. I
might be able to get some time on a 1000 node testbed, so would expect a
bunch of issues to fall out of that.

-- 
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Re: [DISCUSSION] Sorting out issues for 0.96 for (eventual) release

[Stack <st...@duboce.net>]

On Tue, Jan 8, 2013 at 10:32 AM, Andrew Purtell <ap...@apache.org> wrote:

> Some time has elapsed since that discussion, I think we should do it again.
>
>
Go for it Andrew.  Would suggest you tag it on to the tail of the old
discussion or at least reference the old in your setup.


> For example, the notion of running on JDK 7 is new, and depends on other
> communities because quite a bit seems broken on JDK 7 so I hear. The creep
> of "-Djava.net.preferIPv4=true" among Hadoop ecosystem projects is also a
> concern, and it bothers me that HDFS tests seem DOA without it on my dev
> box, so this also isn't something we are going to be able to solve entirely
> on our own. So while I think these are serious concerns, I have mixed
> feelings about them being prerequisites for a 0.96 release.
>
>
Agreed.



> A bunch of hard thoughtful intermingled work is underway in RPC and HFile
> and other places, so that we only need to do a "singularity" once. We
> should do a feature based release for this, not a time based one, is my
> opinion. As for everything else, setting a target and seeing what falls in
> or out based on that is worth doing.
>
>
You call it a 'feature release'.  I see it as we can't release until pb'ing
is done (and, pb'ing can't make hbase N times slower than it was previously
-- smile).

St.Ack

Re: [DISCUSSION] Sorting out issues for 0.96 for (eventual) release

[Stack <st...@duboce.net>]

On Wed, Jan 9, 2013 at 4:45 AM, Nicolas Liochon <nk...@gmail.com> wrote:

> It would be great to have a release soon.
> The migration will require some work from the user point of view: if I'm
> not wrong it's mandatory to rewrite the coprocessors and the filters (even
> if it's not mandatory, it seems to be a reasonable task to include in the
> migration).
>

Just endpoint cps (IIUC).


> For this reason, I think there should be a stable-enough beta release that
> people could use to start this migration. Our contract would be to keep the
> coprocessor & filter API stable between the beta and the RC. But it won't
> be to have no regression nor new features between the beta & the RC. And
> this would send a signal "now we're focusing on the delivery".
>
>
Beta sounds like a good idea given the amount of change in 0.96.  Could
make one soon, as soon as we branch.  I could take care of this if folks
like the idea.  Call it 0.96.0-beta?

For MTTR, there are still a lot of stuff to be done, but it's a never
> ending task. There's my work on assignment, but I don't break the existing
> logic, and it's finishing. MTTR is bigger than that, but there will be new
> ideas, so if we wait for the end of this we will never deliver. And real
> world feedback is valuable. So I setting an arbitrary target would be
> acceptable for this specific point imho (objective for MTTR beeing to be
> always under 1 minute for total recovery time).
>
> Agree on the above.   MTTR is much improved in 0.96 as is (Can we
close HBASE-7007?).  We should get the improvements out in the field.



> May be some stuff could make it as a contrib as well (i.e. secondary
> indexes)?
>

Not mad about carrying contribs in hbase.  We've been there.  No problem
pointing/advertising dependent repositories and doing anything in core to
facilitate dependent/supporting work but would rather not carry the stuff
in core; in general contribs start to become a drag on core dev.

Good on you N,
St.Ack

Re: [DISCUSSION] Sorting out issues for 0.96 for (eventual) release

[Devaraj Das <dd...@hortonworks.com>]

Ooops. Wrong thread. Sorry.

On Thu, Jan 10, 2013 at 1:01 PM, Devaraj Das <dd...@hortonworks.com> wrote:
> I'll run some system tests on a cluster. Will update by tonight or tomorrow.

Re: [DISCUSSION] Sorting out issues for 0.96 for (eventual) release

[Devaraj Das <dd...@hortonworks.com>]

I'll run some system tests on a cluster. Will update by tonight or tomorrow.

On Wed, Jan 9, 2013 at 1:12 PM, Andrew Purtell <ap...@apache.org> wrote:
> Right, it's historical. 0.95 would be implicitly a beta release. Calling it
> 0.96-beta would mean the same thing but different from what we did before
> (0.89 ahead of 0.90 as a "developer preview", which of course became much
> more than that over at FB, but that's another story).
>
> On Wed, Jan 9, 2013 at 12:48 PM, Enis Söztutar <en...@gmail.com> wrote:
>
>> I lack the context here, whether why we have adopted the linux's odd/even
>> naming convention. Anyone kind enough to remind us again? It does not
>> matter that much whether we go with 0.95.0-beta or 0.96.0-beta,
>>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)

Re: [DISCUSSION] Sorting out issues for 0.96 for (eventual) release

[Andrew Purtell <ap...@apache.org>]

Right, it's historical. 0.95 would be implicitly a beta release. Calling it
0.96-beta would mean the same thing but different from what we did before
(0.89 ahead of 0.90 as a "developer preview", which of course became much
more than that over at FB, but that's another story).

On Wed, Jan 9, 2013 at 12:48 PM, Enis Söztutar <en...@gmail.com> wrote:

> I lack the context here, whether why we have adopted the linux's odd/even
> naming convention. Anyone kind enough to remind us again? It does not
> matter that much whether we go with 0.95.0-beta or 0.96.0-beta,
>


-- 
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Re: [DISCUSSION] Sorting out issues for 0.96 for (eventual) release

[Enis Söztutar <en...@gmail.com>]

I was also thinking that 0.96.0 should be labeled beta, just because, we
might want to break compat in a couple more cases.

I lack the context here, whether why we have adopted the linux's odd/even
naming convention. Anyone kind enough to remind us again? It does not
matter that much whether we go with 0.95.0-beta or 0.96.0-beta, but I am
just trying to understand. Also we have been publicizing 0.96 for some
time, so suddenly coming out with a release labeled 0.95 might be
confusing. Can we do 0.96.0-beta then 0.96.1 (non beta)?

One more discussion is that I want to raise supporting windows (HBASE-6814)
as a blocker against 0.96. There is still some more work, but I think we
can get them fixed by the time, 0.96.0 comes out. If not, we can agree that
these changes can go in 0.96.1.

Enis


On Wed, Jan 9, 2013 at 10:03 AM, Andrew Purtell <ap...@apache.org> wrote:

> Sounds good to me. Stack, you said you were going to do this? I'll throw my
> hat in the ring if otherwise.
>
>
> On Wed, Jan 9, 2013 at 9:59 AM, Todd Lipcon <to...@cloudera.com> wrote:
>
> > As a somewhat-bystander of late, I think the 0.89 series worked pretty
> > well for us, and I'm in favor of doing it again, if someone's going to
> > volunteer to do the release management.
> >
> > -Todd
> >
> > On Wed, Jan 9, 2013 at 9:57 AM, Andrew Purtell <ap...@apache.org>
> > wrote:
> > > Like a 0.95-beta release, the same as what we did with 0.89?
> > >
> > > On Wed, Jan 9, 2013 at 4:45 AM, Nicolas Liochon <nk...@gmail.com>
> > wrote:
> > >
> > >> For this reason, I think there should be a stable-enough beta release
> > that
> > >> people could use to start this migration.
> > >>
> > >
> > >
> > > --
> > > Best regards,
> > >
> > >    - Andy
> > >
> > > Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > > (via Tom White)
> >
> >
> >
> > --
> > Todd Lipcon
> > Software Engineer, Cloudera
> >
>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>

Re: [DISCUSSION] Sorting out issues for 0.96 for (eventual) release

[Stack <st...@duboce.net>]

On Wed, Feb 6, 2013 at 2:11 PM, Jonathan Hsieh <jo...@cloudera.com> wrote:

> Can someone summarize where we are at with the RPC currently?  The KV
> stuff is to make sure we don't regress from core get/put perf
> characteristics of 0.94 right?
>

On RPC/KV Serialiization:

A patch that specifies new pb-based rpc format is up here:
https://issues.apache.org/jira/browse/HBASE-7533  Still TODO is reedit of
rpc spec and implementation of how we will pass blocks of keyvalues.  Its
been spec'd, but as you'd expect, after you've done an implementation, its
funny how the spec needs to change (for example, I thought we could pass
EncodedDataBlocks originally).  I need to finish implementation.

Passing blocks of KVs everywhere will not be done for 0.96.  This would
require reaching up into base Mutation objects client-side and ditto on the
server-side refactoring so throughout the pipeline we do not have
dependencies on the current serialization format; rather we'd rely on the
Cell Interface that is out in hbase-common instead.  This refactoring is a
bunch of work and won't happen anytime soon.  I was just going to do it a
few places to prove the passing of blocks-of-kvs mechanism works.

We have to do blocks of KeyValues rather than passing of pb'd KVs because
pb marshalling/unmarshalling is 10x slower than our current dumb passing of
the kv backing byte array.  We can't keep passing the current kv backing
byte array because KVs need to evolve going forward [1].

St.Ack

1.
https://docs.google.com/document/d/1WEtrq-JTIUhlnlnvA0oYRLp0F8MKpEBeBSCFcQiacdw/edit#heading=h.su4gd8tohaza

Re: [DISCUSSION] Sorting out issues for 0.96 for (eventual) release

[Jonathan Hsieh <jo...@cloudera.com>]

I'm am in general +1 for this scope lock and branching soon.  There
are a whole lot of non-trivial refactors/features that are close to
commit but I think will cause pain as we try to stabilize.  And with
some of the internals changes that are being proposed (new fs layout)
and have been prototyped (removing root, etc) it looks like 0.96 will
be the first singularity.. to be followed by another singularity
called 0.98 (or 2.0)

Can someone summarize where we are at with the RPC currently?  The KV
stuff is to make sure we don't regress from core get/put perf
characteristics of 0.94 right?

Snapshot work is currently bogged down trying to get tests to pass on
a snapshot+trunk merge (see HBASE-7290 for issues). :(.  Once I get a
clean run there I plan on formally posting the branch for review and
doing the real cluster testing on the merge snapshot+trunk branch
instead of just the snapshot branch.  On this front, we've got the
failures categorized and at the moment have hacks (some unacceptable,
but pointing towards root cause) to fix all but one of these failures.

Jon.

On Wed, Feb 6, 2013 at 1:53 PM, Devaraj Das <dd...@hortonworks.com> wrote:
> Stack, did you lose the bet to JD yet :-)
>
> But seriously, should we consider branching 0.96 now? I know there are
> quite a few blockers/criticals but they can be fixed/merged even after
> the branching. Given that we want to release sooner a 0.96, I think it
> makes sense to branch sooner as well and lock the scope somewhat (that
> will also hopefully avoid creating more new blockers).
>
> What is the absolute list of things we must wait for before branching?
> Should it be only Snapshots? Should it be RPC, KeyValue serialization
> & Snapshots. There are many other things being developed in trunk and
> many of them are probably okay for 0.96 as well if they are completed
> on time. But I think we should put a stake on the ground and consider
> a date for branching and work towards achieving that. After that it'd
> be up to the RM to consider something for 0.96 or not.
>
> Should we aim for Feb 28 as the date for branching. Then we can have
> an RC in approximately 1.5 - 2 months (April mid-end), and it would
> probably be in line with what Andrew was thinking about when he
> started this thread. Does it sound too aggressive?
>
> Thanks,
> Devaraj.
>
> On Thu, Jan 10, 2013 at 4:02 PM, Stack <st...@duboce.net> wrote:
>> We tried to doc our versioning up on wiki:
>> http://wiki.apache.org/hadoop/Hbase/HBaseVersions?action=edit&editor=text
>> Its out of date and needs moving into the ref guide but there is a
>> section
>> we could reuse that Todd did for 89, the "Development Release", explaining
>> 0.95.
>>
>> (Chatting with Todd, he suggested not waiting for 0.96 to branch but just
>> branch first 0.95 from trunk -- we could do that too though I wouldn't mind
>> the rpc stabilizing first...)
>>
>> St.Ack
>>
>>
>> On Thu, Jan 10, 2013 at 3:38 PM, Andrew Purtell <ap...@apache.org> wrote:
>>
>>> On Thu, Jan 10, 2013 at 3:34 PM, Enis Söztutar <en...@gmail.com> wrote:
>>>
>>> > I dont see why odd/even is better than adding -dev/-beta suffix,
>>> > [...] If historically, 0.89/0.90 worked well, we
>>> > might as well go with it.
>>> >
>>>
>>> Right, the notion is to just do what we did before, having had this
>>> discussion previously, to avoid going over this again.
>>>
>>>
>>> --
>>> Best regards,
>>>
>>>    - Andy
>>>
>>> Problems worthy of attack prove their worth by hitting back. - Piet Hein
>>> (via Tom White)
>>>



-- 
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
// jon@cloudera.com

Re: [DISCUSSION] Sorting out issues for 0.96 for (eventual) release

[Stack <st...@duboce.net>]

No feedback on whether February 15th is ok to branch?  Lets say 22nd then.
 If no complaint, will put it up as branch day.

Thanks all,
St.Ack


On Thu, Feb 7, 2013 at 12:41 AM, Nicolas Liochon <nk...@gmail.com> wrote:

> +1 for the maven style snapshots: at the beginning, we can create them
> without branching.
>
>
> On Thu, Feb 7, 2013 at 12:29 AM, Andrew Purtell <ap...@apache.org>
> wrote:
>
> > Sounds good to me, and you're the RM anyway. :-)
> >
> >
> > On Wed, Feb 6, 2013 at 3:26 PM, Stack <st...@duboce.net> wrote:
> >
> > > On Wed, Feb 6, 2013 at 3:11 PM, Andrew Purtell <ap...@apache.org>
> > > wrote:
> > >
> > > > However, we could branch and start releasing 0.95.<date> developer
> > > preview
> > > > releases pretty soon, right?, to start testing the relative stability
> > as
> > > > these things land.
> > > >
> > > >
> > > Rather than 0.95, I was thinking we could just do maven-style snapshot
> > > releases: i.e. 0.96.0.MAVEN_SUPPLIED_DATE
> > >
> > > St.Ack
> > >
> >
> >
> >
> > --
> > Best regards,
> >
> >    - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
> >
>

Re: [DISCUSSION] Sorting out issues for 0.96 for (eventual) release

[Nicolas Liochon <nk...@gmail.com>]

+1 for the maven style snapshots: at the beginning, we can create them
without branching.


On Thu, Feb 7, 2013 at 12:29 AM, Andrew Purtell <ap...@apache.org> wrote:

> Sounds good to me, and you're the RM anyway. :-)
>
>
> On Wed, Feb 6, 2013 at 3:26 PM, Stack <st...@duboce.net> wrote:
>
> > On Wed, Feb 6, 2013 at 3:11 PM, Andrew Purtell <ap...@apache.org>
> > wrote:
> >
> > > However, we could branch and start releasing 0.95.<date> developer
> > preview
> > > releases pretty soon, right?, to start testing the relative stability
> as
> > > these things land.
> > >
> > >
> > Rather than 0.95, I was thinking we could just do maven-style snapshot
> > releases: i.e. 0.96.0.MAVEN_SUPPLIED_DATE
> >
> > St.Ack
> >
>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>

Re: [DISCUSSION] Sorting out issues for 0.96 for (eventual) release

[Andrew Purtell <ap...@apache.org>]

Sounds good to me, and you're the RM anyway. :-)


On Wed, Feb 6, 2013 at 3:26 PM, Stack <st...@duboce.net> wrote:

> On Wed, Feb 6, 2013 at 3:11 PM, Andrew Purtell <ap...@apache.org>
> wrote:
>
> > However, we could branch and start releasing 0.95.<date> developer
> preview
> > releases pretty soon, right?, to start testing the relative stability as
> > these things land.
> >
> >
> Rather than 0.95, I was thinking we could just do maven-style snapshot
> releases: i.e. 0.96.0.MAVEN_SUPPLIED_DATE
>
> St.Ack
>



-- 
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Re: [DISCUSSION] Sorting out issues for 0.96 for (eventual) release

[Stack <st...@duboce.net>]

On Wed, Feb 6, 2013 at 3:11 PM, Andrew Purtell <ap...@apache.org> wrote:

> However, we could branch and start releasing 0.95.<date> developer preview
> releases pretty soon, right?, to start testing the relative stability as
> these things land.
>
>
Rather than 0.95, I was thinking we could just do maven-style snapshot
releases: i.e. 0.96.0.MAVEN_SUPPLIED_DATE

St.Ack

Re: [DISCUSSION] Sorting out issues for 0.96 for (eventual) release

[Andrew Purtell <ap...@apache.org>]

+1 these need to happen before release:

On Wed, Feb 6, 2013 at 2:29 PM, Stack <st...@duboce.net> wrote:

> Was waiting on the outstanding big code movements -- snapshots coming in,
> prefix-tree, client module moves -- and nailing rpc format /keyvalue
> serialization going forward for 0.96 so we don't have to do it in two
> places.



However, we could branch and start releasing 0.95.<date> developer preview
releases pretty soon, right?, to start testing the relative stability as
these things land.

-- 
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Re: [DISCUSSION] Sorting out issues for 0.96 for (eventual) release

[Stack <st...@duboce.net>]

On Wed, Feb 6, 2013 at 1:53 PM, Devaraj Das <dd...@hortonworks.com> wrote:

> Stack, did you lose the bet to JD yet :-)
>
>
It is not looking good.



> But seriously, should we consider branching 0.96 now? I know there are
> quite a few blockers/criticals but they can be fixed/merged even after
> the branching. Given that we want to release sooner a 0.96, I think it
> makes sense to branch sooner as well and lock the scope somewhat (that
> will also hopefully avoid creating more new blockers).
>
>
Was waiting on the outstanding big code movements -- snapshots coming in,
prefix-tree, client module moves -- and nailing rpc format /keyvalue
serialization going forward for 0.96 so we don't have to do it in two
places.




> Should it be only Snapshots? Should it be RPC, KeyValue serialization
> & Snapshots. There are many other things being developed in trunk and
> many of them are probably okay for 0.96 as well if they are completed
> on time. But I think we should put a stake on the ground and consider
> a date for branching and work towards achieving that. After that it'd
> be up to the RM to consider something for 0.96 or not.
>
>
Agreed.



> Should we aim for Feb 28 as the date for branching. Then we can have
> an RC in approximately 1.5 - 2 months (April mid-end), and it would
> probably be in line with what Andrew was thinking about when he
> started this thread. Does it sound too aggressive?
>

How about Feb15th?

Thanks for revivifying this thread DD.
St.Ack

Re: [DISCUSSION] Sorting out issues for 0.96 for (eventual) release

[Devaraj Das <dd...@hortonworks.com>]

Stack, did you lose the bet to JD yet :-)

But seriously, should we consider branching 0.96 now? I know there are
quite a few blockers/criticals but they can be fixed/merged even after
the branching. Given that we want to release sooner a 0.96, I think it
makes sense to branch sooner as well and lock the scope somewhat (that
will also hopefully avoid creating more new blockers).

What is the absolute list of things we must wait for before branching?
Should it be only Snapshots? Should it be RPC, KeyValue serialization
& Snapshots. There are many other things being developed in trunk and
many of them are probably okay for 0.96 as well if they are completed
on time. But I think we should put a stake on the ground and consider
a date for branching and work towards achieving that. After that it'd
be up to the RM to consider something for 0.96 or not.

Should we aim for Feb 28 as the date for branching. Then we can have
an RC in approximately 1.5 - 2 months (April mid-end), and it would
probably be in line with what Andrew was thinking about when he
started this thread. Does it sound too aggressive?

Thanks,
Devaraj.

On Thu, Jan 10, 2013 at 4:02 PM, Stack <st...@duboce.net> wrote:
> We tried to doc our versioning up on wiki:
> http://wiki.apache.org/hadoop/Hbase/HBaseVersions?action=edit&editor=text
> Its out of date and needs moving into the ref guide but there is a
> section
> we could reuse that Todd did for 89, the "Development Release", explaining
> 0.95.
>
> (Chatting with Todd, he suggested not waiting for 0.96 to branch but just
> branch first 0.95 from trunk -- we could do that too though I wouldn't mind
> the rpc stabilizing first...)
>
> St.Ack
>
>
> On Thu, Jan 10, 2013 at 3:38 PM, Andrew Purtell <ap...@apache.org> wrote:
>
>> On Thu, Jan 10, 2013 at 3:34 PM, Enis Söztutar <en...@gmail.com> wrote:
>>
>> > I dont see why odd/even is better than adding -dev/-beta suffix,
>> > [...] If historically, 0.89/0.90 worked well, we
>> > might as well go with it.
>> >
>>
>> Right, the notion is to just do what we did before, having had this
>> discussion previously, to avoid going over this again.
>>
>>
>> --
>> Best regards,
>>
>>    - Andy
>>
>> Problems worthy of attack prove their worth by hitting back. - Piet Hein
>> (via Tom White)
>>

Re: [DISCUSSION] Sorting out issues for 0.96 for (eventual) release

[Stack <st...@duboce.net>]

We tried to doc our versioning up on wiki:
http://wiki.apache.org/hadoop/Hbase/HBaseVersions?action=edit&editor=text
Its out of date and needs moving into the ref guide but there is a
section
we could reuse that Todd did for 89, the "Development Release", explaining
0.95.

(Chatting with Todd, he suggested not waiting for 0.96 to branch but just
branch first 0.95 from trunk -- we could do that too though I wouldn't mind
the rpc stabilizing first...)

St.Ack


On Thu, Jan 10, 2013 at 3:38 PM, Andrew Purtell <ap...@apache.org> wrote:

> On Thu, Jan 10, 2013 at 3:34 PM, Enis Söztutar <en...@gmail.com> wrote:
>
> > I dont see why odd/even is better than adding -dev/-beta suffix,
> > [...] If historically, 0.89/0.90 worked well, we
> > might as well go with it.
> >
>
> Right, the notion is to just do what we did before, having had this
> discussion previously, to avoid going over this again.
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>

Re: [DISCUSSION] Sorting out issues for 0.96 for (eventual) release

[Andrew Purtell <ap...@apache.org>]

On Thu, Jan 10, 2013 at 3:34 PM, Enis Söztutar <en...@gmail.com> wrote:

> I dont see why odd/even is better than adding -dev/-beta suffix,
> [...] If historically, 0.89/0.90 worked well, we
> might as well go with it.
>

Right, the notion is to just do what we did before, having had this
discussion previously, to avoid going over this again.


-- 
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Re: [DISCUSSION] Sorting out issues for 0.96 for (eventual) release

[Enis Söztutar <en...@gmail.com>]

I dont see why odd/even is better than adding -dev/-beta suffix,
which explicitly indicates that this is a development release, rather than
the user having to consult to documentation to understand it. But agreed
with John in that we don't want to go into a never ending discussion of
which versioning schema to go. If historically, 0.89/0.90 worked well, we
might as well go with it.


On Thu, Jan 10, 2013 at 3:13 PM, Todd Lipcon <to...@cloudera.com> wrote:

> I wrote up some stuff on this back in the 0.89 timeframe here:
> http://wiki.apache.org/hadoop/Hbase/HBaseVersions
>
> Could probably do with a refresh of the text, but should mostly still
> apply.
>
> -Todd
>
> On Thu, Jan 10, 2013 at 3:11 PM, Andrew Purtell <ap...@apache.org>
> wrote:
> > I propose to avoid -dev or -beta, and stick with the even/odd scheme we
> > used for 0.89. We also appended a date stamp instead of minor version,
> e.g.
> > 0.89.20100726. Documenting how this departure from our usual numbering
> > signifies a "developer preview", or whatever we'd like to call it, sounds
> > like a good idea.
> >
> > On Thu, Jan 10, 2013 at 2:55 PM, Jonathan Hsieh <jo...@cloudera.com>
> wrote:
> >
> >> I think that branching scheme makes sense.
> >>
> >> We should probably define the intent of even/odd versioning and compat
> >> rules on the webpage (the how to release instructions and on the
> >> download links) if we are going to do it so we don't have to explain
> >> it over and over.  If we do this ahead of time,  everyone should have
> >> the same expectations knows what this means.
> >>
> >> Also, we could consider probably playing some games with adding -dev
> >> or -beta after an odd version number.  I say this with some
> >> trepidation - over in Hadoop-land they there are some contentious
> >> discussions about version numbering and naming that I'd personally
> >> like to avoid.
> >>
> >> Jon.
> >>
> >> On Thu, Jan 10, 2013 at 2:15 PM, Andrew Purtell <ap...@apache.org>
> >> wrote:
> >> > On Thu, Jan 10, 2013 at 2:00 PM, Stack <st...@duboce.net> wrote:
> >> >
> >> >> How do we want to run it?  Branch 0.96 and then 0.95s are branched
> from
> >> >> 0.96?  (As in the past, 0.95.0, 0.95.1, etc., would come with no
> >> guarantees
> >> >> other than it basically works and it is allowed that 0.95.1 may not
> be
> >> >> compatible with 0.95.0, etc.).
> >> >>
> >> >
> >> >
> >> > +1
> >> >
> >> > --
> >> > Best regards,
> >> >
> >> >    - Andy
> >> >
> >> > Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> >> > (via Tom White)
> >>
> >>
> >>
> >> --
> >> // Jonathan Hsieh (shay)
> >> // Software Engineer, Cloudera
> >> // jon@cloudera.com
> >>
> >
> >
> >
> > --
> > Best regards,
> >
> >    - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>

Re: [DISCUSSION] Sorting out issues for 0.96 for (eventual) release

[Todd Lipcon <to...@cloudera.com>]

I wrote up some stuff on this back in the 0.89 timeframe here:
http://wiki.apache.org/hadoop/Hbase/HBaseVersions

Could probably do with a refresh of the text, but should mostly still apply.

-Todd

On Thu, Jan 10, 2013 at 3:11 PM, Andrew Purtell <ap...@apache.org> wrote:
> I propose to avoid -dev or -beta, and stick with the even/odd scheme we
> used for 0.89. We also appended a date stamp instead of minor version, e.g.
> 0.89.20100726. Documenting how this departure from our usual numbering
> signifies a "developer preview", or whatever we'd like to call it, sounds
> like a good idea.
>
> On Thu, Jan 10, 2013 at 2:55 PM, Jonathan Hsieh <jo...@cloudera.com> wrote:
>
>> I think that branching scheme makes sense.
>>
>> We should probably define the intent of even/odd versioning and compat
>> rules on the webpage (the how to release instructions and on the
>> download links) if we are going to do it so we don't have to explain
>> it over and over.  If we do this ahead of time,  everyone should have
>> the same expectations knows what this means.
>>
>> Also, we could consider probably playing some games with adding -dev
>> or -beta after an odd version number.  I say this with some
>> trepidation - over in Hadoop-land they there are some contentious
>> discussions about version numbering and naming that I'd personally
>> like to avoid.
>>
>> Jon.
>>
>> On Thu, Jan 10, 2013 at 2:15 PM, Andrew Purtell <ap...@apache.org>
>> wrote:
>> > On Thu, Jan 10, 2013 at 2:00 PM, Stack <st...@duboce.net> wrote:
>> >
>> >> How do we want to run it?  Branch 0.96 and then 0.95s are branched from
>> >> 0.96?  (As in the past, 0.95.0, 0.95.1, etc., would come with no
>> guarantees
>> >> other than it basically works and it is allowed that 0.95.1 may not be
>> >> compatible with 0.95.0, etc.).
>> >>
>> >
>> >
>> > +1
>> >
>> > --
>> > Best regards,
>> >
>> >    - Andy
>> >
>> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
>> > (via Tom White)
>>
>>
>>
>> --
>> // Jonathan Hsieh (shay)
>> // Software Engineer, Cloudera
>> // jon@cloudera.com
>>
>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)



-- 
Todd Lipcon
Software Engineer, Cloudera

Re: [DISCUSSION] Sorting out issues for 0.96 for (eventual) release

[Andrew Purtell <ap...@apache.org>]

I propose to avoid -dev or -beta, and stick with the even/odd scheme we
used for 0.89. We also appended a date stamp instead of minor version, e.g.
0.89.20100726. Documenting how this departure from our usual numbering
signifies a "developer preview", or whatever we'd like to call it, sounds
like a good idea.

On Thu, Jan 10, 2013 at 2:55 PM, Jonathan Hsieh <jo...@cloudera.com> wrote:

> I think that branching scheme makes sense.
>
> We should probably define the intent of even/odd versioning and compat
> rules on the webpage (the how to release instructions and on the
> download links) if we are going to do it so we don't have to explain
> it over and over.  If we do this ahead of time,  everyone should have
> the same expectations knows what this means.
>
> Also, we could consider probably playing some games with adding -dev
> or -beta after an odd version number.  I say this with some
> trepidation - over in Hadoop-land they there are some contentious
> discussions about version numbering and naming that I'd personally
> like to avoid.
>
> Jon.
>
> On Thu, Jan 10, 2013 at 2:15 PM, Andrew Purtell <ap...@apache.org>
> wrote:
> > On Thu, Jan 10, 2013 at 2:00 PM, Stack <st...@duboce.net> wrote:
> >
> >> How do we want to run it?  Branch 0.96 and then 0.95s are branched from
> >> 0.96?  (As in the past, 0.95.0, 0.95.1, etc., would come with no
> guarantees
> >> other than it basically works and it is allowed that 0.95.1 may not be
> >> compatible with 0.95.0, etc.).
> >>
> >
> >
> > +1
> >
> > --
> > Best regards,
> >
> >    - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
>
>
>
> --
> // Jonathan Hsieh (shay)
> // Software Engineer, Cloudera
> // jon@cloudera.com
>



-- 
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Re: [DISCUSSION] Sorting out issues for 0.96 for (eventual) release

[Jonathan Hsieh <jo...@cloudera.com>]

I think that branching scheme makes sense.

We should probably define the intent of even/odd versioning and compat
rules on the webpage (the how to release instructions and on the
download links) if we are going to do it so we don't have to explain
it over and over.  If we do this ahead of time,  everyone should have
the same expectations knows what this means.

Also, we could consider probably playing some games with adding -dev
or -beta after an odd version number.  I say this with some
trepidation - over in Hadoop-land they there are some contentious
discussions about version numbering and naming that I'd personally
like to avoid.

Jon.

On Thu, Jan 10, 2013 at 2:15 PM, Andrew Purtell <ap...@apache.org> wrote:
> On Thu, Jan 10, 2013 at 2:00 PM, Stack <st...@duboce.net> wrote:
>
>> How do we want to run it?  Branch 0.96 and then 0.95s are branched from
>> 0.96?  (As in the past, 0.95.0, 0.95.1, etc., would come with no guarantees
>> other than it basically works and it is allowed that 0.95.1 may not be
>> compatible with 0.95.0, etc.).
>>
>
>
> +1
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)



-- 
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
// jon@cloudera.com

Re: [DISCUSSION] Sorting out issues for 0.96 for (eventual) release

[Andrew Purtell <ap...@apache.org>]

On Thu, Jan 10, 2013 at 2:00 PM, Stack <st...@duboce.net> wrote:

> How do we want to run it?  Branch 0.96 and then 0.95s are branched from
> 0.96?  (As in the past, 0.95.0, 0.95.1, etc., would come with no guarantees
> other than it basically works and it is allowed that 0.95.1 may not be
> compatible with 0.95.0, etc.).
>


+1

-- 
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Re: [DISCUSSION] Sorting out issues for 0.96 for (eventual) release

[Stack <st...@duboce.net>]

On Wed, Jan 9, 2013 at 10:03 AM, Andrew Purtell <ap...@apache.org> wrote:

> Sounds good to me. Stack, you said you were going to do this? I'll throw my
> hat in the ring if otherwise.
>
>
I don't mind running it.

How do we want to run it?  Branch 0.96 and then 0.95s are branched from
0.96?  (As in the past, 0.95.0, 0.95.1, etc., would come with no guarantees
other than it basically works and it is allowed that 0.95.1 may not be
compatible with 0.95.0, etc.).

St.Ack

Re: [DISCUSSION] Sorting out issues for 0.96 for (eventual) release

[Andrew Purtell <ap...@apache.org>]

Sounds good to me. Stack, you said you were going to do this? I'll throw my
hat in the ring if otherwise.


On Wed, Jan 9, 2013 at 9:59 AM, Todd Lipcon <to...@cloudera.com> wrote:

> As a somewhat-bystander of late, I think the 0.89 series worked pretty
> well for us, and I'm in favor of doing it again, if someone's going to
> volunteer to do the release management.
>
> -Todd
>
> On Wed, Jan 9, 2013 at 9:57 AM, Andrew Purtell <ap...@apache.org>
> wrote:
> > Like a 0.95-beta release, the same as what we did with 0.89?
> >
> > On Wed, Jan 9, 2013 at 4:45 AM, Nicolas Liochon <nk...@gmail.com>
> wrote:
> >
> >> For this reason, I think there should be a stable-enough beta release
> that
> >> people could use to start this migration.
> >>
> >
> >
> > --
> > Best regards,
> >
> >    - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>



-- 
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Re: [DISCUSSION] Sorting out issues for 0.96 for (eventual) release

[Todd Lipcon <to...@cloudera.com>]

As a somewhat-bystander of late, I think the 0.89 series worked pretty
well for us, and I'm in favor of doing it again, if someone's going to
volunteer to do the release management.

-Todd

On Wed, Jan 9, 2013 at 9:57 AM, Andrew Purtell <ap...@apache.org> wrote:
> Like a 0.95-beta release, the same as what we did with 0.89?
>
> On Wed, Jan 9, 2013 at 4:45 AM, Nicolas Liochon <nk...@gmail.com> wrote:
>
>> For this reason, I think there should be a stable-enough beta release that
>> people could use to start this migration.
>>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)



-- 
Todd Lipcon
Software Engineer, Cloudera

Re: [DISCUSSION] Sorting out issues for 0.96 for (eventual) release

[Andrew Purtell <ap...@apache.org>]

Like a 0.95-beta release, the same as what we did with 0.89?

On Wed, Jan 9, 2013 at 4:45 AM, Nicolas Liochon <nk...@gmail.com> wrote:

> For this reason, I think there should be a stable-enough beta release that
> people could use to start this migration.
>


-- 
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Re: [DISCUSSION] Sorting out issues for 0.96 for (eventual) release

[Nicolas Liochon <nk...@gmail.com>]

It would be great to have a release soon.
The migration will require some work from the user point of view: if I'm
not wrong it's mandatory to rewrite the coprocessors and the filters (even
if it's not mandatory, it seems to be a reasonable task to include in the
migration).
For this reason, I think there should be a stable-enough beta release that
people could use to start this migration. Our contract would be to keep the
coprocessor & filter API stable between the beta and the RC. But it won't
be to have no regression nor new features between the beta & the RC. And
this would send a signal "now we're focusing on the delivery".

For the JDK 1.7, so far the changes we're doing for the support are not
something we would refuse to do in a 96.1 release: I mean we're not going
to say: "to run on 1.7 we need to do X, and X is to risky to be done on a
minor release". So it could be removed from the critical path.

For MTTR, there are still a lot of stuff to be done, but it's a never
ending task. There's my work on assignment, but I don't break the existing
logic, and it's finishing. MTTR is bigger than that, but there will be new
ideas, so if we wait for the end of this we will never deliver. And real
world feedback is valuable. So I setting an arbitrary target would be
acceptable for this specific point imho (objective for MTTR beeing to be
always under 1 minute for total recovery time).

May be some stuff could make it as a contrib as well (i.e. secondary
indexes)?

Cheers,

Nicolas




On Wed, Jan 9, 2013 at 5:05 AM, Andrew Purtell <ap...@apache.org> wrote:

> Ted you seem to have picked some random part of this conversation and
> misunderstood it?
>
> On Tuesday, January 8, 2013, Ted Yu wrote:
>
> > User has the flexibility of running 0.96 on jdk 1.6
> >
> > Cheers
> >
> > On Tue, Jan 8, 2013 at 7:59 PM, Andrew Purtell <apurtell@apache.org
> <javascript:;>>
> > wrote:
> >
> > > What do these two things have to do with each other?
> > >
> > > On Tuesday, January 8, 2013, Ted Yu wrote:
> > >
> > > > bq. depends on other communities because quite a bit seems broken on
> > JDK
> > > 7
> > > > so I hear
> > > >
> > > > I have been running tests using 1.6.0_37 for trunk. Tests run
> smoothly
> > so
> > > > far.
> > > >
> > > > FYI
> > > >
> > > > On Tue, Jan 8, 2013 at 10:32 AM, Andrew Purtell <apurtell@apache.org
> <javascript:;>
> > > <javascript:;>>
> > > > wrote:
> > > >
> > > > > On Tue, Jan 8, 2013 at 10:21 AM, Stack <stack@duboce.net
> <javascript:;>
> > <javascript:;>>
> > > > wrote:
> > > > >
> > > > > > > That said, I think we should make up a list of what people want
> > to
> > > > see
> > > > > > in,
> > > > > > > and what could/should be excluded after we find that consensus.
> > > > > > >
> > > > > > >
> > > > > > High-level, we have had this discussion IMO ([3]).  Would you
> like
> > to
> > > > do
> > > > > it
> > > > > > again Andrew?
> > > > > >
> > > > > > Otherwise, I think folks need to bubble up critical issues (or
> > strike
> > > > > them
> > > > > > down) and if any controversial, lets discuss them?
> > > > > >
> > > > >
> > > > > Some time has elapsed since that discussion, I think we should do
> it
> > > > again.
> > > > >
> > > > > For example, the notion of running on JDK 7 is new, and depends on
> > > other
> > > > > communities because quite a bit seems broken on JDK 7 so I hear.
> The
> > > > creep
> > > > > of "-Djava.net.preferIPv4=true" among Hadoop ecosystem projects is
> > > also a
> > > > > concern, and it bothers me that HDFS tests seem DOA without it on
> my
> > > dev
> > > > > box, so this also isn't something we are going to be able to solve
> > > > entirely
> > > > > on our own. So while I think these are serious concerns, I have
> mixed
> > > > > feelings about them being prerequisites for a 0.96 release.
> > > > >
> > > > > A bunch of hard thoughtful intermingled work is underway in RPC and
> > > HFile
> > > > > and other places, so that we only need to do a "singularity" once.
> We
> > > > > should do a feature based release for this, not a time based one,
> is
> > my
> > > > > opinion. As for everything else, setting a target and seeing what
> > falls
> > > > in
> > > > > or out based on that is worth doing.
> > > > >
> > > > > --
> > > > > Best regards,
> > > > >
> > > > >    - Andy
> > > > >
> > > > > Problems worthy of attack prove their worth by hitting back. - Piet
> > > Hein
> > > > > (via Tom White)
> > > > >
> > > >
> > >
> > >
> > > --
> > > Best regards,
> > >
> > >    - Andy
> > >
> > > Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > > (via Tom White)
> > >
> >
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>

Re: [DISCUSSION] Sorting out issues for 0.96 for (eventual) release

[Andrew Purtell <ap...@apache.org>]

Ted you seem to have picked some random part of this conversation and
misunderstood it?

On Tuesday, January 8, 2013, Ted Yu wrote:

> User has the flexibility of running 0.96 on jdk 1.6
>
> Cheers
>
> On Tue, Jan 8, 2013 at 7:59 PM, Andrew Purtell <apurtell@apache.org<javascript:;>>
> wrote:
>
> > What do these two things have to do with each other?
> >
> > On Tuesday, January 8, 2013, Ted Yu wrote:
> >
> > > bq. depends on other communities because quite a bit seems broken on
> JDK
> > 7
> > > so I hear
> > >
> > > I have been running tests using 1.6.0_37 for trunk. Tests run smoothly
> so
> > > far.
> > >
> > > FYI
> > >
> > > On Tue, Jan 8, 2013 at 10:32 AM, Andrew Purtell <apurtell@apache.org<javascript:;>
> > <javascript:;>>
> > > wrote:
> > >
> > > > On Tue, Jan 8, 2013 at 10:21 AM, Stack <stack@duboce.net<javascript:;>
> <javascript:;>>
> > > wrote:
> > > >
> > > > > > That said, I think we should make up a list of what people want
> to
> > > see
> > > > > in,
> > > > > > and what could/should be excluded after we find that consensus.
> > > > > >
> > > > > >
> > > > > High-level, we have had this discussion IMO ([3]).  Would you like
> to
> > > do
> > > > it
> > > > > again Andrew?
> > > > >
> > > > > Otherwise, I think folks need to bubble up critical issues (or
> strike
> > > > them
> > > > > down) and if any controversial, lets discuss them?
> > > > >
> > > >
> > > > Some time has elapsed since that discussion, I think we should do it
> > > again.
> > > >
> > > > For example, the notion of running on JDK 7 is new, and depends on
> > other
> > > > communities because quite a bit seems broken on JDK 7 so I hear. The
> > > creep
> > > > of "-Djava.net.preferIPv4=true" among Hadoop ecosystem projects is
> > also a
> > > > concern, and it bothers me that HDFS tests seem DOA without it on my
> > dev
> > > > box, so this also isn't something we are going to be able to solve
> > > entirely
> > > > on our own. So while I think these are serious concerns, I have mixed
> > > > feelings about them being prerequisites for a 0.96 release.
> > > >
> > > > A bunch of hard thoughtful intermingled work is underway in RPC and
> > HFile
> > > > and other places, so that we only need to do a "singularity" once. We
> > > > should do a feature based release for this, not a time based one, is
> my
> > > > opinion. As for everything else, setting a target and seeing what
> falls
> > > in
> > > > or out based on that is worth doing.
> > > >
> > > > --
> > > > Best regards,
> > > >
> > > >    - Andy
> > > >
> > > > Problems worthy of attack prove their worth by hitting back. - Piet
> > Hein
> > > > (via Tom White)
> > > >
> > >
> >
> >
> > --
> > Best regards,
> >
> >    - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
> >
>


-- 
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Re: [DISCUSSION] Sorting out issues for 0.96 for (eventual) release

[Ted Yu <yu...@gmail.com>]

User has the flexibility of running 0.96 on jdk 1.6

Cheers

On Tue, Jan 8, 2013 at 7:59 PM, Andrew Purtell <ap...@apache.org> wrote:

> What do these two things have to do with each other?
>
> On Tuesday, January 8, 2013, Ted Yu wrote:
>
> > bq. depends on other communities because quite a bit seems broken on JDK
> 7
> > so I hear
> >
> > I have been running tests using 1.6.0_37 for trunk. Tests run smoothly so
> > far.
> >
> > FYI
> >
> > On Tue, Jan 8, 2013 at 10:32 AM, Andrew Purtell <apurtell@apache.org
> <javascript:;>>
> > wrote:
> >
> > > On Tue, Jan 8, 2013 at 10:21 AM, Stack <stack@duboce.net<javascript:;>>
> > wrote:
> > >
> > > > > That said, I think we should make up a list of what people want to
> > see
> > > > in,
> > > > > and what could/should be excluded after we find that consensus.
> > > > >
> > > > >
> > > > High-level, we have had this discussion IMO ([3]).  Would you like to
> > do
> > > it
> > > > again Andrew?
> > > >
> > > > Otherwise, I think folks need to bubble up critical issues (or strike
> > > them
> > > > down) and if any controversial, lets discuss them?
> > > >
> > >
> > > Some time has elapsed since that discussion, I think we should do it
> > again.
> > >
> > > For example, the notion of running on JDK 7 is new, and depends on
> other
> > > communities because quite a bit seems broken on JDK 7 so I hear. The
> > creep
> > > of "-Djava.net.preferIPv4=true" among Hadoop ecosystem projects is
> also a
> > > concern, and it bothers me that HDFS tests seem DOA without it on my
> dev
> > > box, so this also isn't something we are going to be able to solve
> > entirely
> > > on our own. So while I think these are serious concerns, I have mixed
> > > feelings about them being prerequisites for a 0.96 release.
> > >
> > > A bunch of hard thoughtful intermingled work is underway in RPC and
> HFile
> > > and other places, so that we only need to do a "singularity" once. We
> > > should do a feature based release for this, not a time based one, is my
> > > opinion. As for everything else, setting a target and seeing what falls
> > in
> > > or out based on that is worth doing.
> > >
> > > --
> > > Best regards,
> > >
> > >    - Andy
> > >
> > > Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > > (via Tom White)
> > >
> >
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>

Re: [DISCUSSION] Sorting out issues for 0.96 for (eventual) release

[Andrew Purtell <ap...@apache.org>]

What do these two things have to do with each other?

On Tuesday, January 8, 2013, Ted Yu wrote:

> bq. depends on other communities because quite a bit seems broken on JDK 7
> so I hear
>
> I have been running tests using 1.6.0_37 for trunk. Tests run smoothly so
> far.
>
> FYI
>
> On Tue, Jan 8, 2013 at 10:32 AM, Andrew Purtell <apurtell@apache.org<javascript:;>>
> wrote:
>
> > On Tue, Jan 8, 2013 at 10:21 AM, Stack <stack@duboce.net <javascript:;>>
> wrote:
> >
> > > > That said, I think we should make up a list of what people want to
> see
> > > in,
> > > > and what could/should be excluded after we find that consensus.
> > > >
> > > >
> > > High-level, we have had this discussion IMO ([3]).  Would you like to
> do
> > it
> > > again Andrew?
> > >
> > > Otherwise, I think folks need to bubble up critical issues (or strike
> > them
> > > down) and if any controversial, lets discuss them?
> > >
> >
> > Some time has elapsed since that discussion, I think we should do it
> again.
> >
> > For example, the notion of running on JDK 7 is new, and depends on other
> > communities because quite a bit seems broken on JDK 7 so I hear. The
> creep
> > of "-Djava.net.preferIPv4=true" among Hadoop ecosystem projects is also a
> > concern, and it bothers me that HDFS tests seem DOA without it on my dev
> > box, so this also isn't something we are going to be able to solve
> entirely
> > on our own. So while I think these are serious concerns, I have mixed
> > feelings about them being prerequisites for a 0.96 release.
> >
> > A bunch of hard thoughtful intermingled work is underway in RPC and HFile
> > and other places, so that we only need to do a "singularity" once. We
> > should do a feature based release for this, not a time based one, is my
> > opinion. As for everything else, setting a target and seeing what falls
> in
> > or out based on that is worth doing.
> >
> > --
> > Best regards,
> >
> >    - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
> >
>


-- 
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Re: [DISCUSSION] Sorting out issues for 0.96 for (eventual) release

[Ted Yu <yu...@gmail.com>]

bq. depends on other communities because quite a bit seems broken on JDK 7
so I hear

I have been running tests using 1.6.0_37 for trunk. Tests run smoothly so
far.

FYI

On Tue, Jan 8, 2013 at 10:32 AM, Andrew Purtell <ap...@apache.org> wrote:

> On Tue, Jan 8, 2013 at 10:21 AM, Stack <st...@duboce.net> wrote:
>
> > > That said, I think we should make up a list of what people want to see
> > in,
> > > and what could/should be excluded after we find that consensus.
> > >
> > >
> > High-level, we have had this discussion IMO ([3]).  Would you like to do
> it
> > again Andrew?
> >
> > Otherwise, I think folks need to bubble up critical issues (or strike
> them
> > down) and if any controversial, lets discuss them?
> >
>
> Some time has elapsed since that discussion, I think we should do it again.
>
> For example, the notion of running on JDK 7 is new, and depends on other
> communities because quite a bit seems broken on JDK 7 so I hear. The creep
> of "-Djava.net.preferIPv4=true" among Hadoop ecosystem projects is also a
> concern, and it bothers me that HDFS tests seem DOA without it on my dev
> box, so this also isn't something we are going to be able to solve entirely
> on our own. So while I think these are serious concerns, I have mixed
> feelings about them being prerequisites for a 0.96 release.
>
> A bunch of hard thoughtful intermingled work is underway in RPC and HFile
> and other places, so that we only need to do a "singularity" once. We
> should do a feature based release for this, not a time based one, is my
> opinion. As for everything else, setting a target and seeing what falls in
> or out based on that is worth doing.
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>

Re: [DISCUSSION] Sorting out issues for 0.96 for (eventual) release

[Andrew Purtell <ap...@apache.org>]

On Tue, Jan 8, 2013 at 10:21 AM, Stack <st...@duboce.net> wrote:

> > That said, I think we should make up a list of what people want to see
> in,
> > and what could/should be excluded after we find that consensus.
> >
> >
> High-level, we have had this discussion IMO ([3]).  Would you like to do it
> again Andrew?
>
> Otherwise, I think folks need to bubble up critical issues (or strike them
> down) and if any controversial, lets discuss them?
>

Some time has elapsed since that discussion, I think we should do it again.

For example, the notion of running on JDK 7 is new, and depends on other
communities because quite a bit seems broken on JDK 7 so I hear. The creep
of "-Djava.net.preferIPv4=true" among Hadoop ecosystem projects is also a
concern, and it bothers me that HDFS tests seem DOA without it on my dev
box, so this also isn't something we are going to be able to solve entirely
on our own. So while I think these are serious concerns, I have mixed
feelings about them being prerequisites for a 0.96 release.

A bunch of hard thoughtful intermingled work is underway in RPC and HFile
and other places, so that we only need to do a "singularity" once. We
should do a feature based release for this, not a time based one, is my
opinion. As for everything else, setting a target and seeing what falls in
or out based on that is worth doing.

-- 
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Re: [DISCUSSION] Sorting out issues for 0.96 for (eventual) release

[Stack <st...@duboce.net>]

Thanks for doing the below Andrew.

I was going to write something to try and rally folks around a 0.96
release.  I see loads of work afoot.  I am not sure what percentage can be
judged advancing the 0.96RC cause.

See below.

On Tue, Jan 8, 2013 at 9:51 AM, Andrew Purtell <ap...@apache.org> wrote:

> There are 118 JIRAs targeting 0.96 specifically. 3 blockers, 10 criticals.
>


As I see it, closing above is all that stands in the way of our branching.
 I see different numbers to you if I look on this page [1] (I see many more
issues than you count).

A while back after survey [3], here is what we said would be in 0.96 (You
can see the summary when you list 'versions' [2] in JIRA): "....you will
have to start and stop your cluster bringing it up on 0.96.0. 0.96.0 will
require Hadoop 1.0.0 at least. It will be supported on Hadoop 2.0.0 too.
0.96.0 will be all protobufs all the time; all of its serializations to
zookeeper, to the filesystem, and over rpc will be protobufs. It runs on
jdk7. Metrics will be edited and converted to use Hadoop Metrics2. A
preview of HBase Snapshots and PrefixTreeCompression."

Not mentioned but which will be immediately apparent when 0.96 comes up
will be new UI (smile), Jimmy et als AssignmentManager cleanup, Nicolas and
company's MTTR work -- the list is long.



> Snapshots will be merged in soon. PBing RPC and persistence is well along
> but there are 5945, 6521, 7233, 7448 and others due before release could be
> considered. The work is important and as a consequence 0.96 will still need
> a few months to come together.
>
>
Yeah, these are significant issues but they are being worked on actively at
the moment.

I wish it wasn't months.  It is not good for the project that so much time
has elapsed between major releases.



> That said, I think we should make up a list of what people want to see in,
> and what could/should be excluded after we find that consensus.
>
>
High-level, we have had this discussion IMO ([3]).  Would you like to do it
again Andrew?

Otherwise, I think folks need to bubble up critical issues (or strike them
down) and if any controversial, lets discuss them?


> I wonder if we might be able to start fielding RCs in the second quarter of
> 2013 sometime. I think it will still take some time to bake beyond that. I
> might be able to get some time on a 1000 node testbed, so would expect a
> bunch of issues to fall out of that.
>

I have a bet with JD on when 0.96 will go out.  Going by the timeline you
sling above, it looks like I am going to lose my bet.

Good on you Andrew,
St.Ack


1.
https://issues.apache.org/jira/browse/HBASE/fixforversion/12320040#selectedTab=com.atlassian.jira.plugin.system.project%3Aversion-issues-panel
2.
https://issues.apache.org/jira/plugins/servlet/project-config/HBASE/versions

3. http://search-hadoop.com/m/HOcsdhMt3B1/0.96&subj=0+96+0